2024-09-20T23:56:50.251 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-20T23:56:50.279 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-20T23:56:50.363 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913310' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy mgr-modules: - snap_schedule sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIi0dTXSzYbhqvrZsSP3dhqqQWr7kWtoU6EhYILWD62Biw0AM2MTI7far811NGw2XfQjNW3chpqdq+13DTnyZy4= smithi089.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLohWBl+ZWsSRFvIZTKVxw+A3kB0KExOS3qJdfr07j/RhwLEcxFmRjfGxfXWl6gj6kTMLcNpC6QiYEYZX3V9oLQ= smithi098.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJlMfL8NeHng/R/LkbIXje/HDcOi1rMVYigNsBpGl9qJW1uaARa0CAot6eccmTG5d5NZ8WJ4yn02yAGqYXbayK0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-20T23:56:50.363 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-20T23:56:50.364 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-20T23:56:50.364 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-20T23:56:50.366 INFO:teuthology.task.internal:Checking packages... 2024-09-20T23:56:50.387 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-20T23:56:50.387 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-20T23:56:50.388 INFO:teuthology.packaging:ref: None 2024-09-20T23:56:50.388 INFO:teuthology.packaging:tag: None 2024-09-20T23:56:50.388 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-20T23:56:50.388 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-20T23:56:50.388 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-20T23:56:50.534 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-20T23:56:50.536 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-20T23:56:50.549 INFO:teuthology.task.internal:no buildpackages task found 2024-09-20T23:56:50.549 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-20T23:56:50.626 INFO:teuthology.task.internal:Saving configuration 2024-09-20T23:56:50.647 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-20T23:56:50.671 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-20T23:56:50.698 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-20 23:50:40.072102', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIi0dTXSzYbhqvrZsSP3dhqqQWr7kWtoU6EhYILWD62Biw0AM2MTI7far811NGw2XfQjNW3chpqdq+13DTnyZy4='} 2024-09-20T23:56:50.718 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-20 23:50:40.074804', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLohWBl+ZWsSRFvIZTKVxw+A3kB0KExOS3qJdfr07j/RhwLEcxFmRjfGxfXWl6gj6kTMLcNpC6QiYEYZX3V9oLQ='} 2024-09-20T23:56:50.740 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-20 23:50:40.073573', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJlMfL8NeHng/R/LkbIXje/HDcOi1rMVYigNsBpGl9qJW1uaARa0CAot6eccmTG5d5NZ8WJ4yn02yAGqYXbayK0='} 2024-09-20T23:56:50.740 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-20T23:56:50.752 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-20T23:56:50.752 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-20T23:56:50.752 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-20T23:56:50.752 INFO:teuthology.run_tasks:Running task console_log... 2024-09-20T23:56:50.852 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8ea9254160>, signals=[15]) 2024-09-20T23:56:50.852 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-20T23:56:50.863 INFO:teuthology.task.internal:Opening connections... 2024-09-20T23:56:50.864 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-09-20T23:56:50.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T23:56:50.939 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2024-09-20T23:56:50.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T23:56:51.015 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi098.front.sepia.ceph.com 2024-09-20T23:56:51.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T23:56:51.090 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-20T23:56:51.097 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-09-20T23:56:51.116 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-09-20T23:56:51.117 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-09-20T23:56:51.175 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-09-20T23:56:51.176 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-09-20T23:56:51.176 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-09-20T23:56:51.176 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-09-20T23:56:51.176 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-09-20T23:56:51.176 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-09-20T23:56:51.176 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-20T23:56:51.176 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-09-20T23:56:51.176 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-09-20T23:56:51.176 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-20T23:56:51.176 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-09-20T23:56:51.177 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-20T23:56:51.177 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-20T23:56:51.177 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-20T23:56:51.177 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-09-20T23:56:51.198 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2024-09-20T23:56:51.216 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2024-09-20T23:56:51.216 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2024-09-20T23:56:51.274 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream" 2024-09-20T23:56:51.274 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="9" 2024-09-20T23:56:51.274 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2024-09-20T23:56:51.275 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2024-09-20T23:56:51.275 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="9" 2024-09-20T23:56:51.275 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el9" 2024-09-20T23:56:51.275 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-20T23:56:51.275 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2024-09-20T23:56:51.275 INFO:teuthology.orchestra.run.smithi089.stdout:LOGO="fedora-logo-icon" 2024-09-20T23:56:51.275 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-20T23:56:51.275 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/" 2024-09-20T23:56:51.275 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-20T23:56:51.275 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-20T23:56:51.275 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-20T23:56:51.276 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2024-09-20T23:56:51.297 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2024-09-20T23:56:51.315 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2024-09-20T23:56:51.315 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2024-09-20T23:56:51.373 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2024-09-20T23:56:51.374 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="9" 2024-09-20T23:56:51.374 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2024-09-20T23:56:51.374 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2024-09-20T23:56:51.374 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="9" 2024-09-20T23:56:51.374 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el9" 2024-09-20T23:56:51.374 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-20T23:56:51.374 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2024-09-20T23:56:51.374 INFO:teuthology.orchestra.run.smithi098.stdout:LOGO="fedora-logo-icon" 2024-09-20T23:56:51.374 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-20T23:56:51.374 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2024-09-20T23:56:51.374 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-20T23:56:51.374 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-20T23:56:51.374 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-20T23:56:51.375 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2024-09-20T23:56:51.397 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-20T23:56:51.410 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-20T23:56:51.498 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-20T23:56:51.498 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2024-09-20T23:56:51.500 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2024-09-20T23:56:51.503 DEBUG:teuthology.orchestra.run.smithi098:> test '!' -e /home/ubuntu/cephtest 2024-09-20T23:56:51.520 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-20T23:56:51.539 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-20T23:56:51.540 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2024-09-20T23:56:51.558 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2024-09-20T23:56:51.560 DEBUG:teuthology.orchestra.run.smithi098:> test -z $(ls -A /var/lib/ceph) 2024-09-20T23:56:51.682 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-20T23:56:51.759 INFO:teuthology.run_tasks:Running task kernel... 2024-09-20T23:56:51.778 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-20T23:56:51.778 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-20T23:56:51.778 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-20T23:56:51.779 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-20T23:56:51.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T23:56:51.797 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-20T23:56:51.953 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-20T23:56:51.953 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-20T23:56:51.953 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-09-20T23:56:51.972 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-503.el9.x86_64 2024-09-20T23:56:51.972 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-20T23:56:51.972 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-20T23:56:51.973 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum upgrade -y linux-firmware 2024-09-20T23:56:55.224 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.3 MB 00:01 2024-09-20T23:56:57.099 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-20T23:57:01.583 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-20T23:57:03.377 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 19 kB 00:00 2024-09-20T23:57:04.172 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-20T23:57:09.828 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-20T23:57:11.895 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-20T23:57:11.913 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-20T23:57:11.913 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-09-20T23:57:11.913 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-20T23:57:11.913 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-09-20T23:57:11.913 INFO:teuthology.orchestra.run.smithi027.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-20T23:57:11.913 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-20T23:57:11.913 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-20T23:57:11.913 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-20T23:57:11.913 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 1 Package 2024-09-20T23:57:11.913 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-20T23:57:11.914 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 388 M 2024-09-20T23:57:11.914 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-20T23:57:37.236 INFO:teuthology.orchestra.run.smithi027.stdout:linux-firmware-20240905-146.el9.noarch.rpm 15 MB/s | 388 MB 00:25 2024-09-20T23:57:37.236 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-20T23:57:37.236 INFO:teuthology.orchestra.run.smithi027.stdout:Total 15 MB/s | 388 MB 00:25 2024-09-20T23:57:40.512 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-20T23:57:40.598 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-20T23:57:40.598 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-20T23:57:44.033 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-20T23:57:44.034 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-20T23:57:53.412 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-20T23:57:54.103 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-20T23:57:54.196 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-20T23:57:54.655 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-20T23:57:54.655 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-20T23:57:55.426 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-20T23:57:55.426 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-20T23:57:55.426 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-09-20T23:57:55.426 INFO:teuthology.orchestra.run.smithi027.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-20T23:57:55.426 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-20T23:57:55.426 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-20T23:57:55.774 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-20T23:57:55.775 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-20T23:57:55.921 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-20T23:57:55.921 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-20T23:57:55.924 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi027.front.sepia.ceph.com to complete... 2024-09-20T23:58:08.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-20 23:58:08 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-20T23:58:08.165 INFO:teuthology.orchestra.run.smithi027.stderr:FINISHED --2024-09-20 23:58:08-- 2024-09-20T23:58:08.165 INFO:teuthology.orchestra.run.smithi027.stderr:Total wall clock time: 12s 2024-09-20T23:58:08.165 INFO:teuthology.orchestra.run.smithi027.stderr:Downloaded: 1 files, 343M in 12s (28.8 MB/s) 2024-09-20T23:58:08.168 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-20T23:58:08.168 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-20T23:58:10.334 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying... ######################################## 2024-09-20T23:58:10.395 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing... ######################################## 2024-09-20T23:58:10.459 INFO:teuthology.orchestra.run.smithi027.stdout:Updating / installing... 2024-09-20T23:58:18.784 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-20T23:58:59.902 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-20T23:58:59.903 DEBUG:teuthology.orchestra.run.smithi027:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-20T23:58:59.986 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-20T23:58:59.986 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-20T23:58:59.986 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-20T23:58:59.986 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-20T23:58:59.986 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-20T23:58:59.986 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-20T23:58:59.986 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-20T23:58:59.986 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-20T23:58:59.986 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-20T23:58:59.986 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-20T23:58:59.987 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-20T23:58:59.987 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-20T23:58:59.987 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-20T23:58:59.987 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-20T23:58:59.987 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-20T23:58:59.987 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-20T23:58:59.987 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-20T23:58:59.987 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-20T23:58:59.987 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-20T23:58:59.987 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-20T23:58:59.987 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-20T23:58:59.987 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-20T23:58:59.987 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-20T23:58:59.988 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-20T23:58:59.988 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-20T23:58:59.988 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-20T23:58:59.988 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-20T23:58:59.988 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-20T23:58:59.988 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-20T23:58:59.988 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-20T23:58:59.988 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-20T23:58:59.988 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-20T23:58:59.988 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-20T23:58:59.988 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-20T23:58:59.988 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-20T23:58:59.989 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-20T23:58:59.989 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-20T23:58:59.989 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-20T23:58:59.989 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-20T23:58:59.989 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-20T23:58:59.989 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-20T23:58:59.989 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-20T23:58:59.989 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-20T23:58:59.989 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-20T23:58:59.989 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-20T23:58:59.989 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-20T23:58:59.989 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-20T23:58:59.989 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-20T23:58:59.989 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-20T23:58:59.990 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-20T23:58:59.990 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-20T23:58:59.990 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-20T23:58:59.990 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-20T23:58:59.990 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-20T23:58:59.990 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-20T23:58:59.990 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-20T23:58:59.990 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-20T23:58:59.990 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-20T23:58:59.990 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-20T23:58:59.990 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-20T23:58:59.990 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-20T23:58:59.991 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-20T23:58:59.991 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-20T23:58:59.991 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-20T23:58:59.991 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-20T23:58:59.991 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-20T23:58:59.991 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-20T23:58:59.991 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-20T23:58:59.991 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-20T23:58:59.991 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-20T23:58:59.991 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-20T23:58:59.991 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-20T23:58:59.992 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-20T23:58:59.992 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-20T23:58:59.992 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-20T23:58:59.992 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-20T23:58:59.992 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-20T23:58:59.992 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-20T23:58:59.992 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-20T23:58:59.992 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-20T23:58:59.992 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-20T23:58:59.992 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-20T23:58:59.992 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-20T23:58:59.992 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-20T23:58:59.992 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-20T23:58:59.993 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-20T23:58:59.993 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-20T23:58:59.993 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-20T23:58:59.993 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-20T23:58:59.993 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-20T23:58:59.993 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-20T23:58:59.993 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-20T23:58:59.993 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-20T23:58:59.993 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-20T23:58:59.993 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-20T23:58:59.993 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-20T23:58:59.993 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-20T23:58:59.993 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-20T23:58:59.993 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-20T23:58:59.994 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-20T23:58:59.994 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-20T23:58:59.994 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-20T23:58:59.994 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-20T23:58:59.994 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-20T23:58:59.994 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-20T23:58:59.994 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-20T23:58:59.994 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-20T23:58:59.994 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-20T23:58:59.994 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-20T23:58:59.994 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-20T23:58:59.994 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-20T23:58:59.995 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-20T23:58:59.995 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-20T23:58:59.995 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-20T23:58:59.995 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-20T23:58:59.995 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-20T23:58:59.995 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-20T23:58:59.995 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-20T23:58:59.995 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-20T23:58:59.995 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-20T23:58:59.995 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-20T23:58:59.995 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-20T23:58:59.995 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-20T23:58:59.996 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-20T23:58:59.996 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-20T23:58:59.996 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-20T23:58:59.996 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-20T23:58:59.996 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-20T23:58:59.996 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-20T23:58:59.996 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-20T23:58:59.996 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-20T23:58:59.996 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-20T23:58:59.996 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-20T23:58:59.996 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-20T23:58:59.996 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-20T23:58:59.996 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-20T23:58:59.997 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-20T23:58:59.997 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-20T23:58:59.997 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-20T23:58:59.997 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-20T23:58:59.997 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-20T23:58:59.997 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-20T23:58:59.997 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-20T23:58:59.997 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-20T23:58:59.997 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-20T23:58:59.997 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-20T23:58:59.997 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-20T23:58:59.997 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-20T23:58:59.997 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-20T23:58:59.997 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-20T23:58:59.998 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-20T23:58:59.998 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-20T23:58:59.998 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-20T23:58:59.998 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-20T23:58:59.998 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-20T23:58:59.998 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-20T23:58:59.998 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-20T23:58:59.998 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-20T23:58:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-20T23:58:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-20T23:58:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-20T23:58:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-20T23:58:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-20T23:58:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-20T23:58:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-20T23:58:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-20T23:58:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-20T23:58:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-20T23:58:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-20T23:58:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-20T23:58:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-20T23:58:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-20T23:58:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-20T23:59:00.000 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-20T23:59:00.000 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-20T23:59:00.000 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-20T23:59:00.000 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-20T23:59:00.000 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-20T23:59:00.000 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-20T23:59:00.000 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-20T23:59:00.000 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-20T23:59:00.000 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-20T23:59:00.000 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-20T23:59:00.000 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-20T23:59:00.000 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-20T23:59:00.000 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-20T23:59:00.000 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-20T23:59:00.000 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-20T23:59:00.001 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-20T23:59:00.001 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-20T23:59:00.001 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-20T23:59:00.001 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-20T23:59:00.001 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-20T23:59:00.001 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-20T23:59:00.001 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-20T23:59:00.001 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-20T23:59:00.001 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-20T23:59:00.001 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-20T23:59:00.001 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-20T23:59:00.001 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-20T23:59:00.001 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-20T23:59:00.001 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-20T23:59:00.001 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-20T23:59:00.002 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-20T23:59:00.002 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-20T23:59:00.002 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-20T23:59:00.002 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-20T23:59:00.002 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-20T23:59:00.002 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-20T23:59:00.002 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-20T23:59:00.002 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-20T23:59:00.002 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-20T23:59:00.002 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-20T23:59:00.002 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-20T23:59:00.002 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-20T23:59:00.002 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-20T23:59:00.002 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-20T23:59:00.003 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-20T23:59:00.003 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-20T23:59:00.003 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-20T23:59:00.003 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-20T23:59:00.003 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-20T23:59:00.003 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-20T23:59:00.003 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-20T23:59:00.003 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-20T23:59:00.003 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-20T23:59:00.003 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-20T23:59:00.003 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-20T23:59:00.003 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-20T23:59:00.003 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-20T23:59:00.004 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-20T23:59:00.004 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-20T23:59:00.004 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-20T23:59:00.004 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-20T23:59:00.004 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-20T23:59:00.004 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-20T23:59:00.004 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-20T23:59:00.004 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-20T23:59:00.004 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-20T23:59:00.004 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-20T23:59:00.004 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-20T23:59:00.004 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-20T23:59:00.004 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-20T23:59:00.004 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-20T23:59:00.004 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-20T23:59:00.005 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-20T23:59:00.006 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-20T23:59:00.006 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-20T23:59:00.006 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-20T23:59:00.006 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-20T23:59:00.006 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-20T23:59:00.006 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-20T23:59:00.006 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-20T23:59:00.006 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-20T23:59:00.006 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-20T23:59:00.006 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-20T23:59:00.006 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-20T23:59:00.006 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-20T23:59:00.006 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-20T23:59:00.006 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-20T23:59:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-20T23:59:00.008 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-20T23:59:00.008 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-20T23:59:00.008 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-20T23:59:00.008 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-20T23:59:00.008 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-20T23:59:00.008 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-20T23:59:00.008 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-20T23:59:00.008 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-20T23:59:00.008 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-20T23:59:00.008 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-20T23:59:00.008 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-20T23:59:00.008 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-20T23:59:00.008 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-20T23:59:00.008 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-20T23:59:00.009 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-20T23:59:00.009 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-20T23:59:00.009 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-20T23:59:00.009 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-20T23:59:00.009 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-20T23:59:00.009 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-20T23:59:00.009 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-20T23:59:00.009 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-20T23:59:00.009 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-20T23:59:00.009 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-20T23:59:00.009 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-20T23:59:00.009 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-20T23:59:00.009 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-20T23:59:00.009 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-20T23:59:00.009 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-20T23:59:00.010 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-20T23:59:00.010 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-20T23:59:00.010 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-20T23:59:00.010 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-20T23:59:00.010 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-20T23:59:00.010 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-20T23:59:00.010 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-20T23:59:00.010 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-20T23:59:00.010 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-20T23:59:00.010 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-20T23:59:00.010 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-20T23:59:00.010 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-20T23:59:00.010 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-20T23:59:00.010 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-20T23:59:00.011 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-20T23:59:00.011 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-20T23:59:00.011 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-20T23:59:00.011 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-20T23:59:00.011 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-20T23:59:00.011 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-20T23:59:00.011 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-20T23:59:00.011 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-20T23:59:00.011 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-20T23:59:00.011 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-20T23:59:00.011 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-20T23:59:00.011 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-20T23:59:00.011 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-20T23:59:00.011 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-20T23:59:00.012 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-20T23:59:00.012 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-20T23:59:00.012 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-20T23:59:00.012 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-20T23:59:00.012 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-20T23:59:00.012 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-20T23:59:00.012 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-20T23:59:00.012 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-20T23:59:00.012 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-20T23:59:00.012 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-20T23:59:00.012 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-20T23:59:00.012 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-20T23:59:00.012 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-20T23:59:00.012 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-20T23:59:00.013 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-20T23:59:00.013 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-20T23:59:00.013 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-20T23:59:00.013 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-20T23:59:00.013 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-20T23:59:00.013 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-20T23:59:00.013 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-20T23:59:00.013 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-20T23:59:00.013 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-20T23:59:00.013 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-20T23:59:00.013 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-20T23:59:00.013 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-20T23:59:00.013 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-20T23:59:00.014 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-20T23:59:00.014 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-20T23:59:00.014 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-20T23:59:00.014 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-20T23:59:00.014 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-20T23:59:00.014 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-20T23:59:00.014 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-20T23:59:00.014 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-20T23:59:00.014 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-20T23:59:00.014 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-20T23:59:00.014 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-20T23:59:00.014 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-20T23:59:00.014 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-20T23:59:00.014 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-20T23:59:00.015 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-20T23:59:00.015 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-20T23:59:00.015 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-20T23:59:00.015 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-20T23:59:00.015 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-20T23:59:00.015 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-20T23:59:00.015 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-20T23:59:00.015 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-20T23:59:00.015 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-20T23:59:00.015 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-20T23:59:00.015 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-20T23:59:00.016 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-20T23:59:00.016 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-20T23:59:00.016 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-20T23:59:00.016 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-20T23:59:00.016 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-20T23:59:00.016 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-20T23:59:00.016 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-20T23:59:00.016 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-20T23:59:00.016 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-20T23:59:00.017 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-20T23:59:00.017 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-20T23:59:00.017 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-20T23:59:00.017 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-20T23:59:00.017 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-20T23:59:00.017 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-20T23:59:00.017 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-20T23:59:00.017 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-20T23:59:00.017 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-20T23:59:00.018 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-20T23:59:00.018 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-20T23:59:00.018 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-20T23:59:00.018 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-20T23:59:00.018 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-20T23:59:00.018 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-20T23:59:00.018 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-20T23:59:00.018 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-20T23:59:00.019 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-20T23:59:00.019 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-20T23:59:00.019 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-20T23:59:00.019 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-20T23:59:00.019 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-20T23:59:00.019 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-20T23:59:00.019 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-20T23:59:00.019 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-20T23:59:00.019 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-20T23:59:00.020 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-20T23:59:00.020 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-20T23:59:00.020 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-20T23:59:00.020 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-20T23:59:00.020 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-20T23:59:00.020 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-20T23:59:00.020 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-20T23:59:00.020 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-20T23:59:00.020 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-20T23:59:00.021 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-20T23:59:00.021 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-20T23:59:00.021 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-20T23:59:00.021 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-20T23:59:00.021 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-20T23:59:00.021 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-20T23:59:00.021 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-20T23:59:00.022 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-20T23:59:00.022 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-20T23:59:00.022 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-20T23:59:00.022 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-20T23:59:00.022 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-20T23:59:00.022 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-20T23:59:00.022 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-20T23:59:00.022 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-20T23:59:00.023 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-20T23:59:00.023 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-20T23:59:00.023 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-20T23:59:00.023 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-20T23:59:00.023 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-20T23:59:00.023 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-20T23:59:00.023 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-20T23:59:00.023 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-20T23:59:00.024 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-20T23:59:00.024 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-20T23:59:00.024 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-20T23:59:00.024 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-20T23:59:00.024 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-20T23:59:00.024 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-20T23:59:00.024 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-20T23:59:00.025 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-20T23:59:00.025 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-20T23:59:00.025 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-20T23:59:00.025 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-20T23:59:00.025 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-20T23:59:00.025 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-20T23:59:00.025 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-20T23:59:00.025 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-20T23:59:00.025 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-20T23:59:00.026 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-20T23:59:00.026 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-20T23:59:00.026 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-20T23:59:00.026 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-20T23:59:00.026 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-20T23:59:00.026 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-20T23:59:00.026 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-20T23:59:00.026 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-20T23:59:00.026 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-20T23:59:00.026 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-20T23:59:00.026 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-20T23:59:00.026 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-20T23:59:00.026 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-20T23:59:00.026 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-20T23:59:00.026 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-20T23:59:00.027 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-20T23:59:00.027 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-20T23:59:00.027 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-20T23:59:00.027 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-20T23:59:00.027 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-20T23:59:00.027 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-20T23:59:00.027 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-20T23:59:00.027 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-20T23:59:00.027 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-20T23:59:00.027 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-20T23:59:00.027 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-20T23:59:00.027 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-20T23:59:00.027 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-20T23:59:00.027 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-20T23:59:00.027 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-20T23:59:00.028 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-20T23:59:00.028 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-20T23:59:00.028 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-20T23:59:00.028 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-20T23:59:00.028 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-20T23:59:00.028 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-20T23:59:00.028 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-20T23:59:00.028 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-20T23:59:00.028 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-20T23:59:00.028 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-20T23:59:00.028 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-20T23:59:00.028 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-20T23:59:00.028 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-20T23:59:00.028 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-20T23:59:00.028 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-20T23:59:00.029 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-20T23:59:00.029 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-20T23:59:00.029 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-20T23:59:00.029 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-20T23:59:00.029 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-20T23:59:00.029 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-20T23:59:00.029 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-20T23:59:00.029 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-20T23:59:00.029 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-20T23:59:00.029 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-20T23:59:00.029 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-20T23:59:00.029 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-20T23:59:00.029 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-20T23:59:00.029 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-20T23:59:00.030 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-20T23:59:00.030 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-20T23:59:00.030 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-20T23:59:00.030 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-20T23:59:00.030 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-20T23:59:00.030 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-20T23:59:00.030 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-20T23:59:00.030 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-20T23:59:00.030 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-20T23:59:00.030 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-20T23:59:00.030 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-20T23:59:00.030 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-20T23:59:00.030 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-20T23:59:00.030 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-20T23:59:00.031 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-20T23:59:00.031 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-20T23:59:00.031 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-20T23:59:00.031 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-20T23:59:00.031 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-20T23:59:00.031 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-20T23:59:00.031 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-20T23:59:00.031 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-20T23:59:00.032 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-20T23:59:00.032 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-20T23:59:00.032 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-20T23:59:00.032 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-20T23:59:00.032 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-20T23:59:00.032 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-20T23:59:00.032 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-20T23:59:00.032 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-20T23:59:00.032 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-20T23:59:00.032 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-20T23:59:00.032 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-20T23:59:00.032 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-20T23:59:00.032 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-20T23:59:00.033 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-20T23:59:00.033 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-20T23:59:00.033 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-20T23:59:00.033 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-20T23:59:00.033 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-20T23:59:00.033 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-20T23:59:00.033 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-20T23:59:00.033 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-20T23:59:00.033 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-20T23:59:00.033 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-20T23:59:00.033 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-20T23:59:00.033 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-20T23:59:00.033 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-20T23:59:00.033 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-20T23:59:00.034 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-20T23:59:00.034 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-20T23:59:00.034 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-20T23:59:00.034 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-20T23:59:00.034 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-20T23:59:00.034 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-20T23:59:00.034 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-20T23:59:00.034 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-20T23:59:00.034 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-20T23:59:00.034 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-20T23:59:00.034 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-20T23:59:00.034 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-20T23:59:00.034 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-20T23:59:00.034 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-20T23:59:00.035 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-20T23:59:00.035 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-20T23:59:00.035 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-20T23:59:00.035 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-20T23:59:00.035 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-20T23:59:00.035 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-20T23:59:00.035 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-20T23:59:00.035 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-20T23:59:00.035 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-20T23:59:00.035 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-20T23:59:00.035 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-20T23:59:00.035 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-20T23:59:00.035 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-20T23:59:00.036 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-20T23:59:00.037 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-20T23:59:00.038 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-20T23:59:00.038 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-20T23:59:00.038 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-20T23:59:00.038 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-20T23:59:00.038 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-20T23:59:00.038 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-20T23:59:00.038 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-20T23:59:00.038 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-20T23:59:00.038 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-20T23:59:00.038 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-20T23:59:00.038 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-20T23:59:00.038 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-20T23:59:00.038 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-20T23:59:00.038 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-20T23:59:00.039 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-20T23:59:00.039 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-20T23:59:00.039 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-20T23:59:00.039 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-20T23:59:00.039 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-20T23:59:00.039 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-20T23:59:00.039 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-20T23:59:00.039 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-20T23:59:00.039 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-20T23:59:00.039 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-20T23:59:00.039 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-20T23:59:00.039 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-20T23:59:00.039 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-20T23:59:00.039 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-20T23:59:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-20T23:59:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-20T23:59:00.042 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-20T23:59:00.042 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-20T23:59:00.042 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-20T23:59:00.042 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-20T23:59:00.042 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-20T23:59:00.042 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-20T23:59:00.042 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-20T23:59:00.042 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-20T23:59:00.042 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-20T23:59:00.042 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-20T23:59:00.042 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-20T23:59:00.042 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-20T23:59:00.042 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-20T23:59:00.042 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-20T23:59:00.043 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-20T23:59:00.043 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-20T23:59:00.043 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-20T23:59:00.043 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-20T23:59:00.043 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-20T23:59:00.043 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-20T23:59:00.043 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-20T23:59:00.043 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-20T23:59:00.043 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-20T23:59:00.043 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-20T23:59:00.043 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-20T23:59:00.043 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-20T23:59:00.043 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-20T23:59:00.043 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-20T23:59:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-20T23:59:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-20T23:59:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-20T23:59:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-20T23:59:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-20T23:59:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-20T23:59:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-20T23:59:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-20T23:59:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-20T23:59:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-20T23:59:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-20T23:59:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-20T23:59:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-20T23:59:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-20T23:59:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-20T23:59:00.045 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-20T23:59:00.045 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-20T23:59:00.045 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-20T23:59:00.045 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-20T23:59:00.045 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-20T23:59:00.045 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-20T23:59:00.045 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-20T23:59:00.045 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-20T23:59:00.045 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-20T23:59:00.045 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-20T23:59:00.045 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-20T23:59:00.045 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-20T23:59:00.045 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-20T23:59:00.045 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-20T23:59:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-20T23:59:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-20T23:59:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-20T23:59:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-20T23:59:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-20T23:59:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-20T23:59:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-20T23:59:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-20T23:59:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-20T23:59:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-20T23:59:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-20T23:59:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-20T23:59:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-20T23:59:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-20T23:59:00.047 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-20T23:59:00.047 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-20T23:59:00.047 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-20T23:59:00.047 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-20T23:59:00.047 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-20T23:59:00.047 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-20T23:59:00.047 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-20T23:59:00.047 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-20T23:59:00.047 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-20T23:59:00.047 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-20T23:59:00.047 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-20T23:59:00.047 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-20T23:59:00.047 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-20T23:59:00.047 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-20T23:59:00.047 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-20T23:59:00.048 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-20T23:59:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-20T23:59:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-20T23:59:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-20T23:59:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-20T23:59:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-20T23:59:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-20T23:59:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-20T23:59:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-20T23:59:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-20T23:59:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-20T23:59:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-20T23:59:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-20T23:59:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-20T23:59:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-20T23:59:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-20T23:59:00.050 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-20T23:59:00.050 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-20T23:59:00.050 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-20T23:59:00.050 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-20T23:59:00.050 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-20T23:59:00.050 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-20T23:59:00.050 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-20T23:59:00.050 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-20T23:59:00.050 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-20T23:59:00.050 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-20T23:59:00.050 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-20T23:59:00.050 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-20T23:59:00.050 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-20T23:59:00.050 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-20T23:59:00.051 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-20T23:59:00.051 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-20T23:59:00.051 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-20T23:59:00.051 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-20T23:59:00.051 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-20T23:59:00.051 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-20T23:59:00.051 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-20T23:59:00.051 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-20T23:59:00.051 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-20T23:59:00.051 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-20T23:59:00.051 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-20T23:59:00.051 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-20T23:59:00.051 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-20T23:59:00.051 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-20T23:59:00.052 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-20T23:59:00.052 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-20T23:59:00.052 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-20T23:59:00.052 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-20T23:59:00.052 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-20T23:59:00.052 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-20T23:59:00.052 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-20T23:59:00.052 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-20T23:59:00.052 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-20T23:59:00.052 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-20T23:59:00.052 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-20T23:59:00.052 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-20T23:59:00.052 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-20T23:59:00.052 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-20T23:59:00.053 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-20T23:59:00.053 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-20T23:59:00.053 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-20T23:59:00.053 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-20T23:59:00.053 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-20T23:59:00.053 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-20T23:59:00.053 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-20T23:59:00.053 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-20T23:59:00.053 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-20T23:59:00.053 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-20T23:59:00.053 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-20T23:59:00.053 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-20T23:59:00.053 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-20T23:59:00.053 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-20T23:59:00.053 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-20T23:59:00.054 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-20T23:59:00.054 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-20T23:59:00.054 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-20T23:59:00.054 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-20T23:59:00.054 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-20T23:59:00.054 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-20T23:59:00.054 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-20T23:59:00.054 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-20T23:59:00.054 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-20T23:59:00.054 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-20T23:59:00.054 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-20T23:59:00.054 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-20T23:59:00.054 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-20T23:59:00.055 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-20T23:59:00.055 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-20T23:59:00.055 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-20T23:59:00.055 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-20T23:59:00.055 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-20T23:59:00.055 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-20T23:59:00.055 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-20T23:59:00.055 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-20T23:59:00.055 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-20T23:59:00.055 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-20T23:59:00.055 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-20T23:59:00.055 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-20T23:59:00.055 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-20T23:59:00.055 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-20T23:59:00.056 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-20T23:59:00.056 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-20T23:59:00.056 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-20T23:59:00.056 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-20T23:59:00.056 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-20T23:59:00.056 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-20T23:59:00.056 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-20T23:59:00.056 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-20T23:59:00.056 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-20T23:59:00.056 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-20T23:59:00.056 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-20T23:59:00.056 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-20T23:59:00.056 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-20T23:59:00.056 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-20T23:59:00.057 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-20T23:59:00.057 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-20T23:59:00.057 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-20T23:59:00.057 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-20T23:59:00.057 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-20T23:59:00.057 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-20T23:59:00.057 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-20T23:59:00.057 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-20T23:59:00.057 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-20T23:59:00.057 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-20T23:59:00.057 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-20T23:59:00.057 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-20T23:59:00.057 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-20T23:59:00.057 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-20T23:59:00.057 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-20T23:59:00.058 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-20T23:59:00.058 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-20T23:59:00.058 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-20T23:59:00.058 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-20T23:59:00.058 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-20T23:59:00.058 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-20T23:59:00.058 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-20T23:59:00.058 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-20T23:59:00.058 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-20T23:59:00.058 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-20T23:59:00.058 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-20T23:59:00.058 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-20T23:59:00.058 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-20T23:59:00.058 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-20T23:59:00.058 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-20T23:59:00.059 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-20T23:59:00.059 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-20T23:59:00.059 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-20T23:59:00.059 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-20T23:59:00.059 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-20T23:59:00.059 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-20T23:59:00.059 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-20T23:59:00.059 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-20T23:59:00.059 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-20T23:59:00.059 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-20T23:59:00.059 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-20T23:59:00.059 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-20T23:59:00.059 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-20T23:59:00.059 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-20T23:59:00.059 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-20T23:59:00.060 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-20T23:59:00.061 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-20T23:59:00.062 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-20T23:59:00.062 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-20T23:59:00.062 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-20T23:59:00.062 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-20T23:59:00.062 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-20T23:59:00.062 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-20T23:59:00.062 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-20T23:59:00.062 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-20T23:59:00.062 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-20T23:59:00.062 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-20T23:59:00.062 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-20T23:59:00.062 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-20T23:59:00.062 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-20T23:59:00.062 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-20T23:59:00.062 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-20T23:59:00.063 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-20T23:59:00.063 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-20T23:59:00.063 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-20T23:59:00.063 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-20T23:59:00.063 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-20T23:59:00.063 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-20T23:59:00.063 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-20T23:59:00.063 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-20T23:59:00.063 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-20T23:59:00.063 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-20T23:59:00.063 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-20T23:59:00.063 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-20T23:59:00.063 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-20T23:59:00.063 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-20T23:59:00.064 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-20T23:59:00.065 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-20T23:59:00.065 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-20T23:59:00.065 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-20T23:59:00.065 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-20T23:59:00.065 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-20T23:59:00.065 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-20T23:59:00.065 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-20T23:59:00.065 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-20T23:59:00.065 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-20T23:59:00.065 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-20T23:59:00.065 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-20T23:59:00.065 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-20T23:59:00.065 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-20T23:59:00.065 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-20T23:59:00.066 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-20T23:59:00.066 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-20T23:59:00.066 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-20T23:59:00.066 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-20T23:59:00.066 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-20T23:59:00.066 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-20T23:59:00.066 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-20T23:59:00.066 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-20T23:59:00.066 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-20T23:59:00.066 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-20T23:59:00.066 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-20T23:59:00.066 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-20T23:59:00.066 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-20T23:59:00.066 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-20T23:59:00.067 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-20T23:59:00.067 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-20T23:59:00.067 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-20T23:59:00.067 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-20T23:59:00.067 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-20T23:59:00.067 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-20T23:59:00.067 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-20T23:59:00.067 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-20T23:59:00.067 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-20T23:59:00.067 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-20T23:59:00.067 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-20T23:59:00.067 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-20T23:59:00.067 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-20T23:59:00.067 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-20T23:59:00.067 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-20T23:59:00.068 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-20T23:59:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-20T23:59:00.070 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-20T23:59:00.070 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-20T23:59:00.070 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-20T23:59:00.070 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-20T23:59:00.070 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-20T23:59:00.070 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-20T23:59:00.070 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-20T23:59:00.070 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-20T23:59:00.070 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-20T23:59:00.070 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-20T23:59:00.070 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-20T23:59:00.070 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-20T23:59:00.070 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-20T23:59:00.070 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-20T23:59:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-20T23:59:00.072 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-20T23:59:00.073 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-20T23:59:00.074 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-20T23:59:00.074 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-20T23:59:00.074 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-20T23:59:00.074 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-20T23:59:00.074 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-20T23:59:00.074 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-20T23:59:00.074 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-20T23:59:00.074 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-20T23:59:00.074 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-20T23:59:00.074 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-20T23:59:00.074 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-20T23:59:00.074 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-20T23:59:00.074 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-20T23:59:00.074 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-20T23:59:00.074 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-20T23:59:00.075 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-20T23:59:00.076 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-20T23:59:00.077 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-20T23:59:00.078 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-20T23:59:00.078 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-20T23:59:00.078 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-20T23:59:00.078 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-20T23:59:00.078 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-20T23:59:00.078 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-20T23:59:00.078 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-20T23:59:00.078 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-20T23:59:00.078 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-20T23:59:00.078 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-20T23:59:00.078 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-20T23:59:00.078 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-20T23:59:00.078 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-20T23:59:00.078 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-20T23:59:00.078 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-20T23:59:00.079 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-20T23:59:00.080 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-20T23:59:00.081 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-20T23:59:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-20T23:59:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-20T23:59:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-20T23:59:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-20T23:59:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-20T23:59:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-20T23:59:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-20T23:59:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-20T23:59:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-20T23:59:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-20T23:59:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-20T23:59:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-20T23:59:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-20T23:59:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-20T23:59:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-20T23:59:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-20T23:59:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-20T23:59:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-20T23:59:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-20T23:59:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-20T23:59:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-20T23:59:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-20T23:59:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-20T23:59:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-20T23:59:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-20T23:59:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-20T23:59:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-20T23:59:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-20T23:59:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-20T23:59:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-20T23:59:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-20T23:59:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-20T23:59:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-20T23:59:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-20T23:59:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-20T23:59:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-20T23:59:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-20T23:59:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-20T23:59:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-20T23:59:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-20T23:59:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-20T23:59:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-20T23:59:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-20T23:59:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-20T23:59:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-20T23:59:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-20T23:59:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-20T23:59:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-20T23:59:00.089 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-20T23:59:00.090 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-20T23:59:00.091 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-20T23:59:00.091 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-20T23:59:00.091 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-20T23:59:00.091 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-20T23:59:00.091 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-20T23:59:00.091 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-20T23:59:00.091 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-20T23:59:00.091 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-20T23:59:00.091 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-20T23:59:00.091 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-20T23:59:00.091 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-20T23:59:00.091 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-20T23:59:00.091 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-20T23:59:00.091 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-20T23:59:00.091 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-20T23:59:00.092 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-20T23:59:00.092 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-20T23:59:00.092 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-20T23:59:00.092 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-20T23:59:00.092 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-20T23:59:00.092 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-20T23:59:00.092 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-20T23:59:00.092 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-20T23:59:00.092 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-20T23:59:00.092 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-20T23:59:00.092 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-20T23:59:00.092 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-20T23:59:00.092 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-20T23:59:00.092 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-20T23:59:00.093 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-20T23:59:00.093 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-20T23:59:00.093 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-20T23:59:00.093 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-20T23:59:00.093 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-20T23:59:00.093 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-20T23:59:00.093 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-20T23:59:00.093 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-20T23:59:00.093 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-20T23:59:00.093 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-20T23:59:00.093 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-20T23:59:00.093 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-20T23:59:00.093 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-20T23:59:00.093 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-20T23:59:00.094 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-20T23:59:00.094 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-20T23:59:00.094 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-20T23:59:00.094 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-20T23:59:00.094 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-20T23:59:00.094 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-20T23:59:00.094 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-20T23:59:00.094 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-20T23:59:00.094 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-20T23:59:00.094 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-20T23:59:00.094 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-20T23:59:00.094 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-20T23:59:00.094 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-20T23:59:00.094 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-20T23:59:00.094 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-20T23:59:00.095 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-20T23:59:00.095 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-20T23:59:00.095 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-20T23:59:00.095 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-20T23:59:00.095 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-20T23:59:00.095 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-20T23:59:00.095 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-20T23:59:00.095 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-20T23:59:00.095 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-20T23:59:00.095 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-20T23:59:00.095 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-20T23:59:00.095 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-20T23:59:00.095 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-20T23:59:00.095 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-20T23:59:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-20T23:59:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-20T23:59:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-20T23:59:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-20T23:59:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-20T23:59:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-20T23:59:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-20T23:59:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-20T23:59:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-20T23:59:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-20T23:59:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-20T23:59:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-20T23:59:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-20T23:59:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-20T23:59:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-20T23:59:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-20T23:59:00.099 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-20T23:59:00.100 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-20T23:59:00.101 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-20T23:59:00.101 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-20T23:59:00.101 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-20T23:59:00.101 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-20T23:59:00.101 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-20T23:59:00.101 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-20T23:59:00.101 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-20T23:59:00.101 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-20T23:59:00.101 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-20T23:59:00.101 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-20T23:59:00.101 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-20T23:59:00.101 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-20T23:59:00.101 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-20T23:59:00.101 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-20T23:59:00.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-20T23:59:00.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-20T23:59:00.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-20T23:59:00.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-20T23:59:00.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-20T23:59:00.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-20T23:59:00.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-20T23:59:00.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-20T23:59:00.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-20T23:59:00.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-20T23:59:00.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-20T23:59:00.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-20T23:59:00.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-20T23:59:00.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-20T23:59:00.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-20T23:59:00.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-20T23:59:00.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-20T23:59:00.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-20T23:59:00.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-20T23:59:00.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-20T23:59:00.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-20T23:59:00.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-20T23:59:00.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-20T23:59:00.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-20T23:59:00.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-20T23:59:00.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-20T23:59:00.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-20T23:59:00.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-20T23:59:00.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-20T23:59:00.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-20T23:59:00.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-20T23:59:00.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-20T23:59:00.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-20T23:59:00.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-20T23:59:00.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-20T23:59:00.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-20T23:59:00.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-20T23:59:00.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-20T23:59:00.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-20T23:59:00.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-20T23:59:00.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-20T23:59:00.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-20T23:59:00.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-20T23:59:00.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-20T23:59:00.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-20T23:59:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-20T23:59:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-20T23:59:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-20T23:59:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-20T23:59:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-20T23:59:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-20T23:59:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-20T23:59:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-20T23:59:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-20T23:59:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-20T23:59:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-20T23:59:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-20T23:59:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-20T23:59:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-20T23:59:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-20T23:59:00.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-20T23:59:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-20T23:59:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-20T23:59:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-20T23:59:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-20T23:59:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-20T23:59:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-20T23:59:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-20T23:59:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-20T23:59:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-20T23:59:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-20T23:59:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-20T23:59:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-20T23:59:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-20T23:59:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-20T23:59:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-20T23:59:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-20T23:59:00.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-20T23:59:00.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-20T23:59:00.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-20T23:59:00.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-20T23:59:00.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-20T23:59:00.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-20T23:59:00.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-20T23:59:00.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-20T23:59:00.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-20T23:59:00.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-20T23:59:00.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-20T23:59:00.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-20T23:59:00.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-20T23:59:00.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-20T23:59:00.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-20T23:59:00.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-20T23:59:00.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-20T23:59:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-20T23:59:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-20T23:59:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-20T23:59:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-20T23:59:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-20T23:59:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-20T23:59:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-20T23:59:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-20T23:59:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-20T23:59:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-20T23:59:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-20T23:59:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-20T23:59:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-20T23:59:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-20T23:59:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-20T23:59:00.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-20T23:59:00.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-20T23:59:00.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-20T23:59:00.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-20T23:59:00.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-20T23:59:00.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-20T23:59:00.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-20T23:59:00.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-20T23:59:00.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-20T23:59:00.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-20T23:59:00.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-20T23:59:00.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-20T23:59:00.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-20T23:59:00.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-20T23:59:00.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-20T23:59:00.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-20T23:59:00.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-20T23:59:00.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-20T23:59:00.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-20T23:59:00.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-20T23:59:00.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-20T23:59:00.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-20T23:59:00.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-20T23:59:00.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-20T23:59:00.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-20T23:59:00.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-20T23:59:00.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-20T23:59:00.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-20T23:59:00.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-20T23:59:00.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-20T23:59:00.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-20T23:59:00.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-20T23:59:00.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-20T23:59:00.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-20T23:59:00.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-20T23:59:00.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-20T23:59:00.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-20T23:59:00.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-20T23:59:00.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-20T23:59:00.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-20T23:59:00.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-20T23:59:00.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-20T23:59:00.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-20T23:59:00.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-20T23:59:00.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-20T23:59:00.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-20T23:59:00.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-20T23:59:00.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-20T23:59:00.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-20T23:59:00.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-20T23:59:00.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-20T23:59:00.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-20T23:59:00.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-20T23:59:00.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-20T23:59:00.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-20T23:59:00.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-20T23:59:00.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-20T23:59:00.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-20T23:59:00.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-20T23:59:00.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-20T23:59:00.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-20T23:59:00.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-20T23:59:00.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-20T23:59:00.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-20T23:59:00.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-20T23:59:00.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-20T23:59:00.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-20T23:59:00.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-20T23:59:00.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-20T23:59:00.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-20T23:59:00.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-20T23:59:00.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-20T23:59:00.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-20T23:59:00.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-20T23:59:00.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-20T23:59:00.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-20T23:59:00.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-20T23:59:00.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-20T23:59:00.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-20T23:59:00.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-20T23:59:00.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-20T23:59:00.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-20T23:59:00.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-20T23:59:00.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-20T23:59:00.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-20T23:59:00.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-20T23:59:00.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-20T23:59:00.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-20T23:59:00.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-20T23:59:00.127 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-20T23:59:00.127 DEBUG:teuthology.orchestra.run.smithi027:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-20T23:59:00.152 INFO:teuthology.orchestra.run.smithi027.stdout:postinstall scriptlet (using /bin/sh): 2024-09-20T23:59:00.153 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-20T23:59:00.153 INFO:teuthology.orchestra.run.smithi027.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-20T23:59:00.153 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-09-20T23:59:00.153 INFO:teuthology.orchestra.run.smithi027.stdout:for file in vmlinuz System.map config; do 2024-09-20T23:59:00.153 INFO:teuthology.orchestra.run.smithi027.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-20T23:59:00.153 INFO:teuthology.orchestra.run.smithi027.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-20T23:59:00.153 INFO:teuthology.orchestra.run.smithi027.stdout: fi 2024-09-20T23:59:00.153 INFO:teuthology.orchestra.run.smithi027.stdout:done 2024-09-20T23:59:00.153 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-20T23:59:00.153 INFO:teuthology.orchestra.run.smithi027.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-20T23:59:00.153 INFO:teuthology.orchestra.run.smithi027.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-20T23:59:00.153 INFO:teuthology.orchestra.run.smithi027.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-20T23:59:00.153 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-09-20T23:59:00.153 INFO:teuthology.orchestra.run.smithi027.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-20T23:59:00.154 INFO:teuthology.orchestra.run.smithi027.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-20T23:59:00.154 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-09-20T23:59:00.154 INFO:teuthology.orchestra.run.smithi027.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-20T23:59:00.154 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-20T23:59:00.154 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-20T23:59:00.154 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-09-20T23:59:00.155 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2024-09-20T23:59:00.231 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2024-09-20T23:59:00.231 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2024-09-20T23:59:00.232 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.06 2024-09-20T23:59:00.232 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 82.el9 2024-09-20T23:59:00.232 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2024-09-20T23:59:00.232 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-20T23:59:00.232 INFO:teuthology.orchestra.run.smithi027.stdout:Group : Unspecified 2024-09-20T23:59:00.232 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 8274670 2024-09-20T23:59:00.232 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2024-09-20T23:59:00.232 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-20T23:59:00.232 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-20T23:59:00.232 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-20T23:59:00.232 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-20T23:59:00.232 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : builder@centos.org 2024-09-20T23:59:00.232 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2024-09-20T23:59:00.232 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-20T23:59:00.232 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2024-09-20T23:59:00.233 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2024-09-20T23:59:00.233 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-20T23:59:00.233 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-20T23:59:00.233 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-20T23:59:00.233 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-20T23:59:00.233 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2024-09-20T23:59:00.233 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-20T23:59:00.233 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2024-09-20T23:59:00.234 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-20T23:59:00.234 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-20T23:59:00.234 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-20T23:59:00.791 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2024-09-20T23:59:02.242 INFO:teuthology.orchestra.run.smithi027.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-20T23:59:02.260 INFO:teuthology.orchestra.run.smithi027.stderr:done 2024-09-20T23:59:02.262 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-09-20T23:59:02.278 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.nCJeFJLRWM 2024-09-20T23:59:02.278 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nCJeFJLRWM 2024-09-20T23:59:02.345 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.nCJeFJLRWM 2024-09-20T23:59:02.482 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.nCJeFJLRWM is 6KB 2024-09-20T23:59:02.502 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.nCJeFJLRWM 2024-09-20T23:59:02.517 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2024-09-20T23:59:02.590 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-20T23:59:02.591 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-20T23:59:02.591 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-20T23:59:02.591 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-20T23:59:02.592 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-20T23:59:02.753 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2024-09-20T23:59:32.756 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-20T23:59:32.756 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-20T23:59:32.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:00:07.407 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-21T00:00:16.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T00:00:16.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:00:19.471 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-21T00:00:31.472 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T00:00:31.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:00:34.543 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-21T00:00:49.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T00:00:49.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:01:08.079 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-21T00:01:26.079 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T00:01:26.080 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:01:26.688 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-09-21T00:01:27.161 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T00:01:27.161 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-21T00:01:27.161 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-09-21T00:01:27.198 INFO:teuthology.orchestra.run.smithi027.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-21T00:01:27.198 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T00:01:27.198 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T00:01:27.198 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T00:01:28.199 DEBUG:teuthology.parallel:result is None 2024-09-21T00:01:28.200 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T00:01:28.208 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T00:01:28.208 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T00:01:28.212 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T00:01:28.215 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T00:01:28.246 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T00:01:28.253 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T00:01:28.297 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T00:01:28.297 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T00:01:28.301 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T00:01:28.303 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T00:01:28.372 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T00:01:28.392 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T00:01:28.392 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T00:01:28.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:01:28.410 DEBUG:teuthology.orchestra.run.smithi098:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T00:01:28.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:01:28.427 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T00:01:28.431 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T00:01:28.452 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T00:01:28.487 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T00:01:28.505 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T00:01:28.508 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T00:01:28.525 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T00:01:28.597 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T00:01:28.628 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T00:01:28.631 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T00:01:28.639 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T00:01:28.640 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T00:01:28.644 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T00:01:28.646 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T00:01:28.699 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T00:01:28.707 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T00:01:28.707 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T00:01:28.711 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T00:01:28.714 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T00:01:28.743 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T00:01:28.851 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T00:01:28.940 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T00:01:29.005 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T00:01:29.078 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:01:29.078 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T00:01:29.134 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T00:01:29.179 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T00:01:29.239 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T00:01:29.323 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T00:01:29.366 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T00:01:29.366 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T00:01:29.433 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T00:01:29.477 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T00:01:29.525 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T00:01:29.608 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T00:01:29.651 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T00:01:29.652 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T00:01:29.717 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2024-09-21T00:01:29.723 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2024-09-21T00:01:29.727 DEBUG:teuthology.orchestra.run.smithi098:> sudo service rsyslog restart 2024-09-21T00:01:29.765 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T00:01:29.796 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T00:01:29.801 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T00:01:30.283 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T00:01:30.383 INFO:teuthology.task.internal:Starting timer... 2024-09-21T00:01:30.384 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T00:01:30.439 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T00:01:30.486 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T00:01:30.487 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2024-09-21T00:01:30.570 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2024-09-21T00:01:30.574 DEBUG:teuthology.orchestra.run.smithi089:> sudo service auditd rotate 2024-09-21T00:01:30.628 INFO:teuthology.orchestra.run.smithi089.stdout:Rotating logs: 2024-09-21T00:01:30.629 DEBUG:teuthology.orchestra.run.smithi098:> sudo service auditd rotate 2024-09-21T00:01:30.682 INFO:teuthology.orchestra.run.smithi098.stdout:Rotating logs: 2024-09-21T00:01:30.684 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T00:01:30.684 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2024-09-21T00:01:30.729 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2024-09-21T00:01:30.729 DEBUG:teuthology.orchestra.run.smithi089:> /usr/sbin/getenforce 2024-09-21T00:01:30.756 INFO:teuthology.orchestra.run.smithi089.stdout:Permissive 2024-09-21T00:01:30.757 DEBUG:teuthology.orchestra.run.smithi098:> /usr/sbin/getenforce 2024-09-21T00:01:30.777 INFO:teuthology.orchestra.run.smithi098.stdout:Permissive 2024-09-21T00:01:30.777 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi089.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi098.front.sepia.ceph.com': 'permissive'} 2024-09-21T00:01:30.778 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T00:01:30.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:01:30.838 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T00:01:30.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:01:30.868 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T00:01:30.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:01:30.894 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T00:01:30.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2024-09-21T00:01:30.954 DEBUG:teuthology.orchestra.run.smithi089:> sudo /usr/sbin/setenforce permissive 2024-09-21T00:01:30.988 DEBUG:teuthology.orchestra.run.smithi098:> sudo /usr/sbin/setenforce permissive 2024-09-21T00:01:31.021 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T00:01:31.054 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T00:01:31.064 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T00:01:31.065 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi089.front.sepia.ceph.com,smithi098.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T00:06:23.756 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi089.front.sepia.ceph.com'), Remote(name='ubuntu@smithi098.front.sepia.ceph.com')] 2024-09-21T00:06:23.757 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T00:06:23.758 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:06:23.844 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-09-21T00:06:23.916 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T00:06:23.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-21T00:06:23.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:06:23.992 DEBUG:teuthology.orchestra.run.smithi089:> true 2024-09-21T00:06:24.070 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-21T00:06:24.071 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-21T00:06:24.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:06:24.145 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-09-21T00:06:24.235 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-21T00:06:24.235 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T00:06:24.254 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T00:06:24.254 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T00:06:24.255 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T00:06:24.259 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T00:06:24.259 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T00:06:24.261 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T00:06:24.261 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T00:06:24.291 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T00:06:24.306 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T00:06:24.307 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T00:06:24.318 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T00:06:24.321 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T00:06:24.333 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: ntpd: command not found 2024-09-21T00:06:24.344 INFO:teuthology.orchestra.run.smithi089.stdout:506 Cannot talk to daemon 2024-09-21T00:06:24.346 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: ntpd: command not found 2024-09-21T00:06:24.349 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T00:06:24.357 INFO:teuthology.orchestra.run.smithi098.stdout:506 Cannot talk to daemon 2024-09-21T00:06:24.359 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T00:06:24.371 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T00:06:24.373 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T00:06:24.385 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T00:06:24.402 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2024-09-21T00:06:24.427 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2024-09-21T00:06:24.429 INFO:teuthology.orchestra.run.smithi089.stderr:bash: line 1: ntpq: command not found 2024-09-21T00:06:24.432 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T00:06:24.432 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2024-09-21T00:06:24.432 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:06:24.432 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:06:24.432 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:06:24.432 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:06:24.445 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-09-21T00:06:24.448 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T00:06:24.448 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-09-21T00:06:24.448 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:06:24.448 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:06:24.448 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:06:24.448 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:06:24.457 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T00:06:24.487 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T00:06:24.591 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-09-21T00:06:24.598 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T00:06:24.598 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-09-21T00:06:24.598 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:06:24.598 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:06:24.598 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:06:24.598 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:06:24.599 INFO:teuthology.run_tasks:Running task install... 2024-09-21T00:06:24.608 DEBUG:teuthology.task.install:project ceph 2024-09-21T00:06:24.608 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T00:06:24.608 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T00:06:24.608 INFO:teuthology.task.install:Using flavor: default 2024-09-21T00:06:24.613 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T00:06:24.613 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T00:06:24.614 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T00:06:24.614 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:06:24.615 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T00:06:24.616 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:06:24.617 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T00:06:24.617 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:06:24.754 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T00:06:24.755 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T00:06:24.764 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T00:06:24.764 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T00:06:24.776 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T00:06:24.776 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T00:06:24.862 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T00:06:24.863 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:06:24.863 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T00:06:24.873 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T00:06:24.873 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T00:06:24.873 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T00:06:24.881 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T00:06:24.881 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T00:06:24.881 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T00:06:24.899 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T00:06:24.899 DEBUG:teuthology.orchestra.run.smithi098:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T00:06:24.907 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T00:06:24.908 DEBUG:teuthology.orchestra.run.smithi089:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T00:06:24.920 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T00:06:24.920 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T00:06:24.964 DEBUG:teuthology.orchestra.run.smithi098:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T00:06:24.975 DEBUG:teuthology.orchestra.run.smithi089:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T00:06:25.020 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T00:06:25.038 DEBUG:teuthology.orchestra.run.smithi098:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T00:06:25.050 DEBUG:teuthology.orchestra.run.smithi089:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T00:06:25.100 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T00:06:25.102 INFO:teuthology.orchestra.run.smithi098.stdout:check_obsoletes = 1 2024-09-21T00:06:25.104 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum clean all 2024-09-21T00:06:25.114 INFO:teuthology.orchestra.run.smithi089.stdout:check_obsoletes = 1 2024-09-21T00:06:25.116 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum clean all 2024-09-21T00:06:25.156 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2024-09-21T00:06:25.159 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2024-09-21T00:06:25.330 INFO:teuthology.orchestra.run.smithi098.stdout:45 files removed 2024-09-21T00:06:25.341 INFO:teuthology.orchestra.run.smithi089.stdout:45 files removed 2024-09-21T00:06:25.357 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T00:06:25.368 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T00:06:25.449 INFO:teuthology.orchestra.run.smithi027.stdout:45 files removed 2024-09-21T00:06:25.483 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T00:06:25.915 INFO:teuthology.orchestra.run.smithi089.stdout:ceph packages for x86_64 312 kB/s | 84 kB 00:00 2024-09-21T00:06:25.918 INFO:teuthology.orchestra.run.smithi098.stdout:ceph packages for x86_64 297 kB/s | 84 kB 00:00 2024-09-21T00:06:26.186 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 293 kB/s | 84 kB 00:00 2024-09-21T00:06:26.198 INFO:teuthology.orchestra.run.smithi089.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-21T00:06:26.203 INFO:teuthology.orchestra.run.smithi098.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-21T00:06:26.479 INFO:teuthology.orchestra.run.smithi098.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T00:06:26.483 INFO:teuthology.orchestra.run.smithi089.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-21T00:06:26.503 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 89 kB/s | 24 kB 00:00 2024-09-21T00:06:26.788 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-21T00:06:27.816 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - BaseOS 6.4 MB/s | 8.3 MB 00:01 2024-09-21T00:06:27.863 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.3 MB 00:01 2024-09-21T00:06:28.703 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.3 MB 00:02 2024-09-21T00:06:30.147 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 18 MB/s | 20 MB 00:01 2024-09-21T00:06:30.734 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-09-21T00:06:32.804 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 20 MB 00:03 2024-09-21T00:06:34.857 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-21T00:06:36.674 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-21T00:06:37.114 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-21T00:06:37.185 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.5 MB 00:02 2024-09-21T00:06:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-21T00:06:38.606 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-21T00:06:38.686 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 19 kB 00:00 2024-09-21T00:06:39.602 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-21T00:06:39.614 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-21T00:06:44.159 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-21T00:06:44.971 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-21T00:06:45.025 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-21T00:06:46.565 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T00:06:46.565 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T00:06:46.577 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T00:06:46.578 INFO:teuthology.orchestra.run.smithi027.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T00:06:46.579 INFO:teuthology.orchestra.run.smithi027.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T00:06:46.580 INFO:teuthology.orchestra.run.smithi027.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T00:06:46.582 INFO:teuthology.orchestra.run.smithi027.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T00:06:46.584 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T00:06:46.585 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T00:06:46.655 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-21T00:06:46.676 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-09-21T00:06:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-21T00:06:46.676 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-09-21T00:06:46.676 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-21T00:06:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T00:06:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T00:06:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T00:06:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T00:06:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T00:06:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T00:06:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T00:06:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T00:06:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T00:06:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T00:06:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T00:06:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T00:06:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T00:06:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T00:06:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T00:06:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T00:06:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T00:06:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T00:06:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T00:06:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T00:06:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T00:06:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T00:06:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T00:06:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T00:06:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T00:06:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T00:06:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T00:06:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T00:06:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T00:06:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T00:06:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T00:06:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T00:06:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T00:06:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T00:06:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T00:06:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T00:06:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T00:06:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-09-21T00:06:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T00:06:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T00:06:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T00:06:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T00:06:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T00:06:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T00:06:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T00:06:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T00:06:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-21T00:06:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T00:06:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T00:06:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T00:06:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T00:06:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T00:06:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T00:06:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T00:06:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T00:06:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T00:06:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T00:06:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T00:06:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T00:06:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T00:06:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T00:06:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T00:06:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T00:06:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T00:06:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T00:06:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T00:06:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T00:06:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T00:06:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T00:06:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T00:06:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T00:06:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T00:06:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T00:06:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T00:06:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T00:06:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T00:06:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T00:06:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T00:06:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T00:06:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T00:06:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T00:06:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T00:06:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T00:06:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T00:06:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T00:06:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T00:06:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T00:06:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T00:06:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T00:06:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T00:06:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T00:06:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T00:06:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T00:06:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T00:06:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T00:06:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T00:06:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T00:06:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T00:06:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T00:06:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T00:06:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T00:06:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T00:06:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T00:06:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T00:06:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T00:06:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T00:06:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T00:06:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T00:06:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T00:06:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T00:06:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T00:06:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T00:06:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T00:06:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T00:06:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T00:06:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T00:06:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T00:06:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T00:06:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T00:06:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T00:06:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T00:06:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T00:06:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T00:06:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T00:06:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T00:06:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T00:06:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T00:06:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T00:06:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T00:06:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T00:06:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T00:06:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T00:06:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T00:06:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T00:06:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T00:06:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T00:06:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T00:06:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T00:06:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T00:06:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T00:06:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T00:06:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T00:06:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T00:06:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T00:06:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T00:06:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T00:06:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T00:06:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T00:06:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T00:06:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T00:06:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T00:06:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T00:06:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T00:06:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T00:06:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T00:06:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T00:06:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T00:06:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T00:06:46.689 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2024-09-21T00:06:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T00:06:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:06:46.690 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-21T00:06:46.690 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-09-21T00:06:46.690 INFO:teuthology.orchestra.run.smithi027.stdout:Install 149 Packages 2024-09-21T00:06:46.690 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 8 Packages 2024-09-21T00:06:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:06:46.691 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 228 M 2024-09-21T00:06:46.691 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-21T00:06:47.081 INFO:teuthology.orchestra.run.smithi089.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T00:06:47.082 INFO:teuthology.orchestra.run.smithi089.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T00:06:47.093 INFO:teuthology.orchestra.run.smithi089.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T00:06:47.094 INFO:teuthology.orchestra.run.smithi089.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T00:06:47.095 INFO:teuthology.orchestra.run.smithi089.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T00:06:47.095 INFO:teuthology.orchestra.run.smithi089.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T00:06:47.097 INFO:teuthology.orchestra.run.smithi089.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T00:06:47.099 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T00:06:47.100 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T00:06:47.155 INFO:teuthology.orchestra.run.smithi098.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T00:06:47.156 INFO:teuthology.orchestra.run.smithi098.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T00:06:47.162 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-09-21T00:06:47.169 INFO:teuthology.orchestra.run.smithi098.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T00:06:47.170 INFO:teuthology.orchestra.run.smithi098.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T00:06:47.171 INFO:teuthology.orchestra.run.smithi098.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T00:06:47.172 INFO:teuthology.orchestra.run.smithi098.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T00:06:47.174 INFO:teuthology.orchestra.run.smithi098.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T00:06:47.177 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T00:06:47.178 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2024-09-21T00:06:47.178 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-09-21T00:06:47.178 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2024-09-21T00:06:47.178 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-09-21T00:06:47.178 INFO:teuthology.orchestra.run.smithi089.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T00:06:47.178 INFO:teuthology.orchestra.run.smithi089.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T00:06:47.178 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T00:06:47.178 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T00:06:47.178 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T00:06:47.178 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T00:06:47.179 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T00:06:47.179 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T00:06:47.179 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T00:06:47.179 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T00:06:47.179 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T00:06:47.179 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T00:06:47.179 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T00:06:47.179 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T00:06:47.179 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T00:06:47.179 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T00:06:47.179 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T00:06:47.179 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T00:06:47.179 INFO:teuthology.orchestra.run.smithi089.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T00:06:47.180 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T00:06:47.180 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T00:06:47.180 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T00:06:47.180 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T00:06:47.180 INFO:teuthology.orchestra.run.smithi089.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T00:06:47.180 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T00:06:47.180 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T00:06:47.180 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T00:06:47.180 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T00:06:47.180 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T00:06:47.180 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T00:06:47.180 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T00:06:47.180 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T00:06:47.180 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T00:06:47.180 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T00:06:47.181 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T00:06:47.181 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T00:06:47.181 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T00:06:47.181 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrading: 2024-09-21T00:06:47.181 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T00:06:47.181 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T00:06:47.181 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T00:06:47.181 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T00:06:47.181 INFO:teuthology.orchestra.run.smithi089.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T00:06:47.181 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T00:06:47.181 INFO:teuthology.orchestra.run.smithi089.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T00:06:47.181 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T00:06:47.181 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-09-21T00:06:47.182 INFO:teuthology.orchestra.run.smithi089.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T00:06:47.182 INFO:teuthology.orchestra.run.smithi089.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T00:06:47.182 INFO:teuthology.orchestra.run.smithi089.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T00:06:47.182 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T00:06:47.182 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T00:06:47.182 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T00:06:47.182 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T00:06:47.182 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T00:06:47.182 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T00:06:47.182 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T00:06:47.182 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T00:06:47.182 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T00:06:47.182 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T00:06:47.182 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T00:06:47.183 INFO:teuthology.orchestra.run.smithi089.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T00:06:47.183 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T00:06:47.183 INFO:teuthology.orchestra.run.smithi089.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T00:06:47.183 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T00:06:47.183 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T00:06:47.183 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T00:06:47.183 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T00:06:47.183 INFO:teuthology.orchestra.run.smithi089.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T00:06:47.183 INFO:teuthology.orchestra.run.smithi089.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T00:06:47.183 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T00:06:47.183 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T00:06:47.183 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T00:06:47.183 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T00:06:47.184 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T00:06:47.184 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T00:06:47.184 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T00:06:47.184 INFO:teuthology.orchestra.run.smithi089.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T00:06:47.184 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T00:06:47.184 INFO:teuthology.orchestra.run.smithi089.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T00:06:47.184 INFO:teuthology.orchestra.run.smithi089.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T00:06:47.184 INFO:teuthology.orchestra.run.smithi089.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T00:06:47.184 INFO:teuthology.orchestra.run.smithi089.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T00:06:47.184 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T00:06:47.184 INFO:teuthology.orchestra.run.smithi089.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T00:06:47.184 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T00:06:47.184 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T00:06:47.184 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T00:06:47.184 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T00:06:47.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T00:06:47.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T00:06:47.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T00:06:47.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T00:06:47.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T00:06:47.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T00:06:47.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T00:06:47.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T00:06:47.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T00:06:47.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T00:06:47.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T00:06:47.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T00:06:47.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T00:06:47.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T00:06:47.186 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T00:06:47.186 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T00:06:47.186 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T00:06:47.186 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T00:06:47.186 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T00:06:47.186 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T00:06:47.186 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T00:06:47.186 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T00:06:47.186 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T00:06:47.186 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T00:06:47.186 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T00:06:47.186 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T00:06:47.186 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T00:06:47.187 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T00:06:47.187 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T00:06:47.187 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T00:06:47.187 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T00:06:47.187 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T00:06:47.187 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T00:06:47.187 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T00:06:47.187 INFO:teuthology.orchestra.run.smithi089.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T00:06:47.187 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T00:06:47.187 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T00:06:47.187 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T00:06:47.187 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T00:06:47.187 INFO:teuthology.orchestra.run.smithi089.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T00:06:47.187 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T00:06:47.188 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T00:06:47.188 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T00:06:47.188 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T00:06:47.188 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T00:06:47.188 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T00:06:47.188 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T00:06:47.188 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T00:06:47.188 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T00:06:47.188 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T00:06:47.188 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T00:06:47.188 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T00:06:47.188 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T00:06:47.188 INFO:teuthology.orchestra.run.smithi089.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T00:06:47.188 INFO:teuthology.orchestra.run.smithi089.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T00:06:47.188 INFO:teuthology.orchestra.run.smithi089.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T00:06:47.189 INFO:teuthology.orchestra.run.smithi089.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T00:06:47.189 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T00:06:47.189 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T00:06:47.189 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T00:06:47.189 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T00:06:47.189 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T00:06:47.189 INFO:teuthology.orchestra.run.smithi089.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T00:06:47.189 INFO:teuthology.orchestra.run.smithi089.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T00:06:47.189 INFO:teuthology.orchestra.run.smithi089.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T00:06:47.189 INFO:teuthology.orchestra.run.smithi089.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T00:06:47.189 INFO:teuthology.orchestra.run.smithi089.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T00:06:47.189 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T00:06:47.189 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T00:06:47.189 INFO:teuthology.orchestra.run.smithi089.stdout:Installing weak dependencies: 2024-09-21T00:06:47.189 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T00:06:47.190 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:06:47.190 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-09-21T00:06:47.190 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2024-09-21T00:06:47.190 INFO:teuthology.orchestra.run.smithi089.stdout:Install 149 Packages 2024-09-21T00:06:47.190 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrade 8 Packages 2024-09-21T00:06:47.190 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:06:47.190 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T00:06:47.191 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 228 M 2024-09-21T00:06:47.191 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-09-21T00:06:47.241 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-21T00:06:47.260 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-09-21T00:06:47.261 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-09-21T00:06:47.261 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-09-21T00:06:47.261 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-21T00:06:47.261 INFO:teuthology.orchestra.run.smithi098.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T00:06:47.261 INFO:teuthology.orchestra.run.smithi098.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T00:06:47.261 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T00:06:47.261 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T00:06:47.261 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T00:06:47.261 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T00:06:47.261 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T00:06:47.261 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T00:06:47.261 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T00:06:47.262 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T00:06:47.262 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T00:06:47.262 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T00:06:47.262 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T00:06:47.262 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T00:06:47.262 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T00:06:47.262 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T00:06:47.262 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T00:06:47.262 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T00:06:47.262 INFO:teuthology.orchestra.run.smithi098.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T00:06:47.262 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T00:06:47.262 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T00:06:47.263 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T00:06:47.263 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T00:06:47.263 INFO:teuthology.orchestra.run.smithi098.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T00:06:47.263 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T00:06:47.263 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T00:06:47.263 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T00:06:47.263 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T00:06:47.263 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T00:06:47.263 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T00:06:47.263 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T00:06:47.263 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T00:06:47.263 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T00:06:47.263 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T00:06:47.264 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T00:06:47.264 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T00:06:47.264 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T00:06:47.264 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrading: 2024-09-21T00:06:47.264 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T00:06:47.264 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T00:06:47.264 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T00:06:47.264 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T00:06:47.264 INFO:teuthology.orchestra.run.smithi098.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T00:06:47.264 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T00:06:47.264 INFO:teuthology.orchestra.run.smithi098.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T00:06:47.264 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T00:06:47.264 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-09-21T00:06:47.264 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T00:06:47.265 INFO:teuthology.orchestra.run.smithi098.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T00:06:47.265 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T00:06:47.265 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T00:06:47.265 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T00:06:47.265 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T00:06:47.265 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T00:06:47.265 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T00:06:47.265 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T00:06:47.265 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T00:06:47.265 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T00:06:47.265 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T00:06:47.265 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T00:06:47.265 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T00:06:47.265 INFO:teuthology.orchestra.run.smithi098.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T00:06:47.266 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T00:06:47.266 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T00:06:47.266 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T00:06:47.266 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T00:06:47.266 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T00:06:47.266 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T00:06:47.266 INFO:teuthology.orchestra.run.smithi098.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T00:06:47.266 INFO:teuthology.orchestra.run.smithi098.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T00:06:47.266 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T00:06:47.266 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T00:06:47.266 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T00:06:47.266 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T00:06:47.267 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T00:06:47.267 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T00:06:47.267 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T00:06:47.267 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T00:06:47.267 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T00:06:47.267 INFO:teuthology.orchestra.run.smithi098.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T00:06:47.267 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T00:06:47.267 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T00:06:47.267 INFO:teuthology.orchestra.run.smithi098.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T00:06:47.267 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T00:06:47.267 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T00:06:47.267 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T00:06:47.267 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T00:06:47.267 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T00:06:47.267 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T00:06:47.268 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T00:06:47.268 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T00:06:47.268 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T00:06:47.268 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T00:06:47.268 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T00:06:47.268 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T00:06:47.268 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T00:06:47.268 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T00:06:47.268 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T00:06:47.268 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T00:06:47.268 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T00:06:47.268 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T00:06:47.268 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T00:06:47.268 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T00:06:47.269 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T00:06:47.269 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T00:06:47.269 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T00:06:47.269 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T00:06:47.269 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T00:06:47.269 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T00:06:47.269 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T00:06:47.269 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T00:06:47.269 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T00:06:47.269 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T00:06:47.269 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T00:06:47.269 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T00:06:47.269 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T00:06:47.269 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T00:06:47.270 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T00:06:47.270 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T00:06:47.270 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T00:06:47.270 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T00:06:47.270 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T00:06:47.270 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T00:06:47.270 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T00:06:47.270 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T00:06:47.270 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T00:06:47.270 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T00:06:47.270 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T00:06:47.270 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T00:06:47.270 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T00:06:47.270 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T00:06:47.271 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T00:06:47.271 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T00:06:47.271 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T00:06:47.271 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T00:06:47.271 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T00:06:47.271 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T00:06:47.271 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T00:06:47.271 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T00:06:47.271 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T00:06:47.271 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T00:06:47.271 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T00:06:47.271 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T00:06:47.271 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T00:06:47.271 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T00:06:47.271 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T00:06:47.272 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T00:06:47.272 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T00:06:47.272 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T00:06:47.272 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T00:06:47.272 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T00:06:47.272 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T00:06:47.272 INFO:teuthology.orchestra.run.smithi098.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T00:06:47.272 INFO:teuthology.orchestra.run.smithi098.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T00:06:47.272 INFO:teuthology.orchestra.run.smithi098.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T00:06:47.272 INFO:teuthology.orchestra.run.smithi098.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T00:06:47.272 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T00:06:47.272 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T00:06:47.272 INFO:teuthology.orchestra.run.smithi098.stdout:Installing weak dependencies: 2024-09-21T00:06:47.273 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T00:06:47.273 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:06:47.273 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-21T00:06:47.273 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-09-21T00:06:47.273 INFO:teuthology.orchestra.run.smithi098.stdout:Install 149 Packages 2024-09-21T00:06:47.273 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrade 8 Packages 2024-09-21T00:06:47.273 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:06:47.273 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 228 M 2024-09-21T00:06:47.273 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-21T00:06:47.464 INFO:teuthology.orchestra.run.smithi089.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 53 kB/s | 6.6 kB 00:00 2024-09-21T00:06:47.690 INFO:teuthology.orchestra.run.smithi089.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.8 MB/s | 873 kB 00:00 2024-09-21T00:06:47.782 INFO:teuthology.orchestra.run.smithi089.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.6 MB/s | 148 kB 00:00 2024-09-21T00:06:47.865 INFO:teuthology.orchestra.run.smithi089.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-21T00:06:47.903 INFO:teuthology.orchestra.run.smithi027.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 32 kB/s | 6.6 kB 00:00 2024-09-21T00:06:47.957 INFO:teuthology.orchestra.run.smithi089.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 17 MB/s | 1.5 MB 00:00 2024-09-21T00:06:48.001 INFO:teuthology.orchestra.run.smithi098.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 35 kB/s | 6.6 kB 00:00 2024-09-21T00:06:48.032 INFO:teuthology.orchestra.run.smithi089.stdout:(6/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 9.1 MB/s | 2.3 MB 00:00 2024-09-21T00:06:48.300 INFO:teuthology.orchestra.run.smithi089.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 15 MB/s | 5.0 MB 00:00 2024-09-21T00:06:48.500 INFO:teuthology.orchestra.run.smithi089.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 17 MB/s | 20 MB 00:01 2024-09-21T00:06:48.542 INFO:teuthology.orchestra.run.smithi089.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 607 kB/s | 25 kB 00:00 2024-09-21T00:06:48.553 INFO:teuthology.orchestra.run.smithi027.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.3 MB/s | 873 kB 00:00 2024-09-21T00:06:48.745 INFO:teuthology.orchestra.run.smithi027.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 774 kB/s | 148 kB 00:00 2024-09-21T00:06:48.909 INFO:teuthology.orchestra.run.smithi089.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 17 MB/s | 10 MB 00:00 2024-09-21T00:06:48.976 INFO:teuthology.orchestra.run.smithi089.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.3 MB/s | 226 kB 00:00 2024-09-21T00:06:49.043 INFO:teuthology.orchestra.run.smithi089.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 407 kB/s | 27 kB 00:00 2024-09-21T00:06:49.101 INFO:teuthology.orchestra.run.smithi089.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 545 kB/s | 32 kB 00:00 2024-09-21T00:06:49.128 INFO:teuthology.orchestra.run.smithi027.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 3.9 MB/s | 5.5 MB 00:01 2024-09-21T00:06:49.160 INFO:teuthology.orchestra.run.smithi098.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 754 kB/s | 873 kB 00:01 2024-09-21T00:06:49.185 INFO:teuthology.orchestra.run.smithi089.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 9.0 MB/s | 766 kB 00:00 2024-09-21T00:06:49.269 INFO:teuthology.orchestra.run.smithi089.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 1.9 MB/s | 158 kB 00:00 2024-09-21T00:06:49.344 INFO:teuthology.orchestra.run.smithi089.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.7 MB/s | 127 kB 00:00 2024-09-21T00:06:49.360 INFO:teuthology.orchestra.run.smithi098.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 741 kB/s | 148 kB 00:00 2024-09-21T00:06:49.378 INFO:teuthology.orchestra.run.smithi027.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 6.1 MB/s | 1.5 MB 00:00 2024-09-21T00:06:49.419 INFO:teuthology.orchestra.run.smithi089.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 6.6 MB/s | 507 kB 00:00 2024-09-21T00:06:49.761 INFO:teuthology.orchestra.run.smithi089.stdout:(18/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 10 MB/s | 18 MB 00:01 2024-09-21T00:06:49.853 INFO:teuthology.orchestra.run.smithi089.stdout:(19/157): librgw2-19.3.0-5120.g18432218.el9.x86 12 MB/s | 5.3 MB 00:00 2024-09-21T00:06:49.878 INFO:teuthology.orchestra.run.smithi089.stdout:(20/157): python3-ceph-argparse-19.3.0-5120.g18 385 kB/s | 45 kB 00:00 2024-09-21T00:06:49.904 INFO:teuthology.orchestra.run.smithi027.stdout:(6/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 9.5 MB/s | 5.0 MB 00:00 2024-09-21T00:06:49.904 INFO:teuthology.orchestra.run.smithi089.stdout:(21/157): python3-ceph-common-19.3.0-5120.g1843 2.9 MB/s | 148 kB 00:00 2024-09-21T00:06:49.929 INFO:teuthology.orchestra.run.smithi089.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 3.1 MB/s | 161 kB 00:00 2024-09-21T00:06:49.969 INFO:teuthology.orchestra.run.smithi089.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 4.8 MB/s | 321 kB 00:00 2024-09-21T00:06:49.995 INFO:teuthology.orchestra.run.smithi089.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 4.6 MB/s | 307 kB 00:00 2024-09-21T00:06:50.037 INFO:teuthology.orchestra.run.smithi089.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.4 MB/s | 99 kB 00:00 2024-09-21T00:06:50.062 INFO:teuthology.orchestra.run.smithi089.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.3 MB/s | 86 kB 00:00 2024-09-21T00:06:50.087 INFO:teuthology.orchestra.run.smithi027.stdout:(7/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 1.7 MB/s | 2.3 MB 00:01 2024-09-21T00:06:50.146 INFO:teuthology.orchestra.run.smithi089.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.0 MB/s | 170 kB 00:00 2024-09-21T00:06:50.238 INFO:teuthology.orchestra.run.smithi089.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 373 kB/s | 34 kB 00:00 2024-09-21T00:06:50.296 INFO:teuthology.orchestra.run.smithi089.stdout:(29/157): rbd-mirror-19.3.0-5120.g18432218.el9. 12 MB/s | 3.1 MB 00:00 2024-09-21T00:06:50.322 INFO:teuthology.orchestra.run.smithi089.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.8 MB/s | 154 kB 00:00 2024-09-21T00:06:50.694 INFO:teuthology.orchestra.run.smithi098.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 1.9 MB/s | 5.5 MB 00:02 2024-09-21T00:06:50.905 INFO:teuthology.orchestra.run.smithi089.stdout:(31/157): ceph-test-19.3.0-5120.g18432218.el9.x 17 MB/s | 40 MB 00:02 2024-09-21T00:06:50.956 INFO:teuthology.orchestra.run.smithi089.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 5.7 MB/s | 292 kB 00:00 2024-09-21T00:06:50.979 INFO:teuthology.orchestra.run.smithi027.stdout:(8/157): ceph-osd-19.3.0-5120.g18432218.el9.x86 17 MB/s | 18 MB 00:01 2024-09-21T00:06:51.023 INFO:teuthology.orchestra.run.smithi089.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 741 kB/s | 49 kB 00:00 2024-09-21T00:06:51.038 INFO:teuthology.orchestra.run.smithi027.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 434 kB/s | 25 kB 00:00 2024-09-21T00:06:51.090 INFO:teuthology.orchestra.run.smithi089.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 247 kB/s | 16 kB 00:00 2024-09-21T00:06:51.094 INFO:teuthology.orchestra.run.smithi098.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 1.3 MB/s | 2.3 MB 00:01 2024-09-21T00:06:51.211 INFO:teuthology.orchestra.run.smithi098.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 2.9 MB/s | 1.5 MB 00:00 2024-09-21T00:06:51.265 INFO:teuthology.orchestra.run.smithi089.stdout:(35/157): ceph-mgr-dashboard-19.3.0-5120.g18432 15 MB/s | 14 MB 00:00 2024-09-21T00:06:51.290 INFO:teuthology.orchestra.run.smithi089.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 1.4 MB/s | 285 kB 00:00 2024-09-21T00:06:51.382 INFO:teuthology.orchestra.run.smithi089.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 6.6 MB/s | 784 kB 00:00 2024-09-21T00:06:51.407 INFO:teuthology.orchestra.run.smithi089.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 157 kB/s | 18 kB 00:00 2024-09-21T00:06:51.499 INFO:teuthology.orchestra.run.smithi089.stdout:(39/157): ceph-mgr-diskprediction-local-19.3.0- 6.3 MB/s | 7.4 MB 00:01 2024-09-21T00:06:51.875 INFO:teuthology.orchestra.run.smithi089.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 162 kB/s | 80 kB 00:00 2024-09-21T00:06:52.058 INFO:teuthology.orchestra.run.smithi089.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 342 kB/s | 191 kB 00:00 2024-09-21T00:06:52.242 INFO:teuthology.orchestra.run.smithi089.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 960 kB/s | 801 kB 00:00 2024-09-21T00:06:52.267 INFO:teuthology.orchestra.run.smithi089.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 646 kB/s | 253 kB 00:00 2024-09-21T00:06:52.359 INFO:teuthology.orchestra.run.smithi089.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 911 kB/s | 106 kB 00:00 2024-09-21T00:06:52.385 INFO:teuthology.orchestra.run.smithi089.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-21T00:06:52.388 INFO:teuthology.orchestra.run.smithi027.stdout:(10/157): ceph-test-19.3.0-5120.g18432218.el9.x 30 MB/s | 40 MB 00:01 2024-09-21T00:06:52.438 INFO:teuthology.orchestra.run.smithi027.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 4.5 MB/s | 226 kB 00:00 2024-09-21T00:06:52.451 INFO:teuthology.orchestra.run.smithi089.stdout:(46/157): python3-cryptography-36.0.1-4.el9.x86 3.1 MB/s | 1.2 MB 00:00 2024-09-21T00:06:52.477 INFO:teuthology.orchestra.run.smithi089.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-21T00:06:52.480 INFO:teuthology.orchestra.run.smithi027.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 649 kB/s | 27 kB 00:00 2024-09-21T00:06:52.502 INFO:teuthology.orchestra.run.smithi089.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-21T00:06:52.531 INFO:teuthology.orchestra.run.smithi027.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 640 kB/s | 32 kB 00:00 2024-09-21T00:06:52.589 INFO:teuthology.orchestra.run.smithi027.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 13 MB/s | 766 kB 00:00 2024-09-21T00:06:52.602 INFO:teuthology.orchestra.run.smithi089.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-09-21T00:06:52.636 INFO:teuthology.orchestra.run.smithi089.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 655 kB/s | 22 kB 00:00 2024-09-21T00:06:52.648 INFO:teuthology.orchestra.run.smithi027.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 2.7 MB/s | 158 kB 00:00 2024-09-21T00:06:52.670 INFO:teuthology.orchestra.run.smithi089.stdout:(51/157): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-09-21T00:06:52.698 INFO:teuthology.orchestra.run.smithi027.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T00:06:52.711 INFO:teuthology.orchestra.run.smithi089.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-09-21T00:06:52.737 INFO:teuthology.orchestra.run.smithi089.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-09-21T00:06:52.748 INFO:teuthology.orchestra.run.smithi027.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 10 MB/s | 507 kB 00:00 2024-09-21T00:06:52.771 INFO:teuthology.orchestra.run.smithi089.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-09-21T00:06:52.796 INFO:teuthology.orchestra.run.smithi089.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 309 kB/s | 18 kB 00:00 2024-09-21T00:06:52.821 INFO:teuthology.orchestra.run.smithi089.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 220 kB/s | 11 kB 00:00 2024-09-21T00:06:52.847 INFO:teuthology.orchestra.run.smithi089.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T00:06:52.872 INFO:teuthology.orchestra.run.smithi089.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-21T00:06:52.897 INFO:teuthology.orchestra.run.smithi089.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-09-21T00:06:52.928 INFO:teuthology.orchestra.run.smithi098.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 2.7 MB/s | 5.0 MB 00:01 2024-09-21T00:06:52.948 INFO:teuthology.orchestra.run.smithi027.stdout:(18/157): librgw2-19.3.0-5120.g18432218.el9.x86 26 MB/s | 5.3 MB 00:00 2024-09-21T00:06:52.956 INFO:teuthology.orchestra.run.smithi089.stdout:(60/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-21T00:06:52.981 INFO:teuthology.orchestra.run.smithi089.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-21T00:06:52.999 INFO:teuthology.orchestra.run.smithi027.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 906 kB/s | 45 kB 00:00 2024-09-21T00:06:53.006 INFO:teuthology.orchestra.run.smithi089.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 488 kB/s | 24 kB 00:00 2024-09-21T00:06:53.032 INFO:teuthology.orchestra.run.smithi089.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-21T00:06:53.049 INFO:teuthology.orchestra.run.smithi027.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.9 MB/s | 148 kB 00:00 2024-09-21T00:06:53.057 INFO:teuthology.orchestra.run.smithi089.stdout:(64/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-09-21T00:06:53.082 INFO:teuthology.orchestra.run.smithi089.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-21T00:06:53.099 INFO:teuthology.orchestra.run.smithi027.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 3.2 MB/s | 161 kB 00:00 2024-09-21T00:06:53.107 INFO:teuthology.orchestra.run.smithi089.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-21T00:06:53.133 INFO:teuthology.orchestra.run.smithi089.stdout:(67/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-21T00:06:53.158 INFO:teuthology.orchestra.run.smithi027.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 5.4 MB/s | 321 kB 00:00 2024-09-21T00:06:53.208 INFO:teuthology.orchestra.run.smithi027.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T00:06:53.267 INFO:teuthology.orchestra.run.smithi027.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.7 MB/s | 99 kB 00:00 2024-09-21T00:06:53.283 INFO:teuthology.orchestra.run.smithi089.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-21T00:06:53.317 INFO:teuthology.orchestra.run.smithi027.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T00:06:53.325 INFO:teuthology.orchestra.run.smithi089.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-09-21T00:06:53.393 INFO:teuthology.orchestra.run.smithi089.stdout:(70/157): openblas-openmp-0.3.26-2.el9.x86_64.r 16 MB/s | 4.9 MB 00:00 2024-09-21T00:06:53.418 INFO:teuthology.orchestra.run.smithi089.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-21T00:06:53.451 INFO:teuthology.orchestra.run.smithi089.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 5.3 MB/s | 910 kB 00:00 2024-09-21T00:06:53.467 INFO:teuthology.orchestra.run.smithi027.stdout:(26/157): rbd-mirror-19.3.0-5120.g18432218.el9. 21 MB/s | 3.1 MB 00:00 2024-09-21T00:06:53.510 INFO:teuthology.orchestra.run.smithi089.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-09-21T00:06:53.518 INFO:teuthology.orchestra.run.smithi027.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T00:06:53.527 INFO:teuthology.orchestra.run.smithi089.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-21T00:06:53.568 INFO:teuthology.orchestra.run.smithi027.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 688 kB/s | 34 kB 00:00 2024-09-21T00:06:53.577 INFO:teuthology.orchestra.run.smithi089.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-21T00:06:53.619 INFO:teuthology.orchestra.run.smithi027.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T00:06:53.669 INFO:teuthology.orchestra.run.smithi089.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-09-21T00:06:53.694 INFO:teuthology.orchestra.run.smithi089.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-21T00:06:53.728 INFO:teuthology.orchestra.run.smithi089.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-21T00:06:53.753 INFO:teuthology.orchestra.run.smithi089.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-09-21T00:06:53.778 INFO:teuthology.orchestra.run.smithi089.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-21T00:06:53.804 INFO:teuthology.orchestra.run.smithi089.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-21T00:06:53.821 INFO:teuthology.orchestra.run.smithi089.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 828 kB/s | 35 kB 00:00 2024-09-21T00:06:53.853 INFO:teuthology.orchestra.run.smithi027.stdout:(30/157): ceph-common-19.3.0-5120.g18432218.el9 3.2 MB/s | 20 MB 00:06 2024-09-21T00:06:53.913 INFO:teuthology.orchestra.run.smithi089.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-21T00:06:53.938 INFO:teuthology.orchestra.run.smithi089.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-21T00:06:53.964 INFO:teuthology.orchestra.run.smithi089.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-21T00:06:53.989 INFO:teuthology.orchestra.run.smithi089.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T00:06:54.014 INFO:teuthology.orchestra.run.smithi089.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-21T00:06:54.019 INFO:teuthology.orchestra.run.smithi027.stdout:(31/157): ceph-radosgw-19.3.0-5120.g18432218.el 2.7 MB/s | 10 MB 00:03 2024-09-21T00:06:54.048 INFO:teuthology.orchestra.run.smithi089.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-21T00:06:54.074 INFO:teuthology.orchestra.run.smithi089.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-21T00:06:54.098 INFO:teuthology.orchestra.run.smithi089.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T00:06:54.111 INFO:teuthology.orchestra.run.smithi027.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.1 MB/s | 292 kB 00:00 2024-09-21T00:06:54.124 INFO:teuthology.orchestra.run.smithi089.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T00:06:54.149 INFO:teuthology.orchestra.run.smithi089.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T00:06:54.170 INFO:teuthology.orchestra.run.smithi027.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 848 kB/s | 49 kB 00:00 2024-09-21T00:06:54.241 INFO:teuthology.orchestra.run.smithi089.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-21T00:06:54.267 INFO:teuthology.orchestra.run.smithi089.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 356 kB/s | 42 kB 00:00 2024-09-21T00:06:54.292 INFO:teuthology.orchestra.run.smithi089.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-09-21T00:06:54.317 INFO:teuthology.orchestra.run.smithi089.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T00:06:54.320 INFO:teuthology.orchestra.run.smithi027.stdout:(34/157): ceph-mgr-dashboard-19.3.0-5120.g18432 20 MB/s | 14 MB 00:00 2024-09-21T00:06:54.342 INFO:teuthology.orchestra.run.smithi089.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-09-21T00:06:54.345 INFO:teuthology.orchestra.run.smithi027.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 94 kB/s | 16 kB 00:00 2024-09-21T00:06:54.368 INFO:teuthology.orchestra.run.smithi089.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-21T00:06:54.371 INFO:teuthology.orchestra.run.smithi027.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T00:06:54.393 INFO:teuthology.orchestra.run.smithi089.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T00:06:54.413 INFO:teuthology.orchestra.run.smithi027.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 445 kB/s | 18 kB 00:00 2024-09-21T00:06:54.418 INFO:teuthology.orchestra.run.smithi089.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T00:06:54.428 INFO:teuthology.orchestra.run.smithi098.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 3.0 MB/s | 20 MB 00:06 2024-09-21T00:06:54.444 INFO:teuthology.orchestra.run.smithi089.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T00:06:54.469 INFO:teuthology.orchestra.run.smithi089.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T00:06:54.478 INFO:teuthology.orchestra.run.smithi098.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 507 kB/s | 25 kB 00:00 2024-09-21T00:06:54.494 INFO:teuthology.orchestra.run.smithi089.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-21T00:06:54.513 INFO:teuthology.orchestra.run.smithi027.stdout:(38/157): cephadm-19.3.0-5120.g18432218.el9.noa 4.6 MB/s | 784 kB 00:00 2024-09-21T00:06:54.547 INFO:teuthology.orchestra.run.smithi027.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 598 kB/s | 80 kB 00:00 2024-09-21T00:06:54.655 INFO:teuthology.orchestra.run.smithi027.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2024-09-21T00:06:54.703 INFO:teuthology.orchestra.run.smithi098.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 5.1 MB/s | 18 MB 00:03 2024-09-21T00:06:54.739 INFO:teuthology.orchestra.run.smithi027.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-21T00:06:54.770 INFO:teuthology.orchestra.run.smithi098.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.3 MB/s | 226 kB 00:00 2024-09-21T00:06:54.794 INFO:teuthology.orchestra.run.smithi089.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-09-21T00:06:54.812 INFO:teuthology.orchestra.run.smithi098.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 652 kB/s | 27 kB 00:00 2024-09-21T00:06:54.814 INFO:teuthology.orchestra.run.smithi027.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.6 MB/s | 801 kB 00:00 2024-09-21T00:06:54.845 INFO:teuthology.orchestra.run.smithi089.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T00:06:54.862 INFO:teuthology.orchestra.run.smithi098.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 639 kB/s | 32 kB 00:00 2024-09-21T00:06:54.864 INFO:teuthology.orchestra.run.smithi027.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T00:06:54.870 INFO:teuthology.orchestra.run.smithi089.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-21T00:06:54.912 INFO:teuthology.orchestra.run.smithi089.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 700 kB/s | 19 kB 00:00 2024-09-21T00:06:54.923 INFO:teuthology.orchestra.run.smithi027.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-21T00:06:54.979 INFO:teuthology.orchestra.run.smithi098.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 6.4 MB/s | 766 kB 00:00 2024-09-21T00:06:54.990 INFO:teuthology.orchestra.run.smithi027.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.2 MB 00:00 2024-09-21T00:06:55.015 INFO:teuthology.orchestra.run.smithi027.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-09-21T00:06:55.021 INFO:teuthology.orchestra.run.smithi089.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-21T00:06:55.029 INFO:teuthology.orchestra.run.smithi098.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 3.1 MB/s | 158 kB 00:00 2024-09-21T00:06:55.046 INFO:teuthology.orchestra.run.smithi089.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T00:06:55.080 INFO:teuthology.orchestra.run.smithi089.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-21T00:06:55.106 INFO:teuthology.orchestra.run.smithi089.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T00:06:55.124 INFO:teuthology.orchestra.run.smithi027.stdout:(47/157): ceph-mgr-diskprediction-local-19.3.0- 5.8 MB/s | 7.4 MB 00:01 2024-09-21T00:06:55.131 INFO:teuthology.orchestra.run.smithi089.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-21T00:06:55.138 INFO:teuthology.orchestra.run.smithi098.stdout:(16/157): ceph-radosgw-19.3.0-5120.g18432218.el 4.7 MB/s | 10 MB 00:02 2024-09-21T00:06:55.149 INFO:teuthology.orchestra.run.smithi027.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.3 MB/s | 215 kB 00:00 2024-09-21T00:06:55.163 INFO:teuthology.orchestra.run.smithi098.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 949 kB/s | 127 kB 00:00 2024-09-21T00:06:55.172 INFO:teuthology.orchestra.run.smithi089.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-21T00:06:55.198 INFO:teuthology.orchestra.run.smithi089.stdout:(114/157): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-09-21T00:06:55.231 INFO:teuthology.orchestra.run.smithi089.stdout:(115/157): python3-asyncssh-2.13.2-5.el9.noarch 9.3 MB/s | 548 kB 00:00 2024-09-21T00:06:55.285 INFO:teuthology.orchestra.run.smithi098.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 4.0 MB/s | 507 kB 00:00 2024-09-21T00:06:55.285 INFO:teuthology.orchestra.run.smithi089.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.4 MB/s | 32 kB 00:00 2024-09-21T00:06:55.285 INFO:teuthology.orchestra.run.smithi089.stdout:(117/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 517 kB/s | 43 kB 00:00 2024-09-21T00:06:55.305 INFO:teuthology.orchestra.run.smithi098.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 1.1 MB/s | 45 kB 00:00 2024-09-21T00:06:55.307 INFO:teuthology.orchestra.run.smithi089.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-21T00:06:55.324 INFO:teuthology.orchestra.run.smithi027.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 611 kB/s | 107 kB 00:00 2024-09-21T00:06:55.333 INFO:teuthology.orchestra.run.smithi089.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T00:06:55.356 INFO:teuthology.orchestra.run.smithi098.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.9 MB/s | 148 kB 00:00 2024-09-21T00:06:55.358 INFO:teuthology.orchestra.run.smithi089.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.9 MB/s | 358 kB 00:00 2024-09-21T00:06:55.358 INFO:teuthology.orchestra.run.smithi027.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 689 kB 00:00 2024-09-21T00:06:55.383 INFO:teuthology.orchestra.run.smithi089.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-21T00:06:55.392 INFO:teuthology.orchestra.run.smithi027.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 324 kB/s | 22 kB 00:00 2024-09-21T00:06:55.400 INFO:teuthology.orchestra.run.smithi089.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 5.1 MB/s | 220 kB 00:00 2024-09-21T00:06:55.406 INFO:teuthology.orchestra.run.smithi098.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 3.2 MB/s | 161 kB 00:00 2024-09-21T00:06:55.417 INFO:teuthology.orchestra.run.smithi027.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-21T00:06:55.426 INFO:teuthology.orchestra.run.smithi089.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-21T00:06:55.451 INFO:teuthology.orchestra.run.smithi089.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-21T00:06:55.459 INFO:teuthology.orchestra.run.smithi027.stdout:(53/157): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-09-21T00:06:55.468 INFO:teuthology.orchestra.run.smithi089.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-21T00:06:55.482 INFO:teuthology.orchestra.run.smithi098.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 4.2 MB/s | 321 kB 00:00 2024-09-21T00:06:55.484 INFO:teuthology.orchestra.run.smithi027.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-09-21T00:06:55.485 INFO:teuthology.orchestra.run.smithi089.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-21T00:06:55.501 INFO:teuthology.orchestra.run.smithi089.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-21T00:06:55.510 INFO:teuthology.orchestra.run.smithi027.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-21T00:06:55.519 INFO:teuthology.orchestra.run.smithi089.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 1.1 MB/s | 20 kB 00:00 2024-09-21T00:06:55.543 INFO:teuthology.orchestra.run.smithi027.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-09-21T00:06:55.548 INFO:teuthology.orchestra.run.smithi098.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 4.5 MB/s | 307 kB 00:00 2024-09-21T00:06:55.569 INFO:teuthology.orchestra.run.smithi089.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-09-21T00:06:55.577 INFO:teuthology.orchestra.run.smithi027.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.5 MB/s | 171 kB 00:00 2024-09-21T00:06:55.595 INFO:teuthology.orchestra.run.smithi089.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T00:06:55.615 INFO:teuthology.orchestra.run.smithi098.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.5 MB/s | 99 kB 00:00 2024-09-21T00:06:55.627 INFO:teuthology.orchestra.run.smithi027.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-21T00:06:55.628 INFO:teuthology.orchestra.run.smithi089.stdout:(131/157): protobuf-compiler-3.14.0-13.el9.x86_ 761 kB/s | 863 kB 00:01 2024-09-21T00:06:55.653 INFO:teuthology.orchestra.run.smithi027.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-09-21T00:06:55.653 INFO:teuthology.orchestra.run.smithi089.stdout:(132/157): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-21T00:06:55.666 INFO:teuthology.orchestra.run.smithi098.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T00:06:55.704 INFO:teuthology.orchestra.run.smithi089.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-09-21T00:06:55.729 INFO:teuthology.orchestra.run.smithi089.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 717 kB/s | 16 kB 00:00 2024-09-21T00:06:55.744 INFO:teuthology.orchestra.run.smithi027.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 6.2 MB/s | 581 kB 00:00 2024-09-21T00:06:55.754 INFO:teuthology.orchestra.run.smithi089.stdout:(135/157): python3-natsort-7.1.1-5.el9.noarch.r 459 kB/s | 58 kB 00:00 2024-09-21T00:06:55.778 INFO:teuthology.orchestra.run.smithi027.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-21T00:06:55.780 INFO:teuthology.orchestra.run.smithi089.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T00:06:55.805 INFO:teuthology.orchestra.run.smithi089.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-21T00:06:55.812 INFO:teuthology.orchestra.run.smithi027.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 365 kB/s | 24 kB 00:00 2024-09-21T00:06:55.831 INFO:teuthology.orchestra.run.smithi089.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-09-21T00:06:55.856 INFO:teuthology.orchestra.run.smithi089.stdout:(139/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-09-21T00:06:55.862 INFO:teuthology.orchestra.run.smithi027.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 570 kB/s | 47 kB 00:00 2024-09-21T00:06:55.881 INFO:teuthology.orchestra.run.smithi089.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T00:06:55.883 INFO:teuthology.orchestra.run.smithi098.stdout:(26/157): librgw2-19.3.0-5120.g18432218.el9.x86 7.3 MB/s | 5.3 MB 00:00 2024-09-21T00:06:55.906 INFO:teuthology.orchestra.run.smithi089.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-21T00:06:55.920 INFO:teuthology.orchestra.run.smithi027.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-21T00:06:55.932 INFO:teuthology.orchestra.run.smithi089.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-21T00:06:55.933 INFO:teuthology.orchestra.run.smithi098.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T00:06:55.957 INFO:teuthology.orchestra.run.smithi089.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 986 kB/s | 74 kB 00:00 2024-09-21T00:06:55.971 INFO:teuthology.orchestra.run.smithi027.stdout:(65/157): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-21T00:06:55.982 INFO:teuthology.orchestra.run.smithi089.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-21T00:06:55.983 INFO:teuthology.orchestra.run.smithi098.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 685 kB/s | 34 kB 00:00 2024-09-21T00:06:55.996 INFO:teuthology.orchestra.run.smithi027.stdout:(66/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-09-21T00:06:56.007 INFO:teuthology.orchestra.run.smithi089.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T00:06:56.033 INFO:teuthology.orchestra.run.smithi089.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-09-21T00:06:56.034 INFO:teuthology.orchestra.run.smithi098.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T00:06:56.059 INFO:teuthology.orchestra.run.smithi089.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T00:06:56.083 INFO:teuthology.orchestra.run.smithi089.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-21T00:06:56.125 INFO:teuthology.orchestra.run.smithi089.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-21T00:06:56.192 INFO:teuthology.orchestra.run.smithi098.stdout:(30/157): rbd-mirror-19.3.0-5120.g18432218.el9. 5.9 MB/s | 3.1 MB 00:00 2024-09-21T00:06:56.368 INFO:teuthology.orchestra.run.smithi089.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 11 MB/s | 3.2 MB 00:00 2024-09-21T00:06:56.434 INFO:teuthology.orchestra.run.smithi089.stdout:(151/157): librados2-19.3.0-5120.g18432218.el9. 9.0 MB/s | 3.4 MB 00:00 2024-09-21T00:06:56.688 INFO:teuthology.orchestra.run.smithi027.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.4 MB/s | 3.0 MB 00:01 2024-09-21T00:06:56.726 INFO:teuthology.orchestra.run.smithi089.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 140 kB/s | 41 kB 00:00 2024-09-21T00:06:56.876 INFO:teuthology.orchestra.run.smithi089.stdout:(153/157): NetworkManager-1.51.0-1.el9.x86_64.r 3.1 MB/s | 2.3 MB 00:00 2024-09-21T00:06:57.018 INFO:teuthology.orchestra.run.smithi089.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 1.4 MB 00:00 2024-09-21T00:06:57.043 INFO:teuthology.orchestra.run.smithi089.stdout:(155/157): NetworkManager-tui-1.51.0-1.el9.x86_ 785 kB/s | 249 kB 00:00 2024-09-21T00:06:57.126 INFO:teuthology.orchestra.run.smithi098.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 7.9 MB/s | 7.4 MB 00:00 2024-09-21T00:06:57.154 INFO:teuthology.orchestra.run.smithi089.stdout:(156/157): NetworkManager-libnm-1.51.0-1.el9.x8 2.3 MB/s | 1.8 MB 00:00 2024-09-21T00:06:57.193 INFO:teuthology.orchestra.run.smithi098.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.3 MB/s | 292 kB 00:00 2024-09-21T00:06:57.202 INFO:teuthology.orchestra.run.smithi089.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-09-21T00:06:57.209 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-09-21T00:06:57.209 INFO:teuthology.orchestra.run.smithi089.stdout:Total 23 MB/s | 228 MB 00:10 2024-09-21T00:06:57.251 INFO:teuthology.orchestra.run.smithi098.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 848 kB/s | 49 kB 00:00 2024-09-21T00:06:57.305 INFO:teuthology.orchestra.run.smithi098.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 308 kB/s | 16 kB 00:00 2024-09-21T00:06:57.372 INFO:teuthology.orchestra.run.smithi098.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 4.2 MB/s | 285 kB 00:00 2024-09-21T00:06:57.530 INFO:teuthology.orchestra.run.smithi027.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-09-21T00:06:57.581 INFO:teuthology.orchestra.run.smithi098.stdout:(36/157): ceph-mgr-dashboard-19.3.0-5120.g18432 9.2 MB/s | 14 MB 00:01 2024-09-21T00:06:57.623 INFO:teuthology.orchestra.run.smithi098.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 3.1 MB/s | 784 kB 00:00 2024-09-21T00:06:57.640 INFO:teuthology.orchestra.run.smithi098.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 315 kB/s | 18 kB 00:00 2024-09-21T00:06:57.764 INFO:teuthology.orchestra.run.smithi027.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.5 MB/s | 4.4 MB 00:01 2024-09-21T00:06:57.889 INFO:teuthology.orchestra.run.smithi027.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T00:06:58.106 INFO:teuthology.orchestra.run.smithi027.stdout:(71/157): openblas-openmp-0.3.26-2.el9.x86_64.r 2.3 MB/s | 4.9 MB 00:02 2024-09-21T00:06:58.173 INFO:teuthology.orchestra.run.smithi027.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 1.4 MB/s | 910 kB 00:00 2024-09-21T00:06:58.273 INFO:teuthology.orchestra.run.smithi027.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 877 kB/s | 87 kB 00:00 2024-09-21T00:06:58.324 INFO:teuthology.orchestra.run.smithi098.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 114 kB/s | 80 kB 00:00 2024-09-21T00:06:58.447 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-09-21T00:06:58.599 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-09-21T00:06:58.599 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-09-21T00:06:58.698 INFO:teuthology.orchestra.run.smithi027.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 1.7 MB/s | 1.0 MB 00:00 2024-09-21T00:06:58.757 INFO:teuthology.orchestra.run.smithi027.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-21T00:06:58.915 INFO:teuthology.orchestra.run.smithi027.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-09-21T00:06:59.049 INFO:teuthology.orchestra.run.smithi098.stdout:(40/157): ceph-test-19.3.0-5120.g18432218.el9.x 8.7 MB/s | 40 MB 00:04 2024-09-21T00:06:59.099 INFO:teuthology.orchestra.run.smithi027.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-09-21T00:06:59.158 INFO:teuthology.orchestra.run.smithi027.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 441 kB/s | 26 kB 00:00 2024-09-21T00:06:59.185 INFO:teuthology.orchestra.run.smithi098.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 519 kB/s | 801 kB 00:01 2024-09-21T00:06:59.275 INFO:teuthology.orchestra.run.smithi027.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 164 kB/s | 19 kB 00:00 2024-09-21T00:06:59.408 INFO:teuthology.orchestra.run.smithi098.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 176 kB/s | 191 kB 00:01 2024-09-21T00:06:59.450 INFO:teuthology.orchestra.run.smithi027.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 985 kB/s | 172 kB 00:00 2024-09-21T00:06:59.517 INFO:teuthology.orchestra.run.smithi027.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 523 kB/s | 35 kB 00:00 2024-09-21T00:07:00.000 INFO:teuthology.orchestra.run.smithi098.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 266 kB/s | 253 kB 00:00 2024-09-21T00:07:00.083 INFO:teuthology.orchestra.run.smithi098.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 160 kB/s | 106 kB 00:00 2024-09-21T00:07:00.404 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-09-21T00:07:00.405 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-09-21T00:07:00.725 INFO:teuthology.orchestra.run.smithi098.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 186 kB/s | 135 kB 00:00 2024-09-21T00:07:00.776 INFO:teuthology.orchestra.run.smithi098.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 180 kB/s | 125 kB 00:00 2024-09-21T00:07:01.009 INFO:teuthology.orchestra.run.smithi098.stdout:(47/157): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 689 kB 00:00 2024-09-21T00:07:01.084 INFO:teuthology.orchestra.run.smithi027.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 1.8 MB/s | 5.8 MB 00:03 2024-09-21T00:07:01.126 INFO:teuthology.orchestra.run.smithi098.stdout:(48/157): bison-3.7.4-5.el9.x86_64.rpm 7.9 MB/s | 943 kB 00:00 2024-09-21T00:07:01.184 INFO:teuthology.orchestra.run.smithi098.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-21T00:07:01.218 INFO:teuthology.orchestra.run.smithi098.stdout:(50/157): python3-cryptography-36.0.1-4.el9.x86 618 kB/s | 1.2 MB 00:02 2024-09-21T00:07:01.234 INFO:teuthology.orchestra.run.smithi027.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-09-21T00:07:01.243 INFO:teuthology.orchestra.run.smithi098.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 370 kB/s | 22 kB 00:00 2024-09-21T00:07:01.294 INFO:teuthology.orchestra.run.smithi098.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-09-21T00:07:01.385 INFO:teuthology.orchestra.run.smithi027.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-09-21T00:07:01.452 INFO:teuthology.orchestra.run.smithi098.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 1.3 MB/s | 313 kB 00:00 2024-09-21T00:07:01.468 INFO:teuthology.orchestra.run.smithi027.stdout:(85/157): python3-babel-2.9.1-2.el9.noarch.rpm 1.9 MB/s | 6.0 MB 00:03 2024-09-21T00:07:01.502 INFO:teuthology.orchestra.run.smithi098.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-09-21T00:07:01.510 INFO:teuthology.orchestra.run.smithi027.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 616 kB/s | 77 kB 00:00 2024-09-21T00:07:01.527 INFO:teuthology.orchestra.run.smithi027.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 724 kB/s | 42 kB 00:00 2024-09-21T00:07:01.553 INFO:teuthology.orchestra.run.smithi098.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-21T00:07:01.612 INFO:teuthology.orchestra.run.smithi098.stdout:(56/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.4 MB/s | 3.0 MB 00:00 2024-09-21T00:07:01.637 INFO:teuthology.orchestra.run.smithi098.stdout:(57/157): python3-urllib3-1.26.5-6.el9.noarch.r 236 kB/s | 215 kB 00:00 2024-09-21T00:07:01.644 INFO:teuthology.orchestra.run.smithi027.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-21T00:07:01.662 INFO:teuthology.orchestra.run.smithi098.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.5 MB/s | 171 kB 00:00 2024-09-21T00:07:01.687 INFO:teuthology.orchestra.run.smithi098.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-21T00:07:01.711 INFO:teuthology.orchestra.run.smithi027.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 269 kB 00:00 2024-09-21T00:07:01.763 INFO:teuthology.orchestra.run.smithi098.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-09-21T00:07:01.769 INFO:teuthology.orchestra.run.smithi027.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 882 kB/s | 51 kB 00:00 2024-09-21T00:07:01.788 INFO:teuthology.orchestra.run.smithi098.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 5.7 MB/s | 581 kB 00:00 2024-09-21T00:07:01.813 INFO:teuthology.orchestra.run.smithi098.stdout:(62/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 257 kB/s | 45 kB 00:00 2024-09-21T00:07:01.820 INFO:teuthology.orchestra.run.smithi027.stdout:(91/157): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-09-21T00:07:01.838 INFO:teuthology.orchestra.run.smithi098.stdout:(63/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 324 kB/s | 24 kB 00:00 2024-09-21T00:07:01.854 INFO:teuthology.orchestra.run.smithi027.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.3 MB/s | 279 kB 00:00 2024-09-21T00:07:01.864 INFO:teuthology.orchestra.run.smithi098.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-21T00:07:01.889 INFO:teuthology.orchestra.run.smithi098.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-21T00:07:01.914 INFO:teuthology.orchestra.run.smithi098.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-21T00:07:01.920 INFO:teuthology.orchestra.run.smithi027.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 626 kB/s | 42 kB 00:00 2024-09-21T00:07:01.940 INFO:teuthology.orchestra.run.smithi098.stdout:(67/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-21T00:07:01.979 INFO:teuthology.orchestra.run.smithi027.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 762 kB/s | 44 kB 00:00 2024-09-21T00:07:02.148 INFO:teuthology.orchestra.run.smithi098.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-09-21T00:07:02.221 INFO:teuthology.orchestra.run.smithi027.stdout:(95/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.9 MB/s | 5.1 MB 00:02 2024-09-21T00:07:02.246 INFO:teuthology.orchestra.run.smithi027.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 827 kB/s | 221 kB 00:00 2024-09-21T00:07:02.271 INFO:teuthology.orchestra.run.smithi027.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-09-21T00:07:02.274 INFO:teuthology.orchestra.run.smithi098.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-21T00:07:02.297 INFO:teuthology.orchestra.run.smithi027.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-21T00:07:02.332 INFO:teuthology.orchestra.run.smithi098.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-21T00:07:02.398 INFO:teuthology.orchestra.run.smithi098.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T00:07:02.466 INFO:teuthology.orchestra.run.smithi098.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 8.6 MB/s | 4.9 MB 00:00 2024-09-21T00:07:02.472 INFO:teuthology.orchestra.run.smithi027.stdout:(99/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-21T00:07:02.497 INFO:teuthology.orchestra.run.smithi027.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 288 kB/s | 58 kB 00:00 2024-09-21T00:07:02.522 INFO:teuthology.orchestra.run.smithi027.stdout:(101/157): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-09-21T00:07:02.524 INFO:teuthology.orchestra.run.smithi098.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T00:07:02.548 INFO:teuthology.orchestra.run.smithi027.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-21T00:07:02.558 INFO:teuthology.orchestra.run.smithi098.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 6.3 MB/s | 1.0 MB 00:00 2024-09-21T00:07:02.573 INFO:teuthology.orchestra.run.smithi027.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 845 kB/s | 64 kB 00:00 2024-09-21T00:07:02.598 INFO:teuthology.orchestra.run.smithi027.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-09-21T00:07:02.617 INFO:teuthology.orchestra.run.smithi098.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-21T00:07:02.665 INFO:teuthology.orchestra.run.smithi027.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-09-21T00:07:02.684 INFO:teuthology.orchestra.run.smithi098.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-21T00:07:02.691 INFO:teuthology.orchestra.run.smithi027.stdout:(106/157): lua-devel-5.4.4-4.el9.x86_64.rpm 156 kB/s | 22 kB 00:00 2024-09-21T00:07:02.733 INFO:teuthology.orchestra.run.smithi027.stdout:(107/157): gperftools-libs-2.9.1-3.el9.x86_64.r 4.5 MB/s | 308 kB 00:00 2024-09-21T00:07:02.768 INFO:teuthology.orchestra.run.smithi098.stdout:(77/157): postgresql-server-13.16-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-09-21T00:07:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 211 kB/s | 19 kB 00:00 2024-09-21T00:07:02.792 INFO:teuthology.orchestra.run.smithi098.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-21T00:07:02.808 INFO:teuthology.orchestra.run.smithi027.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T00:07:02.842 INFO:teuthology.orchestra.run.smithi027.stdout:(110/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-09-21T00:07:02.851 INFO:teuthology.orchestra.run.smithi098.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 307 kB/s | 26 kB 00:00 2024-09-21T00:07:02.867 INFO:teuthology.orchestra.run.smithi027.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 829 kB/s | 49 kB 00:00 2024-09-21T00:07:02.877 INFO:teuthology.orchestra.run.smithi098.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 229 kB/s | 19 kB 00:00 2024-09-21T00:07:02.909 INFO:teuthology.orchestra.run.smithi027.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-21T00:07:02.910 INFO:teuthology.orchestra.run.smithi098.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-21T00:07:02.935 INFO:teuthology.orchestra.run.smithi098.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 592 kB/s | 35 kB 00:00 2024-09-21T00:07:02.943 INFO:teuthology.orchestra.run.smithi027.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-21T00:07:02.993 INFO:teuthology.orchestra.run.smithi027.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-21T00:07:03.002 INFO:teuthology.orchestra.run.smithi098.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-09-21T00:07:03.018 INFO:teuthology.orchestra.run.smithi027.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T00:07:03.052 INFO:teuthology.orchestra.run.smithi027.stdout:(116/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.9 MB/s | 839 kB 00:00 2024-09-21T00:07:03.078 INFO:teuthology.orchestra.run.smithi027.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 553 kB/s | 32 kB 00:00 2024-09-21T00:07:03.078 INFO:teuthology.orchestra.run.smithi098.stdout:(84/157): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-09-21T00:07:03.102 INFO:teuthology.orchestra.run.smithi027.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 571 kB/s | 14 kB 00:00 2024-09-21T00:07:03.103 INFO:teuthology.orchestra.run.smithi098.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-21T00:07:03.137 INFO:teuthology.orchestra.run.smithi027.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.8 MB/s | 172 kB 00:00 2024-09-21T00:07:03.145 INFO:teuthology.orchestra.run.smithi098.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-21T00:07:03.162 INFO:teuthology.orchestra.run.smithi098.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 719 kB/s | 42 kB 00:00 2024-09-21T00:07:03.171 INFO:teuthology.orchestra.run.smithi027.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.1 MB/s | 358 kB 00:00 2024-09-21T00:07:03.213 INFO:teuthology.orchestra.run.smithi027.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-21T00:07:03.229 INFO:teuthology.orchestra.run.smithi098.stdout:(88/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-09-21T00:07:03.238 INFO:teuthology.orchestra.run.smithi027.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-21T00:07:03.254 INFO:teuthology.orchestra.run.smithi098.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-21T00:07:03.272 INFO:teuthology.orchestra.run.smithi027.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-21T00:07:03.288 INFO:teuthology.orchestra.run.smithi098.stdout:(90/157): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-09-21T00:07:03.297 INFO:teuthology.orchestra.run.smithi027.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-21T00:07:03.313 INFO:teuthology.orchestra.run.smithi098.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.2 MB/s | 279 kB 00:00 2024-09-21T00:07:03.322 INFO:teuthology.orchestra.run.smithi027.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 714 kB/s | 18 kB 00:00 2024-09-21T00:07:03.338 INFO:teuthology.orchestra.run.smithi098.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 608 kB/s | 51 kB 00:00 2024-09-21T00:07:03.348 INFO:teuthology.orchestra.run.smithi027.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 932 kB/s | 23 kB 00:00 2024-09-21T00:07:03.364 INFO:teuthology.orchestra.run.smithi098.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-09-21T00:07:03.365 INFO:teuthology.orchestra.run.smithi027.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 1.2 MB/s | 19 kB 00:00 2024-09-21T00:07:03.382 INFO:teuthology.orchestra.run.smithi027.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-21T00:07:03.389 INFO:teuthology.orchestra.run.smithi098.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 826 kB/s | 42 kB 00:00 2024-09-21T00:07:03.423 INFO:teuthology.orchestra.run.smithi098.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 753 kB/s | 44 kB 00:00 2024-09-21T00:07:03.440 INFO:teuthology.orchestra.run.smithi027.stdout:(129/157): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-09-21T00:07:03.448 INFO:teuthology.orchestra.run.smithi098.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-21T00:07:03.457 INFO:teuthology.orchestra.run.smithi027.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-21T00:07:03.481 INFO:teuthology.orchestra.run.smithi098.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-09-21T00:07:03.482 INFO:teuthology.orchestra.run.smithi027.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T00:07:03.499 INFO:teuthology.orchestra.run.smithi027.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-21T00:07:03.507 INFO:teuthology.orchestra.run.smithi098.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-09-21T00:07:03.549 INFO:teuthology.orchestra.run.smithi098.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-21T00:07:03.566 INFO:teuthology.orchestra.run.smithi027.stdout:(133/157): libarrow-9.0.0-11.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-09-21T00:07:03.574 INFO:teuthology.orchestra.run.smithi098.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 861 kB/s | 58 kB 00:00 2024-09-21T00:07:03.616 INFO:teuthology.orchestra.run.smithi098.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-21T00:07:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:(134/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.1 MB/s | 2.0 MB 00:00 2024-09-21T00:07:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 998 kB/s | 16 kB 00:00 2024-09-21T00:07:03.641 INFO:teuthology.orchestra.run.smithi098.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 954 kB/s | 64 kB 00:00 2024-09-21T00:07:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.7 MB/s | 28 kB 00:00 2024-09-21T00:07:03.675 INFO:teuthology.orchestra.run.smithi098.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-09-21T00:07:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-21T00:07:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-21T00:07:03.735 INFO:teuthology.orchestra.run.smithi027.stdout:(139/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.1 MB/s | 272 kB 00:00 2024-09-21T00:07:03.742 INFO:teuthology.orchestra.run.smithi098.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-21T00:07:03.760 INFO:teuthology.orchestra.run.smithi027.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-21T00:07:03.785 INFO:teuthology.orchestra.run.smithi027.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-21T00:07:03.811 INFO:teuthology.orchestra.run.smithi027.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-21T00:07:03.836 INFO:teuthology.orchestra.run.smithi027.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-21T00:07:03.861 INFO:teuthology.orchestra.run.smithi027.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T00:07:03.887 INFO:teuthology.orchestra.run.smithi027.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T00:07:03.926 INFO:teuthology.orchestra.run.smithi098.stdout:(105/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-21T00:07:03.937 INFO:teuthology.orchestra.run.smithi027.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-09-21T00:07:03.970 INFO:teuthology.orchestra.run.smithi027.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-21T00:07:03.976 INFO:teuthology.orchestra.run.smithi098.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 2.3 MB/s | 548 kB 00:00 2024-09-21T00:07:03.996 INFO:teuthology.orchestra.run.smithi027.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-09-21T00:07:04.001 INFO:teuthology.orchestra.run.smithi098.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-21T00:07:04.026 INFO:teuthology.orchestra.run.smithi098.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 3.1 MB/s | 308 kB 00:00 2024-09-21T00:07:04.046 INFO:teuthology.orchestra.run.smithi027.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T00:07:04.060 INFO:teuthology.orchestra.run.smithi098.stdout:(109/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-09-21T00:07:04.085 INFO:teuthology.orchestra.run.smithi098.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-09-21T00:07:04.110 INFO:teuthology.orchestra.run.smithi098.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 3.6 MB/s | 67 kB 00:00 2024-09-21T00:07:04.136 INFO:teuthology.orchestra.run.smithi098.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.4 MB/s | 151 kB 00:00 2024-09-21T00:07:04.161 INFO:teuthology.orchestra.run.smithi098.stdout:(113/157): liboath-2.6.7-2.el9.x86_64.rpm 482 kB/s | 49 kB 00:00 2024-09-21T00:07:04.203 INFO:teuthology.orchestra.run.smithi098.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-21T00:07:04.228 INFO:teuthology.orchestra.run.smithi098.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T00:07:04.254 INFO:teuthology.orchestra.run.smithi098.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T00:07:04.279 INFO:teuthology.orchestra.run.smithi098.stdout:(117/157): python3-asyncssh-2.13.2-5.el9.noarch 4.5 MB/s | 548 kB 00:00 2024-09-21T00:07:04.305 INFO:teuthology.orchestra.run.smithi098.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-21T00:07:04.330 INFO:teuthology.orchestra.run.smithi098.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T00:07:04.355 INFO:teuthology.orchestra.run.smithi098.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-21T00:07:04.388 INFO:teuthology.orchestra.run.smithi098.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T00:07:04.414 INFO:teuthology.orchestra.run.smithi098.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-21T00:07:04.439 INFO:teuthology.orchestra.run.smithi098.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-21T00:07:04.464 INFO:teuthology.orchestra.run.smithi098.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-21T00:07:04.523 INFO:teuthology.orchestra.run.smithi098.stdout:(125/157): libarrow-9.0.0-11.el9.x86_64.rpm 8.5 MB/s | 4.4 MB 00:00 2024-09-21T00:07:04.530 INFO:teuthology.orchestra.run.smithi027.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 4.8 MB/s | 2.3 MB 00:00 2024-09-21T00:07:04.548 INFO:teuthology.orchestra.run.smithi098.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-09-21T00:07:04.582 INFO:teuthology.orchestra.run.smithi098.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-09-21T00:07:04.597 INFO:teuthology.orchestra.run.smithi027.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 5.3 MB/s | 3.2 MB 00:00 2024-09-21T00:07:04.607 INFO:teuthology.orchestra.run.smithi098.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-09-21T00:07:04.633 INFO:teuthology.orchestra.run.smithi098.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-21T00:07:04.639 INFO:teuthology.orchestra.run.smithi027.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 993 kB/s | 41 kB 00:00 2024-09-21T00:07:04.666 INFO:teuthology.orchestra.run.smithi098.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-21T00:07:04.692 INFO:teuthology.orchestra.run.smithi098.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T00:07:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:(153/157): librados2-19.3.0-5120.g18432218.el9. 4.6 MB/s | 3.4 MB 00:00 2024-09-21T00:07:04.725 INFO:teuthology.orchestra.run.smithi098.stdout:(132/157): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-21T00:07:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.4 MB/s | 249 kB 00:00 2024-09-21T00:07:04.767 INFO:teuthology.orchestra.run.smithi098.stdout:(133/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.4 MB/s | 2.0 MB 00:00 2024-09-21T00:07:04.784 INFO:teuthology.orchestra.run.smithi098.stdout:(134/157): python3-natsort-7.1.1-5.el9.noarch.r 625 kB/s | 58 kB 00:00 2024-09-21T00:07:04.801 INFO:teuthology.orchestra.run.smithi098.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 995 kB/s | 16 kB 00:00 2024-09-21T00:07:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 6.4 MB/s | 1.8 MB 00:00 2024-09-21T00:07:04.826 INFO:teuthology.orchestra.run.smithi098.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-09-21T00:07:04.851 INFO:teuthology.orchestra.run.smithi098.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-21T00:07:04.877 INFO:teuthology.orchestra.run.smithi098.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-21T00:07:04.902 INFO:teuthology.orchestra.run.smithi098.stdout:(139/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-09-21T00:07:04.927 INFO:teuthology.orchestra.run.smithi098.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T00:07:04.953 INFO:teuthology.orchestra.run.smithi098.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-21T00:07:04.978 INFO:teuthology.orchestra.run.smithi098.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T00:07:05.003 INFO:teuthology.orchestra.run.smithi098.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-21T00:07:05.028 INFO:teuthology.orchestra.run.smithi098.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T00:07:05.054 INFO:teuthology.orchestra.run.smithi098.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T00:07:05.070 INFO:teuthology.orchestra.run.smithi027.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.2 MB/s | 2.4 MB 00:00 2024-09-21T00:07:05.079 INFO:teuthology.orchestra.run.smithi098.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-21T00:07:05.104 INFO:teuthology.orchestra.run.smithi098.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T00:07:05.130 INFO:teuthology.orchestra.run.smithi098.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-21T00:07:05.148 INFO:teuthology.orchestra.run.smithi027.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 3.1 MB/s | 1.4 MB 00:00 2024-09-21T00:07:05.154 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-21T00:07:05.155 INFO:teuthology.orchestra.run.smithi027.stdout:Total 12 MB/s | 228 MB 00:18 2024-09-21T00:07:05.172 INFO:teuthology.orchestra.run.smithi098.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-21T00:07:05.522 INFO:teuthology.orchestra.run.smithi098.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.1 MB/s | 3.2 MB 00:00 2024-09-21T00:07:05.598 INFO:teuthology.orchestra.run.smithi098.stdout:(151/157): librados2-19.3.0-5120.g18432218.el9. 6.9 MB/s | 3.4 MB 00:00 2024-09-21T00:07:06.156 INFO:teuthology.orchestra.run.smithi098.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 73 kB/s | 41 kB 00:00 2024-09-21T00:07:06.815 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-09-21T00:07:06.906 INFO:teuthology.orchestra.run.smithi098.stdout:(153/157): NetworkManager-libnm-1.51.0-1.el9.x8 1.3 MB/s | 1.8 MB 00:01 2024-09-21T00:07:06.924 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T00:07:06.963 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T00:07:07.018 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T00:07:07.023 INFO:teuthology.orchestra.run.smithi098.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 287 kB/s | 249 kB 00:00 2024-09-21T00:07:07.050 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T00:07:07.075 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T00:07:07.119 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T00:07:07.274 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-21T00:07:07.478 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-21T00:07:07.478 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-21T00:07:07.482 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T00:07:07.602 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T00:07:07.749 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T00:07:07.786 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T00:07:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T00:07:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T00:07:07.936 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T00:07:08.088 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T00:07:08.147 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T00:07:08.175 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T00:07:08.281 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T00:07:08.309 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T00:07:08.372 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T00:07:08.394 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T00:07:08.415 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T00:07:08.562 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T00:07:08.688 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T00:07:08.809 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T00:07:08.908 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T00:07:08.915 INFO:teuthology.orchestra.run.smithi098.stdout:(155/157): NetworkManager-1.51.0-1.el9.x86_64.r 634 kB/s | 2.3 MB 00:03 2024-09-21T00:07:08.977 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T00:07:09.073 INFO:teuthology.orchestra.run.smithi098.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 640 kB/s | 1.4 MB 00:02 2024-09-21T00:07:09.206 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T00:07:09.315 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T00:07:09.344 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T00:07:09.378 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T00:07:09.451 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T00:07:09.496 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T00:07:09.528 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T00:07:09.572 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T00:07:09.605 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T00:07:09.624 INFO:teuthology.orchestra.run.smithi098.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 942 kB/s | 2.4 MB 00:02 2024-09-21T00:07:09.629 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-21T00:07:09.629 INFO:teuthology.orchestra.run.smithi098.stdout:Total 10 MB/s | 228 MB 00:22 2024-09-21T00:07:09.641 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T00:07:09.655 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-21T00:07:09.656 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-21T00:07:09.710 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T00:07:09.763 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T00:07:09.794 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T00:07:09.829 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T00:07:09.910 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T00:07:09.940 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T00:07:09.983 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T00:07:10.012 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T00:07:10.081 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T00:07:10.127 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T00:07:10.202 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T00:07:10.334 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T00:07:10.388 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T00:07:10.880 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-21T00:07:10.908 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T00:07:11.017 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T00:07:11.034 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-21T00:07:11.034 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-21T00:07:11.046 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T00:07:11.089 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T00:07:11.127 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T00:07:11.169 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T00:07:11.327 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T00:07:11.396 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T00:07:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T00:07:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T00:07:11.625 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T00:07:11.656 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T00:07:11.692 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T00:07:11.739 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T00:07:11.775 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T00:07:11.818 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T00:07:11.860 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T00:07:11.997 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T00:07:12.255 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T00:07:12.306 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T00:07:12.408 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T00:07:12.447 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T00:07:12.486 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T00:07:12.512 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T00:07:12.537 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T00:07:12.611 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T00:07:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T00:07:12.788 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T00:07:12.816 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T00:07:12.839 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-21T00:07:12.840 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-21T00:07:12.850 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T00:07:12.899 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T00:07:12.938 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T00:07:12.968 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T00:07:13.003 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T00:07:13.036 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T00:07:13.078 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T00:07:13.112 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T00:07:13.198 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T00:07:13.260 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T00:07:13.291 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T00:07:13.394 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T00:07:13.442 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T00:07:14.152 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T00:07:14.225 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T00:07:14.257 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T00:07:14.355 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T00:07:14.403 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T00:07:14.449 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T00:07:14.537 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T00:07:14.585 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T00:07:14.768 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T00:07:14.799 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T00:07:14.876 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T00:07:15.525 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T00:07:15.648 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T00:07:16.457 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-21T00:07:16.621 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T00:07:16.789 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T00:07:16.862 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T00:07:16.948 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T00:07:16.972 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T00:07:16.976 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T00:07:17.009 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T00:07:17.024 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T00:07:17.080 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T00:07:17.138 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T00:07:17.337 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T00:07:17.380 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T00:07:17.472 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T00:07:17.502 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T00:07:17.538 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T00:07:17.538 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T00:07:17.655 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T00:07:17.818 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T00:07:17.848 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T00:07:17.946 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T00:07:17.978 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T00:07:18.018 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T00:07:18.043 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T00:07:18.075 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T00:07:18.154 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T00:07:18.177 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T00:07:18.222 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T00:07:18.286 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T00:07:18.311 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T00:07:18.443 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T00:07:18.471 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T00:07:18.553 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T00:07:18.580 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T00:07:18.623 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T00:07:18.807 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T00:07:18.951 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T00:07:19.137 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T00:07:19.161 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-21T00:07:19.238 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T00:07:19.303 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T00:07:19.339 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T00:07:19.433 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T00:07:19.614 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T00:07:19.621 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T00:07:19.655 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T00:07:19.680 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T00:07:19.724 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T00:07:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T00:07:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T00:07:19.847 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T00:07:19.932 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T00:07:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T00:07:20.062 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T00:07:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T00:07:20.119 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T00:07:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T00:07:20.180 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T00:07:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T00:07:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T00:07:20.307 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T00:07:20.331 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T00:07:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T00:07:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T00:07:20.406 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T00:07:20.426 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T00:07:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T00:07:20.436 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T00:07:20.465 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T00:07:20.466 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T00:07:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T00:07:20.581 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T00:07:20.619 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T00:07:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T00:07:20.653 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T00:07:20.676 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T00:07:20.703 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T00:07:20.749 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T00:07:20.793 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T00:07:20.806 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T00:07:20.846 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T00:07:20.902 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T00:07:20.908 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T00:07:20.930 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T00:07:20.952 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T00:07:21.035 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T00:07:21.073 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T00:07:21.078 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T00:07:21.099 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T00:07:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T00:07:21.156 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T00:07:21.156 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T00:07:21.156 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T00:07:21.157 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:07:21.205 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T00:07:21.268 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T00:07:21.396 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T00:07:21.497 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T00:07:21.572 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T00:07:21.796 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T00:07:21.870 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T00:07:21.900 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T00:07:21.935 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T00:07:21.970 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T00:07:21.970 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T00:07:22.004 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T00:07:22.043 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T00:07:22.048 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T00:07:22.091 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T00:07:22.094 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T00:07:22.131 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T00:07:22.147 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T00:07:22.174 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T00:07:22.216 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T00:07:22.244 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T00:07:22.311 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T00:07:22.332 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T00:07:22.366 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T00:07:22.396 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T00:07:22.406 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T00:07:22.432 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T00:07:22.514 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T00:07:22.534 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T00:07:22.543 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T00:07:22.578 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T00:07:22.602 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T00:07:22.607 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T00:07:22.689 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T00:07:22.694 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T00:07:22.728 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T00:07:22.739 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T00:07:22.764 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T00:07:22.813 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T00:07:22.828 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T00:07:22.872 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T00:07:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T00:07:22.945 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T00:07:22.976 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T00:07:22.999 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T00:07:23.158 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T00:07:23.208 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T00:07:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T00:07:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T00:07:23.425 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T00:07:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T00:07:23.528 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T00:07:23.539 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T00:07:23.572 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T00:07:23.637 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T00:07:23.672 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T00:07:23.674 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T00:07:23.717 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T00:07:23.755 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T00:07:23.797 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T00:07:23.841 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T00:07:23.889 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T00:07:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T00:07:23.956 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T00:07:23.960 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T00:07:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T00:07:24.030 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T00:07:24.040 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T00:07:24.077 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T00:07:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T00:07:24.121 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T00:07:24.154 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T00:07:24.185 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T00:07:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T00:07:24.230 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T00:07:24.259 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T00:07:24.290 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T00:07:24.327 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T00:07:24.342 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T00:07:24.374 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T00:07:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T00:07:24.426 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T00:07:24.442 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T00:07:24.469 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T00:07:24.511 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T00:07:24.570 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T00:07:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T00:07:24.647 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T00:07:24.689 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T00:07:24.730 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T00:07:24.833 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T00:07:24.863 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T00:07:24.903 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T00:07:24.929 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T00:07:24.954 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T00:07:25.029 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T00:07:25.155 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T00:07:25.195 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T00:07:25.222 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T00:07:25.257 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T00:07:25.296 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T00:07:25.328 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T00:07:25.358 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T00:07:25.384 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T00:07:25.417 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T00:07:25.459 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T00:07:25.493 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T00:07:25.579 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T00:07:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T00:07:25.624 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T00:07:25.640 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T00:07:25.655 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T00:07:25.674 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T00:07:25.759 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T00:07:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T00:07:25.815 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T00:07:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T00:07:25.912 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T00:07:26.019 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T00:07:26.068 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T00:07:26.317 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T00:07:27.059 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T00:07:27.059 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T00:07:27.060 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T00:07:27.060 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T00:07:27.060 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T00:07:27.060 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T00:07:27.060 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T00:07:27.060 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T00:07:27.061 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T00:07:27.061 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T00:07:27.132 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T00:07:27.163 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T00:07:27.240 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T00:07:27.286 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T00:07:27.442 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T00:07:27.873 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T00:07:27.994 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T00:07:29.083 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T00:07:29.188 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T00:07:29.214 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T00:07:29.224 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T00:07:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T00:07:29.301 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T00:07:29.329 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T00:07:29.362 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T00:07:29.668 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T00:07:29.688 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T00:07:29.734 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T00:07:29.752 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T00:07:29.809 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T00:07:29.839 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T00:07:29.875 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T00:07:29.875 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T00:07:29.925 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T00:07:29.974 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T00:07:30.382 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T00:07:30.412 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T00:07:30.483 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T00:07:30.514 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T00:07:30.556 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T00:07:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T00:07:30.680 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T00:07:30.707 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T00:07:32.747 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T00:07:32.952 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T00:07:33.328 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T00:07:33.364 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T00:07:33.364 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /sys 2024-09-21T00:07:33.364 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /proc 2024-09-21T00:07:33.364 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /mnt 2024-09-21T00:07:33.364 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /var/tmp 2024-09-21T00:07:33.364 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /home 2024-09-21T00:07:33.364 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /root 2024-09-21T00:07:33.364 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /tmp 2024-09-21T00:07:33.364 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:07:33.369 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T00:07:33.519 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T00:07:33.546 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T00:07:33.555 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T00:07:33.563 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T00:07:33.563 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:33.563 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T00:07:33.563 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:07:33.642 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T00:07:33.642 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T00:07:33.642 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T00:07:33.642 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:07:33.673 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T00:07:34.066 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T00:07:34.106 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T00:07:34.234 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T00:07:34.234 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T00:07:34.234 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T00:07:34.234 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:07:34.266 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T00:07:35.158 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T00:07:35.185 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T00:07:35.186 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:35.186 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T00:07:35.186 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T00:07:35.186 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T00:07:35.186 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:07:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T00:07:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T00:07:35.433 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:35.433 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T00:07:35.433 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T00:07:35.433 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T00:07:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:07:35.902 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T00:07:35.925 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T00:07:35.925 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:35.926 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T00:07:35.926 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T00:07:35.926 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T00:07:35.926 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:07:36.032 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T00:07:36.089 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T00:07:36.152 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T00:07:36.338 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T00:07:36.386 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T00:07:36.464 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T00:07:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T00:07:36.684 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T00:07:36.720 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T00:07:36.769 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T00:07:36.796 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T00:07:37.198 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T00:07:37.229 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T00:07:37.261 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T00:07:37.327 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T00:07:37.360 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T00:07:40.530 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T00:07:40.584 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T00:07:41.546 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T00:07:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T00:07:41.690 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T00:07:41.859 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T00:07:41.887 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T00:07:41.911 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T00:07:41.911 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:41.911 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T00:07:41.911 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T00:07:41.911 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T00:07:41.911 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:07:41.954 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T00:07:41.966 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T00:07:42.065 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T00:07:42.110 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T00:07:43.063 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T00:07:43.090 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T00:07:43.114 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T00:07:43.114 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:43.114 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T00:07:43.114 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T00:07:43.114 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T00:07:43.114 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:07:43.158 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T00:07:43.182 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T00:07:43.182 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:43.182 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T00:07:43.182 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:07:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T00:07:43.248 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T00:07:43.248 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:43.248 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T00:07:43.248 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T00:07:43.248 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T00:07:43.249 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:07:43.568 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T00:07:43.591 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T00:07:43.593 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:43.593 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T00:07:43.593 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T00:07:43.593 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T00:07:43.593 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:07:45.609 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T00:07:45.609 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /sys 2024-09-21T00:07:45.609 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /proc 2024-09-21T00:07:45.609 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /mnt 2024-09-21T00:07:45.609 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /var/tmp 2024-09-21T00:07:45.610 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /home 2024-09-21T00:07:45.610 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /root 2024-09-21T00:07:45.610 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /tmp 2024-09-21T00:07:45.610 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:07:45.778 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T00:07:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T00:07:45.794 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:45.794 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T00:07:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:07:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T00:07:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T00:07:47.417 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:47.417 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T00:07:47.417 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T00:07:47.417 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T00:07:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:07:47.572 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T00:07:47.613 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T00:07:47.625 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T00:07:47.626 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2024-09-21T00:07:47.626 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2024-09-21T00:07:47.626 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2024-09-21T00:07:47.626 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2024-09-21T00:07:47.626 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2024-09-21T00:07:47.626 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /root 2024-09-21T00:07:47.626 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2024-09-21T00:07:47.626 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:07:47.654 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T00:07:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T00:07:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T00:07:47.682 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:47.682 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T00:07:47.682 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T00:07:47.682 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T00:07:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:07:47.686 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T00:07:47.713 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T00:07:47.845 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T00:07:47.883 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T00:07:47.883 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:47.883 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T00:07:47.883 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:07:47.893 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T00:07:47.944 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T00:07:47.975 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T00:07:48.004 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T00:07:48.042 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T00:07:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T00:07:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T00:07:48.174 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:48.174 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T00:07:48.174 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T00:07:48.174 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T00:07:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:07:48.281 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T00:07:48.337 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T00:07:48.401 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T00:07:48.588 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T00:07:48.643 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T00:07:48.655 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T00:07:48.691 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T00:07:48.711 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T00:07:48.721 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T00:07:48.810 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T00:07:48.948 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T00:07:48.994 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T00:07:49.025 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T00:07:49.053 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T00:07:49.480 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T00:07:49.510 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T00:07:49.535 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T00:07:49.542 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T00:07:49.608 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T00:07:49.614 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T00:07:49.642 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T00:07:49.655 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T00:07:49.672 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T00:07:49.733 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T00:07:49.733 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:49.733 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T00:07:49.734 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T00:07:49.734 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T00:07:49.734 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:07:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T00:07:49.768 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T00:07:49.793 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T00:07:49.819 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T00:07:49.819 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T00:07:49.912 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T00:07:49.912 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T00:07:49.996 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T00:07:49.997 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T00:07:50.021 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T00:07:50.026 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T00:07:50.026 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T00:07:50.050 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T00:07:50.050 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:50.050 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T00:07:50.050 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T00:07:50.050 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T00:07:50.051 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:07:50.206 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T00:07:50.206 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T00:07:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T00:07:50.363 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T00:07:50.599 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T00:07:50.651 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T00:07:50.652 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:50.652 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T00:07:50.652 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T00:07:50.652 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T00:07:50.652 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:07:50.783 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T00:07:50.844 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T00:07:50.925 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T00:07:51.187 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T00:07:51.253 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T00:07:51.351 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T00:07:51.469 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T00:07:51.637 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T00:07:51.694 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T00:07:51.743 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T00:07:51.776 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T00:07:52.359 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T00:07:52.401 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T00:07:52.452 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T00:07:52.535 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T00:07:52.573 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T00:07:52.836 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T00:07:52.906 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T00:07:53.865 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T00:07:53.890 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T00:07:54.005 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T00:07:54.174 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T00:07:54.200 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T00:07:54.225 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T00:07:54.225 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:54.225 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T00:07:54.225 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T00:07:54.225 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T00:07:54.225 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:07:54.276 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T00:07:54.288 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T00:07:54.386 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T00:07:54.440 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T00:07:55.400 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T00:07:55.428 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T00:07:55.453 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T00:07:55.453 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:55.453 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T00:07:55.453 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T00:07:55.454 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T00:07:55.454 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:07:55.521 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T00:07:55.544 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T00:07:55.544 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:55.544 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T00:07:55.544 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:07:55.588 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T00:07:55.611 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T00:07:55.611 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:55.611 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T00:07:55.611 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T00:07:55.611 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T00:07:55.612 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:07:55.930 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T00:07:55.954 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T00:07:55.954 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:55.954 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T00:07:55.954 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T00:07:55.955 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T00:07:55.955 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:07:56.467 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T00:07:56.546 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T00:07:56.751 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T00:07:56.752 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T00:07:56.752 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T00:07:56.752 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T00:07:56.752 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T00:07:56.752 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T00:07:56.752 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T00:07:56.752 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T00:07:56.752 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T00:07:56.752 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T00:07:56.752 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T00:07:56.752 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T00:07:56.752 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T00:07:56.753 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T00:07:56.753 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T00:07:56.753 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T00:07:56.753 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T00:07:56.753 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T00:07:56.753 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T00:07:56.753 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T00:07:56.753 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T00:07:56.753 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T00:07:56.753 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T00:07:56.753 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T00:07:56.753 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T00:07:56.753 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T00:07:56.753 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T00:07:56.754 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T00:07:56.765 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T00:07:56.765 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T00:07:56.766 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T00:07:56.766 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T00:07:56.766 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T00:07:56.766 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T00:07:56.766 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T00:07:56.766 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T00:07:56.766 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T00:07:56.766 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T00:07:56.766 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T00:07:56.766 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T00:07:56.766 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T00:07:56.766 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T00:07:56.766 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T00:07:56.767 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T00:07:56.767 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T00:07:56.767 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T00:07:56.767 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T00:07:56.767 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T00:07:56.767 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T00:07:56.767 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T00:07:56.767 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T00:07:56.767 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T00:07:56.767 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T00:07:56.767 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T00:07:56.767 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T00:07:56.767 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T00:07:56.767 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T00:07:56.768 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T00:07:56.768 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T00:07:56.768 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T00:07:56.768 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T00:07:56.768 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T00:07:56.768 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T00:07:56.768 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T00:07:56.768 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T00:07:56.768 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T00:07:56.768 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T00:07:56.768 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T00:07:56.768 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T00:07:56.768 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T00:07:56.768 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T00:07:56.769 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T00:07:56.769 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T00:07:56.769 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T00:07:56.769 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T00:07:56.769 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T00:07:56.769 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T00:07:56.769 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T00:07:56.769 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T00:07:56.769 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T00:07:56.769 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T00:07:56.769 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T00:07:56.769 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T00:07:56.769 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T00:07:56.769 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T00:07:56.770 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T00:07:56.770 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T00:07:56.770 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T00:07:56.770 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T00:07:56.770 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T00:07:56.770 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T00:07:56.770 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T00:07:56.770 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T00:07:56.770 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T00:07:56.770 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T00:07:56.770 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T00:07:56.770 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T00:07:56.770 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T00:07:56.771 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T00:07:56.771 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T00:07:56.771 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T00:07:56.771 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T00:07:56.771 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T00:07:56.771 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T00:07:56.771 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T00:07:56.771 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T00:07:56.771 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T00:07:56.771 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T00:07:56.771 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T00:07:56.771 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T00:07:56.771 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T00:07:56.771 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T00:07:56.771 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T00:07:56.772 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T00:07:56.772 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T00:07:56.772 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T00:07:56.772 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T00:07:56.772 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T00:07:56.772 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T00:07:56.772 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T00:07:56.772 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T00:07:56.772 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T00:07:56.772 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T00:07:56.772 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T00:07:56.772 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T00:07:56.772 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T00:07:56.772 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T00:07:56.772 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T00:07:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T00:07:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T00:07:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T00:07:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T00:07:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T00:07:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T00:07:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T00:07:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T00:07:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T00:07:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T00:07:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T00:07:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T00:07:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T00:07:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T00:07:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T00:07:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T00:07:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T00:07:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T00:07:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T00:07:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T00:07:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T00:07:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T00:07:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T00:07:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T00:07:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T00:07:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T00:07:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T00:07:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T00:07:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T00:07:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T00:07:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T00:07:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T00:07:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T00:07:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T00:07:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T00:07:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T00:07:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T00:07:57.707 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T00:07:57.738 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T00:07:57.898 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T00:07:58.094 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T00:07:58.148 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T00:07:58.202 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T00:07:58.202 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:58.203 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T00:07:58.203 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T00:07:58.203 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T00:07:58.203 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:07:58.265 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T00:07:58.293 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T00:07:58.415 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T00:07:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T00:07:59.325 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T00:07:59.325 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:07:59.325 INFO:teuthology.orchestra.run.smithi089.stdout:Upgraded: 2024-09-21T00:07:59.325 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T00:07:59.325 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T00:07:59.325 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T00:07:59.325 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T00:07:59.325 INFO:teuthology.orchestra.run.smithi089.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.325 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.325 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T00:07:59.326 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T00:07:59.326 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-09-21T00:07:59.326 INFO:teuthology.orchestra.run.smithi089.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T00:07:59.326 INFO:teuthology.orchestra.run.smithi089.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T00:07:59.326 INFO:teuthology.orchestra.run.smithi089.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T00:07:59.326 INFO:teuthology.orchestra.run.smithi089.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T00:07:59.326 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.326 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.326 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.326 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.326 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:07:59.326 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.326 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.327 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.327 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:07:59.327 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:07:59.327 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:07:59.327 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:07:59.327 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:07:59.327 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.329 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.329 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:07:59.329 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.329 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.329 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.330 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:07:59.330 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:07:59.330 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.330 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.330 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:07:59.330 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T00:07:59.330 INFO:teuthology.orchestra.run.smithi089.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T00:07:59.330 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T00:07:59.330 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T00:07:59.330 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T00:07:59.330 INFO:teuthology.orchestra.run.smithi089.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T00:07:59.330 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T00:07:59.330 INFO:teuthology.orchestra.run.smithi089.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T00:07:59.331 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T00:07:59.331 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T00:07:59.331 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T00:07:59.331 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.331 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.331 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.331 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T00:07:59.331 INFO:teuthology.orchestra.run.smithi089.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T00:07:59.331 INFO:teuthology.orchestra.run.smithi089.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T00:07:59.331 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T00:07:59.331 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T00:07:59.331 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T00:07:59.331 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.332 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.332 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T00:07:59.332 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.332 INFO:teuthology.orchestra.run.smithi089.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T00:07:59.332 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T00:07:59.332 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T00:07:59.332 INFO:teuthology.orchestra.run.smithi089.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T00:07:59.332 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T00:07:59.332 INFO:teuthology.orchestra.run.smithi089.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T00:07:59.332 INFO:teuthology.orchestra.run.smithi089.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T00:07:59.333 INFO:teuthology.orchestra.run.smithi089.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T00:07:59.333 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T00:07:59.333 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T00:07:59.333 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T00:07:59.333 INFO:teuthology.orchestra.run.smithi089.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T00:07:59.333 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T00:07:59.333 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T00:07:59.333 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T00:07:59.333 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T00:07:59.333 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T00:07:59.333 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T00:07:59.333 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T00:07:59.333 INFO:teuthology.orchestra.run.smithi089.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T00:07:59.333 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T00:07:59.334 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T00:07:59.334 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T00:07:59.334 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.334 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.334 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.334 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T00:07:59.334 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T00:07:59.334 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T00:07:59.334 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T00:07:59.334 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T00:07:59.334 INFO:teuthology.orchestra.run.smithi089.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T00:07:59.334 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T00:07:59.334 INFO:teuthology.orchestra.run.smithi089.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T00:07:59.335 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T00:07:59.335 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T00:07:59.335 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T00:07:59.335 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T00:07:59.335 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T00:07:59.335 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T00:07:59.335 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T00:07:59.335 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T00:07:59.335 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T00:07:59.335 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T00:07:59.335 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T00:07:59.335 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T00:07:59.335 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T00:07:59.336 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T00:07:59.336 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T00:07:59.336 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T00:07:59.336 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T00:07:59.336 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T00:07:59.336 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T00:07:59.336 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T00:07:59.336 INFO:teuthology.orchestra.run.smithi089.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T00:07:59.336 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T00:07:59.336 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T00:07:59.336 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T00:07:59.336 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T00:07:59.336 INFO:teuthology.orchestra.run.smithi089.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T00:07:59.336 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T00:07:59.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T00:07:59.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T00:07:59.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T00:07:59.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T00:07:59.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T00:07:59.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T00:07:59.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T00:07:59.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T00:07:59.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T00:07:59.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T00:07:59.338 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T00:07:59.338 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T00:07:59.338 INFO:teuthology.orchestra.run.smithi089.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T00:07:59.338 INFO:teuthology.orchestra.run.smithi089.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T00:07:59.338 INFO:teuthology.orchestra.run.smithi089.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T00:07:59.338 INFO:teuthology.orchestra.run.smithi089.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T00:07:59.338 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T00:07:59.338 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T00:07:59.338 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T00:07:59.338 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T00:07:59.338 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T00:07:59.338 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T00:07:59.338 INFO:teuthology.orchestra.run.smithi089.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T00:07:59.338 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.338 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.339 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:07:59.339 INFO:teuthology.orchestra.run.smithi089.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T00:07:59.339 INFO:teuthology.orchestra.run.smithi089.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T00:07:59.339 INFO:teuthology.orchestra.run.smithi089.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T00:07:59.339 INFO:teuthology.orchestra.run.smithi089.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T00:07:59.339 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T00:07:59.339 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T00:07:59.339 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T00:07:59.340 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:07:59.340 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-09-21T00:07:59.589 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T00:07:59.626 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T00:07:59.680 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T00:07:59.680 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:59.680 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T00:07:59.680 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T00:07:59.680 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T00:07:59.680 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:07:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T00:07:59.775 DEBUG:teuthology.parallel:result is None 2024-09-21T00:07:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T00:07:59.778 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:59.778 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T00:07:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:07:59.836 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T00:07:59.888 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T00:07:59.889 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:07:59.889 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T00:07:59.889 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T00:07:59.889 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T00:07:59.889 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:07:59.927 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T00:07:59.968 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T00:08:00.008 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T00:08:00.041 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T00:08:00.068 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T00:08:00.248 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T00:08:00.270 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T00:08:00.298 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T00:08:00.324 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T00:08:00.324 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:08:00.324 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T00:08:00.324 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T00:08:00.324 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T00:08:00.324 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:08:00.329 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T00:08:00.388 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T00:08:00.438 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T00:08:01.062 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T00:08:01.294 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T00:08:01.294 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T00:08:01.951 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T00:08:02.033 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T00:08:02.074 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T00:08:02.155 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T00:08:02.186 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T00:08:02.212 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T00:08:02.238 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T00:08:02.238 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T00:08:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T00:08:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T00:08:02.406 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T00:08:02.431 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T00:08:02.436 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T00:08:02.436 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T00:08:02.624 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T00:08:02.625 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T00:08:02.709 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T00:08:02.772 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T00:08:04.959 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T00:08:05.010 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T00:08:05.061 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T00:08:05.098 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T00:08:05.133 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T00:08:05.369 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T00:08:05.421 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T00:08:05.450 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T00:08:05.483 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T00:08:05.518 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T00:08:06.202 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T00:08:06.249 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T00:08:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T00:08:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T00:08:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T00:08:07.820 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T00:08:07.923 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T00:08:07.967 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T00:08:08.000 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T00:08:08.037 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T00:08:08.037 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T00:08:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T00:08:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T00:08:08.262 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T00:08:08.286 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T00:08:08.302 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T00:08:08.302 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T00:08:08.555 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T00:08:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T00:08:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T00:08:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T00:08:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T00:08:08.969 INFO:teuthology.orchestra.run.smithi027.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-21T00:08:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:08:09.305 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T00:08:09.305 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T00:08:09.305 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T00:08:09.305 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T00:08:09.305 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T00:08:09.305 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T00:08:09.305 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T00:08:09.305 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T00:08:09.305 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T00:08:09.306 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T00:08:09.306 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T00:08:09.306 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T00:08:09.306 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T00:08:09.306 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T00:08:09.306 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T00:08:09.306 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T00:08:09.306 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T00:08:09.306 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T00:08:09.306 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T00:08:09.306 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T00:08:09.307 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T00:08:09.307 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T00:08:09.307 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T00:08:09.307 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T00:08:09.307 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T00:08:09.307 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T00:08:09.307 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T00:08:09.307 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T00:08:09.307 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T00:08:09.307 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T00:08:09.307 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T00:08:09.319 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T00:08:09.319 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T00:08:09.319 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T00:08:09.319 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T00:08:09.319 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T00:08:09.319 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T00:08:09.320 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T00:08:09.320 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T00:08:09.320 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T00:08:09.320 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T00:08:09.320 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T00:08:09.320 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T00:08:09.320 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T00:08:09.321 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T00:08:09.321 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T00:08:09.321 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T00:08:09.321 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T00:08:09.321 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T00:08:09.321 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T00:08:09.321 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T00:08:09.321 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T00:08:09.322 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T00:08:09.322 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T00:08:09.322 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T00:08:09.322 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T00:08:09.322 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T00:08:09.322 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T00:08:09.322 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T00:08:09.322 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T00:08:09.323 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T00:08:09.323 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T00:08:09.323 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T00:08:09.323 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T00:08:09.323 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T00:08:09.323 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T00:08:09.323 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T00:08:09.323 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T00:08:09.323 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T00:08:09.324 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T00:08:09.324 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T00:08:09.324 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T00:08:09.324 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T00:08:09.324 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T00:08:09.324 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T00:08:09.324 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T00:08:09.324 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T00:08:09.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T00:08:09.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T00:08:09.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T00:08:09.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T00:08:09.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T00:08:09.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T00:08:09.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T00:08:09.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T00:08:09.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T00:08:09.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T00:08:09.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T00:08:09.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T00:08:09.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T00:08:09.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T00:08:09.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T00:08:09.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T00:08:09.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T00:08:09.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T00:08:09.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T00:08:09.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T00:08:09.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T00:08:09.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T00:08:09.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T00:08:09.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T00:08:09.328 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T00:08:09.328 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T00:08:09.328 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T00:08:09.328 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T00:08:09.328 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T00:08:09.328 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T00:08:09.328 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T00:08:09.328 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T00:08:09.329 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T00:08:09.329 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T00:08:09.329 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T00:08:09.329 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T00:08:09.329 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T00:08:09.329 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T00:08:09.329 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T00:08:09.329 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T00:08:09.329 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T00:08:09.330 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T00:08:09.330 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T00:08:09.330 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T00:08:09.330 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T00:08:09.330 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T00:08:09.330 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T00:08:09.330 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T00:08:09.330 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T00:08:09.334 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T00:08:09.334 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T00:08:09.334 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T00:08:09.334 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T00:08:09.334 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T00:08:09.334 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T00:08:09.334 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T00:08:09.334 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T00:08:09.334 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T00:08:09.335 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T00:08:09.335 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T00:08:09.335 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T00:08:09.335 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T00:08:09.335 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T00:08:09.335 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T00:08:09.335 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T00:08:09.335 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T00:08:09.335 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T00:08:09.335 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T00:08:09.335 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T00:08:09.335 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T00:08:09.335 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T00:08:09.335 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T00:08:09.336 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T00:08:09.336 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T00:08:09.336 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T00:08:09.336 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T00:08:09.336 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T00:08:09.336 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T00:08:09.336 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T00:08:09.336 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T00:08:09.336 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T00:08:09.336 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T00:08:09.336 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T00:08:09.336 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T00:08:09.336 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T00:08:09.336 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T00:08:09.336 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T00:08:09.337 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T00:08:12.061 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T00:08:12.062 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:08:12.062 INFO:teuthology.orchestra.run.smithi098.stdout:Upgraded: 2024-09-21T00:08:12.062 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T00:08:12.062 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T00:08:12.062 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T00:08:12.062 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T00:08:12.062 INFO:teuthology.orchestra.run.smithi098.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.062 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.062 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T00:08:12.062 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T00:08:12.063 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-21T00:08:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T00:08:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T00:08:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T00:08:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T00:08:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.064 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:12.064 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:12.064 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:12.064 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:12.064 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:12.064 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.064 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.064 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:12.064 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.064 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.064 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.064 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:12.064 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:12.065 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.065 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.065 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:12.065 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T00:08:12.065 INFO:teuthology.orchestra.run.smithi098.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T00:08:12.065 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T00:08:12.065 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T00:08:12.065 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T00:08:12.065 INFO:teuthology.orchestra.run.smithi098.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T00:08:12.065 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T00:08:12.065 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T00:08:12.065 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T00:08:12.066 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T00:08:12.066 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T00:08:12.066 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.066 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.066 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.066 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T00:08:12.066 INFO:teuthology.orchestra.run.smithi098.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T00:08:12.066 INFO:teuthology.orchestra.run.smithi098.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T00:08:12.066 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T00:08:12.066 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T00:08:12.066 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T00:08:12.067 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.067 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.067 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T00:08:12.067 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.067 INFO:teuthology.orchestra.run.smithi098.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T00:08:12.067 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T00:08:12.067 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T00:08:12.067 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T00:08:12.067 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T00:08:12.067 INFO:teuthology.orchestra.run.smithi098.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T00:08:12.067 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T00:08:12.067 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T00:08:12.067 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T00:08:12.068 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T00:08:12.068 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T00:08:12.068 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T00:08:12.068 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T00:08:12.068 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T00:08:12.068 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T00:08:12.068 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T00:08:12.068 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T00:08:12.068 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T00:08:12.068 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T00:08:12.068 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T00:08:12.068 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T00:08:12.068 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T00:08:12.068 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T00:08:12.069 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.069 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.069 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.069 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T00:08:12.069 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T00:08:12.069 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T00:08:12.069 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T00:08:12.069 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T00:08:12.069 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T00:08:12.069 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T00:08:12.069 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T00:08:12.069 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T00:08:12.069 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T00:08:12.069 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T00:08:12.070 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T00:08:12.070 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T00:08:12.070 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T00:08:12.070 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T00:08:12.070 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T00:08:12.070 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T00:08:12.070 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T00:08:12.070 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T00:08:12.070 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T00:08:12.070 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T00:08:12.070 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T00:08:12.070 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T00:08:12.070 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T00:08:12.071 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T00:08:12.071 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T00:08:12.071 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T00:08:12.071 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T00:08:12.071 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T00:08:12.071 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T00:08:12.071 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T00:08:12.071 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T00:08:12.071 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T00:08:12.071 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T00:08:12.071 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T00:08:12.071 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T00:08:12.071 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T00:08:12.071 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T00:08:12.072 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T00:08:12.072 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T00:08:12.072 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.072 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.072 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T00:08:12.072 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T00:08:12.072 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T00:08:12.072 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.072 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T00:08:12.072 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T00:08:12.072 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T00:08:12.072 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T00:08:12.072 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T00:08:12.072 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T00:08:12.072 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T00:08:12.073 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T00:08:12.073 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T00:08:12.073 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T00:08:12.073 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T00:08:12.073 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T00:08:12.073 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T00:08:12.073 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T00:08:12.073 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T00:08:12.073 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.073 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.073 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:12.073 INFO:teuthology.orchestra.run.smithi098.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T00:08:12.073 INFO:teuthology.orchestra.run.smithi098.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T00:08:12.073 INFO:teuthology.orchestra.run.smithi098.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T00:08:12.074 INFO:teuthology.orchestra.run.smithi098.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T00:08:12.074 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T00:08:12.074 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T00:08:12.074 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T00:08:12.074 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:08:12.074 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-21T00:08:12.491 DEBUG:teuthology.parallel:result is None 2024-09-21T00:08:15.319 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T00:08:15.319 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T00:08:15.319 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T00:08:15.319 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T00:08:15.319 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T00:08:15.320 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T00:08:15.320 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T00:08:15.320 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T00:08:15.320 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T00:08:15.320 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T00:08:15.320 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T00:08:15.320 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T00:08:15.320 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T00:08:15.320 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T00:08:15.320 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T00:08:15.320 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T00:08:15.320 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T00:08:15.320 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T00:08:15.320 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T00:08:15.321 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T00:08:15.321 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T00:08:15.321 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T00:08:15.321 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T00:08:15.321 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T00:08:15.321 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T00:08:15.321 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T00:08:15.321 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T00:08:15.321 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T00:08:15.321 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T00:08:15.321 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T00:08:15.321 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T00:08:15.321 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T00:08:15.322 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T00:08:15.322 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T00:08:15.322 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T00:08:15.322 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T00:08:15.322 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T00:08:15.322 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T00:08:15.322 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T00:08:15.322 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T00:08:15.322 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T00:08:15.322 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T00:08:15.322 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T00:08:15.322 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T00:08:15.322 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T00:08:15.322 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T00:08:15.323 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T00:08:15.323 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T00:08:15.323 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T00:08:15.323 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T00:08:15.323 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T00:08:15.323 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T00:08:15.323 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T00:08:15.323 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T00:08:15.323 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T00:08:15.323 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T00:08:15.323 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T00:08:15.323 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T00:08:15.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T00:08:15.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T00:08:15.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T00:08:15.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T00:08:15.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T00:08:15.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T00:08:15.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T00:08:15.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T00:08:15.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T00:08:15.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T00:08:15.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T00:08:15.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T00:08:15.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T00:08:15.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T00:08:15.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T00:08:15.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T00:08:15.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T00:08:15.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T00:08:15.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T00:08:15.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T00:08:15.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T00:08:15.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T00:08:15.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T00:08:15.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T00:08:15.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T00:08:15.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T00:08:15.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T00:08:15.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T00:08:15.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T00:08:15.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T00:08:15.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T00:08:15.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T00:08:15.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T00:08:15.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T00:08:15.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T00:08:15.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T00:08:15.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T00:08:15.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T00:08:15.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T00:08:15.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T00:08:15.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T00:08:15.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T00:08:15.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T00:08:15.327 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T00:08:15.327 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T00:08:15.327 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T00:08:15.327 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T00:08:15.327 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T00:08:15.327 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T00:08:15.327 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T00:08:15.327 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T00:08:15.327 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T00:08:15.327 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T00:08:15.327 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T00:08:15.327 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T00:08:15.327 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T00:08:15.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T00:08:15.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T00:08:15.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T00:08:15.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T00:08:15.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T00:08:15.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T00:08:15.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T00:08:15.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T00:08:15.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T00:08:15.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T00:08:15.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T00:08:15.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T00:08:15.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T00:08:15.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T00:08:15.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T00:08:15.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T00:08:15.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T00:08:15.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T00:08:15.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T00:08:15.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T00:08:15.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T00:08:15.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T00:08:15.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T00:08:15.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T00:08:15.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T00:08:15.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T00:08:15.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T00:08:15.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T00:08:15.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T00:08:15.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T00:08:15.330 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T00:08:15.330 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T00:08:15.330 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T00:08:15.330 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T00:08:15.330 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T00:08:15.330 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T00:08:15.330 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T00:08:15.330 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T00:08:15.330 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T00:08:15.330 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T00:08:15.330 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T00:08:15.330 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T00:08:15.330 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T00:08:15.331 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T00:08:15.331 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T00:08:15.331 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T00:08:15.331 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T00:08:15.331 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T00:08:15.331 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T00:08:15.331 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T00:08:17.721 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T00:08:17.721 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:08:17.721 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-09-21T00:08:17.721 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T00:08:17.721 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T00:08:17.721 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T00:08:17.721 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T00:08:17.721 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.722 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.722 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T00:08:17.722 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T00:08:17.722 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-21T00:08:17.722 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T00:08:17.722 INFO:teuthology.orchestra.run.smithi027.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T00:08:17.722 INFO:teuthology.orchestra.run.smithi027.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T00:08:17.722 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T00:08:17.722 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.722 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.722 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.722 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.722 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:17.723 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.723 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.723 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.723 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:17.723 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:17.723 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:17.723 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:17.723 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:17.723 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.723 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.723 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:17.723 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.723 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.723 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.724 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:17.724 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:17.724 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.724 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.724 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:08:17.724 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T00:08:17.724 INFO:teuthology.orchestra.run.smithi027.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T00:08:17.724 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T00:08:17.724 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T00:08:17.724 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T00:08:17.724 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T00:08:17.724 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T00:08:17.724 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T00:08:17.725 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T00:08:17.725 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T00:08:17.725 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T00:08:17.725 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.725 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.725 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.725 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T00:08:17.725 INFO:teuthology.orchestra.run.smithi027.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T00:08:17.725 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T00:08:17.725 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T00:08:17.725 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T00:08:17.725 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T00:08:17.725 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.725 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.726 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T00:08:17.726 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.726 INFO:teuthology.orchestra.run.smithi027.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T00:08:17.726 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T00:08:17.726 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T00:08:17.726 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T00:08:17.726 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T00:08:17.726 INFO:teuthology.orchestra.run.smithi027.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T00:08:17.726 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T00:08:17.726 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T00:08:17.726 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T00:08:17.726 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T00:08:17.726 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T00:08:17.726 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T00:08:17.727 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T00:08:17.727 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T00:08:17.727 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T00:08:17.727 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T00:08:17.727 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T00:08:17.727 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T00:08:17.727 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T00:08:17.727 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T00:08:17.727 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T00:08:17.727 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T00:08:17.727 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T00:08:17.727 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.727 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.727 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.728 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T00:08:17.728 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T00:08:17.728 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T00:08:17.728 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T00:08:17.728 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T00:08:17.728 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T00:08:17.728 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T00:08:17.728 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T00:08:17.728 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T00:08:17.728 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T00:08:17.728 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T00:08:17.728 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T00:08:17.728 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T00:08:17.728 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T00:08:17.728 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T00:08:17.729 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T00:08:17.729 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T00:08:17.729 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T00:08:17.729 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T00:08:17.729 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T00:08:17.729 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T00:08:17.729 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T00:08:17.729 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T00:08:17.729 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T00:08:17.729 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T00:08:17.729 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T00:08:17.729 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T00:08:17.729 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T00:08:17.729 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T00:08:17.730 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T00:08:17.730 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T00:08:17.730 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T00:08:17.730 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T00:08:17.730 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T00:08:17.730 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T00:08:17.730 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T00:08:17.730 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T00:08:17.730 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T00:08:17.730 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T00:08:17.730 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T00:08:17.730 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.730 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.730 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T00:08:17.731 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T00:08:17.731 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T00:08:17.731 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.731 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T00:08:17.731 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T00:08:17.731 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T00:08:17.731 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T00:08:17.731 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T00:08:17.731 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T00:08:17.731 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T00:08:17.731 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T00:08:17.731 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T00:08:17.731 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T00:08:17.731 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T00:08:17.731 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T00:08:17.732 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T00:08:17.732 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T00:08:17.732 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T00:08:17.732 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.732 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.732 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:08:17.732 INFO:teuthology.orchestra.run.smithi027.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T00:08:17.732 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T00:08:17.732 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T00:08:17.732 INFO:teuthology.orchestra.run.smithi027.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T00:08:17.732 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T00:08:17.732 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T00:08:17.733 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T00:08:17.733 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:08:17.733 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-21T00:08:18.566 DEBUG:teuthology.parallel:result is None 2024-09-21T00:08:18.567 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:08:18.720 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T00:08:18.766 INFO:teuthology.orchestra.run.smithi027.stdout:19.3.0-5120.g18432218.el9 2024-09-21T00:08:18.766 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T00:08:18.766 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T00:08:18.768 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:08:18.916 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T00:08:18.943 INFO:teuthology.orchestra.run.smithi089.stdout:19.3.0-5120.g18432218.el9 2024-09-21T00:08:18.944 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T00:08:18.944 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T00:08:18.946 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:08:19.097 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T00:08:19.127 INFO:teuthology.orchestra.run.smithi098.stdout:19.3.0-5120.g18432218.el9 2024-09-21T00:08:19.127 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T00:08:19.127 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T00:08:19.129 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T00:08:19.130 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:08:19.130 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T00:08:19.192 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T00:08:19.193 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T00:08:19.225 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T00:08:19.225 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T00:08:19.256 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T00:08:19.257 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:08:19.257 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T00:08:19.316 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T00:08:19.418 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T00:08:19.419 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T00:08:19.457 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T00:08:19.527 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T00:08:19.527 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T00:08:19.557 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T00:08:19.626 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T00:08:19.626 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:08:19.626 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T00:08:19.686 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T00:08:19.786 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T00:08:19.786 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T00:08:19.811 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T00:08:19.877 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T00:08:19.877 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T00:08:19.901 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T00:08:19.966 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T00:08:19.966 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:08:19.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T00:08:20.024 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T00:08:20.122 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T00:08:20.123 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T00:08:20.148 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T00:08:20.212 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T00:08:20.212 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T00:08:20.238 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T00:08:20.303 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T00:08:20.428 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug', 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'mgr-modules': ['snap_schedule'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-21T00:08:20.428 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:08:20.428 INFO:tasks.cephadm:Cluster fsid is 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:08:20.428 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T00:08:20.429 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.27', 'mon.b': '172.21.15.89', 'mon.c': '172.21.15.98'} 2024-09-21T00:08:20.429 INFO:tasks.cephadm:First mon is mon.a on smithi027 2024-09-21T00:08:20.429 INFO:tasks.cephadm:First mgr is x 2024-09-21T00:08:20.429 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T00:08:20.429 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname $(hostname -s) 2024-09-21T00:08:20.491 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname $(hostname -s) 2024-09-21T00:08:20.516 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname $(hostname -s) 2024-09-21T00:08:20.540 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T00:08:20.541 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:08:20.685 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T00:08:20.823 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:08:20.824 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T00:08:20.825 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T00:08:20.825 DEBUG:teuthology.orchestra.run.smithi027:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T00:08:21.419 INFO:teuthology.orchestra.run.smithi027.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 00:08 /home/ubuntu/cephtest/cephadm 2024-09-21T00:08:21.419 DEBUG:teuthology.orchestra.run.smithi089:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T00:08:21.944 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 00:08 /home/ubuntu/cephtest/cephadm 2024-09-21T00:08:21.945 DEBUG:teuthology.orchestra.run.smithi098:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T00:08:22.473 INFO:teuthology.orchestra.run.smithi098.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 00:08 /home/ubuntu/cephtest/cephadm 2024-09-21T00:08:22.473 DEBUG:teuthology.orchestra.run.smithi027:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T00:08:22.516 DEBUG:teuthology.orchestra.run.smithi089:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T00:08:22.536 DEBUG:teuthology.orchestra.run.smithi098:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T00:08:22.581 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T00:08:22.581 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T00:08:22.585 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T00:08:22.587 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T00:08:24.202 INFO:teuthology.orchestra.run.smithi089.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T00:08:24.242 INFO:teuthology.orchestra.run.smithi098.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T00:08:24.408 INFO:teuthology.orchestra.run.smithi027.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T00:08:56.755 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-21T00:08:56.755 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T00:08:56.755 INFO:teuthology.orchestra.run.smithi098.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T00:08:56.755 INFO:teuthology.orchestra.run.smithi098.stdout: "repo_digests": [ 2024-09-21T00:08:56.755 INFO:teuthology.orchestra.run.smithi098.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T00:08:56.756 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-21T00:08:56.756 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-21T00:08:57.008 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T00:08:57.008 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T00:08:57.008 INFO:teuthology.orchestra.run.smithi089.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T00:08:57.008 INFO:teuthology.orchestra.run.smithi089.stdout: "repo_digests": [ 2024-09-21T00:08:57.009 INFO:teuthology.orchestra.run.smithi089.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T00:08:57.009 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-21T00:08:57.009 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T00:08:58.244 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:08:58.244 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T00:08:58.244 INFO:teuthology.orchestra.run.smithi027.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T00:08:58.244 INFO:teuthology.orchestra.run.smithi027.stdout: "repo_digests": [ 2024-09-21T00:08:58.245 INFO:teuthology.orchestra.run.smithi027.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T00:08:58.245 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T00:08:58.245 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:08:58.276 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph 2024-09-21T00:08:58.335 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph 2024-09-21T00:08:58.368 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph 2024-09-21T00:08:58.401 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /etc/ceph 2024-09-21T00:08:58.458 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /etc/ceph 2024-09-21T00:08:58.487 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 777 /etc/ceph 2024-09-21T00:08:58.517 INFO:tasks.cephadm:Writing seed config... 2024-09-21T00:08:58.518 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T00:08:58.518 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-21T00:08:58.518 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T00:08:58.518 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T00:08:58.518 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T00:08:58.518 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T00:08:58.518 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T00:08:58.518 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T00:08:58.518 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T00:08:58.518 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T00:08:58.518 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T00:08:58.518 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-21T00:08:58.518 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-21T00:08:58.519 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T00:08:58.519 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T00:08:58.519 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-21T00:08:58.519 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T00:08:58.519 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T00:08:58.519 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T00:08:58.519 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T00:08:58.519 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T00:08:58.519 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T00:08:58.519 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-21T00:08:58.520 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:08:58.520 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T00:08:58.555 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9c21de84-77ad-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2024-09-21T00:08:58.555 DEBUG:teuthology.orchestra.run.smithi027:mon.a> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T00:08:58.600 DEBUG:teuthology.orchestra.run.smithi027:mgr.x> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T00:08:58.645 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T00:08:58.645 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.27 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T00:08:58.914 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-21T00:08:58.914 INFO:teuthology.orchestra.run.smithi027.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.27', '--skip-admin-label'] 2024-09-21T00:08:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T00:08:58.915 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying podman|docker is present... 2024-09-21T00:08:58.962 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 5.2.2 2024-09-21T00:08:58.962 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying lvm2 is present... 2024-09-21T00:08:58.962 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying time synchronization is in place... 2024-09-21T00:08:58.976 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T00:08:58.976 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T00:08:58.989 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T00:08:58.989 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T00:08:59.003 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2024-09-21T00:08:59.016 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2024-09-21T00:08:59.016 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2024-09-21T00:08:59.017 INFO:teuthology.orchestra.run.smithi027.stdout:Repeating the final host check... 2024-09-21T00:08:59.061 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 5.2.2 2024-09-21T00:08:59.062 INFO:teuthology.orchestra.run.smithi027.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T00:08:59.062 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl is present 2024-09-21T00:08:59.062 INFO:teuthology.orchestra.run.smithi027.stdout:lvcreate is present 2024-09-21T00:08:59.074 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T00:08:59.075 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T00:08:59.088 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T00:08:59.088 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T00:08:59.101 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2024-09-21T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2024-09-21T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2024-09-21T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout:Host looks OK 2024-09-21T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout:Cluster fsid: 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout:Acquiring lock 140238303562096 on /run/cephadm/9c21de84-77ad-11ef-baf5-efdc52797490.lock 2024-09-21T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout:Lock 140238303562096 acquired on /run/cephadm/9c21de84-77ad-11ef-baf5-efdc52797490.lock 2024-09-21T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 3300 ... 2024-09-21T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 6789 ... 2024-09-21T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout:Base mon IP(s) is [172.21.15.27:3300, 172.21.15.27:6789], mon addrv is [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-09-21T00:08:59.122 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.27 metric 100 2024-09-21T00:08:59.122 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.27 metric 100 2024-09-21T00:08:59.128 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T00:08:59.129 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-21T00:08:59.129 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1762sec hoplimit 64 pref medium 2024-09-21T00:08:59.134 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T00:08:59.134 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-21T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-21T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2024-09-21T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2024-09-21T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T00:08:59.136 INFO:teuthology.orchestra.run.smithi027.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T00:08:59.137 INFO:teuthology.orchestra.run.smithi027.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T00:09:06.534 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T00:09:06.534 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T00:09:06.534 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T00:09:06.534 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T00:09:06.534 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T00:09:06.534 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T00:09:06.534 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T00:09:11.511 INFO:teuthology.orchestra.run.smithi027.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T00:09:11.512 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T00:09:11.512 INFO:teuthology.orchestra.run.smithi027.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T00:09:12.560 INFO:teuthology.orchestra.run.smithi027.stdout:stat: stdout 167 167 2024-09-21T00:09:12.561 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial keys... 2024-09-21T00:09:13.655 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQCpDu5mEjP3AhAACI+9t5akR/JBmQbKysULAw== 2024-09-21T00:09:14.907 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQCqDu5mtWzdERAAYcX15d38ilP25mlQZrmyWQ== 2024-09-21T00:09:15.922 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQCrDu5mrZtFFRAA9alPvpBM/JNXlbkT22DcOA== 2024-09-21T00:09:15.923 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial monmap... 2024-09-21T00:09:16.961 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T00:09:16.962 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T00:09:16.962 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:09:16.962 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T00:09:16.962 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool for a [v2:172.21.15.27:3300,v1:172.21.15.27:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T00:09:16.962 INFO:teuthology.orchestra.run.smithi027.stdout:setting min_mon_release = quincy 2024-09-21T00:09:16.962 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: set fsid to 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:09:16.962 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T00:09:16.962 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:09:16.962 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mon... 2024-09-21T00:09:18.000 INFO:teuthology.orchestra.run.smithi027.stdout:create mon.a on 2024-09-21T00:09:18.295 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T00:09:18.517 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T00:09:18.781 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9c21de84-77ad-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-9c21de84-77ad-11ef-baf5-efdc52797490.target. 2024-09-21T00:09:18.781 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9c21de84-77ad-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-9c21de84-77ad-11ef-baf5-efdc52797490.target. 2024-09-21T00:09:19.120 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mon.a 2024-09-21T00:09:19.120 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T00:09:19.403 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9c21de84-77ad-11ef-baf5-efdc52797490.target.wants/ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-9c21de84-77ad-11ef-baf5-efdc52797490@.service. 2024-09-21T00:09:19.997 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:19 smithi027 podman[30370]: 2024-09-21 00:09:19.652770361 +0000 UTC m=+0.027818499 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:09:19.997 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:19 smithi027 podman[30370]: 2024-09-21 00:09:19.995591577 +0000 UTC m=+0.370639715 container init dc82976f881bcc62658530349d42c33ed56a52ffb3857505a56bff33f1749c48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T00:09:20.101 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T00:09:20.101 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-09-21T00:09:20.117 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T00:09:20.117 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T00:09:20.117 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-09-21T00:09:20.117 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T00:09:20.119 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon to start... 2024-09-21T00:09:20.119 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon... 2024-09-21T00:09:20.268 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:20 smithi027 podman[30370]: 2024-09-21 00:09:20.002291057 +0000 UTC m=+0.377339195 container start dc82976f881bcc62658530349d42c33ed56a52ffb3857505a56bff33f1749c48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T00:09:20.268 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:20 smithi027 bash[30370]: dc82976f881bcc62658530349d42c33ed56a52ffb3857505a56bff33f1749c48 2024-09-21T00:09:20.268 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:20 smithi027 systemd[1]: Started Ceph mon.a for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:09:20.268 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:20 smithi027 ceph-mon[30389]: mkfs 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:09:20.268 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:20 smithi027 ceph-mon[30389]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T00:09:21.394 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:21 smithi027 ceph-mon[30389]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T00:09:21.395 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:21 smithi027 ceph-mon[30389]: monmap epoch 1 2024-09-21T00:09:21.395 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:21 smithi027 ceph-mon[30389]: fsid 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:09:21.395 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:21 smithi027 ceph-mon[30389]: last_changed 2024-09-21T00:09:16.349878+0000 2024-09-21T00:09:21.395 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:21 smithi027 ceph-mon[30389]: created 2024-09-21T00:09:16.349878+0000 2024-09-21T00:09:21.395 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:21 smithi027 ceph-mon[30389]: min_mon_release 19 (squid) 2024-09-21T00:09:21.395 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:21 smithi027 ceph-mon[30389]: election_strategy: 1 2024-09-21T00:09:21.395 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:21 smithi027 ceph-mon[30389]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T00:09:21.395 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:21 smithi027 ceph-mon[30389]: fsmap 2024-09-21T00:09:21.395 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:21 smithi027 ceph-mon[30389]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T00:09:21.395 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:21 smithi027 ceph-mon[30389]: mgrmap e1: no daemons active 2024-09-21T00:09:21.395 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:21 smithi027 ceph-mon[30389]: from='client.? 172.21.15.27:0/1024087248' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T00:09:21.400 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T00:09:21.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout id: 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:09:21.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T00:09:21.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T00:09:21.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout services: 2024-09-21T00:09:21.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.625126s) 2024-09-21T00:09:21.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T00:09:21.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T00:09:21.403 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T00:09:21.403 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout data: 2024-09-21T00:09:21.403 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T00:09:21.403 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T00:09:21.403 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T00:09:21.403 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T00:09:21.403 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T00:09:21.403 INFO:teuthology.orchestra.run.smithi027.stdout:mon is available 2024-09-21T00:09:21.403 INFO:teuthology.orchestra.run.smithi027.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T00:09:22.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:22 smithi027 ceph-mon[30389]: from='client.? 172.21.15.27:0/475141304' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T00:09:22.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:22 smithi027 ceph-mon[30389]: from='client.? 172.21.15.27:0/475141304' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T00:09:22.666 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T00:09:22.666 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T00:09:22.666 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:09:22.666 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T00:09:22.666 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-09-21T00:09:22.666 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T00:09:22.666 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T00:09:22.666 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T00:09:22.666 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T00:09:22.666 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T00:09:22.666 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T00:09:22.666 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T00:09:22.667 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T00:09:22.667 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T00:09:22.667 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T00:09:22.667 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T00:09:22.667 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T00:09:22.667 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T00:09:22.667 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T00:09:22.667 INFO:teuthology.orchestra.run.smithi027.stdout:Generating new minimal ceph.conf... 2024-09-21T00:09:23.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:23 smithi027 ceph-mon[30389]: from='client.? 172.21.15.27:0/292077197' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:09:23.887 INFO:teuthology.orchestra.run.smithi027.stdout:Restarting the monitor... 2024-09-21T00:09:24.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:23 smithi027 systemd[1]: Stopping Ceph mon.a for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:09:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:24 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a[30385]: 2024-09-21T00:09:24.252+0000 7f8342000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:09:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:24 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a[30385]: 2024-09-21T00:09:24.252+0000 7f8342000640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T00:09:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:24 smithi027 podman[30624]: 2024-09-21 00:09:24.335455568 +0000 UTC m=+0.213955041 container died dc82976f881bcc62658530349d42c33ed56a52ffb3857505a56bff33f1749c48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T00:09:24.951 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:24 smithi027 podman[30624]: 2024-09-21 00:09:24.766155095 +0000 UTC m=+0.644654567 container cleanup dc82976f881bcc62658530349d42c33ed56a52ffb3857505a56bff33f1749c48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T00:09:24.952 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:24 smithi027 bash[30624]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a 2024-09-21T00:09:25.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:24 smithi027 podman[30636]: 2024-09-21 00:09:24.950340754 +0000 UTC m=+0.601068868 container remove dc82976f881bcc62658530349d42c33ed56a52ffb3857505a56bff33f1749c48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T00:09:25.528 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 systemd[1]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T00:09:25.529 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 systemd[1]: Stopped Ceph mon.a for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:09:25.529 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 systemd[1]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mon.a.service: Consumed 1.351s CPU time. 2024-09-21T00:09:25.529 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 systemd[1]: Starting Ceph mon.a for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:09:25.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 podman[30727]: 2024-09-21 00:09:25.432677556 +0000 UTC m=+0.028377900 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:09:25.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 podman[30727]: 2024-09-21 00:09:25.542608057 +0000 UTC m=+0.138308391 container create 3adf3fc84023f274831c402345f8068f8ecf738e6dc5c5cac51bf36d19bd0a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T00:09:25.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 podman[30727]: 2024-09-21 00:09:25.753408139 +0000 UTC m=+0.349108486 container init 3adf3fc84023f274831c402345f8068f8ecf738e6dc5c5cac51bf36d19bd0a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:09:25.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 podman[30727]: 2024-09-21 00:09:25.760098653 +0000 UTC m=+0.355798988 container start 3adf3fc84023f274831c402345f8068f8ecf738e6dc5c5cac51bf36d19bd0a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T00:09:25.846 INFO:teuthology.orchestra.run.smithi027.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T00:09:26.127 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 bash[30727]: 3adf3fc84023f274831c402345f8068f8ecf738e6dc5c5cac51bf36d19bd0a2f 2024-09-21T00:09:26.128 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 systemd[1]: Started Ceph mon.a for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:09:26.128 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T00:09:26.128 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T00:09:26.128 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: pidfile_write: ignore empty --pid-file 2024-09-21T00:09:26.128 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: load: jerasure load: lrc 2024-09-21T00:09:26.128 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: RocksDB version: 7.9.2 2024-09-21T00:09:26.129 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Git sha 0 2024-09-21T00:09:26.129 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T00:09:26.129 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: DB SUMMARY 2024-09-21T00:09:26.129 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: DB Session ID: RXADL94YODHPT8Y6TG71 2024-09-21T00:09:26.129 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: CURRENT file: CURRENT 2024-09-21T00:09:26.129 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T00:09:26.129 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T00:09:26.130 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T00:09:26.130 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94262 ; 2024-09-21T00:09:26.130 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.error_if_exists: 0 2024-09-21T00:09:26.130 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.create_if_missing: 0 2024-09-21T00:09:26.130 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.paranoid_checks: 1 2024-09-21T00:09:26.130 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T00:09:26.131 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T00:09:26.131 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T00:09:26.131 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.env: 0x5571619fed60 2024-09-21T00:09:26.131 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T00:09:26.131 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.info_log: 0x55716ce25320 2024-09-21T00:09:26.131 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T00:09:26.131 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.statistics: (nil) 2024-09-21T00:09:26.132 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.use_fsync: 0 2024-09-21T00:09:26.132 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_log_file_size: 0 2024-09-21T00:09:26.132 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T00:09:26.132 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T00:09:26.132 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T00:09:26.132 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T00:09:26.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.allow_fallocate: 1 2024-09-21T00:09:26.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T00:09:26.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T00:09:26.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.use_direct_reads: 0 2024-09-21T00:09:26.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T00:09:26.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T00:09:26.134 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.db_log_dir: 2024-09-21T00:09:26.136 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.wal_dir: 2024-09-21T00:09:26.136 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T00:09:26.136 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T00:09:26.136 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T00:09:26.136 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T00:09:26.136 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T00:09:26.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T00:09:26.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T00:09:26.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T00:09:26.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.write_buffer_manager: 0x55716ce29d60 2024-09-21T00:09:26.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T00:09:26.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T00:09:26.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T00:09:26.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T00:09:26.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T00:09:26.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T00:09:26.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T00:09:26.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T00:09:26.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.unordered_write: 0 2024-09-21T00:09:26.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T00:09:26.139 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T00:09:26.139 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T00:09:26.139 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T00:09:26.139 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.row_cache: None 2024-09-21T00:09:26.139 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.wal_filter: None 2024-09-21T00:09:26.139 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T00:09:26.139 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T00:09:26.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.two_write_queues: 0 2024-09-21T00:09:26.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T00:09:26.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.wal_compression: 0 2024-09-21T00:09:26.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.atomic_flush: 0 2024-09-21T00:09:26.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T00:09:26.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T00:09:26.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T00:09:26.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.log_readahead_size: 0 2024-09-21T00:09:26.141 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T00:09:26.141 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T00:09:26.141 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T00:09:26.141 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T00:09:26.141 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T00:09:26.141 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T00:09:26.141 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T00:09:26.142 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_background_jobs: 2 2024-09-21T00:09:26.142 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_background_compactions: -1 2024-09-21T00:09:26.142 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_subcompactions: 1 2024-09-21T00:09:26.142 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T00:09:26.142 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T00:09:26.142 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T00:09:26.142 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T00:09:26.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T00:09:26.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T00:09:26.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T00:09:26.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T00:09:26.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_open_files: -1 2024-09-21T00:09:26.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T00:09:26.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T00:09:26.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T00:09:26.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T00:09:26.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_background_flushes: -1 2024-09-21T00:09:26.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Compression algorithms supported: 2024-09-21T00:09:26.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: kZSTD supported: 0 2024-09-21T00:09:26.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: kXpressCompression supported: 0 2024-09-21T00:09:26.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: kBZip2Compression supported: 0 2024-09-21T00:09:26.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T00:09:26.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: kLZ4Compression supported: 1 2024-09-21T00:09:26.145 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: kZlibCompression supported: 1 2024-09-21T00:09:26.145 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T00:09:26.145 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: kSnappyCompression supported: 1 2024-09-21T00:09:26.145 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T00:09:26.145 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T00:09:26.145 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T00:09:26.145 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T00:09:26.145 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T00:09:26.145 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.merge_operator: 2024-09-21T00:09:26.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compaction_filter: None 2024-09-21T00:09:26.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T00:09:26.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T00:09:26.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T00:09:26.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T00:09:26.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55716ce24f60) 2024-09-21T00:09:26.146 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks: 1 2024-09-21T00:09:26.146 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T00:09:26.146 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T00:09:26.147 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_top_level_index_and_filter: 1 2024-09-21T00:09:26.147 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_type: 0 2024-09-21T00:09:26.147 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_index_type: 0 2024-09-21T00:09:26.147 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_shortening: 1 2024-09-21T00:09:26.147 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T00:09:26.147 INFO:journalctl@ceph.mon.a.smithi027.stdout: checksum: 4 2024-09-21T00:09:26.147 INFO:journalctl@ceph.mon.a.smithi027.stdout: no_block_cache: 0 2024-09-21T00:09:26.148 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache: 0x557168de9610 2024-09-21T00:09:26.148 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_name: BinnedLRUCache 2024-09-21T00:09:26.148 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_options: 2024-09-21T00:09:26.148 INFO:journalctl@ceph.mon.a.smithi027.stdout: capacity : 536870912 2024-09-21T00:09:26.148 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_shard_bits : 4 2024-09-21T00:09:26.148 INFO:journalctl@ceph.mon.a.smithi027.stdout: strict_capacity_limit : 0 2024-09-21T00:09:26.148 INFO:journalctl@ceph.mon.a.smithi027.stdout: high_pri_pool_ratio: 0.000 2024-09-21T00:09:26.148 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_compressed: (nil) 2024-09-21T00:09:26.149 INFO:journalctl@ceph.mon.a.smithi027.stdout: persistent_cache: (nil) 2024-09-21T00:09:26.149 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size: 4096 2024-09-21T00:09:26.149 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size_deviation: 10 2024-09-21T00:09:26.149 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_restart_interval: 16 2024-09-21T00:09:26.149 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_block_restart_interval: 1 2024-09-21T00:09:26.149 INFO:journalctl@ceph.mon.a.smithi027.stdout: metadata_block_size: 4096 2024-09-21T00:09:26.149 INFO:journalctl@ceph.mon.a.smithi027.stdout: partition_filters: 0 2024-09-21T00:09:26.149 INFO:journalctl@ceph.mon.a.smithi027.stdout: use_delta_encoding: 1 2024-09-21T00:09:26.149 INFO:journalctl@ceph.mon.a.smithi027.stdout: filter_policy: bloomfilter 2024-09-21T00:09:26.149 INFO:journalctl@ceph.mon.a.smithi027.stdout: whole_key_filtering: 1 2024-09-21T00:09:26.149 INFO:journalctl@ceph.mon.a.smithi027.stdout: verify_compression: 0 2024-09-21T00:09:26.150 INFO:journalctl@ceph.mon.a.smithi027.stdout: read_amp_bytes_per_bit: 0 2024-09-21T00:09:26.150 INFO:journalctl@ceph.mon.a.smithi027.stdout: format_version: 5 2024-09-21T00:09:26.150 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_index_compression: 1 2024-09-21T00:09:26.150 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_align: 0 2024-09-21T00:09:26.150 INFO:journalctl@ceph.mon.a.smithi027.stdout: max_auto_readahead_size: 262144 2024-09-21T00:09:26.150 INFO:journalctl@ceph.mon.a.smithi027.stdout: prepopulate_block_cache: 0 2024-09-21T00:09:26.151 INFO:journalctl@ceph.mon.a.smithi027.stdout: initial_auto_readahead_size: 8192 2024-09-21T00:09:26.151 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T00:09:26.151 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T00:09:26.151 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T00:09:26.151 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compression: NoCompression 2024-09-21T00:09:26.151 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T00:09:26.151 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T00:09:26.151 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T00:09:26.152 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.num_levels: 7 2024-09-21T00:09:26.152 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T00:09:26.152 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T00:09:26.152 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T00:09:26.152 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T00:09:26.152 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T00:09:26.152 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T00:09:26.153 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T00:09:26.153 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T00:09:26.153 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T00:09:26.153 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T00:09:26.153 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T00:09:26.153 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T00:09:26.153 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T00:09:26.153 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T00:09:26.154 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T00:09:26.154 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T00:09:26.154 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T00:09:26.154 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T00:09:26.154 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T00:09:26.154 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T00:09:26.154 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T00:09:26.154 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T00:09:26.154 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T00:09:26.154 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T00:09:26.155 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T00:09:26.155 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T00:09:26.155 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T00:09:26.155 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T00:09:26.155 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T00:09:26.155 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T00:09:26.155 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T00:09:26.156 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T00:09:26.156 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T00:09:26.156 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T00:09:26.156 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T00:09:26.156 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T00:09:26.156 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T00:09:26.156 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T00:09:26.156 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T00:09:26.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T00:09:26.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T00:09:26.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T00:09:26.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T00:09:26.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T00:09:26.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T00:09:26.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T00:09:26.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T00:09:26.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T00:09:26.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T00:09:26.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T00:09:26.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T00:09:26.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T00:09:26.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T00:09:26.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T00:09:26.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.inplace_update_support: 0 2024-09-21T00:09:26.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T00:09:26.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T00:09:26.159 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T00:09:26.159 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T00:09:26.159 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.bloom_locality: 0 2024-09-21T00:09:26.159 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.max_successive_merges: 0 2024-09-21T00:09:26.159 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T00:09:26.159 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T00:09:26.159 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T00:09:26.159 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T00:09:26.160 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.ttl: 2592000 2024-09-21T00:09:26.160 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T00:09:26.160 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T00:09:26.160 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T00:09:26.160 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.enable_blob_files: false 2024-09-21T00:09:26.160 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.min_blob_size: 0 2024-09-21T00:09:26.160 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T00:09:26.160 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T00:09:26.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T00:09:26.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T00:09:26.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T00:09:26.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T00:09:26.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T00:09:26.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T00:09:26.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T00:09:26.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T00:09:26.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: de6ffade-5fc0-4d75-a864-b506e5160945 2024-09-21T00:09:26.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726877365873274, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T00:09:26.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T00:09:26.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726877365875365, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90483, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 281, "table_properties": {"data_size": 88553, "index_size": 250, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11466, "raw_average_key_size": 47, "raw_value_size": 82083, "raw_average_value_size": 342, "num_data_blocks": 11, "num_entries": 240, "num_filter_entries": 240, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726877365, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "de6ffade-5fc0-4d75-a864-b506e5160945", "db_session_id": "RXADL94YODHPT8Y6TG71", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T00:09:26.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726877365875550, "job": 1, "event": "recovery_finished"} 2024-09-21T00:09:26.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T00:09:26.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T00:09:26.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55716ce4aa80 2024-09-21T00:09:26.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: DB pointer 0x55716dbec000 2024-09-21T00:09:26.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T00:09:26.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T00:09:26.163 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** DB Stats ** 2024-09-21T00:09:26.163 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T00:09:26.163 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T00:09:26.163 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T00:09:26.164 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T00:09:26.164 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T00:09:26.164 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T00:09:26.164 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T00:09:26.164 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T00:09:26.164 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2024-09-21T00:09:26.164 INFO:journalctl@ceph.mon.a.smithi027.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T00:09:26.165 INFO:journalctl@ceph.mon.a.smithi027.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T00:09:26.165 INFO:journalctl@ceph.mon.a.smithi027.stdout: L0 2/0 90.22 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T00:09:26.165 INFO:journalctl@ceph.mon.a.smithi027.stdout: Sum 2/0 90.22 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T00:09:26.165 INFO:journalctl@ceph.mon.a.smithi027.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T00:09:26.165 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T00:09:26.165 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2024-09-21T00:09:26.165 INFO:journalctl@ceph.mon.a.smithi027.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T00:09:26.165 INFO:journalctl@ceph.mon.a.smithi027.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T00:09:26.166 INFO:journalctl@ceph.mon.a.smithi027.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 66.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T00:09:26.166 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T00:09:26.166 INFO:journalctl@ceph.mon.a.smithi027.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T00:09:26.166 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T00:09:26.166 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T00:09:26.166 INFO:journalctl@ceph.mon.a.smithi027.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T00:09:26.166 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T00:09:26.166 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T00:09:26.166 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T00:09:26.167 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T00:09:26.167 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative compaction: 0.00 GB write, 13.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T00:09:26.167 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval compaction: 0.00 GB write, 13.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T00:09:26.167 INFO:journalctl@ceph.mon.a.smithi027.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T00:09:26.167 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache BinnedLRUCache@0x557168de9610#2 capacity: 512.00 MB usage: 1.28 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-09-21T00:09:26.167 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-09-21T00:09:26.167 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T00:09:26.167 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T00:09:26.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: starting mon.a rank 0 at public addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] at bind addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:09:26.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: mon.a@-1(???) e1 preinit fsid 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:09:26.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: mon.a@-1(???).mds e1 new map 2024-09-21T00:09:26.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: mon.a@-1(???).mds e1 print_map 2024-09-21T00:09:26.168 INFO:journalctl@ceph.mon.a.smithi027.stdout: e1 2024-09-21T00:09:26.168 INFO:journalctl@ceph.mon.a.smithi027.stdout: btime 2024-09-21T00:09:20:149616+0000 2024-09-21T00:09:26.168 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T00:09:26.169 INFO:journalctl@ceph.mon.a.smithi027.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T00:09:26.169 INFO:journalctl@ceph.mon.a.smithi027.stdout: legacy client fscid: -1 2024-09-21T00:09:26.169 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T00:09:26.169 INFO:journalctl@ceph.mon.a.smithi027.stdout: No filesystems configured 2024-09-21T00:09:26.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T00:09:26.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T00:09:26.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T00:09:26.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T00:09:26.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T00:09:26.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T00:09:26.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: monmap epoch 1 2024-09-21T00:09:26.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: fsid 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:09:26.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: last_changed 2024-09-21T00:09:16.349878+0000 2024-09-21T00:09:26.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: created 2024-09-21T00:09:16.349878+0000 2024-09-21T00:09:26.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: min_mon_release 19 (squid) 2024-09-21T00:09:26.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: election_strategy: 1 2024-09-21T00:09:26.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T00:09:26.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: fsmap 2024-09-21T00:09:26.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T00:09:26.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:25 smithi027 ceph-mon[30745]: mgrmap e1: no daemons active 2024-09-21T00:09:27.124 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T00:09:27.151 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T00:09:27.151 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mgr... 2024-09-21T00:09:27.152 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T00:09:27.152 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T00:09:27.502 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.x 2024-09-21T00:09:27.503 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T00:09:27.741 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9c21de84-77ad-11ef-baf5-efdc52797490.target.wants/ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-9c21de84-77ad-11ef-baf5-efdc52797490@.service. 2024-09-21T00:09:27.779 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:27 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/885654746' entity='client.admin' 2024-09-21T00:09:28.067 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:27 smithi027 systemd[1]: Starting Ceph mgr.x for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:09:28.380 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:28 smithi027 podman[30956]: 2024-09-21 00:09:28.00117325 +0000 UTC m=+0.028213077 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:09:28.380 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:28 smithi027 podman[30956]: 2024-09-21 00:09:28.102605491 +0000 UTC m=+0.129645312 container create c336f5fa731be341ff4792f6fae2a737fac1aeb18978046e6a322ab0786e9ab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T00:09:28.381 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:28 smithi027 podman[30956]: 2024-09-21 00:09:28.296629575 +0000 UTC m=+0.323669425 container init c336f5fa731be341ff4792f6fae2a737fac1aeb18978046e6a322ab0786e9ab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-21T00:09:28.381 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:28 smithi027 podman[30956]: 2024-09-21 00:09:28.305189064 +0000 UTC m=+0.332228896 container start c336f5fa731be341ff4792f6fae2a737fac1aeb18978046e6a322ab0786e9ab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T00:09:28.381 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:28 smithi027 bash[30956]: c336f5fa731be341ff4792f6fae2a737fac1aeb18978046e6a322ab0786e9ab4 2024-09-21T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-09-21T00:09:28.426 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-09-21T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T00:09:28.444 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T00:09:28.444 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-09-21T00:09:28.459 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T00:09:28.459 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T00:09:28.459 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-09-21T00:09:28.459 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T00:09:28.459 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr to start... 2024-09-21T00:09:28.459 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr... 2024-09-21T00:09:28.752 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:28 smithi027 systemd[1]: Started Ceph mgr.x for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:09:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:29 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/711006630' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T00:09:30.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T00:09:30.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T00:09:30.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "9c21de84-77ad-11ef-baf5-efdc52797490", 2024-09-21T00:09:30.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T00:09:30.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T00:09:30.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T00:09:30.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T00:09:30.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:30.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T00:09:30.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T00:09:30.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-09-21T00:09:30.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T00:09:30.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T00:09:30.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T00:09:30.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T00:09:30.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-21T00:09:30.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T00:09:30.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T00:09:30.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T00:09:30.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T00:09:30.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:30.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T00:09:30.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T00:09:30.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T00:09:30.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T00:09:30.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T00:09:30.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T00:09:30.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T00:09:30.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T00:09:30.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:30.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T00:09:30.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T00:09:30.100 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T00:09:30.100 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T00:09:30.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T00:09:30.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T00:09:30.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T00:09:30.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T00:09:30.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T00:09:30.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:30.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T00:09:30.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T00:09:30.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T00:09:20:149616+0000", 2024-09-21T00:09:30.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T00:09:30.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T00:09:30.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:30.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T00:09:30.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T00:09:30.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T00:09:30.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T00:09:30.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T00:09:30.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T00:09:30.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T00:09:30.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T00:09:30.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T00:09:30.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:30.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T00:09:30.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T00:09:30.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T00:09:20.150870+0000", 2024-09-21T00:09:30.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T00:09:30.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:30.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T00:09:30.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T00:09:30.102 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (1/15)... 2024-09-21T00:09:33.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:32 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3198810754' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T00:09:33.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T00:09:33.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T00:09:33.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "9c21de84-77ad-11ef-baf5-efdc52797490", 2024-09-21T00:09:33.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T00:09:33.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T00:09:33.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T00:09:33.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T00:09:33.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:33.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T00:09:33.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T00:09:33.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-09-21T00:09:33.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T00:09:33.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T00:09:33.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T00:09:33.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T00:09:33.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-21T00:09:33.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T00:09:33.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T00:09:33.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T00:09:33.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T00:09:33.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:33.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T00:09:33.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T00:09:33.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T00:09:33.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T00:09:33.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T00:09:33.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T00:09:33.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T00:09:33.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T00:09:33.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:33.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T00:09:33.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T00:09:33.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T00:09:33.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T00:09:33.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T00:09:33.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T00:09:33.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T00:09:33.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T00:09:33.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T00:09:33.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:33.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T00:09:33.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T00:09:33.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T00:09:20:149616+0000", 2024-09-21T00:09:33.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T00:09:33.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T00:09:33.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:33.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T00:09:33.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T00:09:33.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T00:09:33.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T00:09:33.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T00:09:33.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T00:09:33.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T00:09:33.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T00:09:33.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T00:09:33.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:33.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T00:09:33.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T00:09:33.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T00:09:20.150870+0000", 2024-09-21T00:09:33.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T00:09:33.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:33.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T00:09:33.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T00:09:33.471 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (2/15)... 2024-09-21T00:09:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:34 smithi027 ceph-mon[30745]: Activating manager daemon x 2024-09-21T00:09:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:34 smithi027 ceph-mon[30745]: mgrmap e2: x(active, starting, since 0.00581471s) 2024-09-21T00:09:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:34 smithi027 ceph-mon[30745]: from='mgr.14100 172.21.15.27:0/3354389194' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T00:09:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:34 smithi027 ceph-mon[30745]: from='mgr.14100 172.21.15.27:0/3354389194' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T00:09:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:34 smithi027 ceph-mon[30745]: from='mgr.14100 172.21.15.27:0/3354389194' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T00:09:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:34 smithi027 ceph-mon[30745]: from='mgr.14100 172.21.15.27:0/3354389194' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:09:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:34 smithi027 ceph-mon[30745]: from='mgr.14100 172.21.15.27:0/3354389194' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T00:09:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:34 smithi027 ceph-mon[30745]: Manager daemon x is now available 2024-09-21T00:09:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:34 smithi027 ceph-mon[30745]: from='mgr.14100 172.21.15.27:0/3354389194' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T00:09:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:34 smithi027 ceph-mon[30745]: from='mgr.14100 172.21.15.27:0/3354389194' entity='mgr.x' 2024-09-21T00:09:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:34 smithi027 ceph-mon[30745]: from='mgr.14100 172.21.15.27:0/3354389194' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T00:09:35.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:34 smithi027 ceph-mon[30745]: from='mgr.14100 172.21.15.27:0/3354389194' entity='mgr.x' 2024-09-21T00:09:35.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:34 smithi027 ceph-mon[30745]: from='mgr.14100 172.21.15.27:0/3354389194' entity='mgr.x' 2024-09-21T00:09:37.042 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:36 smithi027 ceph-mon[30745]: mgrmap e3: x(active, since 1.00901s) 2024-09-21T00:09:37.042 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:36 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/4190857635' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T00:09:37.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T00:09:37.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T00:09:37.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "9c21de84-77ad-11ef-baf5-efdc52797490", 2024-09-21T00:09:37.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T00:09:37.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T00:09:37.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T00:09:37.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T00:09:37.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:37.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T00:09:37.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T00:09:37.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-09-21T00:09:37.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T00:09:37.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T00:09:37.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T00:09:37.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T00:09:37.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-21T00:09:37.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T00:09:37.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T00:09:37.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T00:09:37.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T00:09:37.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:37.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T00:09:37.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T00:09:37.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T00:09:37.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T00:09:37.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T00:09:37.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T00:09:37.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T00:09:37.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T00:09:37.389 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:37.389 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T00:09:37.389 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T00:09:37.389 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T00:09:37.389 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T00:09:37.389 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T00:09:37.389 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T00:09:37.389 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T00:09:37.389 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T00:09:37.389 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T00:09:37.389 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:37.389 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T00:09:37.389 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T00:09:37.390 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T00:09:20:149616+0000", 2024-09-21T00:09:37.390 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T00:09:37.390 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T00:09:37.390 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:37.390 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T00:09:37.390 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T00:09:37.390 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T00:09:37.390 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T00:09:37.390 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T00:09:37.390 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T00:09:37.390 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T00:09:37.390 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T00:09:37.390 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T00:09:37.391 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:37.391 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T00:09:37.391 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T00:09:37.391 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T00:09:20.150870+0000", 2024-09-21T00:09:37.391 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T00:09:37.391 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T00:09:37.391 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T00:09:37.391 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T00:09:37.391 INFO:teuthology.orchestra.run.smithi027.stdout:mgr is available 2024-09-21T00:09:38.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:37 smithi027 ceph-mon[30745]: mgrmap e4: x(active, since 2s) 2024-09-21T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-09-21T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling cephadm module... 2024-09-21T00:09:39.036 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:38 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3931634415' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T00:09:40.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:39 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/741127526' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T00:09:40.067 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:39 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: ignoring --setuser ceph since I am not root 2024-09-21T00:09:40.067 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:39 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: ignoring --setgroup ceph since I am not root 2024-09-21T00:09:40.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:40 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/741127526' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T00:09:40.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:40 smithi027 ceph-mon[30745]: mgrmap e5: x(active, since 5s) 2024-09-21T00:09:41.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:41 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/147825301' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T00:09:42.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T00:09:42.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T00:09:42.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T00:09:42.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T00:09:42.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T00:09:42.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T00:09:42.186 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2024-09-21T00:09:42.186 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 5... 2024-09-21T00:09:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:46 smithi027 ceph-mon[30745]: Active manager daemon x restarted 2024-09-21T00:09:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:46 smithi027 ceph-mon[30745]: Activating manager daemon x 2024-09-21T00:09:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:46 smithi027 ceph-mon[30745]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T00:09:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:46 smithi027 ceph-mon[30745]: mgrmap e6: x(active, starting, since 0.0039853s) 2024-09-21T00:09:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:46 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:09:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:46 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T00:09:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:46 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T00:09:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:46 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T00:09:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:46 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T00:09:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:46 smithi027 ceph-mon[30745]: Manager daemon x is now available 2024-09-21T00:09:47.986 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T00:09:47.986 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T00:09:47.986 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T00:09:47.987 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T00:09:47.987 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 5 is available 2024-09-21T00:09:47.987 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying orchestrator module is enabled... 2024-09-21T00:09:48.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:48 smithi027 ceph-mon[30745]: mgrmap e7: x(active, since 1.00702s) 2024-09-21T00:09:48.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:48 smithi027 ceph-mon[30745]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T00:09:48.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:48 smithi027 ceph-mon[30745]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T00:09:48.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:48 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' 2024-09-21T00:09:48.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:48 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' 2024-09-21T00:09:48.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:48 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' 2024-09-21T00:09:48.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:48 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' 2024-09-21T00:09:48.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:48 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:09:48.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:48 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:09:48.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:48 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T00:09:48.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:48 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T00:09:49.479 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-mon[30745]: Found migration_current of "None". Setting to last migration. 2024-09-21T00:09:49.479 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1218651377' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T00:09:49.479 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1218651377' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T00:09:49.479 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-mon[30745]: mgrmap e8: x(active, since 2s) 2024-09-21T00:09:49.480 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:09:49.297+0000 7f5127600640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:00:09:49] ENGINE Error in HTTPServer.serve 2024-09-21T00:09:49.480 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: Traceback (most recent call last): 2024-09-21T00:09:49.480 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T00:09:49.480 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: self._connections.run(self.expiration_interval) 2024-09-21T00:09:49.480 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T00:09:49.480 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: self._run(expiration_interval) 2024-09-21T00:09:49.480 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T00:09:49.480 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T00:09:49.480 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T00:09:49.480 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T00:09:49.480 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T00:09:49.480 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: s = self.context.wrap_socket( 2024-09-21T00:09:49.480 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T00:09:49.480 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: return self.sslsocket_class._create( 2024-09-21T00:09:49.481 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T00:09:49.481 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: self.do_handshake() 2024-09-21T00:09:49.481 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T00:09:49.481 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: self._sslobj.do_handshake() 2024-09-21T00:09:49.481 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T00:09:49.481 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:09:49.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T00:09:49.783 INFO:teuthology.orchestra.run.smithi027.stdout:Setting orchestrator backend to cephadm... 2024-09-21T00:09:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:50 smithi027 ceph-mon[30745]: [21/Sep/2024:00:09:49] ENGINE Bus STARTING 2024-09-21T00:09:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:50 smithi027 ceph-mon[30745]: [21/Sep/2024:00:09:49] ENGINE Serving on https://172.21.15.27:7150 2024-09-21T00:09:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:50 smithi027 ceph-mon[30745]: [21/Sep/2024:00:09:49] ENGINE Error in HTTPServer.serve 2024-09-21T00:09:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout: Traceback (most recent call last): 2024-09-21T00:09:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T00:09:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._connections.run(self.expiration_interval) 2024-09-21T00:09:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T00:09:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._run(expiration_interval) 2024-09-21T00:09:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T00:09:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T00:09:50.318 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T00:09:50.318 INFO:journalctl@ceph.mon.a.smithi027.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T00:09:50.318 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T00:09:50.318 INFO:journalctl@ceph.mon.a.smithi027.stdout: s = self.context.wrap_socket( 2024-09-21T00:09:50.318 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T00:09:50.318 INFO:journalctl@ceph.mon.a.smithi027.stdout: return self.sslsocket_class._create( 2024-09-21T00:09:50.318 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T00:09:50.318 INFO:journalctl@ceph.mon.a.smithi027.stdout: self.do_handshake() 2024-09-21T00:09:50.318 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T00:09:50.318 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._sslobj.do_handshake() 2024-09-21T00:09:50.318 INFO:journalctl@ceph.mon.a.smithi027.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T00:09:50.319 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:50 smithi027 ceph-mon[30745]: [21/Sep/2024:00:09:49] ENGINE Serving on http://172.21.15.27:8765 2024-09-21T00:09:50.319 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:50 smithi027 ceph-mon[30745]: [21/Sep/2024:00:09:49] ENGINE Bus STARTED 2024-09-21T00:09:50.319 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:50 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:09:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:51 smithi027 ceph-mon[30745]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:09:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:51 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' 2024-09-21T00:09:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:51 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:09:52.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T00:09:52.673 INFO:teuthology.orchestra.run.smithi027.stdout:Generating ssh key... 2024-09-21T00:09:53.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-mon[30745]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:09:54.067 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: Generating public/private rsa key pair. 2024-09-21T00:09:54.067 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: Your identification has been saved in /tmp/tmp0bso7rjh/key 2024-09-21T00:09:54.067 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: Your public key has been saved in /tmp/tmp0bso7rjh/key.pub 2024-09-21T00:09:54.067 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: The key fingerprint is: 2024-09-21T00:09:54.067 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: SHA256:rEMHSOjoct7Ca6jGkfDq6+CI48GwOpscLMoYiY7suxE ceph-9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:09:54.067 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: The key's randomart image is: 2024-09-21T00:09:54.068 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: +---[RSA 3072]----+ 2024-09-21T00:09:54.068 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: | .. | 2024-09-21T00:09:54.068 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: | .. . | 2024-09-21T00:09:54.068 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: | o . . | 2024-09-21T00:09:54.068 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: |o . o | 2024-09-21T00:09:54.068 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: |+E. . S | 2024-09-21T00:09:54.068 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: |**+ . o | 2024-09-21T00:09:54.068 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: |X%.. o | 2024-09-21T00:09:54.068 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: |^=O . . | 2024-09-21T00:09:54.068 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: |^^=o | 2024-09-21T00:09:54.068 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:09:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: +----[SHA256]-----+ 2024-09-21T00:09:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:54 smithi027 ceph-mon[30745]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:09:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:54 smithi027 ceph-mon[30745]: Generating ssh key... 2024-09-21T00:09:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:54 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' 2024-09-21T00:09:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:54 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' 2024-09-21T00:09:55.917 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTLCmGieeSGTg4R5IzqKpxFHPeRjR7NLerftYRhbHqEFPSj0YMYWEWnwf+moaLcQnBhw/jMO1zoAQR+EuSxShxkPywrUxDoXh4kBRtQD2KJ+xcP197kXgI6QpDKeQ+Dwskn2iWTGEckD0ZZjWC5WbIgMgMFjziGAyV0cU40IwLC9ppYvMHfbELsfTzVaxxqVoVTVLyzQGTRtw/0jx0qhW0NMvyVzvi5LwCAaRkLdBdYICABYJv57eNeSdv7fTMq0GYba7d1cc03oqcJoaue5Fr6Uu9tDrpZwI+oxkTDjDvOiWcZtBxFSLMyNPA3rYvkyf6d1X2wqPUyNZolA8bdQm1Mz1KE6V3e6AFDa+K2i0Nr9AklYrRRQ6+UsD7dJzV9fmmzIR9TJQOZBqTwt+zQ1vJNQ6Qo0a/bALee+HhwltZbBayAourTtDcmmBF89NP9wZTiP7NTLrl4F9xQjIDzEIfR6o1yTpc2rlNu6AvKDmHvsvY78YWApONYiyG7wVEuKk= ceph-9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:09:55.918 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T00:09:55.918 INFO:teuthology.orchestra.run.smithi027.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T00:09:55.918 INFO:teuthology.orchestra.run.smithi027.stdout:Adding host smithi027... 2024-09-21T00:09:56.203 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:56 smithi027 ceph-mon[30745]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:09:57.175 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:57 smithi027 ceph-mon[30745]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi027", "addr": "172.21.15.27", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:09:58.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:09:58 smithi027 ceph-mon[30745]: Deploying cephadm binary to smithi027 2024-09-21T00:10:01.693 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Added host 'smithi027' with addr '172.21.15.27' 2024-09-21T00:10:01.693 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mon service... 2024-09-21T00:10:02.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:02 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' 2024-09-21T00:10:02.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:02 smithi027 ceph-mon[30745]: Added host smithi027 2024-09-21T00:10:02.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:02 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:10:03.123 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T00:10:03.123 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mgr service... 2024-09-21T00:10:03.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:03 smithi027 ceph-mon[30745]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:10:03.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:03 smithi027 ceph-mon[30745]: Saving service mon spec with placement count:5 2024-09-21T00:10:03.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:03 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' 2024-09-21T00:10:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T00:10:05.113 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:04 smithi027 ceph-mon[30745]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:10:05.113 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:04 smithi027 ceph-mon[30745]: Saving service mgr spec with placement count:2 2024-09-21T00:10:05.113 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:04 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' 2024-09-21T00:10:07.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:06 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2163802858' entity='client.admin' 2024-09-21T00:10:08.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:07 smithi027 ceph-mon[30745]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:08.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:07 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1601771408' entity='client.admin' 2024-09-21T00:10:08.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:07 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' 2024-09-21T00:10:08.118 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling the dashboard module... 2024-09-21T00:10:09.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:08 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' 2024-09-21T00:10:10.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:09 smithi027 ceph-mon[30745]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:10.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:09 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2498918245' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T00:10:10.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:09 smithi027 ceph-mon[30745]: from='mgr.14118 172.21.15.27:0/3845102063' entity='mgr.x' 2024-09-21T00:10:10.318 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:10 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: ignoring --setuser ceph since I am not root 2024-09-21T00:10:10.318 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:10 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: ignoring --setgroup ceph since I am not root 2024-09-21T00:10:11.174 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:10 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2498918245' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T00:10:11.175 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:10 smithi027 ceph-mon[30745]: mgrmap e9: x(active, since 23s) 2024-09-21T00:10:12.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:11 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3953748751' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T00:10:12.159 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T00:10:12.159 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-21T00:10:12.160 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T00:10:12.160 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T00:10:12.160 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T00:10:12.160 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T00:10:12.160 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2024-09-21T00:10:12.160 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 9... 2024-09-21T00:10:16.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:16 smithi027 ceph-mon[30745]: Active manager daemon x restarted 2024-09-21T00:10:16.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:16 smithi027 ceph-mon[30745]: Activating manager daemon x 2024-09-21T00:10:16.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:16 smithi027 ceph-mon[30745]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T00:10:16.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:16 smithi027 ceph-mon[30745]: mgrmap e10: x(active, starting, since 0.00553967s) 2024-09-21T00:10:16.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:16 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:10:16.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:16 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T00:10:16.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:16 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T00:10:16.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:16 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T00:10:16.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:16 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T00:10:16.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:16 smithi027 ceph-mon[30745]: Manager daemon x is now available 2024-09-21T00:10:17.794 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:17 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:10:17.794 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:17 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T00:10:17.794 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:17 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T00:10:17.794 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:17 smithi027 ceph-mon[30745]: mgrmap e11: x(active, since 1.01072s) 2024-09-21T00:10:18.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T00:10:18.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-21T00:10:18.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T00:10:18.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T00:10:18.098 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 9 is available 2024-09-21T00:10:18.098 INFO:teuthology.orchestra.run.smithi027.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T00:10:18.370 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:10:18.304+0000 7f1b70200640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:00:10:18] ENGINE Error in HTTPServer.serve 2024-09-21T00:10:18.370 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: Traceback (most recent call last): 2024-09-21T00:10:18.370 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T00:10:18.370 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: self._connections.run(self.expiration_interval) 2024-09-21T00:10:18.370 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T00:10:18.370 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: self._run(expiration_interval) 2024-09-21T00:10:18.370 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T00:10:18.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T00:10:18.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T00:10:18.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T00:10:18.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T00:10:18.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: s = self.context.wrap_socket( 2024-09-21T00:10:18.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T00:10:18.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: return self.sslsocket_class._create( 2024-09-21T00:10:18.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T00:10:18.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: self.do_handshake() 2024-09-21T00:10:18.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T00:10:18.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: self._sslobj.do_handshake() 2024-09-21T00:10:18.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T00:10:18.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:10:18.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:18 smithi027 ceph-mon[30745]: [21/Sep/2024:00:10:18] ENGINE Bus STARTING 2024-09-21T00:10:19.762 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:19 smithi027 ceph-mon[30745]: [21/Sep/2024:00:10:18] ENGINE Serving on http://172.21.15.27:8765 2024-09-21T00:10:19.762 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:19 smithi027 ceph-mon[30745]: [21/Sep/2024:00:10:18] ENGINE Serving on https://172.21.15.27:7150 2024-09-21T00:10:19.762 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:19 smithi027 ceph-mon[30745]: [21/Sep/2024:00:10:18] ENGINE Bus STARTED 2024-09-21T00:10:19.762 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:19 smithi027 ceph-mon[30745]: [21/Sep/2024:00:10:18] ENGINE Error in HTTPServer.serve 2024-09-21T00:10:19.762 INFO:journalctl@ceph.mon.a.smithi027.stdout: Traceback (most recent call last): 2024-09-21T00:10:19.762 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T00:10:19.762 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._connections.run(self.expiration_interval) 2024-09-21T00:10:19.762 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T00:10:19.763 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._run(expiration_interval) 2024-09-21T00:10:19.763 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T00:10:19.763 INFO:journalctl@ceph.mon.a.smithi027.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T00:10:19.763 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T00:10:19.763 INFO:journalctl@ceph.mon.a.smithi027.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T00:10:19.763 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T00:10:19.763 INFO:journalctl@ceph.mon.a.smithi027.stdout: s = self.context.wrap_socket( 2024-09-21T00:10:19.763 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T00:10:19.763 INFO:journalctl@ceph.mon.a.smithi027.stdout: return self.sslsocket_class._create( 2024-09-21T00:10:19.763 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T00:10:19.763 INFO:journalctl@ceph.mon.a.smithi027.stdout: self.do_handshake() 2024-09-21T00:10:19.763 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T00:10:19.763 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._sslobj.do_handshake() 2024-09-21T00:10:19.764 INFO:journalctl@ceph.mon.a.smithi027.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T00:10:19.764 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:19 smithi027 ceph-mon[30745]: mgrmap e12: x(active, since 2s) 2024-09-21T00:10:19.764 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:19 smithi027 ceph-mon[30745]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:10:19.764 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:19 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:20.432 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUdB6QNl8rpSdYz4fmrrIZMEBHIKswDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTAwMTAxOVoXDTM0\nMDkyMjAwMTAxOVowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjI3MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAssLp2VXHklNz9+26LCcA5fIyIRNCLHzrPC9J\nFbdPi6OxlkSb5RsIH+w8V/d/KcFuFGY/z6VL0FOCFCKAEt1Ne6OQ0KF9AUao/e0Q\nBVEcShjgnk5Q1v/gPMsKpLylAGakTIs+1lOu0FJWVzvYPHITDkpAkzoFLs4+VzMZ\nQ0wsBA/uiQfZ/SdkbxNxBGQAeq7te3SIblGOFmNwwYCDUvPl44lFUuEMrQsUSg5c\nBOzvlwbMlUzHb3fnAi6bGj7QJFByzKYqIOrRDLbMnBYSO2QrLzCe/LfhxC2w1dfo\nHbH5chI5tG1h518Jha5dvQCrLqJtYKnoh7wRuzd1SUnx9i9K8hP927eVcCKtZ060\nZ3TG5rEfgYOfeSfirgKv4yOQabIZvsbVtqV4vKGG8P0iApsjS3hsLeA/DwVsWqco\nOiiBBZIh47TAZT4ZzN24TaVlPriTTIciZAJ1QL6gg98/IkuHeQqakgWSs72rSULF\n7O9vndkqEKuTQaqsG4p4xsq3BnCk5cSV8N1u3zQRC56bk9x/terpdB/yVk0dfxPp\nKmXE5+CFoIX5y/EsR/IS8B1T+p4WkV5vHlJGA8nE1K8m350S65hdmlpO4uiblOJA\nBI4H4VQDBO6gon0rJLh+gMLnzqBqEdzoYL/FxNWgMhRCTfdqAhHnCHod5rDgRyZi\ncf+fTvMCAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDI3ghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPGzAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQB/\nIcNktpsqFRYb9XA11h9fC5tcjkeTk6IQ+a4xPJeJYe1idOCuMgX28PZN4CsI+frx\n5J6dKrQnBfM8TNmlRs/aLJOK0ZMtjPxXRUYKcTCzztM+RP0XnxBj0f8IbYTfTb/g\nHGpQKDIPyk+SmbsfXSYFUxLpriPwctWx3PkS37+34UhlO+XJr38uRqm8nlg9I8kk\nuWcZ+7ILVti+O8/j5Iws2JTpblFG/x0j1aiAEHgjG+muJ8B61bArUdPxSVjlVv1g\nUXF6A8+iITx2rkmBsNrOmKqGkVzHnI5V4mBT+sro/dfj6pKKKwZytdLev/IdXXYX\nJAu1FoAOsqiRbWv/rSQE8wQP5IX2MvbiZvDsHETjrrKDLe87hruC+Nd5FOTo0Nas\nC/1s7bzSw4eluCojze02UKCVgjdFDrIW6HK1NAKZ5VcDAMEHXls538nOta44WB4c\nDBhzW1YhiUnk459yisCqYlAbtaep+fLhYGW5KL/LA1eOoILMLSawEL3zCdJdp9SC\n1sXStnsSYnxfSY3fH72SDwnilZlj5LJQVlQR+AsyW4/laqHudiKCa9BrvtcNZzog\nbBRnEch2Iy0kZNJM5813ACXriZhvlF40GeGW27jUtsP5wUhEuK1JCBDqVMxDdEuN\ndWxWAFmLWsJvfPoQs6mKGBsPdmyd7JBXon6MLDmymw==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEAssLp2VXHklNz9+26LCcA5fIyIRNCLHzrPC9JFbdPi6OxlkSb\n5RsIH+w8V/d/KcFuFGY/z6VL0FOCFCKAEt1Ne6OQ0KF9AUao/e0QBVEcShjgnk5Q\n1v/gPMsKpLylAGakTIs+1lOu0FJWVzvYPHITDkpAkzoFLs4+VzMZQ0wsBA/uiQfZ\n/SdkbxNxBGQAeq7te3SIblGOFmNwwYCDUvPl44lFUuEMrQsUSg5cBOzvlwbMlUzH\nb3fnAi6bGj7QJFByzKYqIOrRDLbMnBYSO2QrLzCe/LfhxC2w1dfoHbH5chI5tG1h\n518Jha5dvQCrLqJtYKnoh7wRuzd1SUnx9i9K8hP927eVcCKtZ060Z3TG5rEfgYOf\neSfirgKv4yOQabIZvsbVtqV4vKGG8P0iApsjS3hsLeA/DwVsWqcoOiiBBZIh47TA\nZT4ZzN24TaVlPriTTIciZAJ1QL6gg98/IkuHeQqakgWSs72rSULF7O9vndkqEKuT\nQaqsG4p4xsq3BnCk5cSV8N1u3zQRC56bk9x/terpdB/yVk0dfxPpKmXE5+CFoIX5\ny/EsR/IS8B1T+p4WkV5vHlJGA8nE1K8m350S65hdmlpO4uiblOJABI4H4VQDBO6g\non0rJLh+gMLnzqBqEdzoYL/FxNWgMhRCTfdqAhHnCHod5rDgRyZicf+fTvMCAwEA\nAQKCAgA9ZuQKSgOeCFpKK/cJ2W/ejB2Z7NKLE3F9n8Rb2DZTgaurMO/XdzyIMo3b\nrWCZiB52lVSPz3hGG5kPr3YMhMVrlZhVgVK90HaVmgspuF2eOgNVfRMZdpSgbSG/\nyihnBdqjjCTOW7otRkrv9ddlk81l2C1sJL7JAAC71Qp1akGoN4y4tuCyqwg9exET\n8AGWImXiRPnj4tGKcSe235y8LpilWm5cdH71JVwXv+ztqDD30FcPZWsPp8DKqBWQ\no0i9YIC+OMk5u06JLnUDnDUw/0wyDmJd7g93YjZejaXmJU7IVuhJSFUbszJDH8yE\n8ijagOJPFK2RihERHJCF1wdsh3wXLLox3aFc6uCTHQKvA7atR5Hc6CHEH/wao2N1\nQmrAA4R9SRItT4LNTRESi0L6NqnShJFtLjoYa1yS0D1Kl2naIrlyZC0h5sobMEz9\nK7k45CUlfOVcelj41sFtmiOR0QOUev81j8Q/+tqp938ChK0H8oxia1rPEW7cVqa6\nlyy6YNlRigwMa64ssoAwXzuwI6HSYp8g1Oh3YpytORGlQplDtYiWS82FZ20stosI\nRe1K9rE4c0x9ePQwHzvUPrvAxxhPykPkVjcj5tNqM7Ht4HQVTV/5oZcPFdxNQb74\nE4KUCSDAAPEV+nMdvoKhjeLl3PwVKl/0vU/4Z0dvbxduBkOLsQKCAQEA8KBU4BZX\n4dUVhBjpkKz/iHEgdef1tKVHTiFIw2XeqWlJvm9JGYcqDqIxvefYpFYjLi7aofgj\n35Z2SBJ0PG8wYCT3DhWPAD10stYPu1yBblOYmdqATGezxFAL90M7g8J86QFz3C6M\nqr2HCtx8YCUs6dt9ZfXYqALMsavE89ET/B9/VkOJ9UrgDM5z2YHwa+KG6lG8bQEP\nJZtvsHY9DX4i8gip4V9PZZPgSH8ChEBODflt2faVEokA2f69FA9frykDjd631qJR\n6hCgA6sPKB+ou1/G+v7Ov6RTAXeeQwc3pU1R4et/K6SBb2F60TGTyd0DbeMJSoX9\nSbikT8k7hu7wWwKCAQEAvi65KnRGBhRDN4tPDkLyBD4yuSU35V5rRXXzuoSyeEGj\nG06NpIGgZ2blKHz7TtVU+nbNn6d5AtIOwf/Zc5wuUvvJL3UxeHBdjaZCaNR0T3IN\nRDwJD5HmAKsZ1CkG5nVlf+lUwo3s+Ns2beVkGW8qlGTGMx7953LNtaMb837ok9f4\nKWkyrWKNDFfn5M4iUOuJuY5Rp9nnmqPCA8mOk83VZp7yESFwMhbimWu6uwFPJtbP\nssSP37n/OXdhMGXNkbfV+bDtVuhZSydhjXCJyjAS0urI2HTAVxqiStFS2gGNNyvi\nhXdfStuPjVDO7tw8ZwyfliPfMVwBkOpavmlM4NFfSQKCAQAGJmLk8009tV9+Z9CY\nZQOG7jVBLjOGcW74HTup2tGtDMbYuilpIJ4/j3+r9eAAmz788C+gUiJAlaziPK6B\n2Omom2lBD+X+wJNZ+sSeKkgYlMRiskXzLfdXPiCAKlS8El106xeV+g6YUo3ABlCb\neHfEZlq3H2hxKtkmdVhsTtyu5fMwNjuxbD97tE8bsQl/CGQgWwRw2MUVVfqtzOfU\nj3O/Md99phz2YKgr0iszTB1yl/xpxLacqam+6JbWi2oLpQW1O4sBTla/n7snPNj/\nUofSeikwwxBvr3h7oh4GYonWjLpaBtXSPLseKKSl8hlv4Fk1FsdnAzOtDa4NZ/xA\nf5zfAoIBAHpb60wyNN8yaDCzU3C0h6tNLStLk9Ph1JiDFAW73Gtcp4kpGB5wO/hr\nDiLnjnSg8qSWjSMcB9JVvaL5bjlgkblYP49mEicP/FDnWDDEB/UZRwnQqm+3CckC\nTILgxoy9mUkboSnvUDioKTO5c5adnEKPN6mqVed0SH+FarVVwGNmf0Z9ZVqCXodD\ng3IFYbS4ZgrZUxXwL3+hvnDbW/0iYBs/RjTZU6sWaEIXOrOYWwl9CXJV3s8thtS9\nRu1xgjovkd6tA3Z31QNyWeQ7NJu+JE5JOmFxdFJWU3ZZiLm0Z9vIQ1xlegzaiFh+\nVjT9w2r1hzN87tt9+4/2B94cN5cpfekCggEBAOaWbKza+2lRvYJNryZprmRj6z4k\nZfOJhDXGk0yTyLfYkWYPB3LLQnALiEF/HHKJHyK9UZ1HUA7zcQH1+mZtxvy+L7G6\njwqRPIAVCP+cvmL89svazlOTbka8qZAQiCX26kKavnnG71+llxwmbxFLGcM4hRMV\n2a3u1rz1EF8gKhSU2i8Vf9DoJ0PSH89+C5xztMkRPAK8+Iwa4YVTXLV8X0e/v3tG\nmVDrdhEHJ2ar/XYcnCkTrkL2yV5G3L/TmDohL4COBYzINUSLtMOGqMc1lcvMlTuT\nM5hhO7nwRMLZfnv9xbnOqwBg5/kd5A3FCD2shsn/CNqrLIwWafrDeyAr21A=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T00:10:21.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:21 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:21.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:21 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:21.956 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T00:10:22.773 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:22 smithi027 ceph-mon[30745]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:10:23.935 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T00:10:23.936 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial admin user... 2024-09-21T00:10:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:24 smithi027 ceph-mon[30745]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:10:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:24 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$h.tadc9sF5/nEzuhT9GU6uKUVWAoWNpN/szA5tmqtJvAU9yZoOt7K", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726877424, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T00:10:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:Fetching dashboard port number... 2024-09-21T00:10:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:25 smithi027 ceph-mon[30745]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:10:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:25 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:27.058 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:26 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3317411148' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T00:10:27.063 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T00:10:27.082 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T00:10:27.082 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-09-21T00:10:27.095 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T00:10:27.095 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T00:10:27.095 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-09-21T00:10:27.095 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T00:10:27.098 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph Dashboard is now available at: 2024-09-21T00:10:27.098 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:10:27.098 INFO:teuthology.orchestra.run.smithi027.stdout: URL: https://smithi027.front.sepia.ceph.com:8443/ 2024-09-21T00:10:27.098 INFO:teuthology.orchestra.run.smithi027.stdout: User: admin 2024-09-21T00:10:27.098 INFO:teuthology.orchestra.run.smithi027.stdout: Password: j3jk389sw6 2024-09-21T00:10:27.098 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:10:27.099 INFO:teuthology.orchestra.run.smithi027.stdout:Saving cluster configuration to /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config directory 2024-09-21T00:10:28.798 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: ceph telemetry on 2024-09-21T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout:For more information see: 2024-09-21T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T00:10:28.800 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:10:28.800 INFO:teuthology.orchestra.run.smithi027.stdout:Bootstrap complete. 2024-09-21T00:10:28.867 INFO:tasks.cephadm:Fetching config... 2024-09-21T00:10:28.867 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:10:28.867 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T00:10:28.908 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T00:10:28.908 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:10:28.908 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T00:10:28.987 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T00:10:28.988 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:10:28.988 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T00:10:29.086 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T00:10:29.086 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:10:29.086 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T00:10:29.167 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T00:10:29.168 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTLCmGieeSGTg4R5IzqKpxFHPeRjR7NLerftYRhbHqEFPSj0YMYWEWnwf+moaLcQnBhw/jMO1zoAQR+EuSxShxkPywrUxDoXh4kBRtQD2KJ+xcP197kXgI6QpDKeQ+Dwskn2iWTGEckD0ZZjWC5WbIgMgMFjziGAyV0cU40IwLC9ppYvMHfbELsfTzVaxxqVoVTVLyzQGTRtw/0jx0qhW0NMvyVzvi5LwCAaRkLdBdYICABYJv57eNeSdv7fTMq0GYba7d1cc03oqcJoaue5Fr6Uu9tDrpZwI+oxkTDjDvOiWcZtBxFSLMyNPA3rYvkyf6d1X2wqPUyNZolA8bdQm1Mz1KE6V3e6AFDa+K2i0Nr9AklYrRRQ6+UsD7dJzV9fmmzIR9TJQOZBqTwt+zQ1vJNQ6Qo0a/bALee+HhwltZbBayAourTtDcmmBF89NP9wZTiP7NTLrl4F9xQjIDzEIfR6o1yTpc2rlNu6AvKDmHvsvY78YWApONYiyG7wVEuKk= ceph-9c21de84-77ad-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T00:10:29.292 INFO:teuthology.orchestra.run.smithi027.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTLCmGieeSGTg4R5IzqKpxFHPeRjR7NLerftYRhbHqEFPSj0YMYWEWnwf+moaLcQnBhw/jMO1zoAQR+EuSxShxkPywrUxDoXh4kBRtQD2KJ+xcP197kXgI6QpDKeQ+Dwskn2iWTGEckD0ZZjWC5WbIgMgMFjziGAyV0cU40IwLC9ppYvMHfbELsfTzVaxxqVoVTVLyzQGTRtw/0jx0qhW0NMvyVzvi5LwCAaRkLdBdYICABYJv57eNeSdv7fTMq0GYba7d1cc03oqcJoaue5Fr6Uu9tDrpZwI+oxkTDjDvOiWcZtBxFSLMyNPA3rYvkyf6d1X2wqPUyNZolA8bdQm1Mz1KE6V3e6AFDa+K2i0Nr9AklYrRRQ6+UsD7dJzV9fmmzIR9TJQOZBqTwt+zQ1vJNQ6Qo0a/bALee+HhwltZbBayAourTtDcmmBF89NP9wZTiP7NTLrl4F9xQjIDzEIfR6o1yTpc2rlNu6AvKDmHvsvY78YWApONYiyG7wVEuKk= ceph-9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:10:29.293 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:29 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1485140264' entity='client.admin' 2024-09-21T00:10:29.318 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTLCmGieeSGTg4R5IzqKpxFHPeRjR7NLerftYRhbHqEFPSj0YMYWEWnwf+moaLcQnBhw/jMO1zoAQR+EuSxShxkPywrUxDoXh4kBRtQD2KJ+xcP197kXgI6QpDKeQ+Dwskn2iWTGEckD0ZZjWC5WbIgMgMFjziGAyV0cU40IwLC9ppYvMHfbELsfTzVaxxqVoVTVLyzQGTRtw/0jx0qhW0NMvyVzvi5LwCAaRkLdBdYICABYJv57eNeSdv7fTMq0GYba7d1cc03oqcJoaue5Fr6Uu9tDrpZwI+oxkTDjDvOiWcZtBxFSLMyNPA3rYvkyf6d1X2wqPUyNZolA8bdQm1Mz1KE6V3e6AFDa+K2i0Nr9AklYrRRQ6+UsD7dJzV9fmmzIR9TJQOZBqTwt+zQ1vJNQ6Qo0a/bALee+HhwltZbBayAourTtDcmmBF89NP9wZTiP7NTLrl4F9xQjIDzEIfR6o1yTpc2rlNu6AvKDmHvsvY78YWApONYiyG7wVEuKk= ceph-9c21de84-77ad-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T00:10:29.363 INFO:teuthology.orchestra.run.smithi089.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTLCmGieeSGTg4R5IzqKpxFHPeRjR7NLerftYRhbHqEFPSj0YMYWEWnwf+moaLcQnBhw/jMO1zoAQR+EuSxShxkPywrUxDoXh4kBRtQD2KJ+xcP197kXgI6QpDKeQ+Dwskn2iWTGEckD0ZZjWC5WbIgMgMFjziGAyV0cU40IwLC9ppYvMHfbELsfTzVaxxqVoVTVLyzQGTRtw/0jx0qhW0NMvyVzvi5LwCAaRkLdBdYICABYJv57eNeSdv7fTMq0GYba7d1cc03oqcJoaue5Fr6Uu9tDrpZwI+oxkTDjDvOiWcZtBxFSLMyNPA3rYvkyf6d1X2wqPUyNZolA8bdQm1Mz1KE6V3e6AFDa+K2i0Nr9AklYrRRQ6+UsD7dJzV9fmmzIR9TJQOZBqTwt+zQ1vJNQ6Qo0a/bALee+HhwltZbBayAourTtDcmmBF89NP9wZTiP7NTLrl4F9xQjIDzEIfR6o1yTpc2rlNu6AvKDmHvsvY78YWApONYiyG7wVEuKk= ceph-9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:10:29.376 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTLCmGieeSGTg4R5IzqKpxFHPeRjR7NLerftYRhbHqEFPSj0YMYWEWnwf+moaLcQnBhw/jMO1zoAQR+EuSxShxkPywrUxDoXh4kBRtQD2KJ+xcP197kXgI6QpDKeQ+Dwskn2iWTGEckD0ZZjWC5WbIgMgMFjziGAyV0cU40IwLC9ppYvMHfbELsfTzVaxxqVoVTVLyzQGTRtw/0jx0qhW0NMvyVzvi5LwCAaRkLdBdYICABYJv57eNeSdv7fTMq0GYba7d1cc03oqcJoaue5Fr6Uu9tDrpZwI+oxkTDjDvOiWcZtBxFSLMyNPA3rYvkyf6d1X2wqPUyNZolA8bdQm1Mz1KE6V3e6AFDa+K2i0Nr9AklYrRRQ6+UsD7dJzV9fmmzIR9TJQOZBqTwt+zQ1vJNQ6Qo0a/bALee+HhwltZbBayAourTtDcmmBF89NP9wZTiP7NTLrl4F9xQjIDzEIfR6o1yTpc2rlNu6AvKDmHvsvY78YWApONYiyG7wVEuKk= ceph-9c21de84-77ad-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T00:10:29.419 INFO:teuthology.orchestra.run.smithi098.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTLCmGieeSGTg4R5IzqKpxFHPeRjR7NLerftYRhbHqEFPSj0YMYWEWnwf+moaLcQnBhw/jMO1zoAQR+EuSxShxkPywrUxDoXh4kBRtQD2KJ+xcP197kXgI6QpDKeQ+Dwskn2iWTGEckD0ZZjWC5WbIgMgMFjziGAyV0cU40IwLC9ppYvMHfbELsfTzVaxxqVoVTVLyzQGTRtw/0jx0qhW0NMvyVzvi5LwCAaRkLdBdYICABYJv57eNeSdv7fTMq0GYba7d1cc03oqcJoaue5Fr6Uu9tDrpZwI+oxkTDjDvOiWcZtBxFSLMyNPA3rYvkyf6d1X2wqPUyNZolA8bdQm1Mz1KE6V3e6AFDa+K2i0Nr9AklYrRRQ6+UsD7dJzV9fmmzIR9TJQOZBqTwt+zQ1vJNQ6Qo0a/bALee+HhwltZbBayAourTtDcmmBF89NP9wZTiP7NTLrl4F9xQjIDzEIfR6o1yTpc2rlNu6AvKDmHvsvY78YWApONYiyG7wVEuKk= ceph-9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:10:29.431 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T00:10:29.819 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:10:30.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:30 smithi027 ceph-mon[30745]: mgrmap e13: x(active, since 12s) 2024-09-21T00:10:31.802 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T00:10:31.803 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T00:10:32.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:31 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3379043603' entity='client.admin' 2024-09-21T00:10:32.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:31 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:32.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:31 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:32.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:31 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"} : dispatch 2024-09-21T00:10:32.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:31 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:32.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:31 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:10:32.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:31 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:32.175 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:10:33.797 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi089 2024-09-21T00:10:33.797 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T00:10:33.797 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/etc/ceph/ceph.conf 2024-09-21T00:10:33.816 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T00:10:33.816 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:10:33.874 INFO:tasks.cephadm:Adding host smithi089 to orchestrator... 2024-09-21T00:10:33.875 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch host add smithi089 2024-09-21T00:10:34.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:33 smithi027 ceph-mon[30745]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:10:34.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:33 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:34.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:33 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:10:34.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:33 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:10:34.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:33 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:10:34.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:33 smithi027 ceph-mon[30745]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T00:10:34.253 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:10:35.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:35 smithi027 ceph-mon[30745]: Updating smithi027:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:10:35.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:35 smithi027 ceph-mon[30745]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:10:35.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:35 smithi027 ceph-mon[30745]: Updating smithi027:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T00:10:35.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:35 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:35.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:35 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:35.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:35 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:36.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:36 smithi027 ceph-mon[30745]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi089", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:10:38.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:38 smithi027 ceph-mon[30745]: Deploying cephadm binary to smithi089 2024-09-21T00:10:38.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:38 smithi027 ceph-mon[30745]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:38.808 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi089' with addr '172.21.15.89' 2024-09-21T00:10:39.542 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T00:10:39.914 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:10:40.061 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:39 smithi027 ceph-mon[30745]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:40.062 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:39 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:40.062 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:39 smithi027 ceph-mon[30745]: Added host smithi089 2024-09-21T00:10:40.062 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:39 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:10:40.062 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:39 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:40.062 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:39 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:40.714 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:10:40.714 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}] 2024-09-21T00:10:41.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:41 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:41.363 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi098 2024-09-21T00:10:41.364 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T00:10:41.364 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.conf 2024-09-21T00:10:41.383 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T00:10:41.383 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:10:41.441 INFO:tasks.cephadm:Adding host smithi098 to orchestrator... 2024-09-21T00:10:41.441 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch host add smithi098 2024-09-21T00:10:41.811 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:10:42.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:42 smithi027 ceph-mon[30745]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:42.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:42 smithi027 ceph-mon[30745]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:10:44.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:43 smithi027 ceph-mon[30745]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:44.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:43 smithi027 ceph-mon[30745]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:10:44.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:43 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:44.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:43 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:44.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:43 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:44.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:43 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:44.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:43 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"} : dispatch 2024-09-21T00:10:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:43 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:10:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:43 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:10:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:43 smithi027 ceph-mon[30745]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T00:10:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:43 smithi027 ceph-mon[30745]: Updating smithi089:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:10:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:43 smithi027 ceph-mon[30745]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:10:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:43 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:43 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:43 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:45.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:44 smithi027 ceph-mon[30745]: Updating smithi089:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T00:10:45.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:44 smithi027 ceph-mon[30745]: Deploying cephadm binary to smithi098 2024-09-21T00:10:46.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:45 smithi027 ceph-mon[30745]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:46.454 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi098' with addr '172.21.15.98' 2024-09-21T00:10:47.180 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T00:10:47.555 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:10:47.733 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:47 smithi027 ceph-mon[30745]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:47.733 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:47 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:47.733 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:47 smithi027 ceph-mon[30745]: Added host smithi098 2024-09-21T00:10:47.733 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:47 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:10:47.733 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:47 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:47.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:47 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:48.376 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:10:48.376 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}, {"addr": "172.21.15.98", "hostname": "smithi098", "labels": [], "status": ""}] 2024-09-21T00:10:49.047 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:48 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:49.088 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T00:10:49.088 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T00:10:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:10:49.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:49 smithi027 ceph-mon[30745]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:10:49.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:49 smithi027 ceph-mon[30745]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:50.784 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2024-09-21T00:10:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:50 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2731734552' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T00:10:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:50 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:50 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:50 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:50 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:50 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T00:10:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:50 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:10:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:50 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:10:51.484 INFO:tasks.cephadm:Adding mon.a on smithi027 2024-09-21T00:10:51.484 INFO:tasks.cephadm:Adding mon.b on smithi089 2024-09-21T00:10:51.484 INFO:tasks.cephadm:Adding mon.c on smithi098 2024-09-21T00:10:51.484 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi027:172.21.15.27=a;smithi089:172.21.15.89=b;smithi098:172.21.15.98=c' 2024-09-21T00:10:51.718 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:10:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:51 smithi027 ceph-mon[30745]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:51 smithi027 ceph-mon[30745]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T00:10:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:51 smithi027 ceph-mon[30745]: Updating smithi098:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:10:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:51 smithi027 ceph-mon[30745]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:10:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:51 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2731734552' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T00:10:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:51 smithi027 ceph-mon[30745]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T00:10:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:51 smithi027 ceph-mon[30745]: Updating smithi098:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T00:10:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:51 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:51 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:51 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:52.376 INFO:teuthology.orchestra.run.smithi098.stdout:Scheduled mon update... 2024-09-21T00:10:52.964 DEBUG:teuthology.orchestra.run.smithi089:mon.b> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T00:10:52.967 DEBUG:teuthology.orchestra.run.smithi098:mon.c> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T00:10:52.969 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T00:10:52.969 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T00:10:53.633 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:10:53.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:53 smithi027 ceph-mon[30745]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi089:172.21.15.89=b;smithi098:172.21.15.98=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:10:53.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:53 smithi027 ceph-mon[30745]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi089:172.21.15.89=b;smithi098:172.21.15.98=c;count:3 2024-09-21T00:10:53.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:53 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:53.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:53 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:10:53.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:53 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:10:53.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:53 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:10:53.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:53 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:53.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:53 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:10:53.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:53 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:10:53.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:53 smithi027 ceph-mon[30745]: Deploying daemon mon.c on smithi098 2024-09-21T00:10:53.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:53 smithi027 ceph-mon[30745]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:54.877 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:10:54.877 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":1,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","modified":"2024-09-21T00:09:16.349878Z","created":"2024-09-21T00:09:16.349878Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T00:10:54.877 INFO:teuthology.orchestra.run.smithi098.stderr:dumped monmap epoch 1 2024-09-21T00:10:55.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:55 smithi027 ceph-mon[30745]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:55.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:55 smithi027 ceph-mon[30745]: from='client.? 172.21.15.98:0/342139072' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T00:10:56.520 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T00:10:56.521 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T00:10:56.813 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:10:57.527 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:10:57.527 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":1,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","modified":"2024-09-21T00:09:16.349878Z","created":"2024-09-21T00:09:16.349878Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T00:10:57.527 INFO:teuthology.orchestra.run.smithi098.stderr:dumped monmap epoch 1 2024-09-21T00:10:57.674 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 podman[34692]: 2024-09-21 00:10:57.64146532 +0000 UTC m=+0.353827870 container start e211261a2fc96ffd88b03b5787efdcfe94e444afd0bc9ef3e437a71e54343166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-c, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T00:10:57.674 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T00:10:57.674 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T00:10:57.674 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: pidfile_write: ignore empty --pid-file 2024-09-21T00:10:57.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:10:57 smithi027 ceph-mon[30745]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:57.983 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: load: jerasure load: lrc 2024-09-21T00:10:57.983 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: RocksDB version: 7.9.2 2024-09-21T00:10:57.983 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Git sha 0 2024-09-21T00:10:57.983 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T00:10:57.983 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: DB SUMMARY 2024-09-21T00:10:57.983 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: DB Session ID: 5QIYDXVSCJQICR0A28QL 2024-09-21T00:10:57.983 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: CURRENT file: CURRENT 2024-09-21T00:10:57.984 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T00:10:57.984 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T00:10:57.984 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-21T00:10:57.984 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-21T00:10:57.984 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.error_if_exists: 0 2024-09-21T00:10:57.984 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.create_if_missing: 0 2024-09-21T00:10:57.984 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.paranoid_checks: 1 2024-09-21T00:10:57.984 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T00:10:57.984 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T00:10:57.984 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T00:10:57.985 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.env: 0x55c93c50ed60 2024-09-21T00:10:57.985 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T00:10:57.985 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.info_log: 0x55c941587360 2024-09-21T00:10:57.985 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T00:10:57.985 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.statistics: (nil) 2024-09-21T00:10:57.985 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.use_fsync: 0 2024-09-21T00:10:57.985 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_log_file_size: 0 2024-09-21T00:10:57.985 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T00:10:57.985 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T00:10:57.985 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T00:10:57.985 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T00:10:57.985 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.allow_fallocate: 1 2024-09-21T00:10:57.985 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T00:10:57.985 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T00:10:57.986 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.use_direct_reads: 0 2024-09-21T00:10:57.986 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T00:10:57.986 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T00:10:57.986 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.db_log_dir: 2024-09-21T00:10:57.986 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.wal_dir: 2024-09-21T00:10:57.986 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T00:10:57.986 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T00:10:57.986 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T00:10:57.986 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T00:10:57.988 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T00:10:57.988 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T00:10:57.988 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T00:10:57.988 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T00:10:57.988 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.write_buffer_manager: 0x55c94158bd60 2024-09-21T00:10:57.988 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T00:10:57.988 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T00:10:57.988 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T00:10:57.988 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T00:10:57.988 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T00:10:57.989 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T00:10:57.989 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T00:10:57.989 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T00:10:57.989 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.unordered_write: 0 2024-09-21T00:10:57.989 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T00:10:57.989 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T00:10:57.989 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T00:10:57.989 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T00:10:57.989 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.row_cache: None 2024-09-21T00:10:57.989 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.wal_filter: None 2024-09-21T00:10:57.989 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T00:10:57.989 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T00:10:57.989 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.two_write_queues: 0 2024-09-21T00:10:57.990 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T00:10:57.990 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.wal_compression: 0 2024-09-21T00:10:57.990 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.atomic_flush: 0 2024-09-21T00:10:57.990 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T00:10:57.990 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T00:10:57.990 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T00:10:57.990 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.log_readahead_size: 0 2024-09-21T00:10:57.990 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T00:10:57.990 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T00:10:57.990 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T00:10:57.990 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T00:10:57.990 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T00:10:57.991 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T00:10:57.991 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T00:10:57.991 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_background_jobs: 2 2024-09-21T00:10:57.991 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_background_compactions: -1 2024-09-21T00:10:57.991 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_subcompactions: 1 2024-09-21T00:10:57.991 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T00:10:57.991 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T00:10:57.991 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T00:10:57.991 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T00:10:57.991 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T00:10:57.991 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T00:10:57.991 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T00:10:57.991 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T00:10:57.991 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_open_files: -1 2024-09-21T00:10:57.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T00:10:57.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T00:10:57.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T00:10:57.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T00:10:57.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_background_flushes: -1 2024-09-21T00:10:57.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Compression algorithms supported: 2024-09-21T00:10:57.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: kZSTD supported: 0 2024-09-21T00:10:57.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: kXpressCompression supported: 0 2024-09-21T00:10:57.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: kBZip2Compression supported: 0 2024-09-21T00:10:57.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T00:10:57.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: kLZ4Compression supported: 1 2024-09-21T00:10:57.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: kZlibCompression supported: 1 2024-09-21T00:10:57.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T00:10:57.993 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: kSnappyCompression supported: 1 2024-09-21T00:10:57.993 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T00:10:57.993 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T00:10:57.993 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-21T00:10:57.993 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T00:10:57.993 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T00:10:57.993 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.merge_operator: 2024-09-21T00:10:57.993 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compaction_filter: None 2024-09-21T00:10:57.993 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T00:10:57.993 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T00:10:57.993 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T00:10:57.993 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T00:10:57.994 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c941586fa0) 2024-09-21T00:10:57.994 INFO:journalctl@ceph.mon.c.smithi098.stdout: cache_index_and_filter_blocks: 1 2024-09-21T00:10:57.994 INFO:journalctl@ceph.mon.c.smithi098.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T00:10:57.994 INFO:journalctl@ceph.mon.c.smithi098.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T00:10:57.994 INFO:journalctl@ceph.mon.c.smithi098.stdout: pin_top_level_index_and_filter: 1 2024-09-21T00:10:57.994 INFO:journalctl@ceph.mon.c.smithi098.stdout: index_type: 0 2024-09-21T00:10:57.994 INFO:journalctl@ceph.mon.c.smithi098.stdout: data_block_index_type: 0 2024-09-21T00:10:57.994 INFO:journalctl@ceph.mon.c.smithi098.stdout: index_shortening: 1 2024-09-21T00:10:57.994 INFO:journalctl@ceph.mon.c.smithi098.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T00:10:57.994 INFO:journalctl@ceph.mon.c.smithi098.stdout: checksum: 4 2024-09-21T00:10:57.994 INFO:journalctl@ceph.mon.c.smithi098.stdout: no_block_cache: 0 2024-09-21T00:10:57.994 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_cache: 0x55c93d54b610 2024-09-21T00:10:57.994 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_cache_name: BinnedLRUCache 2024-09-21T00:10:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_cache_options: 2024-09-21T00:10:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout: capacity : 536870912 2024-09-21T00:10:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout: num_shard_bits : 4 2024-09-21T00:10:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout: strict_capacity_limit : 0 2024-09-21T00:10:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout: high_pri_pool_ratio: 0.000 2024-09-21T00:10:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_cache_compressed: (nil) 2024-09-21T00:10:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout: persistent_cache: (nil) 2024-09-21T00:10:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_size: 4096 2024-09-21T00:10:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_size_deviation: 10 2024-09-21T00:10:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_restart_interval: 16 2024-09-21T00:10:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout: index_block_restart_interval: 1 2024-09-21T00:10:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout: metadata_block_size: 4096 2024-09-21T00:10:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout: partition_filters: 0 2024-09-21T00:10:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout: use_delta_encoding: 1 2024-09-21T00:10:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout: filter_policy: bloomfilter 2024-09-21T00:10:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout: whole_key_filtering: 1 2024-09-21T00:10:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout: verify_compression: 0 2024-09-21T00:10:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout: read_amp_bytes_per_bit: 0 2024-09-21T00:10:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout: format_version: 5 2024-09-21T00:10:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout: enable_index_compression: 1 2024-09-21T00:10:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_align: 0 2024-09-21T00:10:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout: max_auto_readahead_size: 262144 2024-09-21T00:10:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout: prepopulate_block_cache: 0 2024-09-21T00:10:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout: initial_auto_readahead_size: 8192 2024-09-21T00:10:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T00:10:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T00:10:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T00:10:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compression: NoCompression 2024-09-21T00:10:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T00:10:57.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T00:10:57.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T00:10:57.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.num_levels: 7 2024-09-21T00:10:57.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T00:10:57.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T00:10:57.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T00:10:57.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T00:10:57.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T00:10:57.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T00:10:57.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T00:10:57.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T00:10:57.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T00:10:57.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T00:10:57.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T00:10:57.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T00:10:57.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T00:10:57.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T00:10:57.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T00:10:57.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T00:10:57.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T00:10:57.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T00:10:57.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T00:10:57.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T00:10:57.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T00:10:57.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T00:10:57.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T00:10:57.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T00:10:57.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T00:10:57.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T00:10:57.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T00:10:57.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T00:10:57.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T00:10:57.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T00:10:57.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T00:10:57.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T00:10:57.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T00:10:57.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T00:10:57.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T00:10:57.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T00:10:57.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T00:10:58.000 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T00:10:58.000 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T00:10:58.000 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T00:10:58.000 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T00:10:58.000 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T00:10:58.000 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T00:10:58.000 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T00:10:58.000 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T00:10:58.000 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T00:10:58.000 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T00:10:58.000 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T00:10:58.000 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T00:10:58.000 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T00:10:58.000 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T00:10:58.000 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T00:10:58.001 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T00:10:58.001 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T00:10:58.001 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.inplace_update_support: 0 2024-09-21T00:10:58.001 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T00:10:58.001 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T00:10:58.001 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T00:10:58.001 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T00:10:58.001 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.bloom_locality: 0 2024-09-21T00:10:58.001 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.max_successive_merges: 0 2024-09-21T00:10:58.001 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T00:10:58.001 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T00:10:58.001 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T00:10:58.001 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T00:10:58.002 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.ttl: 2592000 2024-09-21T00:10:58.002 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T00:10:58.002 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T00:10:58.002 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T00:10:58.002 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.enable_blob_files: false 2024-09-21T00:10:58.002 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.min_blob_size: 0 2024-09-21T00:10:58.002 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T00:10:58.002 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T00:10:58.002 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T00:10:58.002 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T00:10:58.002 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T00:10:58.002 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T00:10:58.002 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T00:10:58.002 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T00:10:58.002 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T00:10:58.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T00:10:58.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 596cc557-eb2a-4e02-94aa-021d568dba15 2024-09-21T00:10:58.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726877457678215, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T00:10:58.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T00:10:58.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726877457678650, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726877457, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "596cc557-eb2a-4e02-94aa-021d568dba15", "db_session_id": "5QIYDXVSCJQICR0A28QL", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T00:10:58.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726877457678743, "job": 1, "event": "recovery_finished"} 2024-09-21T00:10:58.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T00:10:58.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T00:10:58.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c9415aaa80 2024-09-21T00:10:58.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: DB pointer 0x55c942350000 2024-09-21T00:10:58.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T00:10:58.003 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T00:10:58.003 INFO:journalctl@ceph.mon.c.smithi098.stdout: ** DB Stats ** 2024-09-21T00:10:58.003 INFO:journalctl@ceph.mon.c.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T00:10:58.004 INFO:journalctl@ceph.mon.c.smithi098.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T00:10:58.004 INFO:journalctl@ceph.mon.c.smithi098.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T00:10:58.004 INFO:journalctl@ceph.mon.c.smithi098.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T00:10:58.004 INFO:journalctl@ceph.mon.c.smithi098.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T00:10:58.004 INFO:journalctl@ceph.mon.c.smithi098.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T00:10:58.004 INFO:journalctl@ceph.mon.c.smithi098.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T00:10:58.004 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T00:10:58.004 INFO:journalctl@ceph.mon.c.smithi098.stdout: ** Compaction Stats [default] ** 2024-09-21T00:10:58.004 INFO:journalctl@ceph.mon.c.smithi098.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T00:10:58.004 INFO:journalctl@ceph.mon.c.smithi098.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T00:10:58.004 INFO:journalctl@ceph.mon.c.smithi098.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T00:10:58.004 INFO:journalctl@ceph.mon.c.smithi098.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T00:10:58.004 INFO:journalctl@ceph.mon.c.smithi098.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T00:10:58.004 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T00:10:58.005 INFO:journalctl@ceph.mon.c.smithi098.stdout: ** Compaction Stats [default] ** 2024-09-21T00:10:58.005 INFO:journalctl@ceph.mon.c.smithi098.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T00:10:58.005 INFO:journalctl@ceph.mon.c.smithi098.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T00:10:58.005 INFO:journalctl@ceph.mon.c.smithi098.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T00:10:58.005 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T00:10:58.005 INFO:journalctl@ceph.mon.c.smithi098.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T00:10:58.005 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T00:10:58.005 INFO:journalctl@ceph.mon.c.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T00:10:58.005 INFO:journalctl@ceph.mon.c.smithi098.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T00:10:58.005 INFO:journalctl@ceph.mon.c.smithi098.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T00:10:58.005 INFO:journalctl@ceph.mon.c.smithi098.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T00:10:58.005 INFO:journalctl@ceph.mon.c.smithi098.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T00:10:58.005 INFO:journalctl@ceph.mon.c.smithi098.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T00:10:58.005 INFO:journalctl@ceph.mon.c.smithi098.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T00:10:58.006 INFO:journalctl@ceph.mon.c.smithi098.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T00:10:58.006 INFO:journalctl@ceph.mon.c.smithi098.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T00:10:58.006 INFO:journalctl@ceph.mon.c.smithi098.stdout: Block cache BinnedLRUCache@0x55c93d54b610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-21T00:10:58.006 INFO:journalctl@ceph.mon.c.smithi098.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T00:10:58.006 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T00:10:58.006 INFO:journalctl@ceph.mon.c.smithi098.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T00:10:58.006 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-21T00:10:58.006 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: using public_addr v2:172.21.15.98:0/0 -> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] 2024-09-21T00:10:58.006 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: starting mon.c rank -1 at public addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] at bind addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:10:58.006 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: mon.c@-1(???) e0 preinit fsid 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:10:58.006 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T00:10:58.006 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T00:10:58.006 INFO:journalctl@ceph.mon.c.smithi098.stdout: e1 2024-09-21T00:10:58.006 INFO:journalctl@ceph.mon.c.smithi098.stdout: btime 2024-09-21T00:09:20:149616+0000 2024-09-21T00:10:58.007 INFO:journalctl@ceph.mon.c.smithi098.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T00:10:58.007 INFO:journalctl@ceph.mon.c.smithi098.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T00:10:58.007 INFO:journalctl@ceph.mon.c.smithi098.stdout: legacy client fscid: -1 2024-09-21T00:10:58.007 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T00:10:58.007 INFO:journalctl@ceph.mon.c.smithi098.stdout: No filesystems configured 2024-09-21T00:10:58.007 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T00:10:58.007 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T00:10:58.007 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T00:10:58.007 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T00:10:58.007 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T00:10:58.007 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T00:10:58.007 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T00:10:58.007 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T00:10:58.007 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T00:10:58.007 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T00:10:58.008 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:58.008 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.008 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: Added host smithi089 2024-09-21T00:10:58.008 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:10:58.008 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.008 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.008 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.008 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:58.008 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:10:58.008 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:58.008 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:10:58.009 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.009 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.009 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.009 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.009 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"} : dispatch 2024-09-21T00:10:58.009 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:10:58.009 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:10:58.009 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T00:10:58.009 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: Updating smithi089:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:10:58.009 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:10:58.009 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.009 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.009 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.009 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: Updating smithi089:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T00:10:58.010 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: Deploying cephadm binary to smithi098 2024-09-21T00:10:58.010 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:58.010 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:58.010 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.010 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: Added host smithi098 2024-09-21T00:10:58.010 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:10:58.010 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.010 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.010 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.010 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:10:58.010 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:58.010 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2731734552' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T00:10:58.010 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.010 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.010 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.011 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.011 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T00:10:58.011 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:10:58.011 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:10:58.011 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:58.011 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T00:10:58.011 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: Updating smithi098:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:10:58.011 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:10:58.011 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2731734552' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T00:10:58.011 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T00:10:58.011 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: Updating smithi098:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T00:10:58.011 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.011 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.011 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi089:172.21.15.89=b;smithi098:172.21.15.98=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:10:58.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi089:172.21.15.89=b;smithi098:172.21.15.98=c;count:3 2024-09-21T00:10:58.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:10:58.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:10:58.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:10:58.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:10:58.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:10:58.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:10:58.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: Deploying daemon mon.c on smithi098 2024-09-21T00:10:58.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:58.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:58.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: from='client.? 172.21.15.98:0/342139072' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T00:10:58.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:10:58.012 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 ceph-mon[34725]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T00:10:58.013 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 bash[34692]: e211261a2fc96ffd88b03b5787efdcfe94e444afd0bc9ef3e437a71e54343166 2024-09-21T00:10:58.013 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:10:57 smithi098 systemd[1]: Started Ceph mon.c for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:10:59.202 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T00:10:59.203 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T00:10:59.462 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:11:03.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:11:03.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: mon.a calling monitor election 2024-09-21T00:11:03.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:03.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:03.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:03.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:03.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: mon.c calling monitor election 2024-09-21T00:11:03.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:03.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:03.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:03.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:03.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T00:11:03.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: monmap epoch 2 2024-09-21T00:11:03.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: fsid 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:11:03.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: last_changed 2024-09-21T00:10:57.708193+0000 2024-09-21T00:11:03.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: created 2024-09-21T00:09:16.349878+0000 2024-09-21T00:11:03.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: min_mon_release 19 (squid) 2024-09-21T00:11:03.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: election_strategy: 1 2024-09-21T00:11:03.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T00:11:03.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T00:11:03.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: fsmap 2024-09-21T00:11:03.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T00:11:03.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: mgrmap e13: x(active, since 46s) 2024-09-21T00:11:03.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: overall HEALTH_OK 2024-09-21T00:11:03.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:03.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:03.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:03.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:11:03.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:02 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:03.160 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:11:03.160 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":2,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","modified":"2024-09-21T00:10:57.708193Z","created":"2024-09-21T00:09:16.349878Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:3300","nonce":0},{"type":"v1","addr":"172.21.15.98:6789","nonce":0}]},"addr":"172.21.15.98:6789/0","public_addr":"172.21.15.98:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T00:11:03.161 INFO:teuthology.orchestra.run.smithi098.stderr:dumped monmap epoch 2 2024-09-21T00:11:03.161 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:11:03.161 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: mon.a calling monitor election 2024-09-21T00:11:03.161 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:03.161 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:03.161 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:03.161 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:03.162 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: mon.c calling monitor election 2024-09-21T00:11:03.162 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:03.162 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:03.162 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:03.162 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:03.162 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T00:11:03.162 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: monmap epoch 2 2024-09-21T00:11:03.162 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: fsid 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:11:03.162 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: last_changed 2024-09-21T00:10:57.708193+0000 2024-09-21T00:11:03.162 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: created 2024-09-21T00:09:16.349878+0000 2024-09-21T00:11:03.162 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: min_mon_release 19 (squid) 2024-09-21T00:11:03.162 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: election_strategy: 1 2024-09-21T00:11:03.162 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T00:11:03.162 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T00:11:03.163 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: fsmap 2024-09-21T00:11:03.163 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T00:11:03.163 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: mgrmap e13: x(active, since 46s) 2024-09-21T00:11:03.163 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: overall HEALTH_OK 2024-09-21T00:11:03.163 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:03.163 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:03.163 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:03.163 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:11:03.163 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:02 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:03.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:03 smithi098 ceph-mon[34725]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:03.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:03 smithi098 ceph-mon[34725]: Deploying daemon mon.b on smithi089 2024-09-21T00:11:03.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:03 smithi098 ceph-mon[34725]: from='client.? 172.21.15.98:0/4006418465' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T00:11:03.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:03 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:04.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:03 smithi027 ceph-mon[30745]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:04.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:03 smithi027 ceph-mon[30745]: Deploying daemon mon.b on smithi089 2024-09-21T00:11:04.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:03 smithi027 ceph-mon[30745]: from='client.? 172.21.15.98:0/4006418465' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T00:11:04.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:03 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:04.719 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T00:11:04.720 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T00:11:04.975 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:11:05.067 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:04 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:11:04.708+0000 7f1bb7600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T00:11:05.673 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:11:05.673 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":2,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","modified":"2024-09-21T00:10:57.708193Z","created":"2024-09-21T00:09:16.349878Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:3300","nonce":0},{"type":"v1","addr":"172.21.15.98:6789","nonce":0}]},"addr":"172.21.15.98:6789/0","public_addr":"172.21.15.98:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T00:11:05.674 INFO:teuthology.orchestra.run.smithi098.stderr:dumped monmap epoch 2 2024-09-21T00:11:05.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:05 smithi098 ceph-mon[34725]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:05.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:05 smithi098 ceph-mon[34725]: from='client.? 172.21.15.98:0/349482382' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T00:11:06.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:05 smithi027 ceph-mon[30745]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:06.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:05 smithi027 ceph-mon[30745]: from='client.? 172.21.15.98:0/349482382' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T00:11:06.481 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:06 smithi089 systemd[1]: Started Ceph mon.b for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:11:07.186 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T00:11:07.187 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T00:11:07.437 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:11:11.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:11:11.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: mon.a calling monitor election 2024-09-21T00:11:11.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: mon.c calling monitor election 2024-09-21T00:11:11.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:11.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:11.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: mon.b calling monitor election 2024-09-21T00:11:11.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:11.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:11.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T00:11:11.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: monmap epoch 3 2024-09-21T00:11:11.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: fsid 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:11:11.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: last_changed 2024-09-21T00:11:06.226745+0000 2024-09-21T00:11:11.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: created 2024-09-21T00:09:16.349878+0000 2024-09-21T00:11:11.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: min_mon_release 19 (squid) 2024-09-21T00:11:11.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: election_strategy: 1 2024-09-21T00:11:11.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T00:11:11.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T00:11:11.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-09-21T00:11:11.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: fsmap 2024-09-21T00:11:11.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T00:11:11.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: mgrmap e13: x(active, since 54s) 2024-09-21T00:11:11.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: overall HEALTH_OK 2024-09-21T00:11:11.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:11.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:11.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:11.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:11 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:11.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:11:11.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: mon.a calling monitor election 2024-09-21T00:11:11.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: mon.c calling monitor election 2024-09-21T00:11:11.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:11.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:11.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: mon.b calling monitor election 2024-09-21T00:11:11.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:11.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.731 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:11.731 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.731 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T00:11:11.731 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: monmap epoch 3 2024-09-21T00:11:11.731 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: fsid 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:11:11.731 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: last_changed 2024-09-21T00:11:06.226745+0000 2024-09-21T00:11:11.731 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: created 2024-09-21T00:09:16.349878+0000 2024-09-21T00:11:11.732 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: min_mon_release 19 (squid) 2024-09-21T00:11:11.732 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: election_strategy: 1 2024-09-21T00:11:11.732 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T00:11:11.732 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T00:11:11.732 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-09-21T00:11:11.732 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: fsmap 2024-09-21T00:11:11.732 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T00:11:11.732 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: mgrmap e13: x(active, since 54s) 2024-09-21T00:11:11.733 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: overall HEALTH_OK 2024-09-21T00:11:11.733 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:11.733 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:11.733 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:11.733 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:11 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:11.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:11:11.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: mon.a calling monitor election 2024-09-21T00:11:11.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: mon.c calling monitor election 2024-09-21T00:11:11.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:11.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:11.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: mon.b calling monitor election 2024-09-21T00:11:11.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:11.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:11.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:11.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T00:11:11.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: monmap epoch 3 2024-09-21T00:11:11.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: fsid 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:11:11.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: last_changed 2024-09-21T00:11:06.226745+0000 2024-09-21T00:11:11.749 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: created 2024-09-21T00:09:16.349878+0000 2024-09-21T00:11:11.749 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: min_mon_release 19 (squid) 2024-09-21T00:11:11.749 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: election_strategy: 1 2024-09-21T00:11:11.749 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T00:11:11.749 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T00:11:11.749 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-09-21T00:11:11.749 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: fsmap 2024-09-21T00:11:11.749 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T00:11:11.750 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: mgrmap e13: x(active, since 54s) 2024-09-21T00:11:11.750 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: overall HEALTH_OK 2024-09-21T00:11:11.750 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:11.750 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:11.750 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:11.750 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:11 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:11:11.780 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":3,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","modified":"2024-09-21T00:11:06.226745Z","created":"2024-09-21T00:09:16.349878Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:3300","nonce":0},{"type":"v1","addr":"172.21.15.98:6789","nonce":0}]},"addr":"172.21.15.98:6789/0","public_addr":"172.21.15.98:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-21T00:11:11.780 INFO:teuthology.orchestra.run.smithi098.stderr:dumped monmap epoch 3 2024-09-21T00:11:12.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:12 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:11:12.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:12 smithi027 ceph-mon[30745]: from='client.? 172.21.15.98:0/659392041' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T00:11:12.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:12 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:12.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:12 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:12.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:12 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:11:12.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:12 smithi089 ceph-mon[34481]: from='client.? 172.21.15.98:0/659392041' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T00:11:12.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:12 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:12.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:12 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:12.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:12 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:11:12.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:12 smithi098 ceph-mon[34725]: from='client.? 172.21.15.98:0/659392041' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T00:11:12.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:12 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:12.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:12 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:13.103 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T00:11:13.104 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T00:11:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:13.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:13.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:13.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:11:13.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T00:11:13.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T00:11:13.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T00:11:13.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: Updating smithi089:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:13.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: Updating smithi098:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:13.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: Updating smithi027:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:13.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:11:13.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T00:11:13.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:13 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:13.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:13.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:13.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:11:13.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T00:11:13.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T00:11:13.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T00:11:13.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: Updating smithi089:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:13.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: Updating smithi098:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:13.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: Updating smithi027:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:13.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.731 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:11:13.731 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T00:11:13.731 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:13 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:13.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:13.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:13.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:11:13.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T00:11:13.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T00:11:13.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T00:11:13.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: Updating smithi089:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:13.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: Updating smithi098:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:13.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: Updating smithi027:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:13.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:13.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:11:13.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T00:11:13.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:13 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:14.413 INFO:teuthology.orchestra.run.smithi027.stdout:# minimal ceph.conf for 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:11:14.413 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2024-09-21T00:11:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:11:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: mon_host = [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] 2024-09-21T00:11:14.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:14 smithi027 ceph-mon[30745]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T00:11:14.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:14 smithi027 ceph-mon[30745]: Reconfiguring daemon mon.a on smithi027 2024-09-21T00:11:14.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:14 smithi089 ceph-mon[34481]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T00:11:14.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:14 smithi089 ceph-mon[34481]: Reconfiguring daemon mon.a on smithi027 2024-09-21T00:11:14.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:14 smithi098 ceph-mon[34725]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T00:11:14.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:14 smithi098 ceph-mon[34725]: Reconfiguring daemon mon.a on smithi027 2024-09-21T00:11:15.312 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T00:11:15.312 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:11:15.312 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T00:11:15.373 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:11:15.373 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:11:15.470 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T00:11:15.470 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T00:11:15.503 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T00:11:15.504 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:11:15.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:15 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3147642639' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:15.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:15 smithi027 ceph-mon[30745]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:15.573 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T00:11:15.573 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T00:11:15.605 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T00:11:15.605 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:11:15.670 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:15 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3147642639' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:15.670 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:15 smithi098 ceph-mon[34725]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:15.675 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-21T00:11:15.675 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-21T00:11:15.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:15 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3147642639' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:15.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:15 smithi089 ceph-mon[34481]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:16.051 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:17.188 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:16 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:17.188 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:16 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:17.188 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:16 smithi089 ceph-mon[34481]: Reconfiguring mon.b (monmap changed)... 2024-09-21T00:11:17.189 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:16 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:11:17.189 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:16 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T00:11:17.189 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:16 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:17.189 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:16 smithi089 ceph-mon[34481]: Reconfiguring daemon mon.b on smithi089 2024-09-21T00:11:17.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:16 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:17.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:16 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:17.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:16 smithi098 ceph-mon[34725]: Reconfiguring mon.b (monmap changed)... 2024-09-21T00:11:17.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:16 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:11:17.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:16 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T00:11:17.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:16 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:17.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:16 smithi098 ceph-mon[34725]: Reconfiguring daemon mon.b on smithi089 2024-09-21T00:11:17.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:16 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:17.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:16 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:17.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:16 smithi027 ceph-mon[30745]: Reconfiguring mon.b (monmap changed)... 2024-09-21T00:11:17.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:16 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:11:17.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:16 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T00:11:17.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:16 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:17.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:16 smithi027 ceph-mon[30745]: Reconfiguring daemon mon.b on smithi089 2024-09-21T00:11:18.167 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: ignoring --setuser ceph since I am not root 2024-09-21T00:11:18.167 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: ignoring --setgroup ceph since I am not root 2024-09-21T00:11:18.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:17 smithi027 ceph-mon[30745]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:18.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:17 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/248534819' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T00:11:18.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:17 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T00:11:18.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:17 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:18.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:17 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:18.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:17 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:11:18.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:17 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T00:11:18.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:17 smithi027 ceph-mon[30745]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:17 smithi089 ceph-mon[34481]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:17 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/248534819' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T00:11:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:17 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T00:11:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:17 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:17 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:17 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:11:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:17 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T00:11:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:17 smithi089 ceph-mon[34481]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:18.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:17 smithi098 ceph-mon[34725]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:18.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:17 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/248534819' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T00:11:18.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:17 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T00:11:18.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:17 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:18.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:17 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' 2024-09-21T00:11:18.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:17 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:11:18.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:17 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T00:11:18.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:17 smithi098 ceph-mon[34725]: from='mgr.14152 172.21.15.27:0/2551101562' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:18.650 INFO:tasks.cephadm:Adding mgr.x on smithi027 2024-09-21T00:11:18.650 INFO:tasks.cephadm:Adding mgr.y on smithi089 2024-09-21T00:11:18.650 INFO:tasks.cephadm:Adding mgr.z on smithi098 2024-09-21T00:11:18.650 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi027=x;smithi089=y;smithi098=z' 2024-09-21T00:11:18.888 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:11:18.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:18 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T00:11:18.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:18 smithi098 ceph-mon[34725]: mgrmap e14: x(active, since 61s) 2024-09-21T00:11:19.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:18 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T00:11:19.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:18 smithi089 ceph-mon[34481]: mgrmap e14: x(active, since 61s) 2024-09-21T00:11:19.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:18 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T00:11:19.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:18 smithi027 ceph-mon[30745]: mgrmap e14: x(active, since 61s) 2024-09-21T00:11:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:24 smithi027 ceph-mon[30745]: Active manager daemon x restarted 2024-09-21T00:11:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:24 smithi027 ceph-mon[30745]: Activating manager daemon x 2024-09-21T00:11:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:24 smithi027 ceph-mon[30745]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T00:11:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:24 smithi027 ceph-mon[30745]: mgrmap e15: x(active, starting, since 0.00545693s) 2024-09-21T00:11:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:11:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T00:11:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T00:11:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T00:11:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T00:11:24.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:24 smithi027 ceph-mon[30745]: Manager daemon x is now available 2024-09-21T00:11:24.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:24 smithi089 ceph-mon[34481]: Active manager daemon x restarted 2024-09-21T00:11:24.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:24 smithi089 ceph-mon[34481]: Activating manager daemon x 2024-09-21T00:11:24.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:24 smithi089 ceph-mon[34481]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T00:11:24.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:24 smithi089 ceph-mon[34481]: mgrmap e15: x(active, starting, since 0.00545693s) 2024-09-21T00:11:24.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:11:24.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:24.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:24.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T00:11:24.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T00:11:24.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T00:11:24.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T00:11:24.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:24 smithi089 ceph-mon[34481]: Manager daemon x is now available 2024-09-21T00:11:24.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:24 smithi098 ceph-mon[34725]: Active manager daemon x restarted 2024-09-21T00:11:24.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:24 smithi098 ceph-mon[34725]: Activating manager daemon x 2024-09-21T00:11:24.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:24 smithi098 ceph-mon[34725]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T00:11:24.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:24 smithi098 ceph-mon[34725]: mgrmap e15: x(active, starting, since 0.00545693s) 2024-09-21T00:11:24.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:11:24.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:11:24.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:11:24.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T00:11:24.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T00:11:24.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T00:11:24.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T00:11:24.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:24 smithi098 ceph-mon[34725]: Manager daemon x is now available 2024-09-21T00:11:25.258 INFO:teuthology.orchestra.run.smithi098.stdout:Scheduled mgr update... 2024-09-21T00:11:25.565 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:25 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:11:25.565 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:25 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T00:11:25.565 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T00:11:25.565 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:25 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T00:11:25.565 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T00:11:25.565 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:25 smithi098 ceph-mon[34725]: mgrmap e16: x(active, since 1.01313s) 2024-09-21T00:11:25.565 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:25.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:25 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:11:25.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:25 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T00:11:25.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T00:11:25.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:25 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T00:11:25.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T00:11:25.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:25 smithi027 ceph-mon[30745]: mgrmap e16: x(active, since 1.01313s) 2024-09-21T00:11:25.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:25.635 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:25 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:11:25.635 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:25 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T00:11:25.635 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T00:11:25.635 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:25 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T00:11:25.635 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T00:11:25.635 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:25 smithi089 ceph-mon[34481]: mgrmap e16: x(active, since 1.01313s) 2024-09-21T00:11:25.635 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:25.867 DEBUG:teuthology.orchestra.run.smithi089:mgr.y> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T00:11:25.869 DEBUG:teuthology.orchestra.run.smithi098:mgr.z> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T00:11:25.872 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T00:11:25.872 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T00:11:25.876 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T00:11:25.878 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T00:11:25.924 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T00:11:25.924 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T00:11:25.970 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T00:11:25.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T00:11:26.016 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T00:11:26.016 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T00:11:26.063 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T00:11:26.063 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T00:11:26.111 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T00:11:26.112 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T00:11:26.160 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T00:11:26.160 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T00:11:26.214 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T00:11:26.214 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T00:11:26.268 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T00:11:26.268 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T00:11:26.325 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T00:11:26.325 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T00:11:26.381 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T00:11:26.381 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T00:11:26.435 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T00:11:26.435 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T00:11:26.497 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T00:11:26.497 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T00:11:26.552 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T00:11:26.552 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T00:11:26.603 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T00:11:26.603 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T00:11:26.665 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T00:11:26.665 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T00:11:26.734 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T00:11:26.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T00:11:26.785 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T00:11:26.786 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T00:11:26.786 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:26.883 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T00:11:26.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T00:11:26.894 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-21T00:11:26.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mgr debug_cephsqlite 20 2024-09-21T00:11:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:26.943 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T00:11:26.943 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T00:11:27.002 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T00:11:27.002 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T00:11:27.071 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T00:11:27.071 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T00:11:27.131 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-21T00:11:27.131 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-21T00:11:27.220 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-21T00:11:27.220 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-21T00:11:27.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-mon[30745]: Saving service mgr spec with placement smithi027=x;smithi089=y;smithi098=z;count:3 2024-09-21T00:11:27.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:27.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:27.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.222 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:11:27.078+0000 7fbefd600640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:00:11:27] ENGINE Error in HTTPServer.serve 2024-09-21T00:11:27.222 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: Traceback (most recent call last): 2024-09-21T00:11:27.222 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T00:11:27.222 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: self._connections.run(self.expiration_interval) 2024-09-21T00:11:27.222 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T00:11:27.222 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: self._run(expiration_interval) 2024-09-21T00:11:27.222 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T00:11:27.223 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T00:11:27.223 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T00:11:27.223 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T00:11:27.223 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T00:11:27.223 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: s = self.context.wrap_socket( 2024-09-21T00:11:27.223 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T00:11:27.223 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: return self.sslsocket_class._create( 2024-09-21T00:11:27.223 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T00:11:27.223 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: self.do_handshake() 2024-09-21T00:11:27.223 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T00:11:27.223 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: self._sslobj.do_handshake() 2024-09-21T00:11:27.224 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T00:11:27.224 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:11:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:11:27.224 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:27.295 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-21T00:11:27.295 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-21T00:11:27.362 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T00:11:27.362 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T00:11:27.424 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T00:11:27.424 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T00:11:27.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:27 smithi089 ceph-mon[34481]: Saving service mgr spec with placement smithi027=x;smithi089=y;smithi098=z;count:3 2024-09-21T00:11:27.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:27 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.491 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T00:11:27.491 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T00:11:27.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:27 smithi098 ceph-mon[34725]: Saving service mgr spec with placement smithi027=x;smithi089=y;smithi098=z;count:3 2024-09-21T00:11:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:27 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:27.565 DEBUG:tasks.cephadm:set 30 configs 2024-09-21T00:11:27.566 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi027') 2024-09-21T00:11:28.332 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:28 smithi027 ceph-mon[30745]: [21/Sep/2024:00:11:26] ENGINE Bus STARTING 2024-09-21T00:11:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:28 smithi027 ceph-mon[30745]: [21/Sep/2024:00:11:27] ENGINE Serving on https://172.21.15.27:7150 2024-09-21T00:11:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:28 smithi027 ceph-mon[30745]: mgrmap e17: x(active, since 2s) 2024-09-21T00:11:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:28 smithi027 ceph-mon[30745]: [21/Sep/2024:00:11:27] ENGINE Error in HTTPServer.serve 2024-09-21T00:11:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout: Traceback (most recent call last): 2024-09-21T00:11:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T00:11:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._connections.run(self.expiration_interval) 2024-09-21T00:11:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T00:11:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._run(expiration_interval) 2024-09-21T00:11:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T00:11:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T00:11:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T00:11:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T00:11:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T00:11:28.334 INFO:journalctl@ceph.mon.a.smithi027.stdout: s = self.context.wrap_socket( 2024-09-21T00:11:28.334 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T00:11:28.334 INFO:journalctl@ceph.mon.a.smithi027.stdout: return self.sslsocket_class._create( 2024-09-21T00:11:28.334 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T00:11:28.334 INFO:journalctl@ceph.mon.a.smithi027.stdout: self.do_handshake() 2024-09-21T00:11:28.334 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T00:11:28.334 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._sslobj.do_handshake() 2024-09-21T00:11:28.334 INFO:journalctl@ceph.mon.a.smithi027.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T00:11:28.334 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:28 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:28.334 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:28 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:28.334 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:28 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:28.334 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:28 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:28 smithi089 ceph-mon[34481]: [21/Sep/2024:00:11:26] ENGINE Bus STARTING 2024-09-21T00:11:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:28 smithi089 ceph-mon[34481]: [21/Sep/2024:00:11:27] ENGINE Serving on https://172.21.15.27:7150 2024-09-21T00:11:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:28 smithi089 ceph-mon[34481]: mgrmap e17: x(active, since 2s) 2024-09-21T00:11:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:28 smithi089 ceph-mon[34481]: [21/Sep/2024:00:11:27] ENGINE Error in HTTPServer.serve 2024-09-21T00:11:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout: Traceback (most recent call last): 2024-09-21T00:11:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T00:11:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout: self._connections.run(self.expiration_interval) 2024-09-21T00:11:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T00:11:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout: self._run(expiration_interval) 2024-09-21T00:11:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T00:11:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T00:11:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T00:11:28.480 INFO:journalctl@ceph.mon.b.smithi089.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T00:11:28.480 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T00:11:28.480 INFO:journalctl@ceph.mon.b.smithi089.stdout: s = self.context.wrap_socket( 2024-09-21T00:11:28.480 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T00:11:28.480 INFO:journalctl@ceph.mon.b.smithi089.stdout: return self.sslsocket_class._create( 2024-09-21T00:11:28.480 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T00:11:28.480 INFO:journalctl@ceph.mon.b.smithi089.stdout: self.do_handshake() 2024-09-21T00:11:28.480 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T00:11:28.480 INFO:journalctl@ceph.mon.b.smithi089.stdout: self._sslobj.do_handshake() 2024-09-21T00:11:28.480 INFO:journalctl@ceph.mon.b.smithi089.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T00:11:28.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:28 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:28.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:28 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:28.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:28 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:28.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:28 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:28 smithi098 ceph-mon[34725]: [21/Sep/2024:00:11:26] ENGINE Bus STARTING 2024-09-21T00:11:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:28 smithi098 ceph-mon[34725]: [21/Sep/2024:00:11:27] ENGINE Serving on https://172.21.15.27:7150 2024-09-21T00:11:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:28 smithi098 ceph-mon[34725]: mgrmap e17: x(active, since 2s) 2024-09-21T00:11:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:28 smithi098 ceph-mon[34725]: [21/Sep/2024:00:11:27] ENGINE Error in HTTPServer.serve 2024-09-21T00:11:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout: Traceback (most recent call last): 2024-09-21T00:11:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T00:11:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout: self._connections.run(self.expiration_interval) 2024-09-21T00:11:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T00:11:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout: self._run(expiration_interval) 2024-09-21T00:11:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T00:11:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T00:11:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T00:11:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T00:11:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T00:11:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout: s = self.context.wrap_socket( 2024-09-21T00:11:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T00:11:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout: return self.sslsocket_class._create( 2024-09-21T00:11:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T00:11:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout: self.do_handshake() 2024-09-21T00:11:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T00:11:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout: self._sslobj.do_handshake() 2024-09-21T00:11:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T00:11:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:28 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:28 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:28 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:28 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:29.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:29 smithi027 ceph-mon[30745]: [21/Sep/2024:00:11:27] ENGINE Serving on http://172.21.15.27:8765 2024-09-21T00:11:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:29 smithi027 ceph-mon[30745]: [21/Sep/2024:00:11:27] ENGINE Bus STARTED 2024-09-21T00:11:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:29 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:29 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:29.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:29 smithi098 ceph-mon[34725]: [21/Sep/2024:00:11:27] ENGINE Serving on http://172.21.15.27:8765 2024-09-21T00:11:29.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:29 smithi098 ceph-mon[34725]: [21/Sep/2024:00:11:27] ENGINE Bus STARTED 2024-09-21T00:11:29.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:29 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:29.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:29 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:29.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:29 smithi089 ceph-mon[34481]: [21/Sep/2024:00:11:27] ENGINE Serving on http://172.21.15.27:8765 2024-09-21T00:11:29.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:29 smithi089 ceph-mon[34481]: [21/Sep/2024:00:11:27] ENGINE Bus STARTED 2024-09-21T00:11:29.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:29 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:29.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:29 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:29.510 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:31.273 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:30 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:30 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:30 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:31.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:30 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:31.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:30 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:31.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:30 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:11:31.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:30 smithi027 ceph-mon[30745]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T00:11:31.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:30 smithi027 ceph-mon[30745]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T00:11:31.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:30 smithi027 ceph-mon[30745]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T00:11:31.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:30 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/709083058' entity='client.admin' 2024-09-21T00:11:31.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:30 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:30 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:30 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.275 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:30 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.293 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:30 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.293 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:30 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.293 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:30 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:31.293 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:30 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:31.293 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:30 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:31.293 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:30 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:11:31.293 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:30 smithi098 ceph-mon[34725]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T00:11:31.293 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:30 smithi098 ceph-mon[34725]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T00:11:31.294 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:30 smithi098 ceph-mon[34725]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T00:11:31.294 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:30 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/709083058' entity='client.admin' 2024-09-21T00:11:31.294 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:30 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.294 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:30 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.294 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:30 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.294 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:30 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.393 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:31.404 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:30 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:30 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:30 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:30 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"} : dispatch 2024-09-21T00:11:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:30 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:30 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:11:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:30 smithi089 ceph-mon[34481]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T00:11:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:30 smithi089 ceph-mon[34481]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T00:11:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:30 smithi089 ceph-mon[34481]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T00:11:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:30 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/709083058' entity='client.admin' 2024-09-21T00:11:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:30 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:30 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:30 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:30 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:31.706 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi027') 2024-09-21T00:11:31.706 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi027') 2024-09-21T00:11:32.209 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: Updating smithi089:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:32.209 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: Updating smithi098:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:32.209 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: Updating smithi027:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:32.209 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:11:32.209 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:11:32.209 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: Updating smithi089:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T00:11:32.209 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: Updating smithi098:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T00:11:32.209 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:11:32.209 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: Updating smithi027:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T00:11:32.209 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:32.210 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:32.210 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:32.210 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:32.210 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:32.210 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T00:11:32.210 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T00:11:32.210 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:31 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:32.238 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: Updating smithi089:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:32.239 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: Updating smithi098:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:32.239 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: Updating smithi027:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:32.239 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:11:32.239 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:11:32.239 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: Updating smithi089:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T00:11:32.239 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: Updating smithi098:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T00:11:32.239 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:11:32.239 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: Updating smithi027:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T00:11:32.239 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:32.239 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:32.239 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:32.240 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:32.240 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:32.240 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T00:11:32.240 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T00:11:32.240 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:31 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: Updating smithi089:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: Updating smithi098:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: Updating smithi027:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T00:11:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:11:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:11:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: Updating smithi089:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T00:11:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: Updating smithi098:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T00:11:32.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-09-21T00:11:32.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: Updating smithi027:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T00:11:32.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:32.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:32.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:32.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:32.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:32.481 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T00:11:32.481 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T00:11:32.481 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:31 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:33.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:32 smithi027 ceph-mon[30745]: Deploying daemon mgr.z on smithi098 2024-09-21T00:11:33.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:32 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:33.209 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:32 smithi098 ceph-mon[34725]: Deploying daemon mgr.z on smithi098 2024-09-21T00:11:33.210 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:32 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:33.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:32 smithi089 ceph-mon[34481]: Deploying daemon mgr.z on smithi098 2024-09-21T00:11:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:32 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:33.495 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 00:11:33 smithi098 systemd[1]: Started Ceph mgr.z for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:11:33.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:33.311+0000 7fd9f5000640 1 -- 172.21.15.27:0/4095917307 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd9d403fb70 msgr2=0x7fd9d4042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:33.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:33.311+0000 7fd9f5000640 1 --2- 172.21.15.27:0/4095917307 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd9d403fb70 0x7fd9d4042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00096c0 tx=0x7fd9e0005a20 comp rx=0 tx=0).stop 2024-09-21T00:11:33.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:33.311+0000 7fd9f5000640 1 -- 172.21.15.27:0/4095917307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9f010c020 msgr2=0x7fd9f01a1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:33.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:33.311+0000 7fd9f5000640 1 --2- 172.21.15.27:0/4095917307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9f010c020 0x7fd9f01a1e10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8009fd0 tx=0x7fd9e800ef10 comp rx=0 tx=0).stop 2024-09-21T00:11:33.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:33.311+0000 7fd9f5000640 1 -- 172.21.15.27:0/4095917307 shutdown_connections 2024-09-21T00:11:33.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:33.311+0000 7fd9f5000640 1 --2- 172.21.15.27:0/4095917307 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd9d403fb70 0x7fd9d4042080 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:33.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:33.311+0000 7fd9f5000640 1 --2- 172.21.15.27:0/4095917307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9f010c020 0x7fd9f01a1e10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:33.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:33.311+0000 7fd9f5000640 1 --2- 172.21.15.27:0/4095917307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9f010b6d0 0x7fd9f019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:33.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:33.311+0000 7fd9f5000640 1 --2- 172.21.15.27:0/4095917307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f0105d90 0x7fd9f019c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:33.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:33.311+0000 7fd9f5000640 1 -- 172.21.15.27:0/4095917307 >> 172.21.15.27:0/4095917307 conn(0x7fd9f0101860 msgr2=0x7fd9f010acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:33.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:33.311+0000 7fd9f5000640 1 -- 172.21.15.27:0/4095917307 shutdown_connections 2024-09-21T00:11:33.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:33.311+0000 7fd9f5000640 1 -- 172.21.15.27:0/4095917307 wait complete. 2024-09-21T00:11:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:33.871 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:33.871 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:33.872 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:33.875 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:33.878 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:34.217 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:34.219 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:34.220 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:34.222 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:34.441 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:34 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:34.441 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:34 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:34.441 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:34 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:34.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:34 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:34.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:34 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:34.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:34 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:34.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:34 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:34.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:34 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T00:11:34.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:34 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T00:11:34.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:34 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:34.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:34 smithi027 ceph-mon[30745]: Deploying daemon mgr.y on smithi089 2024-09-21T00:11:34.446 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:34.446 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:34.448 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:34.455 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:34.461 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:34.461 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:34.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:34 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:34.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:34 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:34.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:34 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:34.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:34 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:34.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:34 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:34.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:34 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:34.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:34 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:34.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:34 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T00:11:34.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:34 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T00:11:34.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:34 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:34.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:34 smithi089 ceph-mon[34481]: Deploying daemon mgr.y on smithi089 2024-09-21T00:11:34.483 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:34 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:34.483 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:34 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:34.483 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:34 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:34.483 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:34 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:34.483 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:34 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:34.483 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:34 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:34.483 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:34 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:34.483 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:34 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T00:11:34.484 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:34 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T00:11:34.484 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:34 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:34.484 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:34 smithi098 ceph-mon[34725]: Deploying daemon mgr.y on smithi089 2024-09-21T00:11:35.359 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:11:35 smithi089 systemd[1]: Starting Ceph mgr.y for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:11:35.615 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:11:35 smithi089 podman[35985]: 2024-09-21 00:11:35.358372766 +0000 UTC m=+0.084283771 container create 69bc3dd205031e490a9797533171f17f72dbb360b46781187bd583e1cb67e5cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T00:11:35.615 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:11:35 smithi089 podman[35985]: 2024-09-21 00:11:35.289945538 +0000 UTC m=+0.015856499 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:11:35.615 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:11:35 smithi089 podman[35985]: 2024-09-21 00:11:35.57676566 +0000 UTC m=+0.302676620 container init 69bc3dd205031e490a9797533171f17f72dbb360b46781187bd583e1cb67e5cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-y, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:11:35.616 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:11:35 smithi089 podman[35985]: 2024-09-21 00:11:35.579660398 +0000 UTC m=+0.305571359 container start 69bc3dd205031e490a9797533171f17f72dbb360b46781187bd583e1cb67e5cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-y, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T00:11:35.616 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:11:35 smithi089 ceph-mgr[36004]: -- 172.21.15.89:0/2147871347 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a2c9f090e0 con 0x55a2cecac000 2024-09-21T00:11:35.978 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:11:35 smithi089 bash[35985]: 69bc3dd205031e490a9797533171f17f72dbb360b46781187bd583e1cb67e5cd 2024-09-21T00:11:35.978 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:11:35 smithi089 systemd[1]: Started Ceph mgr.y for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:11:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.149+0000 7fd729e00640 1 -- 172.21.15.27:0/2946403853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7241023e0 msgr2=0x7fd7241027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.149+0000 7fd729e00640 1 --2- 172.21.15.27:0/2946403853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7241023e0 0x7fd7241027f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd718009900 tx=0x7fd71802a480 comp rx=0 tx=0).stop 2024-09-21T00:11:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.149+0000 7fd729e00640 1 -- 172.21.15.27:0/2946403853 shutdown_connections 2024-09-21T00:11:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.149+0000 7fd729e00640 1 --2- 172.21.15.27:0/2946403853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd72410ff90 0x7fd7241123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.149+0000 7fd729e00640 1 --2- 172.21.15.27:0/2946403853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd724102d30 0x7fd72410fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.149+0000 7fd729e00640 1 --2- 172.21.15.27:0/2946403853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7241023e0 0x7fd7241027f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.149+0000 7fd729e00640 1 -- 172.21.15.27:0/2946403853 >> 172.21.15.27:0/2946403853 conn(0x7fd7240fe2c0 msgr2=0x7fd724100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.149+0000 7fd729e00640 1 -- 172.21.15.27:0/2946403853 shutdown_connections 2024-09-21T00:11:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.149+0000 7fd729e00640 1 -- 172.21.15.27:0/2946403853 wait complete. 2024-09-21T00:11:36.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.151+0000 7fd729e00640 1 Processor -- start 2024-09-21T00:11:36.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.151+0000 7fd729e00640 1 -- start start 2024-09-21T00:11:36.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.152+0000 7fd729e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7241023e0 0x7fd72419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.152+0000 7fd729e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd724102d30 0x7fd72419c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.152+0000 7fd729e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd72410ff90 0x7fd7241a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.152+0000 7fd729e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd724115a80 con 0x7fd724102d30 2024-09-21T00:11:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.152+0000 7fd729e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd724115900 con 0x7fd7241023e0 2024-09-21T00:11:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.152+0000 7fd729e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd724115c00 con 0x7fd72410ff90 2024-09-21T00:11:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.152+0000 7fd728a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7241023e0 0x7fd72419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.152+0000 7fd729400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd72410ff90 0x7fd7241a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.152+0000 7fd728a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7241023e0 0x7fd72419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58028/0 (socket says 172.21.15.27:58028) 2024-09-21T00:11:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.152+0000 7fd728a00640 1 -- 172.21.15.27:0/2949601150 learned_addr learned my addr 172.21.15.27:0/2949601150 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:36.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.152+0000 7fd723e00640 1 --2- 172.21.15.27:0/2949601150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd724102d30 0x7fd72419c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:36.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.153+0000 7fd728a00640 1 -- 172.21.15.27:0/2949601150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd72410ff90 msgr2=0x7fd7241a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.153+0000 7fd728a00640 1 --2- 172.21.15.27:0/2949601150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd72410ff90 0x7fd7241a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.153+0000 7fd728a00640 1 -- 172.21.15.27:0/2949601150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd724102d30 msgr2=0x7fd72419c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.153+0000 7fd728a00640 1 --2- 172.21.15.27:0/2949601150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd724102d30 0x7fd72419c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.153+0000 7fd728a00640 1 -- 172.21.15.27:0/2949601150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7241a23c0 con 0x7fd7241023e0 2024-09-21T00:11:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.153+0000 7fd729400640 1 --2- 172.21.15.27:0/2949601150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd72410ff90 0x7fd7241a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:11:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.153+0000 7fd728a00640 1 --2- 172.21.15.27:0/2949601150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7241023e0 0x7fd72419c450 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd718005940 tx=0x7fd7180040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:36.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.154+0000 7fd721600640 1 -- 172.21.15.27:0/2949601150 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71803f030 con 0x7fd7241023e0 2024-09-21T00:11:36.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.154+0000 7fd721600640 1 -- 172.21.15.27:0/2949601150 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7180044c0 con 0x7fd7241023e0 2024-09-21T00:11:36.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.154+0000 7fd721600640 1 -- 172.21.15.27:0/2949601150 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd718035070 con 0x7fd7241023e0 2024-09-21T00:11:36.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.154+0000 7fd729e00640 1 -- 172.21.15.27:0/2949601150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7241a2650 con 0x7fd7241023e0 2024-09-21T00:11:36.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.154+0000 7fd729e00640 1 -- 172.21.15.27:0/2949601150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7241a8e10 con 0x7fd7241023e0 2024-09-21T00:11:36.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.155+0000 7fd729e00640 1 -- 172.21.15.27:0/2949601150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72406b560 con 0x7fd7241023e0 2024-09-21T00:11:36.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.155+0000 7fd721600640 1 -- 172.21.15.27:0/2949601150 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7fd718042030 con 0x7fd7241023e0 2024-09-21T00:11:36.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.156+0000 7fd721600640 1 --2- 172.21.15.27:0/2949601150 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd70c03fb70 0x7fd70c042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:36.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.156+0000 7fd721600640 1 -- 172.21.15.27:0/2949601150 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd718074aa0 con 0x7fd7241023e0 2024-09-21T00:11:36.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.156+0000 7fd723e00640 1 --2- 172.21.15.27:0/2949601150 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd70c03fb70 0x7fd70c042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:36.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.157+0000 7fd723e00640 1 --2- 172.21.15.27:0/2949601150 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd70c03fb70 0x7fd70c042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd7100095f0 tx=0x7fd710005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:36.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.162+0000 7fd721600640 1 -- 172.21.15.27:0/2949601150 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd7180035f0 con 0x7fd7241023e0 2024-09-21T00:11:36.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.049+0000 7fdd23000640 1 -- 172.21.15.27:0/2133771277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c103b80 msgr2=0x7fdd1c103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:36.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.049+0000 7fdd23000640 1 --2- 172.21.15.27:0/2133771277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c103b80 0x7fdd1c103f90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009900 tx=0x7fdd0c02a3f0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.050+0000 7fdd23000640 1 -- 172.21.15.27:0/2133771277 shutdown_connections 2024-09-21T00:11:36.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.050+0000 7fdd23000640 1 --2- 172.21.15.27:0/2133771277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd1c109e10 0x7fdd1c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.050+0000 7fdd23000640 1 --2- 172.21.15.27:0/2133771277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c1094c0 0x7fdd1c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.050+0000 7fdd23000640 1 --2- 172.21.15.27:0/2133771277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c103b80 0x7fdd1c103f90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.050+0000 7fdd23000640 1 -- 172.21.15.27:0/2133771277 >> 172.21.15.27:0/2133771277 conn(0x7fdd1c0ff610 msgr2=0x7fdd1c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:36.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.050+0000 7fdd23000640 1 -- 172.21.15.27:0/2133771277 shutdown_connections 2024-09-21T00:11:36.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.050+0000 7fdd23000640 1 -- 172.21.15.27:0/2133771277 wait complete. 2024-09-21T00:11:36.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.051+0000 7fdd23000640 1 Processor -- start 2024-09-21T00:11:36.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.051+0000 7fdd23000640 1 -- start start 2024-09-21T00:11:36.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.051+0000 7fdd23000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd1c103b80 0x7fdd1c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:36.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.051+0000 7fdd23000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c1094c0 0x7fdd1c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:36.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.052+0000 7fdd23000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c109e10 0x7fdd1c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:36.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.052+0000 7fdd23000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c113ad0 con 0x7fdd1c1094c0 2024-09-21T00:11:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.052+0000 7fdd23000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c113950 con 0x7fdd1c109e10 2024-09-21T00:11:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.052+0000 7fdd23000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c113c50 con 0x7fdd1c103b80 2024-09-21T00:11:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.052+0000 7fdd13e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c1094c0 0x7fdd1c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.052+0000 7fdd1be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd1c103b80 0x7fdd1c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.052+0000 7fdd1be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd1c103b80 0x7fdd1c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38908/0 (socket says 172.21.15.27:38908) 2024-09-21T00:11:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.052+0000 7fdd13e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c1094c0 0x7fdd1c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47098/0 (socket says 172.21.15.27:47098) 2024-09-21T00:11:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.052+0000 7fdd1be00640 1 -- 172.21.15.27:0/4110310894 learned_addr learned my addr 172.21.15.27:0/4110310894 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.052+0000 7fdd21000640 1 --2- 172.21.15.27:0/4110310894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c109e10 0x7fdd1c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.052+0000 7fdd1be00640 1 -- 172.21.15.27:0/4110310894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c109e10 msgr2=0x7fdd1c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.052+0000 7fdd1be00640 1 --2- 172.21.15.27:0/4110310894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c109e10 0x7fdd1c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.052+0000 7fdd1be00640 1 -- 172.21.15.27:0/4110310894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c1094c0 msgr2=0x7fdd1c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.052+0000 7fdd1be00640 1 --2- 172.21.15.27:0/4110310894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c1094c0 0x7fdd1c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.052+0000 7fdd1be00640 1 -- 172.21.15.27:0/4110310894 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1c1a87e0 con 0x7fdd1c103b80 2024-09-21T00:11:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.053+0000 7fdd1be00640 1 --2- 172.21.15.27:0/4110310894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd1c103b80 0x7fdd1c075710 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c005940 tx=0x7fdd0c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.053+0000 7fdd19600640 1 -- 172.21.15.27:0/4110310894 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c03f030 con 0x7fdd1c103b80 2024-09-21T00:11:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.053+0000 7fdd19600640 1 -- 172.21.15.27:0/4110310894 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd0c0044c0 con 0x7fdd1c103b80 2024-09-21T00:11:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.053+0000 7fdd19600640 1 -- 172.21.15.27:0/4110310894 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c035070 con 0x7fdd1c103b80 2024-09-21T00:11:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.053+0000 7fdd23000640 1 -- 172.21.15.27:0/4110310894 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1c1a8a70 con 0x7fdd1c103b80 2024-09-21T00:11:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.053+0000 7fdd23000640 1 -- 172.21.15.27:0/4110310894 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd1c1a8f00 con 0x7fdd1c103b80 2024-09-21T00:11:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.054+0000 7fdd19600640 1 -- 172.21.15.27:0/4110310894 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7fdd0c042030 con 0x7fdd1c103b80 2024-09-21T00:11:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.055+0000 7fdd23000640 1 -- 172.21.15.27:0/4110310894 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd1c104e00 con 0x7fdd1c103b80 2024-09-21T00:11:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.055+0000 7fdd19600640 1 --2- 172.21.15.27:0/4110310894 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdd0003fb70 0x7fdd00042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.055+0000 7fdd19600640 1 -- 172.21.15.27:0/4110310894 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fdd0c074ae0 con 0x7fdd1c103b80 2024-09-21T00:11:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.055+0000 7fdd13e00640 1 --2- 172.21.15.27:0/4110310894 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdd0003fb70 0x7fdd00042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.056+0000 7fdd13e00640 1 --2- 172.21.15.27:0/4110310894 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdd0003fb70 0x7fdd00042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c108e50 tx=0x7fdd04005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.061+0000 7fdd19600640 1 -- 172.21.15.27:0/4110310894 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdd0c0303a0 con 0x7fdd1c103b80 2024-09-21T00:11:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.273+0000 7fdd23000640 1 -- 172.21.15.27:0/4110310894 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fdd1c02e640 con 0x7fdd1c103b80 2024-09-21T00:11:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.274+0000 7fdd19600640 1 -- 172.21.15.27:0/4110310894 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v16)=0 v16) ==== 108+0+0 (secure 0 0 0) 0x7fdd0c036da0 con 0x7fdd1c103b80 2024-09-21T00:11:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.277+0000 7fdd23000640 1 -- 172.21.15.27:0/4110310894 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdd0003fb70 msgr2=0x7fdd00042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:36.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.277+0000 7fdd23000640 1 --2- 172.21.15.27:0/4110310894 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdd0003fb70 0x7fdd00042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c108e50 tx=0x7fdd04005f90 comp rx=0 tx=0).stop 2024-09-21T00:11:36.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.277+0000 7fdd23000640 1 -- 172.21.15.27:0/4110310894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd1c103b80 msgr2=0x7fdd1c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:36.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.277+0000 7fdd23000640 1 --2- 172.21.15.27:0/4110310894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd1c103b80 0x7fdd1c075710 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c005940 tx=0x7fdd0c0040c0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.277+0000 7fdd23000640 1 -- 172.21.15.27:0/4110310894 shutdown_connections 2024-09-21T00:11:36.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.277+0000 7fdd23000640 1 --2- 172.21.15.27:0/4110310894 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdd0003fb70 0x7fdd00042080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.277+0000 7fdd23000640 1 --2- 172.21.15.27:0/4110310894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c109e10 0x7fdd1c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.277+0000 7fdd23000640 1 --2- 172.21.15.27:0/4110310894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c1094c0 0x7fdd1c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.277+0000 7fdd23000640 1 --2- 172.21.15.27:0/4110310894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd1c103b80 0x7fdd1c075710 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.277+0000 7fdd23000640 1 -- 172.21.15.27:0/4110310894 >> 172.21.15.27:0/4110310894 conn(0x7fdd1c0ff610 msgr2=0x7fdd1c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:36.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.277+0000 7fdd23000640 1 -- 172.21.15.27:0/4110310894 shutdown_connections 2024-09-21T00:11:36.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.277+0000 7fdd23000640 1 -- 172.21.15.27:0/4110310894 wait complete. 2024-09-21T00:11:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.381+0000 7fd729e00640 1 -- 172.21.15.27:0/2949601150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7fd7241027f0 con 0x7fd7241023e0 2024-09-21T00:11:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.386+0000 7fd721600640 1 -- 172.21.15.27:0/2949601150 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v17)=0 v17) ==== 127+0+0 (secure 0 0 0) 0x7fd718039e00 con 0x7fd7241023e0 2024-09-21T00:11:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.389+0000 7fd729e00640 1 -- 172.21.15.27:0/2949601150 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd70c03fb70 msgr2=0x7fd70c042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.389+0000 7fd729e00640 1 --2- 172.21.15.27:0/2949601150 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd70c03fb70 0x7fd70c042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd7100095f0 tx=0x7fd710005f90 comp rx=0 tx=0).stop 2024-09-21T00:11:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.389+0000 7fd729e00640 1 -- 172.21.15.27:0/2949601150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7241023e0 msgr2=0x7fd72419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.389+0000 7fd729e00640 1 --2- 172.21.15.27:0/2949601150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7241023e0 0x7fd72419c450 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd718005940 tx=0x7fd7180040c0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.389+0000 7fd729e00640 1 -- 172.21.15.27:0/2949601150 shutdown_connections 2024-09-21T00:11:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.389+0000 7fd729e00640 1 --2- 172.21.15.27:0/2949601150 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd70c03fb70 0x7fd70c042080 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.389+0000 7fd729e00640 1 --2- 172.21.15.27:0/2949601150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd72410ff90 0x7fd7241a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.389+0000 7fd729e00640 1 --2- 172.21.15.27:0/2949601150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd724102d30 0x7fd72419c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.389+0000 7fd729e00640 1 --2- 172.21.15.27:0/2949601150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7241023e0 0x7fd72419c450 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.389+0000 7fd729e00640 1 -- 172.21.15.27:0/2949601150 >> 172.21.15.27:0/2949601150 conn(0x7fd7240fe2c0 msgr2=0x7fd724112160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:36.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.389+0000 7fd729e00640 1 -- 172.21.15.27:0/2949601150 shutdown_connections 2024-09-21T00:11:36.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:36.389+0000 7fd729e00640 1 -- 172.21.15.27:0/2949601150 wait complete. 2024-09-21T00:11:36.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:36 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:36.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:36 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:36.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:36 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:36.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:36 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:36.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:36 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:36.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:36 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:11:36.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:36 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:36.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:36 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:36.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:36 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:36.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:36 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:36.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:36 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:36.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:36 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:36.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:36 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:11:36.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:36 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:37.092 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:36 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:37.092 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:36 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:37.092 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:36 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:37.092 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:36 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:37.092 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:36 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:37.092 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:36 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:11:37.093 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:36 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:37.784 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:37 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:37.784 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:37 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:37.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:37 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:37.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:37 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:37.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:37 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:37.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:37 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:11:37.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:37 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:37.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:37 smithi027 ceph-mon[30745]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T00:11:37.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:37 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:37.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:37 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:37.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:37 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T00:11:37.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:37 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:37.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:37 smithi027 ceph-mon[30745]: Reconfiguring daemon mgr.x on smithi027 2024-09-21T00:11:37.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:37 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:38.089 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:37 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:38.089 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:37 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:38.089 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:37 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:38.089 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:37 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:38.090 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:37 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:38.090 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:37 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:11:38.090 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:37 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:38.090 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:37 smithi089 ceph-mon[34481]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T00:11:38.090 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:37 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:38.090 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:37 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:38.090 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:37 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T00:11:38.090 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:37 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:38.090 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:37 smithi089 ceph-mon[34481]: Reconfiguring daemon mgr.x on smithi027 2024-09-21T00:11:38.090 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:37 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:38.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.092+0000 7f01f9a00640 1 -- 172.21.15.27:0/247017247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f41094c0 msgr2=0x7f01f41098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:38.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.092+0000 7f01f9a00640 1 --2- 172.21.15.27:0/247017247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f41094c0 0x7f01f41098d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009900 tx=0x7f01e002a3f0 comp rx=0 tx=0).stop 2024-09-21T00:11:38.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.092+0000 7f01f9a00640 1 -- 172.21.15.27:0/247017247 shutdown_connections 2024-09-21T00:11:38.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.092+0000 7f01f9a00640 1 --2- 172.21.15.27:0/247017247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01f4109e10 0x7f01f4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:38.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.092+0000 7f01f9a00640 1 --2- 172.21.15.27:0/247017247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f41094c0 0x7f01f41098d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:38.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.092+0000 7f01f9a00640 1 --2- 172.21.15.27:0/247017247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01f4103b80 0x7f01f4103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:38.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.092+0000 7f01f9a00640 1 -- 172.21.15.27:0/247017247 >> 172.21.15.27:0/247017247 conn(0x7f01f40ff5d0 msgr2=0x7f01f4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:38.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.092+0000 7f01f9a00640 1 -- 172.21.15.27:0/247017247 shutdown_connections 2024-09-21T00:11:38.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.092+0000 7f01f9a00640 1 -- 172.21.15.27:0/247017247 wait complete. 2024-09-21T00:11:38.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.094+0000 7f01f9a00640 1 Processor -- start 2024-09-21T00:11:38.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.094+0000 7f01f9a00640 1 -- start start 2024-09-21T00:11:38.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.094+0000 7f01f9a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f4103b80 0x7f01f4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:38.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f9a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01f41094c0 0x7f01f4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:38.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f9a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01f4109e10 0x7f01f4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:38.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f9a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f01f4113b20 con 0x7f01f4103b80 2024-09-21T00:11:38.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f9a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41139a0 con 0x7f01f4109e10 2024-09-21T00:11:38.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f9a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f01f4113ca0 con 0x7f01f41094c0 2024-09-21T00:11:38.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f4103b80 0x7f01f4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:38.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01f41094c0 0x7f01f4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:38.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01f4109e10 0x7f01f4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:38.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01f41094c0 0x7f01f4075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38964/0 (socket says 172.21.15.27:38964) 2024-09-21T00:11:38.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f4103b80 0x7f01f4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47162/0 (socket says 172.21.15.27:47162) 2024-09-21T00:11:38.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f2000640 1 -- 172.21.15.27:0/2902056494 learned_addr learned my addr 172.21.15.27:0/2902056494 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:38.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f3400640 1 -- 172.21.15.27:0/2902056494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01f41094c0 msgr2=0x7f01f4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:38.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f3400640 1 --2- 172.21.15.27:0/2902056494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01f41094c0 0x7f01f4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:38.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f3400640 1 -- 172.21.15.27:0/2902056494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f4103b80 msgr2=0x7f01f4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:38.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f3400640 1 --2- 172.21.15.27:0/2902056494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f4103b80 0x7f01f4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:38.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.095+0000 7f01f3400640 1 -- 172.21.15.27:0/2902056494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f41a8790 con 0x7f01f4109e10 2024-09-21T00:11:38.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.096+0000 7f01f3400640 1 --2- 172.21.15.27:0/2902056494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01f4109e10 0x7f01f4076190 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00ec50 tx=0x7f01ec00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:38.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.096+0000 7f01eb400640 1 -- 172.21.15.27:0/2902056494 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec003b70 con 0x7f01f4109e10 2024-09-21T00:11:38.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.096+0000 7f01f9a00640 1 -- 172.21.15.27:0/2902056494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f41a8a80 con 0x7f01f4109e10 2024-09-21T00:11:38.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.096+0000 7f01eb400640 1 -- 172.21.15.27:0/2902056494 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01ec003d10 con 0x7f01f4109e10 2024-09-21T00:11:38.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.096+0000 7f01eb400640 1 -- 172.21.15.27:0/2902056494 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec010830 con 0x7f01f4109e10 2024-09-21T00:11:38.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.096+0000 7f01f9a00640 1 -- 172.21.15.27:0/2902056494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f41a8f40 con 0x7f01f4109e10 2024-09-21T00:11:38.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.097+0000 7f01eb400640 1 -- 172.21.15.27:0/2902056494 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f01ec0109d0 con 0x7f01f4109e10 2024-09-21T00:11:38.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.097+0000 7f01f9a00640 1 -- 172.21.15.27:0/2902056494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f4103f90 con 0x7f01f4109e10 2024-09-21T00:11:38.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.098+0000 7f01eb400640 1 --2- 172.21.15.27:0/2902056494 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f01d803fb70 0x7f01d8042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:38.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.098+0000 7f01eb400640 1 -- 172.21.15.27:0/2902056494 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f01ec054cb0 con 0x7f01f4109e10 2024-09-21T00:11:38.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.098+0000 7f01f2a00640 1 --2- 172.21.15.27:0/2902056494 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f01d803fb70 0x7f01d8042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:38.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.099+0000 7f01f2a00640 1 --2- 172.21.15.27:0/2902056494 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f01d803fb70 0x7f01d8042080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f01f4077160 tx=0x7f01e4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:38.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.103+0000 7f01eb400640 1 -- 172.21.15.27:0/2902056494 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f01ec01fd50 con 0x7f01f4109e10 2024-09-21T00:11:38.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:37 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:38.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:37 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:38.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:37 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:38.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:37 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:38.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:37 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:38.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:37 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:11:38.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:37 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:38.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:37 smithi098 ceph-mon[34725]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T00:11:38.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:37 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:38.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:37 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T00:11:38.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:37 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T00:11:38.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:37 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:11:38.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:37 smithi098 ceph-mon[34725]: Reconfiguring daemon mgr.x on smithi027 2024-09-21T00:11:38.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:37 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:38.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.304+0000 7f01f9a00640 1 -- 172.21.15.27:0/2902056494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f01f402e640 con 0x7f01f4109e10 2024-09-21T00:11:38.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.310+0000 7f01eb400640 1 -- 172.21.15.27:0/2902056494 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v19)=0 v19) ==== 119+0+0 (secure 0 0 0) 0x7f01ec00fcc0 con 0x7f01f4109e10 2024-09-21T00:11:38.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.313+0000 7f01f9a00640 1 -- 172.21.15.27:0/2902056494 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f01d803fb70 msgr2=0x7f01d8042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:38.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.313+0000 7f01f9a00640 1 --2- 172.21.15.27:0/2902056494 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f01d803fb70 0x7f01d8042080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f01f4077160 tx=0x7f01e4005f90 comp rx=0 tx=0).stop 2024-09-21T00:11:38.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.313+0000 7f01f9a00640 1 -- 172.21.15.27:0/2902056494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01f4109e10 msgr2=0x7f01f4076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:38.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.313+0000 7f01f9a00640 1 --2- 172.21.15.27:0/2902056494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01f4109e10 0x7f01f4076190 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00ec50 tx=0x7f01ec00c6a0 comp rx=0 tx=0).stop 2024-09-21T00:11:38.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.313+0000 7f01f9a00640 1 -- 172.21.15.27:0/2902056494 shutdown_connections 2024-09-21T00:11:38.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.313+0000 7f01f9a00640 1 --2- 172.21.15.27:0/2902056494 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f01d803fb70 0x7f01d8042080 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:38.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.313+0000 7f01f9a00640 1 --2- 172.21.15.27:0/2902056494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01f4109e10 0x7f01f4076190 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:38.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.313+0000 7f01f9a00640 1 --2- 172.21.15.27:0/2902056494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01f41094c0 0x7f01f4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:38.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.313+0000 7f01f9a00640 1 --2- 172.21.15.27:0/2902056494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f4103b80 0x7f01f4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:38.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.313+0000 7f01f9a00640 1 -- 172.21.15.27:0/2902056494 >> 172.21.15.27:0/2902056494 conn(0x7f01f40ff5d0 msgr2=0x7f01f41019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:38.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.313+0000 7f01f9a00640 1 -- 172.21.15.27:0/2902056494 shutdown_connections 2024-09-21T00:11:38.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.313+0000 7f01f9a00640 1 -- 172.21.15.27:0/2902056494 wait complete. 2024-09-21T00:11:38.817 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:39.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.008+0000 7f9854e00640 1 -- 172.21.15.27:0/1857094958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9850103b60 msgr2=0x7f9850103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:39.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.008+0000 7f9854e00640 1 --2- 172.21.15.27:0/1857094958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9850103b60 0x7f9850103f70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9840009900 tx=0x7f984002a410 comp rx=0 tx=0).stop 2024-09-21T00:11:39.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.008+0000 7f9854e00640 1 -- 172.21.15.27:0/1857094958 shutdown_connections 2024-09-21T00:11:39.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.008+0000 7f9854e00640 1 --2- 172.21.15.27:0/1857094958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9850109df0 0x7f9850110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.008+0000 7f9854e00640 1 --2- 172.21.15.27:0/1857094958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98501094a0 0x7f98501098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.008+0000 7f9854e00640 1 --2- 172.21.15.27:0/1857094958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9850103b60 0x7f9850103f70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.008+0000 7f9854e00640 1 -- 172.21.15.27:0/1857094958 >> 172.21.15.27:0/1857094958 conn(0x7f985007b850 msgr2=0x7f985007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:39.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.009+0000 7f9854e00640 1 -- 172.21.15.27:0/1857094958 shutdown_connections 2024-09-21T00:11:39.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.009+0000 7f9854e00640 1 -- 172.21.15.27:0/1857094958 wait complete. 2024-09-21T00:11:39.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.010+0000 7f9854e00640 1 Processor -- start 2024-09-21T00:11:39.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.010+0000 7f9854e00640 1 -- start start 2024-09-21T00:11:39.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.010+0000 7f9854e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9850103b60 0x7f985019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.010+0000 7f9854e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98501094a0 0x7f985019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.010+0000 7f9854e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9850109df0 0x7f98501a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.010+0000 7f9854e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9850113b00 con 0x7f9850103b60 2024-09-21T00:11:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.010+0000 7f9854e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9850113980 con 0x7f98501094a0 2024-09-21T00:11:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.010+0000 7f9854e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9850113c80 con 0x7f9850109df0 2024-09-21T00:11:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.011+0000 7f984de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9850103b60 0x7f985019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.011+0000 7f984e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9850109df0 0x7f98501a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.011+0000 7f984e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9850109df0 0x7f98501a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39002/0 (socket says 172.21.15.27:39002) 2024-09-21T00:11:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.011+0000 7f984e800640 1 -- 172.21.15.27:0/1663662367 learned_addr learned my addr 172.21.15.27:0/1663662367 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.011+0000 7f984d400640 1 --2- 172.21.15.27:0/1663662367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98501094a0 0x7f985019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.011+0000 7f984e800640 1 -- 172.21.15.27:0/1663662367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98501094a0 msgr2=0x7f985019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.011+0000 7f984e800640 1 --2- 172.21.15.27:0/1663662367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98501094a0 0x7f985019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.011+0000 7f984e800640 1 -- 172.21.15.27:0/1663662367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9850103b60 msgr2=0x7f985019c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.011+0000 7f984e800640 1 --2- 172.21.15.27:0/1663662367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9850103b60 0x7f985019c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.011+0000 7f984e800640 1 -- 172.21.15.27:0/1663662367 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98501a25c0 con 0x7f9850109df0 2024-09-21T00:11:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.011+0000 7f984de00640 1 --2- 172.21.15.27:0/1663662367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9850103b60 0x7f985019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:11:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.012+0000 7f984e800640 1 --2- 172.21.15.27:0/1663662367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9850109df0 0x7f98501a1e90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f984800eb10 tx=0x7f984800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.012+0000 7f9846a00640 1 -- 172.21.15.27:0/1663662367 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848004b20 con 0x7f9850109df0 2024-09-21T00:11:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.012+0000 7f9854e00640 1 -- 172.21.15.27:0/1663662367 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98501a28b0 con 0x7f9850109df0 2024-09-21T00:11:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.012+0000 7f9846a00640 1 -- 172.21.15.27:0/1663662367 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9848004cc0 con 0x7f9850109df0 2024-09-21T00:11:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.012+0000 7f9846a00640 1 -- 172.21.15.27:0/1663662367 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848010670 con 0x7f9850109df0 2024-09-21T00:11:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.013+0000 7f9854e00640 1 -- 172.21.15.27:0/1663662367 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98501a9010 con 0x7f9850109df0 2024-09-21T00:11:39.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.014+0000 7f9854e00640 1 -- 172.21.15.27:0/1663662367 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9850103f70 con 0x7f9850109df0 2024-09-21T00:11:39.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.014+0000 7f9846a00640 1 -- 172.21.15.27:0/1663662367 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f9848005020 con 0x7f9850109df0 2024-09-21T00:11:39.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.015+0000 7f9846a00640 1 --2- 172.21.15.27:0/1663662367 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f983403fb70 0x7f9834042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:39.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.015+0000 7f9846a00640 1 -- 172.21.15.27:0/1663662367 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9848054c60 con 0x7f9850109df0 2024-09-21T00:11:39.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.015+0000 7f984de00640 1 --2- 172.21.15.27:0/1663662367 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f983403fb70 0x7f9834042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:39.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.016+0000 7f984de00640 1 --2- 172.21.15.27:0/1663662367 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f983403fb70 0x7f9834042080 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9840000c00 tx=0x7f9840031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:39.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.019+0000 7f9846a00640 1 -- 172.21.15.27:0/1663662367 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9848013190 con 0x7f9850109df0 2024-09-21T00:11:39.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.266+0000 7f9854e00640 1 -- 172.21.15.27:0/1663662367 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f98501a9350 con 0x7f9850109df0 2024-09-21T00:11:39.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.273+0000 7f9846a00640 1 -- 172.21.15.27:0/1663662367 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v22)=0 v22) ==== 106+0+0 (secure 0 0 0) 0x7f9848011dc0 con 0x7f9850109df0 2024-09-21T00:11:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.276+0000 7f9854e00640 1 -- 172.21.15.27:0/1663662367 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f983403fb70 msgr2=0x7f9834042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.276+0000 7f9854e00640 1 --2- 172.21.15.27:0/1663662367 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f983403fb70 0x7f9834042080 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9840000c00 tx=0x7f9840031400 comp rx=0 tx=0).stop 2024-09-21T00:11:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.276+0000 7f9854e00640 1 -- 172.21.15.27:0/1663662367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9850109df0 msgr2=0x7f98501a1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:39.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.276+0000 7f9854e00640 1 --2- 172.21.15.27:0/1663662367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9850109df0 0x7f98501a1e90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f984800eb10 tx=0x7f984800c2d0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.276+0000 7f9854e00640 1 -- 172.21.15.27:0/1663662367 shutdown_connections 2024-09-21T00:11:39.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.276+0000 7f9854e00640 1 --2- 172.21.15.27:0/1663662367 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f983403fb70 0x7f9834042080 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.276+0000 7f9854e00640 1 --2- 172.21.15.27:0/1663662367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9850109df0 0x7f98501a1e90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.276+0000 7f9854e00640 1 --2- 172.21.15.27:0/1663662367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98501094a0 0x7f985019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.276+0000 7f9854e00640 1 --2- 172.21.15.27:0/1663662367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9850103b60 0x7f985019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.276+0000 7f9854e00640 1 -- 172.21.15.27:0/1663662367 >> 172.21.15.27:0/1663662367 conn(0x7f985007b850 msgr2=0x7f98501072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:39.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.276+0000 7f9854e00640 1 -- 172.21.15.27:0/1663662367 shutdown_connections 2024-09-21T00:11:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.276+0000 7f9854e00640 1 -- 172.21.15.27:0/1663662367 wait complete. 2024-09-21T00:11:39.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.279+0000 7fea2be00640 1 -- 172.21.15.27:0/3840409281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c1023e0 msgr2=0x7fea2c1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:39.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.279+0000 7fea2be00640 1 --2- 172.21.15.27:0/3840409281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c1023e0 0x7fea2c1027f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009900 tx=0x7fea1c032c90 comp rx=0 tx=0).stop 2024-09-21T00:11:39.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.280+0000 7fea2be00640 1 -- 172.21.15.27:0/3840409281 shutdown_connections 2024-09-21T00:11:39.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.280+0000 7fea2be00640 1 --2- 172.21.15.27:0/3840409281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2c10ff90 0x7fea2c1123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.280+0000 7fea2be00640 1 --2- 172.21.15.27:0/3840409281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea2c102d30 0x7fea2c10fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.280+0000 7fea2be00640 1 --2- 172.21.15.27:0/3840409281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c1023e0 0x7fea2c1027f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.280+0000 7fea2be00640 1 -- 172.21.15.27:0/3840409281 >> 172.21.15.27:0/3840409281 conn(0x7fea2c0fe2c0 msgr2=0x7fea2c100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.280+0000 7fea2be00640 1 -- 172.21.15.27:0/3840409281 shutdown_connections 2024-09-21T00:11:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.280+0000 7fea2be00640 1 -- 172.21.15.27:0/3840409281 wait complete. 2024-09-21T00:11:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.281+0000 7fea2be00640 1 Processor -- start 2024-09-21T00:11:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.281+0000 7fea2be00640 1 -- start start 2024-09-21T00:11:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c1023e0 0x7fea2c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea2c102d30 0x7fea2c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2c10ff90 0x7fea2c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c106e80 con 0x7fea2c10ff90 2024-09-21T00:11:39.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c106d00 con 0x7fea2c1023e0 2024-09-21T00:11:39.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2be00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c107000 con 0x7fea2c102d30 2024-09-21T00:11:39.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c1023e0 0x7fea2c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:39.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c1023e0 0x7fea2c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58140/0 (socket says 172.21.15.27:58140) 2024-09-21T00:11:39.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2aa00640 1 -- 172.21.15.27:0/3954727823 learned_addr learned my addr 172.21.15.27:0/3954727823 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:39.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2a000640 1 --2- 172.21.15.27:0/3954727823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea2c102d30 0x7fea2c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:39.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2b400640 1 --2- 172.21.15.27:0/3954727823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2c10ff90 0x7fea2c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:39.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2aa00640 1 -- 172.21.15.27:0/3954727823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea2c102d30 msgr2=0x7fea2c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:39.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2aa00640 1 --2- 172.21.15.27:0/3954727823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea2c102d30 0x7fea2c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2aa00640 1 -- 172.21.15.27:0/3954727823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2c10ff90 msgr2=0x7fea2c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:39.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2aa00640 1 --2- 172.21.15.27:0/3954727823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2c10ff90 0x7fea2c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.282+0000 7fea2aa00640 1 -- 172.21.15.27:0/3954727823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c19ada0 con 0x7fea2c1023e0 2024-09-21T00:11:39.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.283+0000 7fea2aa00640 1 --2- 172.21.15.27:0/3954727823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c1023e0 0x7fea2c198120 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fea1c000c00 tx=0x7fea1c0347f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.283+0000 7fea23400640 1 -- 172.21.15.27:0/3954727823 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c04a020 con 0x7fea2c1023e0 2024-09-21T00:11:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.283+0000 7fea23400640 1 -- 172.21.15.27:0/3954727823 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea1c034cb0 con 0x7fea2c1023e0 2024-09-21T00:11:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.283+0000 7fea23400640 1 -- 172.21.15.27:0/3954727823 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c003a10 con 0x7fea2c1023e0 2024-09-21T00:11:39.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.283+0000 7fea2be00640 1 -- 172.21.15.27:0/3954727823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2c19b030 con 0x7fea2c1023e0 2024-09-21T00:11:39.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.283+0000 7fea2be00640 1 -- 172.21.15.27:0/3954727823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2c1a4a30 con 0x7fea2c1023e0 2024-09-21T00:11:39.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.285+0000 7fea2be00640 1 -- 172.21.15.27:0/3954727823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea2c1027f0 con 0x7fea2c1023e0 2024-09-21T00:11:39.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.286+0000 7fea23400640 1 -- 172.21.15.27:0/3954727823 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7fea1c04d020 con 0x7fea2c1023e0 2024-09-21T00:11:39.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.286+0000 7fea23400640 1 --2- 172.21.15.27:0/3954727823 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fea1003fb70 0x7fea10042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:39.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.286+0000 7fea23400640 1 -- 172.21.15.27:0/3954727823 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fea1c07d980 con 0x7fea2c1023e0 2024-09-21T00:11:39.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.287+0000 7fea2a000640 1 --2- 172.21.15.27:0/3954727823 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fea1003fb70 0x7fea10042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:39.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.288+0000 7fea2a000640 1 --2- 172.21.15.27:0/3954727823 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fea1003fb70 0x7fea10042080 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fea18005fd0 tx=0x7fea18005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:39.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.290+0000 7fea23400640 1 -- 172.21.15.27:0/3954727823 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fea1c054020 con 0x7fea2c1023e0 2024-09-21T00:11:39.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.326+0000 7fea23400640 1 -- 172.21.15.27:0/3954727823 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 18) ==== 103194+0+0 (secure 0 0 0) 0x7fea1c07c1c0 con 0x7fea2c1023e0 2024-09-21T00:11:39.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.511+0000 7fea2be00640 1 -- 172.21.15.27:0/3954727823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fea2c109d80 con 0x7fea2c1023e0 2024-09-21T00:11:39.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.516+0000 7fea23400640 1 -- 172.21.15.27:0/3954727823 <== mon.2 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v23)=0 v23) ==== 120+0+0 (secure 0 0 0) 0x7fea1c04f400 con 0x7fea2c1023e0 2024-09-21T00:11:39.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.519+0000 7fea2be00640 1 -- 172.21.15.27:0/3954727823 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fea1003fb70 msgr2=0x7fea10042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:39.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.519+0000 7fea2be00640 1 --2- 172.21.15.27:0/3954727823 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fea1003fb70 0x7fea10042080 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fea18005fd0 tx=0x7fea18005850 comp rx=0 tx=0).stop 2024-09-21T00:11:39.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.519+0000 7fea2be00640 1 -- 172.21.15.27:0/3954727823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c1023e0 msgr2=0x7fea2c198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:39.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.519+0000 7fea2be00640 1 --2- 172.21.15.27:0/3954727823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c1023e0 0x7fea2c198120 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fea1c000c00 tx=0x7fea1c0347f0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.519+0000 7fea2be00640 1 -- 172.21.15.27:0/3954727823 shutdown_connections 2024-09-21T00:11:39.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.519+0000 7fea2be00640 1 --2- 172.21.15.27:0/3954727823 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fea1003fb70 0x7fea10042080 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.519+0000 7fea2be00640 1 --2- 172.21.15.27:0/3954727823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2c10ff90 0x7fea2c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.519+0000 7fea2be00640 1 --2- 172.21.15.27:0/3954727823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea2c102d30 0x7fea2c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.519+0000 7fea2be00640 1 --2- 172.21.15.27:0/3954727823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c1023e0 0x7fea2c198120 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.519+0000 7fea2be00640 1 -- 172.21.15.27:0/3954727823 >> 172.21.15.27:0/3954727823 conn(0x7fea2c0fe2c0 msgr2=0x7fea2c112120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:39.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.519+0000 7fea2be00640 1 -- 172.21.15.27:0/3954727823 shutdown_connections 2024-09-21T00:11:39.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.519+0000 7fea2be00640 1 -- 172.21.15.27:0/3954727823 wait complete. 2024-09-21T00:11:39.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:39 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:39.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:39 smithi027 ceph-mon[30745]: Standby manager daemon z started 2024-09-21T00:11:39.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:39 smithi027 ceph-mon[30745]: from='mgr.? 172.21.15.98:0/1214986643' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T00:11:39.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:39 smithi027 ceph-mon[30745]: from='mgr.? 172.21.15.98:0/1214986643' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T00:11:39.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:39 smithi027 ceph-mon[30745]: from='mgr.? 172.21.15.98:0/1214986643' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T00:11:39.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:39 smithi027 ceph-mon[30745]: from='mgr.? 172.21.15.98:0/1214986643' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T00:11:39.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:39 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:39.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:39 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:39.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:39 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:39.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.785+0000 7f713a000640 1 -- 172.21.15.27:0/1593965573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71341049b0 msgr2=0x7f7134104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.785+0000 7f713a000640 1 --2- 172.21.15.27:0/1593965573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71341049b0 0x7f7134104dc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7124009900 tx=0x7f712402a410 comp rx=0 tx=0).stop 2024-09-21T00:11:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.785+0000 7f713a000640 1 -- 172.21.15.27:0/1593965573 shutdown_connections 2024-09-21T00:11:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.785+0000 7f713a000640 1 --2- 172.21.15.27:0/1593965573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f713410de10 0x7f7134110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.785+0000 7f713a000640 1 --2- 172.21.15.27:0/1593965573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7134105300 0x7f713410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.785+0000 7f713a000640 1 --2- 172.21.15.27:0/1593965573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71341049b0 0x7f7134104dc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.785+0000 7f713a000640 1 -- 172.21.15.27:0/1593965573 >> 172.21.15.27:0/1593965573 conn(0x7f7134079730 msgr2=0x7f7134079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.785+0000 7f713a000640 1 -- 172.21.15.27:0/1593965573 shutdown_connections 2024-09-21T00:11:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.785+0000 7f713a000640 1 -- 172.21.15.27:0/1593965573 wait complete. 2024-09-21T00:11:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.787+0000 7f713a000640 1 Processor -- start 2024-09-21T00:11:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.787+0000 7f713a000640 1 -- start start 2024-09-21T00:11:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.787+0000 7f713a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71341049b0 0x7f713419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f713a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7134105300 0x7f713419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f713a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713410de10 0x7f71341a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f713a000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7134113ac0 con 0x7f71341049b0 2024-09-21T00:11:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f713a000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7134113940 con 0x7f713410de10 2024-09-21T00:11:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f713a000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7134113c40 con 0x7f7134105300 2024-09-21T00:11:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f7133400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71341049b0 0x7f713419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f7133400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71341049b0 0x7f713419c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47178/0 (socket says 172.21.15.27:47178) 2024-09-21T00:11:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f7132a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7134105300 0x7f713419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f7133400640 1 -- 172.21.15.27:0/1714556388 learned_addr learned my addr 172.21.15.27:0/1714556388 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f7133e00640 1 --2- 172.21.15.27:0/1714556388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713410de10 0x7f71341a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f7132a00640 1 -- 172.21.15.27:0/1714556388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713410de10 msgr2=0x7f71341a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f7132a00640 1 --2- 172.21.15.27:0/1714556388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713410de10 0x7f71341a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f7132a00640 1 -- 172.21.15.27:0/1714556388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71341049b0 msgr2=0x7f713419c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f7132a00640 1 --2- 172.21.15.27:0/1714556388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71341049b0 0x7f713419c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f7132a00640 1 -- 172.21.15.27:0/1714556388 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71341a2620 con 0x7f7134105300 2024-09-21T00:11:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.788+0000 7f7133e00640 1 --2- 172.21.15.27:0/1714556388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713410de10 0x7f71341a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:11:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.789+0000 7f7132a00640 1 --2- 172.21.15.27:0/1714556388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7134105300 0x7f713419cc50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f713406ad30 tx=0x7f712000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.789+0000 7f712be00640 1 -- 172.21.15.27:0/1714556388 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712000cd00 con 0x7f7134105300 2024-09-21T00:11:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.789+0000 7f712be00640 1 -- 172.21.15.27:0/1714556388 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71200035f0 con 0x7f7134105300 2024-09-21T00:11:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.789+0000 7f712be00640 1 -- 172.21.15.27:0/1714556388 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120010690 con 0x7f7134105300 2024-09-21T00:11:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.789+0000 7f713a000640 1 -- 172.21.15.27:0/1714556388 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71341a2910 con 0x7f7134105300 2024-09-21T00:11:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.789+0000 7f713a000640 1 -- 172.21.15.27:0/1714556388 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71341a9070 con 0x7f7134105300 2024-09-21T00:11:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.790+0000 7f712be00640 1 -- 172.21.15.27:0/1714556388 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f7120010830 con 0x7f7134105300 2024-09-21T00:11:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.791+0000 7f713a000640 1 -- 172.21.15.27:0/1714556388 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7134104dc0 con 0x7f7134105300 2024-09-21T00:11:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.791+0000 7f712be00640 1 --2- 172.21.15.27:0/1714556388 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f711803fb70 0x7f7118042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.791+0000 7f712be00640 1 -- 172.21.15.27:0/1714556388 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f71200548b0 con 0x7f7134105300 2024-09-21T00:11:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.791+0000 7f7133400640 1 --2- 172.21.15.27:0/1714556388 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f711803fb70 0x7f7118042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.792+0000 7f7133400640 1 --2- 172.21.15.27:0/1714556388 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f711803fb70 0x7f7118042080 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7124000c00 tx=0x7f7124005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.796+0000 7f712be00640 1 -- 172.21.15.27:0/1714556388 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7120013190 con 0x7f7134105300 2024-09-21T00:11:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.130+0000 7f713a000640 1 -- 172.21.15.27:0/1714556388 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f71341a93a0 con 0x7f7134105300 2024-09-21T00:11:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.137+0000 7f712be00640 1 -- 172.21.15.27:0/1714556388 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v20)=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f7120003ac0 con 0x7f7134105300 2024-09-21T00:11:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.141+0000 7f713a000640 1 -- 172.21.15.27:0/1714556388 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f711803fb70 msgr2=0x7f7118042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.141+0000 7f713a000640 1 --2- 172.21.15.27:0/1714556388 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f711803fb70 0x7f7118042080 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7124000c00 tx=0x7f7124005a20 comp rx=0 tx=0).stop 2024-09-21T00:11:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.142+0000 7f713a000640 1 -- 172.21.15.27:0/1714556388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7134105300 msgr2=0x7f713419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.142+0000 7f713a000640 1 --2- 172.21.15.27:0/1714556388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7134105300 0x7f713419cc50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f713406ad30 tx=0x7f712000ee50 comp rx=0 tx=0).stop 2024-09-21T00:11:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.142+0000 7f713a000640 1 -- 172.21.15.27:0/1714556388 shutdown_connections 2024-09-21T00:11:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.142+0000 7f713a000640 1 --2- 172.21.15.27:0/1714556388 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f711803fb70 0x7f7118042080 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.142+0000 7f713a000640 1 --2- 172.21.15.27:0/1714556388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713410de10 0x7f71341a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.142+0000 7f713a000640 1 --2- 172.21.15.27:0/1714556388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7134105300 0x7f713419cc50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.142+0000 7f713a000640 1 --2- 172.21.15.27:0/1714556388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71341049b0 0x7f713419c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.142+0000 7f713a000640 1 -- 172.21.15.27:0/1714556388 >> 172.21.15.27:0/1714556388 conn(0x7f7134079730 msgr2=0x7f71341036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.142+0000 7f713a000640 1 -- 172.21.15.27:0/1714556388 shutdown_connections 2024-09-21T00:11:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.142+0000 7f713a000640 1 -- 172.21.15.27:0/1714556388 wait complete. 2024-09-21T00:11:39.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:39 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:39.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:39 smithi089 ceph-mon[34481]: Standby manager daemon z started 2024-09-21T00:11:39.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:39 smithi089 ceph-mon[34481]: from='mgr.? 172.21.15.98:0/1214986643' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T00:11:39.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:39 smithi089 ceph-mon[34481]: from='mgr.? 172.21.15.98:0/1214986643' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T00:11:39.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:39 smithi089 ceph-mon[34481]: from='mgr.? 172.21.15.98:0/1214986643' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T00:11:39.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:39 smithi089 ceph-mon[34481]: from='mgr.? 172.21.15.98:0/1214986643' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T00:11:39.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:39 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:39.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:39 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:39.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:39 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:39.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:39 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:39.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:39 smithi098 ceph-mon[34725]: Standby manager daemon z started 2024-09-21T00:11:39.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:39 smithi098 ceph-mon[34725]: from='mgr.? 172.21.15.98:0/1214986643' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T00:11:39.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:39 smithi098 ceph-mon[34725]: from='mgr.? 172.21.15.98:0/1214986643' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T00:11:39.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:39 smithi098 ceph-mon[34725]: from='mgr.? 172.21.15.98:0/1214986643' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T00:11:39.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:39 smithi098 ceph-mon[34725]: from='mgr.? 172.21.15.98:0/1214986643' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T00:11:39.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:39 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:39.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:39 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:39.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:39 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:40.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.896+0000 7f6908c00640 1 -- 172.21.15.27:0/2111644651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6904076080 msgr2=0x7f690410fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:40.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.896+0000 7f6908c00640 1 --2- 172.21.15.27:0/2111644651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6904076080 0x7f690410fc30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f68f0009860 tx=0x7f68f002a340 comp rx=0 tx=0).stop 2024-09-21T00:11:40.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.896+0000 7f6908c00640 1 -- 172.21.15.27:0/2111644651 shutdown_connections 2024-09-21T00:11:40.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.896+0000 7f6908c00640 1 --2- 172.21.15.27:0/2111644651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6904110170 0x7f6904112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.896+0000 7f6908c00640 1 --2- 172.21.15.27:0/2111644651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6904076080 0x7f690410fc30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.896+0000 7f6908c00640 1 --2- 172.21.15.27:0/2111644651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6904075730 0x7f6904075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.896+0000 7f6908c00640 1 -- 172.21.15.27:0/2111644651 >> 172.21.15.27:0/2111644651 conn(0x7f69040ff5e0 msgr2=0x7f6904101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:40.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.896+0000 7f6908c00640 1 -- 172.21.15.27:0/2111644651 shutdown_connections 2024-09-21T00:11:40.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.896+0000 7f6908c00640 1 -- 172.21.15.27:0/2111644651 wait complete. 2024-09-21T00:11:40.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.898+0000 7f6908c00640 1 Processor -- start 2024-09-21T00:11:40.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.898+0000 7f6908c00640 1 -- start start 2024-09-21T00:11:40.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.898+0000 7f6908c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6904075730 0x7f690419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:40.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.898+0000 7f6908c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6904076080 0x7f690419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:40.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.899+0000 7f6908c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6904110170 0x7f69041a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:40.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.899+0000 7f6908c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6904115c70 con 0x7f6904075730 2024-09-21T00:11:40.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.899+0000 7f6908c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6904115af0 con 0x7f6904110170 2024-09-21T00:11:40.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.899+0000 7f6908c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6904115df0 con 0x7f6904076080 2024-09-21T00:11:40.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.899+0000 7f6901e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6904075730 0x7f690419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:40.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.899+0000 7f6902800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6904110170 0x7f69041a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:40.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.899+0000 7f6902800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6904110170 0x7f69041a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58094/0 (socket says 172.21.15.27:58094) 2024-09-21T00:11:40.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.899+0000 7f6902800640 1 -- 172.21.15.27:0/3416724317 learned_addr learned my addr 172.21.15.27:0/3416724317 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:40.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.899+0000 7f6901400640 1 --2- 172.21.15.27:0/3416724317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6904076080 0x7f690419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:40.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.899+0000 7f6902800640 1 -- 172.21.15.27:0/3416724317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6904076080 msgr2=0x7f690419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:40.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.899+0000 7f6902800640 1 --2- 172.21.15.27:0/3416724317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6904076080 0x7f690419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.899+0000 7f6902800640 1 -- 172.21.15.27:0/3416724317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6904075730 msgr2=0x7f690419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:40.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.899+0000 7f6902800640 1 --2- 172.21.15.27:0/3416724317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6904075730 0x7f690419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.900+0000 7f6902800640 1 -- 172.21.15.27:0/3416724317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69041a25b0 con 0x7f6904110170 2024-09-21T00:11:40.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.900+0000 7f6901e00640 1 --2- 172.21.15.27:0/3416724317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6904075730 0x7f690419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:11:40.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.900+0000 7f6902800640 1 --2- 172.21.15.27:0/3416724317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6904110170 0x7f69041a1e80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00eb10 tx=0x7f68fc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:40.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.900+0000 7f68faa00640 1 -- 172.21.15.27:0/3416724317 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc00ce20 con 0x7f6904110170 2024-09-21T00:11:40.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.900+0000 7f68faa00640 1 -- 172.21.15.27:0/3416724317 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68fc005020 con 0x7f6904110170 2024-09-21T00:11:40.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.900+0000 7f68faa00640 1 -- 172.21.15.27:0/3416724317 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc010690 con 0x7f6904110170 2024-09-21T00:11:40.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.900+0000 7f6908c00640 1 -- 172.21.15.27:0/3416724317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69041a28a0 con 0x7f6904110170 2024-09-21T00:11:40.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.900+0000 7f6908c00640 1 -- 172.21.15.27:0/3416724317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69041a9000 con 0x7f6904110170 2024-09-21T00:11:40.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.902+0000 7f68faa00640 1 -- 172.21.15.27:0/3416724317 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f68fc010830 con 0x7f6904110170 2024-09-21T00:11:40.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.902+0000 7f6908c00640 1 -- 172.21.15.27:0/3416724317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6904075b40 con 0x7f6904110170 2024-09-21T00:11:40.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.902+0000 7f68faa00640 1 --2- 172.21.15.27:0/3416724317 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f68e803fb70 0x7f68e8042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:40.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.902+0000 7f68faa00640 1 -- 172.21.15.27:0/3416724317 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f68fc054bd0 con 0x7f6904110170 2024-09-21T00:11:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.903+0000 7f6901e00640 1 --2- 172.21.15.27:0/3416724317 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f68e803fb70 0x7f68e8042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.904+0000 7f6901e00640 1 --2- 172.21.15.27:0/3416724317 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f68e803fb70 0x7f68e8042080 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005fd0 tx=0x7f68f4009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:38.907+0000 7f68faa00640 1 -- 172.21.15.27:0/3416724317 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f68fc016330 con 0x7f6904110170 2024-09-21T00:11:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.235+0000 7f6908c00640 1 -- 172.21.15.27:0/3416724317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f69041a92a0 con 0x7f6904110170 2024-09-21T00:11:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.243+0000 7f68faa00640 1 -- 172.21.15.27:0/3416724317 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v21)=0 v21) ==== 116+0+0 (secure 0 0 0) 0x7f68fc054e20 con 0x7f6904110170 2024-09-21T00:11:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.245+0000 7f6908c00640 1 -- 172.21.15.27:0/3416724317 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f68e803fb70 msgr2=0x7f68e8042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.245+0000 7f6908c00640 1 --2- 172.21.15.27:0/3416724317 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f68e803fb70 0x7f68e8042080 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005fd0 tx=0x7f68f4009260 comp rx=0 tx=0).stop 2024-09-21T00:11:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.245+0000 7f6908c00640 1 -- 172.21.15.27:0/3416724317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6904110170 msgr2=0x7f69041a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.245+0000 7f6908c00640 1 --2- 172.21.15.27:0/3416724317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6904110170 0x7f69041a1e80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00eb10 tx=0x7f68fc00ef10 comp rx=0 tx=0).stop 2024-09-21T00:11:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.246+0000 7f6908c00640 1 -- 172.21.15.27:0/3416724317 shutdown_connections 2024-09-21T00:11:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.246+0000 7f6908c00640 1 --2- 172.21.15.27:0/3416724317 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f68e803fb70 0x7f68e8042080 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.246+0000 7f6908c00640 1 --2- 172.21.15.27:0/3416724317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6904110170 0x7f69041a1e80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.246+0000 7f6908c00640 1 --2- 172.21.15.27:0/3416724317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6904076080 0x7f690419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.246+0000 7f6908c00640 1 --2- 172.21.15.27:0/3416724317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6904075730 0x7f690419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.246+0000 7f6908c00640 1 -- 172.21.15.27:0/3416724317 >> 172.21.15.27:0/3416724317 conn(0x7f69040ff5e0 msgr2=0x7f69041021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:40.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.246+0000 7f6908c00640 1 -- 172.21.15.27:0/3416724317 shutdown_connections 2024-09-21T00:11:40.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:39.246+0000 7f6908c00640 1 -- 172.21.15.27:0/3416724317 wait complete. 2024-09-21T00:11:40.443 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:40.449 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:40.452 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:40.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:40 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T00:11:40.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:40 smithi027 ceph-mon[30745]: mgrmap e18: x(active, since 15s), standbys: z 2024-09-21T00:11:40.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:40 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:40.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:40 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:40.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:40 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T00:11:40.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:40 smithi089 ceph-mon[34481]: mgrmap e18: x(active, since 15s), standbys: z 2024-09-21T00:11:40.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:40 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:40.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:40 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:40.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:40 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T00:11:40.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:40 smithi098 ceph-mon[34725]: mgrmap e18: x(active, since 15s), standbys: z 2024-09-21T00:11:40.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:40 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:40.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:40 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:11:40.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.540+0000 7fb917400640 1 -- 172.21.15.27:0/1012960965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9101049b0 msgr2=0x7fb910104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:40.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.540+0000 7fb917400640 1 --2- 172.21.15.27:0/1012960965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9101049b0 0x7fb910104dc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb904009900 tx=0x7fb90402a400 comp rx=0 tx=0).stop 2024-09-21T00:11:40.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.541+0000 7fb917400640 1 -- 172.21.15.27:0/1012960965 shutdown_connections 2024-09-21T00:11:40.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.541+0000 7fb917400640 1 --2- 172.21.15.27:0/1012960965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb91010de10 0x7fb910110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.541+0000 7fb917400640 1 --2- 172.21.15.27:0/1012960965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb910105300 0x7fb91010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.541+0000 7fb917400640 1 --2- 172.21.15.27:0/1012960965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9101049b0 0x7fb910104dc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.541+0000 7fb917400640 1 -- 172.21.15.27:0/1012960965 >> 172.21.15.27:0/1012960965 conn(0x7fb910079760 msgr2=0x7fb910079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:40.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.541+0000 7fb917400640 1 -- 172.21.15.27:0/1012960965 shutdown_connections 2024-09-21T00:11:40.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.541+0000 7fb917400640 1 -- 172.21.15.27:0/1012960965 wait complete. 2024-09-21T00:11:40.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.542+0000 7fb917400640 1 Processor -- start 2024-09-21T00:11:40.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.542+0000 7fb917400640 1 -- start start 2024-09-21T00:11:40.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb917400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9101049b0 0x7fb91019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb917400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb910105300 0x7fb91019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb917400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb91010de10 0x7fb9101a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb917400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb910113ac0 con 0x7fb91010de10 2024-09-21T00:11:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb917400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb910113940 con 0x7fb9101049b0 2024-09-21T00:11:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb917400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb910113c40 con 0x7fb910105300 2024-09-21T00:11:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb914a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9101049b0 0x7fb91019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb90fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb910105300 0x7fb91019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb914a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9101049b0 0x7fb91019c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57994/0 (socket says 172.21.15.27:57994) 2024-09-21T00:11:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb914a00640 1 -- 172.21.15.27:0/2282776863 learned_addr learned my addr 172.21.15.27:0/2282776863 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb915400640 1 --2- 172.21.15.27:0/2282776863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb91010de10 0x7fb9101a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb914a00640 1 -- 172.21.15.27:0/2282776863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb910105300 msgr2=0x7fb91019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb914a00640 1 --2- 172.21.15.27:0/2282776863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb910105300 0x7fb91019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb914a00640 1 -- 172.21.15.27:0/2282776863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb91010de10 msgr2=0x7fb9101a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb914a00640 1 --2- 172.21.15.27:0/2282776863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb91010de10 0x7fb9101a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.543+0000 7fb914a00640 1 -- 172.21.15.27:0/2282776863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9101a2660 con 0x7fb9101049b0 2024-09-21T00:11:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.544+0000 7fb915400640 1 --2- 172.21.15.27:0/2282776863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb91010de10 0x7fb9101a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:11:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.544+0000 7fb914a00640 1 --2- 172.21.15.27:0/2282776863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9101049b0 0x7fb91019c6f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb904005ab0 tx=0x7fb904003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.544+0000 7fb90d600640 1 -- 172.21.15.27:0/2282776863 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90403f070 con 0x7fb9101049b0 2024-09-21T00:11:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.544+0000 7fb90d600640 1 -- 172.21.15.27:0/2282776863 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb904031b60 con 0x7fb9101049b0 2024-09-21T00:11:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.544+0000 7fb917400640 1 -- 172.21.15.27:0/2282776863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9101a28f0 con 0x7fb9101049b0 2024-09-21T00:11:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.544+0000 7fb90d600640 1 -- 172.21.15.27:0/2282776863 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904035030 con 0x7fb9101049b0 2024-09-21T00:11:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.544+0000 7fb917400640 1 -- 172.21.15.27:0/2282776863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9101a90b0 con 0x7fb9101049b0 2024-09-21T00:11:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.546+0000 7fb917400640 1 -- 172.21.15.27:0/2282776863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb91006c2c0 con 0x7fb9101049b0 2024-09-21T00:11:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.546+0000 7fb90d600640 1 -- 172.21.15.27:0/2282776863 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7fb904042070 con 0x7fb9101049b0 2024-09-21T00:11:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.547+0000 7fb90d600640 1 --2- 172.21.15.27:0/2282776863 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb8f803fb70 0x7fb8f8042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.547+0000 7fb90d600640 1 -- 172.21.15.27:0/2282776863 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb904074d60 con 0x7fb9101049b0 2024-09-21T00:11:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.547+0000 7fb90fe00640 1 --2- 172.21.15.27:0/2282776863 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb8f803fb70 0x7fb8f8042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.548+0000 7fb90fe00640 1 --2- 172.21.15.27:0/2282776863 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb8f803fb70 0x7fb8f8042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0095f0 tx=0x7fb8fc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.552+0000 7fb90d600640 1 -- 172.21.15.27:0/2282776863 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb904045d70 con 0x7fb9101049b0 2024-09-21T00:11:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.762+0000 7fb917400640 1 -- 172.21.15.27:0/2282776863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fb9101a9360 con 0x7fb9101049b0 2024-09-21T00:11:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.769+0000 7fb90d600640 1 -- 172.21.15.27:0/2282776863 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v16)=0 v16) ==== 115+0+0 (secure 0 0 0) 0x7fb904049210 con 0x7fb9101049b0 2024-09-21T00:11:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.772+0000 7fb917400640 1 -- 172.21.15.27:0/2282776863 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb8f803fb70 msgr2=0x7fb8f8042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.772+0000 7fb917400640 1 --2- 172.21.15.27:0/2282776863 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb8f803fb70 0x7fb8f8042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0095f0 tx=0x7fb8fc005f90 comp rx=0 tx=0).stop 2024-09-21T00:11:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.772+0000 7fb917400640 1 -- 172.21.15.27:0/2282776863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9101049b0 msgr2=0x7fb91019c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.772+0000 7fb917400640 1 --2- 172.21.15.27:0/2282776863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9101049b0 0x7fb91019c6f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb904005ab0 tx=0x7fb904003c10 comp rx=0 tx=0).stop 2024-09-21T00:11:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.772+0000 7fb917400640 1 -- 172.21.15.27:0/2282776863 shutdown_connections 2024-09-21T00:11:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.772+0000 7fb917400640 1 --2- 172.21.15.27:0/2282776863 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb8f803fb70 0x7fb8f8042080 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.772+0000 7fb917400640 1 --2- 172.21.15.27:0/2282776863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb91010de10 0x7fb9101a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.772+0000 7fb917400640 1 --2- 172.21.15.27:0/2282776863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb910105300 0x7fb91019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.772+0000 7fb917400640 1 --2- 172.21.15.27:0/2282776863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9101049b0 0x7fb91019c6f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:40.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.773+0000 7fb917400640 1 -- 172.21.15.27:0/2282776863 >> 172.21.15.27:0/2282776863 conn(0x7fb910079760 msgr2=0x7fb91010cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:40.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.773+0000 7fb917400640 1 -- 172.21.15.27:0/2282776863 shutdown_connections 2024-09-21T00:11:40.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:35.773+0000 7fb917400640 1 -- 172.21.15.27:0/2282776863 wait complete. 2024-09-21T00:11:41.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.980+0000 7f253ce00640 1 -- 172.21.15.27:0/3530481703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2538077940 msgr2=0x7f2538077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:41.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.980+0000 7f253ce00640 1 --2- 172.21.15.27:0/3530481703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2538077940 0x7f2538077d50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2528009900 tx=0x7f252802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:11:41.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.980+0000 7f253ce00640 1 -- 172.21.15.27:0/3530481703 shutdown_connections 2024-09-21T00:11:41.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.980+0000 7f253ce00640 1 --2- 172.21.15.27:0/3530481703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2538114740 0x7f2538116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:41.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.980+0000 7f253ce00640 1 --2- 172.21.15.27:0/3530481703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2538078290 0x7f2538114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:41.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.980+0000 7f253ce00640 1 --2- 172.21.15.27:0/3530481703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2538077940 0x7f2538077d50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:41.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.980+0000 7f253ce00640 1 -- 172.21.15.27:0/3530481703 >> 172.21.15.27:0/3530481703 conn(0x7f2538101810 msgr2=0x7f2538103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:41.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.980+0000 7f253ce00640 1 -- 172.21.15.27:0/3530481703 shutdown_connections 2024-09-21T00:11:41.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.980+0000 7f253ce00640 1 -- 172.21.15.27:0/3530481703 wait complete. 2024-09-21T00:11:41.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.981+0000 7f253ce00640 1 Processor -- start 2024-09-21T00:11:41.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.981+0000 7f253ce00640 1 -- start start 2024-09-21T00:11:41.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.982+0000 7f253ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2538077940 0x7f25381a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:41.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.982+0000 7f253ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2538078290 0x7f25381a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:41.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.982+0000 7f253ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2538114740 0x7f25381a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:41.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.982+0000 7f253ce00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f253811a120 con 0x7f2538078290 2024-09-21T00:11:41.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.982+0000 7f253ce00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2538119fa0 con 0x7f2538077940 2024-09-21T00:11:41.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.982+0000 7f253ce00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f253811a2a0 con 0x7f2538114740 2024-09-21T00:11:41.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.982+0000 7f2535400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2538078290 0x7f25381a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:41.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.982+0000 7f2535e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2538077940 0x7f25381a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:41.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.982+0000 7f2535e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2538077940 0x7f25381a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58154/0 (socket says 172.21.15.27:58154) 2024-09-21T00:11:41.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.982+0000 7f2535e00640 1 -- 172.21.15.27:0/1677358447 learned_addr learned my addr 172.21.15.27:0/1677358447 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:41.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.982+0000 7f2535400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2538078290 0x7f25381a1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47260/0 (socket says 172.21.15.27:47260) 2024-09-21T00:11:41.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.982+0000 7f2536800640 1 --2- 172.21.15.27:0/1677358447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2538114740 0x7f25381a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:41.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.983+0000 7f2535400640 1 -- 172.21.15.27:0/1677358447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2538114740 msgr2=0x7f25381a6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.983+0000 7f2535400640 1 --2- 172.21.15.27:0/1677358447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2538114740 0x7f25381a6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.983+0000 7f2535400640 1 -- 172.21.15.27:0/1677358447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2538077940 msgr2=0x7f25381a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.983+0000 7f2535400640 1 --2- 172.21.15.27:0/1677358447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2538077940 0x7f25381a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.983+0000 7f2535400640 1 -- 172.21.15.27:0/1677358447 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25381a6a60 con 0x7f2538078290 2024-09-21T00:11:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.983+0000 7f2536800640 1 --2- 172.21.15.27:0/1677358447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2538114740 0x7f25381a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:11:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.983+0000 7f2535400640 1 --2- 172.21.15.27:0/1677358447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2538078290 0x7f25381a1090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f253806caa0 tx=0x7f252400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.983+0000 7f252ea00640 1 -- 172.21.15.27:0/1677358447 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252400cc40 con 0x7f2538078290 2024-09-21T00:11:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.983+0000 7f253ce00640 1 -- 172.21.15.27:0/1677358447 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25381a6d50 con 0x7f2538078290 2024-09-21T00:11:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.983+0000 7f252ea00640 1 -- 172.21.15.27:0/1677358447 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f252400cde0 con 0x7f2538078290 2024-09-21T00:11:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.983+0000 7f252ea00640 1 -- 172.21.15.27:0/1677358447 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252401c070 con 0x7f2538078290 2024-09-21T00:11:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.983+0000 7f253ce00640 1 -- 172.21.15.27:0/1677358447 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25381ad4b0 con 0x7f2538078290 2024-09-21T00:11:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.984+0000 7f253ce00640 1 -- 172.21.15.27:0/1677358447 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2538077d50 con 0x7f2538078290 2024-09-21T00:11:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.986+0000 7f252ea00640 1 -- 172.21.15.27:0/1677358447 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 103194+0+0 (secure 0 0 0) 0x7f25240049a0 con 0x7f2538078290 2024-09-21T00:11:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.987+0000 7f252ea00640 1 --2- 172.21.15.27:0/1677358447 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f251c07b020 0x7f251c07d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.987+0000 7f252ea00640 1 -- 172.21.15.27:0/1677358447 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f252409ce30 con 0x7f2538078290 2024-09-21T00:11:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.988+0000 7f2535e00640 1 --2- 172.21.15.27:0/1677358447 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f251c07b020 0x7f251c07d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.988+0000 7f2535e00640 1 --2- 172.21.15.27:0/1677358447 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f251c07b020 0x7f251c07d530 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f25280096c0 tx=0x7f2528005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:40.992+0000 7f252ea00640 1 -- 172.21.15.27:0/1677358447 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2524065340 con 0x7f2538078290 2024-09-21T00:11:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:41.207+0000 7f253ce00640 1 -- 172.21.15.27:0/1677358447 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f253802e640 con 0x7f2538078290 2024-09-21T00:11:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:41.212+0000 7f252ea00640 1 -- 172.21.15.27:0/1677358447 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v24)=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f2524069720 con 0x7f2538078290 2024-09-21T00:11:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:41.215+0000 7f253ce00640 1 -- 172.21.15.27:0/1677358447 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f251c07b020 msgr2=0x7f251c07d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:41.215+0000 7f253ce00640 1 --2- 172.21.15.27:0/1677358447 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f251c07b020 0x7f251c07d530 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f25280096c0 tx=0x7f2528005a20 comp rx=0 tx=0).stop 2024-09-21T00:11:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:41.215+0000 7f253ce00640 1 -- 172.21.15.27:0/1677358447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2538078290 msgr2=0x7f25381a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:41.215+0000 7f253ce00640 1 --2- 172.21.15.27:0/1677358447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2538078290 0x7f25381a1090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f253806caa0 tx=0x7f252400ee50 comp rx=0 tx=0).stop 2024-09-21T00:11:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:41.215+0000 7f253ce00640 1 -- 172.21.15.27:0/1677358447 shutdown_connections 2024-09-21T00:11:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:41.215+0000 7f253ce00640 1 --2- 172.21.15.27:0/1677358447 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f251c07b020 0x7f251c07d530 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:41.215+0000 7f253ce00640 1 --2- 172.21.15.27:0/1677358447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2538114740 0x7f25381a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:41.215+0000 7f253ce00640 1 --2- 172.21.15.27:0/1677358447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2538078290 0x7f25381a1090 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:41.215+0000 7f253ce00640 1 --2- 172.21.15.27:0/1677358447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2538077940 0x7f25381a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:41.215+0000 7f253ce00640 1 -- 172.21.15.27:0/1677358447 >> 172.21.15.27:0/1677358447 conn(0x7f2538101810 msgr2=0x7f25381167b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:41.215+0000 7f253ce00640 1 -- 172.21.15.27:0/1677358447 shutdown_connections 2024-09-21T00:11:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:41.215+0000 7f253ce00640 1 -- 172.21.15.27:0/1677358447 wait complete. 2024-09-21T00:11:41.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:41 smithi027 ceph-mon[30745]: Standby manager daemon y started 2024-09-21T00:11:41.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:41 smithi027 ceph-mon[30745]: from='mgr.? 172.21.15.89:0/4019566418' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T00:11:41.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:41 smithi027 ceph-mon[30745]: from='mgr.? 172.21.15.89:0/4019566418' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T00:11:41.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:41 smithi027 ceph-mon[30745]: from='mgr.? 172.21.15.89:0/4019566418' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T00:11:41.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:41 smithi027 ceph-mon[30745]: from='mgr.? 172.21.15.89:0/4019566418' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T00:11:41.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:41 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1677358447' entity='client.admin' 2024-09-21T00:11:41.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:41 smithi089 ceph-mon[34481]: Standby manager daemon y started 2024-09-21T00:11:41.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:41 smithi089 ceph-mon[34481]: from='mgr.? 172.21.15.89:0/4019566418' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T00:11:41.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:41 smithi089 ceph-mon[34481]: from='mgr.? 172.21.15.89:0/4019566418' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T00:11:41.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:41 smithi089 ceph-mon[34481]: from='mgr.? 172.21.15.89:0/4019566418' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T00:11:41.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:41 smithi089 ceph-mon[34481]: from='mgr.? 172.21.15.89:0/4019566418' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T00:11:41.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:41 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1677358447' entity='client.admin' 2024-09-21T00:11:41.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:41 smithi098 ceph-mon[34725]: Standby manager daemon y started 2024-09-21T00:11:41.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:41 smithi098 ceph-mon[34725]: from='mgr.? 172.21.15.89:0/4019566418' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T00:11:41.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:41 smithi098 ceph-mon[34725]: from='mgr.? 172.21.15.89:0/4019566418' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T00:11:41.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:41 smithi098 ceph-mon[34725]: from='mgr.? 172.21.15.89:0/4019566418' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T00:11:41.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:41 smithi098 ceph-mon[34725]: from='mgr.? 172.21.15.89:0/4019566418' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T00:11:41.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:41 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1677358447' entity='client.admin' 2024-09-21T00:11:42.420 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:42 smithi027 ceph-mon[30745]: mgrmap e19: x(active, since 17s), standbys: z, y 2024-09-21T00:11:42.420 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:42 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T00:11:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.708+0000 7fc777e00640 1 -- 172.21.15.27:0/4130125213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7781092f0 msgr2=0x7fc778109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.708+0000 7fc777e00640 1 --2- 172.21.15.27:0/4130125213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7781092f0 0x7fc778109700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc760009900 tx=0x7fc76002a410 comp rx=0 tx=0).stop 2024-09-21T00:11:42.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.708+0000 7fc777e00640 1 -- 172.21.15.27:0/4130125213 shutdown_connections 2024-09-21T00:11:42.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.708+0000 7fc777e00640 1 --2- 172.21.15.27:0/4130125213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc778109c40 0x7fc778110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:42.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.708+0000 7fc777e00640 1 --2- 172.21.15.27:0/4130125213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7781092f0 0x7fc778109700 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:42.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.708+0000 7fc777e00640 1 --2- 172.21.15.27:0/4130125213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7781039b0 0x7fc778103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:42.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.708+0000 7fc777e00640 1 -- 172.21.15.27:0/4130125213 >> 172.21.15.27:0/4130125213 conn(0x7fc7780ff440 msgr2=0x7fc778101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:42.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.710+0000 7fc777e00640 1 -- 172.21.15.27:0/4130125213 shutdown_connections 2024-09-21T00:11:42.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.710+0000 7fc777e00640 1 -- 172.21.15.27:0/4130125213 wait complete. 2024-09-21T00:11:42.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.711+0000 7fc777e00640 1 Processor -- start 2024-09-21T00:11:42.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.711+0000 7fc777e00640 1 -- start start 2024-09-21T00:11:42.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.711+0000 7fc777e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7781039b0 0x7fc77819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:42.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.711+0000 7fc777e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7781092f0 0x7fc77819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:42.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.711+0000 7fc777e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc778109c40 0x7fc7781a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:42.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.711+0000 7fc777e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc778113950 con 0x7fc778109c40 2024-09-21T00:11:42.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.711+0000 7fc777e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc7781137d0 con 0x7fc7781092f0 2024-09-21T00:11:42.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.711+0000 7fc777e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc778113ad0 con 0x7fc7781039b0 2024-09-21T00:11:42.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.712+0000 7fc776a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7781039b0 0x7fc77819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:42.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.712+0000 7fc777400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc778109c40 0x7fc7781a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:42.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.712+0000 7fc776a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7781039b0 0x7fc77819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39080/0 (socket says 172.21.15.27:39080) 2024-09-21T00:11:42.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.712+0000 7fc776a00640 1 -- 172.21.15.27:0/2689237824 learned_addr learned my addr 172.21.15.27:0/2689237824 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:42.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.712+0000 7fc76e000640 1 --2- 172.21.15.27:0/2689237824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7781092f0 0x7fc77819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:42.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.712+0000 7fc776a00640 1 -- 172.21.15.27:0/2689237824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7781092f0 msgr2=0x7fc77819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:42.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.712+0000 7fc776a00640 1 --2- 172.21.15.27:0/2689237824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7781092f0 0x7fc77819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:42.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.712+0000 7fc776a00640 1 -- 172.21.15.27:0/2689237824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc778109c40 msgr2=0x7fc7781a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:42.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.712+0000 7fc776a00640 1 --2- 172.21.15.27:0/2689237824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc778109c40 0x7fc7781a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:42.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.712+0000 7fc776a00640 1 -- 172.21.15.27:0/2689237824 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7781a24f0 con 0x7fc7781039b0 2024-09-21T00:11:42.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.713+0000 7fc776a00640 1 --2- 172.21.15.27:0/2689237824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7781039b0 0x7fc77819c580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc76800e9f0 tx=0x7fc76800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:42.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.713+0000 7fc76fe00640 1 -- 172.21.15.27:0/2689237824 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76800cd80 con 0x7fc7781039b0 2024-09-21T00:11:42.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.713+0000 7fc76fe00640 1 -- 172.21.15.27:0/2689237824 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7680108e0 con 0x7fc7781039b0 2024-09-21T00:11:42.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.713+0000 7fc777e00640 1 -- 172.21.15.27:0/2689237824 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7781a27e0 con 0x7fc7781039b0 2024-09-21T00:11:42.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.713+0000 7fc76fe00640 1 -- 172.21.15.27:0/2689237824 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768018980 con 0x7fc7781039b0 2024-09-21T00:11:42.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.713+0000 7fc777e00640 1 -- 172.21.15.27:0/2689237824 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7781a8f40 con 0x7fc7781039b0 2024-09-21T00:11:42.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.715+0000 7fc777e00640 1 -- 172.21.15.27:0/2689237824 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc778103dc0 con 0x7fc7781039b0 2024-09-21T00:11:42.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.718+0000 7fc76fe00640 1 -- 172.21.15.27:0/2689237824 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fc768010430 con 0x7fc7781039b0 2024-09-21T00:11:42.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.719+0000 7fc76fe00640 1 --2- 172.21.15.27:0/2689237824 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc75c0b6550 0x7fc75c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:42.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.719+0000 7fc76fe00640 1 -- 172.21.15.27:0/2689237824 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc7680e5f30 con 0x7fc7781039b0 2024-09-21T00:11:42.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.719+0000 7fc76e000640 1 --2- 172.21.15.27:0/2689237824 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc75c0b6550 0x7fc75c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:42.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.720+0000 7fc76e000640 1 --2- 172.21.15.27:0/2689237824 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc75c0b6550 0x7fc75c0b8a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc76002aa00 tx=0x7fc760005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:42.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.721+0000 7fc76fe00640 1 -- 172.21.15.27:0/2689237824 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc7680b2700 con 0x7fc7781039b0 2024-09-21T00:11:42.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:42 smithi089 ceph-mon[34481]: mgrmap e19: x(active, since 17s), standbys: z, y 2024-09-21T00:11:42.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:42 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T00:11:42.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:42 smithi098 ceph-mon[34725]: mgrmap e19: x(active, since 17s), standbys: z, y 2024-09-21T00:11:42.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:42 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T00:11:42.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.940+0000 7fc777e00640 1 -- 172.21.15.27:0/2689237824 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fc77819d6b0 con 0x7fc7781039b0 2024-09-21T00:11:42.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.942+0000 7fc76fe00640 1 -- 172.21.15.27:0/2689237824 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v25)=0 v25) ==== 106+0+0 (secure 0 0 0) 0x7fc7680b7640 con 0x7fc7781039b0 2024-09-21T00:11:42.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.944+0000 7fc777e00640 1 -- 172.21.15.27:0/2689237824 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc75c0b6550 msgr2=0x7fc75c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:42.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.944+0000 7fc777e00640 1 --2- 172.21.15.27:0/2689237824 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc75c0b6550 0x7fc75c0b8a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc76002aa00 tx=0x7fc760005a20 comp rx=0 tx=0).stop 2024-09-21T00:11:42.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.944+0000 7fc777e00640 1 -- 172.21.15.27:0/2689237824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7781039b0 msgr2=0x7fc77819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:42.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.944+0000 7fc777e00640 1 --2- 172.21.15.27:0/2689237824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7781039b0 0x7fc77819c580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc76800e9f0 tx=0x7fc76800eeb0 comp rx=0 tx=0).stop 2024-09-21T00:11:42.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.944+0000 7fc777e00640 1 -- 172.21.15.27:0/2689237824 shutdown_connections 2024-09-21T00:11:42.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.944+0000 7fc777e00640 1 --2- 172.21.15.27:0/2689237824 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc75c0b6550 0x7fc75c0b8a60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:42.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.944+0000 7fc777e00640 1 --2- 172.21.15.27:0/2689237824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc778109c40 0x7fc7781a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:42.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.944+0000 7fc777e00640 1 --2- 172.21.15.27:0/2689237824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7781092f0 0x7fc77819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:42.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.944+0000 7fc777e00640 1 --2- 172.21.15.27:0/2689237824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7781039b0 0x7fc77819c580 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:42.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.945+0000 7fc777e00640 1 -- 172.21.15.27:0/2689237824 >> 172.21.15.27:0/2689237824 conn(0x7fc7780ff440 msgr2=0x7fc77810e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:42.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.945+0000 7fc777e00640 1 -- 172.21.15.27:0/2689237824 shutdown_connections 2024-09-21T00:11:42.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.945+0000 7fc777e00640 1 -- 172.21.15.27:0/2689237824 wait complete. 2024-09-21T00:11:43.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.716+0000 7fde7f800640 1 -- 172.21.15.27:0/1488712948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde78076100 msgr2=0x7fde7810fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:43.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.716+0000 7fde7f800640 1 --2- 172.21.15.27:0/1488712948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde78076100 0x7fde7810fb90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fde64009900 tx=0x7fde6402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:11:43.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.717+0000 7fde7f800640 1 -- 172.21.15.27:0/1488712948 shutdown_connections 2024-09-21T00:11:43.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.717+0000 7fde7f800640 1 --2- 172.21.15.27:0/1488712948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde781100d0 0x7fde78112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:43.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.717+0000 7fde7f800640 1 --2- 172.21.15.27:0/1488712948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde78076100 0x7fde7810fb90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:43.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.717+0000 7fde7f800640 1 --2- 172.21.15.27:0/1488712948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde780757b0 0x7fde78075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:43.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.717+0000 7fde7f800640 1 -- 172.21.15.27:0/1488712948 >> 172.21.15.27:0/1488712948 conn(0x7fde780ff5e0 msgr2=0x7fde78101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:43.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.717+0000 7fde7f800640 1 -- 172.21.15.27:0/1488712948 shutdown_connections 2024-09-21T00:11:43.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.717+0000 7fde7f800640 1 -- 172.21.15.27:0/1488712948 wait complete. 2024-09-21T00:11:43.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.718+0000 7fde7f800640 1 Processor -- start 2024-09-21T00:11:43.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.718+0000 7fde7f800640 1 -- start start 2024-09-21T00:11:43.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.719+0000 7fde7f800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde780757b0 0x7fde7819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:43.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.719+0000 7fde7f800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde78076100 0x7fde7819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:43.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.719+0000 7fde7f800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde781100d0 0x7fde781a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:43.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.719+0000 7fde7f800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fde78115c80 con 0x7fde781100d0 2024-09-21T00:11:43.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.719+0000 7fde7f800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fde78115b00 con 0x7fde78076100 2024-09-21T00:11:43.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.719+0000 7fde7f800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fde78115e00 con 0x7fde780757b0 2024-09-21T00:11:43.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.719+0000 7fde7d800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde781100d0 0x7fde781a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:43.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.719+0000 7fde77e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde78076100 0x7fde7819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:43.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.719+0000 7fde7d800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde781100d0 0x7fde781a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43670/0 (socket says 172.21.15.27:43670) 2024-09-21T00:11:43.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.719+0000 7fde7d800640 1 -- 172.21.15.27:0/1459670933 learned_addr learned my addr 172.21.15.27:0/1459670933 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:43.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.719+0000 7fde7ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde780757b0 0x7fde7819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:43.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.719+0000 7fde7d800640 1 -- 172.21.15.27:0/1459670933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde780757b0 msgr2=0x7fde7819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:43.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.719+0000 7fde7d800640 1 --2- 172.21.15.27:0/1459670933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde780757b0 0x7fde7819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:43.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.719+0000 7fde7d800640 1 -- 172.21.15.27:0/1459670933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde78076100 msgr2=0x7fde7819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:43.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.719+0000 7fde7d800640 1 --2- 172.21.15.27:0/1459670933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde78076100 0x7fde7819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:43.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.720+0000 7fde7d800640 1 -- 172.21.15.27:0/1459670933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde781a2640 con 0x7fde781100d0 2024-09-21T00:11:43.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.720+0000 7fde77e00640 1 --2- 172.21.15.27:0/1459670933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde78076100 0x7fde7819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:11:43.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.720+0000 7fde7d800640 1 --2- 172.21.15.27:0/1459670933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde781100d0 0x7fde781a1f10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fde7000eb10 tx=0x7fde7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:43.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.720+0000 7fde75600640 1 -- 172.21.15.27:0/1459670933 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde70004b20 con 0x7fde781100d0 2024-09-21T00:11:43.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.720+0000 7fde75600640 1 -- 172.21.15.27:0/1459670933 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde70004cc0 con 0x7fde781100d0 2024-09-21T00:11:43.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.720+0000 7fde75600640 1 -- 172.21.15.27:0/1459670933 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde70010670 con 0x7fde781100d0 2024-09-21T00:11:43.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.720+0000 7fde7f800640 1 -- 172.21.15.27:0/1459670933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde781a2930 con 0x7fde781100d0 2024-09-21T00:11:43.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.720+0000 7fde7f800640 1 -- 172.21.15.27:0/1459670933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde781a9090 con 0x7fde781100d0 2024-09-21T00:11:43.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.722+0000 7fde7f800640 1 -- 172.21.15.27:0/1459670933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde78075bc0 con 0x7fde781100d0 2024-09-21T00:11:43.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.725+0000 7fde75600640 1 -- 172.21.15.27:0/1459670933 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fde70005000 con 0x7fde781100d0 2024-09-21T00:11:43.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.726+0000 7fde75600640 1 --2- 172.21.15.27:0/1459670933 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fde600b6550 0x7fde600b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:43.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.726+0000 7fde75600640 1 -- 172.21.15.27:0/1459670933 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fde700e5fc0 con 0x7fde781100d0 2024-09-21T00:11:43.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.726+0000 7fde7ce00640 1 --2- 172.21.15.27:0/1459670933 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fde600b6550 0x7fde600b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:43.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.727+0000 7fde7ce00640 1 --2- 172.21.15.27:0/1459670933 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fde600b6550 0x7fde600b8a60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009b60 tx=0x7fde6c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:43.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.729+0000 7fde75600640 1 -- 172.21.15.27:0/1459670933 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fde700ae4e0 con 0x7fde781100d0 2024-09-21T00:11:43.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.940+0000 7fde7f800640 1 -- 172.21.15.27:0/1459670933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fde781a9290 con 0x7fde781100d0 2024-09-21T00:11:43.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.945+0000 7fde75600640 1 -- 172.21.15.27:0/1459670933 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v26)=0 v26) ==== 121+0+0 (secure 0 0 0) 0x7fde700b28c0 con 0x7fde781100d0 2024-09-21T00:11:43.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.947+0000 7fde7f800640 1 -- 172.21.15.27:0/1459670933 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fde600b6550 msgr2=0x7fde600b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.947+0000 7fde7f800640 1 --2- 172.21.15.27:0/1459670933 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fde600b6550 0x7fde600b8a60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009b60 tx=0x7fde6c005f90 comp rx=0 tx=0).stop 2024-09-21T00:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.947+0000 7fde7f800640 1 -- 172.21.15.27:0/1459670933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde781100d0 msgr2=0x7fde781a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.947+0000 7fde7f800640 1 --2- 172.21.15.27:0/1459670933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde781100d0 0x7fde781a1f10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fde7000eb10 tx=0x7fde7000c2d0 comp rx=0 tx=0).stop 2024-09-21T00:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.947+0000 7fde7f800640 1 -- 172.21.15.27:0/1459670933 shutdown_connections 2024-09-21T00:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.947+0000 7fde7f800640 1 --2- 172.21.15.27:0/1459670933 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fde600b6550 0x7fde600b8a60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.947+0000 7fde7f800640 1 --2- 172.21.15.27:0/1459670933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde781100d0 0x7fde781a1f10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.947+0000 7fde7f800640 1 --2- 172.21.15.27:0/1459670933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde78076100 0x7fde7819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.947+0000 7fde7f800640 1 --2- 172.21.15.27:0/1459670933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde780757b0 0x7fde7819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.947+0000 7fde7f800640 1 -- 172.21.15.27:0/1459670933 >> 172.21.15.27:0/1459670933 conn(0x7fde780ff5e0 msgr2=0x7fde78109c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.947+0000 7fde7f800640 1 -- 172.21.15.27:0/1459670933 shutdown_connections 2024-09-21T00:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:43.947+0000 7fde7f800640 1 -- 172.21.15.27:0/1459670933 wait complete. 2024-09-21T00:11:43.976 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:43 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:44.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:43 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:44.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:43 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:45.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.006+0000 7fd788a00640 1 -- 172.21.15.27:0/1034256864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd784109c40 msgr2=0x7fd784110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:45.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.006+0000 7fd788a00640 1 --2- 172.21.15.27:0/1034256864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd784109c40 0x7fd784110470 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009be0 tx=0x7fd77c02a6c0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.006+0000 7fd788a00640 1 -- 172.21.15.27:0/1034256864 shutdown_connections 2024-09-21T00:11:45.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.006+0000 7fd788a00640 1 --2- 172.21.15.27:0/1034256864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd784109c40 0x7fd784110470 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.006+0000 7fd788a00640 1 --2- 172.21.15.27:0/1034256864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7841092f0 0x7fd784109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.006+0000 7fd788a00640 1 --2- 172.21.15.27:0/1034256864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7841039b0 0x7fd784103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.006+0000 7fd788a00640 1 -- 172.21.15.27:0/1034256864 >> 172.21.15.27:0/1034256864 conn(0x7fd7840ff400 msgr2=0x7fd784101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:45.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.006+0000 7fd788a00640 1 -- 172.21.15.27:0/1034256864 shutdown_connections 2024-09-21T00:11:45.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.006+0000 7fd788a00640 1 -- 172.21.15.27:0/1034256864 wait complete. 2024-09-21T00:11:45.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.008+0000 7fd788a00640 1 Processor -- start 2024-09-21T00:11:45.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.008+0000 7fd788a00640 1 -- start start 2024-09-21T00:11:45.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.008+0000 7fd788a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7841039b0 0x7fd78419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:45.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.008+0000 7fd788a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7841092f0 0x7fd78419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:45.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.008+0000 7fd788a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd784109c40 0x7fd7841a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:45.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.008+0000 7fd788a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd784113900 con 0x7fd7841039b0 2024-09-21T00:11:45.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.008+0000 7fd788a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd784113780 con 0x7fd784109c40 2024-09-21T00:11:45.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.008+0000 7fd788a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd784113a80 con 0x7fd7841092f0 2024-09-21T00:11:45.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.009+0000 7fd782a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7841092f0 0x7fd78419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:45.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.009+0000 7fd782a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7841092f0 0x7fd78419ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:34994/0 (socket says 172.21.15.27:34994) 2024-09-21T00:11:45.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.009+0000 7fd782a00640 1 -- 172.21.15.27:0/4116503092 learned_addr learned my addr 172.21.15.27:0/4116503092 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:45.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.009+0000 7fd783e00640 1 --2- 172.21.15.27:0/4116503092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd784109c40 0x7fd7841a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:45.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.009+0000 7fd783400640 1 --2- 172.21.15.27:0/4116503092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7841039b0 0x7fd78419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:45.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.010+0000 7fd782a00640 1 -- 172.21.15.27:0/4116503092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd784109c40 msgr2=0x7fd7841a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:45.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.010+0000 7fd782a00640 1 --2- 172.21.15.27:0/4116503092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd784109c40 0x7fd7841a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.010+0000 7fd782a00640 1 -- 172.21.15.27:0/4116503092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7841039b0 msgr2=0x7fd78419c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:45.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.010+0000 7fd782a00640 1 --2- 172.21.15.27:0/4116503092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7841039b0 0x7fd78419c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.010+0000 7fd782a00640 1 -- 172.21.15.27:0/4116503092 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7841a2400 con 0x7fd7841092f0 2024-09-21T00:11:45.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.010+0000 7fd782a00640 1 --2- 172.21.15.27:0/4116503092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7841092f0 0x7fd78419ca30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd7700058c0 tx=0x7fd77000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:45.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.011+0000 7fd77be00640 1 -- 172.21.15.27:0/4116503092 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77000cd30 con 0x7fd7841092f0 2024-09-21T00:11:45.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.011+0000 7fd77be00640 1 -- 172.21.15.27:0/4116503092 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7700035f0 con 0x7fd7841092f0 2024-09-21T00:11:45.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.011+0000 7fd77be00640 1 -- 172.21.15.27:0/4116503092 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7700044e0 con 0x7fd7841092f0 2024-09-21T00:11:45.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.011+0000 7fd788a00640 1 -- 172.21.15.27:0/4116503092 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7841a26f0 con 0x7fd7841092f0 2024-09-21T00:11:45.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.011+0000 7fd788a00640 1 -- 172.21.15.27:0/4116503092 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7841a8e50 con 0x7fd7841092f0 2024-09-21T00:11:45.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.012+0000 7fd788a00640 1 -- 172.21.15.27:0/4116503092 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd784103dc0 con 0x7fd7841092f0 2024-09-21T00:11:45.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.015+0000 7fd77be00640 1 -- 172.21.15.27:0/4116503092 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fd770004030 con 0x7fd7841092f0 2024-09-21T00:11:45.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.016+0000 7fd77be00640 1 --2- 172.21.15.27:0/4116503092 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd7680b6550 0x7fd7680b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:45.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.017+0000 7fd77be00640 1 -- 172.21.15.27:0/4116503092 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd7700e59a0 con 0x7fd7841092f0 2024-09-21T00:11:45.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.017+0000 7fd783400640 1 --2- 172.21.15.27:0/4116503092 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd7680b6550 0x7fd7680b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:45.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.018+0000 7fd783400640 1 --2- 172.21.15.27:0/4116503092 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd7680b6550 0x7fd7680b8a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd774009bd0 tx=0x7fd774005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:45.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.020+0000 7fd77be00640 1 -- 172.21.15.27:0/4116503092 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd7700add90 con 0x7fd7841092f0 2024-09-21T00:11:45.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:44 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1459670933' entity='client.admin' 2024-09-21T00:11:45.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:44 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1459670933' entity='client.admin' 2024-09-21T00:11:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.229+0000 7fd788a00640 1 -- 172.21.15.27:0/4116503092 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fd7841a9110 con 0x7fd7841092f0 2024-09-21T00:11:45.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.238+0000 7fd77be00640 1 -- 172.21.15.27:0/4116503092 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7fd7700b2170 con 0x7fd7841092f0 2024-09-21T00:11:45.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.240+0000 7fd788a00640 1 -- 172.21.15.27:0/4116503092 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd7680b6550 msgr2=0x7fd7680b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:45.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.240+0000 7fd788a00640 1 --2- 172.21.15.27:0/4116503092 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd7680b6550 0x7fd7680b8a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd774009bd0 tx=0x7fd774005f90 comp rx=0 tx=0).stop 2024-09-21T00:11:45.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.240+0000 7fd788a00640 1 -- 172.21.15.27:0/4116503092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7841092f0 msgr2=0x7fd78419ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:45.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.240+0000 7fd788a00640 1 --2- 172.21.15.27:0/4116503092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7841092f0 0x7fd78419ca30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd7700058c0 tx=0x7fd77000ee50 comp rx=0 tx=0).stop 2024-09-21T00:11:45.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.240+0000 7fd788a00640 1 -- 172.21.15.27:0/4116503092 shutdown_connections 2024-09-21T00:11:45.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.240+0000 7fd788a00640 1 --2- 172.21.15.27:0/4116503092 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd7680b6550 0x7fd7680b8a60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.240+0000 7fd788a00640 1 --2- 172.21.15.27:0/4116503092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd784109c40 0x7fd7841a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.240+0000 7fd788a00640 1 --2- 172.21.15.27:0/4116503092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7841092f0 0x7fd78419ca30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.240+0000 7fd788a00640 1 --2- 172.21.15.27:0/4116503092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7841039b0 0x7fd78419c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.240+0000 7fd788a00640 1 -- 172.21.15.27:0/4116503092 >> 172.21.15.27:0/4116503092 conn(0x7fd7840ff400 msgr2=0x7fd7841068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:45.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.241+0000 7fd788a00640 1 -- 172.21.15.27:0/4116503092 shutdown_connections 2024-09-21T00:11:45.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.241+0000 7fd788a00640 1 -- 172.21.15.27:0/4116503092 wait complete. 2024-09-21T00:11:45.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:44 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1459670933' entity='client.admin' 2024-09-21T00:11:45.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.796+0000 7faf82e00640 1 -- 172.21.15.27:0/3430417795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf7c109c40 msgr2=0x7faf7c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:45.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.796+0000 7faf82e00640 1 --2- 172.21.15.27:0/3430417795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf7c109c40 0x7faf7c110470 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faf74009ca0 tx=0x7faf7402a8c0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.796+0000 7faf82e00640 1 -- 172.21.15.27:0/3430417795 shutdown_connections 2024-09-21T00:11:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.796+0000 7faf82e00640 1 --2- 172.21.15.27:0/3430417795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf7c109c40 0x7faf7c110470 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.796+0000 7faf82e00640 1 --2- 172.21.15.27:0/3430417795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf7c1092f0 0x7faf7c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.796+0000 7faf82e00640 1 --2- 172.21.15.27:0/3430417795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf7c1039b0 0x7faf7c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.796+0000 7faf82e00640 1 -- 172.21.15.27:0/3430417795 >> 172.21.15.27:0/3430417795 conn(0x7faf7c0ff440 msgr2=0x7faf7c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.796+0000 7faf82e00640 1 -- 172.21.15.27:0/3430417795 shutdown_connections 2024-09-21T00:11:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.797+0000 7faf82e00640 1 -- 172.21.15.27:0/3430417795 wait complete. 2024-09-21T00:11:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.798+0000 7faf82e00640 1 Processor -- start 2024-09-21T00:11:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.798+0000 7faf82e00640 1 -- start start 2024-09-21T00:11:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.798+0000 7faf82e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf7c1039b0 0x7faf7c19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.798+0000 7faf82e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf7c1092f0 0x7faf7c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:45.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.798+0000 7faf82e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf7c109c40 0x7faf7c1a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:45.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.798+0000 7faf82e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c113900 con 0x7faf7c1092f0 2024-09-21T00:11:45.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.798+0000 7faf82e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c113780 con 0x7faf7c1039b0 2024-09-21T00:11:45.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.798+0000 7faf82e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c113a80 con 0x7faf7c109c40 2024-09-21T00:11:45.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.798+0000 7faf81000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf7c1092f0 0x7faf7c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:45.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.799+0000 7faf81000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf7c1092f0 0x7faf7c19ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43684/0 (socket says 172.21.15.27:43684) 2024-09-21T00:11:45.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.799+0000 7faf81000640 1 -- 172.21.15.27:0/354045519 learned_addr learned my addr 172.21.15.27:0/354045519 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:45.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.799+0000 7faf81a00640 1 --2- 172.21.15.27:0/354045519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf7c1039b0 0x7faf7c19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:45.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.799+0000 7faf82400640 1 --2- 172.21.15.27:0/354045519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf7c109c40 0x7faf7c1a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:45.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.799+0000 7faf81000640 1 -- 172.21.15.27:0/354045519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf7c109c40 msgr2=0x7faf7c1a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:45.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.799+0000 7faf81000640 1 --2- 172.21.15.27:0/354045519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf7c109c40 0x7faf7c1a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.799+0000 7faf81000640 1 -- 172.21.15.27:0/354045519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf7c1039b0 msgr2=0x7faf7c19c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:45.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.799+0000 7faf81000640 1 --2- 172.21.15.27:0/354045519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf7c1039b0 0x7faf7c19c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.799+0000 7faf81000640 1 -- 172.21.15.27:0/354045519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c1a2420 con 0x7faf7c1092f0 2024-09-21T00:11:45.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.799+0000 7faf82400640 1 --2- 172.21.15.27:0/354045519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf7c109c40 0x7faf7c1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:11:45.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.799+0000 7faf81000640 1 --2- 172.21.15.27:0/354045519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf7c1092f0 0x7faf7c19ca50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faf6800e9c0 tx=0x7faf6800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:45.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.800+0000 7faf7a000640 1 -- 172.21.15.27:0/354045519 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6800ccc0 con 0x7faf7c1092f0 2024-09-21T00:11:45.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.800+0000 7faf7a000640 1 -- 172.21.15.27:0/354045519 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf680054d0 con 0x7faf7c1092f0 2024-09-21T00:11:45.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.800+0000 7faf7a000640 1 -- 172.21.15.27:0/354045519 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf68010430 con 0x7faf7c1092f0 2024-09-21T00:11:45.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.800+0000 7faf82e00640 1 -- 172.21.15.27:0/354045519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7c1a2710 con 0x7faf7c1092f0 2024-09-21T00:11:45.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.800+0000 7faf82e00640 1 -- 172.21.15.27:0/354045519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7c1a8e70 con 0x7faf7c1092f0 2024-09-21T00:11:45.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.801+0000 7faf82e00640 1 -- 172.21.15.27:0/354045519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7c103dc0 con 0x7faf7c1092f0 2024-09-21T00:11:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.803+0000 7faf7a000640 1 -- 172.21.15.27:0/354045519 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7faf68005020 con 0x7faf7c1092f0 2024-09-21T00:11:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.806+0000 7faf7a000640 1 --2- 172.21.15.27:0/354045519 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7faf640b6550 0x7faf640b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.806+0000 7faf7a000640 1 -- 172.21.15.27:0/354045519 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7faf680e5b60 con 0x7faf7c1092f0 2024-09-21T00:11:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.806+0000 7faf81a00640 1 --2- 172.21.15.27:0/354045519 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7faf640b6550 0x7faf640b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.807+0000 7faf81a00640 1 --2- 172.21.15.27:0/354045519 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7faf640b6550 0x7faf640b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7faf700095f0 tx=0x7faf70005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.809+0000 7faf7a000640 1 -- 172.21.15.27:0/354045519 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faf680adf50 con 0x7faf7c1092f0 2024-09-21T00:11:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.029+0000 7faf82e00640 1 -- 172.21.15.27:0/354045519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7faf7c1a91a0 con 0x7faf7c1092f0 2024-09-21T00:11:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.035+0000 7faf7a000640 1 -- 172.21.15.27:0/354045519 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v27)=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7faf680b2330 con 0x7faf7c1092f0 2024-09-21T00:11:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.038+0000 7faf82e00640 1 -- 172.21.15.27:0/354045519 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7faf640b6550 msgr2=0x7faf640b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.038+0000 7faf82e00640 1 --2- 172.21.15.27:0/354045519 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7faf640b6550 0x7faf640b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7faf700095f0 tx=0x7faf70005f90 comp rx=0 tx=0).stop 2024-09-21T00:11:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.038+0000 7faf82e00640 1 -- 172.21.15.27:0/354045519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf7c1092f0 msgr2=0x7faf7c19ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.038+0000 7faf82e00640 1 --2- 172.21.15.27:0/354045519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf7c1092f0 0x7faf7c19ca50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faf6800e9c0 tx=0x7faf6800ee80 comp rx=0 tx=0).stop 2024-09-21T00:11:45.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.038+0000 7faf82e00640 1 -- 172.21.15.27:0/354045519 shutdown_connections 2024-09-21T00:11:45.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.038+0000 7faf82e00640 1 --2- 172.21.15.27:0/354045519 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7faf640b6550 0x7faf640b8a60 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7faf700095f0 tx=0x7faf70005f90 comp rx=0 tx=0).stop 2024-09-21T00:11:45.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.038+0000 7faf82e00640 1 --2- 172.21.15.27:0/354045519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faf7c109c40 0x7faf7c1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.038+0000 7faf82e00640 1 --2- 172.21.15.27:0/354045519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf7c1092f0 0x7faf7c19ca50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.038+0000 7faf82e00640 1 --2- 172.21.15.27:0/354045519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf7c1039b0 0x7faf7c19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:45.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.038+0000 7faf82e00640 1 -- 172.21.15.27:0/354045519 >> 172.21.15.27:0/354045519 conn(0x7faf7c0ff440 msgr2=0x7faf7c108940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:45.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.039+0000 7faf82e00640 1 -- 172.21.15.27:0/354045519 shutdown_connections 2024-09-21T00:11:45.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.039+0000 7faf82e00640 1 -- 172.21.15.27:0/354045519 wait complete. 2024-09-21T00:11:46.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:46 smithi027 ceph-mon[30745]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:46.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:46 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/354045519' entity='client.admin' 2024-09-21T00:11:46.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:46 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:46.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:46 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.900+0000 7f711c800640 1 -- 172.21.15.27:0/1034479448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71141030f0 msgr2=0x7f7114107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.900+0000 7f711c800640 1 --2- 172.21.15.27:0/1034479448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71141030f0 0x7f7114107620 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7104009930 tx=0x7f710402a440 comp rx=0 tx=0).stop 2024-09-21T00:11:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.900+0000 7f711c800640 1 -- 172.21.15.27:0/1034479448 shutdown_connections 2024-09-21T00:11:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.900+0000 7f711c800640 1 --2- 172.21.15.27:0/1034479448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7114107be0 0x7f71141080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.900+0000 7f711c800640 1 --2- 172.21.15.27:0/1034479448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71141030f0 0x7f7114107620 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.900+0000 7f711c800640 1 --2- 172.21.15.27:0/1034479448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71141027a0 0x7f7114102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.900+0000 7f711c800640 1 -- 172.21.15.27:0/1034479448 >> 172.21.15.27:0/1034479448 conn(0x7f7114079730 msgr2=0x7f7114079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.900+0000 7f711c800640 1 -- 172.21.15.27:0/1034479448 shutdown_connections 2024-09-21T00:11:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.900+0000 7f711c800640 1 -- 172.21.15.27:0/1034479448 wait complete. 2024-09-21T00:11:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.902+0000 7f711c800640 1 Processor -- start 2024-09-21T00:11:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.902+0000 7f711c800640 1 -- start start 2024-09-21T00:11:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.902+0000 7f711c800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71141027a0 0x7f71141a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.903+0000 7f7119e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71141027a0 0x7f71141a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.903+0000 7f7119e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71141027a0 0x7f71141a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43708/0 (socket says 172.21.15.27:43708) 2024-09-21T00:11:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.903+0000 7f711c800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71141030f0 0x7f71141a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.903+0000 7f711c800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7114107be0 0x7f71141a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.903+0000 7f711c800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f711411a0b0 con 0x7f71141027a0 2024-09-21T00:11:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.903+0000 7f711c800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7114119f30 con 0x7f71141030f0 2024-09-21T00:11:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.903+0000 7f711c800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f711411a230 con 0x7f7114107be0 2024-09-21T00:11:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.903+0000 7f7119e00640 1 -- 172.21.15.27:0/2360018982 learned_addr learned my addr 172.21.15.27:0/2360018982 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.904+0000 7f711a800640 1 --2- 172.21.15.27:0/2360018982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7114107be0 0x7f71141a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.904+0000 7f7119400640 1 --2- 172.21.15.27:0/2360018982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71141030f0 0x7f71141a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.905+0000 7f711a800640 1 -- 172.21.15.27:0/2360018982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71141030f0 msgr2=0x7f71141a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.905+0000 7f711a800640 1 --2- 172.21.15.27:0/2360018982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71141030f0 0x7f71141a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.905+0000 7f711a800640 1 -- 172.21.15.27:0/2360018982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71141027a0 msgr2=0x7f71141a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.905+0000 7f711a800640 1 --2- 172.21.15.27:0/2360018982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71141027a0 0x7f71141a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.905+0000 7f711a800640 1 -- 172.21.15.27:0/2360018982 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71141a6a50 con 0x7f7114107be0 2024-09-21T00:11:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.905+0000 7f711a800640 1 --2- 172.21.15.27:0/2360018982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7114107be0 0x7f71141a6320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7110009fd0 tx=0x7f711000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.905+0000 7f710ea00640 1 -- 172.21.15.27:0/2360018982 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711000ce20 con 0x7f7114107be0 2024-09-21T00:11:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.906+0000 7f710ea00640 1 -- 172.21.15.27:0/2360018982 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7110005020 con 0x7f7114107be0 2024-09-21T00:11:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.906+0000 7f711c800640 1 -- 172.21.15.27:0/2360018982 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71141a6d40 con 0x7f7114107be0 2024-09-21T00:11:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.906+0000 7f710ea00640 1 -- 172.21.15.27:0/2360018982 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110010690 con 0x7f7114107be0 2024-09-21T00:11:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.906+0000 7f711c800640 1 -- 172.21.15.27:0/2360018982 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71141ad4a0 con 0x7f7114107be0 2024-09-21T00:11:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.907+0000 7f711c800640 1 -- 172.21.15.27:0/2360018982 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7114107620 con 0x7f7114107be0 2024-09-21T00:11:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.912+0000 7f710ea00640 1 -- 172.21.15.27:0/2360018982 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f71100054d0 con 0x7f7114107be0 2024-09-21T00:11:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.914+0000 7f710ea00640 1 --2- 172.21.15.27:0/2360018982 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f70fc0b64d0 0x7f70fc0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.914+0000 7f710ea00640 1 -- 172.21.15.27:0/2360018982 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f71100e5ff0 con 0x7f7114107be0 2024-09-21T00:11:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.915+0000 7f7119e00640 1 --2- 172.21.15.27:0/2360018982 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f70fc0b64d0 0x7f70fc0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.915+0000 7f7119e00640 1 --2- 172.21.15.27:0/2360018982 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f70fc0b64d0 0x7f70fc0b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f71080058c0 tx=0x7f7108005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:44.917+0000 7f710ea00640 1 -- 172.21.15.27:0/2360018982 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f71100ae3e0 con 0x7f7114107be0 2024-09-21T00:11:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.132+0000 7f711c800640 1 -- 172.21.15.27:0/2360018982 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f71141ad810 con 0x7f7114107be0 2024-09-21T00:11:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.140+0000 7f710ea00640 1 -- 172.21.15.27:0/2360018982 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v28)=0 v28) ==== 120+0+0 (secure 0 0 0) 0x7f71100b27c0 con 0x7f7114107be0 2024-09-21T00:11:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.143+0000 7f711c800640 1 -- 172.21.15.27:0/2360018982 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f70fc0b64d0 msgr2=0x7f70fc0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.143+0000 7f711c800640 1 --2- 172.21.15.27:0/2360018982 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f70fc0b64d0 0x7f70fc0b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f71080058c0 tx=0x7f7108005f90 comp rx=0 tx=0).stop 2024-09-21T00:11:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.143+0000 7f711c800640 1 -- 172.21.15.27:0/2360018982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7114107be0 msgr2=0x7f71141a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.143+0000 7f711c800640 1 --2- 172.21.15.27:0/2360018982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7114107be0 0x7f71141a6320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7110009fd0 tx=0x7f711000ef10 comp rx=0 tx=0).stop 2024-09-21T00:11:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.143+0000 7f711c800640 1 -- 172.21.15.27:0/2360018982 shutdown_connections 2024-09-21T00:11:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.143+0000 7f711c800640 1 --2- 172.21.15.27:0/2360018982 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f70fc0b64d0 0x7f70fc0b89e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.143+0000 7f711c800640 1 --2- 172.21.15.27:0/2360018982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7114107be0 0x7f71141a6320 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.143+0000 7f711c800640 1 --2- 172.21.15.27:0/2360018982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71141030f0 0x7f71141a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.143+0000 7f711c800640 1 --2- 172.21.15.27:0/2360018982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71141027a0 0x7f71141a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.143+0000 7f711c800640 1 -- 172.21.15.27:0/2360018982 >> 172.21.15.27:0/2360018982 conn(0x7f7114079730 msgr2=0x7f71141055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.143+0000 7f711c800640 1 -- 172.21.15.27:0/2360018982 shutdown_connections 2024-09-21T00:11:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:45.143+0000 7f711c800640 1 -- 172.21.15.27:0/2360018982 wait complete. 2024-09-21T00:11:46.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:46 smithi089 ceph-mon[34481]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:46.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:46 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/354045519' entity='client.admin' 2024-09-21T00:11:46.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:46 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:46.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:46 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:46.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:46 smithi098 ceph-mon[34725]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:46.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:46 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/354045519' entity='client.admin' 2024-09-21T00:11:46.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:46 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:46.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:46 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:47.562 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi027') 2024-09-21T00:11:47.562 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi027') 2024-09-21T00:11:47.563 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi027') 2024-09-21T00:11:47.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.819+0000 7f7f9a200640 1 -- 172.21.15.27:0/790301158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f941094c0 msgr2=0x7f7f941098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:47.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.819+0000 7f7f9a200640 1 --2- 172.21.15.27:0/790301158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f941094c0 0x7f7f941098d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009900 tx=0x7f7f8002a3f0 comp rx=0 tx=0).stop 2024-09-21T00:11:47.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.819+0000 7f7f9a200640 1 -- 172.21.15.27:0/790301158 shutdown_connections 2024-09-21T00:11:47.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.819+0000 7f7f9a200640 1 --2- 172.21.15.27:0/790301158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f94109e10 0x7f7f94110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:47.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.819+0000 7f7f9a200640 1 --2- 172.21.15.27:0/790301158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f941094c0 0x7f7f941098d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:47.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.819+0000 7f7f9a200640 1 --2- 172.21.15.27:0/790301158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f94103b80 0x7f7f94103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:47.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.819+0000 7f7f9a200640 1 -- 172.21.15.27:0/790301158 >> 172.21.15.27:0/790301158 conn(0x7f7f940ff590 msgr2=0x7f7f941019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:47.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.819+0000 7f7f9a200640 1 -- 172.21.15.27:0/790301158 shutdown_connections 2024-09-21T00:11:47.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.819+0000 7f7f9a200640 1 -- 172.21.15.27:0/790301158 wait complete. 2024-09-21T00:11:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.820+0000 7f7f9a200640 1 Processor -- start 2024-09-21T00:11:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.820+0000 7f7f9a200640 1 -- start start 2024-09-21T00:11:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f9a200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f94103b80 0x7f7f9419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f9a200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f941094c0 0x7f7f9419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f9a200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f94109e10 0x7f7f941a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f9a200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7f94113ad0 con 0x7f7f94103b80 2024-09-21T00:11:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f9a200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7f94113950 con 0x7f7f94109e10 2024-09-21T00:11:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f9a200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7f94113c50 con 0x7f7f941094c0 2024-09-21T00:11:47.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f93400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f94103b80 0x7f7f9419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:47.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f93400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f94103b80 0x7f7f9419c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43750/0 (socket says 172.21.15.27:43750) 2024-09-21T00:11:47.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f93400640 1 -- 172.21.15.27:0/1787695553 learned_addr learned my addr 172.21.15.27:0/1787695553 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:47.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f93e00640 1 --2- 172.21.15.27:0/1787695553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f94109e10 0x7f7f941a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:47.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f92a00640 1 --2- 172.21.15.27:0/1787695553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f941094c0 0x7f7f9419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:47.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f93e00640 1 -- 172.21.15.27:0/1787695553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f941094c0 msgr2=0x7f7f9419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:47.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f93e00640 1 --2- 172.21.15.27:0/1787695553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f941094c0 0x7f7f9419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:47.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f93e00640 1 -- 172.21.15.27:0/1787695553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f94103b80 msgr2=0x7f7f9419c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:47.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f93e00640 1 --2- 172.21.15.27:0/1787695553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f94103b80 0x7f7f9419c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:47.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.821+0000 7f7f93e00640 1 -- 172.21.15.27:0/1787695553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f941a2610 con 0x7f7f94109e10 2024-09-21T00:11:47.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.822+0000 7f7f93e00640 1 --2- 172.21.15.27:0/1787695553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f94109e10 0x7f7f941a1ee0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009fd0 tx=0x7f7f8c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:47.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.822+0000 7f7f8be00640 1 -- 172.21.15.27:0/1787695553 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00ce20 con 0x7f7f94109e10 2024-09-21T00:11:47.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.822+0000 7f7f8be00640 1 -- 172.21.15.27:0/1787695553 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f8c005020 con 0x7f7f94109e10 2024-09-21T00:11:47.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.822+0000 7f7f9a200640 1 -- 172.21.15.27:0/1787695553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f941a2900 con 0x7f7f94109e10 2024-09-21T00:11:47.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.822+0000 7f7f8be00640 1 -- 172.21.15.27:0/1787695553 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c010690 con 0x7f7f94109e10 2024-09-21T00:11:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.822+0000 7f7f9a200640 1 -- 172.21.15.27:0/1787695553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f941a9060 con 0x7f7f94109e10 2024-09-21T00:11:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.823+0000 7f7f9a200640 1 -- 172.21.15.27:0/1787695553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f94103f90 con 0x7f7f94109e10 2024-09-21T00:11:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.826+0000 7f7f8be00640 1 -- 172.21.15.27:0/1787695553 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f7f8c0054d0 con 0x7f7f94109e10 2024-09-21T00:11:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.827+0000 7f7f8be00640 1 --2- 172.21.15.27:0/1787695553 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7f780b6550 0x7f7f780b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.827+0000 7f7f8be00640 1 -- 172.21.15.27:0/1787695553 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7f8c0e5e30 con 0x7f7f94109e10 2024-09-21T00:11:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.827+0000 7f7f93400640 1 --2- 172.21.15.27:0/1787695553 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7f780b6550 0x7f7f780b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:47.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.828+0000 7f7f93400640 1 --2- 172.21.15.27:0/1787695553 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7f780b6550 0x7f7f780b8a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009c90 tx=0x7f7f84005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:47.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:47.831+0000 7f7f8be00640 1 -- 172.21.15.27:0/1787695553 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7f8c0ae220 con 0x7f7f94109e10 2024-09-21T00:11:48.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.037+0000 7f7f9a200640 1 -- 172.21.15.27:0/1787695553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f7f941a93f0 con 0x7f7f94109e10 2024-09-21T00:11:48.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.046+0000 7f7f8be00640 1 -- 172.21.15.27:0/1787695553 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v30)=0 v30) ==== 118+0+0 (secure 0 0 0) 0x7f7f8c0b2600 con 0x7f7f94109e10 2024-09-21T00:11:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.049+0000 7f7f9a200640 1 -- 172.21.15.27:0/1787695553 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7f780b6550 msgr2=0x7f7f780b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.049+0000 7f7f9a200640 1 --2- 172.21.15.27:0/1787695553 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7f780b6550 0x7f7f780b8a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009c90 tx=0x7f7f84005f90 comp rx=0 tx=0).stop 2024-09-21T00:11:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.049+0000 7f7f9a200640 1 -- 172.21.15.27:0/1787695553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f94109e10 msgr2=0x7f7f941a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.049+0000 7f7f9a200640 1 --2- 172.21.15.27:0/1787695553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f94109e10 0x7f7f941a1ee0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009fd0 tx=0x7f7f8c00ef10 comp rx=0 tx=0).stop 2024-09-21T00:11:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.049+0000 7f7f9a200640 1 -- 172.21.15.27:0/1787695553 shutdown_connections 2024-09-21T00:11:48.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.049+0000 7f7f9a200640 1 --2- 172.21.15.27:0/1787695553 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7f780b6550 0x7f7f780b8a60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.049+0000 7f7f9a200640 1 --2- 172.21.15.27:0/1787695553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f94109e10 0x7f7f941a1ee0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.049+0000 7f7f9a200640 1 --2- 172.21.15.27:0/1787695553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f941094c0 0x7f7f9419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.049+0000 7f7f9a200640 1 --2- 172.21.15.27:0/1787695553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f94103b80 0x7f7f9419c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.049+0000 7f7f9a200640 1 -- 172.21.15.27:0/1787695553 >> 172.21.15.27:0/1787695553 conn(0x7f7f940ff590 msgr2=0x7f7f94105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:48.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.049+0000 7f7f9a200640 1 -- 172.21.15.27:0/1787695553 shutdown_connections 2024-09-21T00:11:48.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.049+0000 7f7f9a200640 1 -- 172.21.15.27:0/1787695553 wait complete. 2024-09-21T00:11:48.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.608+0000 7f5089a00640 1 -- 172.21.15.27:0/82007317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5084109c40 msgr2=0x7f5084110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:48.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.608+0000 7f5089a00640 1 --2- 172.21.15.27:0/82007317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5084109c40 0x7f5084110470 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f507c009ca0 tx=0x7f507c02a8c0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.608+0000 7f5089a00640 1 -- 172.21.15.27:0/82007317 shutdown_connections 2024-09-21T00:11:48.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.608+0000 7f5089a00640 1 --2- 172.21.15.27:0/82007317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5084109c40 0x7f5084110470 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.608+0000 7f5089a00640 1 --2- 172.21.15.27:0/82007317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50841092f0 0x7f5084109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.608+0000 7f5089a00640 1 --2- 172.21.15.27:0/82007317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50841039b0 0x7f5084103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.608+0000 7f5089a00640 1 -- 172.21.15.27:0/82007317 >> 172.21.15.27:0/82007317 conn(0x7f50840ff440 msgr2=0x7f5084101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:48.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.608+0000 7f5089a00640 1 -- 172.21.15.27:0/82007317 shutdown_connections 2024-09-21T00:11:48.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.609+0000 7f5089a00640 1 -- 172.21.15.27:0/82007317 wait complete. 2024-09-21T00:11:48.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.610+0000 7f5089a00640 1 Processor -- start 2024-09-21T00:11:48.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.610+0000 7f5089a00640 1 -- start start 2024-09-21T00:11:48.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.610+0000 7f5089a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50841039b0 0x7f5084075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:48.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.610+0000 7f5089a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50841092f0 0x7f5084075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:48.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.610+0000 7f5089a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5084109c40 0x7f5084076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:48.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.610+0000 7f5089a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5084113950 con 0x7f50841092f0 2024-09-21T00:11:48.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.610+0000 7f5089a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f50841137d0 con 0x7f5084109c40 2024-09-21T00:11:48.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.610+0000 7f5089a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5084113ad0 con 0x7f50841039b0 2024-09-21T00:11:48.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.611+0000 7f5089000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5084109c40 0x7f5084076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:48.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.611+0000 7f5083e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50841039b0 0x7f5084075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:48.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.611+0000 7f5089000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5084109c40 0x7f5084076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58176/0 (socket says 172.21.15.27:58176) 2024-09-21T00:11:48.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.611+0000 7f5083e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50841039b0 0x7f5084075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39068/0 (socket says 172.21.15.27:39068) 2024-09-21T00:11:48.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.611+0000 7f5083e00640 1 -- 172.21.15.27:0/1300646486 learned_addr learned my addr 172.21.15.27:0/1300646486 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:48.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.611+0000 7f507be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50841092f0 0x7f5084075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:48.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.611+0000 7f5083e00640 1 -- 172.21.15.27:0/1300646486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5084109c40 msgr2=0x7f5084076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:48.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.611+0000 7f5083e00640 1 --2- 172.21.15.27:0/1300646486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5084109c40 0x7f5084076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.611+0000 7f5083e00640 1 -- 172.21.15.27:0/1300646486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50841092f0 msgr2=0x7f5084075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.611+0000 7f5083e00640 1 --2- 172.21.15.27:0/1300646486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50841092f0 0x7f5084075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.611+0000 7f5083e00640 1 -- 172.21.15.27:0/1300646486 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50841a85a0 con 0x7f50841039b0 2024-09-21T00:11:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.612+0000 7f5083e00640 1 --2- 172.21.15.27:0/1300646486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50841039b0 0x7f5084075710 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f507400ea60 tx=0x7f507400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.612+0000 7f5081600640 1 -- 172.21.15.27:0/1300646486 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507400cde0 con 0x7f50841039b0 2024-09-21T00:11:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.612+0000 7f5081600640 1 -- 172.21.15.27:0/1300646486 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50740035f0 con 0x7f50841039b0 2024-09-21T00:11:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.612+0000 7f5089a00640 1 -- 172.21.15.27:0/1300646486 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50841a8890 con 0x7f50841039b0 2024-09-21T00:11:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.612+0000 7f5081600640 1 -- 172.21.15.27:0/1300646486 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074010690 con 0x7f50841039b0 2024-09-21T00:11:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.612+0000 7f5089a00640 1 -- 172.21.15.27:0/1300646486 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50841a8e20 con 0x7f50841039b0 2024-09-21T00:11:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.613+0000 7f5089a00640 1 -- 172.21.15.27:0/1300646486 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5084103dc0 con 0x7f50841039b0 2024-09-21T00:11:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.618+0000 7f5081600640 1 -- 172.21.15.27:0/1300646486 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f50740037a0 con 0x7f50841039b0 2024-09-21T00:11:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.620+0000 7f5081600640 1 --2- 172.21.15.27:0/1300646486 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f50680b6550 0x7f50680b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.620+0000 7f5081600640 1 -- 172.21.15.27:0/1300646486 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f50740e5f40 con 0x7f50841039b0 2024-09-21T00:11:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.621+0000 7f507be00640 1 --2- 172.21.15.27:0/1300646486 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f50680b6550 0x7f50680b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.621+0000 7f5081600640 1 -- 172.21.15.27:0/1300646486 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f50740e6350 con 0x7f50841039b0 2024-09-21T00:11:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.621+0000 7f507be00640 1 --2- 172.21.15.27:0/1300646486 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f50680b6550 0x7f50680b8a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5084105c80 tx=0x7f506c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.829+0000 7f5089a00640 1 -- 172.21.15.27:0/1300646486 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f5084076a10 con 0x7f50841039b0 2024-09-21T00:11:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.836+0000 7f5081600640 1 -- 172.21.15.27:0/1300646486 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v25)=0 v25) ==== 125+0+0 (secure 0 0 0) 0x7f5084076a10 con 0x7f50841039b0 2024-09-21T00:11:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.839+0000 7f5089a00640 1 -- 172.21.15.27:0/1300646486 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f50680b6550 msgr2=0x7f50680b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.839+0000 7f5089a00640 1 --2- 172.21.15.27:0/1300646486 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f50680b6550 0x7f50680b8a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5084105c80 tx=0x7f506c005f90 comp rx=0 tx=0).stop 2024-09-21T00:11:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.839+0000 7f5089a00640 1 -- 172.21.15.27:0/1300646486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50841039b0 msgr2=0x7f5084075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.839+0000 7f5089a00640 1 --2- 172.21.15.27:0/1300646486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50841039b0 0x7f5084075710 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f507400ea60 tx=0x7f507400ef20 comp rx=0 tx=0).stop 2024-09-21T00:11:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.839+0000 7f5089a00640 1 -- 172.21.15.27:0/1300646486 shutdown_connections 2024-09-21T00:11:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.839+0000 7f5089a00640 1 --2- 172.21.15.27:0/1300646486 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f50680b6550 0x7f50680b8a60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.839+0000 7f5089a00640 1 --2- 172.21.15.27:0/1300646486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5084109c40 0x7f5084076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.839+0000 7f5089a00640 1 --2- 172.21.15.27:0/1300646486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50841092f0 0x7f5084075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.839+0000 7f5089a00640 1 --2- 172.21.15.27:0/1300646486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50841039b0 0x7f5084075710 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.839+0000 7f5089a00640 1 -- 172.21.15.27:0/1300646486 >> 172.21.15.27:0/1300646486 conn(0x7f50840ff440 msgr2=0x7f5084107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:48.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.839+0000 7f5089a00640 1 -- 172.21.15.27:0/1300646486 shutdown_connections 2024-09-21T00:11:48.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:42.839+0000 7f5089a00640 1 -- 172.21.15.27:0/1300646486 wait complete. 2024-09-21T00:11:48.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:48 smithi027 ceph-mon[30745]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:48.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.512+0000 7f394e200640 1 -- 172.21.15.27:0/478629475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f394810b6c0 msgr2=0x7f394810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:48.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.512+0000 7f394e200640 1 --2- 172.21.15.27:0/478629475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f394810b6c0 0x7f394810bad0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3934009900 tx=0x7f393402a400 comp rx=0 tx=0).stop 2024-09-21T00:11:48.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.513+0000 7f394e200640 1 -- 172.21.15.27:0/478629475 shutdown_connections 2024-09-21T00:11:48.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.513+0000 7f394e200640 1 --2- 172.21.15.27:0/478629475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f394810c010 0x7f3948112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.513+0000 7f394e200640 1 --2- 172.21.15.27:0/478629475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f394810b6c0 0x7f394810bad0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.513+0000 7f394e200640 1 --2- 172.21.15.27:0/478629475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3948105d80 0x7f3948106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.513+0000 7f394e200640 1 -- 172.21.15.27:0/478629475 >> 172.21.15.27:0/478629475 conn(0x7f3948101810 msgr2=0x7f3948103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:48.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.513+0000 7f394e200640 1 -- 172.21.15.27:0/478629475 shutdown_connections 2024-09-21T00:11:48.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.513+0000 7f394e200640 1 -- 172.21.15.27:0/478629475 wait complete. 2024-09-21T00:11:48.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.514+0000 7f394e200640 1 Processor -- start 2024-09-21T00:11:48.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.514+0000 7f394e200640 1 -- start start 2024-09-21T00:11:48.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.515+0000 7f394e200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3948105d80 0x7f394819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:48.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.515+0000 7f394e200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f394810b6c0 0x7f394819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:48.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.515+0000 7f394e200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f394810c010 0x7f39481a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:48.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.515+0000 7f394e200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3948115d20 con 0x7f394810b6c0 2024-09-21T00:11:48.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.515+0000 7f394e200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3948115ba0 con 0x7f3948105d80 2024-09-21T00:11:48.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.515+0000 7f394e200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3948115ea0 con 0x7f394810c010 2024-09-21T00:11:48.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.515+0000 7f3947400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3948105d80 0x7f394819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:48.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.515+0000 7f3947e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f394810c010 0x7f39481a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:48.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.515+0000 7f3947400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3948105d80 0x7f394819c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58040/0 (socket says 172.21.15.27:58040) 2024-09-21T00:11:48.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.515+0000 7f3947400640 1 -- 172.21.15.27:0/4172954767 learned_addr learned my addr 172.21.15.27:0/4172954767 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:48.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.515+0000 7f3946a00640 1 --2- 172.21.15.27:0/4172954767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f394810b6c0 0x7f394819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:48.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.515+0000 7f3947400640 1 -- 172.21.15.27:0/4172954767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f394810c010 msgr2=0x7f39481a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.515+0000 7f3947400640 1 --2- 172.21.15.27:0/4172954767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f394810c010 0x7f39481a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.516+0000 7f3947400640 1 -- 172.21.15.27:0/4172954767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f394810b6c0 msgr2=0x7f394819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.516+0000 7f3947400640 1 --2- 172.21.15.27:0/4172954767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f394810b6c0 0x7f394819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.516+0000 7f3947400640 1 -- 172.21.15.27:0/4172954767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39481a26c0 con 0x7f3948105d80 2024-09-21T00:11:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.516+0000 7f3947e00640 1 --2- 172.21.15.27:0/4172954767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f394810c010 0x7f39481a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:11:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.516+0000 7f3947400640 1 --2- 172.21.15.27:0/4172954767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3948105d80 0x7f394819c750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f393800ea60 tx=0x7f393800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.516+0000 7f393fe00640 1 -- 172.21.15.27:0/4172954767 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393800cde0 con 0x7f3948105d80 2024-09-21T00:11:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.516+0000 7f393fe00640 1 -- 172.21.15.27:0/4172954767 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39380035f0 con 0x7f3948105d80 2024-09-21T00:11:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.516+0000 7f394e200640 1 -- 172.21.15.27:0/4172954767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39481a29b0 con 0x7f3948105d80 2024-09-21T00:11:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.516+0000 7f393fe00640 1 -- 172.21.15.27:0/4172954767 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938010690 con 0x7f3948105d80 2024-09-21T00:11:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.516+0000 7f394e200640 1 -- 172.21.15.27:0/4172954767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39481a9110 con 0x7f3948105d80 2024-09-21T00:11:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.517+0000 7f394e200640 1 -- 172.21.15.27:0/4172954767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3948106190 con 0x7f3948105d80 2024-09-21T00:11:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.517+0000 7f393fe00640 1 -- 172.21.15.27:0/4172954767 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f3938004020 con 0x7f3948105d80 2024-09-21T00:11:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.518+0000 7f393fe00640 1 --2- 172.21.15.27:0/4172954767 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f392c03fb70 0x7f392c042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.518+0000 7f393fe00640 1 -- 172.21.15.27:0/4172954767 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3938054960 con 0x7f3948105d80 2024-09-21T00:11:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.518+0000 7f3946a00640 1 --2- 172.21.15.27:0/4172954767 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f392c03fb70 0x7f392c042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.519+0000 7f3946a00640 1 --2- 172.21.15.27:0/4172954767 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f392c03fb70 0x7f392c042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f393402a9f0 tx=0x7f3934005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.523+0000 7f393fe00640 1 -- 172.21.15.27:0/4172954767 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3938003ac0 con 0x7f3948105d80 2024-09-21T00:11:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.725+0000 7f394e200640 1 -- 172.21.15.27:0/4172954767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7f394802e640 con 0x7f3948105d80 2024-09-21T00:11:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.732+0000 7f393fe00640 1 -- 172.21.15.27:0/4172954767 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v18)=0 v18) ==== 115+0+0 (secure 0 0 0) 0x7f3938021cd0 con 0x7f3948105d80 2024-09-21T00:11:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.735+0000 7f394e200640 1 -- 172.21.15.27:0/4172954767 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f392c03fb70 msgr2=0x7f392c042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.735+0000 7f394e200640 1 --2- 172.21.15.27:0/4172954767 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f392c03fb70 0x7f392c042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f393402a9f0 tx=0x7f3934005a20 comp rx=0 tx=0).stop 2024-09-21T00:11:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.735+0000 7f394e200640 1 -- 172.21.15.27:0/4172954767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3948105d80 msgr2=0x7f394819c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.735+0000 7f394e200640 1 --2- 172.21.15.27:0/4172954767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3948105d80 0x7f394819c750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f393800ea60 tx=0x7f393800ef20 comp rx=0 tx=0).stop 2024-09-21T00:11:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.735+0000 7f394e200640 1 -- 172.21.15.27:0/4172954767 shutdown_connections 2024-09-21T00:11:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.735+0000 7f394e200640 1 --2- 172.21.15.27:0/4172954767 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f392c03fb70 0x7f392c042080 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.735+0000 7f394e200640 1 --2- 172.21.15.27:0/4172954767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f394810c010 0x7f39481a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.735+0000 7f394e200640 1 --2- 172.21.15.27:0/4172954767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f394810b6c0 0x7f394819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.735+0000 7f394e200640 1 --2- 172.21.15.27:0/4172954767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3948105d80 0x7f394819c750 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:48.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.735+0000 7f394e200640 1 -- 172.21.15.27:0/4172954767 >> 172.21.15.27:0/4172954767 conn(0x7f3948101810 msgr2=0x7f3948103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:48.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.736+0000 7f394e200640 1 -- 172.21.15.27:0/4172954767 shutdown_connections 2024-09-21T00:11:48.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:37.736+0000 7f394e200640 1 -- 172.21.15.27:0/4172954767 wait complete. 2024-09-21T00:11:48.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:48 smithi089 ceph-mon[34481]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:48.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:48 smithi098 ceph-mon[34725]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:49.238 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:49 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:49.238 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:49 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1665459997' entity='client.admin' 2024-09-21T00:11:49.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:49 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:49.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:49 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1665459997' entity='client.admin' 2024-09-21T00:11:49.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:49 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:49.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:49 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1665459997' entity='client.admin' 2024-09-21T00:11:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:50 smithi027 ceph-mon[30745]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:50 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/4218632187' entity='client.admin' 2024-09-21T00:11:50.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:50 smithi089 ceph-mon[34481]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:50.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:50 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/4218632187' entity='client.admin' 2024-09-21T00:11:50.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:50 smithi098 ceph-mon[34725]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:50.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:50 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/4218632187' entity='client.admin' 2024-09-21T00:11:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.656+0000 7f3f81600640 1 -- 172.21.15.27:0/1844001419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c105d90 msgr2=0x7f3f7c1061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.656+0000 7f3f81600640 1 --2- 172.21.15.27:0/1844001419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c105d90 0x7f3f7c1061a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009860 tx=0x7f3f6c02a340 comp rx=0 tx=0).stop 2024-09-21T00:11:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.656+0000 7f3f81600640 1 -- 172.21.15.27:0/1844001419 shutdown_connections 2024-09-21T00:11:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.656+0000 7f3f81600640 1 --2- 172.21.15.27:0/1844001419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f7c10c020 0x7f3f7c112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.656+0000 7f3f81600640 1 --2- 172.21.15.27:0/1844001419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c10b6d0 0x7f3f7c10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.656+0000 7f3f81600640 1 --2- 172.21.15.27:0/1844001419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c105d90 0x7f3f7c1061a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.656+0000 7f3f81600640 1 -- 172.21.15.27:0/1844001419 >> 172.21.15.27:0/1844001419 conn(0x7f3f7c101860 msgr2=0x7f3f7c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.656+0000 7f3f81600640 1 -- 172.21.15.27:0/1844001419 shutdown_connections 2024-09-21T00:11:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.656+0000 7f3f81600640 1 -- 172.21.15.27:0/1844001419 wait complete. 2024-09-21T00:11:50.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.657+0000 7f3f81600640 1 Processor -- start 2024-09-21T00:11:50.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.657+0000 7f3f81600640 1 -- start start 2024-09-21T00:11:50.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f81600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c105d90 0x7f3f7c19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:50.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f81600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c10b6d0 0x7f3f7c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:50.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f81600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f7c10c020 0x7f3f7c1a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:50.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f81600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c115ce0 con 0x7f3f7c10b6d0 2024-09-21T00:11:50.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f81600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c115b60 con 0x7f3f7c10c020 2024-09-21T00:11:50.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f81600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c115e60 con 0x7f3f7c105d90 2024-09-21T00:11:50.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f7a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c10b6d0 0x7f3f7c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:50.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f7a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c10b6d0 0x7f3f7c19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43768/0 (socket says 172.21.15.27:43768) 2024-09-21T00:11:50.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f7a000640 1 -- 172.21.15.27:0/1665459997 learned_addr learned my addr 172.21.15.27:0/1665459997 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:50.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f7aa00640 1 --2- 172.21.15.27:0/1665459997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c105d90 0x7f3f7c19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:50.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f7b400640 1 --2- 172.21.15.27:0/1665459997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f7c10c020 0x7f3f7c1a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:50.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f7a000640 1 -- 172.21.15.27:0/1665459997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c105d90 msgr2=0x7f3f7c19c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f7a000640 1 --2- 172.21.15.27:0/1665459997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c105d90 0x7f3f7c19c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f7a000640 1 -- 172.21.15.27:0/1665459997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f7c10c020 msgr2=0x7f3f7c1a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f7a000640 1 --2- 172.21.15.27:0/1665459997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f7c10c020 0x7f3f7c1a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.658+0000 7f3f7a000640 1 -- 172.21.15.27:0/1665459997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c1a2620 con 0x7f3f7c10b6d0 2024-09-21T00:11:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.659+0000 7f3f7a000640 1 --2- 172.21.15.27:0/1665459997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c10b6d0 0x7f3f7c19cc50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3f680058c0 tx=0x7f3f6800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.659+0000 7f3f73400640 1 -- 172.21.15.27:0/1665459997 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6800cd30 con 0x7f3f7c10b6d0 2024-09-21T00:11:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.659+0000 7f3f73400640 1 -- 172.21.15.27:0/1665459997 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f680035f0 con 0x7f3f7c10b6d0 2024-09-21T00:11:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.659+0000 7f3f81600640 1 -- 172.21.15.27:0/1665459997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7c1a2910 con 0x7f3f7c10b6d0 2024-09-21T00:11:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.659+0000 7f3f73400640 1 -- 172.21.15.27:0/1665459997 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f680044e0 con 0x7f3f7c10b6d0 2024-09-21T00:11:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.659+0000 7f3f81600640 1 -- 172.21.15.27:0/1665459997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7c1a9070 con 0x7f3f7c10b6d0 2024-09-21T00:11:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.660+0000 7f3f81600640 1 -- 172.21.15.27:0/1665459997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f7c1061a0 con 0x7f3f7c10b6d0 2024-09-21T00:11:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.663+0000 7f3f73400640 1 -- 172.21.15.27:0/1665459997 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f3f68003aa0 con 0x7f3f7c10b6d0 2024-09-21T00:11:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.665+0000 7f3f73400640 1 --2- 172.21.15.27:0/1665459997 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3f600b6550 0x7f3f600b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.665+0000 7f3f73400640 1 -- 172.21.15.27:0/1665459997 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3f680e59e0 con 0x7f3f7c10b6d0 2024-09-21T00:11:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.665+0000 7f3f7aa00640 1 --2- 172.21.15.27:0/1665459997 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3f600b6550 0x7f3f600b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.666+0000 7f3f7aa00640 1 --2- 172.21.15.27:0/1665459997 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3f600b6550 0x7f3f600b8a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c02a930 tx=0x7f3f6c0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.670+0000 7f3f73400640 1 -- 172.21.15.27:0/1665459997 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3f680addd0 con 0x7f3f7c10b6d0 2024-09-21T00:11:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.872+0000 7f3f81600640 1 -- 172.21.15.27:0/1665459997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f3f7c02e640 con 0x7f3f7c10b6d0 2024-09-21T00:11:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.877+0000 7f3f73400640 1 -- 172.21.15.27:0/1665459997 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v31)=0 v31) ==== 116+0+0 (secure 0 0 0) 0x7f3f680b21b0 con 0x7f3f7c10b6d0 2024-09-21T00:11:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.879+0000 7f3f81600640 1 -- 172.21.15.27:0/1665459997 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3f600b6550 msgr2=0x7f3f600b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.879+0000 7f3f81600640 1 --2- 172.21.15.27:0/1665459997 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3f600b6550 0x7f3f600b8a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c02a930 tx=0x7f3f6c0058d0 comp rx=0 tx=0).stop 2024-09-21T00:11:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.879+0000 7f3f81600640 1 -- 172.21.15.27:0/1665459997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c10b6d0 msgr2=0x7f3f7c19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.879+0000 7f3f81600640 1 --2- 172.21.15.27:0/1665459997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c10b6d0 0x7f3f7c19cc50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3f680058c0 tx=0x7f3f6800ee50 comp rx=0 tx=0).stop 2024-09-21T00:11:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.879+0000 7f3f81600640 1 -- 172.21.15.27:0/1665459997 shutdown_connections 2024-09-21T00:11:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.879+0000 7f3f81600640 1 --2- 172.21.15.27:0/1665459997 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3f600b6550 0x7f3f600b8a60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.879+0000 7f3f81600640 1 --2- 172.21.15.27:0/1665459997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f7c10c020 0x7f3f7c1a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.879+0000 7f3f81600640 1 --2- 172.21.15.27:0/1665459997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c10b6d0 0x7f3f7c19cc50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:50.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.879+0000 7f3f81600640 1 --2- 172.21.15.27:0/1665459997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c105d90 0x7f3f7c19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:50.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.880+0000 7f3f81600640 1 -- 172.21.15.27:0/1665459997 >> 172.21.15.27:0/1665459997 conn(0x7f3f7c101860 msgr2=0x7f3f7c10ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:50.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.880+0000 7f3f81600640 1 -- 172.21.15.27:0/1665459997 shutdown_connections 2024-09-21T00:11:50.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:48.880+0000 7f3f81600640 1 -- 172.21.15.27:0/1665459997 wait complete. 2024-09-21T00:11:51.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.092+0000 7fe935400640 1 -- 172.21.15.27:0/1441497735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9301049c0 msgr2=0x7fe930104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:51.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.092+0000 7fe935400640 1 --2- 172.21.15.27:0/1441497735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9301049c0 0x7fe930104dd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe920009900 tx=0x7fe92002a3f0 comp rx=0 tx=0).stop 2024-09-21T00:11:51.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.092+0000 7fe935400640 1 -- 172.21.15.27:0/1441497735 shutdown_connections 2024-09-21T00:11:51.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.092+0000 7fe935400640 1 --2- 172.21.15.27:0/1441497735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe93010de20 0x7fe930110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:51.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.092+0000 7fe935400640 1 --2- 172.21.15.27:0/1441497735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe930105310 0x7fe93010d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:51.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.092+0000 7fe935400640 1 --2- 172.21.15.27:0/1441497735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9301049c0 0x7fe930104dd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:51.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.092+0000 7fe935400640 1 -- 172.21.15.27:0/1441497735 >> 172.21.15.27:0/1441497735 conn(0x7fe930079730 msgr2=0x7fe930079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:51.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.092+0000 7fe935400640 1 -- 172.21.15.27:0/1441497735 shutdown_connections 2024-09-21T00:11:51.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.092+0000 7fe935400640 1 -- 172.21.15.27:0/1441497735 wait complete. 2024-09-21T00:11:51.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.093+0000 7fe935400640 1 Processor -- start 2024-09-21T00:11:51.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.094+0000 7fe935400640 1 -- start start 2024-09-21T00:11:51.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.094+0000 7fe935400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9301049c0 0x7fe93019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:51.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.094+0000 7fe935400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe930105310 0x7fe93019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:51.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.094+0000 7fe935400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe93010de20 0x7fe9301a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:51.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.094+0000 7fe935400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe930113a80 con 0x7fe9301049c0 2024-09-21T00:11:51.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.094+0000 7fe935400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe930113900 con 0x7fe930105310 2024-09-21T00:11:51.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.094+0000 7fe935400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe930113c00 con 0x7fe93010de20 2024-09-21T00:11:51.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.094+0000 7fe92e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9301049c0 0x7fe93019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:51.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.094+0000 7fe92e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9301049c0 0x7fe93019c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43786/0 (socket says 172.21.15.27:43786) 2024-09-21T00:11:51.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.094+0000 7fe92e800640 1 -- 172.21.15.27:0/4218632187 learned_addr learned my addr 172.21.15.27:0/4218632187 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:51.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.095+0000 7fe92de00640 1 --2- 172.21.15.27:0/4218632187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe930105310 0x7fe93019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:51.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.095+0000 7fe92f200640 1 --2- 172.21.15.27:0/4218632187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe93010de20 0x7fe9301a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:51.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.095+0000 7fe92e800640 1 -- 172.21.15.27:0/4218632187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe93010de20 msgr2=0x7fe9301a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:51.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.095+0000 7fe92e800640 1 --2- 172.21.15.27:0/4218632187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe93010de20 0x7fe9301a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:51.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.095+0000 7fe92e800640 1 -- 172.21.15.27:0/4218632187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe930105310 msgr2=0x7fe93019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:51.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.095+0000 7fe92e800640 1 --2- 172.21.15.27:0/4218632187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe930105310 0x7fe93019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:51.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.095+0000 7fe92e800640 1 -- 172.21.15.27:0/4218632187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9301a25e0 con 0x7fe9301049c0 2024-09-21T00:11:51.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.095+0000 7fe92de00640 1 --2- 172.21.15.27:0/4218632187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe930105310 0x7fe93019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:11:51.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.095+0000 7fe92e800640 1 --2- 172.21.15.27:0/4218632187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9301049c0 0x7fe93019c730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe920032fd0 tx=0x7fe920004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:51.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.095+0000 7fe927400640 1 -- 172.21.15.27:0/4218632187 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92003f030 con 0x7fe9301049c0 2024-09-21T00:11:51.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.095+0000 7fe935400640 1 -- 172.21.15.27:0/4218632187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9301a2870 con 0x7fe9301049c0 2024-09-21T00:11:51.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.095+0000 7fe927400640 1 -- 172.21.15.27:0/4218632187 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe920004490 con 0x7fe9301049c0 2024-09-21T00:11:51.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.095+0000 7fe927400640 1 -- 172.21.15.27:0/4218632187 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920035070 con 0x7fe9301049c0 2024-09-21T00:11:51.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.096+0000 7fe935400640 1 -- 172.21.15.27:0/4218632187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9301a2da0 con 0x7fe9301049c0 2024-09-21T00:11:51.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.097+0000 7fe935400640 1 -- 172.21.15.27:0/4218632187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe93006c1e0 con 0x7fe9301049c0 2024-09-21T00:11:51.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.101+0000 7fe927400640 1 -- 172.21.15.27:0/4218632187 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fe920042030 con 0x7fe9301049c0 2024-09-21T00:11:51.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.102+0000 7fe927400640 1 --2- 172.21.15.27:0/4218632187 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9140b6580 0x7fe9140b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:51.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.102+0000 7fe927400640 1 -- 172.21.15.27:0/4218632187 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe9201060a0 con 0x7fe9301049c0 2024-09-21T00:11:51.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.103+0000 7fe92de00640 1 --2- 172.21.15.27:0/4218632187 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9140b6580 0x7fe9140b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:51.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.103+0000 7fe92de00640 1 --2- 172.21.15.27:0/4218632187 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9140b6580 0x7fe9140b8a90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe93010fcd0 tx=0x7fe91c00a5c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:51.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.105+0000 7fe927400640 1 -- 172.21.15.27:0/4218632187 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe9301a2da0 con 0x7fe9301049c0 2024-09-21T00:11:51.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.309+0000 7fe935400640 1 -- 172.21.15.27:0/4218632187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fe9301a92a0 con 0x7fe9301049c0 2024-09-21T00:11:51.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.314+0000 7fe927400640 1 -- 172.21.15.27:0/4218632187 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v32)=0 v32) ==== 119+0+0 (secure 0 0 0) 0x7fe9200ce490 con 0x7fe9301049c0 2024-09-21T00:11:51.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.316+0000 7fe935400640 1 -- 172.21.15.27:0/4218632187 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9140b6580 msgr2=0x7fe9140b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:51.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.316+0000 7fe935400640 1 --2- 172.21.15.27:0/4218632187 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9140b6580 0x7fe9140b8a90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe93010fcd0 tx=0x7fe91c00a5c0 comp rx=0 tx=0).stop 2024-09-21T00:11:51.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.316+0000 7fe935400640 1 -- 172.21.15.27:0/4218632187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9301049c0 msgr2=0x7fe93019c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:51.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.316+0000 7fe935400640 1 --2- 172.21.15.27:0/4218632187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9301049c0 0x7fe93019c730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe920032fd0 tx=0x7fe920004270 comp rx=0 tx=0).stop 2024-09-21T00:11:51.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.316+0000 7fe935400640 1 -- 172.21.15.27:0/4218632187 shutdown_connections 2024-09-21T00:11:51.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.316+0000 7fe935400640 1 --2- 172.21.15.27:0/4218632187 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9140b6580 0x7fe9140b8a90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:51.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.317+0000 7fe935400640 1 --2- 172.21.15.27:0/4218632187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe93010de20 0x7fe9301a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:51.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.317+0000 7fe935400640 1 --2- 172.21.15.27:0/4218632187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe930105310 0x7fe93019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:51.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.317+0000 7fe935400640 1 --2- 172.21.15.27:0/4218632187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9301049c0 0x7fe93019c730 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:51.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.317+0000 7fe935400640 1 -- 172.21.15.27:0/4218632187 >> 172.21.15.27:0/4218632187 conn(0x7fe930079730 msgr2=0x7fe9300ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:51.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.317+0000 7fe935400640 1 -- 172.21.15.27:0/4218632187 shutdown_connections 2024-09-21T00:11:51.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:49.317+0000 7fe935400640 1 -- 172.21.15.27:0/4218632187 wait complete. 2024-09-21T00:11:52.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:52 smithi027 ceph-mon[30745]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:52.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:52 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:52.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:52 smithi089 ceph-mon[34481]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:52.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:52 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:52.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:52 smithi098 ceph-mon[34725]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:52.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:52 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:54.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:54 smithi027 ceph-mon[30745]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:54.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:54 smithi089 ceph-mon[34481]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:54.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:54 smithi098 ceph-mon[34725]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:55.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.569+0000 7f65bd600640 1 -- 172.21.15.27:0/401229762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65b80757b0 msgr2=0x7f65b8075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:55.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.569+0000 7f65bd600640 1 --2- 172.21.15.27:0/401229762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65b80757b0 0x7f65b8075bc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009900 tx=0x7f65a802a410 comp rx=0 tx=0).stop 2024-09-21T00:11:55.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.569+0000 7f65bd600640 1 -- 172.21.15.27:0/401229762 shutdown_connections 2024-09-21T00:11:55.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.569+0000 7f65bd600640 1 --2- 172.21.15.27:0/401229762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b810fef0 0x7f65b8112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.569+0000 7f65bd600640 1 --2- 172.21.15.27:0/401229762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b8076100 0x7f65b810f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.569+0000 7f65bd600640 1 --2- 172.21.15.27:0/401229762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65b80757b0 0x7f65b8075bc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.569+0000 7f65bd600640 1 -- 172.21.15.27:0/401229762 >> 172.21.15.27:0/401229762 conn(0x7f65b80ff440 msgr2=0x7f65b8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.569+0000 7f65bd600640 1 -- 172.21.15.27:0/401229762 shutdown_connections 2024-09-21T00:11:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.569+0000 7f65bd600640 1 -- 172.21.15.27:0/401229762 wait complete. 2024-09-21T00:11:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.570+0000 7f65bd600640 1 Processor -- start 2024-09-21T00:11:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.571+0000 7f65bd600640 1 -- start start 2024-09-21T00:11:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.571+0000 7f65bd600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65b80757b0 0x7f65b819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.571+0000 7f65bd600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b8076100 0x7f65b819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.571+0000 7f65bd600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b810fef0 0x7f65b81a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.571+0000 7f65bd600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8115af0 con 0x7f65b80757b0 2024-09-21T00:11:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.571+0000 7f65bd600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8115970 con 0x7f65b810fef0 2024-09-21T00:11:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.571+0000 7f65bd600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8115c70 con 0x7f65b8076100 2024-09-21T00:11:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.571+0000 7f65b7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65b80757b0 0x7f65b819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.571+0000 7f65b7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65b80757b0 0x7f65b819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52324/0 (socket says 172.21.15.27:52324) 2024-09-21T00:11:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.571+0000 7f65b7e00640 1 -- 172.21.15.27:0/3871371164 learned_addr learned my addr 172.21.15.27:0/3871371164 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.571+0000 7f65afe00640 1 --2- 172.21.15.27:0/3871371164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b8076100 0x7f65b819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.571+0000 7f65bcc00640 1 --2- 172.21.15.27:0/3871371164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b810fef0 0x7f65b81a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.572+0000 7f65b7e00640 1 -- 172.21.15.27:0/3871371164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b8076100 msgr2=0x7f65b819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.572+0000 7f65b7e00640 1 --2- 172.21.15.27:0/3871371164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b8076100 0x7f65b819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.572+0000 7f65b7e00640 1 -- 172.21.15.27:0/3871371164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b810fef0 msgr2=0x7f65b81a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.572+0000 7f65b7e00640 1 --2- 172.21.15.27:0/3871371164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b810fef0 0x7f65b81a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.572+0000 7f65b7e00640 1 -- 172.21.15.27:0/3871371164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b81a23d0 con 0x7f65b80757b0 2024-09-21T00:11:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.572+0000 7f65bcc00640 1 --2- 172.21.15.27:0/3871371164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b810fef0 0x7f65b81a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:11:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.572+0000 7f65b7e00640 1 --2- 172.21.15.27:0/3871371164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65b80757b0 0x7f65b819c460 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f65a8005ab0 tx=0x7f65a8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.572+0000 7f65b5600640 1 -- 172.21.15.27:0/3871371164 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a803f070 con 0x7f65b80757b0 2024-09-21T00:11:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.572+0000 7f65b5600640 1 -- 172.21.15.27:0/3871371164 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a8031b60 con 0x7f65b80757b0 2024-09-21T00:11:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.572+0000 7f65b5600640 1 -- 172.21.15.27:0/3871371164 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8035030 con 0x7f65b80757b0 2024-09-21T00:11:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.573+0000 7f65bd600640 1 -- 172.21.15.27:0/3871371164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b81a2660 con 0x7f65b80757b0 2024-09-21T00:11:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.573+0000 7f65bd600640 1 -- 172.21.15.27:0/3871371164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b81a8e20 con 0x7f65b80757b0 2024-09-21T00:11:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.574+0000 7f65bd600640 1 -- 172.21.15.27:0/3871371164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b8075bc0 con 0x7f65b80757b0 2024-09-21T00:11:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.576+0000 7f65b5600640 1 -- 172.21.15.27:0/3871371164 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f65a8042070 con 0x7f65b80757b0 2024-09-21T00:11:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.578+0000 7f65b5600640 1 --2- 172.21.15.27:0/3871371164 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f659c0b64d0 0x7f659c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.578+0000 7f65b5600640 1 -- 172.21.15.27:0/3871371164 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f65a8106570 con 0x7f65b80757b0 2024-09-21T00:11:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.578+0000 7f65afe00640 1 --2- 172.21.15.27:0/3871371164 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f659c0b64d0 0x7f659c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.579+0000 7f65afe00640 1 --2- 172.21.15.27:0/3871371164 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f659c0b64d0 0x7f659c0b89e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f65b81121e0 tx=0x7f65a0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.580+0000 7f65b5600640 1 -- 172.21.15.27:0/3871371164 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f65a80d2d40 con 0x7f65b80757b0 2024-09-21T00:11:55.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.779+0000 7f65bd600640 1 -- 172.21.15.27:0/3871371164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f65b81a9050 con 0x7f65b80757b0 2024-09-21T00:11:55.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.784+0000 7f65b5600640 1 -- 172.21.15.27:0/3871371164 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v34)=0 v34) ==== 121+0+0 (secure 0 0 0) 0x7f65a80d7c80 con 0x7f65b80757b0 2024-09-21T00:11:55.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.787+0000 7f65bd600640 1 -- 172.21.15.27:0/3871371164 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f659c0b64d0 msgr2=0x7f659c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:55.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.787+0000 7f65bd600640 1 --2- 172.21.15.27:0/3871371164 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f659c0b64d0 0x7f659c0b89e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f65b81121e0 tx=0x7f65a0005f90 comp rx=0 tx=0).stop 2024-09-21T00:11:55.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.787+0000 7f65bd600640 1 -- 172.21.15.27:0/3871371164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65b80757b0 msgr2=0x7f65b819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:55.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.787+0000 7f65bd600640 1 --2- 172.21.15.27:0/3871371164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65b80757b0 0x7f65b819c460 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f65a8005ab0 tx=0x7f65a8003c10 comp rx=0 tx=0).stop 2024-09-21T00:11:55.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.787+0000 7f65bd600640 1 -- 172.21.15.27:0/3871371164 shutdown_connections 2024-09-21T00:11:55.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.787+0000 7f65bd600640 1 --2- 172.21.15.27:0/3871371164 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f659c0b64d0 0x7f659c0b89e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.787+0000 7f65bd600640 1 --2- 172.21.15.27:0/3871371164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b810fef0 0x7f65b81a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.787+0000 7f65bd600640 1 --2- 172.21.15.27:0/3871371164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b8076100 0x7f65b819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.787+0000 7f65bd600640 1 --2- 172.21.15.27:0/3871371164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65b80757b0 0x7f65b819c460 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.787+0000 7f65bd600640 1 -- 172.21.15.27:0/3871371164 >> 172.21.15.27:0/3871371164 conn(0x7f65b80ff440 msgr2=0x7f65b8109a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.787+0000 7f65bd600640 1 -- 172.21.15.27:0/3871371164 shutdown_connections 2024-09-21T00:11:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:55.787+0000 7f65bd600640 1 -- 172.21.15.27:0/3871371164 wait complete. 2024-09-21T00:11:55.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.659+0000 7f07d4c00640 1 -- 172.21.15.27:0/1082920367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07d010de30 msgr2=0x7f07d0110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:55.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.659+0000 7f07d4c00640 1 --2- 172.21.15.27:0/1082920367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07d010de30 0x7f07d0110320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009ca0 tx=0x7f07c802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.659+0000 7f07d4c00640 1 -- 172.21.15.27:0/1082920367 shutdown_connections 2024-09-21T00:11:55.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.659+0000 7f07d4c00640 1 --2- 172.21.15.27:0/1082920367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07d010de30 0x7f07d0110320 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.659+0000 7f07d4c00640 1 --2- 172.21.15.27:0/1082920367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d0105320 0x7f07d010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.659+0000 7f07d4c00640 1 --2- 172.21.15.27:0/1082920367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07d01049d0 0x7f07d0104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.659+0000 7f07d4c00640 1 -- 172.21.15.27:0/1082920367 >> 172.21.15.27:0/1082920367 conn(0x7f07d0079740 msgr2=0x7f07d0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:55.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.659+0000 7f07d4c00640 1 -- 172.21.15.27:0/1082920367 shutdown_connections 2024-09-21T00:11:55.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.659+0000 7f07d4c00640 1 -- 172.21.15.27:0/1082920367 wait complete. 2024-09-21T00:11:55.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.660+0000 7f07d4c00640 1 Processor -- start 2024-09-21T00:11:55.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.660+0000 7f07d4c00640 1 -- start start 2024-09-21T00:11:55.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.661+0000 7f07d4c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07d01049d0 0x7f07d019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:55.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.661+0000 7f07d4c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d0105320 0x7f07d019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:55.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.661+0000 7f07d4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07d010de30 0x7f07d019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:55.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.661+0000 7f07d4c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f07d0113b00 con 0x7f07d010de30 2024-09-21T00:11:55.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.661+0000 7f07d4c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f07d0113980 con 0x7f07d0105320 2024-09-21T00:11:55.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.661+0000 7f07d4c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f07d0113c80 con 0x7f07d01049d0 2024-09-21T00:11:55.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.661+0000 7f07cde00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07d01049d0 0x7f07d019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:55.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.661+0000 7f07cde00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07d01049d0 0x7f07d019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:35072/0 (socket says 172.21.15.27:35072) 2024-09-21T00:11:55.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.661+0000 7f07cde00640 1 -- 172.21.15.27:0/1184944252 learned_addr learned my addr 172.21.15.27:0/1184944252 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:55.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.661+0000 7f07ce800640 1 --2- 172.21.15.27:0/1184944252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07d010de30 0x7f07d019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:55.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.661+0000 7f07cd400640 1 --2- 172.21.15.27:0/1184944252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d0105320 0x7f07d019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:55.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.662+0000 7f07cd400640 1 -- 172.21.15.27:0/1184944252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07d01049d0 msgr2=0x7f07d019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:55.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.662+0000 7f07cd400640 1 --2- 172.21.15.27:0/1184944252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07d01049d0 0x7f07d019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.662+0000 7f07cd400640 1 -- 172.21.15.27:0/1184944252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07d010de30 msgr2=0x7f07d019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:55.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.662+0000 7f07cd400640 1 --2- 172.21.15.27:0/1184944252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07d010de30 0x7f07d019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.662+0000 7f07cd400640 1 -- 172.21.15.27:0/1184944252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d01a04a0 con 0x7f07d0105320 2024-09-21T00:11:55.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.662+0000 7f07cde00640 1 --2- 172.21.15.27:0/1184944252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07d01049d0 0x7f07d019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:11:55.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.662+0000 7f07cd400640 1 --2- 172.21.15.27:0/1184944252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d0105320 0x7f07d019aad0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f07d006b950 tx=0x7f07bc00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:55.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.662+0000 7f07c6a00640 1 -- 172.21.15.27:0/1184944252 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc00cc40 con 0x7f07d0105320 2024-09-21T00:11:55.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.662+0000 7f07d4c00640 1 -- 172.21.15.27:0/1184944252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d01a0790 con 0x7f07d0105320 2024-09-21T00:11:55.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.662+0000 7f07c6a00640 1 -- 172.21.15.27:0/1184944252 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07bc00cde0 con 0x7f07d0105320 2024-09-21T00:11:55.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.662+0000 7f07c6a00640 1 -- 172.21.15.27:0/1184944252 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc018760 con 0x7f07d0105320 2024-09-21T00:11:55.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.663+0000 7f07d4c00640 1 -- 172.21.15.27:0/1184944252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d01a6ef0 con 0x7f07d0105320 2024-09-21T00:11:55.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.663+0000 7f07d4c00640 1 -- 172.21.15.27:0/1184944252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d006c240 con 0x7f07d0105320 2024-09-21T00:11:55.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.666+0000 7f07c6a00640 1 -- 172.21.15.27:0/1184944252 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f07bc010a90 con 0x7f07d0105320 2024-09-21T00:11:55.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.667+0000 7f07c6a00640 1 --2- 172.21.15.27:0/1184944252 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f07b40b64d0 0x7f07b40b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:55.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.668+0000 7f07c6a00640 1 -- 172.21.15.27:0/1184944252 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f07bc0e5e70 con 0x7f07d0105320 2024-09-21T00:11:55.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.668+0000 7f07cde00640 1 --2- 172.21.15.27:0/1184944252 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f07b40b64d0 0x7f07b40b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:55.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.668+0000 7f07cde00640 1 --2- 172.21.15.27:0/1184944252 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f07b40b64d0 0x7f07b40b89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f07c00058c0 tx=0x7f07c0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:55.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.669+0000 7f07c6a00640 1 -- 172.21.15.27:0/1184944252 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f07bc0b2640 con 0x7f07d0105320 2024-09-21T00:11:55.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.888+0000 7f07d4c00640 1 -- 172.21.15.27:0/1184944252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f07d0104de0 con 0x7f07d0105320 2024-09-21T00:11:55.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.895+0000 7f07c6a00640 1 -- 172.21.15.27:0/1184944252 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v33)=0 v33) ==== 122+0+0 (secure 0 0 0) 0x7f07d0104de0 con 0x7f07d0105320 2024-09-21T00:11:55.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.898+0000 7f07d4c00640 1 -- 172.21.15.27:0/1184944252 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f07b40b64d0 msgr2=0x7f07b40b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:55.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.898+0000 7f07d4c00640 1 --2- 172.21.15.27:0/1184944252 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f07b40b64d0 0x7f07b40b89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f07c00058c0 tx=0x7f07c0005f90 comp rx=0 tx=0).stop 2024-09-21T00:11:55.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.898+0000 7f07d4c00640 1 -- 172.21.15.27:0/1184944252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d0105320 msgr2=0x7f07d019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:55.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.898+0000 7f07d4c00640 1 --2- 172.21.15.27:0/1184944252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d0105320 0x7f07d019aad0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f07d006b950 tx=0x7f07bc00ee50 comp rx=0 tx=0).stop 2024-09-21T00:11:55.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.898+0000 7f07d4c00640 1 -- 172.21.15.27:0/1184944252 shutdown_connections 2024-09-21T00:11:55.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.898+0000 7f07d4c00640 1 --2- 172.21.15.27:0/1184944252 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f07b40b64d0 0x7f07b40b89e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.898+0000 7f07d4c00640 1 --2- 172.21.15.27:0/1184944252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07d010de30 0x7f07d019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.898+0000 7f07d4c00640 1 --2- 172.21.15.27:0/1184944252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d0105320 0x7f07d019aad0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.898+0000 7f07d4c00640 1 --2- 172.21.15.27:0/1184944252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07d01049d0 0x7f07d019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:55.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.898+0000 7f07d4c00640 1 -- 172.21.15.27:0/1184944252 >> 172.21.15.27:0/1184944252 conn(0x7f07d0079740 msgr2=0x7f07d00ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:55.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.898+0000 7f07d4c00640 1 -- 172.21.15.27:0/1184944252 shutdown_connections 2024-09-21T00:11:55.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:51.898+0000 7f07d4c00640 1 -- 172.21.15.27:0/1184944252 wait complete. 2024-09-21T00:11:56.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:56 smithi027 ceph-mon[30745]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:56.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:56 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3871371164' entity='client.admin' 2024-09-21T00:11:56.476 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:56 smithi098 ceph-mon[34725]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:56.477 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:56 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3871371164' entity='client.admin' 2024-09-21T00:11:56.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:56 smithi089 ceph-mon[34481]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:56.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:56 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3871371164' entity='client.admin' 2024-09-21T00:11:58.194 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:11:58.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:58 smithi027 ceph-mon[30745]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:58.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:11:58 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:11:58.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:58 smithi089 ceph-mon[34481]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:58.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:11:58 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:11:58.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:58 smithi098 ceph-mon[34725]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:11:58.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:11:58 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:11:58.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.809+0000 7fc3f0200640 1 -- 172.21.15.27:0/47179738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8114430 msgr2=0x7fc3e8116900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:58.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.809+0000 7fc3f0200640 1 --2- 172.21.15.27:0/47179738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8114430 0x7fc3e8116900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009ca0 tx=0x7fc3e002a8c0 comp rx=0 tx=0).stop 2024-09-21T00:11:58.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.809+0000 7fc3f0200640 1 -- 172.21.15.27:0/47179738 shutdown_connections 2024-09-21T00:11:58.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.809+0000 7fc3f0200640 1 --2- 172.21.15.27:0/47179738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8114430 0x7fc3e8116900 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:58.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.809+0000 7fc3f0200640 1 --2- 172.21.15.27:0/47179738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3e8075dd0 0x7fc3e80761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:58.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.809+0000 7fc3f0200640 1 --2- 172.21.15.27:0/47179738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3e8077380 0x7fc3e80757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:58.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.809+0000 7fc3f0200640 1 -- 172.21.15.27:0/47179738 >> 172.21.15.27:0/47179738 conn(0x7fc3e807b850 msgr2=0x7fc3e807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:58.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.809+0000 7fc3f0200640 1 -- 172.21.15.27:0/47179738 shutdown_connections 2024-09-21T00:11:58.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.809+0000 7fc3f0200640 1 -- 172.21.15.27:0/47179738 wait complete. 2024-09-21T00:11:58.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.810+0000 7fc3f0200640 1 Processor -- start 2024-09-21T00:11:58.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.810+0000 7fc3f0200640 1 -- start start 2024-09-21T00:11:58.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3f0200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3e8075dd0 0x7fc3e81a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:58.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3f0200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8077380 0x7fc3e81a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:58.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3f0200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3e8114430 0x7fc3e81a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:58.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3f0200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e811a070 con 0x7fc3e8077380 2024-09-21T00:11:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3f0200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8119ef0 con 0x7fc3e8114430 2024-09-21T00:11:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3f0200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e811a1f0 con 0x7fc3e8075dd0 2024-09-21T00:11:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3ece00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8077380 0x7fc3e81a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3ee200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3e8114430 0x7fc3e81a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3ece00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8077380 0x7fc3e81a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52328/0 (socket says 172.21.15.27:52328) 2024-09-21T00:11:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3ece00640 1 -- 172.21.15.27:0/422761889 learned_addr learned my addr 172.21.15.27:0/422761889 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3ed800640 1 --2- 172.21.15.27:0/422761889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3e8075dd0 0x7fc3e81a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3ee200640 1 -- 172.21.15.27:0/422761889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3e8075dd0 msgr2=0x7fc3e81a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3ee200640 1 --2- 172.21.15.27:0/422761889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3e8075dd0 0x7fc3e81a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3ee200640 1 -- 172.21.15.27:0/422761889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8077380 msgr2=0x7fc3e81a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3ee200640 1 --2- 172.21.15.27:0/422761889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8077380 0x7fc3e81a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3ee200640 1 -- 172.21.15.27:0/422761889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e81a6950 con 0x7fc3e8114430 2024-09-21T00:11:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.811+0000 7fc3ece00640 1 --2- 172.21.15.27:0/422761889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8077380 0x7fc3e81a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:11:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.812+0000 7fc3ee200640 1 --2- 172.21.15.27:0/422761889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3e8114430 0x7fc3e81a6220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc3e002ade0 tx=0x7fc3e0031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.812+0000 7fc3e6000640 1 -- 172.21.15.27:0/422761889 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0003b70 con 0x7fc3e8114430 2024-09-21T00:11:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.812+0000 7fc3e6000640 1 -- 172.21.15.27:0/422761889 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3e0031910 con 0x7fc3e8114430 2024-09-21T00:11:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.812+0000 7fc3f0200640 1 -- 172.21.15.27:0/422761889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e81a6be0 con 0x7fc3e8114430 2024-09-21T00:11:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.812+0000 7fc3e6000640 1 -- 172.21.15.27:0/422761889 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e002f950 con 0x7fc3e8114430 2024-09-21T00:11:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.812+0000 7fc3f0200640 1 -- 172.21.15.27:0/422761889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e81ad3a0 con 0x7fc3e8114430 2024-09-21T00:11:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.813+0000 7fc3f0200640 1 -- 172.21.15.27:0/422761889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e80761e0 con 0x7fc3e8114430 2024-09-21T00:11:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.815+0000 7fc3e6000640 1 -- 172.21.15.27:0/422761889 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fc3e0004b40 con 0x7fc3e8114430 2024-09-21T00:11:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.817+0000 7fc3e6000640 1 --2- 172.21.15.27:0/422761889 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc3d00b6550 0x7fc3d00b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.817+0000 7fc3e6000640 1 -- 172.21.15.27:0/422761889 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc3e0106940 con 0x7fc3e8114430 2024-09-21T00:11:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.817+0000 7fc3ed800640 1 --2- 172.21.15.27:0/422761889 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc3d00b6550 0x7fc3d00b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.818+0000 7fc3ed800640 1 --2- 172.21.15.27:0/422761889 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc3d00b6550 0x7fc3d00b8a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009bd0 tx=0x7fc3dc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:57.819+0000 7fc3e6000640 1 -- 172.21.15.27:0/422761889 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3e00ced30 con 0x7fc3e8114430 2024-09-21T00:11:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:58.018+0000 7fc3f0200640 1 -- 172.21.15.27:0/422761889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fc3e81ad5d0 con 0x7fc3e8114430 2024-09-21T00:11:58.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:58.025+0000 7fc3e6000640 1 -- 172.21.15.27:0/422761889 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) ==== 138+0+0 (secure 0 0 0) 0x7fc3e0003d10 con 0x7fc3e8114430 2024-09-21T00:11:58.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:58.028+0000 7fc3f0200640 1 -- 172.21.15.27:0/422761889 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc3d00b6550 msgr2=0x7fc3d00b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:58.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:58.028+0000 7fc3f0200640 1 --2- 172.21.15.27:0/422761889 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc3d00b6550 0x7fc3d00b8a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009bd0 tx=0x7fc3dc005f90 comp rx=0 tx=0).stop 2024-09-21T00:11:58.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:58.028+0000 7fc3f0200640 1 -- 172.21.15.27:0/422761889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3e8114430 msgr2=0x7fc3e81a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:58.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:58.028+0000 7fc3f0200640 1 --2- 172.21.15.27:0/422761889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3e8114430 0x7fc3e81a6220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc3e002ade0 tx=0x7fc3e0031660 comp rx=0 tx=0).stop 2024-09-21T00:11:58.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:58.028+0000 7fc3f0200640 1 -- 172.21.15.27:0/422761889 shutdown_connections 2024-09-21T00:11:58.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:58.028+0000 7fc3f0200640 1 --2- 172.21.15.27:0/422761889 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc3d00b6550 0x7fc3d00b8a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:58.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:58.028+0000 7fc3f0200640 1 --2- 172.21.15.27:0/422761889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3e8114430 0x7fc3e81a6220 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:58.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:58.028+0000 7fc3f0200640 1 --2- 172.21.15.27:0/422761889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8077380 0x7fc3e81a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:58.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:58.028+0000 7fc3f0200640 1 --2- 172.21.15.27:0/422761889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3e8075dd0 0x7fc3e81a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:58.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:58.028+0000 7fc3f0200640 1 -- 172.21.15.27:0/422761889 >> 172.21.15.27:0/422761889 conn(0x7fc3e807b850 msgr2=0x7fc3e8113040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:58.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:58.028+0000 7fc3f0200640 1 -- 172.21.15.27:0/422761889 shutdown_connections 2024-09-21T00:11:58.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:58.028+0000 7fc3f0200640 1 -- 172.21.15.27:0/422761889 wait complete. 2024-09-21T00:11:59.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.662+0000 7f6b87200640 1 -- 172.21.15.27:0/4279485644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8810b4d0 msgr2=0x7f6b8810b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:59.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.662+0000 7f6b87200640 1 --2- 172.21.15.27:0/4279485644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8810b4d0 0x7f6b8810b8e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009900 tx=0x7f6b7402a410 comp rx=0 tx=0).stop 2024-09-21T00:11:59.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.662+0000 7f6b87200640 1 -- 172.21.15.27:0/4279485644 shutdown_connections 2024-09-21T00:11:59.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.662+0000 7f6b87200640 1 --2- 172.21.15.27:0/4279485644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b8810be20 0x7f6b88112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:59.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.662+0000 7f6b87200640 1 --2- 172.21.15.27:0/4279485644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8810b4d0 0x7f6b8810b8e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:59.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.662+0000 7f6b87200640 1 --2- 172.21.15.27:0/4279485644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b88105b90 0x7f6b88105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:59.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.662+0000 7f6b87200640 1 -- 172.21.15.27:0/4279485644 >> 172.21.15.27:0/4279485644 conn(0x7f6b881015e0 msgr2=0x7f6b88103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:59.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.662+0000 7f6b87200640 1 -- 172.21.15.27:0/4279485644 shutdown_connections 2024-09-21T00:11:59.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.662+0000 7f6b87200640 1 -- 172.21.15.27:0/4279485644 wait complete. 2024-09-21T00:11:59.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.664+0000 7f6b87200640 1 Processor -- start 2024-09-21T00:11:59.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.664+0000 7f6b87200640 1 -- start start 2024-09-21T00:11:59.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.664+0000 7f6b87200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b88105b90 0x7f6b8819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:59.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.664+0000 7f6b87200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8810b4d0 0x7f6b8819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:59.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.664+0000 7f6b87200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b8810be20 0x7f6b881a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:59.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.664+0000 7f6b87200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6b88115ae0 con 0x7f6b8810b4d0 2024-09-21T00:11:59.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.664+0000 7f6b87200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6b88115960 con 0x7f6b88105b90 2024-09-21T00:11:59.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.664+0000 7f6b87200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6b88115c60 con 0x7f6b8810be20 2024-09-21T00:11:59.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.664+0000 7f6b85400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8810b4d0 0x7f6b8819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:59.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.664+0000 7f6b85e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b88105b90 0x7f6b8819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.664+0000 7f6b85400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8810b4d0 0x7f6b8819cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52348/0 (socket says 172.21.15.27:52348) 2024-09-21T00:11:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.665+0000 7f6b85400640 1 -- 172.21.15.27:0/3809537454 learned_addr learned my addr 172.21.15.27:0/3809537454 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:11:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.664+0000 7f6b85e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b88105b90 0x7f6b8819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49240/0 (socket says 172.21.15.27:49240) 2024-09-21T00:11:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.665+0000 7f6b86800640 1 --2- 172.21.15.27:0/3809537454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b8810be20 0x7f6b881a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.665+0000 7f6b85400640 1 -- 172.21.15.27:0/3809537454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b8810be20 msgr2=0x7f6b881a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.665+0000 7f6b85400640 1 --2- 172.21.15.27:0/3809537454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b8810be20 0x7f6b881a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.665+0000 7f6b85400640 1 -- 172.21.15.27:0/3809537454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b88105b90 msgr2=0x7f6b8819c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.665+0000 7f6b85400640 1 --2- 172.21.15.27:0/3809537454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b88105b90 0x7f6b8819c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.665+0000 7f6b85400640 1 -- 172.21.15.27:0/3809537454 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b881a24a0 con 0x7f6b8810b4d0 2024-09-21T00:11:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.665+0000 7f6b85400640 1 --2- 172.21.15.27:0/3809537454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8810b4d0 0x7f6b8819cad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6b7402a930 tx=0x7f6b74031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.666+0000 7f6b7ea00640 1 -- 172.21.15.27:0/3809537454 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7403f030 con 0x7f6b8810b4d0 2024-09-21T00:11:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.666+0000 7f6b7ea00640 1 -- 172.21.15.27:0/3809537454 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b740043d0 con 0x7f6b8810b4d0 2024-09-21T00:11:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.666+0000 7f6b87200640 1 -- 172.21.15.27:0/3809537454 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b881a2730 con 0x7f6b8810b4d0 2024-09-21T00:11:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.666+0000 7f6b7ea00640 1 -- 172.21.15.27:0/3809537454 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74035070 con 0x7f6b8810b4d0 2024-09-21T00:11:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.666+0000 7f6b87200640 1 -- 172.21.15.27:0/3809537454 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b881a8ef0 con 0x7f6b8810b4d0 2024-09-21T00:11:59.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.667+0000 7f6b87200640 1 -- 172.21.15.27:0/3809537454 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b88106e10 con 0x7f6b8810b4d0 2024-09-21T00:11:59.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.670+0000 7f6b7ea00640 1 -- 172.21.15.27:0/3809537454 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f6b74042030 con 0x7f6b8810b4d0 2024-09-21T00:11:59.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.671+0000 7f6b7ea00640 1 --2- 172.21.15.27:0/3809537454 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6b6c0b6550 0x7f6b6c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:11:59.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.671+0000 7f6b7ea00640 1 -- 172.21.15.27:0/3809537454 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f6b74105e30 con 0x7f6b8810b4d0 2024-09-21T00:11:59.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.671+0000 7f6b85e00640 1 --2- 172.21.15.27:0/3809537454 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6b6c0b6550 0x7f6b6c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:11:59.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.672+0000 7f6b85e00640 1 --2- 172.21.15.27:0/3809537454 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6b6c0b6550 0x7f6b6c0b8a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009bd0 tx=0x7f6b78005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:11:59.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.674+0000 7f6b7ea00640 1 -- 172.21.15.27:0/3809537454 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6b740ce220 con 0x7f6b8810b4d0 2024-09-21T00:11:59.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.874+0000 7f6b87200640 1 -- 172.21.15.27:0/3809537454 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f6b8810dc50 con 0x7f6b8810b4d0 2024-09-21T00:11:59.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.878+0000 7f6b7ea00640 1 -- 172.21.15.27:0/3809537454 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v36)=0 v36) ==== 111+0+0 (secure 0 0 0) 0x7f6b740d2600 con 0x7f6b8810b4d0 2024-09-21T00:11:59.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.881+0000 7f6b87200640 1 -- 172.21.15.27:0/3809537454 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6b6c0b6550 msgr2=0x7f6b6c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:59.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.881+0000 7f6b87200640 1 --2- 172.21.15.27:0/3809537454 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6b6c0b6550 0x7f6b6c0b8a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009bd0 tx=0x7f6b78005f90 comp rx=0 tx=0).stop 2024-09-21T00:11:59.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.881+0000 7f6b87200640 1 -- 172.21.15.27:0/3809537454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8810b4d0 msgr2=0x7f6b8819cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:11:59.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.881+0000 7f6b87200640 1 --2- 172.21.15.27:0/3809537454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8810b4d0 0x7f6b8819cad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6b7402a930 tx=0x7f6b74031a20 comp rx=0 tx=0).stop 2024-09-21T00:11:59.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.881+0000 7f6b87200640 1 -- 172.21.15.27:0/3809537454 shutdown_connections 2024-09-21T00:11:59.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.881+0000 7f6b87200640 1 --2- 172.21.15.27:0/3809537454 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6b6c0b6550 0x7f6b6c0b8a60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:59.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.881+0000 7f6b87200640 1 --2- 172.21.15.27:0/3809537454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b8810be20 0x7f6b881a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:59.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.881+0000 7f6b87200640 1 --2- 172.21.15.27:0/3809537454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8810b4d0 0x7f6b8819cad0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:59.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.881+0000 7f6b87200640 1 --2- 172.21.15.27:0/3809537454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b88105b90 0x7f6b8819c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:11:59.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.881+0000 7f6b87200640 1 -- 172.21.15.27:0/3809537454 >> 172.21.15.27:0/3809537454 conn(0x7f6b881015e0 msgr2=0x7f6b88103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:11:59.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.882+0000 7f6b87200640 1 -- 172.21.15.27:0/3809537454 shutdown_connections 2024-09-21T00:11:59.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:11:59.882+0000 7f6b87200640 1 -- 172.21.15.27:0/3809537454 wait complete. 2024-09-21T00:12:00.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:00 smithi027 ceph-mon[30745]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:00.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:00 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3809537454' entity='client.admin' 2024-09-21T00:12:00.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:00 smithi089 ceph-mon[34481]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:00.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:00 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3809537454' entity='client.admin' 2024-09-21T00:12:00.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:00 smithi098 ceph-mon[34725]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:00.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:00 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3809537454' entity='client.admin' 2024-09-21T00:12:01.299 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi027') 2024-09-21T00:12:01.299 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi027') 2024-09-21T00:12:01.299 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi027') 2024-09-21T00:12:01.299 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi027') 2024-09-21T00:12:01.299 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi027') 2024-09-21T00:12:01.300 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi027') 2024-09-21T00:12:01.300 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi027') 2024-09-21T00:12:01.300 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi027') 2024-09-21T00:12:01.300 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi027') 2024-09-21T00:12:01.300 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi027') 2024-09-21T00:12:01.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:01 smithi027 ceph-mon[30745]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:01.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:01 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:01.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:01 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:01.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:01 smithi027 ceph-mon[30745]: Reconfiguring mon.c (monmap changed)... 2024-09-21T00:12:01.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:01 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:12:01.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:01 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T00:12:01.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:01 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:01.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:01 smithi027 ceph-mon[30745]: Reconfiguring daemon mon.c on smithi098 2024-09-21T00:12:01.834 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:01 smithi098 ceph-mon[34725]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:01.835 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:01 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:01.835 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:01 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:01.835 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:01 smithi098 ceph-mon[34725]: Reconfiguring mon.c (monmap changed)... 2024-09-21T00:12:01.835 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:01 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:12:01.835 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:01 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T00:12:01.835 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:01 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:01.835 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:01 smithi098 ceph-mon[34725]: Reconfiguring daemon mon.c on smithi098 2024-09-21T00:12:01.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:01 smithi089 ceph-mon[34481]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:01.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:01 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:01.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:01 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:01.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:01 smithi089 ceph-mon[34481]: Reconfiguring mon.c (monmap changed)... 2024-09-21T00:12:01.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:01 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T00:12:01.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:01 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T00:12:01.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:01 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:01.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:01 smithi089 ceph-mon[34481]: Reconfiguring daemon mon.c on smithi098 2024-09-21T00:12:02.183 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi027') 2024-09-21T00:12:02.183 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi027') 2024-09-21T00:12:02.183 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi027') 2024-09-21T00:12:02.184 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi027') 2024-09-21T00:12:02.184 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi027') 2024-09-21T00:12:02.184 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi027') 2024-09-21T00:12:02.184 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi027') 2024-09-21T00:12:02.184 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi027') 2024-09-21T00:12:02.184 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi027') 2024-09-21T00:12:02.184 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi027') 2024-09-21T00:12:02.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.733+0000 7f59da400640 1 -- 172.21.15.27:0/3916642522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d4109e10 msgr2=0x7f59d4110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:02.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.733+0000 7f59da400640 1 --2- 172.21.15.27:0/3916642522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d4109e10 0x7f59d4110640 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009ca0 tx=0x7f59cc02a8c0 comp rx=0 tx=0).stop 2024-09-21T00:12:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.733+0000 7f59da400640 1 -- 172.21.15.27:0/3916642522 shutdown_connections 2024-09-21T00:12:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.733+0000 7f59da400640 1 --2- 172.21.15.27:0/3916642522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d4109e10 0x7f59d4110640 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.733+0000 7f59da400640 1 --2- 172.21.15.27:0/3916642522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59d41094c0 0x7f59d41098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.733+0000 7f59da400640 1 --2- 172.21.15.27:0/3916642522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59d4103b80 0x7f59d4103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.733+0000 7f59da400640 1 -- 172.21.15.27:0/3916642522 >> 172.21.15.27:0/3916642522 conn(0x7f59d40ff5d0 msgr2=0x7f59d4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:12:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.733+0000 7f59da400640 1 -- 172.21.15.27:0/3916642522 shutdown_connections 2024-09-21T00:12:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.733+0000 7f59da400640 1 -- 172.21.15.27:0/3916642522 wait complete. 2024-09-21T00:12:02.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.734+0000 7f59da400640 1 Processor -- start 2024-09-21T00:12:02.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.734+0000 7f59da400640 1 -- start start 2024-09-21T00:12:02.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59da400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59d4103b80 0x7f59d419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:02.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59da400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59d41094c0 0x7f59d419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:02.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59da400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d4109e10 0x7f59d41a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59da400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f59d4113b20 con 0x7f59d4109e10 2024-09-21T00:12:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59da400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f59d41139a0 con 0x7f59d4103b80 2024-09-21T00:12:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59da400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f59d4113ca0 con 0x7f59d41094c0 2024-09-21T00:12:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59d3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d4109e10 0x7f59d41a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59d3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59d4103b80 0x7f59d419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59d3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59d4103b80 0x7f59d419c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49258/0 (socket says 172.21.15.27:49258) 2024-09-21T00:12:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59d3400640 1 -- 172.21.15.27:0/3203533537 learned_addr learned my addr 172.21.15.27:0/3203533537 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:12:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59d2a00640 1 --2- 172.21.15.27:0/3203533537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59d41094c0 0x7f59d419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:02.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59d3400640 1 -- 172.21.15.27:0/3203533537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59d41094c0 msgr2=0x7f59d419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:02.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59d3400640 1 --2- 172.21.15.27:0/3203533537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59d41094c0 0x7f59d419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:02.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59d3400640 1 -- 172.21.15.27:0/3203533537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d4109e10 msgr2=0x7f59d41a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:02.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59d3400640 1 --2- 172.21.15.27:0/3203533537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d4109e10 0x7f59d41a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:02.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59d3400640 1 -- 172.21.15.27:0/3203533537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d41a2620 con 0x7f59d4103b80 2024-09-21T00:12:02.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.735+0000 7f59d3e00640 1 --2- 172.21.15.27:0/3203533537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d4109e10 0x7f59d41a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:12:02.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.736+0000 7f59d3400640 1 --2- 172.21.15.27:0/3203533537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59d4103b80 0x7f59d419c6b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f59c400e9f0 tx=0x7f59c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:12:02.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.736+0000 7f59cbe00640 1 -- 172.21.15.27:0/3203533537 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c400cd80 con 0x7f59d4103b80 2024-09-21T00:12:02.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.736+0000 7f59da400640 1 -- 172.21.15.27:0/3203533537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d41a2910 con 0x7f59d4103b80 2024-09-21T00:12:02.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.736+0000 7f59cbe00640 1 -- 172.21.15.27:0/3203533537 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59c40108e0 con 0x7f59d4103b80 2024-09-21T00:12:02.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.736+0000 7f59cbe00640 1 -- 172.21.15.27:0/3203533537 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4018980 con 0x7f59d4103b80 2024-09-21T00:12:02.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.736+0000 7f59da400640 1 -- 172.21.15.27:0/3203533537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d41a9070 con 0x7f59d4103b80 2024-09-21T00:12:02.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.737+0000 7f59da400640 1 -- 172.21.15.27:0/3203533537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d4103f90 con 0x7f59d4103b80 2024-09-21T00:12:02.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.740+0000 7f59cbe00640 1 -- 172.21.15.27:0/3203533537 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f59c4010430 con 0x7f59d4103b80 2024-09-21T00:12:02.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.741+0000 7f59cbe00640 1 --2- 172.21.15.27:0/3203533537 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f59b80b64d0 0x7f59b80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:02.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.742+0000 7f59cbe00640 1 -- 172.21.15.27:0/3203533537 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f59c40e5f80 con 0x7f59d4103b80 2024-09-21T00:12:02.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.742+0000 7f59d2a00640 1 --2- 172.21.15.27:0/3203533537 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f59b80b64d0 0x7f59b80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:02.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.742+0000 7f59d2a00640 1 --2- 172.21.15.27:0/3203533537 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f59b80b64d0 0x7f59b80b89e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f59d406b320 tx=0x7f59c0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:12:02.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.743+0000 7f59cbe00640 1 -- 172.21.15.27:0/3203533537 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f59c40b2750 con 0x7f59d4103b80 2024-09-21T00:12:02.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.945+0000 7f59da400640 1 -- 172.21.15.27:0/3203533537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f59d419d7e0 con 0x7f59d4103b80 2024-09-21T00:12:02.947 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:02 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:02.947 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:02 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:02.947 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:02 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:12:02.947 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:02 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:02.947 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:02 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:12:02.947 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:02 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:02.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.952+0000 7f59cbe00640 1 -- 172.21.15.27:0/3203533537 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v37)=0 v37) ==== 133+0+0 (secure 0 0 0) 0x7f59c40b7690 con 0x7f59d4103b80 2024-09-21T00:12:02.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.954+0000 7f59da400640 1 -- 172.21.15.27:0/3203533537 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f59b80b64d0 msgr2=0x7f59b80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:02.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.954+0000 7f59da400640 1 --2- 172.21.15.27:0/3203533537 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f59b80b64d0 0x7f59b80b89e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f59d406b320 tx=0x7f59c0005f90 comp rx=0 tx=0).stop 2024-09-21T00:12:02.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.954+0000 7f59da400640 1 -- 172.21.15.27:0/3203533537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59d4103b80 msgr2=0x7f59d419c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:02.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.954+0000 7f59da400640 1 --2- 172.21.15.27:0/3203533537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59d4103b80 0x7f59d419c6b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f59c400e9f0 tx=0x7f59c400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:12:02.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.954+0000 7f59da400640 1 -- 172.21.15.27:0/3203533537 shutdown_connections 2024-09-21T00:12:02.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.954+0000 7f59da400640 1 --2- 172.21.15.27:0/3203533537 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f59b80b64d0 0x7f59b80b89e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:02.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.954+0000 7f59da400640 1 --2- 172.21.15.27:0/3203533537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d4109e10 0x7f59d41a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:02.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.954+0000 7f59da400640 1 --2- 172.21.15.27:0/3203533537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59d41094c0 0x7f59d419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:02.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.954+0000 7f59da400640 1 --2- 172.21.15.27:0/3203533537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59d4103b80 0x7f59d419c6b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:02.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.954+0000 7f59da400640 1 -- 172.21.15.27:0/3203533537 >> 172.21.15.27:0/3203533537 conn(0x7f59d40ff5d0 msgr2=0x7f59d4108ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:12:02.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.954+0000 7f59da400640 1 -- 172.21.15.27:0/3203533537 shutdown_connections 2024-09-21T00:12:02.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:02.955+0000 7f59da400640 1 -- 172.21.15.27:0/3203533537 wait complete. 2024-09-21T00:12:03.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:02 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:03.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:02 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:03.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:02 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:12:03.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:02 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:03.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:02 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:12:03.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:02 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:03.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:02 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:03.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:02 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:03.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:02 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:12:03.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:02 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:03.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:02 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:12:03.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:02 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:03.640 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi027') 2024-09-21T00:12:03.640 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi027') 2024-09-21T00:12:03.640 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi027') 2024-09-21T00:12:03.640 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '9c21de84-77ad-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi027') 2024-09-21T00:12:03.640 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T00:12:04.012 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:12:04.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:03 smithi089 ceph-mon[34481]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:04.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:03 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' 2024-09-21T00:12:04.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:03 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:12:04.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:03 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:04.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:03 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:12:04.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:03 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:04.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:03 smithi098 ceph-mon[34725]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:04.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:03 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' 2024-09-21T00:12:04.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:03 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:12:04.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:03 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:04.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:03 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:12:04.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:03 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:04.270 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:03 smithi027 ceph-mon[30745]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:04.271 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:03 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' 2024-09-21T00:12:04.271 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:03 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:12:04.271 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:03 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:04.271 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:03 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:12:04.271 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:03 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:04.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.611+0000 7fe727800640 1 -- 172.21.15.27:0/1487411023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7201094a0 msgr2=0x7fe7201098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:04.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.611+0000 7fe727800640 1 --2- 172.21.15.27:0/1487411023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7201094a0 0x7fe7201098b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe708009900 tx=0x7fe70802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:12:04.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.611+0000 7fe727800640 1 -- 172.21.15.27:0/1487411023 shutdown_connections 2024-09-21T00:12:04.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.611+0000 7fe727800640 1 --2- 172.21.15.27:0/1487411023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe720109df0 0x7fe720110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:04.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.611+0000 7fe727800640 1 --2- 172.21.15.27:0/1487411023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7201094a0 0x7fe7201098b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:04.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.611+0000 7fe727800640 1 --2- 172.21.15.27:0/1487411023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe720103b60 0x7fe720103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:04.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.611+0000 7fe727800640 1 -- 172.21.15.27:0/1487411023 >> 172.21.15.27:0/1487411023 conn(0x7fe72007b850 msgr2=0x7fe72007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:12:04.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.611+0000 7fe727800640 1 -- 172.21.15.27:0/1487411023 shutdown_connections 2024-09-21T00:12:04.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.611+0000 7fe727800640 1 -- 172.21.15.27:0/1487411023 wait complete. 2024-09-21T00:12:04.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.612+0000 7fe727800640 1 Processor -- start 2024-09-21T00:12:04.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.613+0000 7fe727800640 1 -- start start 2024-09-21T00:12:04.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.613+0000 7fe727800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe720103b60 0x7fe72019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:04.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.613+0000 7fe727800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7201094a0 0x7fe72019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:04.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.613+0000 7fe727800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe720109df0 0x7fe7201a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:04.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.613+0000 7fe727800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe720113ab0 con 0x7fe720103b60 2024-09-21T00:12:04.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.613+0000 7fe727800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe720113930 con 0x7fe720109df0 2024-09-21T00:12:04.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.613+0000 7fe727800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe720113c30 con 0x7fe7201094a0 2024-09-21T00:12:04.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.613+0000 7fe717e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7201094a0 0x7fe72019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:04.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.613+0000 7fe725800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe720109df0 0x7fe7201a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:04.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.613+0000 7fe725800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe720109df0 0x7fe7201a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45684/0 (socket says 172.21.15.27:45684) 2024-09-21T00:12:04.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.613+0000 7fe724e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe720103b60 0x7fe72019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:04.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.613+0000 7fe717e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7201094a0 0x7fe72019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38330/0 (socket says 172.21.15.27:38330) 2024-09-21T00:12:04.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.613+0000 7fe725800640 1 -- 172.21.15.27:0/3777742286 learned_addr learned my addr 172.21.15.27:0/3777742286 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:12:04.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.614+0000 7fe725800640 1 -- 172.21.15.27:0/3777742286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7201094a0 msgr2=0x7fe72019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:04.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.614+0000 7fe725800640 1 --2- 172.21.15.27:0/3777742286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7201094a0 0x7fe72019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:04.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.614+0000 7fe725800640 1 -- 172.21.15.27:0/3777742286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe720103b60 msgr2=0x7fe72019c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:04.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.614+0000 7fe725800640 1 --2- 172.21.15.27:0/3777742286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe720103b60 0x7fe72019c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:04.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.614+0000 7fe725800640 1 -- 172.21.15.27:0/3777742286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7201a2610 con 0x7fe720109df0 2024-09-21T00:12:04.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.614+0000 7fe725800640 1 --2- 172.21.15.27:0/3777742286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe720109df0 0x7fe7201a1ee0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe71800eb10 tx=0x7fe71800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:12:04.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.614+0000 7fe71e000640 1 -- 172.21.15.27:0/3777742286 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718004b20 con 0x7fe720109df0 2024-09-21T00:12:04.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.614+0000 7fe727800640 1 -- 172.21.15.27:0/3777742286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7201a2900 con 0x7fe720109df0 2024-09-21T00:12:04.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.614+0000 7fe71e000640 1 -- 172.21.15.27:0/3777742286 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe718004cc0 con 0x7fe720109df0 2024-09-21T00:12:04.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.614+0000 7fe71e000640 1 -- 172.21.15.27:0/3777742286 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718010670 con 0x7fe720109df0 2024-09-21T00:12:04.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.615+0000 7fe727800640 1 -- 172.21.15.27:0/3777742286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7201a9060 con 0x7fe720109df0 2024-09-21T00:12:04.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.616+0000 7fe727800640 1 -- 172.21.15.27:0/3777742286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe720103f70 con 0x7fe720109df0 2024-09-21T00:12:04.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.618+0000 7fe71e000640 1 -- 172.21.15.27:0/3777742286 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fe718005020 con 0x7fe720109df0 2024-09-21T00:12:04.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.620+0000 7fe71e000640 1 --2- 172.21.15.27:0/3777742286 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7040b6550 0x7fe7040b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:04.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.620+0000 7fe71e000640 1 -- 172.21.15.27:0/3777742286 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe7180e5da0 con 0x7fe720109df0 2024-09-21T00:12:04.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.620+0000 7fe724e00640 1 --2- 172.21.15.27:0/3777742286 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7040b6550 0x7fe7040b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:04.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.621+0000 7fe724e00640 1 --2- 172.21.15.27:0/3777742286 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7040b6550 0x7fe7040b8a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe710009bd0 tx=0x7fe710005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:12:04.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.624+0000 7fe71e000640 1 -- 172.21.15.27:0/3777742286 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe7180ae190 con 0x7fe720109df0 2024-09-21T00:12:04.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.825+0000 7fe727800640 1 -- 172.21.15.27:0/3777742286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fe7201a9340 con 0x7fe720109df0 2024-09-21T00:12:04.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.827+0000 7fe71e000640 1 -- 172.21.15.27:0/3777742286 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+21924 (secure 0 0 0) 0x7fe7180b2570 con 0x7fe720109df0 2024-09-21T00:12:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T00:12:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:global dev auth_debug true 2024-09-21T00:12:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T00:12:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T00:12:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:global basic log_to_file true 2024-09-21T00:12:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:global basic log_to_journald false 2024-09-21T00:12:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:global basic log_to_stderr false 2024-09-21T00:12:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_allow_pool_delete true 2024-09-21T00:12:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T00:12:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T00:12:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T00:12:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T00:12:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T00:12:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T00:12:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T00:12:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T00:12:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T00:12:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:global dev ms_die_on_bug true 2024-09-21T00:12:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:global dev ms_die_on_old_message true 2024-09-21T00:12:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:global dev ms_die_on_skipped_message false 2024-09-21T00:12:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T00:12:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T00:12:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T00:12:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T00:12:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T00:12:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced debug_mon 20/20 2024-09-21T00:12:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced debug_ms 1/1 2024-09-21T00:12:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced debug_paxos 20/20 2024-09-21T00:12:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T00:12:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T00:12:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T00:12:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T00:12:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T00:12:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T00:12:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T00:12:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T00:12:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T00:12:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T00:12:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T00:12:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T00:12:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced client_mount_timeout 30 2024-09-21T00:12:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced debug_cephsqlite 20/20 2024-09-21T00:12:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced debug_client 20/20 2024-09-21T00:12:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced debug_mgr 20/20 2024-09-21T00:12:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced debug_ms 1/1 2024-09-21T00:12:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T00:12:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T00:12:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T00:12:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T00:12:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T00:12:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/snap_schedule/allow_m_granularity true 2024-09-21T00:12:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/snap_schedule/dump_on_update true 2024-09-21T00:12:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/snap_schedule/log_level debug 2024-09-21T00:12:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T00:12:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T00:12:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T00:12:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev bdev_debug_aio true 2024-09-21T00:12:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T00:12:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T00:12:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced debug_bluefs 20/20 2024-09-21T00:12:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced debug_bluestore 20/20 2024-09-21T00:12:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced debug_ms 1/1 2024-09-21T00:12:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced debug_osd 20/20 2024-09-21T00:12:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced debug_rocksdb 10/10 2024-09-21T00:12:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T00:12:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_op_order true 2024-09-21T00:12:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T00:12:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_shutdown true 2024-09-21T00:12:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T00:12:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T00:12:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T00:12:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T00:12:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T00:12:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_max_omap_entries_per_request 10000 2024-09-21T00:12:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T00:12:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T00:12:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T00:12:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T00:12:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T00:12:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T00:12:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T00:12:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T00:12:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T00:12:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T00:12:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T00:12:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced debug_mds 20/20 2024-09-21T00:12:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T00:12:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced debug_ms 2/2 2024-09-21T00:12:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T00:12:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T00:12:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_bal_replicate_threshold 1.000000 2024-09-21T00:12:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T00:12:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_bal_split_size 100 2024-09-21T00:12:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:mds dev mds_debug_frag true 2024-09-21T00:12:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:mds dev mds_debug_scatterstat true 2024-09-21T00:12:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T00:12:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:mds dev mds_verify_scatter true 2024-09-21T00:12:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T00:12:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T00:12:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T00:12:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:client advanced client_mount_timeout 600 2024-09-21T00:12:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:client advanced debug_client 20/20 2024-09-21T00:12:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:client advanced debug_ms 1/1 2024-09-21T00:12:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T00:12:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T00:12:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T00:12:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T00:12:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T00:12:04.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.829+0000 7fe727800640 1 -- 172.21.15.27:0/3777742286 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7040b6550 msgr2=0x7fe7040b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:04.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.829+0000 7fe727800640 1 --2- 172.21.15.27:0/3777742286 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7040b6550 0x7fe7040b8a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe710009bd0 tx=0x7fe710005f90 comp rx=0 tx=0).stop 2024-09-21T00:12:04.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.829+0000 7fe727800640 1 -- 172.21.15.27:0/3777742286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe720109df0 msgr2=0x7fe7201a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:04.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.829+0000 7fe727800640 1 --2- 172.21.15.27:0/3777742286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe720109df0 0x7fe7201a1ee0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe71800eb10 tx=0x7fe71800c2d0 comp rx=0 tx=0).stop 2024-09-21T00:12:04.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.829+0000 7fe727800640 1 -- 172.21.15.27:0/3777742286 shutdown_connections 2024-09-21T00:12:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.829+0000 7fe727800640 1 --2- 172.21.15.27:0/3777742286 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7040b6550 0x7fe7040b8a60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.829+0000 7fe727800640 1 --2- 172.21.15.27:0/3777742286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe720109df0 0x7fe7201a1ee0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.829+0000 7fe727800640 1 --2- 172.21.15.27:0/3777742286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7201094a0 0x7fe72019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.829+0000 7fe727800640 1 --2- 172.21.15.27:0/3777742286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe720103b60 0x7fe72019c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.829+0000 7fe727800640 1 -- 172.21.15.27:0/3777742286 >> 172.21.15.27:0/3777742286 conn(0x7fe72007b850 msgr2=0x7fe720105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:12:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.829+0000 7fe727800640 1 -- 172.21.15.27:0/3777742286 shutdown_connections 2024-09-21T00:12:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:04.829+0000 7fe727800640 1 -- 172.21.15.27:0/3777742286 wait complete. 2024-09-21T00:12:05.215 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:04 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3777742286' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T00:12:05.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:04 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3777742286' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T00:12:05.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:04 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3777742286' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T00:12:05.589 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T00:12:05.589 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:12:05.589 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T00:12:05.625 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T00:12:05.625 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2024-09-21T00:12:05.700 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T00:12:05.700 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:12:05.700 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-09-21T00:12:05.701 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:12:05.701 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:12:05.701 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-21 00:10:30.097394769 +0000 2024-09-21T00:12:05.701 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-21 00:05:55.171639544 +0000 2024-09-21T00:12:05.701 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-21 00:05:55.171639544 +0000 2024-09-21T00:12:05.701 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-21 00:05:55.171639544 +0000 2024-09-21T00:12:05.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T00:12:05.798 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-21T00:12:05.798 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-21T00:12:05.799 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000340064 s, 1.5 MB/s 2024-09-21T00:12:05.799 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T00:12:05.836 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2024-09-21T00:12:05.912 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T00:12:05.912 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:12:05.913 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-09-21T00:12:05.913 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:12:05.913 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:12:05.913 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-21 00:10:30.136393789 +0000 2024-09-21T00:12:05.913 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-21 00:05:55.622628221 +0000 2024-09-21T00:12:05.913 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-21 00:05:55.622628221 +0000 2024-09-21T00:12:05.913 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-21 00:05:55.622628221 +0000 2024-09-21T00:12:05.913 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T00:12:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-21T00:12:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-21T00:12:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.00026778 s, 1.9 MB/s 2024-09-21T00:12:06.010 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T00:12:06.047 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2024-09-21T00:12:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T00:12:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:12:06.124 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-09-21T00:12:06.124 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:12:06.124 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:12:06.124 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-21 00:10:30.176392783 +0000 2024-09-21T00:12:06.124 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-21 00:05:56.108616018 +0000 2024-09-21T00:12:06.124 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-21 00:05:56.108616018 +0000 2024-09-21T00:12:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-21 00:05:56.108616018 +0000 2024-09-21T00:12:06.125 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T00:12:06.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:05 smithi027 ceph-mon[30745]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:06.218 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-21T00:12:06.218 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-21T00:12:06.218 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000276384 s, 1.9 MB/s 2024-09-21T00:12:06.221 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T00:12:06.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:05 smithi089 ceph-mon[34481]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:06.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:05 smithi098 ceph-mon[34725]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:06.258 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2024-09-21T00:12:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T00:12:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:12:06.333 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-09-21T00:12:06.333 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:12:06.333 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:12:06.334 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-21 00:10:30.214391827 +0000 2024-09-21T00:12:06.334 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-21 00:05:56.583604092 +0000 2024-09-21T00:12:06.334 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-21 00:05:56.583604092 +0000 2024-09-21T00:12:06.334 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-21 00:05:56.583604092 +0000 2024-09-21T00:12:06.334 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T00:12:06.428 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-21T00:12:06.428 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-21T00:12:06.428 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000288267 s, 1.8 MB/s 2024-09-21T00:12:06.430 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T00:12:06.467 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T00:12:06.467 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T00:12:06.487 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T00:12:06.488 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2024-09-21T00:12:06.546 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T00:12:06.546 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:12:06.547 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T00:12:06.547 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:12:06.547 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:12:06.547 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-21 00:11:35.122067159 +0000 2024-09-21T00:12:06.547 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-21 00:05:08.238375335 +0000 2024-09-21T00:12:06.547 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-21 00:05:08.238375335 +0000 2024-09-21T00:12:06.547 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-21 00:05:08.238375335 +0000 2024-09-21T00:12:06.547 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T00:12:06.617 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-21T00:12:06.617 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-21T00:12:06.617 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000233723 s, 2.2 MB/s 2024-09-21T00:12:06.619 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T00:12:06.677 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2024-09-21T00:12:06.735 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T00:12:06.735 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:12:06.735 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T00:12:06.735 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:12:06.735 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:12:06.735 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-21 00:11:35.122067159 +0000 2024-09-21T00:12:06.735 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-21 00:05:08.561367939 +0000 2024-09-21T00:12:06.736 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-21 00:05:08.561367939 +0000 2024-09-21T00:12:06.736 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-21 00:05:08.561367939 +0000 2024-09-21T00:12:06.736 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T00:12:06.800 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-21T00:12:06.800 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-21T00:12:06.800 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000206691 s, 2.5 MB/s 2024-09-21T00:12:06.802 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T00:12:06.859 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2024-09-21T00:12:06.915 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T00:12:06.915 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:12:06.916 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T00:12:06.916 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:12:06.916 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:12:06.916 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-21 00:11:35.122067159 +0000 2024-09-21T00:12:06.916 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-21 00:05:08.894360314 +0000 2024-09-21T00:12:06.916 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-21 00:05:08.894360314 +0000 2024-09-21T00:12:06.916 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-21 00:05:08.894360314 +0000 2024-09-21T00:12:06.916 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T00:12:06.981 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-21T00:12:06.981 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-21T00:12:06.981 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000204149 s, 2.5 MB/s 2024-09-21T00:12:06.983 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T00:12:07.040 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2024-09-21T00:12:07.095 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T00:12:07.096 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:12:07.096 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T00:12:07.096 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:12:07.096 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:12:07.096 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-21 00:11:35.123067137 +0000 2024-09-21T00:12:07.096 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-21 00:05:09.219352872 +0000 2024-09-21T00:12:07.096 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-21 00:05:09.219352872 +0000 2024-09-21T00:12:07.096 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-21 00:05:09.219352872 +0000 2024-09-21T00:12:07.096 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T00:12:07.160 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-21T00:12:07.160 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-21T00:12:07.160 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000183643 s, 2.8 MB/s 2024-09-21T00:12:07.161 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T00:12:07.217 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T00:12:07.217 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T00:12:07.237 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T00:12:07.237 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_1 2024-09-21T00:12:07.295 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T00:12:07.324 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:12:07.324 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T00:12:07.324 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:12:07.324 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:12:07.324 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-21 00:11:32.806318220 +0000 2024-09-21T00:12:07.324 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-21 00:05:23.420608444 +0000 2024-09-21T00:12:07.324 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-21 00:05:23.420608444 +0000 2024-09-21T00:12:07.324 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-21 00:05:23.420608444 +0000 2024-09-21T00:12:07.324 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T00:12:07.363 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-21T00:12:07.364 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-21T00:12:07.364 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000276883 s, 1.8 MB/s 2024-09-21T00:12:07.365 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T00:12:07.423 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_2 2024-09-21T00:12:07.480 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T00:12:07.481 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:12:07.481 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-21T00:12:07.481 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:12:07.481 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:12:07.481 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-21 00:11:32.806318220 +0000 2024-09-21T00:12:07.481 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-21 00:05:23.745604272 +0000 2024-09-21T00:12:07.481 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-21 00:05:23.745604272 +0000 2024-09-21T00:12:07.481 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-21 00:05:23.745604272 +0000 2024-09-21T00:12:07.481 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T00:12:07.546 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-21T00:12:07.546 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-21T00:12:07.546 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000237008 s, 2.2 MB/s 2024-09-21T00:12:07.547 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T00:12:07.604 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_3 2024-09-21T00:12:07.660 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T00:12:07.660 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:12:07.660 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-21T00:12:07.660 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:12:07.660 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:12:07.660 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-21 00:11:32.807318208 +0000 2024-09-21T00:12:07.660 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-21 00:05:24.060600229 +0000 2024-09-21T00:12:07.660 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-21 00:05:24.060600229 +0000 2024-09-21T00:12:07.660 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-21 00:05:24.060600229 +0000 2024-09-21T00:12:07.661 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T00:12:07.723 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-21T00:12:07.724 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-21T00:12:07.724 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000202181 s, 2.5 MB/s 2024-09-21T00:12:07.725 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T00:12:07.781 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_4 2024-09-21T00:12:07.839 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T00:12:07.839 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:12:07.839 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-21T00:12:07.839 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:12:07.839 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:12:07.839 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-21 00:11:32.807318208 +0000 2024-09-21T00:12:07.839 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-21 00:05:24.383596083 +0000 2024-09-21T00:12:07.839 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-21 00:05:24.383596083 +0000 2024-09-21T00:12:07.840 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-21 00:05:24.383596083 +0000 2024-09-21T00:12:07.840 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T00:12:07.902 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-21T00:12:07.902 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-21T00:12:07.902 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000201884 s, 2.5 MB/s 2024-09-21T00:12:07.903 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T00:12:07.959 INFO:tasks.cephadm:Deploying osd.0 on smithi027 with /dev/vg_nvme/lv_4... 2024-09-21T00:12:07.959 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T00:12:07.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:07 smithi098 ceph-mon[34725]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:08.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:07 smithi089 ceph-mon[34481]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:08.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:07 smithi027 ceph-mon[30745]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:08.336 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:12:09.994 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:09 smithi027 ceph-mon[30745]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:10.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:09 smithi089 ceph-mon[34481]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:10.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:09 smithi098 ceph-mon[34725]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:11.229 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:12:11.263 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi027:vg_nvme/lv_4 2024-09-21T00:12:11.638 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:12:12.051 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:11 smithi027 ceph-mon[30745]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:12.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:11 smithi089 ceph-mon[34481]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:12.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.235+0000 7f2c86000640 1 -- 172.21.15.27:0/2277905399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c801092f0 msgr2=0x7f2c80109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:12.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.235+0000 7f2c86000640 1 --2- 172.21.15.27:0/2277905399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c801092f0 0x7f2c80109700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009900 tx=0x7f2c6c02a410 comp rx=0 tx=0).stop 2024-09-21T00:12:12.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.236+0000 7f2c86000640 1 -- 172.21.15.27:0/2277905399 shutdown_connections 2024-09-21T00:12:12.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.236+0000 7f2c86000640 1 --2- 172.21.15.27:0/2277905399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c80109c40 0x7f2c80110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:12.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.236+0000 7f2c86000640 1 --2- 172.21.15.27:0/2277905399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c801092f0 0x7f2c80109700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:12.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.236+0000 7f2c86000640 1 --2- 172.21.15.27:0/2277905399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c801039b0 0x7f2c80103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:12.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.236+0000 7f2c86000640 1 -- 172.21.15.27:0/2277905399 >> 172.21.15.27:0/2277905399 conn(0x7f2c800ff400 msgr2=0x7f2c80101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:12:12.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.236+0000 7f2c86000640 1 -- 172.21.15.27:0/2277905399 shutdown_connections 2024-09-21T00:12:12.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.236+0000 7f2c86000640 1 -- 172.21.15.27:0/2277905399 wait complete. 2024-09-21T00:12:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.237+0000 7f2c86000640 1 Processor -- start 2024-09-21T00:12:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.237+0000 7f2c86000640 1 -- start start 2024-09-21T00:12:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.237+0000 7f2c86000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c801039b0 0x7f2c8019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.237+0000 7f2c86000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c801092f0 0x7f2c8019a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.237+0000 7f2c86000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c80109c40 0x7f2c8019fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:12.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.237+0000 7f2c86000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c80113900 con 0x7f2c80109c40 2024-09-21T00:12:12.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.237+0000 7f2c86000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2c80113780 con 0x7f2c801092f0 2024-09-21T00:12:12.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.237+0000 7f2c86000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c80113a80 con 0x7f2c801039b0 2024-09-21T00:12:12.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.238+0000 7f2c7fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c801092f0 0x7f2c8019a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:12.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.238+0000 7f2c7fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c801092f0 0x7f2c8019a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45708/0 (socket says 172.21.15.27:45708) 2024-09-21T00:12:12.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.238+0000 7f2c7fe00640 1 -- 172.21.15.27:0/1448230235 learned_addr learned my addr 172.21.15.27:0/1448230235 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:12:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.238+0000 7f2c85600640 1 --2- 172.21.15.27:0/1448230235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c80109c40 0x7f2c8019fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.238+0000 7f2c84c00640 1 --2- 172.21.15.27:0/1448230235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c801039b0 0x7f2c8019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.238+0000 7f2c7fe00640 1 -- 172.21.15.27:0/1448230235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c801039b0 msgr2=0x7f2c8019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.238+0000 7f2c7fe00640 1 --2- 172.21.15.27:0/1448230235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c801039b0 0x7f2c8019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.238+0000 7f2c7fe00640 1 -- 172.21.15.27:0/1448230235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c80109c40 msgr2=0x7f2c8019fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.238+0000 7f2c7fe00640 1 --2- 172.21.15.27:0/1448230235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c80109c40 0x7f2c8019fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:12.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.238+0000 7f2c7fe00640 1 -- 172.21.15.27:0/1448230235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c801a0240 con 0x7f2c801092f0 2024-09-21T00:12:12.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.238+0000 7f2c7fe00640 1 --2- 172.21.15.27:0/1448230235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c801092f0 0x7f2c8019a870 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c02a930 tx=0x7f2c6c031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:12:12.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.239+0000 7f2c7d600640 1 -- 172.21.15.27:0/1448230235 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c03f030 con 0x7f2c801092f0 2024-09-21T00:12:12.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.239+0000 7f2c86000640 1 -- 172.21.15.27:0/1448230235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c801a04d0 con 0x7f2c801092f0 2024-09-21T00:12:12.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.239+0000 7f2c86000640 1 -- 172.21.15.27:0/1448230235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c801a6c90 con 0x7f2c801092f0 2024-09-21T00:12:12.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.239+0000 7f2c7d600640 1 -- 172.21.15.27:0/1448230235 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c6c004490 con 0x7f2c801092f0 2024-09-21T00:12:12.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.239+0000 7f2c7d600640 1 -- 172.21.15.27:0/1448230235 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c049060 con 0x7f2c801092f0 2024-09-21T00:12:12.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.240+0000 7f2c86000640 1 -- 172.21.15.27:0/1448230235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c80103dc0 con 0x7f2c801092f0 2024-09-21T00:12:12.248 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:11 smithi098 ceph-mon[34725]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:12.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.242+0000 7f2c7d600640 1 -- 172.21.15.27:0/1448230235 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f2c6c042030 con 0x7f2c801092f0 2024-09-21T00:12:12.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.244+0000 7f2c7d600640 1 --2- 172.21.15.27:0/1448230235 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2c680b6550 0x7f2c680b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:12.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.244+0000 7f2c7d600640 1 -- 172.21.15.27:0/1448230235 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f2c6c105aa0 con 0x7f2c801092f0 2024-09-21T00:12:12.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.244+0000 7f2c84c00640 1 --2- 172.21.15.27:0/1448230235 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2c680b6550 0x7f2c680b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:12.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.245+0000 7f2c84c00640 1 --2- 172.21.15.27:0/1448230235 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2c680b6550 0x7f2c680b8a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009bd0 tx=0x7f2c74005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:12:12.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.246+0000 7f2c7d600640 1 -- 172.21.15.27:0/1448230235 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2c6c0d2270 con 0x7f2c801092f0 2024-09-21T00:12:12.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:12.454+0000 7f2c86000640 1 -- 172.21.15.27:0/1448230235 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2c80063800 con 0x7f2c680b6550 2024-09-21T00:12:13.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:12 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:12:13.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:12 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:12:13.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:12 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:13.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:12 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:12:13.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:12 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:12:13.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:12 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:13.286 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:12 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:12:13.287 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:12 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:12:13.287 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:12 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:14.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:13 smithi027 ceph-mon[30745]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:14.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:13 smithi027 ceph-mon[30745]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:12:14.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:13 smithi089 ceph-mon[34481]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:14.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:13 smithi089 ceph-mon[34481]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:12:14.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:13 smithi098 ceph-mon[34725]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:14.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:13 smithi098 ceph-mon[34725]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:12:16.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:15 smithi098 ceph-mon[34725]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:16.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:15 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3343305580' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "63ccc094-62d5-446f-844d-1c60d41fda1c"} : dispatch 2024-09-21T00:12:16.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:15 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3343305580' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63ccc094-62d5-446f-844d-1c60d41fda1c"}]': finished 2024-09-21T00:12:16.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:15 smithi098 ceph-mon[34725]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T00:12:16.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:15 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T00:12:16.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:15 smithi027 ceph-mon[30745]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:16.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:15 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3343305580' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "63ccc094-62d5-446f-844d-1c60d41fda1c"} : dispatch 2024-09-21T00:12:16.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:15 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3343305580' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63ccc094-62d5-446f-844d-1c60d41fda1c"}]': finished 2024-09-21T00:12:16.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:15 smithi027 ceph-mon[30745]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T00:12:16.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:15 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T00:12:16.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:15 smithi089 ceph-mon[34481]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:16.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:15 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3343305580' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "63ccc094-62d5-446f-844d-1c60d41fda1c"} : dispatch 2024-09-21T00:12:16.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:15 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3343305580' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63ccc094-62d5-446f-844d-1c60d41fda1c"}]': finished 2024-09-21T00:12:16.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:15 smithi089 ceph-mon[34481]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T00:12:16.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:15 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T00:12:17.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:16 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/107139379' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:12:17.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:16 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/107139379' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:12:17.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:16 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/107139379' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:12:18.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:17 smithi098 ceph-mon[34725]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:18.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:17 smithi027 ceph-mon[30745]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:18.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:17 smithi089 ceph-mon[34481]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:20.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:19 smithi098 ceph-mon[34725]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:20.291 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:19 smithi027 ceph-mon[30745]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:20.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:19 smithi089 ceph-mon[34481]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:21 smithi098 ceph-mon[34725]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:22.252 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:21 smithi027 ceph-mon[30745]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:22.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:21 smithi089 ceph-mon[34481]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:24.204 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:23 smithi027 ceph-mon[30745]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:24.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:23 smithi089 ceph-mon[34481]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:24.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:23 smithi098 ceph-mon[34725]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:25.263 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:25 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T00:12:25.263 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:25 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:25.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:25 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T00:12:25.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:25 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:25.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:25 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T00:12:25.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:25 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:26.117 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:26 smithi027 ceph-mon[30745]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:26.117 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:26 smithi027 ceph-mon[30745]: Deploying daemon osd.0 on smithi027 2024-09-21T00:12:26.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:26 smithi089 ceph-mon[34481]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:26.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:26 smithi089 ceph-mon[34481]: Deploying daemon osd.0 on smithi027 2024-09-21T00:12:26.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:26 smithi098 ceph-mon[34725]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:26.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:26 smithi098 ceph-mon[34725]: Deploying daemon osd.0 on smithi027 2024-09-21T00:12:28.120 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:28 smithi027 ceph-mon[30745]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:28.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:28 smithi089 ceph-mon[34481]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:28 smithi098 ceph-mon[34725]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:30.298 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:30 smithi098 ceph-mon[34725]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:30.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:30 smithi027 ceph-mon[30745]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:30.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:30 smithi089 ceph-mon[34481]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:32.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:32 smithi027 ceph-mon[30745]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:32.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:32 smithi089 ceph-mon[34481]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:32.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:32 smithi098 ceph-mon[34725]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:34.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:34 smithi027 ceph-mon[30745]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:34.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:34 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:12:34.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:34 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:34.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:34 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:34.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:34 smithi089 ceph-mon[34481]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:34.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:34 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:12:34.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:34 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:34.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:34 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:34.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:34 smithi098 ceph-mon[34725]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:34.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:34 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:12:34.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:34 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:34.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:34 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:36.159 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:36 smithi027 ceph-mon[30745]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:36.160 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:36 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:36.160 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:36 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:36.281 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:36 smithi089 ceph-mon[34481]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:36.281 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:36 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:36.281 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:36 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:36.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:36 smithi098 ceph-mon[34725]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:36.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:36 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:36.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:36 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:37.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:37 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:37.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:37 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:12:37.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:37 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:37.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:37 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:37.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:37 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:12:37.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:37 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:37.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:37 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:37.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:37 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:12:37.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:37 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:38.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:38 smithi027 ceph-mon[30745]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:38.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:38 smithi089 ceph-mon[34481]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:38.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:38 smithi098 ceph-mon[34725]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 0 on host 'smithi027' 2024-09-21T00:12:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:38.773+0000 7f2c7d600640 1 -- 172.21.15.27:0/1448230235 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2c80063800 con 0x7f2c680b6550 2024-09-21T00:12:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:38.776+0000 7f2c86000640 1 -- 172.21.15.27:0/1448230235 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2c680b6550 msgr2=0x7f2c680b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:38.776+0000 7f2c86000640 1 --2- 172.21.15.27:0/1448230235 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2c680b6550 0x7f2c680b8a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009bd0 tx=0x7f2c74005f90 comp rx=0 tx=0).stop 2024-09-21T00:12:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:38.776+0000 7f2c86000640 1 -- 172.21.15.27:0/1448230235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c801092f0 msgr2=0x7f2c8019a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:38.776+0000 7f2c86000640 1 --2- 172.21.15.27:0/1448230235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c801092f0 0x7f2c8019a870 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c02a930 tx=0x7f2c6c031a20 comp rx=0 tx=0).stop 2024-09-21T00:12:38.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:38.777+0000 7f2c86000640 1 -- 172.21.15.27:0/1448230235 shutdown_connections 2024-09-21T00:12:38.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:38.777+0000 7f2c86000640 1 --2- 172.21.15.27:0/1448230235 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2c680b6550 0x7f2c680b8a60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:38.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:38.777+0000 7f2c86000640 1 --2- 172.21.15.27:0/1448230235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c80109c40 0x7f2c8019fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:38.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:38.777+0000 7f2c86000640 1 --2- 172.21.15.27:0/1448230235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c801092f0 0x7f2c8019a870 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:38.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:38.777+0000 7f2c86000640 1 --2- 172.21.15.27:0/1448230235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c801039b0 0x7f2c8019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:38.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:38.777+0000 7f2c86000640 1 -- 172.21.15.27:0/1448230235 >> 172.21.15.27:0/1448230235 conn(0x7f2c800ff400 msgr2=0x7f2c80101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:12:38.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:38.777+0000 7f2c86000640 1 -- 172.21.15.27:0/1448230235 shutdown_connections 2024-09-21T00:12:38.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:12:38.777+0000 7f2c86000640 1 -- 172.21.15.27:0/1448230235 wait complete. 2024-09-21T00:12:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:39 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:12:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:39 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:39 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:39 smithi027 ceph-mon[30745]: from='osd.0 [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T00:12:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:39 smithi027 ceph-mon[30745]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T00:12:39.476 DEBUG:teuthology.orchestra.run.smithi027:osd.0> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T00:12:39.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:39 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:12:39.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:39 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:39.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:39 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:39.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:39 smithi089 ceph-mon[34481]: from='osd.0 [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T00:12:39.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:39 smithi089 ceph-mon[34481]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T00:12:39.479 INFO:tasks.cephadm:Deploying osd.1 on smithi089 with /dev/vg_nvme/lv_4... 2024-09-21T00:12:39.479 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T00:12:39.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:39 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:12:39.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:39 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:39.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:39 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:39.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:39 smithi098 ceph-mon[34725]: from='osd.0 [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T00:12:39.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:39 smithi098 ceph-mon[34725]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T00:12:39.730 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T00:12:40.154 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:40 smithi089 ceph-mon[34481]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:40.154 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:40 smithi089 ceph-mon[34481]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T00:12:40.155 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:40 smithi089 ceph-mon[34481]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T00:12:40.155 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:40 smithi089 ceph-mon[34481]: from='osd.0 [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:12:40.155 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:40 smithi089 ceph-mon[34481]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:12:40.155 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:40 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T00:12:40.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:40 smithi027 ceph-mon[30745]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:40.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:40 smithi027 ceph-mon[30745]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T00:12:40.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:40 smithi027 ceph-mon[30745]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T00:12:40.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:40 smithi027 ceph-mon[30745]: from='osd.0 [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:12:40.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:40 smithi027 ceph-mon[30745]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:12:40.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:40 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T00:12:40.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:40 smithi098 ceph-mon[34725]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:40.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:40 smithi098 ceph-mon[34725]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T00:12:40.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:40 smithi098 ceph-mon[34725]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T00:12:40.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:40 smithi098 ceph-mon[34725]: from='osd.0 [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:12:40.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:40 smithi098 ceph-mon[34725]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:12:40.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:40 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T00:12:41.067 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:12:40 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-0[45001]: 2024-09-21T00:12:40.789+0000 7f388fa00640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T00:12:41.068 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:12:40 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-0[45001]: 2024-09-21T00:12:40.793+0000 7f388aa00640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:12:41.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:41 smithi098 ceph-mon[34725]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:41.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:41 smithi098 ceph-mon[34725]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T00:12:41.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:41 smithi098 ceph-mon[34725]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T00:12:41.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:41 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T00:12:41.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:41 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T00:12:42.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:41 smithi027 ceph-mon[30745]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:42.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:41 smithi027 ceph-mon[30745]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T00:12:42.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:41 smithi027 ceph-mon[30745]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T00:12:42.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:41 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T00:12:42.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:41 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T00:12:42.081 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:41 smithi089 ceph-mon[34481]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:42.081 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:41 smithi089 ceph-mon[34481]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T00:12:42.081 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:41 smithi089 ceph-mon[34481]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T00:12:42.081 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:41 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T00:12:42.081 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:41 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T00:12:42.084 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:12:42.111 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi089:vg_nvme/lv_4 2024-09-21T00:12:42.354 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T00:12:42.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.786+0000 7f3bca85b640 1 -- 172.21.15.89:0/1342363224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bc4105d90 msgr2=0x7f3bc41061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:42.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.786+0000 7f3bca85b640 1 --2- 172.21.15.89:0/1342363224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bc4105d90 0x7f3bc41061a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009900 tx=0x7f3bb402a410 comp rx=0 tx=0).stop 2024-09-21T00:12:42.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.786+0000 7f3bca85b640 1 -- 172.21.15.89:0/1342363224 shutdown_connections 2024-09-21T00:12:42.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.786+0000 7f3bca85b640 1 --2- 172.21.15.89:0/1342363224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bc410c020 0x7f3bc4112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:42.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.786+0000 7f3bca85b640 1 --2- 172.21.15.89:0/1342363224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc410b6d0 0x7f3bc410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:42.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.786+0000 7f3bca85b640 1 --2- 172.21.15.89:0/1342363224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bc4105d90 0x7f3bc41061a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:42.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.786+0000 7f3bca85b640 1 -- 172.21.15.89:0/1342363224 >> 172.21.15.89:0/1342363224 conn(0x7f3bc41017c0 msgr2=0x7f3bc4103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:12:42.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.786+0000 7f3bca85b640 1 -- 172.21.15.89:0/1342363224 shutdown_connections 2024-09-21T00:12:42.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.786+0000 7f3bca85b640 1 -- 172.21.15.89:0/1342363224 wait complete. 2024-09-21T00:12:42.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.787+0000 7f3bca85b640 1 Processor -- start 2024-09-21T00:12:42.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.787+0000 7f3bca85b640 1 -- start start 2024-09-21T00:12:42.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.787+0000 7f3bca85b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc4105d90 0x7f3bc419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:42.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.787+0000 7f3bca85b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bc410b6d0 0x7f3bc419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:42.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.787+0000 7f3bca85b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bc410c020 0x7f3bc41a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:42.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.787+0000 7f3bca85b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4115d30 con 0x7f3bc4105d90 2024-09-21T00:12:42.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.787+0000 7f3bca85b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4115bb0 con 0x7f3bc410c020 2024-09-21T00:12:42.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.787+0000 7f3bca85b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4115eb0 con 0x7f3bc410b6d0 2024-09-21T00:12:42.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.788+0000 7f3bc8df8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bc410c020 0x7f3bc41a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:42.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.788+0000 7f3bc8df8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bc410c020 0x7f3bc41a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55548/0 (socket says 172.21.15.89:55548) 2024-09-21T00:12:42.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.788+0000 7f3bc8df8640 1 -- 172.21.15.89:0/3844718802 learned_addr learned my addr 172.21.15.89:0/3844718802 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T00:12:42.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.788+0000 7f3bc3fff640 1 --2- 172.21.15.89:0/3844718802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc4105d90 0x7f3bc419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:42.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.788+0000 7f3bc37fe640 1 --2- 172.21.15.89:0/3844718802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bc410b6d0 0x7f3bc419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:42.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.788+0000 7f3bc8df8640 1 -- 172.21.15.89:0/3844718802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bc410b6d0 msgr2=0x7f3bc419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:42.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.788+0000 7f3bc8df8640 1 --2- 172.21.15.89:0/3844718802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bc410b6d0 0x7f3bc419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:42.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.788+0000 7f3bc8df8640 1 -- 172.21.15.89:0/3844718802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc4105d90 msgr2=0x7f3bc419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:12:42.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.788+0000 7f3bc8df8640 1 --2- 172.21.15.89:0/3844718802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc4105d90 0x7f3bc419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:12:42.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.788+0000 7f3bc8df8640 1 -- 172.21.15.89:0/3844718802 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc41a2630 con 0x7f3bc410c020 2024-09-21T00:12:42.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.788+0000 7f3bc8df8640 1 --2- 172.21.15.89:0/3844718802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bc410c020 0x7f3bc41a1f00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00eaa0 tx=0x7f3bbc00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:12:42.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.793+0000 7f3bc17fa640 1 -- 172.21.15.89:0/3844718802 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00cdb0 con 0x7f3bc410c020 2024-09-21T00:12:42.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.793+0000 7f3bc17fa640 1 -- 172.21.15.89:0/3844718802 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bbc005020 con 0x7f3bc410c020 2024-09-21T00:12:42.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.793+0000 7f3bc17fa640 1 -- 172.21.15.89:0/3844718802 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0116d0 con 0x7f3bc410c020 2024-09-21T00:12:42.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.793+0000 7f3bca85b640 1 -- 172.21.15.89:0/3844718802 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc41a2920 con 0x7f3bc410c020 2024-09-21T00:12:42.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.793+0000 7f3bca85b640 1 -- 172.21.15.89:0/3844718802 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc41a9080 con 0x7f3bc410c020 2024-09-21T00:12:42.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.794+0000 7f3bca85b640 1 -- 172.21.15.89:0/3844718802 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc41061a0 con 0x7f3bc410c020 2024-09-21T00:12:42.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.795+0000 7f3bc17fa640 1 -- 172.21.15.89:0/3844718802 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f3bbc0051d0 con 0x7f3bc410c020 2024-09-21T00:12:42.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.796+0000 7f3bc17fa640 1 --2- 172.21.15.89:0/3844718802 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3bac0b6550 0x7f3bac0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:12:42.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.796+0000 7f3bc17fa640 1 -- 172.21.15.89:0/3844718802 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1982+0+0 (secure 0 0 0) 0x7f3bbc0e5a30 con 0x7f3bc410c020 2024-09-21T00:12:42.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.796+0000 7f3bc3fff640 1 --2- 172.21.15.89:0/3844718802 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3bac0b6550 0x7f3bac0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:12:42.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.797+0000 7f3bc3fff640 1 --2- 172.21.15.89:0/3844718802 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3bac0b6550 0x7f3bac0b8a60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4005ab0 tx=0x7f3bb4005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:12:42.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.800+0000 7f3bc17fa640 1 -- 172.21.15.89:0/3844718802 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3bbc016030 con 0x7f3bc410c020 2024-09-21T00:12:42.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:12:42.989+0000 7f3bca85b640 1 -- 172.21.15.89:0/3844718802 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3bc4063800 con 0x7f3bac0b6550 2024-09-21T00:12:43.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:42 smithi027 ceph-mon[30745]: purged_snaps scrub starts 2024-09-21T00:12:43.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:42 smithi027 ceph-mon[30745]: purged_snaps scrub ok 2024-09-21T00:12:43.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:42 smithi027 ceph-mon[30745]: osd.0 [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090] boot 2024-09-21T00:12:43.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:42 smithi027 ceph-mon[30745]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T00:12:43.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:42 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T00:12:43.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:42 smithi089 ceph-mon[34481]: purged_snaps scrub starts 2024-09-21T00:12:43.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:42 smithi089 ceph-mon[34481]: purged_snaps scrub ok 2024-09-21T00:12:43.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:42 smithi089 ceph-mon[34481]: osd.0 [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090] boot 2024-09-21T00:12:43.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:42 smithi089 ceph-mon[34481]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T00:12:43.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:42 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T00:12:43.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:42 smithi098 ceph-mon[34725]: purged_snaps scrub starts 2024-09-21T00:12:43.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:42 smithi098 ceph-mon[34725]: purged_snaps scrub ok 2024-09-21T00:12:43.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:42 smithi098 ceph-mon[34725]: osd.0 [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090] boot 2024-09-21T00:12:43.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:42 smithi098 ceph-mon[34725]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T00:12:43.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:42 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T00:12:43.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:43.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T00:12:43.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: Detected new or changed devices on smithi027 2024-09-21T00:12:43.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:43.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:43.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:12:43.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:12:43.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: Adjusting osd_memory_target on smithi027 to 17089M 2024-09-21T00:12:43.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:43.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:43.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:12:43.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:43.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:12:43.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:12:43.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:12:43.981 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:44.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:44.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T00:12:44.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: Detected new or changed devices on smithi027 2024-09-21T00:12:44.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:12:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:12:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: Adjusting osd_memory_target on smithi027 to 17089M 2024-09-21T00:12:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:12:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:12:44.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:12:44.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:12:44.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:44.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T00:12:44.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T00:12:44.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: Detected new or changed devices on smithi027 2024-09-21T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: Adjusting osd_memory_target on smithi027 to 17089M 2024-09-21T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:12:44.248 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:46.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:45 smithi027 ceph-mon[30745]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:46.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:45 smithi027 ceph-mon[30745]: from='client.? 172.21.15.89:0/2452096315' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e59a8ed4-3a37-4477-a60a-4abc33d49b4e"} : dispatch 2024-09-21T00:12:46.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:45 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e59a8ed4-3a37-4477-a60a-4abc33d49b4e"} : dispatch 2024-09-21T00:12:46.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:45 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e59a8ed4-3a37-4477-a60a-4abc33d49b4e"}]': finished 2024-09-21T00:12:46.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:45 smithi027 ceph-mon[30745]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T00:12:46.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:45 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:12:46.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:45 smithi089 ceph-mon[34481]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:46.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:45 smithi089 ceph-mon[34481]: from='client.? 172.21.15.89:0/2452096315' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e59a8ed4-3a37-4477-a60a-4abc33d49b4e"} : dispatch 2024-09-21T00:12:46.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:45 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e59a8ed4-3a37-4477-a60a-4abc33d49b4e"} : dispatch 2024-09-21T00:12:46.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:45 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e59a8ed4-3a37-4477-a60a-4abc33d49b4e"}]': finished 2024-09-21T00:12:46.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:45 smithi089 ceph-mon[34481]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T00:12:46.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:45 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:12:46.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:45 smithi098 ceph-mon[34725]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:46.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:45 smithi098 ceph-mon[34725]: from='client.? 172.21.15.89:0/2452096315' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e59a8ed4-3a37-4477-a60a-4abc33d49b4e"} : dispatch 2024-09-21T00:12:46.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:45 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e59a8ed4-3a37-4477-a60a-4abc33d49b4e"} : dispatch 2024-09-21T00:12:46.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:45 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e59a8ed4-3a37-4477-a60a-4abc33d49b4e"}]': finished 2024-09-21T00:12:46.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:45 smithi098 ceph-mon[34725]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T00:12:46.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:45 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:12:47.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:46 smithi027 ceph-mon[30745]: from='client.? 172.21.15.89:0/2551043027' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:12:47.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:46 smithi089 ceph-mon[34481]: from='client.? 172.21.15.89:0/2551043027' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:12:47.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:46 smithi098 ceph-mon[34725]: from='client.? 172.21.15.89:0/2551043027' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:12:48.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:47 smithi027 ceph-mon[30745]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:48.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:47 smithi089 ceph-mon[34481]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:48.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:47 smithi098 ceph-mon[34725]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:50.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:49 smithi027 ceph-mon[30745]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:50.196 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:49 smithi089 ceph-mon[34481]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:50.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:49 smithi098 ceph-mon[34725]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:51 smithi027 ceph-mon[30745]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:52.081 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:51 smithi089 ceph-mon[34481]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:52.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:51 smithi098 ceph-mon[34725]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:53.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:53 smithi089 ceph-mon[34481]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:53.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T00:12:53.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:54.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:53 smithi027 ceph-mon[30745]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:54.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T00:12:54.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:54.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:53 smithi098 ceph-mon[34725]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:54.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T00:12:54.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:12:55.050 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:54 smithi089 ceph-mon[34481]: Deploying daemon osd.1 on smithi089 2024-09-21T00:12:55.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:54 smithi098 ceph-mon[34725]: Deploying daemon osd.1 on smithi089 2024-09-21T00:12:55.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:54 smithi027 ceph-mon[30745]: Deploying daemon osd.1 on smithi089 2024-09-21T00:12:55.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:55 smithi089 ceph-mon[34481]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:56.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:55 smithi098 ceph-mon[34725]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:56.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:55 smithi027 ceph-mon[30745]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:58.094 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:57 smithi089 ceph-mon[34481]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:58.119 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:57 smithi027 ceph-mon[30745]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:12:58.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:57 smithi098 ceph-mon[34725]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:00.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:59 smithi027 ceph-mon[30745]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:00.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:59 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:13:00.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:59 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:00.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:12:59 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:00.129 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:59 smithi089 ceph-mon[34481]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:00.129 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:59 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:13:00.129 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:59 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:00.130 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:12:59 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:00.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:59 smithi098 ceph-mon[34725]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:00.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:59 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:13:00.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:59 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:00.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:12:59 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:02.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:01 smithi089 ceph-mon[34481]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:02.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:01 smithi098 ceph-mon[34725]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:02.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:01 smithi027 ceph-mon[30745]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:03.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:13:03.039+0000 7f3bc17fa640 1 -- 172.21.15.89:0/3844718802 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3bc4063800 con 0x7f3bac0b6550 2024-09-21T00:13:03.040 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 1 on host 'smithi089' 2024-09-21T00:13:03.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:13:03.041+0000 7f3bca85b640 1 -- 172.21.15.89:0/3844718802 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3bac0b6550 msgr2=0x7f3bac0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:13:03.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:13:03.041+0000 7f3bca85b640 1 --2- 172.21.15.89:0/3844718802 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3bac0b6550 0x7f3bac0b8a60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4005ab0 tx=0x7f3bb4005a20 comp rx=0 tx=0).stop 2024-09-21T00:13:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:13:03.041+0000 7f3bca85b640 1 -- 172.21.15.89:0/3844718802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bc410c020 msgr2=0x7f3bc41a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:13:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:13:03.041+0000 7f3bca85b640 1 --2- 172.21.15.89:0/3844718802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bc410c020 0x7f3bc41a1f00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00eaa0 tx=0x7f3bbc00ef60 comp rx=0 tx=0).stop 2024-09-21T00:13:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:13:03.041+0000 7f3bca85b640 1 -- 172.21.15.89:0/3844718802 shutdown_connections 2024-09-21T00:13:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:13:03.041+0000 7f3bca85b640 1 --2- 172.21.15.89:0/3844718802 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3bac0b6550 0x7f3bac0b8a60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:13:03.041+0000 7f3bca85b640 1 --2- 172.21.15.89:0/3844718802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bc410c020 0x7f3bc41a1f00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:13:03.041+0000 7f3bca85b640 1 --2- 172.21.15.89:0/3844718802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bc410b6d0 0x7f3bc419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:13:03.041+0000 7f3bca85b640 1 --2- 172.21.15.89:0/3844718802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc4105d90 0x7f3bc419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:13:03.041+0000 7f3bca85b640 1 -- 172.21.15.89:0/3844718802 >> 172.21.15.89:0/3844718802 conn(0x7f3bc41017c0 msgr2=0x7f3bc410ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:13:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:13:03.041+0000 7f3bca85b640 1 -- 172.21.15.89:0/3844718802 shutdown_connections 2024-09-21T00:13:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:13:03.041+0000 7f3bca85b640 1 -- 172.21.15.89:0/3844718802 wait complete. 2024-09-21T00:13:03.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:03 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:03.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:03 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:03.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:03 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:03.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:03 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:13:03.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:03 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:03.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:03 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:13:03.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:03 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:03.403 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:03 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:03.403 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:03 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:03.403 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:03 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:03.403 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:03 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:13:03.404 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:03 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:03.404 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:03 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:13:03.404 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:03 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:03.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:03 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:03.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:03 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:03.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:03 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:03.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:03 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:13:03.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:03 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:03.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:03 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:13:03.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:03 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:03.680 DEBUG:teuthology.orchestra.run.smithi089:osd.1> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T00:13:03.682 INFO:tasks.cephadm:Deploying osd.2 on smithi098 with /dev/vg_nvme/lv_4... 2024-09-21T00:13:03.683 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T00:13:03.934 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:13:04.315 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:04 smithi098 ceph-mon[34725]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:04.315 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:04 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:04.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:04 smithi027 ceph-mon[30745]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:04.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:04 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:04.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:04 smithi089 ceph-mon[34481]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:04.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:04 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:05.979 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:13:05 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-1[39091]: 2024-09-21T00:13:05.646+0000 7fac4167e640 -1 osd.1 0 log_to_monitors true 2024-09-21T00:13:06.238 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:13:06.264 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi098:vg_nvme/lv_4 2024-09-21T00:13:06.288 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:06 smithi098 ceph-mon[34725]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:06.288 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:06 smithi098 ceph-mon[34725]: from='osd.1 [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T00:13:06.288 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:06 smithi098 ceph-mon[34725]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T00:13:06.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:06 smithi027 ceph-mon[30745]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:06.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:06 smithi027 ceph-mon[30745]: from='osd.1 [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T00:13:06.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:06 smithi027 ceph-mon[30745]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T00:13:06.342 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:06 smithi089 ceph-mon[34481]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:06.342 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:06 smithi089 ceph-mon[34481]: from='osd.1 [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T00:13:06.342 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:06 smithi089 ceph-mon[34481]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T00:13:06.516 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:13:06.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.966+0000 7f25e3311640 1 -- 172.21.15.98:0/213148071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25dc110200 msgr2=0x7f25dc112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:13:06.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.966+0000 7f25e3311640 1 --2- 172.21.15.98:0/213148071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25dc110200 0x7f25dc112620 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f25d4009ca0 tx=0x7f25d402a950 comp rx=0 tx=0).stop 2024-09-21T00:13:06.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.966+0000 7f25e3311640 1 -- 172.21.15.98:0/213148071 shutdown_connections 2024-09-21T00:13:06.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.966+0000 7f25e3311640 1 --2- 172.21.15.98:0/213148071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25dc110200 0x7f25dc112620 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:06.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.966+0000 7f25e3311640 1 --2- 172.21.15.98:0/213148071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25dc102fa0 0x7f25dc10fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:06.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.966+0000 7f25e3311640 1 --2- 172.21.15.98:0/213148071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25dc102650 0x7f25dc102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:06.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.966+0000 7f25e3311640 1 -- 172.21.15.98:0/213148071 >> 172.21.15.98:0/213148071 conn(0x7f25dc0797b0 msgr2=0x7f25dc079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:13:06.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.966+0000 7f25e3311640 1 -- 172.21.15.98:0/213148071 shutdown_connections 2024-09-21T00:13:06.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.966+0000 7f25e3311640 1 -- 172.21.15.98:0/213148071 wait complete. 2024-09-21T00:13:06.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.967+0000 7f25e3311640 1 Processor -- start 2024-09-21T00:13:06.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.967+0000 7f25e3311640 1 -- start start 2024-09-21T00:13:06.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.967+0000 7f25e3311640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25dc102650 0x7f25dc19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:13:06.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.967+0000 7f25e3311640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25dc102fa0 0x7f25dc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:13:06.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.967+0000 7f25e3311640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25dc110200 0x7f25dc1a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:13:06.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.967+0000 7f25e3311640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc115d60 con 0x7f25dc110200 2024-09-21T00:13:06.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.967+0000 7f25e3311640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc115be0 con 0x7f25dc102fa0 2024-09-21T00:13:06.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.967+0000 7f25e3311640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc115ee0 con 0x7f25dc102650 2024-09-21T00:13:06.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.967+0000 7f25e10ad640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25dc102650 0x7f25dc19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:13:06.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.967+0000 7f25e10ad640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25dc102650 0x7f25dc19c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40178/0 (socket says 172.21.15.98:40178) 2024-09-21T00:13:06.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.967+0000 7f25e10ad640 1 -- 172.21.15.98:0/1560376390 learned_addr learned my addr 172.21.15.98:0/1560376390 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-21T00:13:06.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.968+0000 7f25e08ac640 1 --2- 172.21.15.98:0/1560376390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25dc102fa0 0x7f25dc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:13:06.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.968+0000 7f25e10ad640 1 -- 172.21.15.98:0/1560376390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25dc102fa0 msgr2=0x7f25dc19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:13:06.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.968+0000 7f25e10ad640 1 --2- 172.21.15.98:0/1560376390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25dc102fa0 0x7f25dc19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:06.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.968+0000 7f25e10ad640 1 -- 172.21.15.98:0/1560376390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25dc110200 msgr2=0x7f25dc1a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:13:06.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.968+0000 7f25e10ad640 1 --2- 172.21.15.98:0/1560376390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25dc110200 0x7f25dc1a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:06.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.968+0000 7f25e10ad640 1 -- 172.21.15.98:0/1560376390 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25dc1a2700 con 0x7f25dc102650 2024-09-21T00:13:06.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.968+0000 7f25e10ad640 1 --2- 172.21.15.98:0/1560376390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25dc102650 0x7f25dc19c790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f25d800b4a0 tx=0x7f25d800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:13:06.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.968+0000 7f25ce7fc640 1 -- 172.21.15.98:0/1560376390 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8014070 con 0x7f25dc102650 2024-09-21T00:13:06.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.968+0000 7f25ce7fc640 1 -- 172.21.15.98:0/1560376390 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d8010ac0 con 0x7f25dc102650 2024-09-21T00:13:06.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.968+0000 7f25e3311640 1 -- 172.21.15.98:0/1560376390 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25dc1a29f0 con 0x7f25dc102650 2024-09-21T00:13:06.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.968+0000 7f25ce7fc640 1 -- 172.21.15.98:0/1560376390 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8017c30 con 0x7f25dc102650 2024-09-21T00:13:06.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.968+0000 7f25e3311640 1 -- 172.21.15.98:0/1560376390 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25dc1a9150 con 0x7f25dc102650 2024-09-21T00:13:06.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.969+0000 7f25e3311640 1 -- 172.21.15.98:0/1560376390 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25dc102ae0 con 0x7f25dc102650 2024-09-21T00:13:07.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.971+0000 7f25ce7fc640 1 -- 172.21.15.98:0/1560376390 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f25d8017070 con 0x7f25dc102650 2024-09-21T00:13:07.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.971+0000 7f25ce7fc640 1 --2- 172.21.15.98:0/1560376390 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f25c80b64d0 0x7f25c80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:13:07.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.971+0000 7f25ce7fc640 1 -- 172.21.15.98:0/1560376390 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2118+0+0 (secure 0 0 0) 0x7f25d80e5100 con 0x7f25dc102650 2024-09-21T00:13:07.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.972+0000 7f25e08ac640 1 --2- 172.21.15.98:0/1560376390 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f25c80b64d0 0x7f25c80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:13:07.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.972+0000 7f25e08ac640 1 --2- 172.21.15.98:0/1560376390 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f25c80b64d0 0x7f25c80b89e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f25dc103fa0 tx=0x7f25d0008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:13:07.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:06.976+0000 7f25ce7fc640 1 -- 172.21.15.98:0/1560376390 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f25d80ad2a0 con 0x7f25dc102650 2024-09-21T00:13:07.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:07.180+0000 7f25e3311640 1 -- 172.21.15.98:0/1560376390 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f25dc109380 con 0x7f25c80b64d0 2024-09-21T00:13:07.182 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T00:13:07.182 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:13:07.182 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: from='osd.1 [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:13:07.182 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T00:13:07.182 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:13:07.183 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:07.183 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:07.183 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:13:07.183 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:13:07.183 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:07.183 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:07.183 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:13:07.183 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:07.183 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T00:13:07.183 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:13:07.183 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:07 smithi098 ceph-mon[34725]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T00:13:07.471 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T00:13:07.471 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:13:07.471 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: from='osd.1 [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:13:07.471 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T00:13:07.471 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:13:07.471 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:07.471 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:07.471 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:13:07.471 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:13:07.472 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:07.472 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:07.472 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:13:07.472 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:07.472 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T00:13:07.472 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:13:07.472 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-mon[34481]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T00:13:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T00:13:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:13:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: from='osd.1 [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:13:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T00:13:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:13:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:13:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:13:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:07.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:07.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:13:07.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:07.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T00:13:07.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:13:07.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:07 smithi027 ceph-mon[30745]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T00:13:08.093 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-1[39091]: 2024-09-21T00:13:07.779+0000 7fac3de15640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T00:13:08.093 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:13:07 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-1[39091]: 2024-09-21T00:13:07.784+0000 7fac353f6640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:13:08.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:08 smithi098 ceph-mon[34725]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:08.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:08 smithi098 ceph-mon[34725]: Detected new or changed devices on smithi089 2024-09-21T00:13:08.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:08 smithi098 ceph-mon[34725]: Adjusting osd_memory_target on smithi089 to 17087M 2024-09-21T00:13:08.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:08 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:13:08.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:08 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:13:08.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:08 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:13:08.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:08 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:08.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:08 smithi098 ceph-mon[34725]: from='osd.1 ' entity='osd.1' 2024-09-21T00:13:08.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:08 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:13:08.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:08 smithi089 ceph-mon[34481]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:08 smithi089 ceph-mon[34481]: Detected new or changed devices on smithi089 2024-09-21T00:13:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:08 smithi089 ceph-mon[34481]: Adjusting osd_memory_target on smithi089 to 17087M 2024-09-21T00:13:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:08 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:13:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:08 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:13:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:08 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:13:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:08 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:08 smithi089 ceph-mon[34481]: from='osd.1 ' entity='osd.1' 2024-09-21T00:13:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:08 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:13:08.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:08 smithi027 ceph-mon[30745]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:08.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:08 smithi027 ceph-mon[30745]: Detected new or changed devices on smithi089 2024-09-21T00:13:08.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:08 smithi027 ceph-mon[30745]: Adjusting osd_memory_target on smithi089 to 17087M 2024-09-21T00:13:08.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:08 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:13:08.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:08 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:13:08.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:08 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:13:08.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:08 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:08.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:08 smithi027 ceph-mon[30745]: from='osd.1 ' entity='osd.1' 2024-09-21T00:13:08.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:08 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:13:09.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:09 smithi089 ceph-mon[34481]: purged_snaps scrub starts 2024-09-21T00:13:09.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:09 smithi089 ceph-mon[34481]: purged_snaps scrub ok 2024-09-21T00:13:09.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:09 smithi089 ceph-mon[34481]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:13:09.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:09 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:13:09.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:09 smithi089 ceph-mon[34481]: osd.1 [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621] boot 2024-09-21T00:13:09.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:09 smithi089 ceph-mon[34481]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T00:13:09.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:09 smithi098 ceph-mon[34725]: purged_snaps scrub starts 2024-09-21T00:13:09.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:09 smithi098 ceph-mon[34725]: purged_snaps scrub ok 2024-09-21T00:13:09.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:09 smithi098 ceph-mon[34725]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:13:09.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:09 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:13:09.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:09 smithi098 ceph-mon[34725]: osd.1 [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621] boot 2024-09-21T00:13:09.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:09 smithi098 ceph-mon[34725]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T00:13:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:09 smithi027 ceph-mon[30745]: purged_snaps scrub starts 2024-09-21T00:13:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:09 smithi027 ceph-mon[30745]: purged_snaps scrub ok 2024-09-21T00:13:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:09 smithi027 ceph-mon[30745]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:13:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:09 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T00:13:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:09 smithi027 ceph-mon[30745]: osd.1 [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621] boot 2024-09-21T00:13:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:09 smithi027 ceph-mon[30745]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T00:13:10.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:10 smithi089 ceph-mon[34481]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:10.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:10 smithi089 ceph-mon[34481]: from='client.? 172.21.15.98:0/1221303110' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6058a875-d5b2-4459-9407-2fd190cb34a9"} : dispatch 2024-09-21T00:13:10.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:10 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6058a875-d5b2-4459-9407-2fd190cb34a9"} : dispatch 2024-09-21T00:13:10.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:10 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6058a875-d5b2-4459-9407-2fd190cb34a9"}]': finished 2024-09-21T00:13:10.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:10 smithi089 ceph-mon[34481]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T00:13:10.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:10 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:10.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:10 smithi089 ceph-mon[34481]: from='client.? 172.21.15.98:0/3320676906' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:13:10.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:10 smithi098 ceph-mon[34725]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:10.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:10 smithi098 ceph-mon[34725]: from='client.? 172.21.15.98:0/1221303110' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6058a875-d5b2-4459-9407-2fd190cb34a9"} : dispatch 2024-09-21T00:13:10.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:10 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6058a875-d5b2-4459-9407-2fd190cb34a9"} : dispatch 2024-09-21T00:13:10.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:10 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6058a875-d5b2-4459-9407-2fd190cb34a9"}]': finished 2024-09-21T00:13:10.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:10 smithi098 ceph-mon[34725]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T00:13:10.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:10 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:10.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:10 smithi098 ceph-mon[34725]: from='client.? 172.21.15.98:0/3320676906' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:13:10.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:10 smithi027 ceph-mon[30745]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T00:13:10.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:10 smithi027 ceph-mon[30745]: from='client.? 172.21.15.98:0/1221303110' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6058a875-d5b2-4459-9407-2fd190cb34a9"} : dispatch 2024-09-21T00:13:10.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:10 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6058a875-d5b2-4459-9407-2fd190cb34a9"} : dispatch 2024-09-21T00:13:10.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:10 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6058a875-d5b2-4459-9407-2fd190cb34a9"}]': finished 2024-09-21T00:13:10.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:10 smithi027 ceph-mon[30745]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T00:13:10.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:10 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:10.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:10 smithi027 ceph-mon[30745]: from='client.? 172.21.15.98:0/3320676906' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:13:11.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:11 smithi089 ceph-mon[34481]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:11.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:11 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:11.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:11 smithi089 ceph-mon[34481]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T00:13:11.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:11 smithi098 ceph-mon[34725]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:11.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:11 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:11.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:11 smithi098 ceph-mon[34725]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T00:13:11.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:11 smithi027 ceph-mon[30745]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:11.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:11 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:11.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:11 smithi027 ceph-mon[30745]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T00:13:13.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:13 smithi089 ceph-mon[34481]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:13.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:13 smithi098 ceph-mon[34725]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:13.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:13 smithi027 ceph-mon[30745]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:15.640 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:15 smithi098 ceph-mon[34725]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:15.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:15 smithi089 ceph-mon[34481]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:15.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:15 smithi027 ceph-mon[30745]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:17.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:17 smithi089 ceph-mon[34481]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:17.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:17 smithi098 ceph-mon[34725]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:17.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:17 smithi027 ceph-mon[30745]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:18.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:18 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T00:13:18.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:18 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:18.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:18 smithi089 ceph-mon[34481]: Deploying daemon osd.2 on smithi098 2024-09-21T00:13:18.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:18 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T00:13:18.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:18 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:18.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:18 smithi098 ceph-mon[34725]: Deploying daemon osd.2 on smithi098 2024-09-21T00:13:18.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:18 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T00:13:18.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:18 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:18.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:18 smithi027 ceph-mon[30745]: Deploying daemon osd.2 on smithi098 2024-09-21T00:13:19.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:19 smithi089 ceph-mon[34481]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:19.748 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:19 smithi098 ceph-mon[34725]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:19.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:19 smithi027 ceph-mon[30745]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:21.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:21 smithi098 ceph-mon[34725]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:21.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:21 smithi027 ceph-mon[30745]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:21.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:21 smithi089 ceph-mon[34481]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:23.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:23 smithi098 ceph-mon[34725]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:23.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:23 smithi027 ceph-mon[30745]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:23.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:23 smithi089 ceph-mon[34481]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:24.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:13:24.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:24.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:24.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:13:24.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:24.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:24.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:13:24.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:24.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:25.863 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:25 smithi098 ceph-mon[34725]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:25.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:25 smithi089 ceph-mon[34481]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:26.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:25 smithi027 ceph-mon[30745]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:27.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:27.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:27.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:27 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:27.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:27 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:13:27.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:27.366 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:27.366 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:27.366 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:27 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:27.366 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:27 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:13:27.366 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:27 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:27 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:13:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:27.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:27.661+0000 7f25ce7fc640 1 -- 172.21.15.98:0/1560376390 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f25dc109380 con 0x7f25c80b64d0 2024-09-21T00:13:27.663 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 2 on host 'smithi098' 2024-09-21T00:13:27.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:27.663+0000 7f25e3311640 1 -- 172.21.15.98:0/1560376390 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f25c80b64d0 msgr2=0x7f25c80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:13:27.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:27.663+0000 7f25e3311640 1 --2- 172.21.15.98:0/1560376390 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f25c80b64d0 0x7f25c80b89e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f25dc103fa0 tx=0x7f25d0008040 comp rx=0 tx=0).stop 2024-09-21T00:13:27.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:27.663+0000 7f25e3311640 1 -- 172.21.15.98:0/1560376390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25dc102650 msgr2=0x7f25dc19c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:13:27.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:27.663+0000 7f25e3311640 1 --2- 172.21.15.98:0/1560376390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25dc102650 0x7f25dc19c790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f25d800b4a0 tx=0x7f25d800b960 comp rx=0 tx=0).stop 2024-09-21T00:13:27.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:27.663+0000 7f25e3311640 1 -- 172.21.15.98:0/1560376390 shutdown_connections 2024-09-21T00:13:27.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:27.663+0000 7f25e3311640 1 --2- 172.21.15.98:0/1560376390 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f25c80b64d0 0x7f25c80b89e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:27.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:27.663+0000 7f25e3311640 1 --2- 172.21.15.98:0/1560376390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25dc110200 0x7f25dc1a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:27.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:27.663+0000 7f25e3311640 1 --2- 172.21.15.98:0/1560376390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25dc102fa0 0x7f25dc19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:27.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:27.663+0000 7f25e3311640 1 --2- 172.21.15.98:0/1560376390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25dc102650 0x7f25dc19c790 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:27.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:27.663+0000 7f25e3311640 1 -- 172.21.15.98:0/1560376390 >> 172.21.15.98:0/1560376390 conn(0x7f25dc0797b0 msgr2=0x7f25dc108c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:13:27.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:27.663+0000 7f25e3311640 1 -- 172.21.15.98:0/1560376390 shutdown_connections 2024-09-21T00:13:27.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:13:27.663+0000 7f25e3311640 1 -- 172.21.15.98:0/1560376390 wait complete. 2024-09-21T00:13:28.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:28 smithi098 ceph-mon[34725]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:28.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:28 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:13:28.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:28 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:28.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:28 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:28.292 DEBUG:teuthology.orchestra.run.smithi098:osd.2> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T00:13:28.335 INFO:tasks.cephadm:Deploying osd.3 on smithi027 with /dev/vg_nvme/lv_3... 2024-09-21T00:13:28.335 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T00:13:28.392 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:28 smithi027 ceph-mon[30745]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:28.392 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:28 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:13:28.392 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:28 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:28.392 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:28 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:28 smithi089 ceph-mon[34481]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:28 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:13:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:28 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:28 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:28.724 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:13:30.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:30 smithi027 ceph-mon[30745]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:30.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:30 smithi027 ceph-mon[30745]: from='osd.2 [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T00:13:30.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:30 smithi027 ceph-mon[30745]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T00:13:30.446 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:30 smithi098 ceph-mon[34725]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:30.446 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:30 smithi098 ceph-mon[34725]: from='osd.2 [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T00:13:30.446 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:30 smithi098 ceph-mon[34725]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T00:13:30.447 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:13:30 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-2[39902]: 2024-09-21T00:13:30.006+0000 7fa4df961640 -1 osd.2 0 log_to_monitors true 2024-09-21T00:13:30.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:30 smithi089 ceph-mon[34481]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:30.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:30 smithi089 ceph-mon[34481]: from='osd.2 [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T00:13:30.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:30 smithi089 ceph-mon[34481]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T00:13:31.307 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:31 smithi027 ceph-mon[30745]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T00:13:31.307 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:31 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:31.307 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:31 smithi027 ceph-mon[30745]: from='osd.2 [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:13:31.308 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:31 smithi027 ceph-mon[30745]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T00:13:31.308 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:31 smithi027 ceph-mon[30745]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:13:31.308 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:31 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:31.308 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:31 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:31.308 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:31 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:13:31.308 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:31 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:13:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:31 smithi089 ceph-mon[34481]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T00:13:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:31 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:31 smithi089 ceph-mon[34481]: from='osd.2 [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:13:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:31 smithi089 ceph-mon[34481]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T00:13:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:31 smithi089 ceph-mon[34481]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:13:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:31 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:31.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:31 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:31.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:31 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:13:31.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:31 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:13:31.496 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:13:31 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-2[39902]: 2024-09-21T00:13:31.112+0000 7fa4db8e3640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T00:13:31.496 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:13:31 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-2[39902]: 2024-09-21T00:13:31.117+0000 7fa4d76e1640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:13:31.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:31 smithi098 ceph-mon[34725]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T00:13:31.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:31 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:31.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:31 smithi098 ceph-mon[34725]: from='osd.2 [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:13:31.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:31 smithi098 ceph-mon[34725]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T00:13:31.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:31 smithi098 ceph-mon[34725]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:13:31.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:31 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:31.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:31 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:31.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:31 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:13:31.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:31 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:13:32.111 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:13:32.147 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi027:vg_nvme/lv_3 2024-09-21T00:13:32.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:32 smithi089 ceph-mon[34481]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:32 smithi089 ceph-mon[34481]: Detected new or changed devices on smithi098 2024-09-21T00:13:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:32 smithi089 ceph-mon[34481]: Adjusting osd_memory_target on smithi098 to 17088M 2024-09-21T00:13:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:32 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:32 smithi089 ceph-mon[34481]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T00:13:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:32 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:32 smithi089 ceph-mon[34481]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T00:13:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:32 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:32 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:32 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:13:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:32 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:32 smithi098 ceph-mon[34725]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:32 smithi098 ceph-mon[34725]: Detected new or changed devices on smithi098 2024-09-21T00:13:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:32 smithi098 ceph-mon[34725]: Adjusting osd_memory_target on smithi098 to 17088M 2024-09-21T00:13:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:32 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:32.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:32 smithi098 ceph-mon[34725]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T00:13:32.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:32 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:32.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:32 smithi098 ceph-mon[34725]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T00:13:32.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:32 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:32.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:32 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:32.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:32 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:13:32.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:32 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:32.534 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:13:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:32 smithi027 ceph-mon[30745]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:32 smithi027 ceph-mon[30745]: Detected new or changed devices on smithi098 2024-09-21T00:13:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:32 smithi027 ceph-mon[30745]: Adjusting osd_memory_target on smithi098 to 17088M 2024-09-21T00:13:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:32 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:32 smithi027 ceph-mon[30745]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T00:13:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:32 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:32 smithi027 ceph-mon[30745]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T00:13:32.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:32 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:32.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:32 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:32.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:32 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:13:32.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:32 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:33.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.279+0000 7f9544a00640 1 -- 172.21.15.27:0/1612888599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9540077940 msgr2=0x7f9540077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:13:33.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.279+0000 7f9544a00640 1 --2- 172.21.15.27:0/1612888599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9540077940 0x7f9540077d50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9530009900 tx=0x7f953002a410 comp rx=0 tx=0).stop 2024-09-21T00:13:33.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.279+0000 7f9544a00640 1 -- 172.21.15.27:0/1612888599 shutdown_connections 2024-09-21T00:13:33.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.279+0000 7f9544a00640 1 --2- 172.21.15.27:0/1612888599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9540114570 0x7f9540116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:33.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.279+0000 7f9544a00640 1 --2- 172.21.15.27:0/1612888599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9540078290 0x7f9540114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:33.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.279+0000 7f9544a00640 1 --2- 172.21.15.27:0/1612888599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9540077940 0x7f9540077d50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:33.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.279+0000 7f9544a00640 1 -- 172.21.15.27:0/1612888599 >> 172.21.15.27:0/1612888599 conn(0x7f9540101680 msgr2=0x7f9540103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:13:33.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.280+0000 7f9544a00640 1 -- 172.21.15.27:0/1612888599 shutdown_connections 2024-09-21T00:13:33.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.280+0000 7f9544a00640 1 -- 172.21.15.27:0/1612888599 wait complete. 2024-09-21T00:13:33.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.281+0000 7f9544a00640 1 Processor -- start 2024-09-21T00:13:33.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.281+0000 7f9544a00640 1 -- start start 2024-09-21T00:13:33.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f9544a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9540077940 0x7f95401a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:13:33.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f9544a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9540078290 0x7f95401a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:13:33.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f9544a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9540114570 0x7f95401a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:13:33.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f9544a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9540119ed0 con 0x7f9540077940 2024-09-21T00:13:33.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f9544a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9540119d50 con 0x7f9540114570 2024-09-21T00:13:33.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f9544a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f954011a050 con 0x7f9540078290 2024-09-21T00:13:33.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f953e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9540077940 0x7f95401a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:13:33.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f953f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9540114570 0x7f95401a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:13:33.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f953e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9540077940 0x7f95401a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47034/0 (socket says 172.21.15.27:47034) 2024-09-21T00:13:33.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f953de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9540078290 0x7f95401a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:13:33.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f953f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9540114570 0x7f95401a6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:46938/0 (socket says 172.21.15.27:46938) 2024-09-21T00:13:33.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f953e800640 1 -- 172.21.15.27:0/2529759184 learned_addr learned my addr 172.21.15.27:0/2529759184 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:13:33.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f953f200640 1 -- 172.21.15.27:0/2529759184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9540078290 msgr2=0x7f95401a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:13:33.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f953f200640 1 --2- 172.21.15.27:0/2529759184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9540078290 0x7f95401a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:33.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f953f200640 1 -- 172.21.15.27:0/2529759184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9540077940 msgr2=0x7f95401a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:13:33.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f953f200640 1 --2- 172.21.15.27:0/2529759184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9540077940 0x7f95401a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:13:33.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.282+0000 7f953f200640 1 -- 172.21.15.27:0/2529759184 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95401a6870 con 0x7f9540114570 2024-09-21T00:13:33.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.283+0000 7f953f200640 1 --2- 172.21.15.27:0/2529759184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9540114570 0x7f95401a6140 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9538009fd0 tx=0x7f953800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:13:33.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.283+0000 7f9537400640 1 -- 172.21.15.27:0/2529759184 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953800ce20 con 0x7f9540114570 2024-09-21T00:13:33.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.283+0000 7f9537400640 1 -- 172.21.15.27:0/2529759184 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9538005020 con 0x7f9540114570 2024-09-21T00:13:33.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.283+0000 7f9544a00640 1 -- 172.21.15.27:0/2529759184 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95401a6b60 con 0x7f9540114570 2024-09-21T00:13:33.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.283+0000 7f9537400640 1 -- 172.21.15.27:0/2529759184 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9538010690 con 0x7f9540114570 2024-09-21T00:13:33.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.283+0000 7f9544a00640 1 -- 172.21.15.27:0/2529759184 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95401ad2c0 con 0x7f9540114570 2024-09-21T00:13:33.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.284+0000 7f9544a00640 1 -- 172.21.15.27:0/2529759184 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9540077d50 con 0x7f9540114570 2024-09-21T00:13:33.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.286+0000 7f9537400640 1 -- 172.21.15.27:0/2529759184 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f95380054d0 con 0x7f9540114570 2024-09-21T00:13:33.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.288+0000 7f9537400640 1 --2- 172.21.15.27:0/2529759184 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f95240b64d0 0x7f95240b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:13:33.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.288+0000 7f9537400640 1 -- 172.21.15.27:0/2529759184 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3441+0+0 (secure 0 0 0) 0x7f95380e66a0 con 0x7f9540114570 2024-09-21T00:13:33.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.288+0000 7f953e800640 1 --2- 172.21.15.27:0/2529759184 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f95240b64d0 0x7f95240b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:13:33.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.289+0000 7f953e800640 1 --2- 172.21.15.27:0/2529759184 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f95240b64d0 0x7f95240b89e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9530031440 tx=0x7f95300058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:13:33.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.293+0000 7f9537400640 1 -- 172.21.15.27:0/2529759184 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f95380ae270 con 0x7f9540114570 2024-09-21T00:13:33.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:33 smithi089 ceph-mon[34481]: purged_snaps scrub starts 2024-09-21T00:13:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:33 smithi089 ceph-mon[34481]: purged_snaps scrub ok 2024-09-21T00:13:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:33 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:33 smithi089 ceph-mon[34481]: osd.2 [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259] boot 2024-09-21T00:13:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:33 smithi089 ceph-mon[34481]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T00:13:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:33 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:33 smithi089 ceph-mon[34481]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:33 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T00:13:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:33 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T00:13:33.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:33 smithi098 ceph-mon[34725]: purged_snaps scrub starts 2024-09-21T00:13:33.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:33 smithi098 ceph-mon[34725]: purged_snaps scrub ok 2024-09-21T00:13:33.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:33 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:33.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:33 smithi098 ceph-mon[34725]: osd.2 [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259] boot 2024-09-21T00:13:33.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:33 smithi098 ceph-mon[34725]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T00:13:33.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:33 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:33.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:33 smithi098 ceph-mon[34725]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:33.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:33 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T00:13:33.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:33 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T00:13:33.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:33.505+0000 7f9544a00640 1 -- 172.21.15.27:0/2529759184 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9540063800 con 0x7f95240b64d0 2024-09-21T00:13:33.507 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:33 smithi027 ceph-mon[30745]: purged_snaps scrub starts 2024-09-21T00:13:33.507 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:33 smithi027 ceph-mon[30745]: purged_snaps scrub ok 2024-09-21T00:13:33.507 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:33 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:33.507 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:33 smithi027 ceph-mon[30745]: osd.2 [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259] boot 2024-09-21T00:13:33.507 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:33 smithi027 ceph-mon[30745]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T00:13:33.508 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:33 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T00:13:33.508 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:33 smithi027 ceph-mon[30745]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T00:13:33.508 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:33 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T00:13:33.508 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:33 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T00:13:34.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T00:13:34.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 ceph-mon[30745]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T00:13:34.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T00:13:34.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T00:13:34.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 ceph-mon[30745]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:13:34.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:13:34.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:13:34.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:34.445 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T00:13:34.445 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 ceph-mon[34481]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T00:13:34.445 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T00:13:34.445 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T00:13:34.445 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 ceph-mon[34481]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:13:34.445 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:13:34.446 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:13:34.446 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:34.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T00:13:34.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 ceph-mon[34725]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T00:13:34.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T00:13:34.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T00:13:34.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 ceph-mon[34725]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:13:34.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:13:34.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:13:34.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:34.602 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49192]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T00:13:34.602 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49192]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T00:13:34.602 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49192]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T00:13:34.602 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49192]: pam_unix(sudo:session): session closed for user root 2024-09-21T00:13:34.602 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49208]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T00:13:34.603 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49208]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T00:13:34.603 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49208]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T00:13:34.603 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49208]: pam_unix(sudo:session): session closed for user root 2024-09-21T00:13:34.711 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42633]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T00:13:34.711 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42633]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T00:13:34.711 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42633]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T00:13:34.711 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42633]: pam_unix(sudo:session): session closed for user root 2024-09-21T00:13:34.711 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42636]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T00:13:34.712 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42636]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T00:13:34.712 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42636]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T00:13:34.712 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42636]: pam_unix(sudo:session): session closed for user root 2024-09-21T00:13:34.859 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49212]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T00:13:34.859 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49212]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T00:13:34.859 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49212]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T00:13:34.859 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49212]: pam_unix(sudo:session): session closed for user root 2024-09-21T00:13:34.859 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49215]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T00:13:34.859 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49215]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T00:13:34.860 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49215]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T00:13:34.860 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:34 smithi027 sudo[49215]: pam_unix(sudo:session): session closed for user root 2024-09-21T00:13:34.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42639]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T00:13:34.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42639]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T00:13:34.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42639]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T00:13:34.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42639]: pam_unix(sudo:session): session closed for user root 2024-09-21T00:13:34.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42642]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T00:13:34.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42642]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T00:13:34.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42642]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T00:13:34.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:34 smithi089 sudo[42642]: pam_unix(sudo:session): session closed for user root 2024-09-21T00:13:34.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43848]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T00:13:34.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43848]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T00:13:34.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43848]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T00:13:34.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43848]: pam_unix(sudo:session): session closed for user root 2024-09-21T00:13:34.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43851]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T00:13:34.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43851]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T00:13:34.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43851]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T00:13:34.999 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43851]: pam_unix(sudo:session): session closed for user root 2024-09-21T00:13:34.999 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43842]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T00:13:34.999 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43842]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T00:13:34.999 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43842]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T00:13:34.999 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43842]: pam_unix(sudo:session): session closed for user root 2024-09-21T00:13:34.999 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43845]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T00:13:34.999 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43845]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T00:13:34.999 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43845]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T00:13:34.999 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:13:34 smithi098 sudo[43845]: pam_unix(sudo:session): session closed for user root 2024-09-21T00:13:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T00:13:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T00:13:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: pgmap v74: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T00:13:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:13:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:13:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:13:35.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T00:13:35.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T00:13:35.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T00:13:35.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:13:35.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:13:35.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:13:35.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T00:13:35.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:13:35.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:13:35.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:13:35.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:35 smithi027 ceph-mon[30745]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T00:13:35.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T00:13:35.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T00:13:35.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: pgmap v74: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:35.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:35.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:35.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:35.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T00:13:35.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:13:35.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:13:35.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:13:35.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T00:13:35.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T00:13:35.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T00:13:35.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:13:35.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:13:35.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:13:35.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T00:13:35.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:13:35.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:13:35.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:13:35.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:35 smithi089 ceph-mon[34481]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T00:13:35.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T00:13:35.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T00:13:35.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: pgmap v74: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:35.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:35.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:35.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:35.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T00:13:35.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:13:35.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:13:35.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:13:35.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T00:13:35.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T00:13:35.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T00:13:35.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:13:35.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:13:35.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:13:35.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T00:13:35.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T00:13:35.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T00:13:35.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T00:13:35.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:35 smithi098 ceph-mon[34725]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T00:13:36.402 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:36 smithi027 ceph-mon[30745]: osdmap e22: 3 total, 3 up, 3 in 2024-09-21T00:13:36.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:36 smithi089 ceph-mon[34481]: osdmap e22: 3 total, 3 up, 3 in 2024-09-21T00:13:36.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:36 smithi098 ceph-mon[34725]: osdmap e22: 3 total, 3 up, 3 in 2024-09-21T00:13:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:13:37.160+0000 7f9537400640 1 -- 172.21.15.27:0/2529759184 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f95380a9de0 con 0x7f9540114570 2024-09-21T00:13:37.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:37 smithi089 ceph-mon[34481]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:37.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:37 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2811506566' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "dfcdaede-e51c-4985-92af-3e296435a36c"} : dispatch 2024-09-21T00:13:37.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:37 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2811506566' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfcdaede-e51c-4985-92af-3e296435a36c"}]': finished 2024-09-21T00:13:37.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:37 smithi089 ceph-mon[34481]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T00:13:37.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:37 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T00:13:37.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:37 smithi098 ceph-mon[34725]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:37.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:37 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2811506566' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "dfcdaede-e51c-4985-92af-3e296435a36c"} : dispatch 2024-09-21T00:13:37.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:37 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2811506566' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfcdaede-e51c-4985-92af-3e296435a36c"}]': finished 2024-09-21T00:13:37.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:37 smithi098 ceph-mon[34725]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T00:13:37.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:37 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T00:13:37.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:37 smithi027 ceph-mon[30745]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:37.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:37 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2811506566' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "dfcdaede-e51c-4985-92af-3e296435a36c"} : dispatch 2024-09-21T00:13:37.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:37 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2811506566' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfcdaede-e51c-4985-92af-3e296435a36c"}]': finished 2024-09-21T00:13:37.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:37 smithi027 ceph-mon[30745]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T00:13:37.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:37 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T00:13:38.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:38 smithi089 ceph-mon[34481]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T00:13:38.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:38 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3857052218' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:13:38.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:38 smithi098 ceph-mon[34725]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T00:13:38.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:38 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3857052218' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:13:38.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:38 smithi027 ceph-mon[30745]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T00:13:38.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:38 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3857052218' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:13:39.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:39 smithi089 ceph-mon[34481]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:39.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:39 smithi098 ceph-mon[34725]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:39.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:39 smithi027 ceph-mon[30745]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:41.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:41 smithi098 ceph-mon[34725]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:41.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:41 smithi027 ceph-mon[30745]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:41.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:41 smithi089 ceph-mon[34481]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:43.468 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:43 smithi027 ceph-mon[30745]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:43.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:43 smithi089 ceph-mon[34481]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:43.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:43 smithi098 ceph-mon[34725]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:45.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:45 smithi089 ceph-mon[34481]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:45.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:45 smithi098 ceph-mon[34725]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:45.779 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:45 smithi027 ceph-mon[30745]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:46.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:46 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T00:13:46.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:46 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:46.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:46 smithi027 ceph-mon[30745]: Deploying daemon osd.3 on smithi027 2024-09-21T00:13:46.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:46 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T00:13:46.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:46 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:46.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:46 smithi089 ceph-mon[34481]: Deploying daemon osd.3 on smithi027 2024-09-21T00:13:46.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:46 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T00:13:46.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:46 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:46.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:46 smithi098 ceph-mon[34725]: Deploying daemon osd.3 on smithi027 2024-09-21T00:13:47.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:47 smithi027 ceph-mon[30745]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:47.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:47 smithi089 ceph-mon[34481]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:47.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:47 smithi098 ceph-mon[34725]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:49.479 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:49 smithi027 ceph-mon[30745]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:49.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:49 smithi089 ceph-mon[34481]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:49.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:49 smithi098 ceph-mon[34725]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:51.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:51 smithi089 ceph-mon[34481]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:51.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:51 smithi098 ceph-mon[34725]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:51 smithi027 ceph-mon[30745]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:53.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:53 smithi027 ceph-mon[30745]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:53.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:53 smithi089 ceph-mon[34481]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:53.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:53 smithi098 ceph-mon[34725]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:55.607 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:55 smithi027 ceph-mon[30745]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:55.607 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:55 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:13:55.607 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:55 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:55.607 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:55 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:55.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:55 smithi089 ceph-mon[34481]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:55.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:55 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:13:55.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:55 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:55.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:55 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:55.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:55 smithi098 ceph-mon[34725]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:55.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:55 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:13:55.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:55 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:55.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:55 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:57.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:57 smithi027 ceph-mon[30745]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:57.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:57 smithi089 ceph-mon[34481]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:57.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:57 smithi098 ceph-mon[34725]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:58.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:58 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:58.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:58 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:58.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:58 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:58.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:58 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:13:58.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:58 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:58.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:58 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:58.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:58 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:58.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:58 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:58.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:58 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:13:58.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:58 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:58.790 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:58 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:58.790 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:58 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:58.790 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:58 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:13:58.791 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:58 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:13:58.791 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:58 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:13:59.705 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:13:59 smithi027 ceph-mon[30745]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:59.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:13:59 smithi089 ceph-mon[34481]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:13:59.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:13:59 smithi098 ceph-mon[34725]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:14:00.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:00.332+0000 7f9537400640 1 -- 172.21.15.27:0/2529759184 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9540063800 con 0x7f95240b64d0 2024-09-21T00:14:00.334 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 3 on host 'smithi027' 2024-09-21T00:14:00.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:00.336+0000 7f9544a00640 1 -- 172.21.15.27:0/2529759184 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f95240b64d0 msgr2=0x7f95240b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:14:00.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:00.336+0000 7f9544a00640 1 --2- 172.21.15.27:0/2529759184 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f95240b64d0 0x7f95240b89e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9530031440 tx=0x7f95300058d0 comp rx=0 tx=0).stop 2024-09-21T00:14:00.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:00.336+0000 7f9544a00640 1 -- 172.21.15.27:0/2529759184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9540114570 msgr2=0x7f95401a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:14:00.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:00.336+0000 7f9544a00640 1 --2- 172.21.15.27:0/2529759184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9540114570 0x7f95401a6140 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9538009fd0 tx=0x7f953800ef10 comp rx=0 tx=0).stop 2024-09-21T00:14:00.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:00.336+0000 7f9544a00640 1 -- 172.21.15.27:0/2529759184 shutdown_connections 2024-09-21T00:14:00.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:00.336+0000 7f9544a00640 1 --2- 172.21.15.27:0/2529759184 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f95240b64d0 0x7f95240b89e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:00.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:00.336+0000 7f9544a00640 1 --2- 172.21.15.27:0/2529759184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9540114570 0x7f95401a6140 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:00.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:00.336+0000 7f9544a00640 1 --2- 172.21.15.27:0/2529759184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9540078290 0x7f95401a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:00.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:00.336+0000 7f9544a00640 1 --2- 172.21.15.27:0/2529759184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9540077940 0x7f95401a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:00.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:00.336+0000 7f9544a00640 1 -- 172.21.15.27:0/2529759184 >> 172.21.15.27:0/2529759184 conn(0x7f9540101680 msgr2=0x7f9540116600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:14:00.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:00.336+0000 7f9544a00640 1 -- 172.21.15.27:0/2529759184 shutdown_connections 2024-09-21T00:14:00.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:00.336+0000 7f9544a00640 1 -- 172.21.15.27:0/2529759184 wait complete. 2024-09-21T00:14:00.610 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:00 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:14:00.610 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:00 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:00.610 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:00 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:00.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:00 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:14:00.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:00 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:00.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:00 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:00.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:00 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:14:00.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:00 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:00.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:00 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:01.103 DEBUG:teuthology.orchestra.run.smithi027:osd.3> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T00:14:01.107 INFO:tasks.cephadm:Deploying osd.4 on smithi089 with /dev/vg_nvme/lv_3... 2024-09-21T00:14:01.108 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T00:14:01.360 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T00:14:01.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:01 smithi089 ceph-mon[34481]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:14:01.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:01 smithi089 ceph-mon[34481]: from='osd.3 [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T00:14:01.621 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:01 smithi027 ceph-mon[30745]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:14:01.621 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:01 smithi027 ceph-mon[30745]: from='osd.3 [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T00:14:01.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:01 smithi098 ceph-mon[34725]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:14:01.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:01 smithi098 ceph-mon[34725]: from='osd.3 [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T00:14:02.399 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:02 smithi027 ceph-mon[30745]: from='osd.3 [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T00:14:02.399 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:02 smithi027 ceph-mon[30745]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T00:14:02.399 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:02 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T00:14:02.399 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:02 smithi027 ceph-mon[30745]: from='osd.3 [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:14:02.697 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:14:02 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-3[51258]: 2024-09-21T00:14:02.397+0000 7f3674200640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T00:14:02.697 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:14:02 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-3[51258]: 2024-09-21T00:14:02.404+0000 7f366f200640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:14:02.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:02 smithi089 ceph-mon[34481]: from='osd.3 [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T00:14:02.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:02 smithi089 ceph-mon[34481]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T00:14:02.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:02 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T00:14:02.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:02 smithi089 ceph-mon[34481]: from='osd.3 [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:14:02.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:02 smithi098 ceph-mon[34725]: from='osd.3 [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T00:14:02.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:02 smithi098 ceph-mon[34725]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T00:14:02.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:02 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T00:14:02.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:02 smithi098 ceph-mon[34725]: from='osd.3 [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:14:03.653 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:03 smithi089 ceph-mon[34481]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:14:03.653 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:03 smithi089 ceph-mon[34481]: from='osd.3 [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T00:14:03.653 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:03 smithi089 ceph-mon[34481]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T00:14:03.653 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:03 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T00:14:03.653 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:03 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T00:14:03.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:03 smithi098 ceph-mon[34725]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:14:03.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:03 smithi098 ceph-mon[34725]: from='osd.3 [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T00:14:03.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:03 smithi098 ceph-mon[34725]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T00:14:03.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:03 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T00:14:03.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:03 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T00:14:03.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:03 smithi027 ceph-mon[30745]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:14:03.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:03 smithi027 ceph-mon[30745]: from='osd.3 [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T00:14:03.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:03 smithi027 ceph-mon[30745]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T00:14:03.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:03 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T00:14:03.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:03 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T00:14:03.956 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:14:03.989 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi089:vg_nvme/lv_3 2024-09-21T00:14:04.236 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T00:14:04.606 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:04 smithi089 ceph-mon[34481]: purged_snaps scrub starts 2024-09-21T00:14:04.607 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:04 smithi089 ceph-mon[34481]: purged_snaps scrub ok 2024-09-21T00:14:04.607 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:04 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T00:14:04.607 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:04 smithi089 ceph-mon[34481]: osd.3 [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824] boot 2024-09-21T00:14:04.607 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:04 smithi089 ceph-mon[34481]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T00:14:04.607 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:04 smithi089 ceph-mon[34481]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T00:14:04.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.699+0000 7f049cc52640 1 -- 172.21.15.89:0/513200031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102a80 msgr2=0x7f0498102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:14:04.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.699+0000 7f049cc52640 1 --2- 172.21.15.89:0/513200031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102a80 0x7f0498102e90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0488009900 tx=0x7f048802a410 comp rx=0 tx=0).stop 2024-09-21T00:14:04.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.699+0000 7f049cc52640 1 -- 172.21.15.89:0/513200031 shutdown_connections 2024-09-21T00:14:04.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.699+0000 7f049cc52640 1 --2- 172.21.15.89:0/513200031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0498108d10 0x7f049810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:04.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.699+0000 7f049cc52640 1 --2- 172.21.15.89:0/513200031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04981083c0 0x7f04981087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:04.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.699+0000 7f049cc52640 1 --2- 172.21.15.89:0/513200031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102a80 0x7f0498102e90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:04.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.699+0000 7f049cc52640 1 -- 172.21.15.89:0/513200031 >> 172.21.15.89:0/513200031 conn(0x7f0498079740 msgr2=0x7f0498079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:14:04.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.699+0000 7f049cc52640 1 -- 172.21.15.89:0/513200031 shutdown_connections 2024-09-21T00:14:04.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.699+0000 7f049cc52640 1 -- 172.21.15.89:0/513200031 wait complete. 2024-09-21T00:14:04.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049cc52640 1 Processor -- start 2024-09-21T00:14:04.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049cc52640 1 -- start start 2024-09-21T00:14:04.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049cc52640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102a80 0x7f0498199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:14:04.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049cc52640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04981083c0 0x7f0498199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:14:04.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049cc52640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0498108d10 0x7f049819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:14:04.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049cc52640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0498112980 con 0x7f0498108d10 2024-09-21T00:14:04.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049cc52640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0498112800 con 0x7f0498102a80 2024-09-21T00:14:04.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049cc52640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0498112b00 con 0x7f04981083c0 2024-09-21T00:14:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102a80 0x7f0498199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:14:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102a80 0x7f0498199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50336/0 (socket says 172.21.15.89:50336) 2024-09-21T00:14:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049659c640 1 -- 172.21.15.89:0/3237202918 learned_addr learned my addr 172.21.15.89:0/3237202918 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T00:14:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f0496d9d640 1 --2- 172.21.15.89:0/3237202918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0498108d10 0x7f049819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:14:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f048ffff640 1 --2- 172.21.15.89:0/3237202918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04981083c0 0x7f0498199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:14:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049659c640 1 -- 172.21.15.89:0/3237202918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04981083c0 msgr2=0x7f0498199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:14:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049659c640 1 --2- 172.21.15.89:0/3237202918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04981083c0 0x7f0498199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049659c640 1 -- 172.21.15.89:0/3237202918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0498108d10 msgr2=0x7f049819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:14:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049659c640 1 --2- 172.21.15.89:0/3237202918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0498108d10 0x7f049819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f049659c640 1 -- 172.21.15.89:0/3237202918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049819f2e0 con 0x7f0498102a80 2024-09-21T00:14:04.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.700+0000 7f048ffff640 1 --2- 172.21.15.89:0/3237202918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04981083c0 0x7f0498199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:14:04.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.701+0000 7f049659c640 1 --2- 172.21.15.89:0/3237202918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102a80 0x7f0498199370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0488005940 tx=0x7f0488004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:14:04.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.701+0000 7f048f7fe640 1 -- 172.21.15.89:0/3237202918 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048803f030 con 0x7f0498102a80 2024-09-21T00:14:04.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.701+0000 7f048f7fe640 1 -- 172.21.15.89:0/3237202918 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0488003aa0 con 0x7f0498102a80 2024-09-21T00:14:04.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.701+0000 7f048f7fe640 1 -- 172.21.15.89:0/3237202918 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488035070 con 0x7f0498102a80 2024-09-21T00:14:04.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.701+0000 7f049cc52640 1 -- 172.21.15.89:0/3237202918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f049819f570 con 0x7f0498102a80 2024-09-21T00:14:04.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.701+0000 7f049cc52640 1 -- 172.21.15.89:0/3237202918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f049806bb40 con 0x7f0498102a80 2024-09-21T00:14:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.702+0000 7f049cc52640 1 -- 172.21.15.89:0/3237202918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0498103d00 con 0x7f0498102a80 2024-09-21T00:14:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.703+0000 7f048f7fe640 1 -- 172.21.15.89:0/3237202918 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f0488042030 con 0x7f0498102a80 2024-09-21T00:14:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.704+0000 7f048f7fe640 1 --2- 172.21.15.89:0/3237202918 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f04840b6560 0x7f04840b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:14:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.704+0000 7f048f7fe640 1 -- 172.21.15.89:0/3237202918 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3884+0+0 (secure 0 0 0) 0x7f0488106c30 con 0x7f0498102a80 2024-09-21T00:14:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.705+0000 7f048ffff640 1 --2- 172.21.15.89:0/3237202918 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f04840b6560 0x7f04840b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:14:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.705+0000 7f048ffff640 1 --2- 172.21.15.89:0/3237202918 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f04840b6560 0x7f04840b8a70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0498107ed0 tx=0x7f0480005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:14:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.707+0000 7f048f7fe640 1 -- 172.21.15.89:0/3237202918 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f04880ce610 con 0x7f0498102a80 2024-09-21T00:14:04.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:04 smithi098 ceph-mon[34725]: purged_snaps scrub starts 2024-09-21T00:14:04.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:04 smithi098 ceph-mon[34725]: purged_snaps scrub ok 2024-09-21T00:14:04.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:04 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T00:14:04.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:04 smithi098 ceph-mon[34725]: osd.3 [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824] boot 2024-09-21T00:14:04.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:04 smithi098 ceph-mon[34725]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T00:14:04.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:04 smithi098 ceph-mon[34725]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T00:14:04.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:04 smithi027 ceph-mon[30745]: purged_snaps scrub starts 2024-09-21T00:14:04.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:04 smithi027 ceph-mon[30745]: purged_snaps scrub ok 2024-09-21T00:14:04.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:04 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T00:14:04.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:04 smithi027 ceph-mon[30745]: osd.3 [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824] boot 2024-09-21T00:14:04.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:04 smithi027 ceph-mon[30745]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T00:14:04.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:04 smithi027 ceph-mon[30745]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T00:14:04.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:04.898+0000 7f049cc52640 1 -- 172.21.15.89:0/3237202918 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f049806bda0 con 0x7f04840b6560 2024-09-21T00:14:05.601 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: pgmap v94: 1 pgs: 1 remapped+peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:14:05.601 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: from='client.24437 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:14:05.601 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:14:05.601 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:14:05.601 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:05.601 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: Detected new or changed devices on smithi027 2024-09-21T00:14:05.602 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:05.602 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:05.602 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:05.602 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:05.602 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:05.602 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:05.602 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: Adjusting osd_memory_target on smithi027 to 8544M 2024-09-21T00:14:05.602 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:05.602 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:05.602 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:14:05.602 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:05.602 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:05 smithi089 ceph-mon[34481]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T00:14:05.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: pgmap v94: 1 pgs: 1 remapped+peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:14:05.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: from='client.24437 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:14:05.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:14:05.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:14:05.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:05.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: Detected new or changed devices on smithi027 2024-09-21T00:14:05.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:05.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:05.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:05.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:05.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:05.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:05.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: Adjusting osd_memory_target on smithi027 to 8544M 2024-09-21T00:14:05.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:05.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:05.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:14:05.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:05.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:05 smithi098 ceph-mon[34725]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T00:14:05.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: pgmap v94: 1 pgs: 1 remapped+peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T00:14:05.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: from='client.24437 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:14:05.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:14:05.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:14:05.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:05.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: Detected new or changed devices on smithi027 2024-09-21T00:14:05.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:05.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:05.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:05.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:05.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:05.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:05.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: Adjusting osd_memory_target on smithi027 to 8544M 2024-09-21T00:14:05.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:05.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:05.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:14:05.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:05.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:05 smithi027 ceph-mon[30745]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T00:14:07.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:07 smithi089 ceph-mon[34481]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:07 smithi089 ceph-mon[34481]: from='client.? 172.21.15.89:0/3730717936' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7c4af037-8ab6-47c0-b62b-76bd775b3491"} : dispatch 2024-09-21T00:14:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:07 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7c4af037-8ab6-47c0-b62b-76bd775b3491"} : dispatch 2024-09-21T00:14:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:07 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c4af037-8ab6-47c0-b62b-76bd775b3491"}]': finished 2024-09-21T00:14:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:07 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:07 smithi089 ceph-mon[34481]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T00:14:07.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:07 smithi098 ceph-mon[34725]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:07.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:07 smithi098 ceph-mon[34725]: from='client.? 172.21.15.89:0/3730717936' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7c4af037-8ab6-47c0-b62b-76bd775b3491"} : dispatch 2024-09-21T00:14:07.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:07 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7c4af037-8ab6-47c0-b62b-76bd775b3491"} : dispatch 2024-09-21T00:14:07.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:07 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c4af037-8ab6-47c0-b62b-76bd775b3491"}]': finished 2024-09-21T00:14:07.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:07 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:07.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:07 smithi098 ceph-mon[34725]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T00:14:07.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:07 smithi027 ceph-mon[30745]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:07.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:07 smithi027 ceph-mon[30745]: from='client.? 172.21.15.89:0/3730717936' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7c4af037-8ab6-47c0-b62b-76bd775b3491"} : dispatch 2024-09-21T00:14:07.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:07 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7c4af037-8ab6-47c0-b62b-76bd775b3491"} : dispatch 2024-09-21T00:14:07.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:07 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c4af037-8ab6-47c0-b62b-76bd775b3491"}]': finished 2024-09-21T00:14:07.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:07 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:07.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:07 smithi027 ceph-mon[30745]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T00:14:08.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:08 smithi089 ceph-mon[34481]: from='client.? 172.21.15.89:0/2994878832' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:14:08.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:08 smithi098 ceph-mon[34725]: from='client.? 172.21.15.89:0/2994878832' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:14:08.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:08 smithi027 ceph-mon[30745]: from='client.? 172.21.15.89:0/2994878832' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:14:09.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:09 smithi089 ceph-mon[34481]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:09.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:09 smithi098 ceph-mon[34725]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:09.798 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:09 smithi027 ceph-mon[30745]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:11.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:11 smithi089 ceph-mon[34481]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T00:14:11.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:11 smithi098 ceph-mon[34725]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T00:14:11.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:11 smithi027 ceph-mon[30745]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T00:14:13.587 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:13 smithi089 ceph-mon[34481]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T00:14:13.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:13 smithi098 ceph-mon[34725]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T00:14:13.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:13 smithi027 ceph-mon[30745]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T00:14:15.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:15 smithi089 ceph-mon[34481]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T00:14:15.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:15 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T00:14:15.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:15 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:15.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:15 smithi098 ceph-mon[34725]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T00:14:15.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:15 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T00:14:15.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:15 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:15.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:15 smithi027 ceph-mon[30745]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T00:14:15.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:15 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T00:14:15.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:15 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:16.691 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:16 smithi089 ceph-mon[34481]: Deploying daemon osd.4 on smithi089 2024-09-21T00:14:16.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:16 smithi098 ceph-mon[34725]: Deploying daemon osd.4 on smithi089 2024-09-21T00:14:16.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:16 smithi027 ceph-mon[30745]: Deploying daemon osd.4 on smithi089 2024-09-21T00:14:17.698 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:17 smithi089 ceph-mon[34481]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T00:14:17.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:17 smithi098 ceph-mon[34725]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T00:14:17.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:17 smithi027 ceph-mon[30745]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T00:14:19.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:19 smithi089 ceph-mon[34481]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T00:14:19.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:19 smithi098 ceph-mon[34725]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T00:14:19.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:19 smithi027 ceph-mon[30745]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T00:14:21.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:21 smithi089 ceph-mon[34481]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T00:14:21.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:21 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:14:21.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:21 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:21.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:21 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:21.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:21 smithi098 ceph-mon[34725]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T00:14:21.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:21 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:14:21.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:21 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:21.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:21 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:21.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:21 smithi027 ceph-mon[30745]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T00:14:21.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:21 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:14:21.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:21 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:21.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:21 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:23.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:23 smithi089 ceph-mon[34481]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:23.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:23 smithi098 ceph-mon[34725]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:23.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:23 smithi027 ceph-mon[30745]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:25.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:25.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:25.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:25 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:25.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:25 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:14:25.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:25.403 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:25.403 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:25.403 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:25 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:25.403 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:25 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:14:25.403 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:25.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:25.451+0000 7f048f7fe640 1 -- 172.21.15.89:0/3237202918 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f049806bda0 con 0x7f04840b6560 2024-09-21T00:14:25.453 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 4 on host 'smithi089' 2024-09-21T00:14:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:25.454+0000 7f049cc52640 1 -- 172.21.15.89:0/3237202918 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f04840b6560 msgr2=0x7f04840b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:14:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:25.454+0000 7f049cc52640 1 --2- 172.21.15.89:0/3237202918 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f04840b6560 0x7f04840b8a70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0498107ed0 tx=0x7f0480005f90 comp rx=0 tx=0).stop 2024-09-21T00:14:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:25.454+0000 7f049cc52640 1 -- 172.21.15.89:0/3237202918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102a80 msgr2=0x7f0498199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:14:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:25.454+0000 7f049cc52640 1 --2- 172.21.15.89:0/3237202918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102a80 0x7f0498199370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0488005940 tx=0x7f0488004270 comp rx=0 tx=0).stop 2024-09-21T00:14:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:25.454+0000 7f049cc52640 1 -- 172.21.15.89:0/3237202918 shutdown_connections 2024-09-21T00:14:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:25.454+0000 7f049cc52640 1 --2- 172.21.15.89:0/3237202918 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f04840b6560 0x7f04840b8a70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:25.454+0000 7f049cc52640 1 --2- 172.21.15.89:0/3237202918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0498108d10 0x7f049819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:25.454+0000 7f049cc52640 1 --2- 172.21.15.89:0/3237202918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04981083c0 0x7f0498199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:25.454+0000 7f049cc52640 1 --2- 172.21.15.89:0/3237202918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102a80 0x7f0498199370 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:25.454+0000 7f049cc52640 1 -- 172.21.15.89:0/3237202918 >> 172.21.15.89:0/3237202918 conn(0x7f0498079740 msgr2=0x7f04981013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:14:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:25.454+0000 7f049cc52640 1 -- 172.21.15.89:0/3237202918 shutdown_connections 2024-09-21T00:14:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:14:25.454+0000 7f049cc52640 1 -- 172.21.15.89:0/3237202918 wait complete. 2024-09-21T00:14:25.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:25.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:25.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:25 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:25.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:25 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:14:25.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:26.060 DEBUG:teuthology.orchestra.run.smithi089:osd.4> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T00:14:26.062 INFO:tasks.cephadm:Deploying osd.5 on smithi098 with /dev/vg_nvme/lv_3... 2024-09-21T00:14:26.063 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T00:14:26.315 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:14:26.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:26 smithi027 ceph-mon[30745]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:26.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:26 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:14:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:26 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:26 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:26.372 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:26 smithi089 ceph-mon[34481]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:26.372 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:26 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:14:26.372 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:26 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:26.372 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:26 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:26.435 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:26 smithi098 ceph-mon[34725]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:26.435 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:26 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:14:26.435 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:26 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:26.435 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:26 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:27.979 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:14:27 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4[45233]: 2024-09-21T00:14:27.617+0000 7f5b676ed640 -1 osd.4 0 log_to_monitors true 2024-09-21T00:14:28.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:28 smithi027 ceph-mon[30745]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:28 smithi027 ceph-mon[30745]: from='osd.4 [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T00:14:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:28 smithi027 ceph-mon[30745]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T00:14:28.418 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:28 smithi098 ceph-mon[34725]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:28.418 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:28 smithi098 ceph-mon[34725]: from='osd.4 [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T00:14:28.418 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:28 smithi098 ceph-mon[34725]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T00:14:28.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:28 smithi089 ceph-mon[34481]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:28 smithi089 ceph-mon[34481]: from='osd.4 [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T00:14:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:28 smithi089 ceph-mon[34481]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T00:14:28.810 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:14:28.841 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi098:vg_nvme/lv_3 2024-09-21T00:14:29.093 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:14:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T00:14:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: from='osd.4 [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:14:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T00:14:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:14:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:29.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:29.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:29.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:29.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:14:29.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:29.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-mon[34481]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T00:14:29.334 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T00:14:29.334 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:29.334 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: from='osd.4 [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:14:29.334 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T00:14:29.334 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:14:29.334 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:29.334 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:29.335 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:29.335 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:29.335 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:29.335 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:29.335 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:29.335 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:29.335 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:14:29.335 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:29.335 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:29 smithi098 ceph-mon[34725]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T00:14:29.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T00:14:29.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:29.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: from='osd.4 [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:14:29.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T00:14:29.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:14:29.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:29.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:29.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:29.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:29.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:29.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:29.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:29.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:29.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:14:29.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:29.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:29 smithi027 ceph-mon[30745]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T00:14:29.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.594+0000 7f2b1f535640 1 -- 172.21.15.98:0/3443496780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b181094a0 msgr2=0x7f2b181098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:14:29.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.594+0000 7f2b1f535640 1 --2- 172.21.15.98:0/3443496780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b181094a0 0x7f2b181098b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009900 tx=0x7f2b0802a410 comp rx=0 tx=0).stop 2024-09-21T00:14:29.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.594+0000 7f2b1f535640 1 -- 172.21.15.98:0/3443496780 shutdown_connections 2024-09-21T00:14:29.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.594+0000 7f2b1f535640 1 --2- 172.21.15.98:0/3443496780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b18109df0 0x7f2b18110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:29.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.594+0000 7f2b1f535640 1 --2- 172.21.15.98:0/3443496780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b181094a0 0x7f2b181098b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:29.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.594+0000 7f2b1f535640 1 --2- 172.21.15.98:0/3443496780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b18103b60 0x7f2b18103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:29.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.594+0000 7f2b1f535640 1 -- 172.21.15.98:0/3443496780 >> 172.21.15.98:0/3443496780 conn(0x7f2b1807b850 msgr2=0x7f2b1807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:14:29.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.594+0000 7f2b1f535640 1 -- 172.21.15.98:0/3443496780 shutdown_connections 2024-09-21T00:14:29.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.594+0000 7f2b1f535640 1 -- 172.21.15.98:0/3443496780 wait complete. 2024-09-21T00:14:29.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.594+0000 7f2b1f535640 1 Processor -- start 2024-09-21T00:14:29.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1f535640 1 -- start start 2024-09-21T00:14:29.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1f535640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b18103b60 0x7f2b18075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:14:29.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1f535640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b181094a0 0x7f2b18075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:14:29.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1f535640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b18109df0 0x7f2b18076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:14:29.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1f535640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18113ab0 con 0x7f2b18109df0 2024-09-21T00:14:29.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1f535640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18113930 con 0x7f2b18103b60 2024-09-21T00:14:29.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1f535640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18113c30 con 0x7f2b181094a0 2024-09-21T00:14:29.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1cad0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b181094a0 0x7f2b18075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:14:29.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1cad0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b181094a0 0x7f2b18075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33432/0 (socket says 172.21.15.98:33432) 2024-09-21T00:14:29.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1cad0640 1 -- 172.21.15.98:0/1412744960 learned_addr learned my addr 172.21.15.98:0/1412744960 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-21T00:14:29.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1d2d1640 1 --2- 172.21.15.98:0/1412744960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b18103b60 0x7f2b18075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:14:29.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1dad2640 1 --2- 172.21.15.98:0/1412744960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b18109df0 0x7f2b18076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:14:29.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1cad0640 1 -- 172.21.15.98:0/1412744960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b18103b60 msgr2=0x7f2b18075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:14:29.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1cad0640 1 --2- 172.21.15.98:0/1412744960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b18103b60 0x7f2b18075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:29.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1cad0640 1 -- 172.21.15.98:0/1412744960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b18109df0 msgr2=0x7f2b18076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:14:29.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1cad0640 1 --2- 172.21.15.98:0/1412744960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b18109df0 0x7f2b18076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:29.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1cad0640 1 -- 172.21.15.98:0/1412744960 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b181a87c0 con 0x7f2b181094a0 2024-09-21T00:14:29.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1d2d1640 1 --2- 172.21.15.98:0/1412744960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b18103b60 0x7f2b18075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:14:29.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.595+0000 7f2b1cad0640 1 --2- 172.21.15.98:0/1412744960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b181094a0 0x7f2b18075c50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2b08006010 tx=0x7f2b08004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:14:29.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.596+0000 7f2b0e7fc640 1 -- 172.21.15.98:0/1412744960 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0803f030 con 0x7f2b181094a0 2024-09-21T00:14:29.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.596+0000 7f2b0e7fc640 1 -- 172.21.15.98:0/1412744960 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b08003aa0 con 0x7f2b181094a0 2024-09-21T00:14:29.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.596+0000 7f2b1f535640 1 -- 172.21.15.98:0/1412744960 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b181a8a50 con 0x7f2b181094a0 2024-09-21T00:14:29.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.596+0000 7f2b0e7fc640 1 -- 172.21.15.98:0/1412744960 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b08035070 con 0x7f2b181094a0 2024-09-21T00:14:29.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.596+0000 7f2b1f535640 1 -- 172.21.15.98:0/1412744960 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b181a8f30 con 0x7f2b181094a0 2024-09-21T00:14:29.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.597+0000 7f2b1f535640 1 -- 172.21.15.98:0/1412744960 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b18103f70 con 0x7f2b181094a0 2024-09-21T00:14:29.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.599+0000 7f2b0e7fc640 1 -- 172.21.15.98:0/1412744960 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f2b08042030 con 0x7f2b181094a0 2024-09-21T00:14:29.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.600+0000 7f2b0e7fc640 1 --2- 172.21.15.98:0/1412744960 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2b040b6560 0x7f2b040b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:14:29.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.600+0000 7f2b0e7fc640 1 -- 172.21.15.98:0/1412744960 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4036+0+0 (secure 0 0 0) 0x7f2b08106b50 con 0x7f2b181094a0 2024-09-21T00:14:29.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.600+0000 7f2b1d2d1640 1 --2- 172.21.15.98:0/1412744960 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2b040b6560 0x7f2b040b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:14:29.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.600+0000 7f2b1d2d1640 1 --2- 172.21.15.98:0/1412744960 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2b040b6560 0x7f2b040b8a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b18077160 tx=0x7f2b14005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:14:29.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.603+0000 7f2b0e7fc640 1 -- 172.21.15.98:0/1412744960 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2b080d2870 con 0x7f2b181094a0 2024-09-21T00:14:29.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:29.790+0000 7f2b1f535640 1 -- 172.21.15.98:0/1412744960 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2b1810bc10 con 0x7f2b040b6560 2024-09-21T00:14:30.086 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4[45233]: 2024-09-21T00:14:29.738+0000 7f5b63e84640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T00:14:30.086 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:14:29 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4[45233]: 2024-09-21T00:14:29.745+0000 7f5b5b465640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:14:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:30 smithi098 ceph-mon[34725]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:30 smithi098 ceph-mon[34725]: Detected new or changed devices on smithi089 2024-09-21T00:14:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:30 smithi098 ceph-mon[34725]: Adjusting osd_memory_target on smithi089 to 8543M 2024-09-21T00:14:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:30 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:30 smithi098 ceph-mon[34725]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T00:14:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:30 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:30 smithi098 ceph-mon[34725]: from='osd.4 ' entity='osd.4' 2024-09-21T00:14:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:30 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:14:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:30 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:14:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:30 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:30.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:30 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:30.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:30 smithi089 ceph-mon[34481]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:30.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:30 smithi089 ceph-mon[34481]: Detected new or changed devices on smithi089 2024-09-21T00:14:30.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:30 smithi089 ceph-mon[34481]: Adjusting osd_memory_target on smithi089 to 8543M 2024-09-21T00:14:30.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:30 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:30.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:30 smithi089 ceph-mon[34481]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T00:14:30.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:30 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:30.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:30 smithi089 ceph-mon[34481]: from='osd.4 ' entity='osd.4' 2024-09-21T00:14:30.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:30 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:14:30.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:30 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:14:30.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:30 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:30.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:30 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:30.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:30 smithi027 ceph-mon[30745]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:30.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:30 smithi027 ceph-mon[30745]: Detected new or changed devices on smithi089 2024-09-21T00:14:30.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:30 smithi027 ceph-mon[30745]: Adjusting osd_memory_target on smithi089 to 8543M 2024-09-21T00:14:30.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:30 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:30.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:30 smithi027 ceph-mon[30745]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T00:14:30.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:30 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:30.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:30 smithi027 ceph-mon[30745]: from='osd.4 ' entity='osd.4' 2024-09-21T00:14:30.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:30 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:14:30.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:30 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:14:30.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:30 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:30.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:30 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:31.176 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:31 smithi098 ceph-mon[34725]: purged_snaps scrub starts 2024-09-21T00:14:31.176 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:31 smithi098 ceph-mon[34725]: purged_snaps scrub ok 2024-09-21T00:14:31.176 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:31 smithi098 ceph-mon[34725]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:14:31.176 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:31 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:31.176 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:31 smithi098 ceph-mon[34725]: osd.4 [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690] boot 2024-09-21T00:14:31.177 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:31 smithi098 ceph-mon[34725]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T00:14:31.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:31 smithi089 ceph-mon[34481]: purged_snaps scrub starts 2024-09-21T00:14:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:31 smithi089 ceph-mon[34481]: purged_snaps scrub ok 2024-09-21T00:14:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:31 smithi089 ceph-mon[34481]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:14:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:31 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:31 smithi089 ceph-mon[34481]: osd.4 [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690] boot 2024-09-21T00:14:31.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:31 smithi089 ceph-mon[34481]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T00:14:31.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:31 smithi027 ceph-mon[30745]: purged_snaps scrub starts 2024-09-21T00:14:31.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:31 smithi027 ceph-mon[30745]: purged_snaps scrub ok 2024-09-21T00:14:31.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:31 smithi027 ceph-mon[30745]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:14:31.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:31 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T00:14:31.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:31 smithi027 ceph-mon[30745]: osd.4 [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690] boot 2024-09-21T00:14:31.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:31 smithi027 ceph-mon[30745]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T00:14:32.458 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:32 smithi098 ceph-mon[34725]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:32.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:32 smithi089 ceph-mon[34481]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:32 smithi027 ceph-mon[30745]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T00:14:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:33 smithi089 ceph-mon[34481]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T00:14:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:33 smithi089 ceph-mon[34481]: from='client.? 172.21.15.98:0/1275651620' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e996ed0d-88ad-4aa7-891f-dfade4e09cb0"} : dispatch 2024-09-21T00:14:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:33 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e996ed0d-88ad-4aa7-891f-dfade4e09cb0"} : dispatch 2024-09-21T00:14:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:33 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e996ed0d-88ad-4aa7-891f-dfade4e09cb0"}]': finished 2024-09-21T00:14:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:33 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T00:14:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:33 smithi089 ceph-mon[34481]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T00:14:33.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:33 smithi089 ceph-mon[34481]: from='client.? 172.21.15.98:0/2243877550' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:14:33.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:33 smithi098 ceph-mon[34725]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T00:14:33.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:33 smithi098 ceph-mon[34725]: from='client.? 172.21.15.98:0/1275651620' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e996ed0d-88ad-4aa7-891f-dfade4e09cb0"} : dispatch 2024-09-21T00:14:33.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:33 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e996ed0d-88ad-4aa7-891f-dfade4e09cb0"} : dispatch 2024-09-21T00:14:33.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:33 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e996ed0d-88ad-4aa7-891f-dfade4e09cb0"}]': finished 2024-09-21T00:14:33.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:33 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T00:14:33.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:33 smithi098 ceph-mon[34725]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T00:14:33.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:33 smithi098 ceph-mon[34725]: from='client.? 172.21.15.98:0/2243877550' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:14:33.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:33 smithi027 ceph-mon[30745]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T00:14:33.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:33 smithi027 ceph-mon[30745]: from='client.? 172.21.15.98:0/1275651620' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e996ed0d-88ad-4aa7-891f-dfade4e09cb0"} : dispatch 2024-09-21T00:14:33.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:33 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e996ed0d-88ad-4aa7-891f-dfade4e09cb0"} : dispatch 2024-09-21T00:14:33.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:33 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e996ed0d-88ad-4aa7-891f-dfade4e09cb0"}]': finished 2024-09-21T00:14:33.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:33 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T00:14:33.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:33 smithi027 ceph-mon[30745]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T00:14:33.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:33 smithi027 ceph-mon[30745]: from='client.? 172.21.15.98:0/2243877550' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:14:34.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:34 smithi089 ceph-mon[34481]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:34.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:34 smithi098 ceph-mon[34725]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:34.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:34 smithi027 ceph-mon[30745]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:36.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:36 smithi089 ceph-mon[34481]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:36.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:36 smithi098 ceph-mon[34725]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:36.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:36 smithi027 ceph-mon[30745]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:38.388 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:38 smithi098 ceph-mon[34725]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:38.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:38 smithi089 ceph-mon[34481]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:38.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:38 smithi027 ceph-mon[30745]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:40.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:40 smithi098 ceph-mon[34725]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:40.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:40 smithi089 ceph-mon[34481]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:40.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:40 smithi027 ceph-mon[30745]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:41.445 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:41 smithi098 ceph-mon[34725]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:41.445 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:41 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T00:14:41.445 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:41 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:41.445 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:41 smithi098 ceph-mon[34725]: Deploying daemon osd.5 on smithi098 2024-09-21T00:14:41.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:41 smithi089 ceph-mon[34481]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:41.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:41 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T00:14:41.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:41 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:41.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:41 smithi089 ceph-mon[34481]: Deploying daemon osd.5 on smithi098 2024-09-21T00:14:41.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:41 smithi027 ceph-mon[30745]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:41.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:41 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T00:14:41.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:41 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:41.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:41 smithi027 ceph-mon[30745]: Deploying daemon osd.5 on smithi098 2024-09-21T00:14:43.405 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:43 smithi098 ceph-mon[34725]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:43.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:43 smithi089 ceph-mon[34481]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:43.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:43 smithi027 ceph-mon[30745]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:45.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:45 smithi089 ceph-mon[34481]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:45.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:45 smithi098 ceph-mon[34725]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:45.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:45 smithi027 ceph-mon[30745]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:47.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:47 smithi098 ceph-mon[34725]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:47.664 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:47 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:14:47.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:47 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:47.665 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:47 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:47.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:47 smithi089 ceph-mon[34481]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:47.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:47 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:14:47.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:47 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:47.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:47 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:47.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:47 smithi027 ceph-mon[30745]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:47.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:47 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:14:47.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:47 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:47.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:47 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:49.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:49 smithi089 ceph-mon[34481]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:49.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:49 smithi098 ceph-mon[34725]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:49.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:49 smithi027 ceph-mon[30745]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:50.696 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:50 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:50.696 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:50 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:50.696 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:50 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:50.696 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:50 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:14:50.697 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:50 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:50.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:50 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:50.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:50 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:50.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:50 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:50.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:50 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:14:50.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:50 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:51.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:51.052+0000 7f2b0e7fc640 1 -- 172.21.15.98:0/1412744960 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2b1810bc10 con 0x7f2b040b6560 2024-09-21T00:14:51.054 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 5 on host 'smithi098' 2024-09-21T00:14:51.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:51.055+0000 7f2b1f535640 1 -- 172.21.15.98:0/1412744960 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2b040b6560 msgr2=0x7f2b040b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:14:51.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:51.055+0000 7f2b1f535640 1 --2- 172.21.15.98:0/1412744960 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2b040b6560 0x7f2b040b8a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b18077160 tx=0x7f2b14005f90 comp rx=0 tx=0).stop 2024-09-21T00:14:51.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:51.055+0000 7f2b1f535640 1 -- 172.21.15.98:0/1412744960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b181094a0 msgr2=0x7f2b18075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:14:51.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:51.055+0000 7f2b1f535640 1 --2- 172.21.15.98:0/1412744960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b181094a0 0x7f2b18075c50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2b08006010 tx=0x7f2b08004060 comp rx=0 tx=0).stop 2024-09-21T00:14:51.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:51.055+0000 7f2b1f535640 1 -- 172.21.15.98:0/1412744960 shutdown_connections 2024-09-21T00:14:51.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:51.055+0000 7f2b1f535640 1 --2- 172.21.15.98:0/1412744960 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2b040b6560 0x7f2b040b8a70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:51.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:51.055+0000 7f2b1f535640 1 --2- 172.21.15.98:0/1412744960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b18109df0 0x7f2b18076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:51.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:51.055+0000 7f2b1f535640 1 --2- 172.21.15.98:0/1412744960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b181094a0 0x7f2b18075c50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:51.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:51.055+0000 7f2b1f535640 1 --2- 172.21.15.98:0/1412744960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b18103b60 0x7f2b18075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:51.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:51.055+0000 7f2b1f535640 1 -- 172.21.15.98:0/1412744960 >> 172.21.15.98:0/1412744960 conn(0x7f2b1807b850 msgr2=0x7f2b18105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:14:51.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:51.055+0000 7f2b1f535640 1 -- 172.21.15.98:0/1412744960 shutdown_connections 2024-09-21T00:14:51.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:14:51.055+0000 7f2b1f535640 1 -- 172.21.15.98:0/1412744960 wait complete. 2024-09-21T00:14:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:50 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:50 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:50 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:51.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:50 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:14:51.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:50 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:51.688 DEBUG:teuthology.orchestra.run.smithi098:osd.5> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T00:14:51.691 INFO:tasks.cephadm:Deploying osd.6 on smithi027 with /dev/vg_nvme/lv_2... 2024-09-21T00:14:51.691 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T00:14:51.872 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:51 smithi098 ceph-mon[34725]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:51.872 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:51 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:14:51.872 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:51 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:51.872 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:51 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:51.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:51 smithi089 ceph-mon[34481]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:51.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:51 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:14:51.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:51 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:51.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:51 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:51 smithi027 ceph-mon[30745]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:51 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:14:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:51 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:51 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:52.082 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:14:53.496 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:14:53 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:14:53.285+0000 7f937db3b640 -1 osd.5 0 log_to_monitors true 2024-09-21T00:14:53.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:53 smithi027 ceph-mon[30745]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:53.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:53 smithi027 ceph-mon[30745]: from='osd.5 [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T00:14:53.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:53 smithi027 ceph-mon[30745]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T00:14:53.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:53 smithi089 ceph-mon[34481]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:53.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:53 smithi089 ceph-mon[34481]: from='osd.5 [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T00:14:53.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:53 smithi089 ceph-mon[34481]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T00:14:53.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:53 smithi098 ceph-mon[34725]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:53.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:53 smithi098 ceph-mon[34725]: from='osd.5 [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T00:14:53.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:53 smithi098 ceph-mon[34725]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T00:14:54.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:54 smithi089 ceph-mon[34481]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T00:14:54.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:54 smithi089 ceph-mon[34481]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T00:14:54.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:54 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T00:14:54.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:54 smithi089 ceph-mon[34481]: from='osd.5 [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:14:54.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:54 smithi089 ceph-mon[34481]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:14:54.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:54 smithi098 ceph-mon[34725]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T00:14:54.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:54 smithi098 ceph-mon[34725]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T00:14:54.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:54 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T00:14:54.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:54 smithi098 ceph-mon[34725]: from='osd.5 [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:14:54.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:54 smithi098 ceph-mon[34725]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:14:54.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:14:54 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:14:54.643+0000 7f9379abd640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T00:14:54.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:14:54 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:14:54.655+0000 7f93758bb640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:14:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:54 smithi027 ceph-mon[30745]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T00:14:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:54 smithi027 ceph-mon[30745]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T00:14:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:54 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T00:14:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:54 smithi027 ceph-mon[30745]: from='osd.5 [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:14:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:54 smithi027 ceph-mon[30745]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:14:55.620 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:14:55.658 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi027:vg_nvme/lv_2 2024-09-21T00:14:55.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:55.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T00:14:55.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T00:14:55.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T00:14:55.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: Detected new or changed devices on smithi098 2024-09-21T00:14:55.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:55.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:55.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:55.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:55.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:55.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:55.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: Adjusting osd_memory_target on smithi098 to 8544M 2024-09-21T00:14:55.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:55.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:55.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:14:55.980 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:55 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:55.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:55.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T00:14:55.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T00:14:55.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T00:14:55.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: Detected new or changed devices on smithi098 2024-09-21T00:14:55.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:55.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:55.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:55.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:55.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:55.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:55.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: Adjusting osd_memory_target on smithi098 to 8544M 2024-09-21T00:14:55.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:55.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:55.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:14:55.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:55 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:56.039 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:14:56.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:56.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T00:14:56.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T00:14:56.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T00:14:56.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: Detected new or changed devices on smithi098 2024-09-21T00:14:56.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:56.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:56.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:56.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:56.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:56.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:14:56.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: Adjusting osd_memory_target on smithi098 to 8544M 2024-09-21T00:14:56.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:56.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:56.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:14:56.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:55 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:14:56.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.662+0000 7fe039e00640 1 -- 172.21.15.27:0/2858434004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe034075750 msgr2=0x7fe034075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:14:56.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.662+0000 7fe039e00640 1 --2- 172.21.15.27:0/2858434004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe034075750 0x7fe034075b60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe024009900 tx=0x7fe02402a410 comp rx=0 tx=0).stop 2024-09-21T00:14:56.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.663+0000 7fe039e00640 1 -- 172.21.15.27:0/2858434004 shutdown_connections 2024-09-21T00:14:56.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.663+0000 7fe039e00640 1 --2- 172.21.15.27:0/2858434004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe034110120 0x7fe034112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:56.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.663+0000 7fe039e00640 1 --2- 172.21.15.27:0/2858434004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0340760a0 0x7fe03410fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:56.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.663+0000 7fe039e00640 1 --2- 172.21.15.27:0/2858434004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe034075750 0x7fe034075b60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:56.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.663+0000 7fe039e00640 1 -- 172.21.15.27:0/2858434004 >> 172.21.15.27:0/2858434004 conn(0x7fe03407b850 msgr2=0x7fe03407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:14:56.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.663+0000 7fe039e00640 1 -- 172.21.15.27:0/2858434004 shutdown_connections 2024-09-21T00:14:56.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.663+0000 7fe039e00640 1 -- 172.21.15.27:0/2858434004 wait complete. 2024-09-21T00:14:56.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.664+0000 7fe039e00640 1 Processor -- start 2024-09-21T00:14:56.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.664+0000 7fe039e00640 1 -- start start 2024-09-21T00:14:56.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.664+0000 7fe039e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe034075750 0x7fe03419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:14:56.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.664+0000 7fe039e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0340760a0 0x7fe03419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:14:56.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.664+0000 7fe039e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe034110120 0x7fe0341a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:14:56.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.664+0000 7fe039e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe034115c90 con 0x7fe034075750 2024-09-21T00:14:56.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.664+0000 7fe039e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe034115b10 con 0x7fe0340760a0 2024-09-21T00:14:56.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.664+0000 7fe039e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe034115e10 con 0x7fe034110120 2024-09-21T00:14:56.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.665+0000 7fe032a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe034075750 0x7fe03419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:14:56.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.665+0000 7fe032a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe034075750 0x7fe03419c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59650/0 (socket says 172.21.15.27:59650) 2024-09-21T00:14:56.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.665+0000 7fe032a00640 1 -- 172.21.15.27:0/3314077541 learned_addr learned my addr 172.21.15.27:0/3314077541 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:14:56.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.665+0000 7fe032000640 1 --2- 172.21.15.27:0/3314077541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0340760a0 0x7fe03419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:14:56.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.665+0000 7fe033400640 1 --2- 172.21.15.27:0/3314077541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe034110120 0x7fe0341a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:14:56.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.665+0000 7fe032a00640 1 -- 172.21.15.27:0/3314077541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe034110120 msgr2=0x7fe0341a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:14:56.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.665+0000 7fe032a00640 1 --2- 172.21.15.27:0/3314077541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe034110120 0x7fe0341a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:56.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.665+0000 7fe032a00640 1 -- 172.21.15.27:0/3314077541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0340760a0 msgr2=0x7fe03419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:14:56.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.665+0000 7fe032a00640 1 --2- 172.21.15.27:0/3314077541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0340760a0 0x7fe03419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:14:56.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.665+0000 7fe032a00640 1 -- 172.21.15.27:0/3314077541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0341a2670 con 0x7fe034075750 2024-09-21T00:14:56.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.665+0000 7fe032a00640 1 --2- 172.21.15.27:0/3314077541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe034075750 0x7fe03419c700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe024005940 tx=0x7fe0240040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:14:56.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.666+0000 7fe038a00640 1 -- 172.21.15.27:0/3314077541 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02403f030 con 0x7fe034075750 2024-09-21T00:14:56.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.666+0000 7fe038a00640 1 -- 172.21.15.27:0/3314077541 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0240044c0 con 0x7fe034075750 2024-09-21T00:14:56.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.666+0000 7fe039e00640 1 -- 172.21.15.27:0/3314077541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0341a2900 con 0x7fe034075750 2024-09-21T00:14:56.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.666+0000 7fe038a00640 1 -- 172.21.15.27:0/3314077541 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024035070 con 0x7fe034075750 2024-09-21T00:14:56.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.666+0000 7fe039e00640 1 -- 172.21.15.27:0/3314077541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0341a90c0 con 0x7fe034075750 2024-09-21T00:14:56.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.667+0000 7fe039e00640 1 -- 172.21.15.27:0/3314077541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe034075b60 con 0x7fe034075750 2024-09-21T00:14:56.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.669+0000 7fe038a00640 1 -- 172.21.15.27:0/3314077541 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fe024042030 con 0x7fe034075750 2024-09-21T00:14:56.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.671+0000 7fe038a00640 1 --2- 172.21.15.27:0/3314077541 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe0180b65e0 0x7fe0180b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:14:56.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.671+0000 7fe038a00640 1 -- 172.21.15.27:0/3314077541 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4748+0+0 (secure 0 0 0) 0x7fe0241065c0 con 0x7fe034075750 2024-09-21T00:14:56.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.671+0000 7fe032000640 1 --2- 172.21.15.27:0/3314077541 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe0180b65e0 0x7fe0180b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:14:56.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.672+0000 7fe032000640 1 --2- 172.21.15.27:0/3314077541 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe0180b65e0 0x7fe0180b8af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe034103530 tx=0x7fe020005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:14:56.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.675+0000 7fe038a00640 1 -- 172.21.15.27:0/3314077541 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe0240cdba0 con 0x7fe034075750 2024-09-21T00:14:56.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:56 smithi027 ceph-mon[30745]: purged_snaps scrub starts 2024-09-21T00:14:56.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:56 smithi027 ceph-mon[30745]: purged_snaps scrub ok 2024-09-21T00:14:56.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:56 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T00:14:56.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:56 smithi027 ceph-mon[30745]: osd.5 [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562] boot 2024-09-21T00:14:56.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:56 smithi027 ceph-mon[30745]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T00:14:56.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:56 smithi027 ceph-mon[30745]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T00:14:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:14:56.885+0000 7fe039e00640 1 -- 172.21.15.27:0/3314077541 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe034063800 con 0x7fe0180b65e0 2024-09-21T00:14:56.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:56 smithi089 ceph-mon[34481]: purged_snaps scrub starts 2024-09-21T00:14:56.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:56 smithi089 ceph-mon[34481]: purged_snaps scrub ok 2024-09-21T00:14:56.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:56 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T00:14:56.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:56 smithi089 ceph-mon[34481]: osd.5 [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562] boot 2024-09-21T00:14:56.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:56 smithi089 ceph-mon[34481]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T00:14:56.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:56 smithi089 ceph-mon[34481]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T00:14:56.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:56 smithi098 ceph-mon[34725]: purged_snaps scrub starts 2024-09-21T00:14:56.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:56 smithi098 ceph-mon[34725]: purged_snaps scrub ok 2024-09-21T00:14:56.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:56 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T00:14:56.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:56 smithi098 ceph-mon[34725]: osd.5 [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562] boot 2024-09-21T00:14:56.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:56 smithi098 ceph-mon[34725]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T00:14:56.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:56 smithi098 ceph-mon[34725]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T00:14:57.697 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:57 smithi027 ceph-mon[30745]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:57.697 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:57 smithi027 ceph-mon[30745]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:14:57.697 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:57 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:14:57.697 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:57 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:14:57.697 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:57 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:57.697 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:57 smithi027 ceph-mon[30745]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T00:14:57.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:57 smithi089 ceph-mon[34481]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:57.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:57 smithi089 ceph-mon[34481]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:14:57.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:57 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:14:57.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:57 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:14:57.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:57 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:57.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:57 smithi089 ceph-mon[34481]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T00:14:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:57 smithi098 ceph-mon[34725]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T00:14:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:57 smithi098 ceph-mon[34725]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:14:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:57 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:14:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:57 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:14:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:57 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:14:57.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:57 smithi098 ceph-mon[34725]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T00:14:59.790 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:14:59 smithi027 ceph-mon[30745]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:14:59.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:14:59 smithi089 ceph-mon[34481]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:14:59.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:14:59 smithi098 ceph-mon[34725]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:00.918 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:00 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/16251214' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cf080497-57c2-42d0-8c8f-1eed0110702d"} : dispatch 2024-09-21T00:15:00.918 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:00 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cf080497-57c2-42d0-8c8f-1eed0110702d"} : dispatch 2024-09-21T00:15:00.918 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:00 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf080497-57c2-42d0-8c8f-1eed0110702d"}]': finished 2024-09-21T00:15:00.918 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:00 smithi098 ceph-mon[34725]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T00:15:00.918 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:00 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:00.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:00 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/16251214' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cf080497-57c2-42d0-8c8f-1eed0110702d"} : dispatch 2024-09-21T00:15:00.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:00 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cf080497-57c2-42d0-8c8f-1eed0110702d"} : dispatch 2024-09-21T00:15:00.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:00 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf080497-57c2-42d0-8c8f-1eed0110702d"}]': finished 2024-09-21T00:15:00.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:00 smithi089 ceph-mon[34481]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T00:15:00.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:00 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:01.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:00 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/16251214' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cf080497-57c2-42d0-8c8f-1eed0110702d"} : dispatch 2024-09-21T00:15:01.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:00 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cf080497-57c2-42d0-8c8f-1eed0110702d"} : dispatch 2024-09-21T00:15:01.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:00 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf080497-57c2-42d0-8c8f-1eed0110702d"}]': finished 2024-09-21T00:15:01.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:00 smithi027 ceph-mon[30745]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T00:15:01.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:00 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:01.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:01 smithi027 ceph-mon[30745]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-09-21T00:15:01.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:01 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/641743421' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:15:01.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:01 smithi089 ceph-mon[34481]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-09-21T00:15:01.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:01 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/641743421' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:15:01.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:01 smithi098 ceph-mon[34725]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-09-21T00:15:01.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:01 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/641743421' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:15:03.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:03 smithi089 ceph-mon[34481]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-21T00:15:03.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:03 smithi098 ceph-mon[34725]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-21T00:15:04.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:03 smithi027 ceph-mon[30745]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-21T00:15:05.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:05 smithi089 ceph-mon[34481]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T00:15:05.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:05 smithi098 ceph-mon[34725]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T00:15:06.018 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:05 smithi027 ceph-mon[30745]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T00:15:07.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:07 smithi089 ceph-mon[34481]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T00:15:07.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:07 smithi098 ceph-mon[34725]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T00:15:08.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:07 smithi027 ceph-mon[30745]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T00:15:09.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:09 smithi027 ceph-mon[30745]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T00:15:09.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:09 smithi089 ceph-mon[34481]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T00:15:09.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:09 smithi098 ceph-mon[34725]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T00:15:10.779 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:10 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T00:15:10.780 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:10 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:10.780 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:10 smithi027 ceph-mon[30745]: Deploying daemon osd.6 on smithi027 2024-09-21T00:15:10.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:10 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T00:15:10.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:10 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:10.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:10 smithi089 ceph-mon[34481]: Deploying daemon osd.6 on smithi027 2024-09-21T00:15:10.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:10 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T00:15:10.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:10 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:10.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:10 smithi098 ceph-mon[34725]: Deploying daemon osd.6 on smithi027 2024-09-21T00:15:11.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:11 smithi089 ceph-mon[34481]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:11.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:11 smithi098 ceph-mon[34725]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:12.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:11 smithi027 ceph-mon[30745]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:13.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:13 smithi089 ceph-mon[34481]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:13.988 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:13 smithi027 ceph-mon[30745]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:13.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:13 smithi098 ceph-mon[34725]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:15.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:15 smithi089 ceph-mon[34481]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:15.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:15 smithi098 ceph-mon[34725]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:16.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:15 smithi027 ceph-mon[30745]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:17.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:17 smithi027 ceph-mon[30745]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:17.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:17 smithi089 ceph-mon[34481]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:17.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:17 smithi098 ceph-mon[34725]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:19.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:19 smithi089 ceph-mon[34481]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:19.982 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:19 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:15:19.982 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:19 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:19.982 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:19 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:19.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:19 smithi098 ceph-mon[34725]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:19.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:19 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:15:19.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:19 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:19.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:19 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:20.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:19 smithi027 ceph-mon[30745]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:20.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:19 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:15:20.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:19 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:20.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:19 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:21.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:21 smithi089 ceph-mon[34481]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:21.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:21 smithi098 ceph-mon[34725]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:22.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:21 smithi027 ceph-mon[30745]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:23.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:22 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:23.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:22 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:23.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:22 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:23.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:22 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:15:23.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:22 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:23.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:22 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:23.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:22 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:23.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:22 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:23.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:22 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:15:23.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:22 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:23.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:22 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:23.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:22 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:23.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:22 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:23.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:22 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:15:23.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:22 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:23.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:15:23.990+0000 7fe038a00640 1 -- 172.21.15.27:0/3314077541 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe034063800 con 0x7fe0180b65e0 2024-09-21T00:15:23.992 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 6 on host 'smithi027' 2024-09-21T00:15:23.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:15:23.993+0000 7fe039e00640 1 -- 172.21.15.27:0/3314077541 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe0180b65e0 msgr2=0x7fe0180b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:15:23.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:15:23.993+0000 7fe039e00640 1 --2- 172.21.15.27:0/3314077541 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe0180b65e0 0x7fe0180b8af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe034103530 tx=0x7fe020005f90 comp rx=0 tx=0).stop 2024-09-21T00:15:23.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:15:23.993+0000 7fe039e00640 1 -- 172.21.15.27:0/3314077541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe034075750 msgr2=0x7fe03419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:15:23.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:15:23.993+0000 7fe039e00640 1 --2- 172.21.15.27:0/3314077541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe034075750 0x7fe03419c700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe024005940 tx=0x7fe0240040c0 comp rx=0 tx=0).stop 2024-09-21T00:15:23.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:15:23.993+0000 7fe039e00640 1 -- 172.21.15.27:0/3314077541 shutdown_connections 2024-09-21T00:15:23.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:15:23.993+0000 7fe039e00640 1 --2- 172.21.15.27:0/3314077541 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe0180b65e0 0x7fe0180b8af0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:23.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:15:23.993+0000 7fe039e00640 1 --2- 172.21.15.27:0/3314077541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe034110120 0x7fe0341a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:23.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:15:23.993+0000 7fe039e00640 1 --2- 172.21.15.27:0/3314077541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0340760a0 0x7fe03419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:23.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:15:23.993+0000 7fe039e00640 1 --2- 172.21.15.27:0/3314077541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe034075750 0x7fe03419c700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:23.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:15:23.993+0000 7fe039e00640 1 -- 172.21.15.27:0/3314077541 >> 172.21.15.27:0/3314077541 conn(0x7fe03407b850 msgr2=0x7fe034101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:15:23.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:15:23.993+0000 7fe039e00640 1 -- 172.21.15.27:0/3314077541 shutdown_connections 2024-09-21T00:15:23.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:15:23.993+0000 7fe039e00640 1 -- 172.21.15.27:0/3314077541 wait complete. 2024-09-21T00:15:24.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:24 smithi027 ceph-mon[30745]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:24.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:15:24.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:24.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:24.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:24 smithi089 ceph-mon[34481]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:24.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:15:24.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:24.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:24.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:24 smithi098 ceph-mon[34725]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:24.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:15:24.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:24.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:24.879 DEBUG:teuthology.orchestra.run.smithi027:osd.6> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T00:15:24.884 INFO:tasks.cephadm:Deploying osd.7 on smithi089 with /dev/vg_nvme/lv_2... 2024-09-21T00:15:24.885 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T00:15:25.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:25 smithi027 ceph-mon[30745]: from='osd.6 [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T00:15:25.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:25 smithi027 ceph-mon[30745]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T00:15:25.140 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T00:15:25.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:25 smithi089 ceph-mon[34481]: from='osd.6 [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T00:15:25.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:25 smithi089 ceph-mon[34481]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T00:15:25.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:25 smithi098 ceph-mon[34725]: from='osd.6 [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T00:15:25.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:25 smithi098 ceph-mon[34725]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T00:15:26.279 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:26 smithi089 ceph-mon[34481]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:26.280 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:26 smithi089 ceph-mon[34481]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T00:15:26.280 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:26 smithi089 ceph-mon[34481]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T00:15:26.280 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:26 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:26.280 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:26 smithi089 ceph-mon[34481]: from='osd.6 [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:15:26.280 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:26 smithi089 ceph-mon[34481]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:15:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:26 smithi027 ceph-mon[30745]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:26 smithi027 ceph-mon[30745]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T00:15:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:26 smithi027 ceph-mon[30745]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T00:15:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:26 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:26 smithi027 ceph-mon[30745]: from='osd.6 [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:15:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:26 smithi027 ceph-mon[30745]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:15:26.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:26 smithi098 ceph-mon[34725]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:26.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:26 smithi098 ceph-mon[34725]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T00:15:26.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:26 smithi098 ceph-mon[34725]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T00:15:26.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:26 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:26.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:26 smithi098 ceph-mon[34725]: from='osd.6 [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:15:26.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:26 smithi098 ceph-mon[34725]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:15:26.976 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:15:26 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:15:26.864+0000 7fb7ce200640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T00:15:26.976 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:15:26 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:15:26.887+0000 7fb7c3800640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:15:27.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:27 smithi089 ceph-mon[34481]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T00:15:27.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:27 smithi089 ceph-mon[34481]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T00:15:27.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:27 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:27.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:27 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:27.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:27 smithi089 ceph-mon[34481]: from='osd.6 ' entity='osd.6' 2024-09-21T00:15:27.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:27 smithi027 ceph-mon[30745]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T00:15:27.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:27 smithi027 ceph-mon[30745]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T00:15:27.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:27 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:27.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:27 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:27.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:27 smithi027 ceph-mon[30745]: from='osd.6 ' entity='osd.6' 2024-09-21T00:15:27.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:27 smithi098 ceph-mon[34725]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T00:15:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:27 smithi098 ceph-mon[34725]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T00:15:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:27 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:27 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:27 smithi098 ceph-mon[34725]: from='osd.6 ' entity='osd.6' 2024-09-21T00:15:27.585 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:15:27.612 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi089:vg_nvme/lv_2 2024-09-21T00:15:27.868 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T00:15:28.149 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:28 smithi089 ceph-mon[34481]: purged_snaps scrub starts 2024-09-21T00:15:28.149 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:28 smithi089 ceph-mon[34481]: purged_snaps scrub ok 2024-09-21T00:15:28.149 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:28 smithi089 ceph-mon[34481]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:28.149 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:28 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:28.149 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:28 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:28.149 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:28 smithi089 ceph-mon[34481]: osd.6 [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857] boot 2024-09-21T00:15:28.149 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:28 smithi089 ceph-mon[34481]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T00:15:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:28 smithi027 ceph-mon[30745]: purged_snaps scrub starts 2024-09-21T00:15:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:28 smithi027 ceph-mon[30745]: purged_snaps scrub ok 2024-09-21T00:15:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:28 smithi027 ceph-mon[30745]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:28 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:28 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:28 smithi027 ceph-mon[30745]: osd.6 [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857] boot 2024-09-21T00:15:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:28 smithi027 ceph-mon[30745]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T00:15:28.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.386+0000 7f1727e5a640 1 -- 172.21.15.89:0/3559079912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1720102a60 msgr2=0x7f1720102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:15:28.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.386+0000 7f1727e5a640 1 --2- 172.21.15.89:0/3559079912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1720102a60 0x7f1720102e70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1714009900 tx=0x7f171402a410 comp rx=0 tx=0).stop 2024-09-21T00:15:28.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.386+0000 7f1727e5a640 1 -- 172.21.15.89:0/3559079912 shutdown_connections 2024-09-21T00:15:28.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.386+0000 7f1727e5a640 1 --2- 172.21.15.89:0/3559079912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1720108cf0 0x7f172010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.386+0000 7f1727e5a640 1 --2- 172.21.15.89:0/3559079912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17201083a0 0x7f17201087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.386+0000 7f1727e5a640 1 --2- 172.21.15.89:0/3559079912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1720102a60 0x7f1720102e70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.386+0000 7f1727e5a640 1 -- 172.21.15.89:0/3559079912 >> 172.21.15.89:0/3559079912 conn(0x7f1720079730 msgr2=0x7f1720079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:15:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.387+0000 7f1727e5a640 1 -- 172.21.15.89:0/3559079912 shutdown_connections 2024-09-21T00:15:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.387+0000 7f1727e5a640 1 -- 172.21.15.89:0/3559079912 wait complete. 2024-09-21T00:15:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.387+0000 7f1727e5a640 1 Processor -- start 2024-09-21T00:15:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.387+0000 7f1727e5a640 1 -- start start 2024-09-21T00:15:28.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.387+0000 7f1727e5a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1720102a60 0x7f172019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:15:28.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.387+0000 7f1727e5a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17201083a0 0x7f172019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:15:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1727e5a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1720108cf0 0x7f172019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:15:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1727e5a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f17201129c0 con 0x7f17201083a0 2024-09-21T00:15:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1727e5a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1720112840 con 0x7f1720102a60 2024-09-21T00:15:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1727e5a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1720112b40 con 0x7f1720108cf0 2024-09-21T00:15:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1725bf6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1720102a60 0x7f172019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:15:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1725bf6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1720102a60 0x7f172019b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40988/0 (socket says 172.21.15.89:40988) 2024-09-21T00:15:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1725bf6640 1 -- 172.21.15.89:0/3344419391 learned_addr learned my addr 172.21.15.89:0/3344419391 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T00:15:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f17253f5640 1 --2- 172.21.15.89:0/3344419391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17201083a0 0x7f172019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:15:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f17263f7640 1 --2- 172.21.15.89:0/3344419391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1720108cf0 0x7f172019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:15:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1725bf6640 1 -- 172.21.15.89:0/3344419391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1720108cf0 msgr2=0x7f172019bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:15:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1725bf6640 1 --2- 172.21.15.89:0/3344419391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1720108cf0 0x7f172019bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1725bf6640 1 -- 172.21.15.89:0/3344419391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17201083a0 msgr2=0x7f172019b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:15:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1725bf6640 1 --2- 172.21.15.89:0/3344419391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17201083a0 0x7f172019b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1725bf6640 1 -- 172.21.15.89:0/3344419391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17201a65c0 con 0x7f1720102a60 2024-09-21T00:15:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1725bf6640 1 --2- 172.21.15.89:0/3344419391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1720102a60 0x7f172019b330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1714006010 tx=0x7f17140042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:15:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1712ffd640 1 -- 172.21.15.89:0/3344419391 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171403f030 con 0x7f1720102a60 2024-09-21T00:15:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1712ffd640 1 -- 172.21.15.89:0/3344419391 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1714003800 con 0x7f1720102a60 2024-09-21T00:15:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1727e5a640 1 -- 172.21.15.89:0/3344419391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17201a6850 con 0x7f1720102a60 2024-09-21T00:15:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1712ffd640 1 -- 172.21.15.89:0/3344419391 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714035070 con 0x7f1720102a60 2024-09-21T00:15:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.388+0000 7f1727e5a640 1 -- 172.21.15.89:0/3344419391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17201a6d80 con 0x7f1720102a60 2024-09-21T00:15:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.389+0000 7f1727e5a640 1 -- 172.21.15.89:0/3344419391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1720102e70 con 0x7f1720102a60 2024-09-21T00:15:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.391+0000 7f1712ffd640 1 -- 172.21.15.89:0/3344419391 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f1714003cb0 con 0x7f1720102a60 2024-09-21T00:15:28.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.392+0000 7f1712ffd640 1 --2- 172.21.15.89:0/3344419391 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f170c0b6610 0x7f170c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:15:28.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.392+0000 7f1712ffd640 1 -- 172.21.15.89:0/3344419391 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5180+0+0 (secure 0 0 0) 0x7f1714107240 con 0x7f1720102a60 2024-09-21T00:15:28.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.392+0000 7f17253f5640 1 --2- 172.21.15.89:0/3344419391 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f170c0b6610 0x7f170c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:15:28.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.393+0000 7f17253f5640 1 --2- 172.21.15.89:0/3344419391 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f170c0b6610 0x7f170c0b8b20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1720100b70 tx=0x7f171c00a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:15:28.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.400+0000 7f1712ffd640 1 -- 172.21.15.89:0/3344419391 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f17140ce710 con 0x7f1720102a60 2024-09-21T00:15:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:28 smithi098 ceph-mon[34725]: purged_snaps scrub starts 2024-09-21T00:15:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:28 smithi098 ceph-mon[34725]: purged_snaps scrub ok 2024-09-21T00:15:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:28 smithi098 ceph-mon[34725]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T00:15:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:28 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:28 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T00:15:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:28 smithi098 ceph-mon[34725]: osd.6 [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857] boot 2024-09-21T00:15:28.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:28 smithi098 ceph-mon[34725]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T00:15:28.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:28.594+0000 7f1727e5a640 1 -- 172.21.15.89:0/3344419391 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1720063800 con 0x7f170c0b6610 2024-09-21T00:15:29.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:29 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:15:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:29 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:15:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:29 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:29 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:29 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:29 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:29 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:29 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:15:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:29 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:15:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:29 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:15:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:29 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:29 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:29 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:29 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:29.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:29 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:29.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:29 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:15:29.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:29.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:29 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:15:29.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:29 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:15:29.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:29 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:29.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:29.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:29.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:29 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:29 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:29 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:29.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:29.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:29 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:29.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:29 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:15:29.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:30.313 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:30 smithi089 ceph-mon[34481]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:30.314 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:30 smithi089 ceph-mon[34481]: from='client.24512 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:15:30.314 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:30 smithi089 ceph-mon[34481]: Detected new or changed devices on smithi027 2024-09-21T00:15:30.314 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:30 smithi089 ceph-mon[34481]: Adjusting osd_memory_target on smithi027 to 5696M 2024-09-21T00:15:30.314 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:30 smithi089 ceph-mon[34481]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T00:15:30.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:30 smithi027 ceph-mon[30745]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:30.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:30 smithi027 ceph-mon[30745]: from='client.24512 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:15:30.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:30 smithi027 ceph-mon[30745]: Detected new or changed devices on smithi027 2024-09-21T00:15:30.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:30 smithi027 ceph-mon[30745]: Adjusting osd_memory_target on smithi027 to 5696M 2024-09-21T00:15:30.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:30 smithi027 ceph-mon[30745]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T00:15:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:30 smithi098 ceph-mon[34725]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:30 smithi098 ceph-mon[34725]: from='client.24512 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:15:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:30 smithi098 ceph-mon[34725]: Detected new or changed devices on smithi027 2024-09-21T00:15:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:30 smithi098 ceph-mon[34725]: Adjusting osd_memory_target on smithi027 to 5696M 2024-09-21T00:15:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:30 smithi098 ceph-mon[34725]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T00:15:32.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:32 smithi027 ceph-mon[30745]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:32.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:32 smithi027 ceph-mon[30745]: from='client.? 172.21.15.89:0/4016091089' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6"} : dispatch 2024-09-21T00:15:32.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:32 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6"} : dispatch 2024-09-21T00:15:32.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:32 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6"}]': finished 2024-09-21T00:15:32.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:32 smithi027 ceph-mon[30745]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T00:15:32.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:32 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T00:15:32.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:32 smithi027 ceph-mon[30745]: from='client.? 172.21.15.89:0/3695367895' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:15:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:32 smithi089 ceph-mon[34481]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:32 smithi089 ceph-mon[34481]: from='client.? 172.21.15.89:0/4016091089' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6"} : dispatch 2024-09-21T00:15:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:32 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6"} : dispatch 2024-09-21T00:15:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:32 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6"}]': finished 2024-09-21T00:15:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:32 smithi089 ceph-mon[34481]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T00:15:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:32 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T00:15:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:32 smithi089 ceph-mon[34481]: from='client.? 172.21.15.89:0/3695367895' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:15:32.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:32 smithi098 ceph-mon[34725]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:32 smithi098 ceph-mon[34725]: from='client.? 172.21.15.89:0/4016091089' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6"} : dispatch 2024-09-21T00:15:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:32 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6"} : dispatch 2024-09-21T00:15:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:32 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6"}]': finished 2024-09-21T00:15:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:32 smithi098 ceph-mon[34725]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T00:15:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:32 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T00:15:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:32 smithi098 ceph-mon[34725]: from='client.? 172.21.15.89:0/3695367895' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:15:34.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:34 smithi027 ceph-mon[30745]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:34.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:34 smithi089 ceph-mon[34481]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:34.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:34 smithi098 ceph-mon[34725]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:36.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:36 smithi089 ceph-mon[34481]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:36.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:36 smithi027 ceph-mon[30745]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:36.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:36 smithi098 ceph-mon[34725]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:38.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:38 smithi027 ceph-mon[30745]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:38.369 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:38 smithi089 ceph-mon[34481]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:38.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:38 smithi098 ceph-mon[34725]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:40.312 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:40 smithi089 ceph-mon[34481]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:40.313 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:40 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T00:15:40.313 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:40 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:40.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:40 smithi027 ceph-mon[30745]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:40.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:40 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T00:15:40.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:40 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:40.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:40 smithi098 ceph-mon[34725]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:40.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:40 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T00:15:40.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:40 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:41.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:41 smithi027 ceph-mon[30745]: Deploying daemon osd.7 on smithi089 2024-09-21T00:15:41.367 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:41 smithi089 ceph-mon[34481]: Deploying daemon osd.7 on smithi089 2024-09-21T00:15:41.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:41 smithi098 ceph-mon[34725]: Deploying daemon osd.7 on smithi089 2024-09-21T00:15:42.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:42 smithi027 ceph-mon[30745]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:42.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:42 smithi089 ceph-mon[34481]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:42.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:42 smithi098 ceph-mon[34725]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:44 smithi089 ceph-mon[34481]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:44.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:44 smithi098 ceph-mon[34725]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:44.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:44 smithi027 ceph-mon[30745]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:46.405 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:46 smithi089 ceph-mon[34481]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:46.405 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:46 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:15:46.406 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:46 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:46.406 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:46 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:46.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:46 smithi098 ceph-mon[34725]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:46.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:46 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:15:46.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:46 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:46.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:46 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:46.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:46 smithi027 ceph-mon[30745]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:46.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:46 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:15:46.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:46 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:46.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:46 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:48.438 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:48 smithi089 ceph-mon[34481]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:48.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:48 smithi098 ceph-mon[34725]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:48.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:48 smithi027 ceph-mon[30745]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:49.772 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 7 on host 'smithi089' 2024-09-21T00:15:49.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:49.771+0000 7f1712ffd640 1 -- 172.21.15.89:0/3344419391 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1720063800 con 0x7f170c0b6610 2024-09-21T00:15:49.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:49.773+0000 7f1727e5a640 1 -- 172.21.15.89:0/3344419391 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f170c0b6610 msgr2=0x7f170c0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:15:49.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:49.773+0000 7f1727e5a640 1 --2- 172.21.15.89:0/3344419391 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f170c0b6610 0x7f170c0b8b20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1720100b70 tx=0x7f171c00a2d0 comp rx=0 tx=0).stop 2024-09-21T00:15:49.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:49.773+0000 7f1727e5a640 1 -- 172.21.15.89:0/3344419391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1720102a60 msgr2=0x7f172019b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:15:49.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:49.773+0000 7f1727e5a640 1 --2- 172.21.15.89:0/3344419391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1720102a60 0x7f172019b330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1714006010 tx=0x7f17140042f0 comp rx=0 tx=0).stop 2024-09-21T00:15:49.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:49.773+0000 7f1727e5a640 1 -- 172.21.15.89:0/3344419391 shutdown_connections 2024-09-21T00:15:49.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:49.773+0000 7f1727e5a640 1 --2- 172.21.15.89:0/3344419391 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f170c0b6610 0x7f170c0b8b20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:49.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:49.773+0000 7f1727e5a640 1 --2- 172.21.15.89:0/3344419391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1720108cf0 0x7f172019bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:49.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:49.773+0000 7f1727e5a640 1 --2- 172.21.15.89:0/3344419391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17201083a0 0x7f172019b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:49.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:49.773+0000 7f1727e5a640 1 --2- 172.21.15.89:0/3344419391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1720102a60 0x7f172019b330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:49.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:49.773+0000 7f1727e5a640 1 -- 172.21.15.89:0/3344419391 >> 172.21.15.89:0/3344419391 conn(0x7f1720079730 msgr2=0x7f1720101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:15:49.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:49.774+0000 7f1727e5a640 1 -- 172.21.15.89:0/3344419391 shutdown_connections 2024-09-21T00:15:49.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:15:49.774+0000 7f1727e5a640 1 -- 172.21.15.89:0/3344419391 wait complete. 2024-09-21T00:15:49.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:49 smithi089 ceph-mon[34481]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:49.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:49 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:49.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:49 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:49.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:49 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:49.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:49 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:15:49.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:49 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:49.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:49 smithi098 ceph-mon[34725]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:49.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:49 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:49.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:49 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:49.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:49 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:49.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:49 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:15:49.997 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:49 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:50.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:49 smithi027 ceph-mon[30745]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:50.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:49 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:50.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:49 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:50.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:49 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:50.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:49 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:15:50.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:49 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:50.515 DEBUG:teuthology.orchestra.run.smithi089:osd.7> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T00:15:50.518 INFO:tasks.cephadm:Deploying osd.8 on smithi098 with /dev/vg_nvme/lv_2... 2024-09-21T00:15:50.518 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T00:15:50.772 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:15:50.824 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:50 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:15:50.824 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:50 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:50.824 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:50 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:50.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:50 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:15:50.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:50 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:50.892 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:50 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:50 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:15:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:50 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:50 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:51.855 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:51 smithi089 ceph-mon[34481]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:51.856 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:51 smithi089 ceph-mon[34481]: from='osd.7 [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T00:15:51.856 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:51 smithi089 ceph-mon[34481]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T00:15:51.856 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:15:51 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:15:51.571+0000 7ff9ff1aa640 -1 osd.7 0 log_to_monitors true 2024-09-21T00:15:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:51 smithi027 ceph-mon[30745]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:51 smithi027 ceph-mon[30745]: from='osd.7 [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T00:15:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:51 smithi027 ceph-mon[30745]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T00:15:52.195 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:51 smithi098 ceph-mon[34725]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:52.195 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:51 smithi098 ceph-mon[34725]: from='osd.7 [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T00:15:52.195 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:51 smithi098 ceph-mon[34725]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T00:15:52.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:52 smithi089 ceph-mon[34481]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T00:15:52.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:52 smithi089 ceph-mon[34481]: from='osd.7 [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:15:52.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:52 smithi089 ceph-mon[34481]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T00:15:52.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:52 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T00:15:52.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:52 smithi089 ceph-mon[34481]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:15:52.979 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:15:52 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:15:52.783+0000 7ff9fb12c640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T00:15:52.979 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:15:52 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:15:52.793+0000 7ff9f6f2a640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:15:53.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:52 smithi027 ceph-mon[30745]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T00:15:53.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:52 smithi027 ceph-mon[30745]: from='osd.7 [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:15:53.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:52 smithi027 ceph-mon[30745]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T00:15:53.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:52 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T00:15:53.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:52 smithi027 ceph-mon[30745]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:15:53.067 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:52 smithi098 ceph-mon[34725]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T00:15:53.068 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:52 smithi098 ceph-mon[34725]: from='osd.7 [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:15:53.068 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:52 smithi098 ceph-mon[34725]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T00:15:53.068 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:52 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T00:15:53.068 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:52 smithi098 ceph-mon[34725]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:15:53.804 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:15:53.832 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi098:vg_nvme/lv_2 2024-09-21T00:15:54.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:54.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T00:15:54.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T00:15:54.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T00:15:54.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T00:15:54.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:54.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:54.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:54.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:54.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:15:54.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:53 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:54.079 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:15:54.200 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:54.200 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T00:15:54.200 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T00:15:54.200 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T00:15:54.200 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T00:15:54.201 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:54.201 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:54.201 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.201 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.201 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.201 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.201 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.201 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.201 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:54.201 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:54.201 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:15:54.201 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:53 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:54.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T00:15:54.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T00:15:54.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T00:15:54.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T00:15:54.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T00:15:54.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:54.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:54.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:15:54.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:54.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:54.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:15:54.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:53 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:15:54.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.555+0000 7f4ded6f7640 1 -- 172.21.15.98:0/2823238797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de8108cf0 msgr2=0x7f4de810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:15:54.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.555+0000 7f4ded6f7640 1 --2- 172.21.15.98:0/2823238797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de8108cf0 0x7f4de810f520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009ca0 tx=0x7f4de002a8c0 comp rx=0 tx=0).stop 2024-09-21T00:15:54.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.555+0000 7f4ded6f7640 1 -- 172.21.15.98:0/2823238797 shutdown_connections 2024-09-21T00:15:54.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.555+0000 7f4ded6f7640 1 --2- 172.21.15.98:0/2823238797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de8108cf0 0x7f4de810f520 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:54.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.555+0000 7f4ded6f7640 1 --2- 172.21.15.98:0/2823238797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de81083a0 0x7f4de81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:54.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.555+0000 7f4ded6f7640 1 --2- 172.21.15.98:0/2823238797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4de8102a60 0x7f4de8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:54.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.555+0000 7f4ded6f7640 1 -- 172.21.15.98:0/2823238797 >> 172.21.15.98:0/2823238797 conn(0x7f4de8079730 msgr2=0x7f4de8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:15:54.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.555+0000 7f4ded6f7640 1 -- 172.21.15.98:0/2823238797 shutdown_connections 2024-09-21T00:15:54.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.555+0000 7f4ded6f7640 1 -- 172.21.15.98:0/2823238797 wait complete. 2024-09-21T00:15:54.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.556+0000 7f4ded6f7640 1 Processor -- start 2024-09-21T00:15:54.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.556+0000 7f4ded6f7640 1 -- start start 2024-09-21T00:15:54.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.556+0000 7f4ded6f7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de8102a60 0x7f4de819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:15:54.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.556+0000 7f4ded6f7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de81083a0 0x7f4de819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:15:54.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.556+0000 7f4ded6f7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4de8108cf0 0x7f4de819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:15:54.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.556+0000 7f4ded6f7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8112a40 con 0x7f4de81083a0 2024-09-21T00:15:54.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.556+0000 7f4ded6f7640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81128c0 con 0x7f4de8108cf0 2024-09-21T00:15:54.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.556+0000 7f4ded6f7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8112bc0 con 0x7f4de8102a60 2024-09-21T00:15:54.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.556+0000 7f4de6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de8102a60 0x7f4de819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:15:54.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.556+0000 7f4de6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de8102a60 0x7f4de819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48058/0 (socket says 172.21.15.98:48058) 2024-09-21T00:15:54.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.556+0000 7f4de6ffd640 1 -- 172.21.15.98:0/707579472 learned_addr learned my addr 172.21.15.98:0/707579472 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-21T00:15:54.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.557+0000 7f4de77fe640 1 --2- 172.21.15.98:0/707579472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4de8108cf0 0x7f4de819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:15:54.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.557+0000 7f4de6ffd640 1 -- 172.21.15.98:0/707579472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4de8108cf0 msgr2=0x7f4de819fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:15:54.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.557+0000 7f4de67fc640 1 --2- 172.21.15.98:0/707579472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de81083a0 0x7f4de819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:15:54.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.557+0000 7f4de6ffd640 1 --2- 172.21.15.98:0/707579472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4de8108cf0 0x7f4de819fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:54.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.557+0000 7f4de6ffd640 1 -- 172.21.15.98:0/707579472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de81083a0 msgr2=0x7f4de819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:15:54.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.557+0000 7f4de6ffd640 1 --2- 172.21.15.98:0/707579472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de81083a0 0x7f4de819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:15:54.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.557+0000 7f4de6ffd640 1 -- 172.21.15.98:0/707579472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de81a0570 con 0x7f4de8102a60 2024-09-21T00:15:54.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.557+0000 7f4de67fc640 1 --2- 172.21.15.98:0/707579472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de81083a0 0x7f4de819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:15:54.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.557+0000 7f4de6ffd640 1 --2- 172.21.15.98:0/707579472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de8102a60 0x7f4de819a600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800b4a0 tx=0x7f4dd800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:15:54.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.557+0000 7f4dcffff640 1 -- 172.21.15.98:0/707579472 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8014070 con 0x7f4de8102a60 2024-09-21T00:15:54.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.557+0000 7f4ded6f7640 1 -- 172.21.15.98:0/707579472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de81a0860 con 0x7f4de8102a60 2024-09-21T00:15:54.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.557+0000 7f4dcffff640 1 -- 172.21.15.98:0/707579472 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd8010ac0 con 0x7f4de8102a60 2024-09-21T00:15:54.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.557+0000 7f4dcffff640 1 -- 172.21.15.98:0/707579472 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8017c30 con 0x7f4de8102a60 2024-09-21T00:15:54.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.557+0000 7f4ded6f7640 1 -- 172.21.15.98:0/707579472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de81a6fc0 con 0x7f4de8102a60 2024-09-21T00:15:54.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.558+0000 7f4ded6f7640 1 -- 172.21.15.98:0/707579472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de8102e70 con 0x7f4de8102a60 2024-09-21T00:15:54.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.560+0000 7f4dcffff640 1 -- 172.21.15.98:0/707579472 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f4dd8017070 con 0x7f4de8102a60 2024-09-21T00:15:54.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.561+0000 7f4dcffff640 1 --2- 172.21.15.98:0/707579472 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4dd00b6560 0x7f4dd00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:15:54.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.561+0000 7f4dcffff640 1 -- 172.21.15.98:0/707579472 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5645+0+0 (secure 0 0 0) 0x7f4dd80e6040 con 0x7f4de8102a60 2024-09-21T00:15:54.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.561+0000 7f4de67fc640 1 --2- 172.21.15.98:0/707579472 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4dd00b6560 0x7f4dd00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:15:54.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.562+0000 7f4de67fc640 1 --2- 172.21.15.98:0/707579472 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4dd00b6560 0x7f4dd00b8a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc006fd0 tx=0x7f4ddc008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:15:54.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.564+0000 7f4dcffff640 1 -- 172.21.15.98:0/707579472 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4dd80ad340 con 0x7f4de8102a60 2024-09-21T00:15:54.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:15:54.749+0000 7f4ded6f7640 1 -- 172.21.15.98:0/707579472 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4de8101c70 con 0x7f4dd00b6560 2024-09-21T00:15:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:54 smithi027 ceph-mon[30745]: purged_snaps scrub starts 2024-09-21T00:15:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:54 smithi027 ceph-mon[30745]: purged_snaps scrub ok 2024-09-21T00:15:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:54 smithi027 ceph-mon[30745]: Detected new or changed devices on smithi089 2024-09-21T00:15:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:54 smithi027 ceph-mon[30745]: Adjusting osd_memory_target on smithi089 to 5695M 2024-09-21T00:15:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:54 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T00:15:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:54 smithi027 ceph-mon[30745]: osd.7 [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546] boot 2024-09-21T00:15:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:54 smithi027 ceph-mon[30745]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T00:15:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:54 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:15:55.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:54 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:15:55.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:54 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:55.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:54 smithi027 ceph-mon[30745]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T00:15:55.128 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:54 smithi098 ceph-mon[34725]: purged_snaps scrub starts 2024-09-21T00:15:55.128 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:54 smithi098 ceph-mon[34725]: purged_snaps scrub ok 2024-09-21T00:15:55.128 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:54 smithi098 ceph-mon[34725]: Detected new or changed devices on smithi089 2024-09-21T00:15:55.128 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:54 smithi098 ceph-mon[34725]: Adjusting osd_memory_target on smithi089 to 5695M 2024-09-21T00:15:55.128 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:54 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T00:15:55.128 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:54 smithi098 ceph-mon[34725]: osd.7 [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546] boot 2024-09-21T00:15:55.129 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:54 smithi098 ceph-mon[34725]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T00:15:55.129 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:54 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:15:55.129 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:54 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:15:55.129 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:54 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:55.129 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:54 smithi098 ceph-mon[34725]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T00:15:55.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:54 smithi089 ceph-mon[34481]: purged_snaps scrub starts 2024-09-21T00:15:55.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:54 smithi089 ceph-mon[34481]: purged_snaps scrub ok 2024-09-21T00:15:55.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:54 smithi089 ceph-mon[34481]: Detected new or changed devices on smithi089 2024-09-21T00:15:55.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:54 smithi089 ceph-mon[34481]: Adjusting osd_memory_target on smithi089 to 5695M 2024-09-21T00:15:55.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:54 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T00:15:55.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:54 smithi089 ceph-mon[34481]: osd.7 [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546] boot 2024-09-21T00:15:55.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:54 smithi089 ceph-mon[34481]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T00:15:55.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:54 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:15:55.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:54 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:15:55.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:54 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:15:55.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:54 smithi089 ceph-mon[34481]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T00:15:56.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:55 smithi027 ceph-mon[30745]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:15:56.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:55 smithi027 ceph-mon[30745]: from='client.24520 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:15:56.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:55 smithi027 ceph-mon[30745]: osdmap e50: 8 total, 8 up, 8 in 2024-09-21T00:15:56.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:55 smithi089 ceph-mon[34481]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:15:56.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:55 smithi089 ceph-mon[34481]: from='client.24520 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:15:56.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:55 smithi089 ceph-mon[34481]: osdmap e50: 8 total, 8 up, 8 in 2024-09-21T00:15:56.230 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:55 smithi098 ceph-mon[34725]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:15:56.230 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:55 smithi098 ceph-mon[34725]: from='client.24520 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:15:56.230 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:55 smithi098 ceph-mon[34725]: osdmap e50: 8 total, 8 up, 8 in 2024-09-21T00:15:58.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:57 smithi027 ceph-mon[30745]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:15:58.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:57 smithi027 ceph-mon[30745]: from='client.? 172.21.15.98:0/4191536505' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0b558458-fc2d-4ed1-b72a-e2ea8cf811b0"} : dispatch 2024-09-21T00:15:58.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:57 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0b558458-fc2d-4ed1-b72a-e2ea8cf811b0"} : dispatch 2024-09-21T00:15:58.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:57 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b558458-fc2d-4ed1-b72a-e2ea8cf811b0"}]': finished 2024-09-21T00:15:58.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:57 smithi027 ceph-mon[30745]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T00:15:58.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:57 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T00:15:58.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:57 smithi089 ceph-mon[34481]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:15:58.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:57 smithi089 ceph-mon[34481]: from='client.? 172.21.15.98:0/4191536505' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0b558458-fc2d-4ed1-b72a-e2ea8cf811b0"} : dispatch 2024-09-21T00:15:58.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:57 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0b558458-fc2d-4ed1-b72a-e2ea8cf811b0"} : dispatch 2024-09-21T00:15:58.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:57 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b558458-fc2d-4ed1-b72a-e2ea8cf811b0"}]': finished 2024-09-21T00:15:58.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:57 smithi089 ceph-mon[34481]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T00:15:58.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:57 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T00:15:58.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:57 smithi098 ceph-mon[34725]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:15:58.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:57 smithi098 ceph-mon[34725]: from='client.? 172.21.15.98:0/4191536505' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0b558458-fc2d-4ed1-b72a-e2ea8cf811b0"} : dispatch 2024-09-21T00:15:58.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:57 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0b558458-fc2d-4ed1-b72a-e2ea8cf811b0"} : dispatch 2024-09-21T00:15:58.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:57 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b558458-fc2d-4ed1-b72a-e2ea8cf811b0"}]': finished 2024-09-21T00:15:58.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:57 smithi098 ceph-mon[34725]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T00:15:58.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:57 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T00:15:59.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:58 smithi027 ceph-mon[30745]: from='client.? 172.21.15.98:0/2124585786' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:15:59.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:58 smithi089 ceph-mon[34481]: from='client.? 172.21.15.98:0/2124585786' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:15:59.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:58 smithi098 ceph-mon[34725]: from='client.? 172.21.15.98:0/2124585786' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:16:00.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:15:59 smithi027 ceph-mon[30745]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:00.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:15:59 smithi098 ceph-mon[34725]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:00.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:15:59 smithi089 ceph-mon[34481]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:02.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:01 smithi027 ceph-mon[30745]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T00:16:02.128 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:01 smithi098 ceph-mon[34725]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T00:16:02.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:01 smithi089 ceph-mon[34481]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T00:16:03.894 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:03 smithi098 ceph-mon[34725]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-21T00:16:04.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:03 smithi089 ceph-mon[34481]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-21T00:16:04.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:03 smithi027 ceph-mon[30745]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-21T00:16:06.179 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:05 smithi098 ceph-mon[34725]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T00:16:06.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:05 smithi089 ceph-mon[34481]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T00:16:06.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:05 smithi027 ceph-mon[30745]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T00:16:06.919 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:06 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T00:16:06.919 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:06 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:07.130 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:06 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T00:16:07.130 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:06 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:07.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:06 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T00:16:07.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:06 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:08.103 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:07 smithi098 ceph-mon[34725]: Deploying daemon osd.8 on smithi098 2024-09-21T00:16:08.103 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:07 smithi098 ceph-mon[34725]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T00:16:08.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:07 smithi089 ceph-mon[34481]: Deploying daemon osd.8 on smithi098 2024-09-21T00:16:08.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:07 smithi089 ceph-mon[34481]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T00:16:08.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:07 smithi027 ceph-mon[30745]: Deploying daemon osd.8 on smithi098 2024-09-21T00:16:08.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:07 smithi027 ceph-mon[30745]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T00:16:09.912 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:09 smithi098 ceph-mon[34725]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T00:16:10.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:09 smithi089 ceph-mon[34481]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T00:16:10.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:09 smithi027 ceph-mon[30745]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T00:16:11.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:11 smithi098 ceph-mon[34725]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T00:16:12.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:11 smithi089 ceph-mon[34481]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T00:16:12.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:11 smithi027 ceph-mon[30745]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T00:16:12.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:12 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:16:12.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:12 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:12.992 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:12 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:13.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:12 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:16:13.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:12 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:13.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:12 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:13.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:12 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:16:13.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:12 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:13.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:12 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:13.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:13 smithi098 ceph-mon[34725]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:14.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:13 smithi089 ceph-mon[34481]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:14.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:13 smithi027 ceph-mon[30745]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:16.093 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:15 smithi098 ceph-mon[34725]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:16.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:16:16.137+0000 7f4dcffff640 1 -- 172.21.15.98:0/707579472 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4de8101c70 con 0x7f4dd00b6560 2024-09-21T00:16:16.139 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 8 on host 'smithi098' 2024-09-21T00:16:16.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:16:16.140+0000 7f4ded6f7640 1 -- 172.21.15.98:0/707579472 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4dd00b6560 msgr2=0x7f4dd00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:16:16.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:16:16.140+0000 7f4ded6f7640 1 --2- 172.21.15.98:0/707579472 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4dd00b6560 0x7f4dd00b8a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc006fd0 tx=0x7f4ddc008040 comp rx=0 tx=0).stop 2024-09-21T00:16:16.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:16:16.140+0000 7f4ded6f7640 1 -- 172.21.15.98:0/707579472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de8102a60 msgr2=0x7f4de819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:16:16.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:16:16.140+0000 7f4ded6f7640 1 --2- 172.21.15.98:0/707579472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de8102a60 0x7f4de819a600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800b4a0 tx=0x7f4dd800b960 comp rx=0 tx=0).stop 2024-09-21T00:16:16.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:16:16.140+0000 7f4ded6f7640 1 -- 172.21.15.98:0/707579472 shutdown_connections 2024-09-21T00:16:16.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:16:16.140+0000 7f4ded6f7640 1 --2- 172.21.15.98:0/707579472 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4dd00b6560 0x7f4dd00b8a70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:16.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:16:16.140+0000 7f4ded6f7640 1 --2- 172.21.15.98:0/707579472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4de8108cf0 0x7f4de819fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:16.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:16:16.140+0000 7f4ded6f7640 1 --2- 172.21.15.98:0/707579472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de81083a0 0x7f4de819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:16.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:16:16.140+0000 7f4ded6f7640 1 --2- 172.21.15.98:0/707579472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de8102a60 0x7f4de819a600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:16.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:16:16.140+0000 7f4ded6f7640 1 -- 172.21.15.98:0/707579472 >> 172.21.15.98:0/707579472 conn(0x7f4de8079730 msgr2=0x7f4de8101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:16:16.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:16:16.140+0000 7f4ded6f7640 1 -- 172.21.15.98:0/707579472 shutdown_connections 2024-09-21T00:16:16.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:16:16.140+0000 7f4ded6f7640 1 -- 172.21.15.98:0/707579472 wait complete. 2024-09-21T00:16:16.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:15 smithi089 ceph-mon[34481]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:16.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:15 smithi027 ceph-mon[30745]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:16.790 DEBUG:teuthology.orchestra.run.smithi098:osd.8> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T00:16:16.792 INFO:tasks.cephadm:Deploying osd.9 on smithi027 with /dev/vg_nvme/lv_1... 2024-09-21T00:16:16.792 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T00:16:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:16:17.283 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:17 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:17.283 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:17 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:17.283 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:17 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:17.283 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:17 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:17.383 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:17 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:17.383 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:17 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:17.383 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:17 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:17.383 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:17 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:17.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:17 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:17.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:17 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:17.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:17 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:17.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:17 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:18.388 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:18 smithi027 ceph-mon[30745]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:18.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:18 smithi098 ceph-mon[34725]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:18.438 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:18 smithi089 ceph-mon[34481]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:18.701 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:16:18 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:16:18.395+0000 7ff39ed6d640 -1 osd.8 0 log_to_monitors true 2024-09-21T00:16:19.277 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:19 smithi027 ceph-mon[30745]: from='osd.8 [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T00:16:19.278 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:19 smithi027 ceph-mon[30745]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T00:16:19.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:19 smithi089 ceph-mon[34481]: from='osd.8 [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T00:16:19.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:19 smithi089 ceph-mon[34481]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T00:16:19.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:19 smithi098 ceph-mon[34725]: from='osd.8 [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T00:16:19.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:19 smithi098 ceph-mon[34725]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T00:16:20.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:20.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T00:16:20.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T00:16:20.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T00:16:20.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='osd.8 [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:16:20.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:16:20.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: Detected new or changed devices on smithi098 2024-09-21T00:16:20.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:20.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:20.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: Adjusting osd_memory_target on smithi098 to 5696M 2024-09-21T00:16:20.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:20.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:20.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:16:20.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:20.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:20 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:16:20.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:20.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T00:16:20.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T00:16:20.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T00:16:20.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='osd.8 [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:16:20.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:16:20.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: Detected new or changed devices on smithi098 2024-09-21T00:16:20.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:20.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:20.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: Adjusting osd_memory_target on smithi098 to 5696M 2024-09-21T00:16:20.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:20.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:20.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:16:20.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:20.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:16:20.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:20.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T00:16:20.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T00:16:20.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T00:16:20.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='osd.8 [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:16:20.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:16:20.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: Detected new or changed devices on smithi098 2024-09-21T00:16:20.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:20.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:20.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:20.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: Adjusting osd_memory_target on smithi098 to 5696M 2024-09-21T00:16:20.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:20.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:20.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:16:20.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:20.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:20 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:16:20.672 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:16:20.705 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi027:vg_nvme/lv_1 2024-09-21T00:16:21.089 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:16:21.150 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:16:20.822+0000 7ff39b504640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T00:16:21.150 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:16:20 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:16:20.831+0000 7ff392ae5640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:16:21.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:21 smithi027 ceph-mon[30745]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T00:16:21.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:21 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T00:16:21.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:21 smithi027 ceph-mon[30745]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T00:16:21.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:21 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T00:16:21.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:21 smithi027 ceph-mon[30745]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:21.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:21 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:21.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:21 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:16:21.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:21 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:21.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:21 smithi027 ceph-mon[30745]: from='osd.8 ' entity='osd.8' 2024-09-21T00:16:21.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:21 smithi027 ceph-mon[30745]: osd.8 [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485] boot 2024-09-21T00:16:21.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:21 smithi027 ceph-mon[30745]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T00:16:21.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:21 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T00:16:21.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:21 smithi089 ceph-mon[34481]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T00:16:21.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:21 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T00:16:21.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:21 smithi089 ceph-mon[34481]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T00:16:21.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:21 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T00:16:21.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:21 smithi089 ceph-mon[34481]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:21.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:21 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:21.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:21 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:16:21.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:21 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:21.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:21 smithi089 ceph-mon[34481]: from='osd.8 ' entity='osd.8' 2024-09-21T00:16:21.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:21 smithi089 ceph-mon[34481]: osd.8 [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485] boot 2024-09-21T00:16:21.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:21 smithi089 ceph-mon[34481]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T00:16:21.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:21 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T00:16:21.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:21 smithi098 ceph-mon[34725]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T00:16:21.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:21 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T00:16:21.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:21 smithi098 ceph-mon[34725]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T00:16:21.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:21 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T00:16:21.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:21 smithi098 ceph-mon[34725]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T00:16:21.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:21 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:21.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:21 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:16:21.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:21 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:21.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:21 smithi098 ceph-mon[34725]: from='osd.8 ' entity='osd.8' 2024-09-21T00:16:21.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:21 smithi098 ceph-mon[34725]: osd.8 [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485] boot 2024-09-21T00:16:21.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:21 smithi098 ceph-mon[34725]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T00:16:21.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:21 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T00:16:21.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.731+0000 7f2460200640 1 -- 172.21.15.27:0/2240031296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f245810b6c0 msgr2=0x7f245810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:16:21.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.731+0000 7f2460200640 1 --2- 172.21.15.27:0/2240031296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f245810b6c0 0x7f245810bad0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2444009900 tx=0x7f244402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:16:21.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.731+0000 7f2460200640 1 -- 172.21.15.27:0/2240031296 shutdown_connections 2024-09-21T00:16:21.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.731+0000 7f2460200640 1 --2- 172.21.15.27:0/2240031296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f245810c010 0x7f2458112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:21.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.731+0000 7f2460200640 1 --2- 172.21.15.27:0/2240031296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f245810b6c0 0x7f245810bad0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:21.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.731+0000 7f2460200640 1 --2- 172.21.15.27:0/2240031296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2458105d80 0x7f2458106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:21.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.731+0000 7f2460200640 1 -- 172.21.15.27:0/2240031296 >> 172.21.15.27:0/2240031296 conn(0x7f2458101810 msgr2=0x7f2458103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:16:21.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.731+0000 7f2460200640 1 -- 172.21.15.27:0/2240031296 shutdown_connections 2024-09-21T00:16:21.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.732+0000 7f2460200640 1 -- 172.21.15.27:0/2240031296 wait complete. 2024-09-21T00:16:21.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.733+0000 7f2460200640 1 Processor -- start 2024-09-21T00:16:21.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.733+0000 7f2460200640 1 -- start start 2024-09-21T00:16:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.733+0000 7f2460200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2458105d80 0x7f245819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:16:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.733+0000 7f2460200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f245810b6c0 0x7f245819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:16:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.733+0000 7f2460200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f245810c010 0x7f24581a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:16:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.733+0000 7f2460200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2458115cd0 con 0x7f245810b6c0 2024-09-21T00:16:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.733+0000 7f2460200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2458115b50 con 0x7f245810c010 2024-09-21T00:16:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.733+0000 7f2460200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2458115e50 con 0x7f2458105d80 2024-09-21T00:16:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.734+0000 7f245ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f245810b6c0 0x7f245819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:16:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.734+0000 7f245ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f245810b6c0 0x7f245819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33974/0 (socket says 172.21.15.27:33974) 2024-09-21T00:16:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.734+0000 7f245ce00640 1 -- 172.21.15.27:0/474994674 learned_addr learned my addr 172.21.15.27:0/474994674 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:16:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.734+0000 7f245d800640 1 --2- 172.21.15.27:0/474994674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2458105d80 0x7f245819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:16:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.734+0000 7f245e200640 1 --2- 172.21.15.27:0/474994674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f245810c010 0x7f24581a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:16:21.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.734+0000 7f245ce00640 1 -- 172.21.15.27:0/474994674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2458105d80 msgr2=0x7f245819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:16:21.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.734+0000 7f245ce00640 1 --2- 172.21.15.27:0/474994674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2458105d80 0x7f245819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:21.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.734+0000 7f245ce00640 1 -- 172.21.15.27:0/474994674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f245810c010 msgr2=0x7f24581a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:16:21.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.734+0000 7f245ce00640 1 --2- 172.21.15.27:0/474994674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f245810c010 0x7f24581a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:21.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.734+0000 7f245ce00640 1 -- 172.21.15.27:0/474994674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24581a2610 con 0x7f245810b6c0 2024-09-21T00:16:21.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.735+0000 7f245ce00640 1 --2- 172.21.15.27:0/474994674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f245810b6c0 0x7f245819cc40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f244402a910 tx=0x7f2444031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:16:21.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.735+0000 7f2456000640 1 -- 172.21.15.27:0/474994674 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244403f030 con 0x7f245810b6c0 2024-09-21T00:16:21.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.735+0000 7f2456000640 1 -- 172.21.15.27:0/474994674 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2444004490 con 0x7f245810b6c0 2024-09-21T00:16:21.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.735+0000 7f2460200640 1 -- 172.21.15.27:0/474994674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24581a28a0 con 0x7f245810b6c0 2024-09-21T00:16:21.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.735+0000 7f2456000640 1 -- 172.21.15.27:0/474994674 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444035070 con 0x7f245810b6c0 2024-09-21T00:16:21.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.735+0000 7f2460200640 1 -- 172.21.15.27:0/474994674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24581a9060 con 0x7f245810b6c0 2024-09-21T00:16:21.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.736+0000 7f2460200640 1 -- 172.21.15.27:0/474994674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2458106190 con 0x7f245810b6c0 2024-09-21T00:16:21.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.739+0000 7f2456000640 1 -- 172.21.15.27:0/474994674 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f2444042030 con 0x7f245810b6c0 2024-09-21T00:16:21.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.740+0000 7f2456000640 1 --2- 172.21.15.27:0/474994674 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f24400b6560 0x7f24400b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:16:21.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.740+0000 7f2456000640 1 -- 172.21.15.27:0/474994674 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6077+0+0 (secure 0 0 0) 0x7f2444107220 con 0x7f245810b6c0 2024-09-21T00:16:21.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.741+0000 7f245d800640 1 --2- 172.21.15.27:0/474994674 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f24400b6560 0x7f24400b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:16:21.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.741+0000 7f245d800640 1 --2- 172.21.15.27:0/474994674 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f24400b6560 0x7f24400b8a70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f244c009bd0 tx=0x7f244c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:16:21.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.743+0000 7f2456000640 1 -- 172.21.15.27:0/474994674 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f24440d26b0 con 0x7f245810b6c0 2024-09-21T00:16:21.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:21.952+0000 7f2460200640 1 -- 172.21.15.27:0/474994674 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f245810de60 con 0x7f24400b6560 2024-09-21T00:16:22.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:22 smithi089 ceph-mon[34481]: purged_snaps scrub starts 2024-09-21T00:16:22.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:22 smithi089 ceph-mon[34481]: purged_snaps scrub ok 2024-09-21T00:16:22.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:22 smithi089 ceph-mon[34481]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:16:22.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:22 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:16:22.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:22 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:16:22.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:22 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:22.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:22 smithi089 ceph-mon[34481]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T00:16:22.493 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:22 smithi027 ceph-mon[30745]: purged_snaps scrub starts 2024-09-21T00:16:22.493 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:22 smithi027 ceph-mon[30745]: purged_snaps scrub ok 2024-09-21T00:16:22.493 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:22 smithi027 ceph-mon[30745]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:16:22.493 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:22 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:16:22.493 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:22 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:16:22.493 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:22 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:22.493 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:22 smithi027 ceph-mon[30745]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T00:16:22.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:22 smithi098 ceph-mon[34725]: purged_snaps scrub starts 2024-09-21T00:16:22.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:22 smithi098 ceph-mon[34725]: purged_snaps scrub ok 2024-09-21T00:16:22.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:22 smithi098 ceph-mon[34725]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:16:22.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:22 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:16:22.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:22 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:16:22.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:22 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:22.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:22 smithi098 ceph-mon[34725]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T00:16:24.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:23 smithi027 ceph-mon[30745]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 491 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:24.173 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:23 smithi027 ceph-mon[30745]: osdmap e56: 9 total, 9 up, 9 in 2024-09-21T00:16:24.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:23 smithi098 ceph-mon[34725]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 491 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:24.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:23 smithi098 ceph-mon[34725]: osdmap e56: 9 total, 9 up, 9 in 2024-09-21T00:16:24.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:23 smithi089 ceph-mon[34481]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 491 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:24.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:23 smithi089 ceph-mon[34481]: osdmap e56: 9 total, 9 up, 9 in 2024-09-21T00:16:26.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:25 smithi098 ceph-mon[34725]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 492 MiB used, 804 GiB / 805 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-09-21T00:16:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:25 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3860366655' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b"} : dispatch 2024-09-21T00:16:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:25 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3860366655' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b"}]': finished 2024-09-21T00:16:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:25 smithi098 ceph-mon[34725]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T00:16:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:25 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:25 smithi027 ceph-mon[30745]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 492 MiB used, 804 GiB / 805 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-09-21T00:16:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:25 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3860366655' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b"} : dispatch 2024-09-21T00:16:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:25 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3860366655' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b"}]': finished 2024-09-21T00:16:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:25 smithi027 ceph-mon[30745]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T00:16:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:25 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:26.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:25 smithi089 ceph-mon[34481]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 492 MiB used, 804 GiB / 805 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-09-21T00:16:26.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:25 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3860366655' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b"} : dispatch 2024-09-21T00:16:26.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:25 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3860366655' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b"}]': finished 2024-09-21T00:16:26.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:25 smithi089 ceph-mon[34481]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T00:16:26.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:25 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:27.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:26 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/655264015' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:16:27.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:26 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/655264015' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:16:27.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:26 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/655264015' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:16:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:27 smithi098 ceph-mon[34725]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-21T00:16:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:27 smithi027 ceph-mon[30745]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-21T00:16:28.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:27 smithi089 ceph-mon[34481]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-21T00:16:30.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:29 smithi098 ceph-mon[34725]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-21T00:16:30.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:29 smithi027 ceph-mon[30745]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-21T00:16:30.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:29 smithi089 ceph-mon[34481]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-21T00:16:32.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:31 smithi027 ceph-mon[30745]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T00:16:32.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:31 smithi089 ceph-mon[34481]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T00:16:32.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:31 smithi098 ceph-mon[34725]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T00:16:34.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:34 smithi027 ceph-mon[30745]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T00:16:34.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:34 smithi089 ceph-mon[34481]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T00:16:34.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:34 smithi098 ceph-mon[34725]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T00:16:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:35 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T00:16:35.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:35 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:35.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:35 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T00:16:35.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:35 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:35.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:35 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T00:16:35.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:35 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:36.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:36 smithi027 ceph-mon[30745]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:36.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:36 smithi027 ceph-mon[30745]: Deploying daemon osd.9 on smithi027 2024-09-21T00:16:36.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:36 smithi089 ceph-mon[34481]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:36.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:36 smithi089 ceph-mon[34481]: Deploying daemon osd.9 on smithi027 2024-09-21T00:16:36.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:36 smithi098 ceph-mon[34725]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:36.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:36 smithi098 ceph-mon[34725]: Deploying daemon osd.9 on smithi027 2024-09-21T00:16:38.281 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:38 smithi089 ceph-mon[34481]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:38.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:38 smithi027 ceph-mon[30745]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:38.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:38 smithi098 ceph-mon[34725]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:40.230 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:40 smithi027 ceph-mon[30745]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:40.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:40 smithi089 ceph-mon[34481]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:40.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:40 smithi098 ceph-mon[34725]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:42.314 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:42 smithi027 ceph-mon[30745]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:42.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:42 smithi089 ceph-mon[34481]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:42.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:42 smithi098 ceph-mon[34725]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:44.287 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:44 smithi027 ceph-mon[30745]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:44.287 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:44 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:16:44.287 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:44 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:44.287 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:44 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:44.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:44 smithi089 ceph-mon[34481]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:44.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:44 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:16:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:44 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:44 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:44.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:44 smithi098 ceph-mon[34725]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:44.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:44 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:16:44.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:44 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:44.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:44 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:46.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:46 smithi027 ceph-mon[30745]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:46.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:46 smithi089 ceph-mon[34481]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:46.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:46 smithi098 ceph-mon[34725]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:48.216 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:47 smithi027 ceph-mon[30745]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:48.216 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:47 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:48.216 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:47 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:48.216 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:47 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:48.216 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:47 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:16:48.217 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:47 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:48.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:47 smithi089 ceph-mon[34481]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:48.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:47 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:48.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:47 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:48.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:47 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:48.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:47 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:16:48.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:47 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:48.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:47 smithi098 ceph-mon[34725]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:48.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:47 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:48.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:47 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:48.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:47 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:48.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:47 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:16:48.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:47 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:48.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:48.754+0000 7f2456000640 1 -- 172.21.15.27:0/474994674 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f245810de60 con 0x7f24400b6560 2024-09-21T00:16:48.757 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 9 on host 'smithi027' 2024-09-21T00:16:48.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:48.758+0000 7f2460200640 1 -- 172.21.15.27:0/474994674 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f24400b6560 msgr2=0x7f24400b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:16:48.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:48.758+0000 7f2460200640 1 --2- 172.21.15.27:0/474994674 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f24400b6560 0x7f24400b8a70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f244c009bd0 tx=0x7f244c005f90 comp rx=0 tx=0).stop 2024-09-21T00:16:48.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:48.758+0000 7f2460200640 1 -- 172.21.15.27:0/474994674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f245810b6c0 msgr2=0x7f245819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:16:48.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:48.758+0000 7f2460200640 1 --2- 172.21.15.27:0/474994674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f245810b6c0 0x7f245819cc40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f244402a910 tx=0x7f2444031a20 comp rx=0 tx=0).stop 2024-09-21T00:16:48.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:48.758+0000 7f2460200640 1 -- 172.21.15.27:0/474994674 shutdown_connections 2024-09-21T00:16:48.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:48.758+0000 7f2460200640 1 --2- 172.21.15.27:0/474994674 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f24400b6560 0x7f24400b8a70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:48.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:48.758+0000 7f2460200640 1 --2- 172.21.15.27:0/474994674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f245810c010 0x7f24581a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:48.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:48.758+0000 7f2460200640 1 --2- 172.21.15.27:0/474994674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f245810b6c0 0x7f245819cc40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:48.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:48.758+0000 7f2460200640 1 --2- 172.21.15.27:0/474994674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2458105d80 0x7f245819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:48.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:48.758+0000 7f2460200640 1 -- 172.21.15.27:0/474994674 >> 172.21.15.27:0/474994674 conn(0x7f2458101810 msgr2=0x7f2458103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:16:48.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:48.758+0000 7f2460200640 1 -- 172.21.15.27:0/474994674 shutdown_connections 2024-09-21T00:16:48.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:16:48.758+0000 7f2460200640 1 -- 172.21.15.27:0/474994674 wait complete. 2024-09-21T00:16:49.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:48 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:16:49.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:48 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:49.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:48 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:49.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:48 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:16:49.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:48 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:49.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:48 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:49.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:48 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:16:49.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:48 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:49.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:48 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:49.584 DEBUG:teuthology.orchestra.run.smithi027:osd.9> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T00:16:49.587 INFO:tasks.cephadm:Deploying osd.10 on smithi089 with /dev/vg_nvme/lv_1... 2024-09-21T00:16:49.587 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T00:16:49.842 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T00:16:50.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:49 smithi027 ceph-mon[30745]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:50.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:49 smithi027 ceph-mon[30745]: from='osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T00:16:50.215 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:49 smithi089 ceph-mon[34481]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:50.215 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:49 smithi089 ceph-mon[34481]: from='osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T00:16:50.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:49 smithi098 ceph-mon[34725]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:50.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:49 smithi098 ceph-mon[34725]: from='osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T00:16:51.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:50 smithi027 ceph-mon[30745]: from='osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T00:16:51.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:50 smithi027 ceph-mon[30745]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T00:16:51.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:50 smithi027 ceph-mon[30745]: from='osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:16:51.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:50 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:51.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:50 smithi098 ceph-mon[34725]: from='osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T00:16:51.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:50 smithi098 ceph-mon[34725]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T00:16:51.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:50 smithi098 ceph-mon[34725]: from='osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:16:51.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:50 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:51.335 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:50 smithi089 ceph-mon[34481]: from='osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T00:16:51.335 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:50 smithi089 ceph-mon[34481]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T00:16:51.335 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:50 smithi089 ceph-mon[34481]: from='osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T00:16:51.335 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:50 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:51.785 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:16:51 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:16:51.749+0000 7f0dab400640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T00:16:51.785 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:16:51 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:16:51.782+0000 7f0da1400640 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:16:52.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:51 smithi027 ceph-mon[30745]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:52.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:51 smithi027 ceph-mon[30745]: from='osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T00:16:52.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:51 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:52.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:51 smithi027 ceph-mon[30745]: osdmap e59: 10 total, 9 up, 10 in 2024-09-21T00:16:52.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:51 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:52.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:51 smithi027 ceph-mon[30745]: from='osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113]' entity='osd.9' 2024-09-21T00:16:52.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:51 smithi098 ceph-mon[34725]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:52.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:51 smithi098 ceph-mon[34725]: from='osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T00:16:52.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:51 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:52.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:51 smithi098 ceph-mon[34725]: osdmap e59: 10 total, 9 up, 10 in 2024-09-21T00:16:52.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:51 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:52.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:51 smithi098 ceph-mon[34725]: from='osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113]' entity='osd.9' 2024-09-21T00:16:52.467 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:51 smithi089 ceph-mon[34481]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:52.468 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:51 smithi089 ceph-mon[34481]: from='osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T00:16:52.468 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:51 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:52.468 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:51 smithi089 ceph-mon[34481]: osdmap e59: 10 total, 9 up, 10 in 2024-09-21T00:16:52.468 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:51 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:52.468 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:51 smithi089 ceph-mon[34481]: from='osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113]' entity='osd.9' 2024-09-21T00:16:52.470 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T00:16:52.498 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi089:vg_nvme/lv_1 2024-09-21T00:16:52.765 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T00:16:53.054 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:52 smithi089 ceph-mon[34481]: purged_snaps scrub starts 2024-09-21T00:16:53.054 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:52 smithi089 ceph-mon[34481]: purged_snaps scrub ok 2024-09-21T00:16:53.054 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:52 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:53.054 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:52 smithi089 ceph-mon[34481]: osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113] boot 2024-09-21T00:16:53.054 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:52 smithi089 ceph-mon[34481]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T00:16:53.054 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:52 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:53.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:52 smithi098 ceph-mon[34725]: purged_snaps scrub starts 2024-09-21T00:16:53.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:52 smithi098 ceph-mon[34725]: purged_snaps scrub ok 2024-09-21T00:16:53.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:52 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:53.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:52 smithi098 ceph-mon[34725]: osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113] boot 2024-09-21T00:16:53.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:52 smithi098 ceph-mon[34725]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T00:16:53.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:52 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:53.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.272+0000 7fa8e6581640 1 -- 172.21.15.89:0/2351353118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8e01083a0 msgr2=0x7fa8e01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:16:53.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.272+0000 7fa8e6581640 1 --2- 172.21.15.89:0/2351353118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8e01083a0 0x7fa8e01087b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009900 tx=0x7fa8c802a410 comp rx=0 tx=0).stop 2024-09-21T00:16:53.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.272+0000 7fa8e6581640 1 -- 172.21.15.89:0/2351353118 shutdown_connections 2024-09-21T00:16:53.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.272+0000 7fa8e6581640 1 --2- 172.21.15.89:0/2351353118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e0108cf0 0x7fa8e010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:53.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.272+0000 7fa8e6581640 1 --2- 172.21.15.89:0/2351353118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8e01083a0 0x7fa8e01087b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:53.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.272+0000 7fa8e6581640 1 --2- 172.21.15.89:0/2351353118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8e0102a60 0x7fa8e0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:53.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.272+0000 7fa8e6581640 1 -- 172.21.15.89:0/2351353118 >> 172.21.15.89:0/2351353118 conn(0x7fa8e0079730 msgr2=0x7fa8e0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:16:53.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.272+0000 7fa8e6581640 1 -- 172.21.15.89:0/2351353118 shutdown_connections 2024-09-21T00:16:53.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.272+0000 7fa8e6581640 1 -- 172.21.15.89:0/2351353118 wait complete. 2024-09-21T00:16:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.273+0000 7fa8e6581640 1 Processor -- start 2024-09-21T00:16:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.273+0000 7fa8e6581640 1 -- start start 2024-09-21T00:16:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8e6581640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e0102a60 0x7fa8e019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:16:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8e6581640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8e01083a0 0x7fa8e019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:16:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8e6581640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8e0108cf0 0x7fa8e019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:16:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8e6581640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e0112960 con 0x7fa8e0102a60 2024-09-21T00:16:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8e6581640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e01127e0 con 0x7fa8e0108cf0 2024-09-21T00:16:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8e6581640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e0112ae0 con 0x7fa8e01083a0 2024-09-21T00:16:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8e4b1e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8e0108cf0 0x7fa8e019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:16:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8e4b1e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8e0108cf0 0x7fa8e019bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58488/0 (socket says 172.21.15.89:58488) 2024-09-21T00:16:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8e4b1e640 1 -- 172.21.15.89:0/2215019290 learned_addr learned my addr 172.21.15.89:0/2215019290 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T00:16:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8dffff640 1 --2- 172.21.15.89:0/2215019290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e0102a60 0x7fa8e019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:16:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8d7fff640 1 --2- 172.21.15.89:0/2215019290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8e01083a0 0x7fa8e019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:16:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8e4b1e640 1 -- 172.21.15.89:0/2215019290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8e01083a0 msgr2=0x7fa8e019b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:16:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8e4b1e640 1 --2- 172.21.15.89:0/2215019290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8e01083a0 0x7fa8e019b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8e4b1e640 1 -- 172.21.15.89:0/2215019290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e0102a60 msgr2=0x7fa8e019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:16:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8e4b1e640 1 --2- 172.21.15.89:0/2215019290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e0102a60 0x7fa8e019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:16:53.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8e4b1e640 1 -- 172.21.15.89:0/2215019290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e01a65c0 con 0x7fa8e0108cf0 2024-09-21T00:16:53.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8d7fff640 1 --2- 172.21.15.89:0/2215019290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8e01083a0 0x7fa8e019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:16:53.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.274+0000 7fa8e4b1e640 1 --2- 172.21.15.89:0/2215019290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8e0108cf0 0x7fa8e019bd30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009fd0 tx=0x7fa8d800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:16:53.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.275+0000 7fa8ddffb640 1 -- 172.21.15.89:0/2215019290 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8017070 con 0x7fa8e0108cf0 2024-09-21T00:16:53.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.275+0000 7fa8e6581640 1 -- 172.21.15.89:0/2215019290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e01a68b0 con 0x7fa8e0108cf0 2024-09-21T00:16:53.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.275+0000 7fa8ddffb640 1 -- 172.21.15.89:0/2215019290 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d8003e20 con 0x7fa8e0108cf0 2024-09-21T00:16:53.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.275+0000 7fa8ddffb640 1 -- 172.21.15.89:0/2215019290 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8010460 con 0x7fa8e0108cf0 2024-09-21T00:16:53.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.275+0000 7fa8e6581640 1 -- 172.21.15.89:0/2215019290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e01a6dc0 con 0x7fa8e0108cf0 2024-09-21T00:16:53.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.276+0000 7fa8e6581640 1 -- 172.21.15.89:0/2215019290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e0102e70 con 0x7fa8e0108cf0 2024-09-21T00:16:53.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.277+0000 7fa8ddffb640 1 -- 172.21.15.89:0/2215019290 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fa8d8003970 con 0x7fa8e0108cf0 2024-09-21T00:16:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.278+0000 7fa8ddffb640 1 --2- 172.21.15.89:0/2215019290 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa8cc0b65e0 0x7fa8cc0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:16:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.278+0000 7fa8ddffb640 1 -- 172.21.15.89:0/2215019290 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6476+0+0 (secure 0 0 0) 0x7fa8d80e68c0 con 0x7fa8e0108cf0 2024-09-21T00:16:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.279+0000 7fa8dffff640 1 --2- 172.21.15.89:0/2215019290 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa8cc0b65e0 0x7fa8cc0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:16:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.279+0000 7fa8dffff640 1 --2- 172.21.15.89:0/2215019290 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa8cc0b65e0 0x7fa8cc0b8af0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa8e006ba20 tx=0x7fa8d000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:16:53.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.286+0000 7fa8ddffb640 1 -- 172.21.15.89:0/2215019290 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa8d80ad900 con 0x7fa8e0108cf0 2024-09-21T00:16:53.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:52 smithi027 ceph-mon[30745]: purged_snaps scrub starts 2024-09-21T00:16:53.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:52 smithi027 ceph-mon[30745]: purged_snaps scrub ok 2024-09-21T00:16:53.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:52 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:53.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:52 smithi027 ceph-mon[30745]: osd.9 [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113] boot 2024-09-21T00:16:53.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:52 smithi027 ceph-mon[30745]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T00:16:53.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:52 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T00:16:53.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:16:53.480+0000 7fa8e6581640 1 -- 172.21.15.89:0/2215019290 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa8e0063800 con 0x7fa8cc0b65e0 2024-09-21T00:16:54.090 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:54.091 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:54.091 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:54.091 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.091 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.091 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.091 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.091 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.091 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.091 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.091 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.091 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:54.091 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:16:54.091 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:16:54.092 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:54.092 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:54.092 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:16:54.092 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:53 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:54.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:54.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:54.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:54.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:54.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:16:54.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:16:54.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:54.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:54.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:16:54.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:53 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:54.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-21T00:16:54.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:54.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:54.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T00:16:54.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:54.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:16:54.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:16:54.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:54.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:16:54.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:16:54.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:53 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:16:55.147 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:55 smithi089 ceph-mon[34481]: Detected new or changed devices on smithi027 2024-09-21T00:16:55.147 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:55 smithi089 ceph-mon[34481]: Adjusting osd_memory_target on smithi027 to 4272M 2024-09-21T00:16:55.147 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:55 smithi089 ceph-mon[34481]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:16:55.147 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:55 smithi089 ceph-mon[34481]: osdmap e61: 10 total, 10 up, 10 in 2024-09-21T00:16:55.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:55 smithi027 ceph-mon[30745]: Detected new or changed devices on smithi027 2024-09-21T00:16:55.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:55 smithi027 ceph-mon[30745]: Adjusting osd_memory_target on smithi027 to 4272M 2024-09-21T00:16:55.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:55 smithi027 ceph-mon[30745]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:16:55.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:55 smithi027 ceph-mon[30745]: osdmap e61: 10 total, 10 up, 10 in 2024-09-21T00:16:55.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:55 smithi098 ceph-mon[34725]: Detected new or changed devices on smithi027 2024-09-21T00:16:55.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:55 smithi098 ceph-mon[34725]: Adjusting osd_memory_target on smithi027 to 4272M 2024-09-21T00:16:55.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:55 smithi098 ceph-mon[34725]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:16:55.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:55 smithi098 ceph-mon[34725]: osdmap e61: 10 total, 10 up, 10 in 2024-09-21T00:16:56.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:56 smithi027 ceph-mon[30745]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:16:56.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:56 smithi027 ceph-mon[30745]: from='client.? 172.21.15.89:0/1776948147' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "324dd6ce-68fe-4c76-959b-1c9fb0ae92a7"} : dispatch 2024-09-21T00:16:56.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:56 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "324dd6ce-68fe-4c76-959b-1c9fb0ae92a7"} : dispatch 2024-09-21T00:16:56.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:56 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "324dd6ce-68fe-4c76-959b-1c9fb0ae92a7"}]': finished 2024-09-21T00:16:56.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:56 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:16:56.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:56 smithi027 ceph-mon[30745]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T00:16:56.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:56 smithi089 ceph-mon[34481]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:16:56.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:56 smithi089 ceph-mon[34481]: from='client.? 172.21.15.89:0/1776948147' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "324dd6ce-68fe-4c76-959b-1c9fb0ae92a7"} : dispatch 2024-09-21T00:16:56.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:56 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "324dd6ce-68fe-4c76-959b-1c9fb0ae92a7"} : dispatch 2024-09-21T00:16:56.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:56 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "324dd6ce-68fe-4c76-959b-1c9fb0ae92a7"}]': finished 2024-09-21T00:16:56.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:56 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:16:56.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:56 smithi089 ceph-mon[34481]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T00:16:56.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:56 smithi098 ceph-mon[34725]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:16:56.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:56 smithi098 ceph-mon[34725]: from='client.? 172.21.15.89:0/1776948147' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "324dd6ce-68fe-4c76-959b-1c9fb0ae92a7"} : dispatch 2024-09-21T00:16:56.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:56 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "324dd6ce-68fe-4c76-959b-1c9fb0ae92a7"} : dispatch 2024-09-21T00:16:56.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:56 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "324dd6ce-68fe-4c76-959b-1c9fb0ae92a7"}]': finished 2024-09-21T00:16:56.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:56 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:16:56.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:56 smithi098 ceph-mon[34725]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T00:16:57.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:57 smithi027 ceph-mon[30745]: from='client.? 172.21.15.89:0/1404547721' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:16:57.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:57 smithi089 ceph-mon[34481]: from='client.? 172.21.15.89:0/1404547721' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:16:57.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:57 smithi098 ceph-mon[34725]: from='client.? 172.21.15.89:0/1404547721' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:16:58.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:16:58 smithi027 ceph-mon[30745]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:16:58.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:16:58 smithi089 ceph-mon[34481]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:16:58.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:16:58 smithi098 ceph-mon[34725]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:00.298 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:00 smithi098 ceph-mon[34725]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:00.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:00 smithi027 ceph-mon[30745]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:00.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:00 smithi089 ceph-mon[34481]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:02.284 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:02 smithi089 ceph-mon[34481]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:02.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:02 smithi027 ceph-mon[30745]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:02.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:02 smithi098 ceph-mon[34725]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:04.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:04 smithi027 ceph-mon[30745]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:04.349 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:04 smithi089 ceph-mon[34481]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:04.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:04 smithi098 ceph-mon[34725]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:05.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:05 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T00:17:05.117 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:05 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:05.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:05 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T00:17:05.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:05 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:05.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:05 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T00:17:05.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:05 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:06.201 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:06 smithi089 ceph-mon[34481]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:06.201 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:06 smithi089 ceph-mon[34481]: Deploying daemon osd.10 on smithi089 2024-09-21T00:17:06.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:06 smithi027 ceph-mon[30745]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:06.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:06 smithi027 ceph-mon[30745]: Deploying daemon osd.10 on smithi089 2024-09-21T00:17:06.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:06 smithi098 ceph-mon[34725]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:06.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:06 smithi098 ceph-mon[34725]: Deploying daemon osd.10 on smithi089 2024-09-21T00:17:08.209 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:08 smithi089 ceph-mon[34481]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:08.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:08 smithi027 ceph-mon[30745]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:08.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:08 smithi098 ceph-mon[34725]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:10.189 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:10 smithi089 ceph-mon[34481]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:10.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:10 smithi027 ceph-mon[30745]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:10.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:10 smithi098 ceph-mon[34725]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:11.147 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:11 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:17:11.147 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:11 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:11.147 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:11 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:11.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:11 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:17:11.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:11 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:11.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:11 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:11.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:11 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:17:11.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:11 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:11.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:11 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:12.214 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:12 smithi089 ceph-mon[34481]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:12.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:12 smithi027 ceph-mon[30745]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:12.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:12 smithi098 ceph-mon[34725]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:14.249 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 10 on host 'smithi089' 2024-09-21T00:17:14.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:17:14.247+0000 7fa8ddffb640 1 -- 172.21.15.89:0/2215019290 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa8e0063800 con 0x7fa8cc0b65e0 2024-09-21T00:17:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:17:14.250+0000 7fa8e6581640 1 -- 172.21.15.89:0/2215019290 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa8cc0b65e0 msgr2=0x7fa8cc0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:17:14.250+0000 7fa8e6581640 1 --2- 172.21.15.89:0/2215019290 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa8cc0b65e0 0x7fa8cc0b8af0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa8e006ba20 tx=0x7fa8d000b390 comp rx=0 tx=0).stop 2024-09-21T00:17:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:17:14.250+0000 7fa8e6581640 1 -- 172.21.15.89:0/2215019290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8e0108cf0 msgr2=0x7fa8e019bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:17:14.250+0000 7fa8e6581640 1 --2- 172.21.15.89:0/2215019290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8e0108cf0 0x7fa8e019bd30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009fd0 tx=0x7fa8d800b260 comp rx=0 tx=0).stop 2024-09-21T00:17:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:17:14.250+0000 7fa8e6581640 1 -- 172.21.15.89:0/2215019290 shutdown_connections 2024-09-21T00:17:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:17:14.250+0000 7fa8e6581640 1 --2- 172.21.15.89:0/2215019290 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa8cc0b65e0 0x7fa8cc0b8af0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:17:14.250+0000 7fa8e6581640 1 --2- 172.21.15.89:0/2215019290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8e0108cf0 0x7fa8e019bd30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:17:14.250+0000 7fa8e6581640 1 --2- 172.21.15.89:0/2215019290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8e01083a0 0x7fa8e019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:17:14.250+0000 7fa8e6581640 1 --2- 172.21.15.89:0/2215019290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e0102a60 0x7fa8e019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:17:14.250+0000 7fa8e6581640 1 -- 172.21.15.89:0/2215019290 >> 172.21.15.89:0/2215019290 conn(0x7fa8e0079730 msgr2=0x7fa8e0104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:17:14.250+0000 7fa8e6581640 1 -- 172.21.15.89:0/2215019290 shutdown_connections 2024-09-21T00:17:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T00:17:14.250+0000 7fa8e6581640 1 -- 172.21.15.89:0/2215019290 wait complete. 2024-09-21T00:17:14.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:14 smithi089 ceph-mon[34481]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:14.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:14 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:14.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:14 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:14.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:14 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:14.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:14 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:17:14.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:14 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:14.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:14 smithi098 ceph-mon[34725]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:14.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:14 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:14.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:14 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:14.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:14 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:14.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:14 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:17:14.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:14 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:14.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:14 smithi027 ceph-mon[30745]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:14.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:14 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:14.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:14 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:14.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:14 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:14.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:14 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:17:14.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:14 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:14.932 DEBUG:teuthology.orchestra.run.smithi089:osd.10> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T00:17:14.935 INFO:tasks.cephadm:Deploying osd.11 on smithi098 with /dev/vg_nvme/lv_1... 2024-09-21T00:17:14.935 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T00:17:15.195 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:17:15.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:15 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:17:15.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:15 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:15.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:15 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:15.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:15 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:17:15.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:15 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:15.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:15 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:15.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:15 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:17:15.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:15 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:15.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:15 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:16.120 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:16 smithi089 ceph-mon[34481]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:16.333 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:16 smithi098 ceph-mon[34725]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:16.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:16 smithi027 ceph-mon[30745]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:16.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:17:16 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:17:16.520+0000 7f97da8bc640 -1 osd.10 0 log_to_monitors true 2024-09-21T00:17:17.195 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:17 smithi098 ceph-mon[34725]: from='osd.10 [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T00:17:17.195 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:17 smithi098 ceph-mon[34725]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T00:17:17.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:17 smithi089 ceph-mon[34481]: from='osd.10 [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T00:17:17.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:17 smithi089 ceph-mon[34481]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T00:17:17.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:17 smithi027 ceph-mon[30745]: from='osd.10 [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T00:17:17.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:17 smithi027 ceph-mon[30745]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T00:17:17.825 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T00:17:17.853 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi098:vg_nvme/lv_1 2024-09-21T00:17:18.101 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:17:18.382 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:18.383 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T00:17:18.383 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:17:18.383 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='osd.10 [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:17:18.383 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T00:17:18.383 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:17:18.383 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:18.383 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:18.383 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.383 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.383 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.383 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.383 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.383 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.384 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.384 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.384 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:18.384 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:18.384 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:17:18.384 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:18 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:18.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:18.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T00:17:18.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:17:18.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='osd.10 [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:17:18.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T00:17:18.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:17:18.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:18.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:18.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:18.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:18.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:17:18.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:18.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:18.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T00:17:18.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:17:18.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='osd.10 [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:17:18.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T00:17:18.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T00:17:18.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:18.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:18.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:18.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:18.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:18.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:17:18.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:18 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:18.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.601+0000 7fa5cedbe640 1 -- 172.21.15.98:0/3615503935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c8107690 msgr2=0x7fa5c8107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:18.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.601+0000 7fa5cedbe640 1 --2- 172.21.15.98:0/3615503935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c8107690 0x7fa5c8107aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009900 tx=0x7fa5b802a410 comp rx=0 tx=0).stop 2024-09-21T00:17:18.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.601+0000 7fa5cedbe640 1 -- 172.21.15.98:0/3615503935 shutdown_connections 2024-09-21T00:17:18.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.601+0000 7fa5cedbe640 1 --2- 172.21.15.98:0/3615503935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c8103950 0x7fa5c810f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:18.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.601+0000 7fa5cedbe640 1 --2- 172.21.15.98:0/3615503935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5c8103000 0x7fa5c8103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:18.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.601+0000 7fa5cedbe640 1 --2- 172.21.15.98:0/3615503935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c8107690 0x7fa5c8107aa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:18.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.601+0000 7fa5cedbe640 1 -- 172.21.15.98:0/3615503935 >> 172.21.15.98:0/3615503935 conn(0x7fa5c8079ea0 msgr2=0x7fa5c807a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:18.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.601+0000 7fa5cedbe640 1 -- 172.21.15.98:0/3615503935 shutdown_connections 2024-09-21T00:17:18.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.601+0000 7fa5cedbe640 1 -- 172.21.15.98:0/3615503935 wait complete. 2024-09-21T00:17:18.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5cedbe640 1 Processor -- start 2024-09-21T00:17:18.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5cedbe640 1 -- start start 2024-09-21T00:17:18.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5cedbe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5c8103000 0x7fa5c819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:18.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5cedbe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c8103950 0x7fa5c819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:18.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5cedbe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c8107690 0x7fa5c819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:18.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5cedbe640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8113ba0 con 0x7fa5c8107690 2024-09-21T00:17:18.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5cedbe640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8113a20 con 0x7fa5c8103000 2024-09-21T00:17:18.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5cedbe640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8113d20 con 0x7fa5c8103950 2024-09-21T00:17:18.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5c7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c8103950 0x7fa5c819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:18.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5c7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c8103950 0x7fa5c819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33190/0 (socket says 172.21.15.98:33190) 2024-09-21T00:17:18.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5c7fff640 1 -- 172.21.15.98:0/1461715200 learned_addr learned my addr 172.21.15.98:0/1461715200 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-21T00:17:18.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5ccb5a640 1 --2- 172.21.15.98:0/1461715200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5c8103000 0x7fa5c819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:18.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5cd35b640 1 --2- 172.21.15.98:0/1461715200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c8107690 0x7fa5c819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:18.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5c7fff640 1 -- 172.21.15.98:0/1461715200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5c8103000 msgr2=0x7fa5c819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:18.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5c7fff640 1 --2- 172.21.15.98:0/1461715200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5c8103000 0x7fa5c819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:18.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5c7fff640 1 -- 172.21.15.98:0/1461715200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c8107690 msgr2=0x7fa5c819fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:18.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.602+0000 7fa5c7fff640 1 --2- 172.21.15.98:0/1461715200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c8107690 0x7fa5c819fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.603+0000 7fa5c7fff640 1 -- 172.21.15.98:0/1461715200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c81a0520 con 0x7fa5c8103950 2024-09-21T00:17:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.603+0000 7fa5c7fff640 1 --2- 172.21.15.98:0/1461715200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c8103950 0x7fa5c819ab50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc005fd0 tx=0x7fa5bc00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.603+0000 7fa5c5ffb640 1 -- 172.21.15.98:0/1461715200 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc017070 con 0x7fa5c8103950 2024-09-21T00:17:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.603+0000 7fa5c5ffb640 1 -- 172.21.15.98:0/1461715200 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5bc009c30 con 0x7fa5c8103950 2024-09-21T00:17:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.603+0000 7fa5cedbe640 1 -- 172.21.15.98:0/1461715200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c81a0810 con 0x7fa5c8103950 2024-09-21T00:17:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.603+0000 7fa5c5ffb640 1 -- 172.21.15.98:0/1461715200 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc003d10 con 0x7fa5c8103950 2024-09-21T00:17:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.603+0000 7fa5cedbe640 1 -- 172.21.15.98:0/1461715200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c81a6f70 con 0x7fa5c8103950 2024-09-21T00:17:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.604+0000 7fa5cedbe640 1 -- 172.21.15.98:0/1461715200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c8103410 con 0x7fa5c8103950 2024-09-21T00:17:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.606+0000 7fa5c5ffb640 1 -- 172.21.15.98:0/1461715200 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fa5bc003eb0 con 0x7fa5c8103950 2024-09-21T00:17:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.607+0000 7fa5c5ffb640 1 --2- 172.21.15.98:0/1461715200 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa5b00b65e0 0x7fa5b00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.607+0000 7fa5c5ffb640 1 -- 172.21.15.98:0/1461715200 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6629+0+0 (secure 0 0 0) 0x7fa5bc0e6970 con 0x7fa5c8103950 2024-09-21T00:17:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.607+0000 7fa5ccb5a640 1 --2- 172.21.15.98:0/1461715200 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa5b00b65e0 0x7fa5b00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.608+0000 7fa5ccb5a640 1 --2- 172.21.15.98:0/1461715200 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa5b00b65e0 0x7fa5b00b8af0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8031440 tx=0x7fa5b8009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:18.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.611+0000 7fa5c5ffb640 1 -- 172.21.15.98:0/1461715200 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa5bc020030 con 0x7fa5c8103950 2024-09-21T00:17:18.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:18.800+0000 7fa5cedbe640 1 -- 172.21.15.98:0/1461715200 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa5c8063800 con 0x7fa5b00b65e0 2024-09-21T00:17:19.095 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:17:18.806+0000 7f97d683e640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T00:17:19.095 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:17:18 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:17:18.818+0000 7f97ce634640 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:17:19.438 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:19 smithi098 ceph-mon[34725]: Detected new or changed devices on smithi089 2024-09-21T00:17:19.438 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:19 smithi098 ceph-mon[34725]: Adjusting osd_memory_target on smithi089 to 4271M 2024-09-21T00:17:19.438 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:19 smithi098 ceph-mon[34725]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T00:17:19.438 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:19 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:17:19.438 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:19 smithi098 ceph-mon[34725]: osdmap e64: 11 total, 10 up, 11 in 2024-09-21T00:17:19.439 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:19 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:17:19.439 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:19 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:17:19.439 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:19 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:17:19.439 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:19 smithi098 ceph-mon[34725]: from='osd.10 ' entity='osd.10' 2024-09-21T00:17:19.439 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:19 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:19.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:19 smithi089 ceph-mon[34481]: Detected new or changed devices on smithi089 2024-09-21T00:17:19.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:19 smithi089 ceph-mon[34481]: Adjusting osd_memory_target on smithi089 to 4271M 2024-09-21T00:17:19.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:19 smithi089 ceph-mon[34481]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T00:17:19.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:19 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:17:19.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:19 smithi089 ceph-mon[34481]: osdmap e64: 11 total, 10 up, 11 in 2024-09-21T00:17:19.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:19 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:17:19.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:19 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:17:19.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:19 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:17:19.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:19 smithi089 ceph-mon[34481]: from='osd.10 ' entity='osd.10' 2024-09-21T00:17:19.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:19 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:19.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:19 smithi027 ceph-mon[30745]: Detected new or changed devices on smithi089 2024-09-21T00:17:19.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:19 smithi027 ceph-mon[30745]: Adjusting osd_memory_target on smithi089 to 4271M 2024-09-21T00:17:19.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:19 smithi027 ceph-mon[30745]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T00:17:19.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:19 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:17:19.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:19 smithi027 ceph-mon[30745]: osdmap e64: 11 total, 10 up, 11 in 2024-09-21T00:17:19.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:19 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:17:19.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:19 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T00:17:19.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:19 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T00:17:19.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:19 smithi027 ceph-mon[30745]: from='osd.10 ' entity='osd.10' 2024-09-21T00:17:19.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:19 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:20.193 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:20 smithi098 ceph-mon[34725]: purged_snaps scrub starts 2024-09-21T00:17:20.193 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:20 smithi098 ceph-mon[34725]: purged_snaps scrub ok 2024-09-21T00:17:20.193 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:20 smithi098 ceph-mon[34725]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:20.193 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:20 smithi098 ceph-mon[34725]: from='client.24595 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:17:20.193 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:20 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:17:20.193 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:20 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:17:20.193 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:20 smithi098 ceph-mon[34725]: osd.10 [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225] boot 2024-09-21T00:17:20.194 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:20 smithi098 ceph-mon[34725]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T00:17:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:20 smithi089 ceph-mon[34481]: purged_snaps scrub starts 2024-09-21T00:17:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:20 smithi089 ceph-mon[34481]: purged_snaps scrub ok 2024-09-21T00:17:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:20 smithi089 ceph-mon[34481]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:20 smithi089 ceph-mon[34481]: from='client.24595 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:17:20.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:20 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:17:20.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:20 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:17:20.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:20 smithi089 ceph-mon[34481]: osd.10 [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225] boot 2024-09-21T00:17:20.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:20 smithi089 ceph-mon[34481]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T00:17:20.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:20 smithi027 ceph-mon[30745]: purged_snaps scrub starts 2024-09-21T00:17:20.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:20 smithi027 ceph-mon[30745]: purged_snaps scrub ok 2024-09-21T00:17:20.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:20 smithi027 ceph-mon[30745]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T00:17:20.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:20 smithi027 ceph-mon[30745]: from='client.24595 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:17:20.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:20 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:17:20.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:20 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T00:17:20.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:20 smithi027 ceph-mon[30745]: osd.10 [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225] boot 2024-09-21T00:17:20.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:20 smithi027 ceph-mon[30745]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T00:17:21.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:21 smithi098 ceph-mon[34725]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T00:17:21.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:21 smithi089 ceph-mon[34481]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T00:17:21.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:21 smithi027 ceph-mon[30745]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T00:17:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:22 smithi098 ceph-mon[34725]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:22 smithi098 ceph-mon[34725]: osdmap e67: 11 total, 11 up, 11 in 2024-09-21T00:17:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:22 smithi098 ceph-mon[34725]: from='client.? 172.21.15.98:0/425115623' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5e34686e-350a-491e-b9d7-59174ea90944"} : dispatch 2024-09-21T00:17:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:22 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5e34686e-350a-491e-b9d7-59174ea90944"} : dispatch 2024-09-21T00:17:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:22 smithi098 ceph-mon[34725]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e34686e-350a-491e-b9d7-59174ea90944"}]': finished 2024-09-21T00:17:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:22 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:22 smithi098 ceph-mon[34725]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T00:17:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:22 smithi098 ceph-mon[34725]: from='client.? 172.21.15.98:0/2428814520' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:17:22.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:22 smithi089 ceph-mon[34481]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:22.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:22 smithi089 ceph-mon[34481]: osdmap e67: 11 total, 11 up, 11 in 2024-09-21T00:17:22.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:22 smithi089 ceph-mon[34481]: from='client.? 172.21.15.98:0/425115623' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5e34686e-350a-491e-b9d7-59174ea90944"} : dispatch 2024-09-21T00:17:22.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:22 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5e34686e-350a-491e-b9d7-59174ea90944"} : dispatch 2024-09-21T00:17:22.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:22 smithi089 ceph-mon[34481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e34686e-350a-491e-b9d7-59174ea90944"}]': finished 2024-09-21T00:17:22.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:22 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:22.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:22 smithi089 ceph-mon[34481]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T00:17:22.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:22 smithi089 ceph-mon[34481]: from='client.? 172.21.15.98:0/2428814520' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:17:22.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:22 smithi027 ceph-mon[30745]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:22.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:22 smithi027 ceph-mon[30745]: osdmap e67: 11 total, 11 up, 11 in 2024-09-21T00:17:22.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:22 smithi027 ceph-mon[30745]: from='client.? 172.21.15.98:0/425115623' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5e34686e-350a-491e-b9d7-59174ea90944"} : dispatch 2024-09-21T00:17:22.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:22 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5e34686e-350a-491e-b9d7-59174ea90944"} : dispatch 2024-09-21T00:17:22.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:22 smithi027 ceph-mon[30745]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e34686e-350a-491e-b9d7-59174ea90944"}]': finished 2024-09-21T00:17:22.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:22 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:22.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:22 smithi027 ceph-mon[30745]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T00:17:22.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:22 smithi027 ceph-mon[30745]: from='client.? 172.21.15.98:0/2428814520' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T00:17:24.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:24 smithi089 ceph-mon[34481]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:24.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:24 smithi098 ceph-mon[34725]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:24.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:24 smithi027 ceph-mon[30745]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:26.446 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:26 smithi098 ceph-mon[34725]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-21T00:17:26.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:26 smithi089 ceph-mon[34481]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-21T00:17:26.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:26 smithi027 ceph-mon[30745]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-21T00:17:28.466 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:28 smithi098 ceph-mon[34725]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-21T00:17:28.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:28 smithi089 ceph-mon[34481]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-21T00:17:28.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:28 smithi027 ceph-mon[30745]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-21T00:17:30.233 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:30 smithi098 ceph-mon[34725]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T00:17:30.233 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:30 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T00:17:30.233 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:30 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:30.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:30 smithi089 ceph-mon[34481]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T00:17:30.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:30 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T00:17:30.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:30 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:30.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:30 smithi027 ceph-mon[30745]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T00:17:30.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:30 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T00:17:30.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:30 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:31.239 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:31 smithi098 ceph-mon[34725]: Deploying daemon osd.11 on smithi098 2024-09-21T00:17:31.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:31 smithi089 ceph-mon[34481]: Deploying daemon osd.11 on smithi098 2024-09-21T00:17:31.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:31 smithi027 ceph-mon[30745]: Deploying daemon osd.11 on smithi098 2024-09-21T00:17:32.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:32 smithi098 ceph-mon[34725]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T00:17:32.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:32 smithi089 ceph-mon[34481]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T00:17:32.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:32 smithi027 ceph-mon[30745]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T00:17:34.300 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:34 smithi098 ceph-mon[34725]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T00:17:34.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:34 smithi089 ceph-mon[34481]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T00:17:34.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:34 smithi027 ceph-mon[30745]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T00:17:36.432 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:36 smithi098 ceph-mon[34725]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T00:17:36.432 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:36 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:17:36.433 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:36 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:36.433 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:36 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:36.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:36 smithi089 ceph-mon[34481]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T00:17:36.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:36 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:17:36.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:36 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:36.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:36 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:36.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:36 smithi027 ceph-mon[30745]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T00:17:36.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:36 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:17:36.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:36 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:36.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:36 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:38.466 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:38 smithi098 ceph-mon[34725]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:38.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:38 smithi089 ceph-mon[34481]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:38.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:38 smithi027 ceph-mon[30745]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:38.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:38.835+0000 7fa5c5ffb640 1 -- 172.21.15.98:0/1461715200 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa5c8063800 con 0x7fa5b00b65e0 2024-09-21T00:17:38.838 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 11 on host 'smithi098' 2024-09-21T00:17:38.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:38.838+0000 7fa5cedbe640 1 -- 172.21.15.98:0/1461715200 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa5b00b65e0 msgr2=0x7fa5b00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:38.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:38.838+0000 7fa5cedbe640 1 --2- 172.21.15.98:0/1461715200 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa5b00b65e0 0x7fa5b00b8af0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8031440 tx=0x7fa5b8009560 comp rx=0 tx=0).stop 2024-09-21T00:17:38.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:38.838+0000 7fa5cedbe640 1 -- 172.21.15.98:0/1461715200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c8103950 msgr2=0x7fa5c819ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:38.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:38.838+0000 7fa5cedbe640 1 --2- 172.21.15.98:0/1461715200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c8103950 0x7fa5c819ab50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc005fd0 tx=0x7fa5bc00dde0 comp rx=0 tx=0).stop 2024-09-21T00:17:38.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:38.838+0000 7fa5cedbe640 1 -- 172.21.15.98:0/1461715200 shutdown_connections 2024-09-21T00:17:38.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:38.838+0000 7fa5cedbe640 1 --2- 172.21.15.98:0/1461715200 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa5b00b65e0 0x7fa5b00b8af0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:38.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:38.838+0000 7fa5cedbe640 1 --2- 172.21.15.98:0/1461715200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c8107690 0x7fa5c819fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:38.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:38.838+0000 7fa5cedbe640 1 --2- 172.21.15.98:0/1461715200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c8103950 0x7fa5c819ab50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:38.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:38.838+0000 7fa5cedbe640 1 --2- 172.21.15.98:0/1461715200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5c8103000 0x7fa5c819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:38.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:38.838+0000 7fa5cedbe640 1 -- 172.21.15.98:0/1461715200 >> 172.21.15.98:0/1461715200 conn(0x7fa5c8079ea0 msgr2=0x7fa5c8106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:38.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:38.838+0000 7fa5cedbe640 1 -- 172.21.15.98:0/1461715200 shutdown_connections 2024-09-21T00:17:38.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:38.838+0000 7fa5cedbe640 1 -- 172.21.15.98:0/1461715200 wait complete. 2024-09-21T00:17:39.795 DEBUG:teuthology.orchestra.run.smithi098:osd.11> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T00:17:39.797 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T00:17:39.797 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T00:17:40.196 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:17:40.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:39 smithi089 ceph-mon[34481]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:40.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:39 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:40.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:39 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:40.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:39 smithi098 ceph-mon[34725]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:40.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:39 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:40.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:39 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:40.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:39 smithi027 ceph-mon[30745]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:40.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:39 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:40.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:39 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.849+0000 7f7deb800640 1 -- 172.21.15.27:0/210433038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7de410c010 msgr2=0x7f7de4112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.849+0000 7f7deb800640 1 --2- 172.21.15.27:0/210433038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7de410c010 0x7f7de4112840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009be0 tx=0x7f7ddc02a6d0 comp rx=0 tx=0).stop 2024-09-21T00:17:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.849+0000 7f7deb800640 1 -- 172.21.15.27:0/210433038 shutdown_connections 2024-09-21T00:17:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.849+0000 7f7deb800640 1 --2- 172.21.15.27:0/210433038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7de410c010 0x7f7de4112840 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.849+0000 7f7deb800640 1 --2- 172.21.15.27:0/210433038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7de410b6c0 0x7f7de410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.849+0000 7f7deb800640 1 --2- 172.21.15.27:0/210433038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7de4105d80 0x7f7de4106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.849+0000 7f7deb800640 1 -- 172.21.15.27:0/210433038 >> 172.21.15.27:0/210433038 conn(0x7f7de4101810 msgr2=0x7f7de4103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.849+0000 7f7deb800640 1 -- 172.21.15.27:0/210433038 shutdown_connections 2024-09-21T00:17:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.849+0000 7f7deb800640 1 -- 172.21.15.27:0/210433038 wait complete. 2024-09-21T00:17:40.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.850+0000 7f7deb800640 1 Processor -- start 2024-09-21T00:17:40.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.851+0000 7f7deb800640 1 -- start start 2024-09-21T00:17:40.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.851+0000 7f7deb800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7de4105d80 0x7f7de419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:40.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.851+0000 7f7deb800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7de410b6c0 0x7f7de419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:40.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.851+0000 7f7deb800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7de410c010 0x7f7de41a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:40.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.851+0000 7f7deb800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7de4115cd0 con 0x7f7de410c010 2024-09-21T00:17:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.851+0000 7f7deb800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7de4115b50 con 0x7f7de410b6c0 2024-09-21T00:17:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.851+0000 7f7deb800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7de4115e50 con 0x7f7de4105d80 2024-09-21T00:17:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.851+0000 7f7de8e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7de4105d80 0x7f7de419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.851+0000 7f7de8e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7de4105d80 0x7f7de419c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:36452/0 (socket says 172.21.15.27:36452) 2024-09-21T00:17:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.851+0000 7f7de8e00640 1 -- 172.21.15.27:0/1980874249 learned_addr learned my addr 172.21.15.27:0/1980874249 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:17:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.852+0000 7f7de9800640 1 --2- 172.21.15.27:0/1980874249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7de410c010 0x7f7de41a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.852+0000 7f7de3e00640 1 --2- 172.21.15.27:0/1980874249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7de410b6c0 0x7f7de419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.852+0000 7f7de8e00640 1 -- 172.21.15.27:0/1980874249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7de410b6c0 msgr2=0x7f7de419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.852+0000 7f7de8e00640 1 --2- 172.21.15.27:0/1980874249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7de410b6c0 0x7f7de419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.852+0000 7f7de8e00640 1 -- 172.21.15.27:0/1980874249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7de410c010 msgr2=0x7f7de41a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.852+0000 7f7de8e00640 1 --2- 172.21.15.27:0/1980874249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7de410c010 0x7f7de41a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.852+0000 7f7de8e00640 1 -- 172.21.15.27:0/1980874249 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de41a2630 con 0x7f7de4105d80 2024-09-21T00:17:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.852+0000 7f7de8e00640 1 --2- 172.21.15.27:0/1980874249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7de4105d80 0x7f7de419c6c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800d950 tx=0x7f7dd800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.853+0000 7f7de1600640 1 -- 172.21.15.27:0/1980874249 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8017030 con 0x7f7de4105d80 2024-09-21T00:17:40.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.853+0000 7f7de1600640 1 -- 172.21.15.27:0/1980874249 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd8004020 con 0x7f7de4105d80 2024-09-21T00:17:40.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.853+0000 7f7deb800640 1 -- 172.21.15.27:0/1980874249 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de41a2920 con 0x7f7de4105d80 2024-09-21T00:17:40.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.853+0000 7f7de1600640 1 -- 172.21.15.27:0/1980874249 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80047a0 con 0x7f7de4105d80 2024-09-21T00:17:40.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.853+0000 7f7deb800640 1 -- 172.21.15.27:0/1980874249 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de41a9080 con 0x7f7de4105d80 2024-09-21T00:17:40.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.854+0000 7f7deb800640 1 -- 172.21.15.27:0/1980874249 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de4106190 con 0x7f7de4105d80 2024-09-21T00:17:40.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.856+0000 7f7de1600640 1 -- 172.21.15.27:0/1980874249 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f7dd8009a30 con 0x7f7de4105d80 2024-09-21T00:17:40.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.858+0000 7f7de1600640 1 --2- 172.21.15.27:0/1980874249 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7dcc0b65e0 0x7f7dcc0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:40.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.858+0000 7f7de1600640 1 -- 172.21.15.27:0/1980874249 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7024+0+0 (secure 0 0 0) 0x7f7dd80e6900 con 0x7f7de4105d80 2024-09-21T00:17:40.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.858+0000 7f7de3e00640 1 --2- 172.21.15.27:0/1980874249 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7dcc0b65e0 0x7f7dcc0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:40.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.859+0000 7f7de3e00640 1 --2- 172.21.15.27:0/1980874249 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7dcc0b65e0 0x7f7dcc0b8af0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00096c0 tx=0x7f7dd0009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:40.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:40.859+0000 7f7de1600640 1 -- 172.21.15.27:0/1980874249 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7dd80ad6a0 con 0x7f7de4105d80 2024-09-21T00:17:41.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:41.063+0000 7f7deb800640 1 -- 172.21.15.27:0/1980874249 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f7de419d7f0 con 0x7f7de4105d80 2024-09-21T00:17:41.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:41.063+0000 7f7de1600640 1 -- 172.21.15.27:0/1980874249 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f7dd80b1a80 con 0x7f7de4105d80 2024-09-21T00:17:41.065 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:17:41.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:41.066+0000 7f7deb800640 1 -- 172.21.15.27:0/1980874249 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7dcc0b65e0 msgr2=0x7f7dcc0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:41.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:41.066+0000 7f7deb800640 1 --2- 172.21.15.27:0/1980874249 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7dcc0b65e0 0x7f7dcc0b8af0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00096c0 tx=0x7f7dd0009260 comp rx=0 tx=0).stop 2024-09-21T00:17:41.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:41.066+0000 7f7deb800640 1 -- 172.21.15.27:0/1980874249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7de4105d80 msgr2=0x7f7de419c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:41.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:41.066+0000 7f7deb800640 1 --2- 172.21.15.27:0/1980874249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7de4105d80 0x7f7de419c6c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800d950 tx=0x7f7dd800de10 comp rx=0 tx=0).stop 2024-09-21T00:17:41.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:41.066+0000 7f7deb800640 1 -- 172.21.15.27:0/1980874249 shutdown_connections 2024-09-21T00:17:41.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:41.066+0000 7f7deb800640 1 --2- 172.21.15.27:0/1980874249 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7dcc0b65e0 0x7f7dcc0b8af0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:41.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:41.066+0000 7f7deb800640 1 --2- 172.21.15.27:0/1980874249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7de410c010 0x7f7de41a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:41.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:41.066+0000 7f7deb800640 1 --2- 172.21.15.27:0/1980874249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7de410b6c0 0x7f7de419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:41.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:41.066+0000 7f7deb800640 1 --2- 172.21.15.27:0/1980874249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7de4105d80 0x7f7de419c6c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:41.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:41.066+0000 7f7deb800640 1 -- 172.21.15.27:0/1980874249 >> 172.21.15.27:0/1980874249 conn(0x7f7de4101810 msgr2=0x7f7de4108c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:41.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:41.066+0000 7f7deb800640 1 -- 172.21.15.27:0/1980874249 shutdown_connections 2024-09-21T00:17:41.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:41.066+0000 7f7deb800640 1 -- 172.21.15.27:0/1980874249 wait complete. 2024-09-21T00:17:41.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:40 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:41.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:40 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:41.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:40 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:41.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:40 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:41.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:40 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:41.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:40 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:41.846 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:17:41 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:17:41.726+0000 7fcbb8711640 -1 osd.11 0 log_to_monitors true 2024-09-21T00:17:41.984 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726877839,"num_in_osds":12,"osd_in_since":1726877841,"num_remapped_pgs":0} 2024-09-21T00:17:42.146 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:41 smithi098 ceph-mon[34725]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:42.146 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:41 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1980874249' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T00:17:42.146 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:41 smithi098 ceph-mon[34725]: from='osd.11 [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T00:17:42.146 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:41 smithi098 ceph-mon[34725]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T00:17:42.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:41 smithi089 ceph-mon[34481]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:42.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:41 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1980874249' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T00:17:42.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:41 smithi089 ceph-mon[34481]: from='osd.11 [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T00:17:42.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:41 smithi089 ceph-mon[34481]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T00:17:42.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:41 smithi027 ceph-mon[30745]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:42.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:41 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1980874249' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T00:17:42.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:41 smithi027 ceph-mon[30745]: from='osd.11 [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T00:17:42.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:41 smithi027 ceph-mon[30745]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T00:17:42.985 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T00:17:43.117 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:17:42 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:17:42.974+0000 7fcbb4693640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T00:17:43.118 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:17:42 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:17:42.993+0000 7fcbb0491640 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:17:43.118 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:42 smithi098 ceph-mon[34725]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T00:17:43.118 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:42 smithi098 ceph-mon[34725]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T00:17:43.118 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:42 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:43.118 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:42 smithi098 ceph-mon[34725]: from='osd.11 [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:17:43.118 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:42 smithi098 ceph-mon[34725]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:17:43.118 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:42 smithi098 ceph-mon[34725]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T00:17:43.118 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:42 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:43.118 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:42 smithi098 ceph-mon[34725]: osdmap e70: 12 total, 11 up, 12 in 2024-09-21T00:17:43.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:42 smithi027 ceph-mon[30745]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T00:17:43.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:42 smithi027 ceph-mon[30745]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T00:17:43.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:42 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:43.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:42 smithi027 ceph-mon[30745]: from='osd.11 [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:17:43.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:42 smithi027 ceph-mon[30745]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:17:43.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:42 smithi027 ceph-mon[30745]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T00:17:43.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:42 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:43.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:42 smithi027 ceph-mon[30745]: osdmap e70: 12 total, 11 up, 12 in 2024-09-21T00:17:43.372 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:17:43.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:42 smithi089 ceph-mon[34481]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T00:17:43.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:42 smithi089 ceph-mon[34481]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T00:17:43.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:42 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:43.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:42 smithi089 ceph-mon[34481]: from='osd.11 [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:17:43.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:42 smithi089 ceph-mon[34481]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T00:17:43.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:42 smithi089 ceph-mon[34481]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T00:17:43.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:42 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:43.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:42 smithi089 ceph-mon[34481]: osdmap e70: 12 total, 11 up, 12 in 2024-09-21T00:17:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.157+0000 7f1aa3200640 1 -- 172.21.15.27:0/3292860742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1aa4114280 msgr2=0x7f1aa4116750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.157+0000 7f1aa3200640 1 --2- 172.21.15.27:0/3292860742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1aa4114280 0x7f1aa4116750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009ca0 tx=0x7f1a9c02a8c0 comp rx=0 tx=0).stop 2024-09-21T00:17:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.158+0000 7f1aa3200640 1 -- 172.21.15.27:0/3292860742 shutdown_connections 2024-09-21T00:17:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.158+0000 7f1aa3200640 1 --2- 172.21.15.27:0/3292860742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1aa4114280 0x7f1aa4116750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:44.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.158+0000 7f1aa3200640 1 --2- 172.21.15.27:0/3292860742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1aa4075dd0 0x7f1aa40761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:44.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.158+0000 7f1aa3200640 1 --2- 172.21.15.27:0/3292860742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1aa4077380 0x7f1aa40757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:44.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.158+0000 7f1aa3200640 1 -- 172.21.15.27:0/3292860742 >> 172.21.15.27:0/3292860742 conn(0x7f1aa40ff440 msgr2=0x7f1aa4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:44.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.158+0000 7f1aa3200640 1 -- 172.21.15.27:0/3292860742 shutdown_connections 2024-09-21T00:17:44.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.158+0000 7f1aa3200640 1 -- 172.21.15.27:0/3292860742 wait complete. 2024-09-21T00:17:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.159+0000 7f1aa3200640 1 Processor -- start 2024-09-21T00:17:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.159+0000 7f1aa3200640 1 -- start start 2024-09-21T00:17:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa3200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1aa4075dd0 0x7f1aa41a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa3200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1aa4077380 0x7f1aa41a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa3200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1aa4114280 0x7f1aa41a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa3200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4119f10 con 0x7f1aa4075dd0 2024-09-21T00:17:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa3200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4119d90 con 0x7f1aa4114280 2024-09-21T00:17:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa3200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa411a090 con 0x7f1aa4077380 2024-09-21T00:17:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1aa4075dd0 0x7f1aa41a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1aa4075dd0 0x7f1aa41a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60220/0 (socket says 172.21.15.27:60220) 2024-09-21T00:17:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa1e00640 1 -- 172.21.15.27:0/2362283539 learned_addr learned my addr 172.21.15.27:0/2362283539 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:17:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa2800640 1 --2- 172.21.15.27:0/2362283539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1aa4114280 0x7f1aa41a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa1400640 1 --2- 172.21.15.27:0/2362283539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1aa4077380 0x7f1aa41a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa1e00640 1 -- 172.21.15.27:0/2362283539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1aa4077380 msgr2=0x7f1aa41a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa1e00640 1 --2- 172.21.15.27:0/2362283539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1aa4077380 0x7f1aa41a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa1e00640 1 -- 172.21.15.27:0/2362283539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1aa4114280 msgr2=0x7f1aa41a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa1e00640 1 --2- 172.21.15.27:0/2362283539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1aa4114280 0x7f1aa41a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.160+0000 7f1aa1e00640 1 -- 172.21.15.27:0/2362283539 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa41a6870 con 0x7f1aa4075dd0 2024-09-21T00:17:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.161+0000 7f1aa1e00640 1 --2- 172.21.15.27:0/2362283539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1aa4075dd0 0x7f1aa41a0900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400d950 tx=0x7f1a9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.161+0000 7f1a9aa00640 1 -- 172.21.15.27:0/2362283539 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a94017070 con 0x7f1aa4075dd0 2024-09-21T00:17:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.161+0000 7f1aa3200640 1 -- 172.21.15.27:0/2362283539 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa41a6b60 con 0x7f1aa4075dd0 2024-09-21T00:17:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.161+0000 7f1a9aa00640 1 -- 172.21.15.27:0/2362283539 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a94009c30 con 0x7f1aa4075dd0 2024-09-21T00:17:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.161+0000 7f1a9aa00640 1 -- 172.21.15.27:0/2362283539 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400f620 con 0x7f1aa4075dd0 2024-09-21T00:17:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.161+0000 7f1aa3200640 1 -- 172.21.15.27:0/2362283539 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa41ad2c0 con 0x7f1aa4075dd0 2024-09-21T00:17:44.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.162+0000 7f1aa3200640 1 -- 172.21.15.27:0/2362283539 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa40761e0 con 0x7f1aa4075dd0 2024-09-21T00:17:44.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.165+0000 7f1a9aa00640 1 -- 172.21.15.27:0/2362283539 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f1a94005030 con 0x7f1aa4075dd0 2024-09-21T00:17:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.166+0000 7f1a9aa00640 1 --2- 172.21.15.27:0/2362283539 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a880b6560 0x7f1a880b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:44.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.167+0000 7f1a9aa00640 1 -- 172.21.15.27:0/2362283539 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f1a940e6ae0 con 0x7f1aa4075dd0 2024-09-21T00:17:44.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.167+0000 7f1aa1400640 1 --2- 172.21.15.27:0/2362283539 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a880b6560 0x7f1a880b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:44.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.167+0000 7f1aa1400640 1 --2- 172.21.15.27:0/2362283539 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a880b6560 0x7f1a880b8a70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1a900095f0 tx=0x7f1a90005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:44.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.169+0000 7f1a9aa00640 1 -- 172.21.15.27:0/2362283539 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1a94009de0 con 0x7f1aa4075dd0 2024-09-21T00:17:44.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:44.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:44.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:44.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:44.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:17:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:17:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:17:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:44.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:44.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:44.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:44.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:44.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:44.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:44.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:44.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:17:44.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:44.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:17:44.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:44.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:17:44.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:44.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:44.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.370+0000 7f1aa3200640 1 -- 172.21.15.27:0/2362283539 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1aa41a1a30 con 0x7f1aa4075dd0 2024-09-21T00:17:44.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.371+0000 7f1a9aa00640 1 -- 172.21.15.27:0/2362283539 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f1a940b1a80 con 0x7f1aa4075dd0 2024-09-21T00:17:44.372 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:17:44.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.373+0000 7f1aa3200640 1 -- 172.21.15.27:0/2362283539 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a880b6560 msgr2=0x7f1a880b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:44.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.373+0000 7f1aa3200640 1 --2- 172.21.15.27:0/2362283539 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a880b6560 0x7f1a880b8a70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1a900095f0 tx=0x7f1a90005f90 comp rx=0 tx=0).stop 2024-09-21T00:17:44.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.373+0000 7f1aa3200640 1 -- 172.21.15.27:0/2362283539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1aa4075dd0 msgr2=0x7f1aa41a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:44.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.373+0000 7f1aa3200640 1 --2- 172.21.15.27:0/2362283539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1aa4075dd0 0x7f1aa41a0900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400d950 tx=0x7f1a9400de10 comp rx=0 tx=0).stop 2024-09-21T00:17:44.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.373+0000 7f1aa3200640 1 -- 172.21.15.27:0/2362283539 shutdown_connections 2024-09-21T00:17:44.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.373+0000 7f1aa3200640 1 --2- 172.21.15.27:0/2362283539 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a880b6560 0x7f1a880b8a70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:44.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.373+0000 7f1aa3200640 1 --2- 172.21.15.27:0/2362283539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1aa4114280 0x7f1aa41a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:44.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.373+0000 7f1aa3200640 1 --2- 172.21.15.27:0/2362283539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1aa4077380 0x7f1aa41a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:44.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.373+0000 7f1aa3200640 1 --2- 172.21.15.27:0/2362283539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1aa4075dd0 0x7f1aa41a0900 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:44.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.373+0000 7f1aa3200640 1 -- 172.21.15.27:0/2362283539 >> 172.21.15.27:0/2362283539 conn(0x7f1aa40ff440 msgr2=0x7f1aa4110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:44.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.374+0000 7f1aa3200640 1 -- 172.21.15.27:0/2362283539 shutdown_connections 2024-09-21T00:17:44.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:44.374+0000 7f1aa3200640 1 -- 172.21.15.27:0/2362283539 wait complete. 2024-09-21T00:17:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T00:17:44.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:44.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:44.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:17:44.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:44.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:17:44.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:44.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:17:44.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:44.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:45.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:44 smithi098 ceph-mon[34725]: purged_snaps scrub starts 2024-09-21T00:17:45.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:44 smithi098 ceph-mon[34725]: purged_snaps scrub ok 2024-09-21T00:17:45.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:44 smithi098 ceph-mon[34725]: Detected new or changed devices on smithi098 2024-09-21T00:17:45.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:44 smithi098 ceph-mon[34725]: Adjusting osd_memory_target on smithi098 to 4272M 2024-09-21T00:17:45.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:44 smithi098 ceph-mon[34725]: osd.11 [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064] boot 2024-09-21T00:17:45.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:44 smithi098 ceph-mon[34725]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T00:17:45.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:44 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:45.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:44 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2362283539' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T00:17:45.270 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:44 smithi027 ceph-mon[30745]: purged_snaps scrub starts 2024-09-21T00:17:45.270 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:44 smithi027 ceph-mon[30745]: purged_snaps scrub ok 2024-09-21T00:17:45.270 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:44 smithi027 ceph-mon[30745]: Detected new or changed devices on smithi098 2024-09-21T00:17:45.270 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:44 smithi027 ceph-mon[30745]: Adjusting osd_memory_target on smithi098 to 4272M 2024-09-21T00:17:45.271 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:44 smithi027 ceph-mon[30745]: osd.11 [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064] boot 2024-09-21T00:17:45.271 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:44 smithi027 ceph-mon[30745]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T00:17:45.271 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:44 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:45.271 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:44 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2362283539' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T00:17:45.315 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":71,"num_osds":12,"num_up_osds":12,"osd_up_since":1726877863,"num_in_osds":12,"osd_in_since":1726877841,"num_remapped_pgs":0} 2024-09-21T00:17:45.316 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T00:17:45.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:44 smithi089 ceph-mon[34481]: purged_snaps scrub starts 2024-09-21T00:17:45.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:44 smithi089 ceph-mon[34481]: purged_snaps scrub ok 2024-09-21T00:17:45.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:44 smithi089 ceph-mon[34481]: Detected new or changed devices on smithi098 2024-09-21T00:17:45.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:44 smithi089 ceph-mon[34481]: Adjusting osd_memory_target on smithi098 to 4272M 2024-09-21T00:17:45.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:44 smithi089 ceph-mon[34481]: osd.11 [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064] boot 2024-09-21T00:17:45.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:44 smithi089 ceph-mon[34481]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T00:17:45.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:44 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T00:17:45.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:44 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2362283539' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T00:17:45.695 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:17:46.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:45 smithi027 ceph-mon[30745]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:46.115 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:45 smithi027 ceph-mon[30745]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T00:17:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.298+0000 7fe480a00640 1 -- 172.21.15.27:0/223403876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47c109df0 msgr2=0x7fe47c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.298+0000 7fe480a00640 1 --2- 172.21.15.27:0/223403876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47c109df0 0x7fe47c110620 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe474009ca0 tx=0x7fe47402a8c0 comp rx=0 tx=0).stop 2024-09-21T00:17:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.298+0000 7fe480a00640 1 -- 172.21.15.27:0/223403876 shutdown_connections 2024-09-21T00:17:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.298+0000 7fe480a00640 1 --2- 172.21.15.27:0/223403876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47c109df0 0x7fe47c110620 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.298+0000 7fe480a00640 1 --2- 172.21.15.27:0/223403876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe47c1094a0 0x7fe47c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.298+0000 7fe480a00640 1 --2- 172.21.15.27:0/223403876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe47c103b60 0x7fe47c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.298+0000 7fe480a00640 1 -- 172.21.15.27:0/223403876 >> 172.21.15.27:0/223403876 conn(0x7fe47c07b850 msgr2=0x7fe47c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.299+0000 7fe480a00640 1 -- 172.21.15.27:0/223403876 shutdown_connections 2024-09-21T00:17:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.299+0000 7fe480a00640 1 -- 172.21.15.27:0/223403876 wait complete. 2024-09-21T00:17:46.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.300+0000 7fe480a00640 1 Processor -- start 2024-09-21T00:17:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.300+0000 7fe480a00640 1 -- start start 2024-09-21T00:17:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.300+0000 7fe480a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47c103b60 0x7fe47c19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.300+0000 7fe480a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe47c1094a0 0x7fe47c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.300+0000 7fe480a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe47c109df0 0x7fe47c1a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.300+0000 7fe480a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c113b00 con 0x7fe47c103b60 2024-09-21T00:17:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.300+0000 7fe480a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c113980 con 0x7fe47c1094a0 2024-09-21T00:17:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.300+0000 7fe480a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c113c80 con 0x7fe47c109df0 2024-09-21T00:17:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.301+0000 7fe479e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47c103b60 0x7fe47c19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.301+0000 7fe479e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47c103b60 0x7fe47c19c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60238/0 (socket says 172.21.15.27:60238) 2024-09-21T00:17:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.301+0000 7fe479e00640 1 -- 172.21.15.27:0/415298884 learned_addr learned my addr 172.21.15.27:0/415298884 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:17:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.301+0000 7fe47a800640 1 --2- 172.21.15.27:0/415298884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe47c109df0 0x7fe47c1a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.301+0000 7fe479400640 1 --2- 172.21.15.27:0/415298884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe47c1094a0 0x7fe47c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:46.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.301+0000 7fe479e00640 1 -- 172.21.15.27:0/415298884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe47c109df0 msgr2=0x7fe47c1a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:46.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.301+0000 7fe479e00640 1 --2- 172.21.15.27:0/415298884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe47c109df0 0x7fe47c1a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:46.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.301+0000 7fe479e00640 1 -- 172.21.15.27:0/415298884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe47c1094a0 msgr2=0x7fe47c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:46.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.301+0000 7fe479e00640 1 --2- 172.21.15.27:0/415298884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe47c1094a0 0x7fe47c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:46.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.301+0000 7fe479e00640 1 -- 172.21.15.27:0/415298884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47c1a26a0 con 0x7fe47c103b60 2024-09-21T00:17:46.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.301+0000 7fe479400640 1 --2- 172.21.15.27:0/415298884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe47c1094a0 0x7fe47c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:17:46.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.301+0000 7fe479e00640 1 --2- 172.21.15.27:0/415298884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47c103b60 0x7fe47c19c730 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00e9f0 tx=0x7fe46c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:46.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.302+0000 7fe472a00640 1 -- 172.21.15.27:0/415298884 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46c00cd80 con 0x7fe47c103b60 2024-09-21T00:17:46.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.302+0000 7fe472a00640 1 -- 172.21.15.27:0/415298884 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe46c0054d0 con 0x7fe47c103b60 2024-09-21T00:17:46.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.302+0000 7fe480a00640 1 -- 172.21.15.27:0/415298884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe47c1a2990 con 0x7fe47c103b60 2024-09-21T00:17:46.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.302+0000 7fe472a00640 1 -- 172.21.15.27:0/415298884 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46c010690 con 0x7fe47c103b60 2024-09-21T00:17:46.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.302+0000 7fe480a00640 1 -- 172.21.15.27:0/415298884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe47c1a90f0 con 0x7fe47c103b60 2024-09-21T00:17:46.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.303+0000 7fe480a00640 1 -- 172.21.15.27:0/415298884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe47c103f70 con 0x7fe47c103b60 2024-09-21T00:17:46.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.306+0000 7fe472a00640 1 -- 172.21.15.27:0/415298884 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fe46c004e30 con 0x7fe47c103b60 2024-09-21T00:17:46.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.307+0000 7fe472a00640 1 --2- 172.21.15.27:0/415298884 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe4600b65e0 0x7fe4600b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:46.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.307+0000 7fe472a00640 1 -- 172.21.15.27:0/415298884 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fe46c0e7620 con 0x7fe47c103b60 2024-09-21T00:17:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.307+0000 7fe479400640 1 --2- 172.21.15.27:0/415298884 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe4600b65e0 0x7fe4600b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.308+0000 7fe479400640 1 --2- 172.21.15.27:0/415298884 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe4600b65e0 0x7fe4600b8af0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe47c106370 tx=0x7fe468005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.309+0000 7fe472a00640 1 -- 172.21.15.27:0/415298884 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe46c056710 con 0x7fe47c103b60 2024-09-21T00:17:46.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:45 smithi089 ceph-mon[34481]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:46.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:45 smithi089 ceph-mon[34481]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T00:17:46.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:45 smithi098 ceph-mon[34725]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-21T00:17:46.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:45 smithi098 ceph-mon[34725]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T00:17:46.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.509+0000 7fe480a00640 1 -- 172.21.15.27:0/415298884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe47c19d860 con 0x7fe47c103b60 2024-09-21T00:17:46.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.510+0000 7fe472a00640 1 -- 172.21.15.27:0/415298884 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18683 (secure 0 0 0) 0x7fe46c0ae1e0 con 0x7fe47c103b60 2024-09-21T00:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":72,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","created":"2024-09-21T00:09:20.149938+0000","modified":"2024-09-21T00:17:44.987278+0000","last_up_change":"2024-09-21T00:17:43.978785+0000","last_in_change":"2024-09-21T00:17:21.325177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:13:32.718543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63ccc094-62d5-446f-844d-1c60d41fda1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6803","nonce":354370090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6805","nonce":354370090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6809","nonce":354370090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6807","nonce":354370090}]},"public_addr":"172.21.15.27:6803/354370090","cluster_addr":"172.21.15.27:6805/354370090","heartbeat_back_addr":"172.21.15.27:6809/354370090","heartbeat_front_addr":"172.21.15.27:6807/354370090","state":["exists","up"]},{"osd":1,"uuid":"e59a8ed4-3a37-4477-a60a-4abc33d49b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6801","nonce":922528621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6803","nonce":922528621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6807","nonce":922528621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6805","nonce":922528621}]},"public_addr":"172.21.15.89:6801/922528621","cluster_addr":"172.21.15.89:6803/922528621","heartbeat_back_addr":"172.21.15.89:6807/922528621","heartbeat_front_addr":"172.21.15.89:6805/922528621","state":["exists","up"]},{"osd":2,"uuid":"6058a875-d5b2-4459-9407-2fd190cb34a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6801","nonce":257636259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6803","nonce":257636259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6807","nonce":257636259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6805","nonce":257636259}]},"public_addr":"172.21.15.98:6801/257636259","cluster_addr":"172.21.15.98:6803/257636259","heartbeat_back_addr":"172.21.15.98:6807/257636259","heartbeat_front_addr":"172.21.15.98:6805/257636259","state":["exists","up"]},{"osd":3,"uuid":"dfcdaede-e51c-4985-92af-3e296435a36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6811","nonce":821088824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6813","nonce":821088824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6817","nonce":821088824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6815","nonce":821088824}]},"public_addr":"172.21.15.27:6811/821088824","cluster_addr":"172.21.15.27:6813/821088824","heartbeat_back_addr":"172.21.15.27:6817/821088824","heartbeat_front_addr":"172.21.15.27:6815/821088824","state":["exists","up"]},{"osd":4,"uuid":"7c4af037-8ab6-47c0-b62b-76bd775b3491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6809","nonce":4001038690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6811","nonce":4001038690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6815","nonce":4001038690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6813","nonce":4001038690}]},"public_addr":"172.21.15.89:6809/4001038690","cluster_addr":"172.21.15.89:6811/4001038690","heartbeat_back_addr":"172.21.15.89:6815/4001038690","heartbeat_front_addr":"172.21.15.89:6813/4001038690","state":["exists","up"]},{"osd":5,"uuid":"e996ed0d-88ad-4aa7-891f-dfade4e09cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6809","nonce":3029903562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6811","nonce":3029903562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6815","nonce":3029903562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6813","nonce":3029903562}]},"public_addr":"172.21.15.98:6809/3029903562","cluster_addr":"172.21.15.98:6811/3029903562","heartbeat_back_addr":"172.21.15.98:6815/3029903562","heartbeat_front_addr":"172.21.15.98:6813/3029903562","state":["exists","up"]},{"osd":6,"uuid":"cf080497-57c2-42d0-8c8f-1eed0110702d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6819","nonce":1286808857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6821","nonce":1286808857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6825","nonce":1286808857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6823","nonce":1286808857}]},"public_addr":"172.21.15.27:6819/1286808857","cluster_addr":"172.21.15.27:6821/1286808857","heartbeat_back_addr":"172.21.15.27:6825/1286808857","heartbeat_front_addr":"172.21.15.27:6823/1286808857","state":["exists","up"]},{"osd":7,"uuid":"ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6817","nonce":2479349546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6819","nonce":2479349546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6823","nonce":2479349546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6821","nonce":2479349546}]},"public_addr":"172.21.15.89:6817/2479349546","cluster_addr":"172.21.15.89:6819/2479349546","heartbeat_back_addr":"172.21.15.89:6823/2479349546","heartbeat_front_addr":"172.21.15.89:6821/2479349546","state":["exists","up"]},{"osd":8,"uuid":"0b558458-fc2d-4ed1-b72a-e2ea8cf811b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6817","nonce":2830381485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6819","nonce":2830381485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6823","nonce":2830381485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6821","nonce":2830381485}]},"public_addr":"172.21.15.98:6817/2830381485","cluster_addr":"172.21.15.98:6819/2830381485","heartbeat_back_addr":"172.21.15.98:6823/2830381485","heartbeat_front_addr":"172.21.15.98:6821/2830381485","state":["exists","up"]},{"osd":9,"uuid":"1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6827","nonce":3033977113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6829","nonce":3033977113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6833","nonce":3033977113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6831","nonce":3033977113}]},"public_addr":"172.21.15.27:6827/3033977113","cluster_addr":"172.21.15.27:6829/3033977113","heartbeat_back_addr":"172.21.15.27:6833/3033977113","heartbeat_front_addr":"172.21.15.27:6831/3033977113","state":["exists","up"]},{"osd":10,"uuid":"324dd6ce-68fe-4c76-959b-1c9fb0ae92a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6825","nonce":2370332225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6827","nonce":2370332225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6831","nonce":2370332225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6829","nonce":2370332225}]},"public_addr":"172.21.15.89:6825/2370332225","cluster_addr":"172.21.15.89:6827/2370332225","heartbeat_back_addr":"172.21.15.89:6831/2370332225","heartbeat_front_addr":"172.21.15.89:6829/2370332225","state":["exists","up"]},{"osd":11,"uuid":"5e34686e-350a-491e-b9d7-59174ea90944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6825","nonce":2795999064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6827","nonce":2795999064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6831","nonce":2795999064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6829","nonce":2795999064}]},"public_addr":"172.21.15.98:6825/2795999064","cluster_addr":"172.21.15.98:6827/2795999064","heartbeat_back_addr":"172.21.15.98:6831/2795999064","heartbeat_front_addr":"172.21.15.98:6829/2795999064","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:12:39.841958+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:06.684417+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:31.041378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:01.612108+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:28.592326+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:54.303714+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:25.523475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:52.523483+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:19.430503+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:50.599120+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:17.520452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:42.725550+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/502883064":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/831048090":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/3880378905":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/793991452":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/2077956479":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6800/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6801/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/1098432608":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6800/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/528019907":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/2173601508":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:6800/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/764776233":"2024-09-22T00:11:24.231101+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:17:46.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.512+0000 7fe480a00640 1 -- 172.21.15.27:0/415298884 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe4600b65e0 msgr2=0x7fe4600b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:46.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.512+0000 7fe480a00640 1 --2- 172.21.15.27:0/415298884 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe4600b65e0 0x7fe4600b8af0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe47c106370 tx=0x7fe468005f90 comp rx=0 tx=0).stop 2024-09-21T00:17:46.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.512+0000 7fe480a00640 1 -- 172.21.15.27:0/415298884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47c103b60 msgr2=0x7fe47c19c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:46.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.512+0000 7fe480a00640 1 --2- 172.21.15.27:0/415298884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47c103b60 0x7fe47c19c730 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00e9f0 tx=0x7fe46c00edf0 comp rx=0 tx=0).stop 2024-09-21T00:17:46.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.512+0000 7fe480a00640 1 -- 172.21.15.27:0/415298884 shutdown_connections 2024-09-21T00:17:46.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.512+0000 7fe480a00640 1 --2- 172.21.15.27:0/415298884 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe4600b65e0 0x7fe4600b8af0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:46.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.512+0000 7fe480a00640 1 --2- 172.21.15.27:0/415298884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe47c109df0 0x7fe47c1a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:46.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.512+0000 7fe480a00640 1 --2- 172.21.15.27:0/415298884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe47c1094a0 0x7fe47c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:46.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.512+0000 7fe480a00640 1 --2- 172.21.15.27:0/415298884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47c103b60 0x7fe47c19c730 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:46.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.512+0000 7fe480a00640 1 -- 172.21.15.27:0/415298884 >> 172.21.15.27:0/415298884 conn(0x7fe47c07b850 msgr2=0x7fe47c107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:46.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.513+0000 7fe480a00640 1 -- 172.21.15.27:0/415298884 shutdown_connections 2024-09-21T00:17:46.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:46.513+0000 7fe480a00640 1 -- 172.21.15.27:0/415298884 wait complete. 2024-09-21T00:17:47.228 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:47 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/415298884' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:17:47.266 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T00:13:32.718543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T00:17:47.267 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T00:17:47.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:47 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/415298884' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:17:47.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:47 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/415298884' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:17:47.644 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:17:48.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:48 smithi027 ceph-mon[30745]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.293+0000 7fc33ac00640 1 -- 172.21.15.27:0/4248689153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33410b500 msgr2=0x7fc33410b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:48.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.293+0000 7fc33ac00640 1 --2- 172.21.15.27:0/4248689153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33410b500 0x7fc33410b910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc320009900 tx=0x7fc32002a410 comp rx=0 tx=0).stop 2024-09-21T00:17:48.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.293+0000 7fc33ac00640 1 -- 172.21.15.27:0/4248689153 shutdown_connections 2024-09-21T00:17:48.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.293+0000 7fc33ac00640 1 --2- 172.21.15.27:0/4248689153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc33410be50 0x7fc334112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:48.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.293+0000 7fc33ac00640 1 --2- 172.21.15.27:0/4248689153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33410b500 0x7fc33410b910 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:48.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.293+0000 7fc33ac00640 1 --2- 172.21.15.27:0/4248689153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc334105bc0 0x7fc334105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:48.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.293+0000 7fc33ac00640 1 -- 172.21.15.27:0/4248689153 >> 172.21.15.27:0/4248689153 conn(0x7fc334101670 msgr2=0x7fc334103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:48.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.293+0000 7fc33ac00640 1 -- 172.21.15.27:0/4248689153 shutdown_connections 2024-09-21T00:17:48.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.293+0000 7fc33ac00640 1 -- 172.21.15.27:0/4248689153 wait complete. 2024-09-21T00:17:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.294+0000 7fc33ac00640 1 Processor -- start 2024-09-21T00:17:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.294+0000 7fc33ac00640 1 -- start start 2024-09-21T00:17:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.295+0000 7fc33ac00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc334105bc0 0x7fc33419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.295+0000 7fc33ac00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc33410b500 0x7fc33419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.295+0000 7fc33ac00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33410be50 0x7fc3341a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.295+0000 7fc33ac00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc334115b10 con 0x7fc33410be50 2024-09-21T00:17:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.295+0000 7fc33ac00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc334115990 con 0x7fc334105bc0 2024-09-21T00:17:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.295+0000 7fc33ac00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc334115c90 con 0x7fc33410b500 2024-09-21T00:17:48.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.295+0000 7fc338e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc33410b500 0x7fc33419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.295+0000 7fc339800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc334105bc0 0x7fc33419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.295+0000 7fc33a200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33410be50 0x7fc3341a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.295+0000 7fc338e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc33410b500 0x7fc33419ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:35390/0 (socket says 172.21.15.27:35390) 2024-09-21T00:17:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.295+0000 7fc338e00640 1 -- 172.21.15.27:0/2013241221 learned_addr learned my addr 172.21.15.27:0/2013241221 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:17:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.295+0000 7fc339800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc334105bc0 0x7fc33419c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39862/0 (socket says 172.21.15.27:39862) 2024-09-21T00:17:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.296+0000 7fc33a200640 1 -- 172.21.15.27:0/2013241221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc33410b500 msgr2=0x7fc33419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.296+0000 7fc33a200640 1 --2- 172.21.15.27:0/2013241221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc33410b500 0x7fc33419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.296+0000 7fc33a200640 1 -- 172.21.15.27:0/2013241221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc334105bc0 msgr2=0x7fc33419c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.296+0000 7fc33a200640 1 --2- 172.21.15.27:0/2013241221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc334105bc0 0x7fc33419c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.296+0000 7fc33a200640 1 -- 172.21.15.27:0/2013241221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3341a2450 con 0x7fc33410be50 2024-09-21T00:17:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.296+0000 7fc33a200640 1 --2- 172.21.15.27:0/2013241221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33410be50 0x7fc3341a1d20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009fd0 tx=0x7fc32c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.296+0000 7fc332000640 1 -- 172.21.15.27:0/2013241221 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c00ce20 con 0x7fc33410be50 2024-09-21T00:17:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.296+0000 7fc332000640 1 -- 172.21.15.27:0/2013241221 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc32c005020 con 0x7fc33410be50 2024-09-21T00:17:48.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.296+0000 7fc332000640 1 -- 172.21.15.27:0/2013241221 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c010690 con 0x7fc33410be50 2024-09-21T00:17:48.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.296+0000 7fc33ac00640 1 -- 172.21.15.27:0/2013241221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3341a2740 con 0x7fc33410be50 2024-09-21T00:17:48.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.297+0000 7fc33ac00640 1 -- 172.21.15.27:0/2013241221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3341a8ea0 con 0x7fc33410be50 2024-09-21T00:17:48.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.298+0000 7fc33ac00640 1 -- 172.21.15.27:0/2013241221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc334105fd0 con 0x7fc33410be50 2024-09-21T00:17:48.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.300+0000 7fc332000640 1 -- 172.21.15.27:0/2013241221 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fc32c0054d0 con 0x7fc33410be50 2024-09-21T00:17:48.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.302+0000 7fc332000640 1 --2- 172.21.15.27:0/2013241221 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc31c0b65e0 0x7fc31c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:48.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.302+0000 7fc332000640 1 -- 172.21.15.27:0/2013241221 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fc32c0e7620 con 0x7fc33410be50 2024-09-21T00:17:48.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.302+0000 7fc339800640 1 --2- 172.21.15.27:0/2013241221 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc31c0b65e0 0x7fc31c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:48.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.303+0000 7fc339800640 1 --2- 172.21.15.27:0/2013241221 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc31c0b65e0 0x7fc31c0b8af0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc3280095f0 tx=0x7fc328005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:48.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.306+0000 7fc332000640 1 -- 172.21.15.27:0/2013241221 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc32c0ae280 con 0x7fc33410be50 2024-09-21T00:17:48.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:48 smithi089 ceph-mon[34481]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:48.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:48 smithi098 ceph-mon[34725]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:48.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.504+0000 7fc33ac00640 1 -- 172.21.15.27:0/2013241221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc33410dca0 con 0x7fc33410be50 2024-09-21T00:17:48.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.504+0000 7fc332000640 1 -- 172.21.15.27:0/2013241221 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7fc32c0b2660 con 0x7fc33410be50 2024-09-21T00:17:48.505 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-21T00:17:48.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.508+0000 7fc33ac00640 1 -- 172.21.15.27:0/2013241221 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc31c0b65e0 msgr2=0x7fc31c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.508+0000 7fc33ac00640 1 --2- 172.21.15.27:0/2013241221 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc31c0b65e0 0x7fc31c0b8af0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc3280095f0 tx=0x7fc328005f90 comp rx=0 tx=0).stop 2024-09-21T00:17:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.508+0000 7fc33ac00640 1 -- 172.21.15.27:0/2013241221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33410be50 msgr2=0x7fc3341a1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.508+0000 7fc33ac00640 1 --2- 172.21.15.27:0/2013241221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33410be50 0x7fc3341a1d20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009fd0 tx=0x7fc32c00ef10 comp rx=0 tx=0).stop 2024-09-21T00:17:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.508+0000 7fc33ac00640 1 -- 172.21.15.27:0/2013241221 shutdown_connections 2024-09-21T00:17:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.508+0000 7fc33ac00640 1 --2- 172.21.15.27:0/2013241221 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc31c0b65e0 0x7fc31c0b8af0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.508+0000 7fc33ac00640 1 --2- 172.21.15.27:0/2013241221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33410be50 0x7fc3341a1d20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.508+0000 7fc33ac00640 1 --2- 172.21.15.27:0/2013241221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc33410b500 0x7fc33419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.508+0000 7fc33ac00640 1 --2- 172.21.15.27:0/2013241221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc334105bc0 0x7fc33419c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.508+0000 7fc33ac00640 1 -- 172.21.15.27:0/2013241221 >> 172.21.15.27:0/2013241221 conn(0x7fc334101670 msgr2=0x7fc334103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.509+0000 7fc33ac00640 1 -- 172.21.15.27:0/2013241221 shutdown_connections 2024-09-21T00:17:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:48.509+0000 7fc33ac00640 1 -- 172.21.15.27:0/2013241221 wait complete. 2024-09-21T00:17:49.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:49 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2013241221' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T00:17:49.359 INFO:tasks.cephadm:Adding mds.a on smithi027 2024-09-21T00:17:49.359 INFO:tasks.cephadm:Adding mds.d on smithi027 2024-09-21T00:17:49.359 INFO:tasks.cephadm:Adding mds.g on smithi027 2024-09-21T00:17:49.359 INFO:tasks.cephadm:Adding mds.j on smithi027 2024-09-21T00:17:49.359 INFO:tasks.cephadm:Adding mds.b on smithi089 2024-09-21T00:17:49.359 INFO:tasks.cephadm:Adding mds.e on smithi089 2024-09-21T00:17:49.360 INFO:tasks.cephadm:Adding mds.h on smithi089 2024-09-21T00:17:49.360 INFO:tasks.cephadm:Adding mds.k on smithi089 2024-09-21T00:17:49.360 INFO:tasks.cephadm:Adding mds.c on smithi098 2024-09-21T00:17:49.360 INFO:tasks.cephadm:Adding mds.f on smithi098 2024-09-21T00:17:49.360 INFO:tasks.cephadm:Adding mds.i on smithi098 2024-09-21T00:17:49.360 INFO:tasks.cephadm:Adding mds.l on smithi098 2024-09-21T00:17:49.360 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l' 2024-09-21T00:17:49.389 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:49 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2013241221' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T00:17:49.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:49 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2013241221' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T00:17:49.619 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:17:50.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.125+0000 7f64396c7640 1 -- 172.21.15.98:0/3623471677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64341027e0 msgr2=0x7f6434110610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:50.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.125+0000 7f64396c7640 1 --2- 172.21.15.98:0/3623471677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64341027e0 0x7f6434110610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f642c009ca0 tx=0x7f642c02a8c0 comp rx=0 tx=0).stop 2024-09-21T00:17:50.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.125+0000 7f64396c7640 1 -- 172.21.15.98:0/3623471677 shutdown_connections 2024-09-21T00:17:50.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.125+0000 7f64396c7640 1 --2- 172.21.15.98:0/3623471677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64341027e0 0x7f6434110610 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:50.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.125+0000 7f64396c7640 1 --2- 172.21.15.98:0/3623471677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6434101e90 0x7f64341022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:50.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.126+0000 7f64396c7640 1 --2- 172.21.15.98:0/3623471677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64341087a0 0x7f6434108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:50.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.126+0000 7f64396c7640 1 -- 172.21.15.98:0/3623471677 >> 172.21.15.98:0/3623471677 conn(0x7f6434079ec0 msgr2=0x7f643407a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:50.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.126+0000 7f64396c7640 1 -- 172.21.15.98:0/3623471677 shutdown_connections 2024-09-21T00:17:50.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.126+0000 7f64396c7640 1 -- 172.21.15.98:0/3623471677 wait complete. 2024-09-21T00:17:50.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.126+0000 7f64396c7640 1 Processor -- start 2024-09-21T00:17:50.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.127+0000 7f64396c7640 1 -- start start 2024-09-21T00:17:50.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.127+0000 7f64396c7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6434101e90 0x7f643419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:50.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.127+0000 7f64396c7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64341027e0 0x7f643419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:50.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.127+0000 7f64396c7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64341087a0 0x7f643419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:50.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.127+0000 7f64396c7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6434113bb0 con 0x7f6434101e90 2024-09-21T00:17:50.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.127+0000 7f64396c7640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6434113a30 con 0x7f64341027e0 2024-09-21T00:17:50.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.127+0000 7f64396c7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6434113d30 con 0x7f64341087a0 2024-09-21T00:17:50.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.127+0000 7f64337fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64341087a0 0x7f643419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:50.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.127+0000 7f64337fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64341087a0 0x7f643419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51712/0 (socket says 172.21.15.98:51712) 2024-09-21T00:17:50.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.127+0000 7f64337fe640 1 -- 172.21.15.98:0/924557079 learned_addr learned my addr 172.21.15.98:0/924557079 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-21T00:17:50.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.127+0000 7f64327fc640 1 --2- 172.21.15.98:0/924557079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64341027e0 0x7f643419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:50.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.128+0000 7f64337fe640 1 -- 172.21.15.98:0/924557079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64341027e0 msgr2=0x7f643419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:50.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.128+0000 7f64337fe640 1 --2- 172.21.15.98:0/924557079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64341027e0 0x7f643419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:50.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.128+0000 7f64337fe640 1 -- 172.21.15.98:0/924557079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6434101e90 msgr2=0x7f643419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:50.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.128+0000 7f64337fe640 1 --2- 172.21.15.98:0/924557079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6434101e90 0x7f643419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:50.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.128+0000 7f64337fe640 1 -- 172.21.15.98:0/924557079 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64341a0450 con 0x7f64341087a0 2024-09-21T00:17:50.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.128+0000 7f64327fc640 1 --2- 172.21.15.98:0/924557079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64341027e0 0x7f643419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:17:50.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.128+0000 7f64337fe640 1 --2- 172.21.15.98:0/924557079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64341087a0 0x7f643419fd20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f642c005cc0 tx=0x7f642c02f380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:50.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.128+0000 7f641bfff640 1 -- 172.21.15.98:0/924557079 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c003bf0 con 0x7f64341087a0 2024-09-21T00:17:50.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.128+0000 7f641bfff640 1 -- 172.21.15.98:0/924557079 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f642c003d90 con 0x7f64341087a0 2024-09-21T00:17:50.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.128+0000 7f64396c7640 1 -- 172.21.15.98:0/924557079 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64341a06e0 con 0x7f64341087a0 2024-09-21T00:17:50.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.128+0000 7f641bfff640 1 -- 172.21.15.98:0/924557079 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c02fbc0 con 0x7f64341087a0 2024-09-21T00:17:50.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.128+0000 7f64396c7640 1 -- 172.21.15.98:0/924557079 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64341a6ea0 con 0x7f64341087a0 2024-09-21T00:17:50.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.129+0000 7f64396c7640 1 -- 172.21.15.98:0/924557079 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64341022a0 con 0x7f64341087a0 2024-09-21T00:17:50.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.132+0000 7f641bfff640 1 -- 172.21.15.98:0/924557079 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f642c02fd60 con 0x7f64341087a0 2024-09-21T00:17:50.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.133+0000 7f641bfff640 1 --2- 172.21.15.98:0/924557079 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f641c0b6560 0x7f641c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:50.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.133+0000 7f641bfff640 1 -- 172.21.15.98:0/924557079 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f642c108740 con 0x7f64341087a0 2024-09-21T00:17:50.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.133+0000 7f6432ffd640 1 --2- 172.21.15.98:0/924557079 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f641c0b6560 0x7f641c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:50.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.133+0000 7f6432ffd640 1 --2- 172.21.15.98:0/924557079 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f641c0b6560 0x7f641c0b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6424009710 tx=0x7f6424006c90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:50.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.135+0000 7f641bfff640 1 -- 172.21.15.98:0/924557079 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f642c0cf3a0 con 0x7f64341087a0 2024-09-21T00:17:50.267 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:50 smithi098 ceph-mon[34725]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:50.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:50 smithi027 ceph-mon[30745]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:50.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.330+0000 7f64396c7640 1 -- 172.21.15.98:0/924557079 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l", "target": ["mon-mgr", ""]}) -- 0x7f6434063800 con 0x7f641c0b6560 2024-09-21T00:17:50.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.343+0000 7f641bfff640 1 -- 172.21.15.98:0/924557079 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6434063800 con 0x7f641c0b6560 2024-09-21T00:17:50.344 INFO:teuthology.orchestra.run.smithi098.stdout:Scheduled mds.all update... 2024-09-21T00:17:50.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.345+0000 7f64396c7640 1 -- 172.21.15.98:0/924557079 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f641c0b6560 msgr2=0x7f641c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:50.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.345+0000 7f64396c7640 1 --2- 172.21.15.98:0/924557079 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f641c0b6560 0x7f641c0b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6424009710 tx=0x7f6424006c90 comp rx=0 tx=0).stop 2024-09-21T00:17:50.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.345+0000 7f64396c7640 1 -- 172.21.15.98:0/924557079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64341087a0 msgr2=0x7f643419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:50.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.345+0000 7f64396c7640 1 --2- 172.21.15.98:0/924557079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64341087a0 0x7f643419fd20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f642c005cc0 tx=0x7f642c02f380 comp rx=0 tx=0).stop 2024-09-21T00:17:50.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.345+0000 7f64396c7640 1 -- 172.21.15.98:0/924557079 shutdown_connections 2024-09-21T00:17:50.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.345+0000 7f64396c7640 1 --2- 172.21.15.98:0/924557079 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f641c0b6560 0x7f641c0b8a70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:50.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.345+0000 7f64396c7640 1 --2- 172.21.15.98:0/924557079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64341087a0 0x7f643419fd20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:50.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.345+0000 7f64396c7640 1 --2- 172.21.15.98:0/924557079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64341027e0 0x7f643419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:50.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.345+0000 7f64396c7640 1 --2- 172.21.15.98:0/924557079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6434101e90 0x7f643419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:50.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.345+0000 7f64396c7640 1 -- 172.21.15.98:0/924557079 >> 172.21.15.98:0/924557079 conn(0x7f6434079ec0 msgr2=0x7f6434105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:50.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.345+0000 7f64396c7640 1 -- 172.21.15.98:0/924557079 shutdown_connections 2024-09-21T00:17:50.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T00:17:50.345+0000 7f64396c7640 1 -- 172.21.15.98:0/924557079 wait complete. 2024-09-21T00:17:50.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:50 smithi089 ceph-mon[34481]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:51.037 DEBUG:teuthology.orchestra.run.smithi027:mds.a> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T00:17:51.041 DEBUG:teuthology.orchestra.run.smithi027:mds.d> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T00:17:51.085 DEBUG:teuthology.orchestra.run.smithi027:mds.g> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T00:17:51.130 DEBUG:teuthology.orchestra.run.smithi027:mds.j> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T00:17:51.177 DEBUG:teuthology.orchestra.run.smithi089:mds.b> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T00:17:51.180 DEBUG:teuthology.orchestra.run.smithi089:mds.e> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T00:17:51.223 DEBUG:teuthology.orchestra.run.smithi089:mds.h> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T00:17:51.266 DEBUG:teuthology.orchestra.run.smithi089:mds.k> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T00:17:51.310 DEBUG:teuthology.orchestra.run.smithi098:mds.c> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T00:17:51.313 DEBUG:teuthology.orchestra.run.smithi098:mds.f> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T00:17:51.356 DEBUG:teuthology.orchestra.run.smithi098:mds.i> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T00:17:51.399 DEBUG:teuthology.orchestra.run.smithi098:mds.l> sudo journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T00:17:51.442 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T00:17:51.442 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:17:51.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:51 smithi089 ceph-mon[34481]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:51.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:51 smithi089 ceph-mon[34481]: from='client.24646 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:17:51.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:51 smithi089 ceph-mon[34481]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l;count:12 2024-09-21T00:17:51.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:51 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:51.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:51 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:17:51.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:51 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:51.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:51 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:17:51.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:51 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:51.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:51 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:51.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:51 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:51.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:51 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:17:51.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:51 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:51.480 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:51 smithi089 ceph-mon[34481]: Deploying daemon mds.l on smithi098 2024-09-21T00:17:51.641 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:51 smithi098 ceph-mon[34725]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:51.642 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:51 smithi098 ceph-mon[34725]: from='client.24646 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:17:51.642 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:51 smithi098 ceph-mon[34725]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l;count:12 2024-09-21T00:17:51.642 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:51 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:51.642 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:51 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:17:51.642 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:51 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:51.642 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:51 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:17:51.642 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:51 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:51.642 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:51 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:51.643 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:51 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:51.643 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:51 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:17:51.643 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:51 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:51.643 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:51 smithi098 ceph-mon[34725]: Deploying daemon mds.l on smithi098 2024-09-21T00:17:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:51 smithi027 ceph-mon[30745]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:51 smithi027 ceph-mon[30745]: from='client.24646 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:17:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:51 smithi027 ceph-mon[30745]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l;count:12 2024-09-21T00:17:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:51 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:51 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:17:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:51 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:51 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:17:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:51 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:51 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:51 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:51.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:51 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:17:51.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:51 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:51.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:51 smithi027 ceph-mon[30745]: Deploying daemon mds.l on smithi098 2024-09-21T00:17:51.821 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:17:52.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.491+0000 7f9695a00640 1 -- 172.21.15.27:0/2614998537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9690109e10 msgr2=0x7f9690110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:52.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.491+0000 7f9695a00640 1 --2- 172.21.15.27:0/2614998537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9690109e10 0x7f9690110640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9688009be0 tx=0x7f968802a6c0 comp rx=0 tx=0).stop 2024-09-21T00:17:52.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.491+0000 7f9695a00640 1 -- 172.21.15.27:0/2614998537 shutdown_connections 2024-09-21T00:17:52.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.491+0000 7f9695a00640 1 --2- 172.21.15.27:0/2614998537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9690109e10 0x7f9690110640 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:52.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.491+0000 7f9695a00640 1 --2- 172.21.15.27:0/2614998537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96901094c0 0x7f96901098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:52.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.491+0000 7f9695a00640 1 --2- 172.21.15.27:0/2614998537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9690103b80 0x7f9690103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:52.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.491+0000 7f9695a00640 1 -- 172.21.15.27:0/2614998537 >> 172.21.15.27:0/2614998537 conn(0x7f96900ff5d0 msgr2=0x7f9690101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:52.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.491+0000 7f9695a00640 1 -- 172.21.15.27:0/2614998537 shutdown_connections 2024-09-21T00:17:52.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.491+0000 7f9695a00640 1 -- 172.21.15.27:0/2614998537 wait complete. 2024-09-21T00:17:52.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.492+0000 7f9695a00640 1 Processor -- start 2024-09-21T00:17:52.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.493+0000 7f9695a00640 1 -- start start 2024-09-21T00:17:52.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.493+0000 7f9695a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9690103b80 0x7f969019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.493+0000 7f9695a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96901094c0 0x7f969019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.493+0000 7f9695a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9690109e10 0x7f969019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.493+0000 7f9695a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9690113ad0 con 0x7f9690103b80 2024-09-21T00:17:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.493+0000 7f9695a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9690113950 con 0x7f9690109e10 2024-09-21T00:17:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.493+0000 7f9695a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9690113c50 con 0x7f96901094c0 2024-09-21T00:17:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.493+0000 7f9687e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96901094c0 0x7f969019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.493+0000 7f9687e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96901094c0 0x7f969019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:35402/0 (socket says 172.21.15.27:35402) 2024-09-21T00:17:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.493+0000 7f9687e00640 1 -- 172.21.15.27:0/3244742593 learned_addr learned my addr 172.21.15.27:0/3244742593 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:17:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.493+0000 7f968f400640 1 --2- 172.21.15.27:0/3244742593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9690109e10 0x7f969019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.494+0000 7f968ea00640 1 --2- 172.21.15.27:0/3244742593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9690103b80 0x7f969019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.494+0000 7f9687e00640 1 -- 172.21.15.27:0/3244742593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9690109e10 msgr2=0x7f969019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.494+0000 7f9687e00640 1 --2- 172.21.15.27:0/3244742593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9690109e10 0x7f969019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.494+0000 7f9687e00640 1 -- 172.21.15.27:0/3244742593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9690103b80 msgr2=0x7f969019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.494+0000 7f9687e00640 1 --2- 172.21.15.27:0/3244742593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9690103b80 0x7f969019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.494+0000 7f9687e00640 1 -- 172.21.15.27:0/3244742593 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96901a03b0 con 0x7f96901094c0 2024-09-21T00:17:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.494+0000 7f968ea00640 1 --2- 172.21.15.27:0/3244742593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9690103b80 0x7f969019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:17:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.494+0000 7f9687e00640 1 --2- 172.21.15.27:0/3244742593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96901094c0 0x7f969019a9e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f967800e9f0 tx=0x7f967800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:52.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.494+0000 7f9687400640 1 -- 172.21.15.27:0/3244742593 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967800ccc0 con 0x7f96901094c0 2024-09-21T00:17:52.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.495+0000 7f9695a00640 1 -- 172.21.15.27:0/3244742593 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96901a06a0 con 0x7f96901094c0 2024-09-21T00:17:52.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.495+0000 7f9687400640 1 -- 172.21.15.27:0/3244742593 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96780108e0 con 0x7f96901094c0 2024-09-21T00:17:52.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.495+0000 7f9687400640 1 -- 172.21.15.27:0/3244742593 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96780187a0 con 0x7f96901094c0 2024-09-21T00:17:52.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.495+0000 7f9695a00640 1 -- 172.21.15.27:0/3244742593 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96901a6e00 con 0x7f96901094c0 2024-09-21T00:17:52.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.496+0000 7f9695a00640 1 -- 172.21.15.27:0/3244742593 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9690103f90 con 0x7f96901094c0 2024-09-21T00:17:52.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.498+0000 7f9687400640 1 -- 172.21.15.27:0/3244742593 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f9678010430 con 0x7f96901094c0 2024-09-21T00:17:52.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.500+0000 7f9687400640 1 --2- 172.21.15.27:0/3244742593 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96740b6560 0x7f96740b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:52.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.500+0000 7f9687400640 1 -- 172.21.15.27:0/3244742593 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f96780e75d0 con 0x7f96901094c0 2024-09-21T00:17:52.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.500+0000 7f968ea00640 1 --2- 172.21.15.27:0/3244742593 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96740b6560 0x7f96740b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:52.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.501+0000 7f968ea00640 1 --2- 172.21.15.27:0/3244742593 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96740b6560 0x7f96740b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9680005fd0 tx=0x7f9680009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:52.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.501+0000 7f9687400640 1 -- 172.21.15.27:0/3244742593 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f96780ae190 con 0x7f96901094c0 2024-09-21T00:17:52.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.742+0000 7f9695a00640 1 -- 172.21.15.27:0/3244742593 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f969019b570 con 0x7f96901094c0 2024-09-21T00:17:52.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.742+0000 7f9687400640 1 -- 172.21.15.27:0/3244742593 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f96780b2570 con 0x7f96901094c0 2024-09-21T00:17:52.744 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:17:52.744 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"btime":"2024-09-21T00:09:20:149616+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T00:17:52.744 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 1 2024-09-21T00:17:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.745+0000 7f9695a00640 1 -- 172.21.15.27:0/3244742593 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96740b6560 msgr2=0x7f96740b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.745+0000 7f9695a00640 1 --2- 172.21.15.27:0/3244742593 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96740b6560 0x7f96740b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9680005fd0 tx=0x7f9680009260 comp rx=0 tx=0).stop 2024-09-21T00:17:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.745+0000 7f9695a00640 1 -- 172.21.15.27:0/3244742593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96901094c0 msgr2=0x7f969019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.745+0000 7f9695a00640 1 --2- 172.21.15.27:0/3244742593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96901094c0 0x7f969019a9e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f967800e9f0 tx=0x7f967800eeb0 comp rx=0 tx=0).stop 2024-09-21T00:17:52.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.745+0000 7f9695a00640 1 -- 172.21.15.27:0/3244742593 shutdown_connections 2024-09-21T00:17:52.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.745+0000 7f9695a00640 1 --2- 172.21.15.27:0/3244742593 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96740b6560 0x7f96740b8a70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:52.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.745+0000 7f9695a00640 1 --2- 172.21.15.27:0/3244742593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9690109e10 0x7f969019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:52.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.745+0000 7f9695a00640 1 --2- 172.21.15.27:0/3244742593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96901094c0 0x7f969019a9e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:52.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.745+0000 7f9695a00640 1 --2- 172.21.15.27:0/3244742593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9690103b80 0x7f969019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:52.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.745+0000 7f9695a00640 1 -- 172.21.15.27:0/3244742593 >> 172.21.15.27:0/3244742593 conn(0x7f96900ff5d0 msgr2=0x7f96901019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:52.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.745+0000 7f9695a00640 1 -- 172.21.15.27:0/3244742593 shutdown_connections 2024-09-21T00:17:52.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:52.745+0000 7f9695a00640 1 -- 172.21.15.27:0/3244742593 wait complete. 2024-09-21T00:17:52.996 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 00:17:52 smithi098 systemd[1]: Starting Ceph mds.l for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:17:53.388 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:53 smithi098 ceph-mon[34725]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:53.388 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:53 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3244742593' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:17:53.389 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 00:17:53 smithi098 podman[63562]: 2024-09-21 00:17:53.011803167 +0000 UTC m=+0.015628011 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:17:53.389 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 00:17:53 smithi098 podman[63562]: 2024-09-21 00:17:53.123741142 +0000 UTC m=+0.127565985 container create e5614accb06847eb34f6a1d31d6b96fc7e13e44a893c0de2d4dec197dc584074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-l, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:17:53.389 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 00:17:53 smithi098 podman[63562]: 2024-09-21 00:17:53.351768119 +0000 UTC m=+0.355592964 container init e5614accb06847eb34f6a1d31d6b96fc7e13e44a893c0de2d4dec197dc584074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T00:17:53.389 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 00:17:53 smithi098 podman[63562]: 2024-09-21 00:17:53.354604826 +0000 UTC m=+0.358429680 container start e5614accb06847eb34f6a1d31d6b96fc7e13e44a893c0de2d4dec197dc584074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-l, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T00:17:53.389 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 00:17:53 smithi098 ceph-mds[63581]: -- 172.21.15.98:0/4011620993 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55866182f860 con 0x5586659d7180 2024-09-21T00:17:53.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:53 smithi027 ceph-mon[30745]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:53.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:53 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3244742593' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:17:53.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:53 smithi089 ceph-mon[34481]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:53.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:53 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3244742593' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:17:53.745 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 00:17:53 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-l[63577]: starting mds.l at 2024-09-21T00:17:53.745 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 00:17:53 smithi098 bash[63562]: e5614accb06847eb34f6a1d31d6b96fc7e13e44a893c0de2d4dec197dc584074 2024-09-21T00:17:53.746 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 00:17:53 smithi098 systemd[1]: Started Ceph mds.l for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:17:54.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:54 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.98:6832/3448953882,v1:172.21.15.98:6833/3448953882] up:boot 2024-09-21T00:17:54.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:54 smithi089 ceph-mon[34481]: fsmap 1 up:standby 2024-09-21T00:17:54.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:54 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T00:17:54.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:54 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:54.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:54 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:54.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:54 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:54.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:54 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:54.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:54 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:54.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:54 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:17:54.481 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:54 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:54.481 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:54 smithi089 ceph-mon[34481]: Deploying daemon mds.k on smithi089 2024-09-21T00:17:54.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:54 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.98:6832/3448953882,v1:172.21.15.98:6833/3448953882] up:boot 2024-09-21T00:17:54.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:54 smithi098 ceph-mon[34725]: fsmap 1 up:standby 2024-09-21T00:17:54.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:54 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T00:17:54.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:54 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:54.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:54 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:54.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:54 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:54.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:54 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:54.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:54 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:54.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:54 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:17:54.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:54 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:54.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:54 smithi098 ceph-mon[34725]: Deploying daemon mds.k on smithi089 2024-09-21T00:17:54.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:54 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.98:6832/3448953882,v1:172.21.15.98:6833/3448953882] up:boot 2024-09-21T00:17:54.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:54 smithi027 ceph-mon[30745]: fsmap 1 up:standby 2024-09-21T00:17:54.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:54 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T00:17:54.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:54 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:54.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:54 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:54.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:54 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:54.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:54 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:54.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:54 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:54.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:54 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:17:54.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:54 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:54.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:54 smithi027 ceph-mon[30745]: Deploying daemon mds.k on smithi089 2024-09-21T00:17:55.595 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:17:55.721 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:55 smithi089 ceph-mon[34481]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:55.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:55 smithi027 ceph-mon[30745]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:55.984 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:17:55.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:55 smithi098 ceph-mon[34725]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:56.478 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 00:17:56 smithi089 systemd[1]: Started Ceph mds.k for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:17:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.638+0000 7fd3c8200640 1 -- 172.21.15.27:0/1138029467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c0109e00 msgr2=0x7fd3c0110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.638+0000 7fd3c8200640 1 --2- 172.21.15.27:0/1138029467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c0109e00 0x7fd3c0110630 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009ca0 tx=0x7fd3b802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:17:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.638+0000 7fd3c8200640 1 -- 172.21.15.27:0/1138029467 shutdown_connections 2024-09-21T00:17:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.638+0000 7fd3c8200640 1 --2- 172.21.15.27:0/1138029467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c0109e00 0x7fd3c0110630 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.638+0000 7fd3c8200640 1 --2- 172.21.15.27:0/1138029467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c01094b0 0x7fd3c01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.638+0000 7fd3c8200640 1 --2- 172.21.15.27:0/1138029467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3c0103b70 0x7fd3c0103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.638+0000 7fd3c8200640 1 -- 172.21.15.27:0/1138029467 >> 172.21.15.27:0/1138029467 conn(0x7fd3c00ff5e0 msgr2=0x7fd3c0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.638+0000 7fd3c8200640 1 -- 172.21.15.27:0/1138029467 shutdown_connections 2024-09-21T00:17:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.638+0000 7fd3c8200640 1 -- 172.21.15.27:0/1138029467 wait complete. 2024-09-21T00:17:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.639+0000 7fd3c8200640 1 Processor -- start 2024-09-21T00:17:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.639+0000 7fd3c8200640 1 -- start start 2024-09-21T00:17:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.640+0000 7fd3c8200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c0103b70 0x7fd3c019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.640+0000 7fd3c8200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3c01094b0 0x7fd3c019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.640+0000 7fd3c8200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c0109e00 0x7fd3c01a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.640+0000 7fd3c8200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c0113b10 con 0x7fd3c0103b70 2024-09-21T00:17:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.640+0000 7fd3c8200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c0113990 con 0x7fd3c0109e00 2024-09-21T00:17:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.640+0000 7fd3c8200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c0113c90 con 0x7fd3c01094b0 2024-09-21T00:17:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.640+0000 7fd3c5800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c0103b70 0x7fd3c019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.640+0000 7fd3c6200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c0109e00 0x7fd3c01a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.640+0000 7fd3c5800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c0103b70 0x7fd3c019c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51860/0 (socket says 172.21.15.27:51860) 2024-09-21T00:17:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.640+0000 7fd3c5800640 1 -- 172.21.15.27:0/1076968261 learned_addr learned my addr 172.21.15.27:0/1076968261 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:17:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.640+0000 7fd3c4e00640 1 --2- 172.21.15.27:0/1076968261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3c01094b0 0x7fd3c019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.641+0000 7fd3c6200640 1 -- 172.21.15.27:0/1076968261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3c01094b0 msgr2=0x7fd3c019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.641+0000 7fd3c6200640 1 --2- 172.21.15.27:0/1076968261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3c01094b0 0x7fd3c019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.641+0000 7fd3c6200640 1 -- 172.21.15.27:0/1076968261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c0103b70 msgr2=0x7fd3c019c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.641+0000 7fd3c6200640 1 --2- 172.21.15.27:0/1076968261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c0103b70 0x7fd3c019c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.641+0000 7fd3c6200640 1 -- 172.21.15.27:0/1076968261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c01a25b0 con 0x7fd3c0109e00 2024-09-21T00:17:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.641+0000 7fd3c4e00640 1 --2- 172.21.15.27:0/1076968261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3c01094b0 0x7fd3c019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:17:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.641+0000 7fd3c6200640 1 --2- 172.21.15.27:0/1076968261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c0109e00 0x7fd3c01a1e80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8005e30 tx=0x7fd3b80317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.641+0000 7fd3be000640 1 -- 172.21.15.27:0/1076968261 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8003b70 con 0x7fd3c0109e00 2024-09-21T00:17:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.641+0000 7fd3c8200640 1 -- 172.21.15.27:0/1076968261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c01a2840 con 0x7fd3c0109e00 2024-09-21T00:17:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.641+0000 7fd3c8200640 1 -- 172.21.15.27:0/1076968261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c01a9000 con 0x7fd3c0109e00 2024-09-21T00:17:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.641+0000 7fd3be000640 1 -- 172.21.15.27:0/1076968261 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3b8003d10 con 0x7fd3c0109e00 2024-09-21T00:17:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.641+0000 7fd3be000640 1 -- 172.21.15.27:0/1076968261 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b802f970 con 0x7fd3c0109e00 2024-09-21T00:17:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.643+0000 7fd3c8200640 1 -- 172.21.15.27:0/1076968261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c0103f80 con 0x7fd3c0109e00 2024-09-21T00:17:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.645+0000 7fd3be000640 1 -- 172.21.15.27:0/1076968261 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fd3b802fb10 con 0x7fd3c0109e00 2024-09-21T00:17:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.647+0000 7fd3be000640 1 --2- 172.21.15.27:0/1076968261 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd3a80b6660 0x7fd3a80b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:17:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.648+0000 7fd3be000640 1 -- 172.21.15.27:0/1076968261 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fd3b8108030 con 0x7fd3c0109e00 2024-09-21T00:17:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.648+0000 7fd3c5800640 1 --2- 172.21.15.27:0/1076968261 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd3a80b6660 0x7fd3a80b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:17:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.648+0000 7fd3c5800640 1 --2- 172.21.15.27:0/1076968261 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd3a80b6660 0x7fd3a80b8b70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009cd0 tx=0x7fd3b4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:17:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.651+0000 7fd3be000640 1 -- 172.21.15.27:0/1076968261 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd3b80cebf0 con 0x7fd3c0109e00 2024-09-21T00:17:56.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.895+0000 7fd3c8200640 1 -- 172.21.15.27:0/1076968261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3c01a92a0 con 0x7fd3c0109e00 2024-09-21T00:17:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.896+0000 7fd3be000640 1 -- 172.21.15.27:0/1076968261 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2257 (secure 0 0 0) 0x7fd3b80d2fd0 con 0x7fd3c0109e00 2024-09-21T00:17:56.897 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:17:56.897 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":3,"btime":"2024-09-21T00:17:56:540406+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T00:17:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 3 2024-09-21T00:17:56.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.898+0000 7fd3c8200640 1 -- 172.21.15.27:0/1076968261 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd3a80b6660 msgr2=0x7fd3a80b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:56.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.898+0000 7fd3c8200640 1 --2- 172.21.15.27:0/1076968261 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd3a80b6660 0x7fd3a80b8b70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009cd0 tx=0x7fd3b4005f90 comp rx=0 tx=0).stop 2024-09-21T00:17:56.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.898+0000 7fd3c8200640 1 -- 172.21.15.27:0/1076968261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c0109e00 msgr2=0x7fd3c01a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:17:56.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.898+0000 7fd3c8200640 1 --2- 172.21.15.27:0/1076968261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c0109e00 0x7fd3c01a1e80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8005e30 tx=0x7fd3b80317c0 comp rx=0 tx=0).stop 2024-09-21T00:17:56.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.898+0000 7fd3c8200640 1 -- 172.21.15.27:0/1076968261 shutdown_connections 2024-09-21T00:17:56.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.898+0000 7fd3c8200640 1 --2- 172.21.15.27:0/1076968261 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd3a80b6660 0x7fd3a80b8b70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:56.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.898+0000 7fd3c8200640 1 --2- 172.21.15.27:0/1076968261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c0109e00 0x7fd3c01a1e80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:56.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.898+0000 7fd3c8200640 1 --2- 172.21.15.27:0/1076968261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3c01094b0 0x7fd3c019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:56.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.898+0000 7fd3c8200640 1 --2- 172.21.15.27:0/1076968261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c0103b70 0x7fd3c019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:17:56.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.898+0000 7fd3c8200640 1 -- 172.21.15.27:0/1076968261 >> 172.21.15.27:0/1076968261 conn(0x7fd3c00ff5e0 msgr2=0x7fd3c0106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:17:56.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.899+0000 7fd3c8200640 1 -- 172.21.15.27:0/1076968261 shutdown_connections 2024-09-21T00:17:56.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:17:56.899+0000 7fd3c8200640 1 -- 172.21.15.27:0/1076968261 wait complete. 2024-09-21T00:17:57.504 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:57 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:57.504 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:57 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:57.504 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:57 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:57.504 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:57 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:57.505 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:57 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:57.505 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:57 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:17:57.505 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:57 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:57.505 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:57 smithi089 ceph-mon[34481]: Deploying daemon mds.j on smithi027 2024-09-21T00:17:57.505 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:57 smithi089 ceph-mon[34481]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:57.505 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:57 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.89:6832/191964694,v1:172.21.15.89:6833/191964694] up:boot 2024-09-21T00:17:57.505 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:57 smithi089 ceph-mon[34481]: fsmap 2 up:standby 2024-09-21T00:17:57.505 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:57 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T00:17:57.505 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:57 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1076968261' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:17:57.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:57 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:57.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:57 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:57.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:57 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:57.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:57 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:57.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:57 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:57.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:57 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:17:57.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:57 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:57.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:57 smithi027 ceph-mon[30745]: Deploying daemon mds.j on smithi027 2024-09-21T00:17:57.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:57 smithi027 ceph-mon[30745]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:57.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:57 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.89:6832/191964694,v1:172.21.15.89:6833/191964694] up:boot 2024-09-21T00:17:57.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:57 smithi027 ceph-mon[30745]: fsmap 2 up:standby 2024-09-21T00:17:57.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:57 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T00:17:57.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:57 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1076968261' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:17:57.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:57 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:57.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:57 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:57.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:57 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:17:57.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:57 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:57.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:57 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:17:57.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:57 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:17:57.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:57 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:17:57.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:57 smithi098 ceph-mon[34725]: Deploying daemon mds.j on smithi027 2024-09-21T00:17:57.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:57 smithi098 ceph-mon[34725]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:57.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:57 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.89:6832/191964694,v1:172.21.15.89:6833/191964694] up:boot 2024-09-21T00:17:57.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:57 smithi098 ceph-mon[34725]: fsmap 2 up:standby 2024-09-21T00:17:57.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:57 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T00:17:57.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:57 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1076968261' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:17:59.317 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:17:59 smithi027 systemd[1]: Starting Ceph mds.j for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:17:59.595 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:17:59 smithi027 ceph-mon[30745]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:59.595 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:17:59 smithi027 podman[69335]: 2024-09-21 00:17:59.246219606 +0000 UTC m=+0.027959155 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:17:59.595 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:17:59 smithi027 podman[69335]: 2024-09-21 00:17:59.483585374 +0000 UTC m=+0.265324959 container create 8f3c7975b433443c4d7aefdf58d73401de893890d55154726883bdcabad44fd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-j, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T00:17:59.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:17:59 smithi089 ceph-mon[34481]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:59.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:17:59 smithi098 ceph-mon[34725]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:17:59.946 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:17:59 smithi027 podman[69335]: 2024-09-21 00:17:59.703282279 +0000 UTC m=+0.485021827 container init 8f3c7975b433443c4d7aefdf58d73401de893890d55154726883bdcabad44fd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T00:17:59.946 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:17:59 smithi027 podman[69335]: 2024-09-21 00:17:59.710475614 +0000 UTC m=+0.492215171 container start 8f3c7975b433443c4d7aefdf58d73401de893890d55154726883bdcabad44fd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-j, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T00:17:59.946 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:17:59 smithi027 bash[69335]: 8f3c7975b433443c4d7aefdf58d73401de893890d55154726883bdcabad44fd0 2024-09-21T00:17:59.947 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:17:59 smithi027 systemd[1]: Started Ceph mds.j for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:17:59.947 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:17:59 smithi027 ceph-mds[69355]: -- 172.21.15.27:0/2542984759 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b199ae9860 con 0x55b19dc91a80 2024-09-21T00:17:59.947 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:17:59 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-j[69350]: starting mds.j at 2024-09-21T00:18:00.857 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:18:00.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:00 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:00.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:00 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:00.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:00 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:00.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:00 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:00.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:00 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:00.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:00 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:00.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:00 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:00.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:00 smithi098 ceph-mon[34725]: Deploying daemon mds.d on smithi027 2024-09-21T00:18:00.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:00 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.27:6834/4068274538,v1:172.21.15.27:6835/4068274538] up:boot 2024-09-21T00:18:00.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:00 smithi098 ceph-mon[34725]: fsmap 3 up:standby 2024-09-21T00:18:00.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:00 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T00:18:01.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:00 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:01.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:00 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:01.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:00 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:01.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:00 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:01.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:00 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:01.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:00 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:01.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:00 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:01.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:00 smithi089 ceph-mon[34481]: Deploying daemon mds.d on smithi027 2024-09-21T00:18:01.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:00 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.27:6834/4068274538,v1:172.21.15.27:6835/4068274538] up:boot 2024-09-21T00:18:01.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:00 smithi089 ceph-mon[34481]: fsmap 3 up:standby 2024-09-21T00:18:01.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:00 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T00:18:01.276 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:00 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:01.277 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:00 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:01.277 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:00 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:01.277 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:00 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:01.277 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:00 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:01.277 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:00 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:01.277 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:00 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:01.277 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:00 smithi027 ceph-mon[30745]: Deploying daemon mds.d on smithi027 2024-09-21T00:18:01.277 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:00 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.27:6834/4068274538,v1:172.21.15.27:6835/4068274538] up:boot 2024-09-21T00:18:01.277 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:00 smithi027 ceph-mon[30745]: fsmap 3 up:standby 2024-09-21T00:18:01.277 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:00 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T00:18:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:02.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:02 smithi098 ceph-mon[34725]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:02.249 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:02 smithi027 ceph-mon[30745]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:02.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:02 smithi089 ceph-mon[34481]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:02.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.587+0000 7fe779800640 1 -- 172.21.15.27:0/110510115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410b6d0 msgr2=0x7fe77410bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:02.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.587+0000 7fe779800640 1 --2- 172.21.15.27:0/110510115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410b6d0 0x7fe77410bae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe760009900 tx=0x7fe76002a410 comp rx=0 tx=0).stop 2024-09-21T00:18:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.587+0000 7fe779800640 1 -- 172.21.15.27:0/110510115 shutdown_connections 2024-09-21T00:18:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.587+0000 7fe779800640 1 --2- 172.21.15.27:0/110510115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe77410c020 0x7fe774112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.587+0000 7fe779800640 1 --2- 172.21.15.27:0/110510115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410b6d0 0x7fe77410bae0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.587+0000 7fe779800640 1 --2- 172.21.15.27:0/110510115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe774105d90 0x7fe7741061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.587+0000 7fe779800640 1 -- 172.21.15.27:0/110510115 >> 172.21.15.27:0/110510115 conn(0x7fe774101860 msgr2=0x7fe774103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.588+0000 7fe779800640 1 -- 172.21.15.27:0/110510115 shutdown_connections 2024-09-21T00:18:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.588+0000 7fe779800640 1 -- 172.21.15.27:0/110510115 wait complete. 2024-09-21T00:18:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.589+0000 7fe779800640 1 Processor -- start 2024-09-21T00:18:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.589+0000 7fe779800640 1 -- start start 2024-09-21T00:18:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.589+0000 7fe779800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe774105d90 0x7fe77419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.589+0000 7fe779800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe77410b6d0 0x7fe77419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.589+0000 7fe779800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410c020 0x7fe7741a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.589+0000 7fe779800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe774115ce0 con 0x7fe77410c020 2024-09-21T00:18:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.589+0000 7fe779800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe774115b60 con 0x7fe774105d90 2024-09-21T00:18:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.589+0000 7fe779800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe774115e60 con 0x7fe77410b6d0 2024-09-21T00:18:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.590+0000 7fe772a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe774105d90 0x7fe77419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.590+0000 7fe773400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410c020 0x7fe7741a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.590+0000 7fe772a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe774105d90 0x7fe77419c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38438/0 (socket says 172.21.15.27:38438) 2024-09-21T00:18:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.590+0000 7fe772a00640 1 -- 172.21.15.27:0/3257202797 learned_addr learned my addr 172.21.15.27:0/3257202797 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.590+0000 7fe772000640 1 --2- 172.21.15.27:0/3257202797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe77410b6d0 0x7fe77419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.590+0000 7fe772a00640 1 -- 172.21.15.27:0/3257202797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe77410b6d0 msgr2=0x7fe77419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.590+0000 7fe772a00640 1 --2- 172.21.15.27:0/3257202797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe77410b6d0 0x7fe77419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.590+0000 7fe772a00640 1 -- 172.21.15.27:0/3257202797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410c020 msgr2=0x7fe7741a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.590+0000 7fe772a00640 1 --2- 172.21.15.27:0/3257202797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410c020 0x7fe7741a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.590+0000 7fe772a00640 1 -- 172.21.15.27:0/3257202797 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7741a2620 con 0x7fe774105d90 2024-09-21T00:18:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.590+0000 7fe773400640 1 --2- 172.21.15.27:0/3257202797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410c020 0x7fe7741a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:18:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.591+0000 7fe772a00640 1 --2- 172.21.15.27:0/3257202797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe774105d90 0x7fe77419c6b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e9f0 tx=0x7fe76400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.591+0000 7fe76b400640 1 -- 172.21.15.27:0/3257202797 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76400cd80 con 0x7fe774105d90 2024-09-21T00:18:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.591+0000 7fe76b400640 1 -- 172.21.15.27:0/3257202797 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7640108e0 con 0x7fe774105d90 2024-09-21T00:18:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.591+0000 7fe76b400640 1 -- 172.21.15.27:0/3257202797 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764018980 con 0x7fe774105d90 2024-09-21T00:18:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.591+0000 7fe779800640 1 -- 172.21.15.27:0/3257202797 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7741a2910 con 0x7fe774105d90 2024-09-21T00:18:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.591+0000 7fe779800640 1 -- 172.21.15.27:0/3257202797 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7741a9070 con 0x7fe774105d90 2024-09-21T00:18:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.593+0000 7fe779800640 1 -- 172.21.15.27:0/3257202797 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7741061a0 con 0x7fe774105d90 2024-09-21T00:18:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.596+0000 7fe76b400640 1 -- 172.21.15.27:0/3257202797 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fe764010430 con 0x7fe774105d90 2024-09-21T00:18:02.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.597+0000 7fe76b400640 1 --2- 172.21.15.27:0/3257202797 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7580b65e0 0x7fe7580b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:02.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.598+0000 7fe76b400640 1 -- 172.21.15.27:0/3257202797 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fe7640e7760 con 0x7fe774105d90 2024-09-21T00:18:02.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.598+0000 7fe772000640 1 --2- 172.21.15.27:0/3257202797 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7580b65e0 0x7fe7580b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:02.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.598+0000 7fe76b400640 1 -- 172.21.15.27:0/3257202797 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe7640b2700 con 0x7fe774105d90 2024-09-21T00:18:02.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.599+0000 7fe772000640 1 --2- 172.21.15.27:0/3257202797 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7580b65e0 0x7fe7580b8af0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe760009230 tx=0x7fe760031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:02.817 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:18:02 smithi027 systemd[1]: Starting Ceph mds.d for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:18:02.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.873+0000 7fe779800640 1 -- 172.21.15.27:0/3257202797 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe77410de70 con 0x7fe774105d90 2024-09-21T00:18:02.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.874+0000 7fe76b400640 1 -- 172.21.15.27:0/3257202797 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3082 (secure 0 0 0) 0x7fe7640b7640 con 0x7fe774105d90 2024-09-21T00:18:02.875 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:18:02.875 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":4,"btime":"2024-09-21T00:18:00:375417+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-09-21T00:18:02.876 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 4 2024-09-21T00:18:02.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.877+0000 7fe779800640 1 -- 172.21.15.27:0/3257202797 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7580b65e0 msgr2=0x7fe7580b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:02.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.877+0000 7fe779800640 1 --2- 172.21.15.27:0/3257202797 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7580b65e0 0x7fe7580b8af0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe760009230 tx=0x7fe760031400 comp rx=0 tx=0).stop 2024-09-21T00:18:02.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.877+0000 7fe779800640 1 -- 172.21.15.27:0/3257202797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe774105d90 msgr2=0x7fe77419c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:02.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.877+0000 7fe779800640 1 --2- 172.21.15.27:0/3257202797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe774105d90 0x7fe77419c6b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e9f0 tx=0x7fe76400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:18:02.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.877+0000 7fe779800640 1 -- 172.21.15.27:0/3257202797 shutdown_connections 2024-09-21T00:18:02.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.877+0000 7fe779800640 1 --2- 172.21.15.27:0/3257202797 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7580b65e0 0x7fe7580b8af0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:02.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.877+0000 7fe779800640 1 --2- 172.21.15.27:0/3257202797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410c020 0x7fe7741a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:02.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.877+0000 7fe779800640 1 --2- 172.21.15.27:0/3257202797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe77410b6d0 0x7fe77419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:02.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.877+0000 7fe779800640 1 --2- 172.21.15.27:0/3257202797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe774105d90 0x7fe77419c6b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:02.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.877+0000 7fe779800640 1 -- 172.21.15.27:0/3257202797 >> 172.21.15.27:0/3257202797 conn(0x7fe774101860 msgr2=0x7fe774108ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:02.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.877+0000 7fe779800640 1 -- 172.21.15.27:0/3257202797 shutdown_connections 2024-09-21T00:18:02.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:02.877+0000 7fe779800640 1 -- 172.21.15.27:0/3257202797 wait complete. 2024-09-21T00:18:03.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:03 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3257202797' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:03.317 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:18:02 smithi027 podman[69710]: 2024-09-21 00:18:02.869199871 +0000 UTC m=+0.028619227 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:18:03.317 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:18:02 smithi027 podman[69710]: 2024-09-21 00:18:02.985050828 +0000 UTC m=+0.144470156 container create e2ac6703cc5d1e07568936bdbc61ca156ad7c043a381d4d05e5596d390abfa9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T00:18:03.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:03 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3257202797' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:03.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:03 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3257202797' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:03.727 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:18:03 smithi027 podman[69710]: 2024-09-21 00:18:03.448106025 +0000 UTC m=+0.607525347 container init e2ac6703cc5d1e07568936bdbc61ca156ad7c043a381d4d05e5596d390abfa9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-d, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T00:18:03.727 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:18:03 smithi027 podman[69710]: 2024-09-21 00:18:03.455099393 +0000 UTC m=+0.614518715 container start e2ac6703cc5d1e07568936bdbc61ca156ad7c043a381d4d05e5596d390abfa9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T00:18:03.727 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:18:03 smithi027 bash[69710]: e2ac6703cc5d1e07568936bdbc61ca156ad7c043a381d4d05e5596d390abfa9c 2024-09-21T00:18:03.727 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:18:03 smithi027 systemd[1]: Started Ceph mds.d for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:18:03.728 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:18:03 smithi027 ceph-mds[69741]: -- 172.21.15.27:0/3082131861 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564fa479d860 con 0x564fa8945a80 2024-09-21T00:18:03.728 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:18:03 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-d[69737]: starting mds.d at 2024-09-21T00:18:04.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:04 smithi027 ceph-mon[30745]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:04.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:04 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:04.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:04 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:04.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:04 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:04.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:04 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:04.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:04 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:04.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:04 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:04.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:04 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:04.335 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:04 smithi089 ceph-mon[34481]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:04.336 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:04 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:04.336 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:04 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:04.336 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:04 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:04.336 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:04 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:04.336 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:04 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:04.337 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:04 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:04.337 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:04 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:04.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:04 smithi098 ceph-mon[34725]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:04.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:04 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:04.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:04 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:04.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:04 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:04.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:04 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:04.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:04 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:04.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:04 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:04.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:04 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:05.315 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:05 smithi089 ceph-mon[34481]: Deploying daemon mds.e on smithi089 2024-09-21T00:18:05.315 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:05 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.27:6836/1265394249,v1:172.21.15.27:6837/1265394249] up:boot 2024-09-21T00:18:05.316 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:05 smithi089 ceph-mon[34481]: fsmap 4 up:standby 2024-09-21T00:18:05.316 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:05 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T00:18:05.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:05 smithi027 ceph-mon[30745]: Deploying daemon mds.e on smithi089 2024-09-21T00:18:05.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:05 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.27:6836/1265394249,v1:172.21.15.27:6837/1265394249] up:boot 2024-09-21T00:18:05.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:05 smithi027 ceph-mon[30745]: fsmap 4 up:standby 2024-09-21T00:18:05.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:05 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T00:18:05.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:05 smithi098 ceph-mon[34725]: Deploying daemon mds.e on smithi089 2024-09-21T00:18:05.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:05 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.27:6836/1265394249,v1:172.21.15.27:6837/1265394249] up:boot 2024-09-21T00:18:05.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:05 smithi098 ceph-mon[34725]: fsmap 4 up:standby 2024-09-21T00:18:05.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:05 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T00:18:05.979 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 00:18:05 smithi089 podman[62554]: 2024-09-21 00:18:05.909300385 +0000 UTC m=+0.127771217 container create 976b1886d114aa685091894c12951f091f01de9572ea6acabeb07252d094852e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-e, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:18:06.109 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:18:06.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:06 smithi027 ceph-mon[30745]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:06.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:06 smithi089 ceph-mon[34481]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:06.479 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 00:18:06 smithi089 podman[62554]: 2024-09-21 00:18:06.103905242 +0000 UTC m=+0.322376030 container init 976b1886d114aa685091894c12951f091f01de9572ea6acabeb07252d094852e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T00:18:06.479 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 00:18:06 smithi089 podman[62554]: 2024-09-21 00:18:06.107115685 +0000 UTC m=+0.325586480 container start 976b1886d114aa685091894c12951f091f01de9572ea6acabeb07252d094852e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T00:18:06.479 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 00:18:06 smithi089 ceph-mds[62573]: -- 172.21.15.89:0/1618738561 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f1e883860 con 0x558f22a2b600 2024-09-21T00:18:06.479 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 00:18:06 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-e[62569]: starting mds.e at 2024-09-21T00:18:06.479 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 00:18:06 smithi089 bash[62554]: 976b1886d114aa685091894c12951f091f01de9572ea6acabeb07252d094852e 2024-09-21T00:18:06.479 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 00:18:06 smithi089 systemd[1]: Started Ceph mds.e for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:18:06.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:06 smithi098 ceph-mon[34725]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:06.512 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.131+0000 7f4380400640 1 -- 172.21.15.27:0/1303381788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f437810c010 msgr2=0x7f4378112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.131+0000 7f4380400640 1 --2- 172.21.15.27:0/1303381788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f437810c010 0x7f4378112840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4370009be0 tx=0x7f437002a6c0 comp rx=0 tx=0).stop 2024-09-21T00:18:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.131+0000 7f4380400640 1 -- 172.21.15.27:0/1303381788 shutdown_connections 2024-09-21T00:18:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.131+0000 7f4380400640 1 --2- 172.21.15.27:0/1303381788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f437810c010 0x7f4378112840 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.131+0000 7f4380400640 1 --2- 172.21.15.27:0/1303381788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f437810b6c0 0x7f437810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.131+0000 7f4380400640 1 --2- 172.21.15.27:0/1303381788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4378105d80 0x7f4378106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.131+0000 7f4380400640 1 -- 172.21.15.27:0/1303381788 >> 172.21.15.27:0/1303381788 conn(0x7f4378101810 msgr2=0x7f4378103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.131+0000 7f4380400640 1 -- 172.21.15.27:0/1303381788 shutdown_connections 2024-09-21T00:18:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.131+0000 7f4380400640 1 -- 172.21.15.27:0/1303381788 wait complete. 2024-09-21T00:18:07.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.132+0000 7f4380400640 1 Processor -- start 2024-09-21T00:18:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.132+0000 7f4380400640 1 -- start start 2024-09-21T00:18:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.133+0000 7f4380400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4378105d80 0x7f437819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.133+0000 7f4380400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f437810b6c0 0x7f437819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.133+0000 7f4380400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f437810c010 0x7f43781a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.133+0000 7f4380400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4378115cd0 con 0x7f437810c010 2024-09-21T00:18:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.133+0000 7f4380400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4378115b50 con 0x7f437810b6c0 2024-09-21T00:18:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.133+0000 7f4380400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4378115e50 con 0x7f4378105d80 2024-09-21T00:18:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.133+0000 7f437da00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4378105d80 0x7f437819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.133+0000 7f437da00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4378105d80 0x7f437819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:60190/0 (socket says 172.21.15.27:60190) 2024-09-21T00:18:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.133+0000 7f437da00640 1 -- 172.21.15.27:0/1236180149 learned_addr learned my addr 172.21.15.27:0/1236180149 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.133+0000 7f437e400640 1 --2- 172.21.15.27:0/1236180149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f437810c010 0x7f43781a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.133+0000 7f437d000640 1 --2- 172.21.15.27:0/1236180149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f437810b6c0 0x7f437819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.134+0000 7f437da00640 1 -- 172.21.15.27:0/1236180149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f437810b6c0 msgr2=0x7f437819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.134+0000 7f437da00640 1 --2- 172.21.15.27:0/1236180149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f437810b6c0 0x7f437819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.134+0000 7f437da00640 1 -- 172.21.15.27:0/1236180149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f437810c010 msgr2=0x7f43781a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.134+0000 7f437da00640 1 --2- 172.21.15.27:0/1236180149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f437810c010 0x7f43781a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.134+0000 7f437da00640 1 -- 172.21.15.27:0/1236180149 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43781a25d0 con 0x7f4378105d80 2024-09-21T00:18:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.134+0000 7f437da00640 1 --2- 172.21.15.27:0/1236180149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4378105d80 0x7f437819c660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f436c00e9c0 tx=0x7f436c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.134+0000 7f4376000640 1 -- 172.21.15.27:0/1236180149 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c00ccc0 con 0x7f4378105d80 2024-09-21T00:18:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.134+0000 7f4376000640 1 -- 172.21.15.27:0/1236180149 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f436c0054d0 con 0x7f4378105d80 2024-09-21T00:18:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.134+0000 7f4380400640 1 -- 172.21.15.27:0/1236180149 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43781a28c0 con 0x7f4378105d80 2024-09-21T00:18:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.134+0000 7f4376000640 1 -- 172.21.15.27:0/1236180149 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c010430 con 0x7f4378105d80 2024-09-21T00:18:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.134+0000 7f4380400640 1 -- 172.21.15.27:0/1236180149 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43781a9020 con 0x7f4378105d80 2024-09-21T00:18:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.135+0000 7f4380400640 1 -- 172.21.15.27:0/1236180149 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4378106190 con 0x7f4378105d80 2024-09-21T00:18:07.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.137+0000 7f4376000640 1 -- 172.21.15.27:0/1236180149 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f436c004f80 con 0x7f4378105d80 2024-09-21T00:18:07.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.139+0000 7f4376000640 1 --2- 172.21.15.27:0/1236180149 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f43600b65e0 0x7f43600b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:07.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.139+0000 7f4376000640 1 -- 172.21.15.27:0/1236180149 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f436c0e73f0 con 0x7f4378105d80 2024-09-21T00:18:07.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.139+0000 7f437d000640 1 --2- 172.21.15.27:0/1236180149 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f43600b65e0 0x7f43600b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:07.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.140+0000 7f437d000640 1 --2- 172.21.15.27:0/1236180149 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f43600b65e0 0x7f43600b8af0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f43781050c0 tx=0x7f4364005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.143+0000 7f4376000640 1 -- 172.21.15.27:0/1236180149 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f436c0adfb0 con 0x7f4378105d80 2024-09-21T00:18:07.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.382+0000 7f4380400640 1 -- 172.21.15.27:0/1236180149 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f437819d790 con 0x7f4378105d80 2024-09-21T00:18:07.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.383+0000 7f4376000640 1 -- 172.21.15.27:0/1236180149 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4729 (secure 0 0 0) 0x7f436c0b2390 con 0x7f4378105d80 2024-09-21T00:18:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:18:07.385 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":6,"btime":"2024-09-21T00:18:07:034959+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T00:18:07.385 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 6 2024-09-21T00:18:07.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.385+0000 7f4380400640 1 -- 172.21.15.27:0/1236180149 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f43600b65e0 msgr2=0x7f43600b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:07.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.385+0000 7f4380400640 1 --2- 172.21.15.27:0/1236180149 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f43600b65e0 0x7f43600b8af0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f43781050c0 tx=0x7f4364005f90 comp rx=0 tx=0).stop 2024-09-21T00:18:07.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.385+0000 7f4380400640 1 -- 172.21.15.27:0/1236180149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4378105d80 msgr2=0x7f437819c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:07.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.385+0000 7f4380400640 1 --2- 172.21.15.27:0/1236180149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4378105d80 0x7f437819c660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f436c00e9c0 tx=0x7f436c00ee80 comp rx=0 tx=0).stop 2024-09-21T00:18:07.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.385+0000 7f4380400640 1 -- 172.21.15.27:0/1236180149 shutdown_connections 2024-09-21T00:18:07.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.385+0000 7f4380400640 1 --2- 172.21.15.27:0/1236180149 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f43600b65e0 0x7f43600b8af0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:07.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.385+0000 7f4380400640 1 --2- 172.21.15.27:0/1236180149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f437810c010 0x7f43781a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:07.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.385+0000 7f4380400640 1 --2- 172.21.15.27:0/1236180149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f437810b6c0 0x7f437819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:07.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.385+0000 7f4380400640 1 --2- 172.21.15.27:0/1236180149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4378105d80 0x7f437819c660 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:07.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.385+0000 7f4380400640 1 -- 172.21.15.27:0/1236180149 >> 172.21.15.27:0/1236180149 conn(0x7f4378101810 msgr2=0x7f437810acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:07.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.386+0000 7f4380400640 1 -- 172.21.15.27:0/1236180149 shutdown_connections 2024-09-21T00:18:07.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:07.386+0000 7f4380400640 1 -- 172.21.15.27:0/1236180149 wait complete. 2024-09-21T00:18:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:07 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:07 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:07 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:07 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:07 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:07 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:07 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:07 smithi027 ceph-mon[30745]: Deploying daemon mds.f on smithi098 2024-09-21T00:18:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:07 smithi027 ceph-mon[30745]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:07.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:07 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.89:6834/811483028,v1:172.21.15.89:6835/811483028] up:boot 2024-09-21T00:18:07.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:07 smithi027 ceph-mon[30745]: fsmap 5 up:standby 2024-09-21T00:18:07.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:07 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T00:18:07.690 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:07 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:07.690 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:07 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:07.690 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:07 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:07.690 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:07 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:07.690 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:07 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:07.690 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:07 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:07.690 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:07 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:07.690 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:07 smithi098 ceph-mon[34725]: Deploying daemon mds.f on smithi098 2024-09-21T00:18:07.690 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:07 smithi098 ceph-mon[34725]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:07.690 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:07 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.89:6834/811483028,v1:172.21.15.89:6835/811483028] up:boot 2024-09-21T00:18:07.690 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:07 smithi098 ceph-mon[34725]: fsmap 5 up:standby 2024-09-21T00:18:07.691 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:07 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T00:18:07.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:07 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:07 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:07 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:07 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:07 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:07 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:07 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:07 smithi089 ceph-mon[34481]: Deploying daemon mds.f on smithi098 2024-09-21T00:18:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:07 smithi089 ceph-mon[34481]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:07 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.89:6834/811483028,v1:172.21.15.89:6835/811483028] up:boot 2024-09-21T00:18:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:07 smithi089 ceph-mon[34481]: fsmap 5 up:standby 2024-09-21T00:18:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:07 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T00:18:08.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:08 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1236180149' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:08.496 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:18:08 smithi098 systemd[1]: Starting Ceph mds.f for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:18:08.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:08 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1236180149' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:08.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:08 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1236180149' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:08.796 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:18:08 smithi098 podman[63837]: 2024-09-21 00:18:08.417506661 +0000 UTC m=+0.015616960 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:18:08.796 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:18:08 smithi098 podman[63837]: 2024-09-21 00:18:08.531782384 +0000 UTC m=+0.129892729 container create fac7a7a53ad7db4321c4b5e6ca451c68d53836b71b3ad4c9e1413fafb322e9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-f, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T00:18:08.796 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:18:08 smithi098 podman[63837]: 2024-09-21 00:18:08.759550244 +0000 UTC m=+0.357660543 container init fac7a7a53ad7db4321c4b5e6ca451c68d53836b71b3ad4c9e1413fafb322e9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-f, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T00:18:08.796 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:18:08 smithi098 podman[63837]: 2024-09-21 00:18:08.762329582 +0000 UTC m=+0.360439882 container start fac7a7a53ad7db4321c4b5e6ca451c68d53836b71b3ad4c9e1413fafb322e9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-f, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T00:18:09.245 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:18:08 smithi098 ceph-mds[63857]: -- 172.21.15.98:0/3212855312 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555ff85fd860 con 0x555ffc7a5600 2024-09-21T00:18:09.246 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:18:08 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-f[63853]: starting mds.f at 2024-09-21T00:18:09.246 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:18:08 smithi098 bash[63837]: fac7a7a53ad7db4321c4b5e6ca451c68d53836b71b3ad4c9e1413fafb322e9da 2024-09-21T00:18:09.246 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:18:08 smithi098 systemd[1]: Started Ceph mds.f for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:18:10.131 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:18:10.188 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:09 smithi027 ceph-mon[30745]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:10.188 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:09 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:10.188 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:09 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:10.188 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:09 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:10.189 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:09 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:10.189 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:09 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:10.189 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:09 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:10.189 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:09 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:10.189 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:09 smithi027 ceph-mon[30745]: Deploying daemon mds.c on smithi098 2024-09-21T00:18:10.189 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:09 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.98:6834/388589971,v1:172.21.15.98:6835/388589971] up:boot 2024-09-21T00:18:10.189 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:09 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T00:18:10.189 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:09 smithi027 ceph-mon[30745]: fsmap 6 up:standby 2024-09-21T00:18:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:09 smithi089 ceph-mon[34481]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:09 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:09 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:09 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:09 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:09 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:09 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:09 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:09 smithi089 ceph-mon[34481]: Deploying daemon mds.c on smithi098 2024-09-21T00:18:10.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:09 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.98:6834/388589971,v1:172.21.15.98:6835/388589971] up:boot 2024-09-21T00:18:10.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:09 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T00:18:10.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:09 smithi089 ceph-mon[34481]: fsmap 6 up:standby 2024-09-21T00:18:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:09 smithi098 ceph-mon[34725]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:09 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:09 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:09 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:09 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:09 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:09 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:09 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:09 smithi098 ceph-mon[34725]: Deploying daemon mds.c on smithi098 2024-09-21T00:18:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:09 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.98:6834/388589971,v1:172.21.15.98:6835/388589971] up:boot 2024-09-21T00:18:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:09 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T00:18:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:09 smithi098 ceph-mon[34725]: fsmap 6 up:standby 2024-09-21T00:18:10.524 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:10.996 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:18:10 smithi098 systemd[1]: Starting Ceph mds.c for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:18:11.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.155+0000 7f6148a00640 1 -- 172.21.15.27:0/2889104063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6144105bc0 msgr2=0x7f6144105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:11.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.155+0000 7f6148a00640 1 --2- 172.21.15.27:0/2889104063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6144105bc0 0x7f6144105fd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6134009900 tx=0x7f613402a410 comp rx=0 tx=0).stop 2024-09-21T00:18:11.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.155+0000 7f6148a00640 1 -- 172.21.15.27:0/2889104063 shutdown_connections 2024-09-21T00:18:11.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.155+0000 7f6148a00640 1 --2- 172.21.15.27:0/2889104063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f614410be50 0x7f6144112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:11.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.155+0000 7f6148a00640 1 --2- 172.21.15.27:0/2889104063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f614410b500 0x7f614410b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:11.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.155+0000 7f6148a00640 1 --2- 172.21.15.27:0/2889104063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6144105bc0 0x7f6144105fd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:11.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.155+0000 7f6148a00640 1 -- 172.21.15.27:0/2889104063 >> 172.21.15.27:0/2889104063 conn(0x7f6144101610 msgr2=0x7f6144103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:11.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.155+0000 7f6148a00640 1 -- 172.21.15.27:0/2889104063 shutdown_connections 2024-09-21T00:18:11.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.155+0000 7f6148a00640 1 -- 172.21.15.27:0/2889104063 wait complete. 2024-09-21T00:18:11.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.156+0000 7f6148a00640 1 Processor -- start 2024-09-21T00:18:11.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.157+0000 7f6148a00640 1 -- start start 2024-09-21T00:18:11.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.157+0000 7f6148a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6144105bc0 0x7f614419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:11.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.157+0000 7f6148a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f614410b500 0x7f614419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:11.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.157+0000 7f6148a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f614410be50 0x7f61441a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:11.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.157+0000 7f6148a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6144115b60 con 0x7f6144105bc0 2024-09-21T00:18:11.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.157+0000 7f6148a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f61441159e0 con 0x7f614410be50 2024-09-21T00:18:11.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.157+0000 7f6148a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6144115ce0 con 0x7f614410b500 2024-09-21T00:18:11.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.157+0000 7f6141e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f614410b500 0x7f614419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:11.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.157+0000 7f6141e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f614410b500 0x7f614419cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:60216/0 (socket says 172.21.15.27:60216) 2024-09-21T00:18:11.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.157+0000 7f6141e00640 1 -- 172.21.15.27:0/3864654662 learned_addr learned my addr 172.21.15.27:0/3864654662 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:11.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.157+0000 7f6142800640 1 --2- 172.21.15.27:0/3864654662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6144105bc0 0x7f614419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:11.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.157+0000 7f6143200640 1 --2- 172.21.15.27:0/3864654662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f614410be50 0x7f61441a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:11.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.158+0000 7f6141e00640 1 -- 172.21.15.27:0/3864654662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f614410be50 msgr2=0x7f61441a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:11.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.158+0000 7f6141e00640 1 --2- 172.21.15.27:0/3864654662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f614410be50 0x7f61441a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:11.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.158+0000 7f6141e00640 1 -- 172.21.15.27:0/3864654662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6144105bc0 msgr2=0x7f614419c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:11.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.158+0000 7f6141e00640 1 --2- 172.21.15.27:0/3864654662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6144105bc0 0x7f614419c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:11.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.158+0000 7f6141e00640 1 -- 172.21.15.27:0/3864654662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61441a2480 con 0x7f614410b500 2024-09-21T00:18:11.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.158+0000 7f6141e00640 1 --2- 172.21.15.27:0/3864654662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f614410b500 0x7f614419cab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f613000e9c0 tx=0x7f613000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:11.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.158+0000 7f613b400640 1 -- 172.21.15.27:0/3864654662 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613000ccc0 con 0x7f614410b500 2024-09-21T00:18:11.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.158+0000 7f6148a00640 1 -- 172.21.15.27:0/3864654662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61441a2770 con 0x7f614410b500 2024-09-21T00:18:11.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.158+0000 7f613b400640 1 -- 172.21.15.27:0/3864654662 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61300054d0 con 0x7f614410b500 2024-09-21T00:18:11.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.158+0000 7f613b400640 1 -- 172.21.15.27:0/3864654662 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130010430 con 0x7f614410b500 2024-09-21T00:18:11.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.159+0000 7f6148a00640 1 -- 172.21.15.27:0/3864654662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61441a8ed0 con 0x7f614410b500 2024-09-21T00:18:11.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.160+0000 7f6148a00640 1 -- 172.21.15.27:0/3864654662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6144105fd0 con 0x7f614410b500 2024-09-21T00:18:11.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.162+0000 7f613b400640 1 -- 172.21.15.27:0/3864654662 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f6130005020 con 0x7f614410b500 2024-09-21T00:18:11.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.163+0000 7f613b400640 1 --2- 172.21.15.27:0/3864654662 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f61280b6560 0x7f61280b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:11.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.164+0000 7f613b400640 1 -- 172.21.15.27:0/3864654662 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f61300e73c0 con 0x7f614410b500 2024-09-21T00:18:11.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.164+0000 7f6142800640 1 --2- 172.21.15.27:0/3864654662 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f61280b6560 0x7f61280b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:11.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.164+0000 7f6142800640 1 --2- 172.21.15.27:0/3864654662 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f61280b6560 0x7f61280b8a70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6134005ab0 tx=0x7f6134005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:11.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.168+0000 7f613b400640 1 -- 172.21.15.27:0/3864654662 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f61300adf80 con 0x7f614410b500 2024-09-21T00:18:11.289 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:18:11 smithi098 podman[64106]: 2024-09-21 00:18:10.913993998 +0000 UTC m=+0.015996917 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:18:11.289 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:18:11 smithi098 podman[64106]: 2024-09-21 00:18:11.055094926 +0000 UTC m=+0.157097835 container create 9bacc637e50c55d3b16f4a1738e4decb617ef0748b64d8f5ebb12d0565c4423f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-c, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T00:18:11.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.426+0000 7f6148a00640 1 -- 172.21.15.27:0/3864654662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f614402e640 con 0x7f614410b500 2024-09-21T00:18:11.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.427+0000 7f613b400640 1 -- 172.21.15.27:0/3864654662 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5551 (secure 0 0 0) 0x7f61300b2360 con 0x7f614410b500 2024-09-21T00:18:11.429 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:18:11.429 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"btime":"2024-09-21T00:18:09:279513+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24703,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-21T00:18:11.430 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2024-09-21T00:18:11.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.430+0000 7f6148a00640 1 -- 172.21.15.27:0/3864654662 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f61280b6560 msgr2=0x7f61280b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:11.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.430+0000 7f6148a00640 1 --2- 172.21.15.27:0/3864654662 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f61280b6560 0x7f61280b8a70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6134005ab0 tx=0x7f6134005a20 comp rx=0 tx=0).stop 2024-09-21T00:18:11.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.430+0000 7f6148a00640 1 -- 172.21.15.27:0/3864654662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f614410b500 msgr2=0x7f614419cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:11.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.430+0000 7f6148a00640 1 --2- 172.21.15.27:0/3864654662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f614410b500 0x7f614419cab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f613000e9c0 tx=0x7f613000ee80 comp rx=0 tx=0).stop 2024-09-21T00:18:11.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.430+0000 7f6148a00640 1 -- 172.21.15.27:0/3864654662 shutdown_connections 2024-09-21T00:18:11.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.430+0000 7f6148a00640 1 --2- 172.21.15.27:0/3864654662 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f61280b6560 0x7f61280b8a70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:11.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.430+0000 7f6148a00640 1 --2- 172.21.15.27:0/3864654662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f614410be50 0x7f61441a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:11.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.430+0000 7f6148a00640 1 --2- 172.21.15.27:0/3864654662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f614410b500 0x7f614419cab0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:11.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.430+0000 7f6148a00640 1 --2- 172.21.15.27:0/3864654662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6144105bc0 0x7f614419c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:11.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.430+0000 7f6148a00640 1 -- 172.21.15.27:0/3864654662 >> 172.21.15.27:0/3864654662 conn(0x7f6144101610 msgr2=0x7f6144103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:11.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.430+0000 7f6148a00640 1 -- 172.21.15.27:0/3864654662 shutdown_connections 2024-09-21T00:18:11.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:11.430+0000 7f6148a00640 1 -- 172.21.15.27:0/3864654662 wait complete. 2024-09-21T00:18:11.745 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:18:11 smithi098 podman[64106]: 2024-09-21 00:18:11.319316708 +0000 UTC m=+0.421319617 container init 9bacc637e50c55d3b16f4a1738e4decb617ef0748b64d8f5ebb12d0565c4423f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-c, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-21T00:18:11.746 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:18:11 smithi098 podman[64106]: 2024-09-21 00:18:11.322423834 +0000 UTC m=+0.424426745 container start 9bacc637e50c55d3b16f4a1738e4decb617ef0748b64d8f5ebb12d0565c4423f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T00:18:11.746 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:18:11 smithi098 ceph-mds[64125]: -- 172.21.15.98:0/1198536879 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a9e1d5860 con 0x560aa237d600 2024-09-21T00:18:11.746 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:18:11 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-c[64121]: starting mds.c at 2024-09-21T00:18:11.746 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:18:11 smithi098 bash[64106]: 9bacc637e50c55d3b16f4a1738e4decb617ef0748b64d8f5ebb12d0565c4423f 2024-09-21T00:18:11.746 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:18:11 smithi098 systemd[1]: Started Ceph mds.c for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:18:12.183 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:11 smithi089 ceph-mon[34481]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:12.183 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:11 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3864654662' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:12.183 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:11 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:12.183 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:11 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:12.183 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:11 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:12.183 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:11 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:12.183 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:11 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:12.183 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:11 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:12.183 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:11 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:12.215 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:11 smithi027 ceph-mon[30745]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:12.215 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:11 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3864654662' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:12.215 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:11 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:12.215 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:11 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:12.215 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:11 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:12.215 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:11 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:12.215 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:11 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:12.215 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:11 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:12.215 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:11 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:12.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:11 smithi098 ceph-mon[34725]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:12.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:11 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3864654662' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:12.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:11 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:12.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:11 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:12.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:11 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:12.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:11 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:12.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:11 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:12.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:11 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:12.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:11 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:13.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:12 smithi089 ceph-mon[34481]: Deploying daemon mds.h on smithi089 2024-09-21T00:18:13.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:12 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.98:6836/3858073278,v1:172.21.15.98:6837/3858073278] up:boot 2024-09-21T00:18:13.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:12 smithi089 ceph-mon[34481]: fsmap 7 up:standby 2024-09-21T00:18:13.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:12 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T00:18:13.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:12 smithi098 ceph-mon[34725]: Deploying daemon mds.h on smithi089 2024-09-21T00:18:13.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:12 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.98:6836/3858073278,v1:172.21.15.98:6837/3858073278] up:boot 2024-09-21T00:18:13.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:12 smithi098 ceph-mon[34725]: fsmap 7 up:standby 2024-09-21T00:18:13.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:12 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T00:18:13.283 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:12 smithi027 ceph-mon[30745]: Deploying daemon mds.h on smithi089 2024-09-21T00:18:13.283 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:12 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.98:6836/3858073278,v1:172.21.15.98:6837/3858073278] up:boot 2024-09-21T00:18:13.283 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:12 smithi027 ceph-mon[30745]: fsmap 7 up:standby 2024-09-21T00:18:13.283 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:12 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T00:18:13.729 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:18:13 smithi089 systemd[1]: Starting Ceph mds.h for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:18:14.068 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:13 smithi089 ceph-mon[34481]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:14.068 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:18:13 smithi089 podman[62825]: 2024-09-21 00:18:13.653914566 +0000 UTC m=+0.015786259 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:18:14.068 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:18:13 smithi089 podman[62825]: 2024-09-21 00:18:13.782054796 +0000 UTC m=+0.143926513 container create 3483ea512a72f742480e7b141d9e39827b3ce214a614ed3154e2a8176ec9e254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:18:14.068 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:18:14 smithi089 podman[62825]: 2024-09-21 00:18:14.067555531 +0000 UTC m=+0.429427207 container init 3483ea512a72f742480e7b141d9e39827b3ce214a614ed3154e2a8176ec9e254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T00:18:14.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:13 smithi098 ceph-mon[34725]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:14.255 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:18:14.307 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:13 smithi027 ceph-mon[30745]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:14.478 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:18:14 smithi089 podman[62825]: 2024-09-21 00:18:14.070574057 +0000 UTC m=+0.432445732 container start 3483ea512a72f742480e7b141d9e39827b3ce214a614ed3154e2a8176ec9e254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T00:18:14.478 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:18:14 smithi089 ceph-mds[62844]: -- 172.21.15.89:0/3778622218 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d4f7da7860 con 0x55d4fbf4fa80 2024-09-21T00:18:14.479 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:18:14 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h[62840]: starting mds.h at 2024-09-21T00:18:14.479 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:18:14 smithi089 bash[62825]: 3483ea512a72f742480e7b141d9e39827b3ce214a614ed3154e2a8176ec9e254 2024-09-21T00:18:14.479 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:18:14 smithi089 systemd[1]: Started Ceph mds.h for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:18:14.643 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:15.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.266+0000 7f3b71c00640 1 -- 172.21.15.27:0/1485277165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b6c114740 msgr2=0x7f3b6c116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.266+0000 7f3b71c00640 1 --2- 172.21.15.27:0/1485277165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b6c114740 0x7f3b6c116b60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009be0 tx=0x7f3b6402a6c0 comp rx=0 tx=0).stop 2024-09-21T00:18:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.266+0000 7f3b71c00640 1 -- 172.21.15.27:0/1485277165 shutdown_connections 2024-09-21T00:18:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.266+0000 7f3b71c00640 1 --2- 172.21.15.27:0/1485277165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b6c114740 0x7f3b6c116b60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.266+0000 7f3b71c00640 1 --2- 172.21.15.27:0/1485277165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b6c078290 0x7f3b6c114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.266+0000 7f3b71c00640 1 --2- 172.21.15.27:0/1485277165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b6c077940 0x7f3b6c077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.266+0000 7f3b71c00640 1 -- 172.21.15.27:0/1485277165 >> 172.21.15.27:0/1485277165 conn(0x7f3b6c101810 msgr2=0x7f3b6c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.266+0000 7f3b71c00640 1 -- 172.21.15.27:0/1485277165 shutdown_connections 2024-09-21T00:18:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.266+0000 7f3b71c00640 1 -- 172.21.15.27:0/1485277165 wait complete. 2024-09-21T00:18:15.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.267+0000 7f3b71c00640 1 Processor -- start 2024-09-21T00:18:15.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.267+0000 7f3b71c00640 1 -- start start 2024-09-21T00:18:15.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.267+0000 7f3b71c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b6c077940 0x7f3b6c1a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:15.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.267+0000 7f3b71c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b6c078290 0x7f3b6c1a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:15.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.267+0000 7f3b71c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b6c114740 0x7f3b6c1a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:15.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.267+0000 7f3b71c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c11a0b0 con 0x7f3b6c078290 2024-09-21T00:18:15.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.267+0000 7f3b71c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c119f30 con 0x7f3b6c077940 2024-09-21T00:18:15.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.267+0000 7f3b71c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c11a230 con 0x7f3b6c114740 2024-09-21T00:18:15.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.268+0000 7f3b6aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b6c077940 0x7f3b6c1a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:15.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.268+0000 7f3b6aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b6c077940 0x7f3b6c1a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:48390/0 (socket says 172.21.15.27:48390) 2024-09-21T00:18:15.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.268+0000 7f3b6aa00640 1 -- 172.21.15.27:0/2690146992 learned_addr learned my addr 172.21.15.27:0/2690146992 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:15.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.268+0000 7f3b6a000640 1 --2- 172.21.15.27:0/2690146992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b6c078290 0x7f3b6c1a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:15.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.268+0000 7f3b6b400640 1 --2- 172.21.15.27:0/2690146992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b6c114740 0x7f3b6c1a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:15.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.268+0000 7f3b6aa00640 1 -- 172.21.15.27:0/2690146992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b6c114740 msgr2=0x7f3b6c1a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:15.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.268+0000 7f3b6aa00640 1 --2- 172.21.15.27:0/2690146992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b6c114740 0x7f3b6c1a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:15.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.268+0000 7f3b6aa00640 1 -- 172.21.15.27:0/2690146992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b6c078290 msgr2=0x7f3b6c1a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:15.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.268+0000 7f3b6aa00640 1 --2- 172.21.15.27:0/2690146992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b6c078290 0x7f3b6c1a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:15.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.268+0000 7f3b6aa00640 1 -- 172.21.15.27:0/2690146992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6c1a6a30 con 0x7f3b6c077940 2024-09-21T00:18:15.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.268+0000 7f3b6aa00640 1 --2- 172.21.15.27:0/2690146992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b6c077940 0x7f3b6c1a0ac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00e9c0 tx=0x7f3b5c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:15.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.269+0000 7f3b63400640 1 -- 172.21.15.27:0/2690146992 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00cde0 con 0x7f3b6c077940 2024-09-21T00:18:15.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.269+0000 7f3b71c00640 1 -- 172.21.15.27:0/2690146992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6c1a6d20 con 0x7f3b6c077940 2024-09-21T00:18:15.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.269+0000 7f3b63400640 1 -- 172.21.15.27:0/2690146992 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b5c0035f0 con 0x7f3b6c077940 2024-09-21T00:18:15.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.269+0000 7f3b63400640 1 -- 172.21.15.27:0/2690146992 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c010690 con 0x7f3b6c077940 2024-09-21T00:18:15.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.269+0000 7f3b71c00640 1 -- 172.21.15.27:0/2690146992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b6c1ad480 con 0x7f3b6c077940 2024-09-21T00:18:15.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.270+0000 7f3b71c00640 1 -- 172.21.15.27:0/2690146992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b6c077d50 con 0x7f3b6c077940 2024-09-21T00:18:15.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.272+0000 7f3b63400640 1 -- 172.21.15.27:0/2690146992 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f3b5c004000 con 0x7f3b6c077940 2024-09-21T00:18:15.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.274+0000 7f3b63400640 1 --2- 172.21.15.27:0/2690146992 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3b500b6560 0x7f3b500b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.274+0000 7f3b63400640 1 -- 172.21.15.27:0/2690146992 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f3b5c0e7360 con 0x7f3b6c077940 2024-09-21T00:18:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.274+0000 7f3b6a000640 1 --2- 172.21.15.27:0/2690146992 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3b500b6560 0x7f3b500b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.275+0000 7f3b6a000640 1 --2- 172.21.15.27:0/2690146992 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3b500b6560 0x7f3b500b8a70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c07a1e0 tx=0x7f3b58005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:15.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.278+0000 7f3b63400640 1 -- 172.21.15.27:0/2690146992 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3b5c009510 con 0x7f3b6c077940 2024-09-21T00:18:15.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:15 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:15.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:15 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:15.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:15 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:15.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:15 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:15.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:15 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:15.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:15 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:15.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:15 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:15.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:15 smithi089 ceph-mon[34481]: Deploying daemon mds.i on smithi098 2024-09-21T00:18:15.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:15 smithi089 ceph-mon[34481]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:15.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:15 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.89:6836/3044996985,v1:172.21.15.89:6837/3044996985] up:boot 2024-09-21T00:18:15.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:15 smithi089 ceph-mon[34481]: fsmap 8 up:standby 2024-09-21T00:18:15.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:15 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T00:18:15.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:15 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:15.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:15 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:15.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:15 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:15.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:15 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:15.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:15 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:15.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:15 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:15.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:15 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:15.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:15 smithi098 ceph-mon[34725]: Deploying daemon mds.i on smithi098 2024-09-21T00:18:15.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:15 smithi098 ceph-mon[34725]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:15.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:15 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.89:6836/3044996985,v1:172.21.15.89:6837/3044996985] up:boot 2024-09-21T00:18:15.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:15 smithi098 ceph-mon[34725]: fsmap 8 up:standby 2024-09-21T00:18:15.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:15 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T00:18:15.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.519+0000 7f3b71c00640 1 -- 172.21.15.27:0/2690146992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b6c10f930 con 0x7f3b6c077940 2024-09-21T00:18:15.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:15 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:15.521 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:15 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:15.521 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:15 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:15.521 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:15 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:15.521 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:15 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:15.521 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:15 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:15.521 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:15 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:15.521 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:15 smithi027 ceph-mon[30745]: Deploying daemon mds.i on smithi098 2024-09-21T00:18:15.521 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:15 smithi027 ceph-mon[30745]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:15.521 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:15 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.89:6836/3044996985,v1:172.21.15.89:6837/3044996985] up:boot 2024-09-21T00:18:15.521 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:15 smithi027 ceph-mon[30745]: fsmap 8 up:standby 2024-09-21T00:18:15.521 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:15 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T00:18:15.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.520+0000 7f3b63400640 1 -- 172.21.15.27:0/2690146992 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7201 (secure 0 0 0) 0x7f3b5c0adf20 con 0x7f3b6c077940 2024-09-21T00:18:15.522 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:18:15.522 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":9,"btime":"2024-09-21T00:18:14:928552+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24703,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24715,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24722,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T00:18:15.522 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 9 2024-09-21T00:18:15.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.523+0000 7f3b71c00640 1 -- 172.21.15.27:0/2690146992 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3b500b6560 msgr2=0x7f3b500b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:15.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.523+0000 7f3b71c00640 1 --2- 172.21.15.27:0/2690146992 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3b500b6560 0x7f3b500b8a70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c07a1e0 tx=0x7f3b58005f90 comp rx=0 tx=0).stop 2024-09-21T00:18:15.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.523+0000 7f3b71c00640 1 -- 172.21.15.27:0/2690146992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b6c077940 msgr2=0x7f3b6c1a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:15.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.523+0000 7f3b71c00640 1 --2- 172.21.15.27:0/2690146992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b6c077940 0x7f3b6c1a0ac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00e9c0 tx=0x7f3b5c00ee80 comp rx=0 tx=0).stop 2024-09-21T00:18:15.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.523+0000 7f3b71c00640 1 -- 172.21.15.27:0/2690146992 shutdown_connections 2024-09-21T00:18:15.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.523+0000 7f3b71c00640 1 --2- 172.21.15.27:0/2690146992 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3b500b6560 0x7f3b500b8a70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:15.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.523+0000 7f3b71c00640 1 --2- 172.21.15.27:0/2690146992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b6c114740 0x7f3b6c1a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:15.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.523+0000 7f3b71c00640 1 --2- 172.21.15.27:0/2690146992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b6c078290 0x7f3b6c1a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:15.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.523+0000 7f3b71c00640 1 --2- 172.21.15.27:0/2690146992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b6c077940 0x7f3b6c1a0ac0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:15.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.523+0000 7f3b71c00640 1 -- 172.21.15.27:0/2690146992 >> 172.21.15.27:0/2690146992 conn(0x7f3b6c101810 msgr2=0x7f3b6c116750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:15.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.523+0000 7f3b71c00640 1 -- 172.21.15.27:0/2690146992 shutdown_connections 2024-09-21T00:18:15.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:15.523+0000 7f3b71c00640 1 -- 172.21.15.27:0/2690146992 wait complete. 2024-09-21T00:18:16.245 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:18:16 smithi098 systemd[1]: Starting Ceph mds.i for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:18:16.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:16 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2690146992' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:16.590 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:16 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2690146992' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:16.590 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:18:16 smithi098 podman[64374]: 2024-09-21 00:18:16.23301377 +0000 UTC m=+0.015856639 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:18:16.591 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:18:16 smithi098 podman[64374]: 2024-09-21 00:18:16.362099217 +0000 UTC m=+0.144942077 container create 6d1c23916cc1da5cfe00dec7332208f9bb400324cd546e3b7da6068111e05e8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-i, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T00:18:16.591 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:18:16 smithi098 podman[64374]: 2024-09-21 00:18:16.590244813 +0000 UTC m=+0.373087672 container init 6d1c23916cc1da5cfe00dec7332208f9bb400324cd546e3b7da6068111e05e8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-i, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T00:18:16.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:16 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2690146992' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:16.996 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:18:16 smithi098 podman[64374]: 2024-09-21 00:18:16.593101044 +0000 UTC m=+0.375943903 container start 6d1c23916cc1da5cfe00dec7332208f9bb400324cd546e3b7da6068111e05e8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-i, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:18:16.996 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:18:16 smithi098 ceph-mds[64393]: -- 172.21.15.98:0/4085067848 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c3060ad860 con 0x55c30a255a80 2024-09-21T00:18:16.996 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:18:16 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-i[64389]: starting mds.i at 2024-09-21T00:18:16.996 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:18:16 smithi098 bash[64374]: 6d1c23916cc1da5cfe00dec7332208f9bb400324cd546e3b7da6068111e05e8a 2024-09-21T00:18:16.996 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:18:16 smithi098 systemd[1]: Started Ceph mds.i for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:18:17.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:17 smithi098 ceph-mon[34725]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:17.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:17 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:17.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:17 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:17.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:17 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:17.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:17 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:17.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:17 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:17.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:17 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:17.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:17 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:17.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:17 smithi098 ceph-mon[34725]: Deploying daemon mds.g on smithi027 2024-09-21T00:18:17.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:17 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] up:boot 2024-09-21T00:18:17.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:17 smithi098 ceph-mon[34725]: fsmap 9 up:standby 2024-09-21T00:18:17.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:17 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T00:18:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:17 smithi027 ceph-mon[30745]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:17 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:17 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:17 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:17 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:17 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:17 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:17 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:17 smithi027 ceph-mon[30745]: Deploying daemon mds.g on smithi027 2024-09-21T00:18:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:17 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] up:boot 2024-09-21T00:18:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:17 smithi027 ceph-mon[30745]: fsmap 9 up:standby 2024-09-21T00:18:18.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:17 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T00:18:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:17 smithi089 ceph-mon[34481]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:17 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:17 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:17 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:17 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:17 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:17 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:17 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:17 smithi089 ceph-mon[34481]: Deploying daemon mds.g on smithi027 2024-09-21T00:18:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:17 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] up:boot 2024-09-21T00:18:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:17 smithi089 ceph-mon[34481]: fsmap 9 up:standby 2024-09-21T00:18:18.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:17 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T00:18:18.343 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:18:18.762 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:19.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.567+0000 7fc72a000640 1 -- 172.21.15.27:0/3744643641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc724105bc0 msgr2=0x7fc724105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:19.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.567+0000 7fc72a000640 1 --2- 172.21.15.27:0/3744643641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc724105bc0 0x7fc724105fd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc718009900 tx=0x7fc71802a410 comp rx=0 tx=0).stop 2024-09-21T00:18:19.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.567+0000 7fc72a000640 1 -- 172.21.15.27:0/3744643641 shutdown_connections 2024-09-21T00:18:19.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.567+0000 7fc72a000640 1 --2- 172.21.15.27:0/3744643641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc72410be50 0x7fc724112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:19.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.567+0000 7fc72a000640 1 --2- 172.21.15.27:0/3744643641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc72410b500 0x7fc72410b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:19.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.567+0000 7fc72a000640 1 --2- 172.21.15.27:0/3744643641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc724105bc0 0x7fc724105fd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:19.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.567+0000 7fc72a000640 1 -- 172.21.15.27:0/3744643641 >> 172.21.15.27:0/3744643641 conn(0x7fc724101670 msgr2=0x7fc724103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:19.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.567+0000 7fc72a000640 1 -- 172.21.15.27:0/3744643641 shutdown_connections 2024-09-21T00:18:19.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.567+0000 7fc72a000640 1 -- 172.21.15.27:0/3744643641 wait complete. 2024-09-21T00:18:19.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.569+0000 7fc72a000640 1 Processor -- start 2024-09-21T00:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.569+0000 7fc72a000640 1 -- start start 2024-09-21T00:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.569+0000 7fc72a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc724105bc0 0x7fc72419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.569+0000 7fc72a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc72410b500 0x7fc72419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.569+0000 7fc72a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc72410be50 0x7fc7241a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.569+0000 7fc72a000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc724115b60 con 0x7fc72410be50 2024-09-21T00:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.569+0000 7fc72a000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc7241159e0 con 0x7fc72410b500 2024-09-21T00:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.569+0000 7fc72a000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc724115ce0 con 0x7fc724105bc0 2024-09-21T00:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.570+0000 7fc729600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc72410be50 0x7fc7241a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.570+0000 7fc728c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc724105bc0 0x7fc72419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:19.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.570+0000 7fc723e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc72410b500 0x7fc72419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:19.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.570+0000 7fc728c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc724105bc0 0x7fc72419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:57858/0 (socket says 172.21.15.27:57858) 2024-09-21T00:18:19.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.570+0000 7fc729600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc72410be50 0x7fc7241a1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39738/0 (socket says 172.21.15.27:39738) 2024-09-21T00:18:19.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.570+0000 7fc723e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc72410b500 0x7fc72419caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:48418/0 (socket says 172.21.15.27:48418) 2024-09-21T00:18:19.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.570+0000 7fc728c00640 1 -- 172.21.15.27:0/3428610923 learned_addr learned my addr 172.21.15.27:0/3428610923 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.571+0000 7fc729600640 1 -- 172.21.15.27:0/3428610923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc724105bc0 msgr2=0x7fc72419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.571+0000 7fc729600640 1 --2- 172.21.15.27:0/3428610923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc724105bc0 0x7fc72419c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.571+0000 7fc729600640 1 -- 172.21.15.27:0/3428610923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc72410b500 msgr2=0x7fc72419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.571+0000 7fc729600640 1 --2- 172.21.15.27:0/3428610923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc72410b500 0x7fc72419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.571+0000 7fc729600640 1 -- 172.21.15.27:0/3428610923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7241a24c0 con 0x7fc72410be50 2024-09-21T00:18:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.571+0000 7fc723e00640 1 --2- 172.21.15.27:0/3428610923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc72410b500 0x7fc72419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:18:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.571+0000 7fc729600640 1 --2- 172.21.15.27:0/3428610923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc72410be50 0x7fc7241a1d90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00ec50 tx=0x7fc71c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.571+0000 7fc721600640 1 -- 172.21.15.27:0/3428610923 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c003b70 con 0x7fc72410be50 2024-09-21T00:18:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.571+0000 7fc72a000640 1 -- 172.21.15.27:0/3428610923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7241a27b0 con 0x7fc72410be50 2024-09-21T00:18:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.572+0000 7fc721600640 1 -- 172.21.15.27:0/3428610923 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc71c003d10 con 0x7fc72410be50 2024-09-21T00:18:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.572+0000 7fc721600640 1 -- 172.21.15.27:0/3428610923 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c010830 con 0x7fc72410be50 2024-09-21T00:18:19.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.572+0000 7fc72a000640 1 -- 172.21.15.27:0/3428610923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7241a8f10 con 0x7fc72410be50 2024-09-21T00:18:19.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.573+0000 7fc72a000640 1 -- 172.21.15.27:0/3428610923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc724105fd0 con 0x7fc72410be50 2024-09-21T00:18:19.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.576+0000 7fc721600640 1 -- 172.21.15.27:0/3428610923 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fc71c003ec0 con 0x7fc72410be50 2024-09-21T00:18:19.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.577+0000 7fc721600640 1 --2- 172.21.15.27:0/3428610923 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc70c0b6660 0x7fc70c0b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:19.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.578+0000 7fc721600640 1 -- 172.21.15.27:0/3428610923 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fc71c0e7520 con 0x7fc72410be50 2024-09-21T00:18:19.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.578+0000 7fc728c00640 1 --2- 172.21.15.27:0/3428610923 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc70c0b6660 0x7fc70c0b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:19.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.579+0000 7fc728c00640 1 --2- 172.21.15.27:0/3428610923 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc70c0b6660 0x7fc70c0b8b70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc7180096c0 tx=0x7fc718005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:19.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.583+0000 7fc721600640 1 -- 172.21.15.27:0/3428610923 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc71c0ae0e0 con 0x7fc72410be50 2024-09-21T00:18:19.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.835+0000 7fc72a000640 1 -- 172.21.15.27:0/3428610923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc72402e640 con 0x7fc72410be50 2024-09-21T00:18:19.837 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:19 smithi027 ceph-mon[30745]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:19.837 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 00:18:19 smithi027 podman[70433]: 2024-09-21 00:18:19.769068021 +0000 UTC m=+0.167064920 container create a57428cef61dca59ef0b5f18c2971d9fef7b75bf1dff85554be4618522328b16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-g, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T00:18:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.836+0000 7fc721600640 1 -- 172.21.15.27:0/3428610923 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8028 (secure 0 0 0) 0x7fc71c0b24c0 con 0x7fc72410be50 2024-09-21T00:18:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:18:19.838 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":10,"btime":"2024-09-21T00:18:17:252313+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24703,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24715,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24722,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24727,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T00:18:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 10 2024-09-21T00:18:19.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.839+0000 7fc72a000640 1 -- 172.21.15.27:0/3428610923 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc70c0b6660 msgr2=0x7fc70c0b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:19.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.839+0000 7fc72a000640 1 --2- 172.21.15.27:0/3428610923 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc70c0b6660 0x7fc70c0b8b70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc7180096c0 tx=0x7fc718005a20 comp rx=0 tx=0).stop 2024-09-21T00:18:19.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.839+0000 7fc72a000640 1 -- 172.21.15.27:0/3428610923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc72410be50 msgr2=0x7fc7241a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:19.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.839+0000 7fc72a000640 1 --2- 172.21.15.27:0/3428610923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc72410be50 0x7fc7241a1d90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00ec50 tx=0x7fc71c00c6a0 comp rx=0 tx=0).stop 2024-09-21T00:18:19.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.839+0000 7fc72a000640 1 -- 172.21.15.27:0/3428610923 shutdown_connections 2024-09-21T00:18:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.839+0000 7fc72a000640 1 --2- 172.21.15.27:0/3428610923 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc70c0b6660 0x7fc70c0b8b70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.839+0000 7fc72a000640 1 --2- 172.21.15.27:0/3428610923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc72410be50 0x7fc7241a1d90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.839+0000 7fc72a000640 1 --2- 172.21.15.27:0/3428610923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc72410b500 0x7fc72419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.839+0000 7fc72a000640 1 --2- 172.21.15.27:0/3428610923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc724105bc0 0x7fc72419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.839+0000 7fc72a000640 1 -- 172.21.15.27:0/3428610923 >> 172.21.15.27:0/3428610923 conn(0x7fc724101670 msgr2=0x7fc724103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.839+0000 7fc72a000640 1 -- 172.21.15.27:0/3428610923 shutdown_connections 2024-09-21T00:18:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:19.839+0000 7fc72a000640 1 -- 172.21.15.27:0/3428610923 wait complete. 2024-09-21T00:18:19.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:19 smithi098 ceph-mon[34725]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:19 smithi089 ceph-mon[34481]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:20.794 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 00:18:20 smithi027 podman[70433]: 2024-09-21 00:18:20.685875403 +0000 UTC m=+1.083872324 container init a57428cef61dca59ef0b5f18c2971d9fef7b75bf1dff85554be4618522328b16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-g, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:18:20.795 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 00:18:20 smithi027 podman[70433]: 2024-09-21 00:18:20.692305157 +0000 UTC m=+1.090302050 container start a57428cef61dca59ef0b5f18c2971d9fef7b75bf1dff85554be4618522328b16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-g, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T00:18:20.795 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:20 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3428610923' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:21.067 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 00:18:20 smithi027 bash[70433]: a57428cef61dca59ef0b5f18c2971d9fef7b75bf1dff85554be4618522328b16 2024-09-21T00:18:21.067 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 00:18:20 smithi027 systemd[1]: Started Ceph mds.g for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:18:21.067 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 00:18:20 smithi027 ceph-mds[70464]: -- 172.21.15.27:0/4002020425 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55acbaf8f860 con 0x55acbf137600 2024-09-21T00:18:21.067 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 00:18:20 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-g[70459]: starting mds.g at 2024-09-21T00:18:21.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:20 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3428610923' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:21.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:20 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3428610923' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:22.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:21 smithi027 ceph-mon[30745]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:22.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:21 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:22.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:21 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:22.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:21 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:22.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:21 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:22.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:21 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:22.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:21 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:22.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:21 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:22.071 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:21 smithi027 ceph-mon[30745]: Deploying daemon mds.a on smithi027 2024-09-21T00:18:22.071 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:21 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.27:6838/3523607333,v1:172.21.15.27:6839/3523607333] up:boot 2024-09-21T00:18:22.071 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:21 smithi027 ceph-mon[30745]: fsmap 10 up:standby 2024-09-21T00:18:22.071 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:21 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T00:18:22.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:21 smithi089 ceph-mon[34481]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:22.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:21 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:22.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:21 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:22.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:21 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:22.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:21 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:22.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:21 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:22.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:21 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:22.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:21 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:22.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:21 smithi089 ceph-mon[34481]: Deploying daemon mds.a on smithi027 2024-09-21T00:18:22.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:21 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.27:6838/3523607333,v1:172.21.15.27:6839/3523607333] up:boot 2024-09-21T00:18:22.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:21 smithi089 ceph-mon[34481]: fsmap 10 up:standby 2024-09-21T00:18:22.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:21 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T00:18:22.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:21 smithi098 ceph-mon[34725]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:21 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:21 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:21 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:21 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:21 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:21 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:21 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:21 smithi098 ceph-mon[34725]: Deploying daemon mds.a on smithi027 2024-09-21T00:18:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:21 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.27:6838/3523607333,v1:172.21.15.27:6839/3523607333] up:boot 2024-09-21T00:18:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:21 smithi098 ceph-mon[34725]: fsmap 10 up:standby 2024-09-21T00:18:22.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:21 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T00:18:22.584 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:18:23.002 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:23.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.682+0000 7f4c5be00640 1 -- 172.21.15.27:0/3133895382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c54109df0 msgr2=0x7f4c54110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:23.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.682+0000 7f4c5be00640 1 --2- 172.21.15.27:0/3133895382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c54109df0 0x7f4c54110620 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009900 tx=0x7f4c4c02a410 comp rx=0 tx=0).stop 2024-09-21T00:18:23.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.682+0000 7f4c5be00640 1 -- 172.21.15.27:0/3133895382 shutdown_connections 2024-09-21T00:18:23.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.682+0000 7f4c5be00640 1 --2- 172.21.15.27:0/3133895382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c54109df0 0x7f4c54110620 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:23.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.682+0000 7f4c5be00640 1 --2- 172.21.15.27:0/3133895382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c541094a0 0x7f4c541098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:23.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.682+0000 7f4c5be00640 1 --2- 172.21.15.27:0/3133895382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c54103b60 0x7f4c54103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:23.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.682+0000 7f4c5be00640 1 -- 172.21.15.27:0/3133895382 >> 172.21.15.27:0/3133895382 conn(0x7f4c5407b850 msgr2=0x7f4c5407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:23.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.682+0000 7f4c5be00640 1 -- 172.21.15.27:0/3133895382 shutdown_connections 2024-09-21T00:18:23.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.682+0000 7f4c5be00640 1 -- 172.21.15.27:0/3133895382 wait complete. 2024-09-21T00:18:23.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.684+0000 7f4c5be00640 1 Processor -- start 2024-09-21T00:18:23.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.684+0000 7f4c5be00640 1 -- start start 2024-09-21T00:18:23.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.685+0000 7f4c5be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c54103b60 0x7f4c5419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:23.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.685+0000 7f4c5be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c541094a0 0x7f4c5419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:23.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.685+0000 7f4c5be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c54109df0 0x7f4c5419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:23.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.685+0000 7f4c5be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c54113b00 con 0x7f4c54109df0 2024-09-21T00:18:23.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.685+0000 7f4c5be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4c54113980 con 0x7f4c54103b60 2024-09-21T00:18:23.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.685+0000 7f4c5be00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4c54113c80 con 0x7f4c541094a0 2024-09-21T00:18:23.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.685+0000 7f4c59400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c54103b60 0x7f4c5419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:23.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.685+0000 7f4c59400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c54103b60 0x7f4c5419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57806/0 (socket says 172.21.15.27:57806) 2024-09-21T00:18:23.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.685+0000 7f4c59e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c54109df0 0x7f4c5419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:23.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.685+0000 7f4c59400640 1 -- 172.21.15.27:0/1234418208 learned_addr learned my addr 172.21.15.27:0/1234418208 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:23.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.685+0000 7f4c58a00640 1 --2- 172.21.15.27:0/1234418208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c541094a0 0x7f4c5419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:23.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.686+0000 7f4c59400640 1 -- 172.21.15.27:0/1234418208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c541094a0 msgr2=0x7f4c5419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:23.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.686+0000 7f4c59400640 1 --2- 172.21.15.27:0/1234418208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c541094a0 0x7f4c5419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:23.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.686+0000 7f4c59400640 1 -- 172.21.15.27:0/1234418208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c54109df0 msgr2=0x7f4c5419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:23.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.686+0000 7f4c59400640 1 --2- 172.21.15.27:0/1234418208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c54109df0 0x7f4c5419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:23.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.686+0000 7f4c59400640 1 -- 172.21.15.27:0/1234418208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c541a0380 con 0x7f4c54103b60 2024-09-21T00:18:23.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.686+0000 7f4c59e00640 1 --2- 172.21.15.27:0/1234418208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c54109df0 0x7f4c5419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:18:23.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.686+0000 7f4c59400640 1 --2- 172.21.15.27:0/1234418208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c54103b60 0x7f4c5419a410 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800ebc0 tx=0x7f4c4800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:23.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.686+0000 7f4c52000640 1 -- 172.21.15.27:0/1234418208 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48010630 con 0x7f4c54103b60 2024-09-21T00:18:23.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.687+0000 7f4c52000640 1 -- 172.21.15.27:0/1234418208 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c48010c80 con 0x7f4c54103b60 2024-09-21T00:18:23.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.687+0000 7f4c5be00640 1 -- 172.21.15.27:0/1234418208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c541a0670 con 0x7f4c54103b60 2024-09-21T00:18:23.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.687+0000 7f4c52000640 1 -- 172.21.15.27:0/1234418208 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48018b60 con 0x7f4c54103b60 2024-09-21T00:18:23.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.687+0000 7f4c5be00640 1 -- 172.21.15.27:0/1234418208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c540755f0 con 0x7f4c54103b60 2024-09-21T00:18:23.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.688+0000 7f4c5be00640 1 -- 172.21.15.27:0/1234418208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c54103f70 con 0x7f4c54103b60 2024-09-21T00:18:23.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.691+0000 7f4c52000640 1 -- 172.21.15.27:0/1234418208 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f4c48018d00 con 0x7f4c54103b60 2024-09-21T00:18:23.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.692+0000 7f4c52000640 1 --2- 172.21.15.27:0/1234418208 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4c3c0b6560 0x7f4c3c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:23.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.693+0000 7f4c52000640 1 -- 172.21.15.27:0/1234418208 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f4c480e7900 con 0x7f4c54103b60 2024-09-21T00:18:23.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.693+0000 7f4c58a00640 1 --2- 172.21.15.27:0/1234418208 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4c3c0b6560 0x7f4c3c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:23.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.694+0000 7f4c58a00640 1 --2- 172.21.15.27:0/1234418208 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4c3c0b6560 0x7f4c3c0b8a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4c54109110 tx=0x7f4c40005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:23.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.694+0000 7f4c52000640 1 -- 172.21.15.27:0/1234418208 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c480ae4c0 con 0x7f4c54103b60 2024-09-21T00:18:23.937 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:23 smithi027 ceph-mon[30745]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:23.937 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:18:23 smithi027 podman[70805]: 2024-09-21 00:18:23.797052619 +0000 UTC m=+0.034246645 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:18:23.938 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:18:23 smithi027 podman[70805]: 2024-09-21 00:18:23.936921856 +0000 UTC m=+0.174115881 container create c5af4bbf1b58f760a8d77456a01ed693dbfdacbd1ef89e082dd0e4b82a16f810 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T00:18:23.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.971+0000 7f4c5be00640 1 -- 172.21.15.27:0/1234418208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c5419b540 con 0x7f4c54103b60 2024-09-21T00:18:23.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.972+0000 7f4c52000640 1 -- 172.21.15.27:0/1234418208 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8854 (secure 0 0 0) 0x7f4c480b28a0 con 0x7f4c54103b60 2024-09-21T00:18:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:18:23.973 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":11,"btime":"2024-09-21T00:18:21:751002+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24703,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24715,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24722,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24727,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T00:18:23.973 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 11 2024-09-21T00:18:23.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.974+0000 7f4c5be00640 1 -- 172.21.15.27:0/1234418208 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4c3c0b6560 msgr2=0x7f4c3c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:23.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.974+0000 7f4c5be00640 1 --2- 172.21.15.27:0/1234418208 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4c3c0b6560 0x7f4c3c0b8a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4c54109110 tx=0x7f4c40005f90 comp rx=0 tx=0).stop 2024-09-21T00:18:23.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.974+0000 7f4c5be00640 1 -- 172.21.15.27:0/1234418208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c54103b60 msgr2=0x7f4c5419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:23.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.974+0000 7f4c5be00640 1 --2- 172.21.15.27:0/1234418208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c54103b60 0x7f4c5419a410 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800ebc0 tx=0x7f4c4800c6a0 comp rx=0 tx=0).stop 2024-09-21T00:18:23.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.974+0000 7f4c5be00640 1 -- 172.21.15.27:0/1234418208 shutdown_connections 2024-09-21T00:18:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.974+0000 7f4c5be00640 1 --2- 172.21.15.27:0/1234418208 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4c3c0b6560 0x7f4c3c0b8a70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.974+0000 7f4c5be00640 1 --2- 172.21.15.27:0/1234418208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c54109df0 0x7f4c5419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.974+0000 7f4c5be00640 1 --2- 172.21.15.27:0/1234418208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c541094a0 0x7f4c5419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.974+0000 7f4c5be00640 1 --2- 172.21.15.27:0/1234418208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c54103b60 0x7f4c5419a410 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.974+0000 7f4c5be00640 1 -- 172.21.15.27:0/1234418208 >> 172.21.15.27:0/1234418208 conn(0x7f4c5407b850 msgr2=0x7f4c54107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.974+0000 7f4c5be00640 1 -- 172.21.15.27:0/1234418208 shutdown_connections 2024-09-21T00:18:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:23.974+0000 7f4c5be00640 1 -- 172.21.15.27:0/1234418208 wait complete. 2024-09-21T00:18:24.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:23 smithi089 ceph-mon[34481]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:24.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:23 smithi098 ceph-mon[34725]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:24.567 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:18:24 smithi027 podman[70805]: 2024-09-21 00:18:24.501143004 +0000 UTC m=+0.738337042 container init c5af4bbf1b58f760a8d77456a01ed693dbfdacbd1ef89e082dd0e4b82a16f810 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T00:18:24.567 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:18:24 smithi027 podman[70805]: 2024-09-21 00:18:24.507612995 +0000 UTC m=+0.744807027 container start c5af4bbf1b58f760a8d77456a01ed693dbfdacbd1ef89e082dd0e4b82a16f810 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-a, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T00:18:24.887 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:18:24 smithi027 bash[70805]: c5af4bbf1b58f760a8d77456a01ed693dbfdacbd1ef89e082dd0e4b82a16f810 2024-09-21T00:18:24.887 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:18:24 smithi027 systemd[1]: Started Ceph mds.a for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:18:24.887 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:18:24 smithi027 ceph-mds[70838]: -- 172.21.15.27:0/2708083626 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55749c3b7860 con 0x5574a055fa80 2024-09-21T00:18:24.887 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:18:24 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-a[70832]: starting mds.a at 2024-09-21T00:18:25.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:24 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1234418208' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:25.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:25.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:25.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:25.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:25.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:25.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:25.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:25.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:24 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1234418208' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:25.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:25.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:25.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:25.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:25.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:25.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:25.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:25.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:24 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1234418208' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:25.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:25.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:25.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:25.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:25.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T00:18:25.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T00:18:25.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:26.151 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:25 smithi089 ceph-mon[34481]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:26.151 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:25 smithi089 ceph-mon[34481]: Deploying daemon mds.b on smithi089 2024-09-21T00:18:26.151 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:25 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T00:18:26.151 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:25 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] up:boot 2024-09-21T00:18:26.151 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:25 smithi089 ceph-mon[34481]: fsmap 11 up:standby 2024-09-21T00:18:26.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:25 smithi098 ceph-mon[34725]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:25 smithi098 ceph-mon[34725]: Deploying daemon mds.b on smithi089 2024-09-21T00:18:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:25 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T00:18:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:25 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] up:boot 2024-09-21T00:18:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:25 smithi098 ceph-mon[34725]: fsmap 11 up:standby 2024-09-21T00:18:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:25 smithi027 ceph-mon[30745]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:25 smithi027 ceph-mon[30745]: Deploying daemon mds.b on smithi089 2024-09-21T00:18:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:25 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T00:18:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:25 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] up:boot 2024-09-21T00:18:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:25 smithi027 ceph-mon[30745]: fsmap 11 up:standby 2024-09-21T00:18:26.728 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:18:26 smithi089 systemd[1]: Starting Ceph mds.b for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:18:27.042 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:18:27.157 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:18:26 smithi089 podman[63095]: 2024-09-21 00:18:26.732770598 +0000 UTC m=+0.015841223 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:18:27.157 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:18:26 smithi089 podman[63095]: 2024-09-21 00:18:26.862149986 +0000 UTC m=+0.145220607 container create a4b42d6d83682708340db1d7de0b3bd8b309a3f9567daf82bf738f89490983aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-b, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T00:18:27.157 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:18:27 smithi089 podman[63095]: 2024-09-21 00:18:27.156840209 +0000 UTC m=+0.439910820 container init a4b42d6d83682708340db1d7de0b3bd8b309a3f9567daf82bf738f89490983aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:18:27.449 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:27.478 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:18:27 smithi089 podman[63095]: 2024-09-21 00:18:27.159905701 +0000 UTC m=+0.442976317 container start a4b42d6d83682708340db1d7de0b3bd8b309a3f9567daf82bf738f89490983aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:18:27.478 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:18:27 smithi089 ceph-mds[63114]: -- 172.21.15.89:0/3777868098 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e0cb331860 con 0x55e0cf4d9600 2024-09-21T00:18:27.478 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:18:27 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-b[63110]: starting mds.b at 2024-09-21T00:18:27.478 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:18:27 smithi089 bash[63095]: a4b42d6d83682708340db1d7de0b3bd8b309a3f9567daf82bf738f89490983aa 2024-09-21T00:18:27.478 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:18:27 smithi089 systemd[1]: Started Ceph mds.b for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:18:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.125+0000 7fd008c00640 1 -- 172.21.15.27:0/583504081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0041100e0 msgr2=0x7fd004112520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.125+0000 7fd008c00640 1 --2- 172.21.15.27:0/583504081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0041100e0 0x7fd004112520 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009ca0 tx=0x7fcffc02a8c0 comp rx=0 tx=0).stop 2024-09-21T00:18:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.125+0000 7fd008c00640 1 -- 172.21.15.27:0/583504081 shutdown_connections 2024-09-21T00:18:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.125+0000 7fd008c00640 1 --2- 172.21.15.27:0/583504081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0041100e0 0x7fd004112520 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.125+0000 7fd008c00640 1 --2- 172.21.15.27:0/583504081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd004076100 0x7fd00410fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.125+0000 7fd008c00640 1 --2- 172.21.15.27:0/583504081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0040757b0 0x7fd004075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.125+0000 7fd008c00640 1 -- 172.21.15.27:0/583504081 >> 172.21.15.27:0/583504081 conn(0x7fd0040ff590 msgr2=0x7fd0041019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.125+0000 7fd008c00640 1 -- 172.21.15.27:0/583504081 shutdown_connections 2024-09-21T00:18:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.125+0000 7fd008c00640 1 -- 172.21.15.27:0/583504081 wait complete. 2024-09-21T00:18:28.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.126+0000 7fd008c00640 1 Processor -- start 2024-09-21T00:18:28.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.127+0000 7fd008c00640 1 -- start start 2024-09-21T00:18:28.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.127+0000 7fd008c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0040757b0 0x7fd00419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:28.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.127+0000 7fd008c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd004076100 0x7fd00419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:28.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.128+0000 7fd008c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0041100e0 0x7fd0041a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:28.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.128+0000 7fd008c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd004115ce0 con 0x7fd004076100 2024-09-21T00:18:28.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.128+0000 7fd008c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd004115b60 con 0x7fd0041100e0 2024-09-21T00:18:28.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.128+0000 7fd008c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd004115e60 con 0x7fd0040757b0 2024-09-21T00:18:28.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.128+0000 7fd002800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0041100e0 0x7fd0041a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:28.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.128+0000 7fd001400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd004076100 0x7fd00419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:28.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.128+0000 7fd002800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0041100e0 0x7fd0041a1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57824/0 (socket says 172.21.15.27:57824) 2024-09-21T00:18:28.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.128+0000 7fd002800640 1 -- 172.21.15.27:0/1176063532 learned_addr learned my addr 172.21.15.27:0/1176063532 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:28.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.128+0000 7fd001e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0040757b0 0x7fd00419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.128+0000 7fd002800640 1 -- 172.21.15.27:0/1176063532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0040757b0 msgr2=0x7fd00419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.128+0000 7fd002800640 1 --2- 172.21.15.27:0/1176063532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0040757b0 0x7fd00419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.128+0000 7fd002800640 1 -- 172.21.15.27:0/1176063532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd004076100 msgr2=0x7fd00419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.128+0000 7fd002800640 1 --2- 172.21.15.27:0/1176063532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd004076100 0x7fd00419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.128+0000 7fd002800640 1 -- 172.21.15.27:0/1176063532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0041a25e0 con 0x7fd0041100e0 2024-09-21T00:18:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.129+0000 7fd002800640 1 --2- 172.21.15.27:0/1176063532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0041100e0 0x7fd0041a1eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcffc02f540 tx=0x7fcffc0316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.129+0000 7fcffaa00640 1 -- 172.21.15.27:0/1176063532 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc003b70 con 0x7fd0041100e0 2024-09-21T00:18:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.129+0000 7fcffaa00640 1 -- 172.21.15.27:0/1176063532 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcffc003d10 con 0x7fd0041100e0 2024-09-21T00:18:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.129+0000 7fcffaa00640 1 -- 172.21.15.27:0/1176063532 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc02f990 con 0x7fd0041100e0 2024-09-21T00:18:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.129+0000 7fd008c00640 1 -- 172.21.15.27:0/1176063532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0041a2870 con 0x7fd0041100e0 2024-09-21T00:18:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.129+0000 7fd008c00640 1 -- 172.21.15.27:0/1176063532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0041a9030 con 0x7fd0041100e0 2024-09-21T00:18:28.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.130+0000 7fd008c00640 1 -- 172.21.15.27:0/1176063532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd004075bc0 con 0x7fd0041100e0 2024-09-21T00:18:28.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.133+0000 7fcffaa00640 1 -- 172.21.15.27:0/1176063532 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fcffc02fb30 con 0x7fd0041100e0 2024-09-21T00:18:28.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.134+0000 7fcffaa00640 1 --2- 172.21.15.27:0/1176063532 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcfe80b65e0 0x7fcfe80b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:28.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.134+0000 7fcffaa00640 1 -- 172.21.15.27:0/1176063532 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fcffc108320 con 0x7fd0041100e0 2024-09-21T00:18:28.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.135+0000 7fd001e00640 1 --2- 172.21.15.27:0/1176063532 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcfe80b65e0 0x7fcfe80b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:28.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.135+0000 7fd001e00640 1 --2- 172.21.15.27:0/1176063532 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcfe80b65e0 0x7fcfe80b8af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009bd0 tx=0x7fcff4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.138+0000 7fcffaa00640 1 -- 172.21.15.27:0/1176063532 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcffc0ceee0 con 0x7fd0041100e0 2024-09-21T00:18:28.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:27 smithi089 ceph-mon[34481]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:28.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:28.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:28.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:28.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:28.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:27 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:28.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:27 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:18:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:27 smithi098 ceph-mon[34725]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:27 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:27 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:18:28.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:27 smithi027 ceph-mon[30745]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:27 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:27 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:18:28.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.392+0000 7fd008c00640 1 -- 172.21.15.27:0/1176063532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd00419d7a0 con 0x7fd0041100e0 2024-09-21T00:18:28.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.393+0000 7fcffaa00640 1 -- 172.21.15.27:0/1176063532 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10503 (secure 0 0 0) 0x7fcffc0d32c0 con 0x7fd0041100e0 2024-09-21T00:18:28.395 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:18:28.395 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":13,"btime":"2024-09-21T00:18:27:889132+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24703,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24715,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24722,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24727,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24761,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T00:18:28.395 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 13 2024-09-21T00:18:28.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.396+0000 7fd008c00640 1 -- 172.21.15.27:0/1176063532 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcfe80b65e0 msgr2=0x7fcfe80b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:28.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.396+0000 7fd008c00640 1 --2- 172.21.15.27:0/1176063532 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcfe80b65e0 0x7fcfe80b8af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009bd0 tx=0x7fcff4005f90 comp rx=0 tx=0).stop 2024-09-21T00:18:28.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.396+0000 7fd008c00640 1 -- 172.21.15.27:0/1176063532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0041100e0 msgr2=0x7fd0041a1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:28.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.396+0000 7fd008c00640 1 --2- 172.21.15.27:0/1176063532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0041100e0 0x7fd0041a1eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcffc02f540 tx=0x7fcffc0316f0 comp rx=0 tx=0).stop 2024-09-21T00:18:28.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.396+0000 7fd008c00640 1 -- 172.21.15.27:0/1176063532 shutdown_connections 2024-09-21T00:18:28.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.396+0000 7fd008c00640 1 --2- 172.21.15.27:0/1176063532 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcfe80b65e0 0x7fcfe80b8af0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:28.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.396+0000 7fd008c00640 1 --2- 172.21.15.27:0/1176063532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0041100e0 0x7fd0041a1eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:28.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.396+0000 7fd008c00640 1 --2- 172.21.15.27:0/1176063532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd004076100 0x7fd00419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:28.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.396+0000 7fd008c00640 1 --2- 172.21.15.27:0/1176063532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0040757b0 0x7fd00419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:28.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.396+0000 7fd008c00640 1 -- 172.21.15.27:0/1176063532 >> 172.21.15.27:0/1176063532 conn(0x7fd0040ff590 msgr2=0x7fd004109bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:28.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.396+0000 7fd008c00640 1 -- 172.21.15.27:0/1176063532 shutdown_connections 2024-09-21T00:18:28.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:28.396+0000 7fd008c00640 1 -- 172.21.15.27:0/1176063532 wait complete. 2024-09-21T00:18:29.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:28 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] up:boot 2024-09-21T00:18:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:28 smithi089 ceph-mon[34481]: fsmap 12 up:standby 2024-09-21T00:18:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:28 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T00:18:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:28 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1176063532' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:28 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:29.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:28 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:29.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:28 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] up:boot 2024-09-21T00:18:29.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:28 smithi098 ceph-mon[34725]: fsmap 12 up:standby 2024-09-21T00:18:29.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:28 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T00:18:29.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:28 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1176063532' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:29.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:28 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:29.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:28 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:28 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] up:boot 2024-09-21T00:18:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:28 smithi027 ceph-mon[30745]: fsmap 12 up:standby 2024-09-21T00:18:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:28 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T00:18:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:28 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1176063532' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:28 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:29.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:28 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:29.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T00:18:30.164 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:29 smithi098 ceph-mon[34725]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:30.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:30.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:30.165 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:30.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:29 smithi089 ceph-mon[34481]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:30.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:30.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:30.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:30.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:29 smithi027 ceph-mon[30745]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:30.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:30.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:30.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:30.417 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:31.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.036+0000 7fcb4c800640 1 -- 172.21.15.27:0/1646863873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb441049b0 msgr2=0x7fcb44104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:31.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.036+0000 7fcb4c800640 1 --2- 172.21.15.27:0/1646863873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb441049b0 0x7fcb44104dc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009900 tx=0x7fcb3802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:18:31.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.036+0000 7fcb4c800640 1 -- 172.21.15.27:0/1646863873 shutdown_connections 2024-09-21T00:18:31.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.036+0000 7fcb4c800640 1 --2- 172.21.15.27:0/1646863873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb4410de10 0x7fcb44110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:31.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.036+0000 7fcb4c800640 1 --2- 172.21.15.27:0/1646863873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb44105300 0x7fcb4410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:31.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.036+0000 7fcb4c800640 1 --2- 172.21.15.27:0/1646863873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb441049b0 0x7fcb44104dc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:31.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.036+0000 7fcb4c800640 1 -- 172.21.15.27:0/1646863873 >> 172.21.15.27:0/1646863873 conn(0x7fcb44079730 msgr2=0x7fcb44079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:31.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.036+0000 7fcb4c800640 1 -- 172.21.15.27:0/1646863873 shutdown_connections 2024-09-21T00:18:31.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.036+0000 7fcb4c800640 1 -- 172.21.15.27:0/1646863873 wait complete. 2024-09-21T00:18:31.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.037+0000 7fcb4c800640 1 Processor -- start 2024-09-21T00:18:31.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.037+0000 7fcb4c800640 1 -- start start 2024-09-21T00:18:31.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.038+0000 7fcb4c800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb441049b0 0x7fcb441026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:31.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.038+0000 7fcb4c800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb44105300 0x7fcb44102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:31.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.038+0000 7fcb4c800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb4410de10 0x7fcb44103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:31.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.038+0000 7fcb4c800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcb44113ae0 con 0x7fcb441049b0 2024-09-21T00:18:31.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.038+0000 7fcb4c800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcb44113960 con 0x7fcb44105300 2024-09-21T00:18:31.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.038+0000 7fcb4c800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcb44113c60 con 0x7fcb4410de10 2024-09-21T00:18:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.038+0000 7fcb4a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb4410de10 0x7fcb44103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.038+0000 7fcb49e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb441049b0 0x7fcb441026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.038+0000 7fcb49400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb44105300 0x7fcb44102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.038+0000 7fcb4a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb4410de10 0x7fcb44103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:52136/0 (socket says 172.21.15.27:52136) 2024-09-21T00:18:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.038+0000 7fcb4a800640 1 -- 172.21.15.27:0/172417233 learned_addr learned my addr 172.21.15.27:0/172417233 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.038+0000 7fcb49400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb44105300 0x7fcb44102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57828/0 (socket says 172.21.15.27:57828) 2024-09-21T00:18:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.039+0000 7fcb4a800640 1 -- 172.21.15.27:0/172417233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb44105300 msgr2=0x7fcb44102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.039+0000 7fcb4a800640 1 --2- 172.21.15.27:0/172417233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb44105300 0x7fcb44102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.039+0000 7fcb4a800640 1 -- 172.21.15.27:0/172417233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb441049b0 msgr2=0x7fcb441026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.039+0000 7fcb4a800640 1 --2- 172.21.15.27:0/172417233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb441049b0 0x7fcb441026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.039+0000 7fcb4a800640 1 -- 172.21.15.27:0/172417233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb441a8810 con 0x7fcb4410de10 2024-09-21T00:18:31.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.039+0000 7fcb4a800640 1 --2- 172.21.15.27:0/172417233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb4410de10 0x7fcb44103170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000eb10 tx=0x7fcb4000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:31.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.039+0000 7fcb3ea00640 1 -- 172.21.15.27:0/172417233 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40004b20 con 0x7fcb4410de10 2024-09-21T00:18:31.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.039+0000 7fcb3ea00640 1 -- 172.21.15.27:0/172417233 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb40004cc0 con 0x7fcb4410de10 2024-09-21T00:18:31.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.040+0000 7fcb3ea00640 1 -- 172.21.15.27:0/172417233 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40010670 con 0x7fcb4410de10 2024-09-21T00:18:31.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.040+0000 7fcb4c800640 1 -- 172.21.15.27:0/172417233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4419a2c0 con 0x7fcb4410de10 2024-09-21T00:18:31.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.040+0000 7fcb4c800640 1 -- 172.21.15.27:0/172417233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4419a780 con 0x7fcb4410de10 2024-09-21T00:18:31.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.041+0000 7fcb4c800640 1 -- 172.21.15.27:0/172417233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb44104dc0 con 0x7fcb4410de10 2024-09-21T00:18:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.043+0000 7fcb3ea00640 1 -- 172.21.15.27:0/172417233 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fcb40005020 con 0x7fcb4410de10 2024-09-21T00:18:31.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.045+0000 7fcb3ea00640 1 --2- 172.21.15.27:0/172417233 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcb2c0b6560 0x7fcb2c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:31.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.045+0000 7fcb3ea00640 1 -- 172.21.15.27:0/172417233 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fcb400e75c0 con 0x7fcb4410de10 2024-09-21T00:18:31.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.045+0000 7fcb49e00640 1 --2- 172.21.15.27:0/172417233 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcb2c0b6560 0x7fcb2c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.046+0000 7fcb49e00640 1 --2- 172.21.15.27:0/172417233 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcb2c0b6560 0x7fcb2c0b8a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcb38031440 tx=0x7fcb38005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:31.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.053+0000 7fcb3ea00640 1 -- 172.21.15.27:0/172417233 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcb400ae180 con 0x7fcb4410de10 2024-09-21T00:18:31.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.287+0000 7fcb4c800640 1 -- 172.21.15.27:0/172417233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcb4419abe0 con 0x7fcb4410de10 2024-09-21T00:18:31.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.287+0000 7fcb3ea00640 1 -- 172.21.15.27:0/172417233 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7fcb400b2560 con 0x7fcb4410de10 2024-09-21T00:18:31.288 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:18:31.288 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-21T00:18:31.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:18:31.289 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-09-21T00:18:31.289 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-09-21T00:18:31.289 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:18:31.289 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-21T00:18:31.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.291+0000 7fcb4c800640 1 -- 172.21.15.27:0/172417233 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcb2c0b6560 msgr2=0x7fcb2c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:31.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.291+0000 7fcb4c800640 1 --2- 172.21.15.27:0/172417233 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcb2c0b6560 0x7fcb2c0b8a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcb38031440 tx=0x7fcb38005a20 comp rx=0 tx=0).stop 2024-09-21T00:18:31.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.291+0000 7fcb4c800640 1 -- 172.21.15.27:0/172417233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb4410de10 msgr2=0x7fcb44103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:31.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.291+0000 7fcb4c800640 1 --2- 172.21.15.27:0/172417233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb4410de10 0x7fcb44103170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000eb10 tx=0x7fcb4000c2d0 comp rx=0 tx=0).stop 2024-09-21T00:18:31.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.291+0000 7fcb4c800640 1 -- 172.21.15.27:0/172417233 shutdown_connections 2024-09-21T00:18:31.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.291+0000 7fcb4c800640 1 --2- 172.21.15.27:0/172417233 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcb2c0b6560 0x7fcb2c0b8a70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:31.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.291+0000 7fcb4c800640 1 --2- 172.21.15.27:0/172417233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb4410de10 0x7fcb44103170 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:31.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.291+0000 7fcb4c800640 1 --2- 172.21.15.27:0/172417233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb44105300 0x7fcb44102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:31.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.291+0000 7fcb4c800640 1 --2- 172.21.15.27:0/172417233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb441049b0 0x7fcb441026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:31.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.291+0000 7fcb4c800640 1 -- 172.21.15.27:0/172417233 >> 172.21.15.27:0/172417233 conn(0x7fcb44079730 msgr2=0x7fcb441079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:31.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.291+0000 7fcb4c800640 1 -- 172.21.15.27:0/172417233 shutdown_connections 2024-09-21T00:18:31.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:31.291+0000 7fcb4c800640 1 -- 172.21.15.27:0/172417233 wait complete. 2024-09-21T00:18:32.171 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T00:18:32.171 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T00:18:32.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:32 smithi089 ceph-mon[34481]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:32 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:32 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:32 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:32 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:18:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:32 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:32.481 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:32 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:18:32.481 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:32 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/172417233' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T00:18:32.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:32 smithi098 ceph-mon[34725]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:32.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:32 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:32.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:32 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:32 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:32 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:18:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:32 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:32 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:18:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:32 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/172417233' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T00:18:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:32 smithi027 ceph-mon[30745]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:32 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:32 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:32 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:32 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:18:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:32 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:32 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:18:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:32 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/172417233' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T00:18:33.118 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.749+0000 7fdba2600640 1 -- 172.21.15.27:0/1775151871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c1094c0 msgr2=0x7fdb9c1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.749+0000 7fdba2600640 1 --2- 172.21.15.27:0/1775151871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c1094c0 0x7fdb9c1098d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009900 tx=0x7fdb8402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:18:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.749+0000 7fdba2600640 1 -- 172.21.15.27:0/1775151871 shutdown_connections 2024-09-21T00:18:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.749+0000 7fdba2600640 1 --2- 172.21.15.27:0/1775151871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c109e10 0x7fdb9c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.749+0000 7fdba2600640 1 --2- 172.21.15.27:0/1775151871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c1094c0 0x7fdb9c1098d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.749+0000 7fdba2600640 1 --2- 172.21.15.27:0/1775151871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb9c103b80 0x7fdb9c103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.749+0000 7fdba2600640 1 -- 172.21.15.27:0/1775151871 >> 172.21.15.27:0/1775151871 conn(0x7fdb9c0ff590 msgr2=0x7fdb9c1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.749+0000 7fdba2600640 1 -- 172.21.15.27:0/1775151871 shutdown_connections 2024-09-21T00:18:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.749+0000 7fdba2600640 1 -- 172.21.15.27:0/1775151871 wait complete. 2024-09-21T00:18:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.750+0000 7fdba2600640 1 Processor -- start 2024-09-21T00:18:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.750+0000 7fdba2600640 1 -- start start 2024-09-21T00:18:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.751+0000 7fdba2600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c103b80 0x7fdb9c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.751+0000 7fdba2600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb9c1094c0 0x7fdb9c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.751+0000 7fdba2600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c109e10 0x7fdb9c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.751+0000 7fdba2600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c113ad0 con 0x7fdb9c103b80 2024-09-21T00:18:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.751+0000 7fdba2600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c113950 con 0x7fdb9c1094c0 2024-09-21T00:18:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.751+0000 7fdba2600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c113c50 con 0x7fdb9c109e10 2024-09-21T00:18:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.751+0000 7fdb9b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c103b80 0x7fdb9c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.751+0000 7fdb9b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c103b80 0x7fdb9c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35240/0 (socket says 172.21.15.27:35240) 2024-09-21T00:18:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.751+0000 7fdb93e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb9c1094c0 0x7fdb9c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.751+0000 7fdb9b400640 1 -- 172.21.15.27:0/4028717640 learned_addr learned my addr 172.21.15.27:0/4028717640 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.751+0000 7fdb9be00640 1 --2- 172.21.15.27:0/4028717640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c109e10 0x7fdb9c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.752+0000 7fdb93e00640 1 -- 172.21.15.27:0/4028717640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c109e10 msgr2=0x7fdb9c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.752+0000 7fdb93e00640 1 --2- 172.21.15.27:0/4028717640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c109e10 0x7fdb9c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.752+0000 7fdb93e00640 1 -- 172.21.15.27:0/4028717640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c103b80 msgr2=0x7fdb9c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.752+0000 7fdb93e00640 1 --2- 172.21.15.27:0/4028717640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c103b80 0x7fdb9c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.752+0000 7fdb93e00640 1 -- 172.21.15.27:0/4028717640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c1a8820 con 0x7fdb9c1094c0 2024-09-21T00:18:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.752+0000 7fdb93e00640 1 --2- 172.21.15.27:0/4028717640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb9c1094c0 0x7fdb9c075c50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb8402a910 tx=0x7fdb84031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.752+0000 7fdb98c00640 1 -- 172.21.15.27:0/4028717640 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8403f070 con 0x7fdb9c1094c0 2024-09-21T00:18:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.753+0000 7fdb98c00640 1 -- 172.21.15.27:0/4028717640 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb84004490 con 0x7fdb9c1094c0 2024-09-21T00:18:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.753+0000 7fdb98c00640 1 -- 172.21.15.27:0/4028717640 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84035070 con 0x7fdb9c1094c0 2024-09-21T00:18:33.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.753+0000 7fdba2600640 1 -- 172.21.15.27:0/4028717640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c1a8ab0 con 0x7fdb9c1094c0 2024-09-21T00:18:33.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.753+0000 7fdba2600640 1 -- 172.21.15.27:0/4028717640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c1a8fe0 con 0x7fdb9c1094c0 2024-09-21T00:18:33.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.754+0000 7fdba2600640 1 -- 172.21.15.27:0/4028717640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9c103f90 con 0x7fdb9c1094c0 2024-09-21T00:18:33.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.758+0000 7fdb98c00640 1 -- 172.21.15.27:0/4028717640 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fdb8403b460 con 0x7fdb9c1094c0 2024-09-21T00:18:33.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.760+0000 7fdb98c00640 1 --2- 172.21.15.27:0/4028717640 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdb800b65e0 0x7fdb800b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:33.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.760+0000 7fdb98c00640 1 -- 172.21.15.27:0/4028717640 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fdb84107680 con 0x7fdb9c1094c0 2024-09-21T00:18:33.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.760+0000 7fdb9b400640 1 --2- 172.21.15.27:0/4028717640 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdb800b65e0 0x7fdb800b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:33.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.760+0000 7fdb98c00640 1 -- 172.21.15.27:0/4028717640 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdb840d2620 con 0x7fdb9c1094c0 2024-09-21T00:18:33.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.761+0000 7fdb9b400640 1 --2- 172.21.15.27:0/4028717640 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdb800b65e0 0x7fdb800b8af0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c077160 tx=0x7fdb8c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:33.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:33 smithi027 ceph-mon[30745]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:33.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:33 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:33.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:33 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:33.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:33 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:33.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:33 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:33.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:33.976+0000 7fdba2600640 1 -- 172.21.15.27:0/4028717640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fdb9c076a10 con 0x7fdb9c1094c0 2024-09-21T00:18:33.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:33 smithi089 ceph-mon[34481]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:33.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:33 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:33.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:33 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:33.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:33 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:33.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:33 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:33.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:33 smithi098 ceph-mon[34725]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:33.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:33 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:33.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:33 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:33.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:33 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:33.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:33 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:34.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:34.611+0000 7fdb98c00640 1 -- 172.21.15.27:0/4028717640 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7fdb9c076a10 con 0x7fdb9c1094c0 2024-09-21T00:18:34.613 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-09-21T00:18:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:34.616+0000 7fdba2600640 1 -- 172.21.15.27:0/4028717640 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdb800b65e0 msgr2=0x7fdb800b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:34.616+0000 7fdba2600640 1 --2- 172.21.15.27:0/4028717640 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdb800b65e0 0x7fdb800b8af0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c077160 tx=0x7fdb8c005f90 comp rx=0 tx=0).stop 2024-09-21T00:18:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:34.616+0000 7fdba2600640 1 -- 172.21.15.27:0/4028717640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb9c1094c0 msgr2=0x7fdb9c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:34.616+0000 7fdba2600640 1 --2- 172.21.15.27:0/4028717640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb9c1094c0 0x7fdb9c075c50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb8402a910 tx=0x7fdb84031a20 comp rx=0 tx=0).stop 2024-09-21T00:18:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:34.616+0000 7fdba2600640 1 -- 172.21.15.27:0/4028717640 shutdown_connections 2024-09-21T00:18:34.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:34.616+0000 7fdba2600640 1 --2- 172.21.15.27:0/4028717640 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdb800b65e0 0x7fdb800b8af0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:34.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:34.616+0000 7fdba2600640 1 --2- 172.21.15.27:0/4028717640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c109e10 0x7fdb9c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:34.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:34.616+0000 7fdba2600640 1 --2- 172.21.15.27:0/4028717640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb9c1094c0 0x7fdb9c075c50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:34.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:34.616+0000 7fdba2600640 1 --2- 172.21.15.27:0/4028717640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c103b80 0x7fdb9c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:34.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:34.616+0000 7fdba2600640 1 -- 172.21.15.27:0/4028717640 >> 172.21.15.27:0/4028717640 conn(0x7fdb9c0ff590 msgr2=0x7fdb9c107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:34.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:34.616+0000 7fdba2600640 1 -- 172.21.15.27:0/4028717640 shutdown_connections 2024-09-21T00:18:34.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:34.616+0000 7fdba2600640 1 -- 172.21.15.27:0/4028717640 wait complete. 2024-09-21T00:18:34.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:34 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/4028717640' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T00:18:34.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:34 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T00:18:34.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:34 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:34.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:34 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:34.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:34 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:34.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:34 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:18:34.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:34 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:34.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:34 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/4028717640' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T00:18:34.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:34 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T00:18:34.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:34 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:34.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:34 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:34.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:34 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:34.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:34 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:18:34.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:34 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:35.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:34 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/4028717640' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T00:18:35.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:34 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T00:18:35.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:34 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:35.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:34 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:35.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:34 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:18:35.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:34 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:18:35.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:34 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:18:35.548 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-21T00:18:35.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:35 smithi027 ceph-mon[30745]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:35.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:35 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T00:18:35.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:35 smithi027 ceph-mon[30745]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T00:18:35.945 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:35.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:35 smithi089 ceph-mon[34481]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:35.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:35 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T00:18:35.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:35 smithi089 ceph-mon[34481]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T00:18:35.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:35 smithi098 ceph-mon[34725]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:35.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:35 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T00:18:35.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:35 smithi098 ceph-mon[34725]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T00:18:36.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.588+0000 7f9f9ca00640 1 -- 172.21.15.27:0/1441201830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98105bb0 msgr2=0x7f9f98105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.588+0000 7f9f9ca00640 1 --2- 172.21.15.27:0/1441201830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98105bb0 0x7f9f98105fc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009900 tx=0x7f9f8802a410 comp rx=0 tx=0).stop 2024-09-21T00:18:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.588+0000 7f9f9ca00640 1 -- 172.21.15.27:0/1441201830 shutdown_connections 2024-09-21T00:18:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.588+0000 7f9f9ca00640 1 --2- 172.21.15.27:0/1441201830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f9810be40 0x7f9f98112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.588+0000 7f9f9ca00640 1 --2- 172.21.15.27:0/1441201830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f9810b4f0 0x7f9f9810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.588+0000 7f9f9ca00640 1 --2- 172.21.15.27:0/1441201830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98105bb0 0x7f9f98105fc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.588+0000 7f9f9ca00640 1 -- 172.21.15.27:0/1441201830 >> 172.21.15.27:0/1441201830 conn(0x7f9f98101680 msgr2=0x7f9f98103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.588+0000 7f9f9ca00640 1 -- 172.21.15.27:0/1441201830 shutdown_connections 2024-09-21T00:18:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.588+0000 7f9f9ca00640 1 -- 172.21.15.27:0/1441201830 wait complete. 2024-09-21T00:18:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.589+0000 7f9f9ca00640 1 Processor -- start 2024-09-21T00:18:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.590+0000 7f9f9ca00640 1 -- start start 2024-09-21T00:18:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.590+0000 7f9f9ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98105bb0 0x7f9f9819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.590+0000 7f9f9ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f9810b4f0 0x7f9f9819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.590+0000 7f9f9ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f9810be40 0x7f9f981a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.590+0000 7f9f9ca00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98115b50 con 0x7f9f98105bb0 2024-09-21T00:18:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.590+0000 7f9f9ca00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9f981159d0 con 0x7f9f9810b4f0 2024-09-21T00:18:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.590+0000 7f9f9ca00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98115cd0 con 0x7f9f9810be40 2024-09-21T00:18:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.590+0000 7f9f96800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98105bb0 0x7f9f9819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.590+0000 7f9f8de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f9810b4f0 0x7f9f9819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.590+0000 7f9f96800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98105bb0 0x7f9f9819c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35250/0 (socket says 172.21.15.27:35250) 2024-09-21T00:18:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.590+0000 7f9f96800640 1 -- 172.21.15.27:0/3947945840 learned_addr learned my addr 172.21.15.27:0/3947945840 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.590+0000 7f9f97200640 1 --2- 172.21.15.27:0/3947945840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f9810be40 0x7f9f981a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.591+0000 7f9f8de00640 1 -- 172.21.15.27:0/3947945840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f9810be40 msgr2=0x7f9f981a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.591+0000 7f9f8de00640 1 --2- 172.21.15.27:0/3947945840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f9810be40 0x7f9f981a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.591+0000 7f9f8de00640 1 -- 172.21.15.27:0/3947945840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98105bb0 msgr2=0x7f9f9819c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.591+0000 7f9f8de00640 1 --2- 172.21.15.27:0/3947945840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98105bb0 0x7f9f9819c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.591+0000 7f9f8de00640 1 -- 172.21.15.27:0/3947945840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f981a24b0 con 0x7f9f9810b4f0 2024-09-21T00:18:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.591+0000 7f9f8de00640 1 --2- 172.21.15.27:0/3947945840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f9810b4f0 0x7f9f9819cae0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9f9806c8c0 tx=0x7f9f8000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.591+0000 7f9f8fe00640 1 -- 172.21.15.27:0/3947945840 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8000cd00 con 0x7f9f9810b4f0 2024-09-21T00:18:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.591+0000 7f9f8fe00640 1 -- 172.21.15.27:0/3947945840 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f800035f0 con 0x7f9f9810b4f0 2024-09-21T00:18:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.591+0000 7f9f9ca00640 1 -- 172.21.15.27:0/3947945840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f981a27a0 con 0x7f9f9810b4f0 2024-09-21T00:18:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.592+0000 7f9f8fe00640 1 -- 172.21.15.27:0/3947945840 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80010690 con 0x7f9f9810b4f0 2024-09-21T00:18:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.592+0000 7f9f9ca00640 1 -- 172.21.15.27:0/3947945840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f981a8f00 con 0x7f9f9810b4f0 2024-09-21T00:18:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.593+0000 7f9f9ca00640 1 -- 172.21.15.27:0/3947945840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f98105fc0 con 0x7f9f9810b4f0 2024-09-21T00:18:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.595+0000 7f9f8fe00640 1 -- 172.21.15.27:0/3947945840 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f9f800037a0 con 0x7f9f9810b4f0 2024-09-21T00:18:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.597+0000 7f9f8fe00640 1 --2- 172.21.15.27:0/3947945840 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9f7c0b6560 0x7f9f7c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.597+0000 7f9f8fe00640 1 -- 172.21.15.27:0/3947945840 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7712+0+0 (secure 0 0 0) 0x7f9f800e7710 con 0x7f9f9810b4f0 2024-09-21T00:18:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.597+0000 7f9f96800640 1 --2- 172.21.15.27:0/3947945840 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9f7c0b6560 0x7f9f7c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.598+0000 7f9f96800640 1 --2- 172.21.15.27:0/3947945840 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9f7c0b6560 0x7f9f7c0b8a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9f880096c0 tx=0x7f9f88005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:36.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.599+0000 7f9f8fe00640 1 -- 172.21.15.27:0/3947945840 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9f800ae180 con 0x7f9f9810b4f0 2024-09-21T00:18:36.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:36.804+0000 7f9f9ca00640 1 -- 172.21.15.27:0/3947945840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f9f9810dc70 con 0x7f9f9810b4f0 2024-09-21T00:18:36.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:36 smithi027 ceph-mon[30745]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T00:18:36.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:36 smithi027 ceph-mon[30745]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T00:18:36.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:36 smithi089 ceph-mon[34481]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T00:18:36.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:36 smithi089 ceph-mon[34481]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T00:18:36.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:36 smithi098 ceph-mon[34725]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T00:18:36.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:36 smithi098 ceph-mon[34725]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T00:18:37.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:37.637+0000 7f9f8fe00640 1 -- 172.21.15.27:0/3947945840 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v76) ==== 171+0+0 (secure 0 0 0) 0x7f9f800b2560 con 0x7f9f9810b4f0 2024-09-21T00:18:37.639 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-09-21T00:18:37.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:37.642+0000 7f9f9ca00640 1 -- 172.21.15.27:0/3947945840 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9f7c0b6560 msgr2=0x7f9f7c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:37.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:37.642+0000 7f9f9ca00640 1 --2- 172.21.15.27:0/3947945840 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9f7c0b6560 0x7f9f7c0b8a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9f880096c0 tx=0x7f9f88005a20 comp rx=0 tx=0).stop 2024-09-21T00:18:37.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:37.642+0000 7f9f9ca00640 1 -- 172.21.15.27:0/3947945840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f9810b4f0 msgr2=0x7f9f9819cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:37.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:37.642+0000 7f9f9ca00640 1 --2- 172.21.15.27:0/3947945840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f9810b4f0 0x7f9f9819cae0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9f9806c8c0 tx=0x7f9f8000ee50 comp rx=0 tx=0).stop 2024-09-21T00:18:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:37.642+0000 7f9f9ca00640 1 -- 172.21.15.27:0/3947945840 shutdown_connections 2024-09-21T00:18:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:37.642+0000 7f9f9ca00640 1 --2- 172.21.15.27:0/3947945840 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9f7c0b6560 0x7f9f7c0b8a70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:37.642+0000 7f9f9ca00640 1 --2- 172.21.15.27:0/3947945840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f9810be40 0x7f9f981a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:37.642+0000 7f9f9ca00640 1 --2- 172.21.15.27:0/3947945840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f9810b4f0 0x7f9f9819cae0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:37.642+0000 7f9f9ca00640 1 --2- 172.21.15.27:0/3947945840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98105bb0 0x7f9f9819c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:37.642+0000 7f9f9ca00640 1 -- 172.21.15.27:0/3947945840 >> 172.21.15.27:0/3947945840 conn(0x7f9f98101680 msgr2=0x7f9f98103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:37.643+0000 7f9f9ca00640 1 -- 172.21.15.27:0/3947945840 shutdown_connections 2024-09-21T00:18:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:37.643+0000 7f9f9ca00640 1 -- 172.21.15.27:0/3947945840 wait complete. 2024-09-21T00:18:37.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:37 smithi089 ceph-mon[34481]: pgmap v278: 33 pgs: 3 creating+peering, 20 unknown, 10 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:37.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:37 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3947945840' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T00:18:37.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:37 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T00:18:37.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:37 smithi098 ceph-mon[34725]: pgmap v278: 33 pgs: 3 creating+peering, 20 unknown, 10 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:37.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:37 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3947945840' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T00:18:37.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:37 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T00:18:38.022 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:37 smithi027 ceph-mon[30745]: pgmap v278: 33 pgs: 3 creating+peering, 20 unknown, 10 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:38.022 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:37 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3947945840' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T00:18:38.023 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:37 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T00:18:38.445 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T00:18:38.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:38 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T00:18:38.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:38 smithi027 ceph-mon[30745]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T00:18:38.850 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:38.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:38 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T00:18:38.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:38 smithi089 ceph-mon[34481]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T00:18:38.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:38 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T00:18:38.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:38 smithi098 ceph-mon[34725]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T00:18:39.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.552+0000 7f80ade00640 1 -- 172.21.15.27:0/1666537906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80a8109c20 msgr2=0x7f80a8110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:39.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.552+0000 7f80ade00640 1 --2- 172.21.15.27:0/1666537906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80a8109c20 0x7f80a8110450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009ca0 tx=0x7f80a002a8c0 comp rx=0 tx=0).stop 2024-09-21T00:18:39.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.553+0000 7f80ade00640 1 -- 172.21.15.27:0/1666537906 shutdown_connections 2024-09-21T00:18:39.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.553+0000 7f80ade00640 1 --2- 172.21.15.27:0/1666537906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80a8109c20 0x7f80a8110450 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:39.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.553+0000 7f80ade00640 1 --2- 172.21.15.27:0/1666537906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a81092d0 0x7f80a81096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:39.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.553+0000 7f80ade00640 1 --2- 172.21.15.27:0/1666537906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80a8103990 0x7f80a8103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:39.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.553+0000 7f80ade00640 1 -- 172.21.15.27:0/1666537906 >> 172.21.15.27:0/1666537906 conn(0x7f80a80ff440 msgr2=0x7f80a8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:39.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.553+0000 7f80ade00640 1 -- 172.21.15.27:0/1666537906 shutdown_connections 2024-09-21T00:18:39.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.553+0000 7f80ade00640 1 -- 172.21.15.27:0/1666537906 wait complete. 2024-09-21T00:18:39.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.554+0000 7f80ade00640 1 Processor -- start 2024-09-21T00:18:39.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.554+0000 7f80ade00640 1 -- start start 2024-09-21T00:18:39.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.554+0000 7f80ade00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a8103990 0x7f80a819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:39.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.554+0000 7f80ade00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80a81092d0 0x7f80a819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:39.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.554+0000 7f80ade00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80a8109c20 0x7f80a819fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:39.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.554+0000 7f80ade00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f80a81138e0 con 0x7f80a8103990 2024-09-21T00:18:39.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.554+0000 7f80ade00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f80a8113760 con 0x7f80a81092d0 2024-09-21T00:18:39.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.554+0000 7f80ade00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f80a8113a60 con 0x7f80a8109c20 2024-09-21T00:18:39.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.555+0000 7f80aca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a8103990 0x7f80a819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:39.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.555+0000 7f80a7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80a81092d0 0x7f80a819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:39.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.555+0000 7f80a7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80a81092d0 0x7f80a819a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:55192/0 (socket says 172.21.15.27:55192) 2024-09-21T00:18:39.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.555+0000 7f80a7e00640 1 -- 172.21.15.27:0/1841065181 learned_addr learned my addr 172.21.15.27:0/1841065181 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:39.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.555+0000 7f80aca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a8103990 0x7f80a819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35266/0 (socket says 172.21.15.27:35266) 2024-09-21T00:18:39.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.555+0000 7f80ad400640 1 --2- 172.21.15.27:0/1841065181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80a8109c20 0x7f80a819fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:39.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.555+0000 7f80a7e00640 1 -- 172.21.15.27:0/1841065181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80a8109c20 msgr2=0x7f80a819fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:39.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.555+0000 7f80a7e00640 1 --2- 172.21.15.27:0/1841065181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80a8109c20 0x7f80a819fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:39.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.555+0000 7f80a7e00640 1 -- 172.21.15.27:0/1841065181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a8103990 msgr2=0x7f80a819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:39.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.555+0000 7f80a7e00640 1 --2- 172.21.15.27:0/1841065181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a8103990 0x7f80a819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:39.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.555+0000 7f80a7e00640 1 -- 172.21.15.27:0/1841065181 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a81a01a0 con 0x7f80a81092d0 2024-09-21T00:18:39.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.555+0000 7f80ad400640 1 --2- 172.21.15.27:0/1841065181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80a8109c20 0x7f80a819fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:18:39.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.555+0000 7f80a7e00640 1 --2- 172.21.15.27:0/1841065181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80a81092d0 0x7f80a819a7d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f809400e9f0 tx=0x7f809400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:39.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.556+0000 7f80a5600640 1 -- 172.21.15.27:0/1841065181 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809400ccc0 con 0x7f80a81092d0 2024-09-21T00:18:39.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.556+0000 7f80ade00640 1 -- 172.21.15.27:0/1841065181 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a81a0490 con 0x7f80a81092d0 2024-09-21T00:18:39.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.556+0000 7f80a5600640 1 -- 172.21.15.27:0/1841065181 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80940108e0 con 0x7f80a81092d0 2024-09-21T00:18:39.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.556+0000 7f80a5600640 1 -- 172.21.15.27:0/1841065181 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094018860 con 0x7f80a81092d0 2024-09-21T00:18:39.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.556+0000 7f80ade00640 1 -- 172.21.15.27:0/1841065181 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a81a6bf0 con 0x7f80a81092d0 2024-09-21T00:18:39.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.557+0000 7f80ade00640 1 -- 172.21.15.27:0/1841065181 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a8103da0 con 0x7f80a81092d0 2024-09-21T00:18:39.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.560+0000 7f80a5600640 1 -- 172.21.15.27:0/1841065181 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f8094010430 con 0x7f80a81092d0 2024-09-21T00:18:39.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.562+0000 7f80a5600640 1 --2- 172.21.15.27:0/1841065181 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f80900b6560 0x7f80900b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:39.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.562+0000 7f80a5600640 1 -- 172.21.15.27:0/1841065181 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8094+0+0 (secure 0 0 0) 0x7f80940e7660 con 0x7f80a81092d0 2024-09-21T00:18:39.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.562+0000 7f80aca00640 1 --2- 172.21.15.27:0/1841065181 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f80900b6560 0x7f80900b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:39.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.563+0000 7f80aca00640 1 --2- 172.21.15.27:0/1841065181 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f80900b6560 0x7f80900b8a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f809c009bd0 tx=0x7f809c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:39.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.565+0000 7f80a5600640 1 -- 172.21.15.27:0/1841065181 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f80940adf30 con 0x7f80a81092d0 2024-09-21T00:18:39.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:39.808+0000 7f80ade00640 1 -- 172.21.15.27:0/1841065181 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f80a802e640 con 0x7f80a81092d0 2024-09-21T00:18:39.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:39 smithi027 ceph-mon[30745]: pgmap v281: 97 pgs: 23 creating+peering, 50 unknown, 24 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:39.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:39 smithi027 ceph-mon[30745]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T00:18:39.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:39 smithi089 ceph-mon[34481]: pgmap v281: 97 pgs: 23 creating+peering, 50 unknown, 24 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:39.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:39 smithi089 ceph-mon[34481]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T00:18:39.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:39 smithi098 ceph-mon[34725]: pgmap v281: 97 pgs: 23 creating+peering, 50 unknown, 24 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:39.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:39 smithi098 ceph-mon[34725]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T00:18:40.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:40.665+0000 7f80a5600640 1 -- 172.21.15.27:0/1841065181 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f80940b2310 con 0x7f80a81092d0 2024-09-21T00:18:40.666 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T00:18:40.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:40.669+0000 7f80ade00640 1 -- 172.21.15.27:0/1841065181 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f80900b6560 msgr2=0x7f80900b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:40.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:40.669+0000 7f80ade00640 1 --2- 172.21.15.27:0/1841065181 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f80900b6560 0x7f80900b8a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f809c009bd0 tx=0x7f809c005f90 comp rx=0 tx=0).stop 2024-09-21T00:18:40.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:40.669+0000 7f80ade00640 1 -- 172.21.15.27:0/1841065181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80a81092d0 msgr2=0x7f80a819a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:40.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:40.669+0000 7f80ade00640 1 --2- 172.21.15.27:0/1841065181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80a81092d0 0x7f80a819a7d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f809400e9f0 tx=0x7f809400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:18:40.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:40.669+0000 7f80ade00640 1 -- 172.21.15.27:0/1841065181 shutdown_connections 2024-09-21T00:18:40.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:40.669+0000 7f80ade00640 1 --2- 172.21.15.27:0/1841065181 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f80900b6560 0x7f80900b8a70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:40.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:40.669+0000 7f80ade00640 1 --2- 172.21.15.27:0/1841065181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80a8109c20 0x7f80a819fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:40.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:40.669+0000 7f80ade00640 1 --2- 172.21.15.27:0/1841065181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80a81092d0 0x7f80a819a7d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:40.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:40.669+0000 7f80ade00640 1 --2- 172.21.15.27:0/1841065181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a8103990 0x7f80a819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:40.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:40.669+0000 7f80ade00640 1 -- 172.21.15.27:0/1841065181 >> 172.21.15.27:0/1841065181 conn(0x7f80a80ff440 msgr2=0x7f80a8106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:40.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:40.670+0000 7f80ade00640 1 -- 172.21.15.27:0/1841065181 shutdown_connections 2024-09-21T00:18:40.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:40.670+0000 7f80ade00640 1 -- 172.21.15.27:0/1841065181 wait complete. 2024-09-21T00:18:40.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:40 smithi089 ceph-mon[34481]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T00:18:40.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:40 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1841065181' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T00:18:40.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:40 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T00:18:40.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:40 smithi089 ceph-mon[34481]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T00:18:40.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:40 smithi098 ceph-mon[34725]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T00:18:40.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:40 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1841065181' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T00:18:40.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:40 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T00:18:40.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:40 smithi098 ceph-mon[34725]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T00:18:41.039 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:40 smithi027 ceph-mon[30745]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T00:18:41.039 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:40 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1841065181' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T00:18:41.039 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:40 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T00:18:41.039 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:40 smithi027 ceph-mon[30745]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T00:18:41.518 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-21T00:18:41.518 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-21T00:18:41.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:41 smithi098 ceph-mon[34725]: pgmap v284: 97 pgs: 20 creating+peering, 11 unknown, 66 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:41.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:41 smithi098 ceph-mon[34725]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T00:18:41.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:41 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T00:18:41.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:41 smithi098 ceph-mon[34725]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T00:18:41.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:41 smithi098 ceph-mon[34725]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T00:18:41.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:41 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] up:active 2024-09-21T00:18:41.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:41 smithi098 ceph-mon[34725]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T00:18:41.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:41 smithi027 ceph-mon[30745]: pgmap v284: 97 pgs: 20 creating+peering, 11 unknown, 66 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:41.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:41 smithi027 ceph-mon[30745]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T00:18:41.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:41 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T00:18:41.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:41 smithi027 ceph-mon[30745]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T00:18:41.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:41 smithi027 ceph-mon[30745]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T00:18:41.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:41 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] up:active 2024-09-21T00:18:41.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:41 smithi027 ceph-mon[30745]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T00:18:41.904 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:41.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:41 smithi089 ceph-mon[34481]: pgmap v284: 97 pgs: 20 creating+peering, 11 unknown, 66 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:41.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:41 smithi089 ceph-mon[34481]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T00:18:41.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:41 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T00:18:41.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:41 smithi089 ceph-mon[34481]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T00:18:41.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:41 smithi089 ceph-mon[34481]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T00:18:41.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:41 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] up:active 2024-09-21T00:18:41.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:41 smithi089 ceph-mon[34481]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T00:18:42.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.571+0000 7f3df8800640 1 -- 172.21.15.27:0/562001214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3df0103b60 msgr2=0x7f3df0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:42.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.571+0000 7f3df8800640 1 --2- 172.21.15.27:0/562001214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3df0103b60 0x7f3df0103f70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009900 tx=0x7f3de402a410 comp rx=0 tx=0).stop 2024-09-21T00:18:42.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.572+0000 7f3df8800640 1 -- 172.21.15.27:0/562001214 shutdown_connections 2024-09-21T00:18:42.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.572+0000 7f3df8800640 1 --2- 172.21.15.27:0/562001214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3df0109df0 0x7f3df0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:42.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.572+0000 7f3df8800640 1 --2- 172.21.15.27:0/562001214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3df01094a0 0x7f3df01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:42.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.572+0000 7f3df8800640 1 --2- 172.21.15.27:0/562001214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3df0103b60 0x7f3df0103f70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:42.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.572+0000 7f3df8800640 1 -- 172.21.15.27:0/562001214 >> 172.21.15.27:0/562001214 conn(0x7f3df007b850 msgr2=0x7f3df007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:42.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.572+0000 7f3df8800640 1 -- 172.21.15.27:0/562001214 shutdown_connections 2024-09-21T00:18:42.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.572+0000 7f3df8800640 1 -- 172.21.15.27:0/562001214 wait complete. 2024-09-21T00:18:42.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.573+0000 7f3df8800640 1 Processor -- start 2024-09-21T00:18:42.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.573+0000 7f3df8800640 1 -- start start 2024-09-21T00:18:42.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.573+0000 7f3df8800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3df0103b60 0x7f3df019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:42.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.573+0000 7f3df8800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3df01094a0 0x7f3df019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:42.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.573+0000 7f3df8800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3df0109df0 0x7f3df01a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:42.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.573+0000 7f3df8800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0113ab0 con 0x7f3df0109df0 2024-09-21T00:18:42.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.573+0000 7f3df8800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0113930 con 0x7f3df0103b60 2024-09-21T00:18:42.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.573+0000 7f3df8800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0113c30 con 0x7f3df01094a0 2024-09-21T00:18:42.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.574+0000 7f3df5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3df0103b60 0x7f3df019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:42.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.574+0000 7f3df6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3df0109df0 0x7f3df01a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:42.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.574+0000 7f3df5400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3df01094a0 0x7f3df019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:42.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.574+0000 7f3df5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3df0103b60 0x7f3df019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:55212/0 (socket says 172.21.15.27:55212) 2024-09-21T00:18:42.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.574+0000 7f3df5e00640 1 -- 172.21.15.27:0/3792629353 learned_addr learned my addr 172.21.15.27:0/3792629353 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.574+0000 7f3df5e00640 1 -- 172.21.15.27:0/3792629353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3df01094a0 msgr2=0x7f3df019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.574+0000 7f3df5e00640 1 --2- 172.21.15.27:0/3792629353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3df01094a0 0x7f3df019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.574+0000 7f3df5e00640 1 -- 172.21.15.27:0/3792629353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3df0109df0 msgr2=0x7f3df01a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.574+0000 7f3df5e00640 1 --2- 172.21.15.27:0/3792629353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3df0109df0 0x7f3df01a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.574+0000 7f3df5e00640 1 -- 172.21.15.27:0/3792629353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df01a2630 con 0x7f3df0103b60 2024-09-21T00:18:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.574+0000 7f3df5400640 1 --2- 172.21.15.27:0/3792629353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3df01094a0 0x7f3df019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:18:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.575+0000 7f3df5e00640 1 --2- 172.21.15.27:0/3792629353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3df0103b60 0x7f3df019c6c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3de4005940 tx=0x7f3de40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.575+0000 7f3deaa00640 1 -- 172.21.15.27:0/3792629353 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de403f030 con 0x7f3df0103b60 2024-09-21T00:18:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.575+0000 7f3df8800640 1 -- 172.21.15.27:0/3792629353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df01a28c0 con 0x7f3df0103b60 2024-09-21T00:18:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.575+0000 7f3deaa00640 1 -- 172.21.15.27:0/3792629353 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de40044c0 con 0x7f3df0103b60 2024-09-21T00:18:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.575+0000 7f3deaa00640 1 -- 172.21.15.27:0/3792629353 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4035070 con 0x7f3df0103b60 2024-09-21T00:18:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.575+0000 7f3df8800640 1 -- 172.21.15.27:0/3792629353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df01a9080 con 0x7f3df0103b60 2024-09-21T00:18:42.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.576+0000 7f3df8800640 1 -- 172.21.15.27:0/3792629353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df0103f70 con 0x7f3df0103b60 2024-09-21T00:18:42.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.579+0000 7f3deaa00640 1 -- 172.21.15.27:0/3792629353 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f3de4042030 con 0x7f3df0103b60 2024-09-21T00:18:42.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.580+0000 7f3deaa00640 1 --2- 172.21.15.27:0/3792629353 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3dd80b6560 0x7f3dd80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:42.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.580+0000 7f3deaa00640 1 -- 172.21.15.27:0/3792629353 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8194+0+0 (secure 0 0 0) 0x7f3de4107910 con 0x7f3df0103b60 2024-09-21T00:18:42.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.581+0000 7f3df5400640 1 --2- 172.21.15.27:0/3792629353 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3dd80b6560 0x7f3dd80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:42.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.581+0000 7f3df5400640 1 --2- 172.21.15.27:0/3792629353 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3dd80b6560 0x7f3dd80b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3df0108f50 tx=0x7f3de0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:42.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.582+0000 7f3deaa00640 1 -- 172.21.15.27:0/3792629353 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3de40d2540 con 0x7f3df0103b60 2024-09-21T00:18:42.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:42.791+0000 7f3df8800640 1 -- 172.21.15.27:0/3792629353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f3df01a9280 con 0x7f3df0103b60 2024-09-21T00:18:43.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:43.675+0000 7f3deaa00640 1 -- 172.21.15.27:0/3792629353 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) ==== 157+0+0 (secure 0 0 0) 0x7f3de40d7480 con 0x7f3df0103b60 2024-09-21T00:18:43.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:43.678+0000 7f3df8800640 1 -- 172.21.15.27:0/3792629353 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3dd80b6560 msgr2=0x7f3dd80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:43.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:43.678+0000 7f3df8800640 1 --2- 172.21.15.27:0/3792629353 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3dd80b6560 0x7f3dd80b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3df0108f50 tx=0x7f3de0005f90 comp rx=0 tx=0).stop 2024-09-21T00:18:43.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:43.678+0000 7f3df8800640 1 -- 172.21.15.27:0/3792629353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3df0103b60 msgr2=0x7f3df019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:43.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:43.678+0000 7f3df8800640 1 --2- 172.21.15.27:0/3792629353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3df0103b60 0x7f3df019c6c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3de4005940 tx=0x7f3de40040c0 comp rx=0 tx=0).stop 2024-09-21T00:18:43.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:43.678+0000 7f3df8800640 1 -- 172.21.15.27:0/3792629353 shutdown_connections 2024-09-21T00:18:43.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:43.678+0000 7f3df8800640 1 --2- 172.21.15.27:0/3792629353 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3dd80b6560 0x7f3dd80b8a70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:43.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:43.678+0000 7f3df8800640 1 --2- 172.21.15.27:0/3792629353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3df0109df0 0x7f3df01a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:43.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:43.678+0000 7f3df8800640 1 --2- 172.21.15.27:0/3792629353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3df01094a0 0x7f3df019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:43.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:43.678+0000 7f3df8800640 1 --2- 172.21.15.27:0/3792629353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3df0103b60 0x7f3df019c6c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:43.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:43.678+0000 7f3df8800640 1 -- 172.21.15.27:0/3792629353 >> 172.21.15.27:0/3792629353 conn(0x7f3df007b850 msgr2=0x7f3df01072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:43.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:43.678+0000 7f3df8800640 1 -- 172.21.15.27:0/3792629353 shutdown_connections 2024-09-21T00:18:43.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:43.678+0000 7f3df8800640 1 -- 172.21.15.27:0/3792629353 wait complete. 2024-09-21T00:18:43.978 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:18:43 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-b[63110]: 2024-09-21T00:18:43.898+0000 7f986b756640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:18:43.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:43 smithi089 ceph-mon[34481]: pgmap v286: 97 pgs: 20 creating+peering, 77 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 654 B/s wr, 5 op/s 2024-09-21T00:18:43.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:43 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3792629353' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T00:18:43.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:43 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T00:18:43.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:43 smithi098 ceph-mon[34725]: pgmap v286: 97 pgs: 20 creating+peering, 77 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 654 B/s wr, 5 op/s 2024-09-21T00:18:43.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:43 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3792629353' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T00:18:43.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:43 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T00:18:44.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:43 smithi027 ceph-mon[30745]: pgmap v286: 97 pgs: 20 creating+peering, 77 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 654 B/s wr, 5 op/s 2024-09-21T00:18:44.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:43 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3792629353' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T00:18:44.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:43 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T00:18:44.594 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T00:18:44.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:44 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T00:18:44.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:44 smithi027 ceph-mon[30745]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T00:18:44.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:44 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T00:18:44.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:44 smithi089 ceph-mon[34481]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T00:18:44.984 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:44.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:44 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T00:18:44.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:44 smithi098 ceph-mon[34725]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T00:18:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.661+0000 7f3f1e800640 1 -- 172.21.15.27:0/444180815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f18109e10 msgr2=0x7f3f18110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.661+0000 7f3f1e800640 1 --2- 172.21.15.27:0/444180815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f18109e10 0x7f3f18110640 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009ca0 tx=0x7f3f1002a8c0 comp rx=0 tx=0).stop 2024-09-21T00:18:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.661+0000 7f3f1e800640 1 -- 172.21.15.27:0/444180815 shutdown_connections 2024-09-21T00:18:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.661+0000 7f3f1e800640 1 --2- 172.21.15.27:0/444180815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f18109e10 0x7f3f18110640 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.661+0000 7f3f1e800640 1 --2- 172.21.15.27:0/444180815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f181094c0 0x7f3f181098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.661+0000 7f3f1e800640 1 --2- 172.21.15.27:0/444180815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f18103b80 0x7f3f18103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.661+0000 7f3f1e800640 1 -- 172.21.15.27:0/444180815 >> 172.21.15.27:0/444180815 conn(0x7f3f180ff5d0 msgr2=0x7f3f18101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.662+0000 7f3f1e800640 1 -- 172.21.15.27:0/444180815 shutdown_connections 2024-09-21T00:18:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.662+0000 7f3f1e800640 1 -- 172.21.15.27:0/444180815 wait complete. 2024-09-21T00:18:45.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.663+0000 7f3f1e800640 1 Processor -- start 2024-09-21T00:18:45.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.663+0000 7f3f1e800640 1 -- start start 2024-09-21T00:18:45.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.663+0000 7f3f1e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f18103b80 0x7f3f1819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:45.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.663+0000 7f3f1e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f181094c0 0x7f3f1819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:45.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.663+0000 7f3f1e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f18109e10 0x7f3f181a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:45.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.663+0000 7f3f1e800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f18113ad0 con 0x7f3f18103b80 2024-09-21T00:18:45.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.663+0000 7f3f1e800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3f18113950 con 0x7f3f181094c0 2024-09-21T00:18:45.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.663+0000 7f3f1e800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3f18113c50 con 0x7f3f18109e10 2024-09-21T00:18:45.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.664+0000 7f3f17400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f18103b80 0x7f3f1819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:45.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.664+0000 7f3f17400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f18103b80 0x7f3f1819c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35484/0 (socket says 172.21.15.27:35484) 2024-09-21T00:18:45.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.664+0000 7f3f17400640 1 -- 172.21.15.27:0/3269398674 learned_addr learned my addr 172.21.15.27:0/3269398674 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:45.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.664+0000 7f3f0fe00640 1 --2- 172.21.15.27:0/3269398674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f181094c0 0x7f3f1819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:45.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.664+0000 7f3f17e00640 1 --2- 172.21.15.27:0/3269398674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f18109e10 0x7f3f181a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:45.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.664+0000 7f3f17400640 1 -- 172.21.15.27:0/3269398674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f18109e10 msgr2=0x7f3f181a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:45.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.664+0000 7f3f17400640 1 --2- 172.21.15.27:0/3269398674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f18109e10 0x7f3f181a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:45.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.664+0000 7f3f17400640 1 -- 172.21.15.27:0/3269398674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f181094c0 msgr2=0x7f3f1819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:45.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.664+0000 7f3f17400640 1 --2- 172.21.15.27:0/3269398674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f181094c0 0x7f3f1819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:45.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.664+0000 7f3f17400640 1 -- 172.21.15.27:0/3269398674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f181a25b0 con 0x7f3f18103b80 2024-09-21T00:18:45.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.665+0000 7f3f17400640 1 --2- 172.21.15.27:0/3269398674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f18103b80 0x7f3f1819c640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800ea60 tx=0x7f3f0800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:45.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.665+0000 7f3f14c00640 1 -- 172.21.15.27:0/3269398674 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0800cde0 con 0x7f3f18103b80 2024-09-21T00:18:45.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.665+0000 7f3f1e800640 1 -- 172.21.15.27:0/3269398674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f181a28a0 con 0x7f3f18103b80 2024-09-21T00:18:45.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.665+0000 7f3f14c00640 1 -- 172.21.15.27:0/3269398674 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f080035f0 con 0x7f3f18103b80 2024-09-21T00:18:45.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.665+0000 7f3f14c00640 1 -- 172.21.15.27:0/3269398674 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08010690 con 0x7f3f18103b80 2024-09-21T00:18:45.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.665+0000 7f3f1e800640 1 -- 172.21.15.27:0/3269398674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f181a9000 con 0x7f3f18103b80 2024-09-21T00:18:45.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.666+0000 7f3f1e800640 1 -- 172.21.15.27:0/3269398674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f18103f90 con 0x7f3f18103b80 2024-09-21T00:18:45.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.669+0000 7f3f14c00640 1 -- 172.21.15.27:0/3269398674 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f3f080037a0 con 0x7f3f18103b80 2024-09-21T00:18:45.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.671+0000 7f3f14c00640 1 --2- 172.21.15.27:0/3269398674 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3efc0b65e0 0x7f3efc0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:45.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.671+0000 7f3f14c00640 1 -- 172.21.15.27:0/3269398674 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8492+0+0 (secure 0 0 0) 0x7f3f080e7ae0 con 0x7f3f18103b80 2024-09-21T00:18:45.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.671+0000 7f3f0fe00640 1 --2- 172.21.15.27:0/3269398674 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3efc0b65e0 0x7f3efc0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:45.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.672+0000 7f3f0fe00640 1 --2- 172.21.15.27:0/3269398674 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3efc0b65e0 0x7f3efc0b8af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3f18105bc0 tx=0x7f3f00005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:45.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.674+0000 7f3f14c00640 1 -- 172.21.15.27:0/3269398674 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3f080ae270 con 0x7f3f18103b80 2024-09-21T00:18:45.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:45 smithi027 ceph-mon[30745]: pgmap v288: 97 pgs: 97 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-21T00:18:45.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:45.874+0000 7f3f1e800640 1 -- 172.21.15.27:0/3269398674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3f1819d770 con 0x7f3f18103b80 2024-09-21T00:18:45.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:45 smithi089 ceph-mon[34481]: pgmap v288: 97 pgs: 97 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-21T00:18:45.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:45 smithi098 ceph-mon[34725]: pgmap v288: 97 pgs: 97 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-21T00:18:46.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:46 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3269398674' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T00:18:46.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:46 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3269398674' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T00:18:47.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:46 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3269398674' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T00:18:47.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:47.697+0000 7f3f14c00640 1 -- 172.21.15.27:0/3269398674 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) ==== 228+0+0 (secure 0 0 0) 0x7f3f080b2650 con 0x7f3f18103b80 2024-09-21T00:18:47.700 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data_ec' created 2024-09-21T00:18:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:47.701+0000 7f3f1e800640 1 -- 172.21.15.27:0/3269398674 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3efc0b65e0 msgr2=0x7f3efc0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:47.701+0000 7f3f1e800640 1 --2- 172.21.15.27:0/3269398674 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3efc0b65e0 0x7f3efc0b8af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3f18105bc0 tx=0x7f3f00005f90 comp rx=0 tx=0).stop 2024-09-21T00:18:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:47.701+0000 7f3f1e800640 1 -- 172.21.15.27:0/3269398674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f18103b80 msgr2=0x7f3f1819c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:47.701+0000 7f3f1e800640 1 --2- 172.21.15.27:0/3269398674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f18103b80 0x7f3f1819c640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800ea60 tx=0x7f3f0800ef20 comp rx=0 tx=0).stop 2024-09-21T00:18:47.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:47.702+0000 7f3f1e800640 1 -- 172.21.15.27:0/3269398674 shutdown_connections 2024-09-21T00:18:47.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:47.702+0000 7f3f1e800640 1 --2- 172.21.15.27:0/3269398674 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3efc0b65e0 0x7f3efc0b8af0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:47.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:47.702+0000 7f3f1e800640 1 --2- 172.21.15.27:0/3269398674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f18109e10 0x7f3f181a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:47.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:47.702+0000 7f3f1e800640 1 --2- 172.21.15.27:0/3269398674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f181094c0 0x7f3f1819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:47.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:47.702+0000 7f3f1e800640 1 --2- 172.21.15.27:0/3269398674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f18103b80 0x7f3f1819c640 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:47.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:47.702+0000 7f3f1e800640 1 -- 172.21.15.27:0/3269398674 >> 172.21.15.27:0/3269398674 conn(0x7f3f180ff5d0 msgr2=0x7f3f18108b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:47.702+0000 7f3f1e800640 1 -- 172.21.15.27:0/3269398674 shutdown_connections 2024-09-21T00:18:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:47.702+0000 7f3f1e800640 1 -- 172.21.15.27:0/3269398674 wait complete. 2024-09-21T00:18:47.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:47 smithi089 ceph-mon[34481]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-21T00:18:47.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:47 smithi089 ceph-mon[34481]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T00:18:47.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:47 smithi098 ceph-mon[34725]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-21T00:18:47.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:47 smithi098 ceph-mon[34725]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T00:18:48.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:47 smithi027 ceph-mon[30745]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-21T00:18:48.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:47 smithi027 ceph-mon[30745]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T00:18:48.593 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-21T00:18:48.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:48 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3269398674' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T00:18:48.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:48 smithi027 ceph-mon[30745]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T00:18:48.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:48 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3269398674' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T00:18:48.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:48 smithi089 ceph-mon[34481]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T00:18:48.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:48 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3269398674' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T00:18:48.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:48 smithi098 ceph-mon[34725]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T00:18:49.002 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.740+0000 7f1a56000640 1 -- 172.21.15.27:0/909561739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a50103990 msgr2=0x7f1a50103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.740+0000 7f1a56000640 1 --2- 172.21.15.27:0/909561739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a50103990 0x7f1a50103da0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009900 tx=0x7f1a4402a410 comp rx=0 tx=0).stop 2024-09-21T00:18:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.740+0000 7f1a56000640 1 -- 172.21.15.27:0/909561739 shutdown_connections 2024-09-21T00:18:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.740+0000 7f1a56000640 1 --2- 172.21.15.27:0/909561739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a50109c20 0x7f1a50110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.740+0000 7f1a56000640 1 --2- 172.21.15.27:0/909561739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a501092d0 0x7f1a501096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.740+0000 7f1a56000640 1 --2- 172.21.15.27:0/909561739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a50103990 0x7f1a50103da0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.740+0000 7f1a56000640 1 -- 172.21.15.27:0/909561739 >> 172.21.15.27:0/909561739 conn(0x7f1a500ff3e0 msgr2=0x7f1a50101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.740+0000 7f1a56000640 1 -- 172.21.15.27:0/909561739 shutdown_connections 2024-09-21T00:18:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.740+0000 7f1a56000640 1 -- 172.21.15.27:0/909561739 wait complete. 2024-09-21T00:18:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.741+0000 7f1a56000640 1 Processor -- start 2024-09-21T00:18:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.742+0000 7f1a56000640 1 -- start start 2024-09-21T00:18:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.742+0000 7f1a56000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a50103990 0x7f1a5019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.742+0000 7f1a56000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a501092d0 0x7f1a5019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.742+0000 7f1a56000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a50109c20 0x7f1a501a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.742+0000 7f1a56000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1a50113930 con 0x7f1a50109c20 2024-09-21T00:18:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.742+0000 7f1a56000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1a501137b0 con 0x7f1a501092d0 2024-09-21T00:18:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.742+0000 7f1a56000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1a50113ab0 con 0x7f1a50103990 2024-09-21T00:18:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.742+0000 7f1a54c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a50103990 0x7f1a5019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.742+0000 7f1a55600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a50109c20 0x7f1a501a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.742+0000 7f1a54c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a50103990 0x7f1a5019c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:52804/0 (socket says 172.21.15.27:52804) 2024-09-21T00:18:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.742+0000 7f1a54c00640 1 -- 172.21.15.27:0/2042965493 learned_addr learned my addr 172.21.15.27:0/2042965493 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.742+0000 7f1a4fe00640 1 --2- 172.21.15.27:0/2042965493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a501092d0 0x7f1a5019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:49.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.743+0000 7f1a54c00640 1 -- 172.21.15.27:0/2042965493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a501092d0 msgr2=0x7f1a5019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:49.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.743+0000 7f1a54c00640 1 --2- 172.21.15.27:0/2042965493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a501092d0 0x7f1a5019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:49.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.743+0000 7f1a54c00640 1 -- 172.21.15.27:0/2042965493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a50109c20 msgr2=0x7f1a501a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:49.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.743+0000 7f1a54c00640 1 --2- 172.21.15.27:0/2042965493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a50109c20 0x7f1a501a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:49.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.743+0000 7f1a54c00640 1 -- 172.21.15.27:0/2042965493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a501a2490 con 0x7f1a50103990 2024-09-21T00:18:49.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.743+0000 7f1a54c00640 1 --2- 172.21.15.27:0/2042965493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a50103990 0x7f1a5019c520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1a44005ab0 tx=0x7f1a44003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.744+0000 7f1a4d600640 1 -- 172.21.15.27:0/2042965493 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4403f070 con 0x7f1a50103990 2024-09-21T00:18:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.744+0000 7f1a4d600640 1 -- 172.21.15.27:0/2042965493 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a44031b60 con 0x7f1a50103990 2024-09-21T00:18:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.744+0000 7f1a56000640 1 -- 172.21.15.27:0/2042965493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a501a2720 con 0x7f1a50103990 2024-09-21T00:18:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.744+0000 7f1a4d600640 1 -- 172.21.15.27:0/2042965493 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44035030 con 0x7f1a50103990 2024-09-21T00:18:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.744+0000 7f1a56000640 1 -- 172.21.15.27:0/2042965493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a501a8ee0 con 0x7f1a50103990 2024-09-21T00:18:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.745+0000 7f1a56000640 1 -- 172.21.15.27:0/2042965493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a50104c10 con 0x7f1a50103990 2024-09-21T00:18:49.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.747+0000 7f1a4d600640 1 -- 172.21.15.27:0/2042965493 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f1a44042070 con 0x7f1a50103990 2024-09-21T00:18:49.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.748+0000 7f1a4d600640 1 --2- 172.21.15.27:0/2042965493 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a380b6560 0x7f1a380b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:49.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.749+0000 7f1a4d600640 1 -- 172.21.15.27:0/2042965493 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8985+0+0 (secure 0 0 0) 0x7f1a44107d80 con 0x7f1a50103990 2024-09-21T00:18:49.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.749+0000 7f1a4fe00640 1 --2- 172.21.15.27:0/2042965493 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a380b6560 0x7f1a380b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:49.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.749+0000 7f1a4fe00640 1 --2- 172.21.15.27:0/2042965493 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a380b6560 0x7f1a380b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1a50105a20 tx=0x7f1a3c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:49.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.753+0000 7f1a4d600640 1 -- 172.21.15.27:0/2042965493 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1a440d26a0 con 0x7f1a50103990 2024-09-21T00:18:49.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:49 smithi027 ceph-mon[30745]: pgmap v292: 129 pgs: 13 creating+peering, 19 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-09-21T00:18:49.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:49 smithi027 ceph-mon[30745]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T00:18:49.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:49.951+0000 7f1a56000640 1 -- 172.21.15.27:0/2042965493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f1a501a90b0 con 0x7f1a50103990 2024-09-21T00:18:49.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:49 smithi089 ceph-mon[34481]: pgmap v292: 129 pgs: 13 creating+peering, 19 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-09-21T00:18:49.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:49 smithi089 ceph-mon[34481]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T00:18:49.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:49 smithi098 ceph-mon[34725]: pgmap v292: 129 pgs: 13 creating+peering, 19 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-09-21T00:18:49.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:49 smithi098 ceph-mon[34725]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T00:18:50.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:50.725+0000 7f1a4d600640 1 -- 172.21.15.27:0/2042965493 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7f1a440d75e0 con 0x7f1a50103990 2024-09-21T00:18:50.728 INFO:teuthology.orchestra.run.smithi027.stderr:set pool 4 allow_ec_overwrites to true 2024-09-21T00:18:50.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:50.728+0000 7f1a56000640 1 -- 172.21.15.27:0/2042965493 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a380b6560 msgr2=0x7f1a380b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:50.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:50.728+0000 7f1a56000640 1 --2- 172.21.15.27:0/2042965493 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a380b6560 0x7f1a380b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1a50105a20 tx=0x7f1a3c005f90 comp rx=0 tx=0).stop 2024-09-21T00:18:50.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:50.728+0000 7f1a56000640 1 -- 172.21.15.27:0/2042965493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a50103990 msgr2=0x7f1a5019c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:50.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:50.728+0000 7f1a56000640 1 --2- 172.21.15.27:0/2042965493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a50103990 0x7f1a5019c520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1a44005ab0 tx=0x7f1a44003c10 comp rx=0 tx=0).stop 2024-09-21T00:18:50.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:50.729+0000 7f1a56000640 1 -- 172.21.15.27:0/2042965493 shutdown_connections 2024-09-21T00:18:50.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:50.729+0000 7f1a56000640 1 --2- 172.21.15.27:0/2042965493 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a380b6560 0x7f1a380b8a70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:50.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:50.729+0000 7f1a56000640 1 --2- 172.21.15.27:0/2042965493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a50109c20 0x7f1a501a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:50.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:50.729+0000 7f1a56000640 1 --2- 172.21.15.27:0/2042965493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a501092d0 0x7f1a5019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:50.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:50.729+0000 7f1a56000640 1 --2- 172.21.15.27:0/2042965493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a50103990 0x7f1a5019c520 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:50.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:50.729+0000 7f1a56000640 1 -- 172.21.15.27:0/2042965493 >> 172.21.15.27:0/2042965493 conn(0x7f1a500ff3e0 msgr2=0x7f1a501089b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:50.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:50.729+0000 7f1a56000640 1 -- 172.21.15.27:0/2042965493 shutdown_connections 2024-09-21T00:18:50.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:50.729+0000 7f1a56000640 1 -- 172.21.15.27:0/2042965493 wait complete. 2024-09-21T00:18:50.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:50 smithi089 ceph-mon[34481]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T00:18:50.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:50 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2042965493' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T00:18:50.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:50 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T00:18:50.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:50 smithi098 ceph-mon[34725]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T00:18:50.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:50 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2042965493' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T00:18:50.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:50 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T00:18:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:50 smithi027 ceph-mon[30745]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T00:18:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:50 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2042965493' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T00:18:51.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:50 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T00:18:51.537 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-21T00:18:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:51 smithi027 ceph-mon[30745]: pgmap v295: 129 pgs: 13 creating+peering, 12 unknown, 104 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:51 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T00:18:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:51 smithi027 ceph-mon[30745]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T00:18:51.932 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:51.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:51 smithi089 ceph-mon[34481]: pgmap v295: 129 pgs: 13 creating+peering, 12 unknown, 104 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:51.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:51 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T00:18:51.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:51 smithi089 ceph-mon[34481]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T00:18:51.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:51 smithi098 ceph-mon[34725]: pgmap v295: 129 pgs: 13 creating+peering, 12 unknown, 104 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:51.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:51 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T00:18:51.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:51 smithi098 ceph-mon[34725]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T00:18:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.605+0000 7f39d0c00640 1 -- 172.21.15.27:0/1538135520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39cc1094a0 msgr2=0x7f39cc1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.605+0000 7f39d0c00640 1 --2- 172.21.15.27:0/1538135520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39cc1094a0 0x7f39cc1098b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009900 tx=0x7f39b402a410 comp rx=0 tx=0).stop 2024-09-21T00:18:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.605+0000 7f39d0c00640 1 -- 172.21.15.27:0/1538135520 shutdown_connections 2024-09-21T00:18:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.605+0000 7f39d0c00640 1 --2- 172.21.15.27:0/1538135520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39cc109df0 0x7f39cc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.605+0000 7f39d0c00640 1 --2- 172.21.15.27:0/1538135520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39cc1094a0 0x7f39cc1098b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.605+0000 7f39d0c00640 1 --2- 172.21.15.27:0/1538135520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39cc103b60 0x7f39cc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.605+0000 7f39d0c00640 1 -- 172.21.15.27:0/1538135520 >> 172.21.15.27:0/1538135520 conn(0x7f39cc07b850 msgr2=0x7f39cc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.605+0000 7f39d0c00640 1 -- 172.21.15.27:0/1538135520 shutdown_connections 2024-09-21T00:18:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.605+0000 7f39d0c00640 1 -- 172.21.15.27:0/1538135520 wait complete. 2024-09-21T00:18:52.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.606+0000 7f39d0c00640 1 Processor -- start 2024-09-21T00:18:52.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.606+0000 7f39d0c00640 1 -- start start 2024-09-21T00:18:52.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.607+0000 7f39d0c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39cc103b60 0x7f39cc19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.607+0000 7f39d0c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39cc1094a0 0x7f39cc19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.607+0000 7f39d0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39cc109df0 0x7f39cc19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.607+0000 7f39d0c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc113b00 con 0x7f39cc109df0 2024-09-21T00:18:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.607+0000 7f39d0c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc113980 con 0x7f39cc1094a0 2024-09-21T00:18:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.607+0000 7f39d0c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc113c80 con 0x7f39cc103b60 2024-09-21T00:18:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.607+0000 7f39c9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39cc103b60 0x7f39cc19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.607+0000 7f39ca800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39cc109df0 0x7f39cc19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.607+0000 7f39c9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39cc103b60 0x7f39cc19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:52836/0 (socket says 172.21.15.27:52836) 2024-09-21T00:18:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.607+0000 7f39c9e00640 1 -- 172.21.15.27:0/3464259962 learned_addr learned my addr 172.21.15.27:0/3464259962 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.607+0000 7f39c1400640 1 --2- 172.21.15.27:0/3464259962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39cc1094a0 0x7f39cc19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:52.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.608+0000 7f39c9e00640 1 -- 172.21.15.27:0/3464259962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39cc1094a0 msgr2=0x7f39cc19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:52.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.608+0000 7f39c9e00640 1 --2- 172.21.15.27:0/3464259962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39cc1094a0 0x7f39cc19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:52.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.608+0000 7f39c9e00640 1 -- 172.21.15.27:0/3464259962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39cc109df0 msgr2=0x7f39cc19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:52.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.608+0000 7f39c9e00640 1 --2- 172.21.15.27:0/3464259962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39cc109df0 0x7f39cc19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:52.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.608+0000 7f39c9e00640 1 -- 172.21.15.27:0/3464259962 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39cc1a03e0 con 0x7f39cc103b60 2024-09-21T00:18:52.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.608+0000 7f39ca800640 1 --2- 172.21.15.27:0/3464259962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39cc109df0 0x7f39cc19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:18:52.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.608+0000 7f39c9e00640 1 --2- 172.21.15.27:0/3464259962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39cc103b60 0x7f39cc19a470 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00e9f0 tx=0x7f39bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:52.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.608+0000 7f39c3400640 1 -- 172.21.15.27:0/3464259962 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc00cd80 con 0x7f39cc103b60 2024-09-21T00:18:52.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.608+0000 7f39d0c00640 1 -- 172.21.15.27:0/3464259962 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39cc1a06d0 con 0x7f39cc103b60 2024-09-21T00:18:52.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.608+0000 7f39c3400640 1 -- 172.21.15.27:0/3464259962 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39bc0108e0 con 0x7f39cc103b60 2024-09-21T00:18:52.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.608+0000 7f39c3400640 1 -- 172.21.15.27:0/3464259962 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc018980 con 0x7f39cc103b60 2024-09-21T00:18:52.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.608+0000 7f39d0c00640 1 -- 172.21.15.27:0/3464259962 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39cc1a6e30 con 0x7f39cc103b60 2024-09-21T00:18:52.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.609+0000 7f39d0c00640 1 -- 172.21.15.27:0/3464259962 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39cc103f70 con 0x7f39cc103b60 2024-09-21T00:18:52.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.612+0000 7f39c3400640 1 -- 172.21.15.27:0/3464259962 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f39bc010430 con 0x7f39cc103b60 2024-09-21T00:18:52.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.614+0000 7f39c3400640 1 --2- 172.21.15.27:0/3464259962 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f39b00b6560 0x7f39b00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:52.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.614+0000 7f39c3400640 1 -- 172.21.15.27:0/3464259962 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8985+0+0 (secure 0 0 0) 0x7f39bc0e7e70 con 0x7f39cc103b60 2024-09-21T00:18:52.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.614+0000 7f39c1400640 1 --2- 172.21.15.27:0/3464259962 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f39b00b6560 0x7f39b00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:52.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.615+0000 7f39c1400640 1 --2- 172.21.15.27:0/3464259962 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f39b00b6560 0x7f39b00b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f39b402aa00 tx=0x7f39b4005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:52.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.615+0000 7f39c3400640 1 -- 172.21.15.27:0/3464259962 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f39bc0b2790 con 0x7f39cc103b60 2024-09-21T00:18:52.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:52.857+0000 7f39d0c00640 1 -- 172.21.15.27:0/3464259962 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f39cc10bbc0 con 0x7f39cc103b60 2024-09-21T00:18:53.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:53.740+0000 7f39c3400640 1 -- 172.21.15.27:0/3464259962 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7f39bc0b76d0 con 0x7f39cc103b60 2024-09-21T00:18:53.742 INFO:teuthology.orchestra.run.smithi027.stderr:added data pool 4 to fsmap 2024-09-21T00:18:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:53.743+0000 7f39d0c00640 1 -- 172.21.15.27:0/3464259962 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f39b00b6560 msgr2=0x7f39b00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:53.743+0000 7f39d0c00640 1 --2- 172.21.15.27:0/3464259962 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f39b00b6560 0x7f39b00b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f39b402aa00 tx=0x7f39b4005a20 comp rx=0 tx=0).stop 2024-09-21T00:18:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:53.743+0000 7f39d0c00640 1 -- 172.21.15.27:0/3464259962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39cc103b60 msgr2=0x7f39cc19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:53.743+0000 7f39d0c00640 1 --2- 172.21.15.27:0/3464259962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39cc103b60 0x7f39cc19a470 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00e9f0 tx=0x7f39bc00eeb0 comp rx=0 tx=0).stop 2024-09-21T00:18:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:53.743+0000 7f39d0c00640 1 -- 172.21.15.27:0/3464259962 shutdown_connections 2024-09-21T00:18:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:53.743+0000 7f39d0c00640 1 --2- 172.21.15.27:0/3464259962 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f39b00b6560 0x7f39b00b8a70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:53.744+0000 7f39d0c00640 1 --2- 172.21.15.27:0/3464259962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39cc109df0 0x7f39cc19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:53.744+0000 7f39d0c00640 1 --2- 172.21.15.27:0/3464259962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39cc1094a0 0x7f39cc19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:53.744+0000 7f39d0c00640 1 --2- 172.21.15.27:0/3464259962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39cc103b60 0x7f39cc19a470 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:53.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:53.744+0000 7f39d0c00640 1 -- 172.21.15.27:0/3464259962 >> 172.21.15.27:0/3464259962 conn(0x7f39cc07b850 msgr2=0x7f39cc1054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:53.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:53.744+0000 7f39d0c00640 1 -- 172.21.15.27:0/3464259962 shutdown_connections 2024-09-21T00:18:53.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:53.744+0000 7f39d0c00640 1 -- 172.21.15.27:0/3464259962 wait complete. 2024-09-21T00:18:53.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:53 smithi098 ceph-mon[34725]: pgmap v297: 129 pgs: 13 creating+peering, 116 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:53.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:53 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3464259962' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T00:18:53.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:53 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T00:18:54.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:53 smithi027 ceph-mon[30745]: pgmap v297: 129 pgs: 13 creating+peering, 116 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:54.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:53 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3464259962' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T00:18:54.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:53 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T00:18:54.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:53 smithi089 ceph-mon[34481]: pgmap v297: 129 pgs: 13 creating+peering, 116 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:54.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:53 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3464259962' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T00:18:54.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:53 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T00:18:54.723 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:18:54.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:54 smithi098 ceph-mon[34725]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T00:18:54.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:54 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T00:18:54.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:54 smithi098 ceph-mon[34725]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T00:18:55.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:54 smithi027 ceph-mon[30745]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T00:18:55.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:54 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T00:18:55.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:54 smithi027 ceph-mon[30745]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T00:18:55.114 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:55.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:54 smithi089 ceph-mon[34481]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T00:18:55.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:54 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T00:18:55.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:54 smithi089 ceph-mon[34481]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T00:18:55.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.755+0000 7ffa35c00640 1 -- 172.21.15.27:0/4209966683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa301039b0 msgr2=0x7ffa30103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:55.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.755+0000 7ffa35c00640 1 --2- 172.21.15.27:0/4209966683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa301039b0 0x7ffa30103dc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ffa20009860 tx=0x7ffa2002a340 comp rx=0 tx=0).stop 2024-09-21T00:18:55.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.756+0000 7ffa35c00640 1 -- 172.21.15.27:0/4209966683 shutdown_connections 2024-09-21T00:18:55.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.756+0000 7ffa35c00640 1 --2- 172.21.15.27:0/4209966683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa30109c40 0x7ffa30110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:55.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.756+0000 7ffa35c00640 1 --2- 172.21.15.27:0/4209966683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa301092f0 0x7ffa30109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:55.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.756+0000 7ffa35c00640 1 --2- 172.21.15.27:0/4209966683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa301039b0 0x7ffa30103dc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:55.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.756+0000 7ffa35c00640 1 -- 172.21.15.27:0/4209966683 >> 172.21.15.27:0/4209966683 conn(0x7ffa300ff440 msgr2=0x7ffa30101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:55.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.756+0000 7ffa35c00640 1 -- 172.21.15.27:0/4209966683 shutdown_connections 2024-09-21T00:18:55.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.756+0000 7ffa35c00640 1 -- 172.21.15.27:0/4209966683 wait complete. 2024-09-21T00:18:55.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.757+0000 7ffa35c00640 1 Processor -- start 2024-09-21T00:18:55.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.757+0000 7ffa35c00640 1 -- start start 2024-09-21T00:18:55.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.757+0000 7ffa35c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa301039b0 0x7ffa3019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:55.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.757+0000 7ffa35c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa301092f0 0x7ffa3019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:55.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.757+0000 7ffa35c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa30109c40 0x7ffa3019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:55.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.757+0000 7ffa35c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffa30113900 con 0x7ffa30109c40 2024-09-21T00:18:55.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.757+0000 7ffa35c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffa30113780 con 0x7ffa301039b0 2024-09-21T00:18:55.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.757+0000 7ffa35c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffa30113a80 con 0x7ffa301092f0 2024-09-21T00:18:55.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.758+0000 7ffa2f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa301092f0 0x7ffa3019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:55.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.758+0000 7ffa2fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa301039b0 0x7ffa3019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:55.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.758+0000 7ffa35200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa30109c40 0x7ffa3019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:55.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.758+0000 7ffa2f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa301092f0 0x7ffa3019a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38610/0 (socket says 172.21.15.27:38610) 2024-09-21T00:18:55.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.758+0000 7ffa2f400640 1 -- 172.21.15.27:0/4018599999 learned_addr learned my addr 172.21.15.27:0/4018599999 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:55.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.758+0000 7ffa2fe00640 1 -- 172.21.15.27:0/4018599999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa301092f0 msgr2=0x7ffa3019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:55.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.758+0000 7ffa2fe00640 1 --2- 172.21.15.27:0/4018599999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa301092f0 0x7ffa3019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:55.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.758+0000 7ffa2fe00640 1 -- 172.21.15.27:0/4018599999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa30109c40 msgr2=0x7ffa3019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:55.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.758+0000 7ffa2fe00640 1 --2- 172.21.15.27:0/4018599999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa30109c40 0x7ffa3019fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:55.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.758+0000 7ffa2fe00640 1 -- 172.21.15.27:0/4018599999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa301a0260 con 0x7ffa301039b0 2024-09-21T00:18:55.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.758+0000 7ffa2f400640 1 --2- 172.21.15.27:0/4018599999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa301092f0 0x7ffa3019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:18:55.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.759+0000 7ffa2fe00640 1 --2- 172.21.15.27:0/4018599999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa301039b0 0x7ffa3019a2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffa20035040 tx=0x7ffa20032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:55.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.759+0000 7ffa2cc00640 1 -- 172.21.15.27:0/4018599999 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20037040 con 0x7ffa301039b0 2024-09-21T00:18:55.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.759+0000 7ffa2cc00640 1 -- 172.21.15.27:0/4018599999 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa20004050 con 0x7ffa301039b0 2024-09-21T00:18:55.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.759+0000 7ffa35c00640 1 -- 172.21.15.27:0/4018599999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa301a04f0 con 0x7ffa301039b0 2024-09-21T00:18:55.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.759+0000 7ffa2cc00640 1 -- 172.21.15.27:0/4018599999 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa200035f0 con 0x7ffa301039b0 2024-09-21T00:18:55.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.759+0000 7ffa35c00640 1 -- 172.21.15.27:0/4018599999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa301a6cb0 con 0x7ffa301039b0 2024-09-21T00:18:55.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.760+0000 7ffa35c00640 1 -- 172.21.15.27:0/4018599999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa30104c30 con 0x7ffa301039b0 2024-09-21T00:18:55.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.762+0000 7ffa2cc00640 1 -- 172.21.15.27:0/4018599999 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7ffa20043020 con 0x7ffa301039b0 2024-09-21T00:18:55.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.764+0000 7ffa2cc00640 1 --2- 172.21.15.27:0/4018599999 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ffa140b65e0 0x7ffa140b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:55.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.764+0000 7ffa2cc00640 1 -- 172.21.15.27:0/4018599999 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7ffa2006fe00 con 0x7ffa301039b0 2024-09-21T00:18:55.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.764+0000 7ffa2f400640 1 --2- 172.21.15.27:0/4018599999 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ffa140b65e0 0x7ffa140b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:55.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.765+0000 7ffa2f400640 1 --2- 172.21.15.27:0/4018599999 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ffa140b65e0 0x7ffa140b8af0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffa30108e60 tx=0x7ffa1c009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:55.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:55.766+0000 7ffa2cc00640 1 -- 172.21.15.27:0/4018599999 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ffa2004a060 con 0x7ffa301039b0 2024-09-21T00:18:55.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:55 smithi098 ceph-mon[34725]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:56.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:56.010+0000 7ffa35c00640 1 -- 172.21.15.27:0/4018599999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa3002e640 con 0x7ffa301039b0 2024-09-21T00:18:56.012 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:55 smithi027 ceph-mon[30745]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:56.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:56.011+0000 7ffa2cc00640 1 -- 172.21.15.27:0/4018599999 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11821 (secure 0 0 0) 0x7ffa200ccfe0 con 0x7ffa301039b0 2024-09-21T00:18:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:18:56.013 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"btime":"2024-09-21T00:18:53:726209+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24703,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24715,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24722,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24727,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:18:52.860449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24761,"qdb_cluster":[24761]},"id":1}]} 2024-09-21T00:18:56.013 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 16 2024-09-21T00:18:56.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:56.014+0000 7ffa35c00640 1 -- 172.21.15.27:0/4018599999 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ffa140b65e0 msgr2=0x7ffa140b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:56.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:56.014+0000 7ffa35c00640 1 --2- 172.21.15.27:0/4018599999 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ffa140b65e0 0x7ffa140b8af0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffa30108e60 tx=0x7ffa1c009260 comp rx=0 tx=0).stop 2024-09-21T00:18:56.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:56.014+0000 7ffa35c00640 1 -- 172.21.15.27:0/4018599999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa301039b0 msgr2=0x7ffa3019a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:56.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:56.014+0000 7ffa35c00640 1 --2- 172.21.15.27:0/4018599999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa301039b0 0x7ffa3019a2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffa20035040 tx=0x7ffa20032d50 comp rx=0 tx=0).stop 2024-09-21T00:18:56.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:56.014+0000 7ffa35c00640 1 -- 172.21.15.27:0/4018599999 shutdown_connections 2024-09-21T00:18:56.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:56.014+0000 7ffa35c00640 1 --2- 172.21.15.27:0/4018599999 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ffa140b65e0 0x7ffa140b8af0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:56.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:56.014+0000 7ffa35c00640 1 --2- 172.21.15.27:0/4018599999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa30109c40 0x7ffa3019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:56.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:56.014+0000 7ffa35c00640 1 --2- 172.21.15.27:0/4018599999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa301092f0 0x7ffa3019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:56.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:56.014+0000 7ffa35c00640 1 --2- 172.21.15.27:0/4018599999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa301039b0 0x7ffa3019a2f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:56.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:56.014+0000 7ffa35c00640 1 -- 172.21.15.27:0/4018599999 >> 172.21.15.27:0/4018599999 conn(0x7ffa300ff440 msgr2=0x7ffa30101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:56.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:56.014+0000 7ffa35c00640 1 -- 172.21.15.27:0/4018599999 shutdown_connections 2024-09-21T00:18:56.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:56.014+0000 7ffa35c00640 1 -- 172.21.15.27:0/4018599999 wait complete. 2024-09-21T00:18:56.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:55 smithi089 ceph-mon[34481]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:56.869 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T00:18:57.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:56 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/4018599999' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:57.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:56 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/4018599999' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:57.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:56 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/4018599999' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:18:57.259 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:57.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.891+0000 7f2de3200640 1 -- 172.21.15.27:0/995084771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2de4114260 msgr2=0x7f2de4116730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.891+0000 7f2de3200640 1 --2- 172.21.15.27:0/995084771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2de4114260 0x7f2de4116730 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009ca0 tx=0x7f2ddc02a8c0 comp rx=0 tx=0).stop 2024-09-21T00:18:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.891+0000 7f2de3200640 1 -- 172.21.15.27:0/995084771 shutdown_connections 2024-09-21T00:18:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.891+0000 7f2de3200640 1 --2- 172.21.15.27:0/995084771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2de4114260 0x7f2de4116730 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.891+0000 7f2de3200640 1 --2- 172.21.15.27:0/995084771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2de4075dd0 0x7f2de40761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.891+0000 7f2de3200640 1 --2- 172.21.15.27:0/995084771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2de4077380 0x7f2de40757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.891+0000 7f2de3200640 1 -- 172.21.15.27:0/995084771 >> 172.21.15.27:0/995084771 conn(0x7f2de40ff3e0 msgr2=0x7f2de4101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.891+0000 7f2de3200640 1 -- 172.21.15.27:0/995084771 shutdown_connections 2024-09-21T00:18:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.891+0000 7f2de3200640 1 -- 172.21.15.27:0/995084771 wait complete. 2024-09-21T00:18:57.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.892+0000 7f2de3200640 1 Processor -- start 2024-09-21T00:18:57.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.892+0000 7f2de3200640 1 -- start start 2024-09-21T00:18:57.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.893+0000 7f2de3200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2de4075dd0 0x7f2de41a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:57.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.893+0000 7f2de3200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2de4077380 0x7f2de41a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:57.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.893+0000 7f2de3200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2de4114260 0x7f2de41a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:57.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.893+0000 7f2de3200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4119ea0 con 0x7f2de4075dd0 2024-09-21T00:18:57.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.893+0000 7f2de3200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4119d20 con 0x7f2de4077380 2024-09-21T00:18:57.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.893+0000 7f2de3200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2de411a020 con 0x7f2de4114260 2024-09-21T00:18:57.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.893+0000 7f2de1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2de4075dd0 0x7f2de41a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:57.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.893+0000 7f2de1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2de4075dd0 0x7f2de41a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46258/0 (socket says 172.21.15.27:46258) 2024-09-21T00:18:57.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.893+0000 7f2de1e00640 1 -- 172.21.15.27:0/959436936 learned_addr learned my addr 172.21.15.27:0/959436936 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:57.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.893+0000 7f2de1400640 1 --2- 172.21.15.27:0/959436936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2de4077380 0x7f2de41a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:57.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.893+0000 7f2de2800640 1 --2- 172.21.15.27:0/959436936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2de4114260 0x7f2de41a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:57.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.893+0000 7f2de1e00640 1 -- 172.21.15.27:0/959436936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2de4114260 msgr2=0x7f2de41a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:57.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.893+0000 7f2de1e00640 1 --2- 172.21.15.27:0/959436936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2de4114260 0x7f2de41a60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:57.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.893+0000 7f2de1e00640 1 -- 172.21.15.27:0/959436936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2de4077380 msgr2=0x7f2de41a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:57.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.893+0000 7f2de1e00640 1 --2- 172.21.15.27:0/959436936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2de4077380 0x7f2de41a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:57.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.894+0000 7f2de1e00640 1 -- 172.21.15.27:0/959436936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de41a6800 con 0x7f2de4075dd0 2024-09-21T00:18:57.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.894+0000 7f2de1400640 1 --2- 172.21.15.27:0/959436936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2de4077380 0x7f2de41a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:18:57.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.894+0000 7f2de1e00640 1 --2- 172.21.15.27:0/959436936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2de4075dd0 0x7f2de41a0890 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400e9f0 tx=0x7f2dd400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:57.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.894+0000 7f2ddaa00640 1 -- 172.21.15.27:0/959436936 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd400cd80 con 0x7f2de4075dd0 2024-09-21T00:18:57.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.894+0000 7f2ddaa00640 1 -- 172.21.15.27:0/959436936 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dd40054d0 con 0x7f2de4075dd0 2024-09-21T00:18:57.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.894+0000 7f2ddaa00640 1 -- 172.21.15.27:0/959436936 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4010690 con 0x7f2de4075dd0 2024-09-21T00:18:57.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.894+0000 7f2de3200640 1 -- 172.21.15.27:0/959436936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de41a6af0 con 0x7f2de4075dd0 2024-09-21T00:18:57.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.894+0000 7f2de3200640 1 -- 172.21.15.27:0/959436936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de41ad250 con 0x7f2de4075dd0 2024-09-21T00:18:57.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.895+0000 7f2de3200640 1 -- 172.21.15.27:0/959436936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de40761e0 con 0x7f2de4075dd0 2024-09-21T00:18:57.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.900+0000 7f2ddaa00640 1 -- 172.21.15.27:0/959436936 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f2dd4004e30 con 0x7f2de4075dd0 2024-09-21T00:18:57.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.901+0000 7f2ddaa00640 1 --2- 172.21.15.27:0/959436936 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2dc80b65e0 0x7f2dc80b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:57.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.901+0000 7f2ddaa00640 1 -- 172.21.15.27:0/959436936 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f2dd40e7bb0 con 0x7f2de4075dd0 2024-09-21T00:18:57.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.902+0000 7f2de1400640 1 --2- 172.21.15.27:0/959436936 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2dc80b65e0 0x7f2dc80b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:57.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.902+0000 7f2de1400640 1 --2- 172.21.15.27:0/959436936 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2dc80b65e0 0x7f2dc80b8af0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2de41161d0 tx=0x7f2dd0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:57.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:57.904+0000 7f2ddaa00640 1 -- 172.21.15.27:0/959436936 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2dd40ae130 con 0x7f2de4075dd0 2024-09-21T00:18:58.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:57 smithi027 ceph-mon[30745]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:58.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:58.109+0000 7f2de3200640 1 -- 172.21.15.27:0/959436936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2de41a19c0 con 0x7f2de4075dd0 2024-09-21T00:18:58.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:58.111+0000 7f2ddaa00640 1 -- 172.21.15.27:0/959436936 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24427 (secure 0 0 0) 0x7f2dd40b2510 con 0x7f2de4075dd0 2024-09-21T00:18:58.112 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:18:58.113 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":86,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","created":"2024-09-21T00:09:20.149938+0000","modified":"2024-09-21T00:18:52.860270+0000","last_up_change":"2024-09-21T00:17:43.978785+0000","last_in_change":"2024-09-21T00:17:21.325177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:13:32.718543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:18:33.992414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:18:36.817691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:18:46.707770+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63ccc094-62d5-446f-844d-1c60d41fda1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6803","nonce":354370090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6805","nonce":354370090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6809","nonce":354370090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6807","nonce":354370090}]},"public_addr":"172.21.15.27:6803/354370090","cluster_addr":"172.21.15.27:6805/354370090","heartbeat_back_addr":"172.21.15.27:6809/354370090","heartbeat_front_addr":"172.21.15.27:6807/354370090","state":["exists","up"]},{"osd":1,"uuid":"e59a8ed4-3a37-4477-a60a-4abc33d49b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6801","nonce":922528621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6803","nonce":922528621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6807","nonce":922528621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6805","nonce":922528621}]},"public_addr":"172.21.15.89:6801/922528621","cluster_addr":"172.21.15.89:6803/922528621","heartbeat_back_addr":"172.21.15.89:6807/922528621","heartbeat_front_addr":"172.21.15.89:6805/922528621","state":["exists","up"]},{"osd":2,"uuid":"6058a875-d5b2-4459-9407-2fd190cb34a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6801","nonce":257636259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6803","nonce":257636259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6807","nonce":257636259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6805","nonce":257636259}]},"public_addr":"172.21.15.98:6801/257636259","cluster_addr":"172.21.15.98:6803/257636259","heartbeat_back_addr":"172.21.15.98:6807/257636259","heartbeat_front_addr":"172.21.15.98:6805/257636259","state":["exists","up"]},{"osd":3,"uuid":"dfcdaede-e51c-4985-92af-3e296435a36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6811","nonce":821088824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6813","nonce":821088824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6817","nonce":821088824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6815","nonce":821088824}]},"public_addr":"172.21.15.27:6811/821088824","cluster_addr":"172.21.15.27:6813/821088824","heartbeat_back_addr":"172.21.15.27:6817/821088824","heartbeat_front_addr":"172.21.15.27:6815/821088824","state":["exists","up"]},{"osd":4,"uuid":"7c4af037-8ab6-47c0-b62b-76bd775b3491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6809","nonce":4001038690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6811","nonce":4001038690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6815","nonce":4001038690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6813","nonce":4001038690}]},"public_addr":"172.21.15.89:6809/4001038690","cluster_addr":"172.21.15.89:6811/4001038690","heartbeat_back_addr":"172.21.15.89:6815/4001038690","heartbeat_front_addr":"172.21.15.89:6813/4001038690","state":["exists","up"]},{"osd":5,"uuid":"e996ed0d-88ad-4aa7-891f-dfade4e09cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6809","nonce":3029903562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6811","nonce":3029903562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6815","nonce":3029903562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6813","nonce":3029903562}]},"public_addr":"172.21.15.98:6809/3029903562","cluster_addr":"172.21.15.98:6811/3029903562","heartbeat_back_addr":"172.21.15.98:6815/3029903562","heartbeat_front_addr":"172.21.15.98:6813/3029903562","state":["exists","up"]},{"osd":6,"uuid":"cf080497-57c2-42d0-8c8f-1eed0110702d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6819","nonce":1286808857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6821","nonce":1286808857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6825","nonce":1286808857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6823","nonce":1286808857}]},"public_addr":"172.21.15.27:6819/1286808857","cluster_addr":"172.21.15.27:6821/1286808857","heartbeat_back_addr":"172.21.15.27:6825/1286808857","heartbeat_front_addr":"172.21.15.27:6823/1286808857","state":["exists","up"]},{"osd":7,"uuid":"ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6817","nonce":2479349546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6819","nonce":2479349546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6823","nonce":2479349546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6821","nonce":2479349546}]},"public_addr":"172.21.15.89:6817/2479349546","cluster_addr":"172.21.15.89:6819/2479349546","heartbeat_back_addr":"172.21.15.89:6823/2479349546","heartbeat_front_addr":"172.21.15.89:6821/2479349546","state":["exists","up"]},{"osd":8,"uuid":"0b558458-fc2d-4ed1-b72a-e2ea8cf811b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6817","nonce":2830381485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6819","nonce":2830381485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6823","nonce":2830381485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6821","nonce":2830381485}]},"public_addr":"172.21.15.98:6817/2830381485","cluster_addr":"172.21.15.98:6819/2830381485","heartbeat_back_addr":"172.21.15.98:6823/2830381485","heartbeat_front_addr":"172.21.15.98:6821/2830381485","state":["exists","up"]},{"osd":9,"uuid":"1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6827","nonce":3033977113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6829","nonce":3033977113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6833","nonce":3033977113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6831","nonce":3033977113}]},"public_addr":"172.21.15.27:6827/3033977113","cluster_addr":"172.21.15.27:6829/3033977113","heartbeat_back_addr":"172.21.15.27:6833/3033977113","heartbeat_front_addr":"172.21.15.27:6831/3033977113","state":["exists","up"]},{"osd":10,"uuid":"324dd6ce-68fe-4c76-959b-1c9fb0ae92a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6825","nonce":2370332225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6827","nonce":2370332225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6831","nonce":2370332225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6829","nonce":2370332225}]},"public_addr":"172.21.15.89:6825/2370332225","cluster_addr":"172.21.15.89:6827/2370332225","heartbeat_back_addr":"172.21.15.89:6831/2370332225","heartbeat_front_addr":"172.21.15.89:6829/2370332225","state":["exists","up"]},{"osd":11,"uuid":"5e34686e-350a-491e-b9d7-59174ea90944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6825","nonce":2795999064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6827","nonce":2795999064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6831","nonce":2795999064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6829","nonce":2795999064}]},"public_addr":"172.21.15.98:6825/2795999064","cluster_addr":"172.21.15.98:6827/2795999064","heartbeat_back_addr":"172.21.15.98:6831/2795999064","heartbeat_front_addr":"172.21.15.98:6829/2795999064","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:12:39.841958+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:06.684417+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:31.041378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:01.612108+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:28.592326+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:54.303714+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:25.523475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:52.523483+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:19.430503+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:50.599120+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:17.520452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:42.725550+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/502883064":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/831048090":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/3880378905":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/793991452":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/2077956479":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6800/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6801/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/1098432608":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6800/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/528019907":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/2173601508":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:6800/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/764776233":"2024-09-22T00:11:24.231101+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:18:58.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:58.114+0000 7f2de3200640 1 -- 172.21.15.27:0/959436936 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2dc80b65e0 msgr2=0x7f2dc80b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:58.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:58.114+0000 7f2de3200640 1 --2- 172.21.15.27:0/959436936 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2dc80b65e0 0x7f2dc80b8af0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2de41161d0 tx=0x7f2dd0005f90 comp rx=0 tx=0).stop 2024-09-21T00:18:58.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:58.114+0000 7f2de3200640 1 -- 172.21.15.27:0/959436936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2de4075dd0 msgr2=0x7f2de41a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:58.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:58.114+0000 7f2de3200640 1 --2- 172.21.15.27:0/959436936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2de4075dd0 0x7f2de41a0890 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400e9f0 tx=0x7f2dd400edf0 comp rx=0 tx=0).stop 2024-09-21T00:18:58.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:58.114+0000 7f2de3200640 1 -- 172.21.15.27:0/959436936 shutdown_connections 2024-09-21T00:18:58.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:58.114+0000 7f2de3200640 1 --2- 172.21.15.27:0/959436936 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2dc80b65e0 0x7f2dc80b8af0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:58.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:58.114+0000 7f2de3200640 1 --2- 172.21.15.27:0/959436936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2de4114260 0x7f2de41a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:58.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:58.114+0000 7f2de3200640 1 --2- 172.21.15.27:0/959436936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2de4077380 0x7f2de41a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:58.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:58.114+0000 7f2de3200640 1 --2- 172.21.15.27:0/959436936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2de4075dd0 0x7f2de41a0890 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:58.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:58.114+0000 7f2de3200640 1 -- 172.21.15.27:0/959436936 >> 172.21.15.27:0/959436936 conn(0x7f2de40ff3e0 msgr2=0x7f2de4110160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:58.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:58.114+0000 7f2de3200640 1 -- 172.21.15.27:0/959436936 shutdown_connections 2024-09-21T00:18:58.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:58.114+0000 7f2de3200640 1 -- 172.21.15.27:0/959436936 wait complete. 2024-09-21T00:18:58.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:57 smithi089 ceph-mon[34481]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:58.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:57 smithi098 ceph-mon[34725]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:18:58.846 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T00:18:59.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:58 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/959436936' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:18:59.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:58 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/959436936' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:18:59.244 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:18:59.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:58 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/959436936' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:18:59.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.851+0000 7f18f4400640 1 -- 172.21.15.27:0/1233804079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18ec103b80 msgr2=0x7f18ec103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:59.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.851+0000 7f18f4400640 1 --2- 172.21.15.27:0/1233804079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18ec103b80 0x7f18ec103f90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009900 tx=0x7f18e002a410 comp rx=0 tx=0).stop 2024-09-21T00:18:59.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.851+0000 7f18f4400640 1 -- 172.21.15.27:0/1233804079 shutdown_connections 2024-09-21T00:18:59.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.851+0000 7f18f4400640 1 --2- 172.21.15.27:0/1233804079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18ec109e10 0x7f18ec110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:59.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.851+0000 7f18f4400640 1 --2- 172.21.15.27:0/1233804079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec1094c0 0x7f18ec1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:59.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.851+0000 7f18f4400640 1 --2- 172.21.15.27:0/1233804079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18ec103b80 0x7f18ec103f90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:59.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.851+0000 7f18f4400640 1 -- 172.21.15.27:0/1233804079 >> 172.21.15.27:0/1233804079 conn(0x7f18ec0ff5d0 msgr2=0x7f18ec101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:18:59.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.851+0000 7f18f4400640 1 -- 172.21.15.27:0/1233804079 shutdown_connections 2024-09-21T00:18:59.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.851+0000 7f18f4400640 1 -- 172.21.15.27:0/1233804079 wait complete. 2024-09-21T00:18:59.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.853+0000 7f18f4400640 1 Processor -- start 2024-09-21T00:18:59.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.853+0000 7f18f4400640 1 -- start start 2024-09-21T00:18:59.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.854+0000 7f18f4400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec103b80 0x7f18ec19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:59.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.854+0000 7f18f4400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18ec1094c0 0x7f18ec19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:59.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.854+0000 7f18f4400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18ec109e10 0x7f18ec19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:59.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.854+0000 7f18f4400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec113b20 con 0x7f18ec1094c0 2024-09-21T00:18:59.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.854+0000 7f18f4400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec1139a0 con 0x7f18ec109e10 2024-09-21T00:18:59.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.854+0000 7f18f4400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec113ca0 con 0x7f18ec103b80 2024-09-21T00:18:59.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.854+0000 7f18f1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18ec1094c0 0x7f18ec19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:59.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.854+0000 7f18f1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18ec1094c0 0x7f18ec19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46276/0 (socket says 172.21.15.27:46276) 2024-09-21T00:18:59.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.854+0000 7f18f1000640 1 -- 172.21.15.27:0/749695080 learned_addr learned my addr 172.21.15.27:0/749695080 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:18:59.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.854+0000 7f18f1a00640 1 --2- 172.21.15.27:0/749695080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec103b80 0x7f18ec19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:59.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.854+0000 7f18f2400640 1 --2- 172.21.15.27:0/749695080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18ec109e10 0x7f18ec19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:59.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.854+0000 7f18f1000640 1 -- 172.21.15.27:0/749695080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec103b80 msgr2=0x7f18ec19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:59.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.854+0000 7f18f1000640 1 --2- 172.21.15.27:0/749695080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec103b80 0x7f18ec19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:59.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.854+0000 7f18f1000640 1 -- 172.21.15.27:0/749695080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18ec109e10 msgr2=0x7f18ec19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:18:59.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.854+0000 7f18f1000640 1 --2- 172.21.15.27:0/749695080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18ec109e10 0x7f18ec19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:18:59.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.855+0000 7f18f1000640 1 -- 172.21.15.27:0/749695080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec1a0440 con 0x7f18ec1094c0 2024-09-21T00:18:59.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.855+0000 7f18f1a00640 1 --2- 172.21.15.27:0/749695080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec103b80 0x7f18ec19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:18:59.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.855+0000 7f18f1000640 1 --2- 172.21.15.27:0/749695080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18ec1094c0 0x7f18ec19aa70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f18d800e9c0 tx=0x7f18d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:59.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.855+0000 7f18ea000640 1 -- 172.21.15.27:0/749695080 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d800ccc0 con 0x7f18ec1094c0 2024-09-21T00:18:59.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.855+0000 7f18ea000640 1 -- 172.21.15.27:0/749695080 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18d80054d0 con 0x7f18ec1094c0 2024-09-21T00:18:59.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.855+0000 7f18f4400640 1 -- 172.21.15.27:0/749695080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ec1a0730 con 0x7f18ec1094c0 2024-09-21T00:18:59.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.855+0000 7f18ea000640 1 -- 172.21.15.27:0/749695080 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8010430 con 0x7f18ec1094c0 2024-09-21T00:18:59.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.855+0000 7f18f4400640 1 -- 172.21.15.27:0/749695080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ec1a6e90 con 0x7f18ec1094c0 2024-09-21T00:18:59.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.856+0000 7f18f4400640 1 -- 172.21.15.27:0/749695080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ec103f90 con 0x7f18ec1094c0 2024-09-21T00:18:59.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.859+0000 7f18ea000640 1 -- 172.21.15.27:0/749695080 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f18d8005000 con 0x7f18ec1094c0 2024-09-21T00:18:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.863+0000 7f18ea000640 1 --2- 172.21.15.27:0/749695080 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f18d40b65e0 0x7f18d40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:18:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.863+0000 7f18ea000640 1 -- 172.21.15.27:0/749695080 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f18d80e7b90 con 0x7f18ec1094c0 2024-09-21T00:18:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.863+0000 7f18f1a00640 1 --2- 172.21.15.27:0/749695080 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f18d40b65e0 0x7f18d40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:18:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.864+0000 7f18f1a00640 1 --2- 172.21.15.27:0/749695080 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f18d40b65e0 0x7f18d40b8af0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f18e0005ab0 tx=0x7f18e0005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:18:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:18:59.865+0000 7f18ea000640 1 -- 172.21.15.27:0/749695080 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f18d8056710 con 0x7f18ec1094c0 2024-09-21T00:19:00.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:18:59 smithi027 ceph-mon[30745]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:00.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:00.067+0000 7f18f4400640 1 -- 172.21.15.27:0/749695080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18ec19b600 con 0x7f18ec1094c0 2024-09-21T00:19:00.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:00.069+0000 7f18ea000640 1 -- 172.21.15.27:0/749695080 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24427 (secure 0 0 0) 0x7f18ec19b600 con 0x7f18ec1094c0 2024-09-21T00:19:00.071 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:19:00.071 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":86,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","created":"2024-09-21T00:09:20.149938+0000","modified":"2024-09-21T00:18:52.860270+0000","last_up_change":"2024-09-21T00:17:43.978785+0000","last_in_change":"2024-09-21T00:17:21.325177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:13:32.718543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:18:33.992414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:18:36.817691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:18:46.707770+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63ccc094-62d5-446f-844d-1c60d41fda1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6803","nonce":354370090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6805","nonce":354370090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6809","nonce":354370090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6807","nonce":354370090}]},"public_addr":"172.21.15.27:6803/354370090","cluster_addr":"172.21.15.27:6805/354370090","heartbeat_back_addr":"172.21.15.27:6809/354370090","heartbeat_front_addr":"172.21.15.27:6807/354370090","state":["exists","up"]},{"osd":1,"uuid":"e59a8ed4-3a37-4477-a60a-4abc33d49b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6801","nonce":922528621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6803","nonce":922528621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6807","nonce":922528621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6805","nonce":922528621}]},"public_addr":"172.21.15.89:6801/922528621","cluster_addr":"172.21.15.89:6803/922528621","heartbeat_back_addr":"172.21.15.89:6807/922528621","heartbeat_front_addr":"172.21.15.89:6805/922528621","state":["exists","up"]},{"osd":2,"uuid":"6058a875-d5b2-4459-9407-2fd190cb34a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6801","nonce":257636259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6803","nonce":257636259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6807","nonce":257636259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6805","nonce":257636259}]},"public_addr":"172.21.15.98:6801/257636259","cluster_addr":"172.21.15.98:6803/257636259","heartbeat_back_addr":"172.21.15.98:6807/257636259","heartbeat_front_addr":"172.21.15.98:6805/257636259","state":["exists","up"]},{"osd":3,"uuid":"dfcdaede-e51c-4985-92af-3e296435a36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6811","nonce":821088824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6813","nonce":821088824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6817","nonce":821088824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6815","nonce":821088824}]},"public_addr":"172.21.15.27:6811/821088824","cluster_addr":"172.21.15.27:6813/821088824","heartbeat_back_addr":"172.21.15.27:6817/821088824","heartbeat_front_addr":"172.21.15.27:6815/821088824","state":["exists","up"]},{"osd":4,"uuid":"7c4af037-8ab6-47c0-b62b-76bd775b3491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6809","nonce":4001038690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6811","nonce":4001038690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6815","nonce":4001038690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6813","nonce":4001038690}]},"public_addr":"172.21.15.89:6809/4001038690","cluster_addr":"172.21.15.89:6811/4001038690","heartbeat_back_addr":"172.21.15.89:6815/4001038690","heartbeat_front_addr":"172.21.15.89:6813/4001038690","state":["exists","up"]},{"osd":5,"uuid":"e996ed0d-88ad-4aa7-891f-dfade4e09cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6809","nonce":3029903562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6811","nonce":3029903562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6815","nonce":3029903562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6813","nonce":3029903562}]},"public_addr":"172.21.15.98:6809/3029903562","cluster_addr":"172.21.15.98:6811/3029903562","heartbeat_back_addr":"172.21.15.98:6815/3029903562","heartbeat_front_addr":"172.21.15.98:6813/3029903562","state":["exists","up"]},{"osd":6,"uuid":"cf080497-57c2-42d0-8c8f-1eed0110702d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6819","nonce":1286808857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6821","nonce":1286808857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6825","nonce":1286808857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6823","nonce":1286808857}]},"public_addr":"172.21.15.27:6819/1286808857","cluster_addr":"172.21.15.27:6821/1286808857","heartbeat_back_addr":"172.21.15.27:6825/1286808857","heartbeat_front_addr":"172.21.15.27:6823/1286808857","state":["exists","up"]},{"osd":7,"uuid":"ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6817","nonce":2479349546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6819","nonce":2479349546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6823","nonce":2479349546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6821","nonce":2479349546}]},"public_addr":"172.21.15.89:6817/2479349546","cluster_addr":"172.21.15.89:6819/2479349546","heartbeat_back_addr":"172.21.15.89:6823/2479349546","heartbeat_front_addr":"172.21.15.89:6821/2479349546","state":["exists","up"]},{"osd":8,"uuid":"0b558458-fc2d-4ed1-b72a-e2ea8cf811b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6817","nonce":2830381485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6819","nonce":2830381485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6823","nonce":2830381485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6821","nonce":2830381485}]},"public_addr":"172.21.15.98:6817/2830381485","cluster_addr":"172.21.15.98:6819/2830381485","heartbeat_back_addr":"172.21.15.98:6823/2830381485","heartbeat_front_addr":"172.21.15.98:6821/2830381485","state":["exists","up"]},{"osd":9,"uuid":"1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6827","nonce":3033977113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6829","nonce":3033977113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6833","nonce":3033977113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6831","nonce":3033977113}]},"public_addr":"172.21.15.27:6827/3033977113","cluster_addr":"172.21.15.27:6829/3033977113","heartbeat_back_addr":"172.21.15.27:6833/3033977113","heartbeat_front_addr":"172.21.15.27:6831/3033977113","state":["exists","up"]},{"osd":10,"uuid":"324dd6ce-68fe-4c76-959b-1c9fb0ae92a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6825","nonce":2370332225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6827","nonce":2370332225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6831","nonce":2370332225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6829","nonce":2370332225}]},"public_addr":"172.21.15.89:6825/2370332225","cluster_addr":"172.21.15.89:6827/2370332225","heartbeat_back_addr":"172.21.15.89:6831/2370332225","heartbeat_front_addr":"172.21.15.89:6829/2370332225","state":["exists","up"]},{"osd":11,"uuid":"5e34686e-350a-491e-b9d7-59174ea90944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6825","nonce":2795999064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6827","nonce":2795999064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6831","nonce":2795999064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6829","nonce":2795999064}]},"public_addr":"172.21.15.98:6825/2795999064","cluster_addr":"172.21.15.98:6827/2795999064","heartbeat_back_addr":"172.21.15.98:6831/2795999064","heartbeat_front_addr":"172.21.15.98:6829/2795999064","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:12:39.841958+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:06.684417+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:31.041378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:01.612108+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:28.592326+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:54.303714+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:25.523475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:52.523483+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:19.430503+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:50.599120+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:17.520452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:42.725550+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/502883064":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/831048090":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/3880378905":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/793991452":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/2077956479":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6800/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6801/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/1098432608":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6800/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/528019907":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/2173601508":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:6800/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/764776233":"2024-09-22T00:11:24.231101+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:19:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:00.071+0000 7f18f4400640 1 -- 172.21.15.27:0/749695080 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f18d40b65e0 msgr2=0x7f18d40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:00.071+0000 7f18f4400640 1 --2- 172.21.15.27:0/749695080 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f18d40b65e0 0x7f18d40b8af0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f18e0005ab0 tx=0x7f18e0005a20 comp rx=0 tx=0).stop 2024-09-21T00:19:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:00.071+0000 7f18f4400640 1 -- 172.21.15.27:0/749695080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18ec1094c0 msgr2=0x7f18ec19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:00.071+0000 7f18f4400640 1 --2- 172.21.15.27:0/749695080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18ec1094c0 0x7f18ec19aa70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f18d800e9c0 tx=0x7f18d800ee80 comp rx=0 tx=0).stop 2024-09-21T00:19:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:00.072+0000 7f18f4400640 1 -- 172.21.15.27:0/749695080 shutdown_connections 2024-09-21T00:19:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:00.072+0000 7f18f4400640 1 --2- 172.21.15.27:0/749695080 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f18d40b65e0 0x7f18d40b8af0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:00.072+0000 7f18f4400640 1 --2- 172.21.15.27:0/749695080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18ec109e10 0x7f18ec19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:00.072+0000 7f18f4400640 1 --2- 172.21.15.27:0/749695080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18ec1094c0 0x7f18ec19aa70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:00.072+0000 7f18f4400640 1 --2- 172.21.15.27:0/749695080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec103b80 0x7f18ec19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:00.072+0000 7f18f4400640 1 -- 172.21.15.27:0/749695080 >> 172.21.15.27:0/749695080 conn(0x7f18ec0ff5d0 msgr2=0x7f18ec1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:00.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:00.072+0000 7f18f4400640 1 -- 172.21.15.27:0/749695080 shutdown_connections 2024-09-21T00:19:00.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:00.072+0000 7f18f4400640 1 -- 172.21.15.27:0/749695080 wait complete. 2024-09-21T00:19:00.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:18:59 smithi098 ceph-mon[34725]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:00.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:18:59 smithi089 ceph-mon[34481]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:00.976 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:19:01.028 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:00 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/749695080' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:19:01.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:00 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/749695080' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:19:01.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:00 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/749695080' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:19:01.364 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.949+0000 7f7b78a00640 1 -- 172.21.15.27:0/1791612674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b74103990 msgr2=0x7f7b74103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.949+0000 7f7b78a00640 1 --2- 172.21.15.27:0/1791612674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b74103990 0x7f7b74103da0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009900 tx=0x7f7b6402a410 comp rx=0 tx=0).stop 2024-09-21T00:19:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.950+0000 7f7b78a00640 1 -- 172.21.15.27:0/1791612674 shutdown_connections 2024-09-21T00:19:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.950+0000 7f7b78a00640 1 --2- 172.21.15.27:0/1791612674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b74109c20 0x7f7b74110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:01.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.950+0000 7f7b78a00640 1 --2- 172.21.15.27:0/1791612674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b741092d0 0x7f7b741096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:01.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.950+0000 7f7b78a00640 1 --2- 172.21.15.27:0/1791612674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b74103990 0x7f7b74103da0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:01.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.950+0000 7f7b78a00640 1 -- 172.21.15.27:0/1791612674 >> 172.21.15.27:0/1791612674 conn(0x7f7b740ff3e0 msgr2=0x7f7b74101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:01.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.950+0000 7f7b78a00640 1 -- 172.21.15.27:0/1791612674 shutdown_connections 2024-09-21T00:19:01.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.950+0000 7f7b78a00640 1 -- 172.21.15.27:0/1791612674 wait complete. 2024-09-21T00:19:01.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.951+0000 7f7b78a00640 1 Processor -- start 2024-09-21T00:19:01.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.951+0000 7f7b78a00640 1 -- start start 2024-09-21T00:19:01.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.951+0000 7f7b78a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b74103990 0x7f7b74075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:01.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.951+0000 7f7b78a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b741092d0 0x7f7b74075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:01.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.951+0000 7f7b78a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b74109c20 0x7f7b74076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.951+0000 7f7b78a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74113930 con 0x7f7b741092d0 2024-09-21T00:19:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.951+0000 7f7b78a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741137b0 con 0x7f7b74103990 2024-09-21T00:19:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.951+0000 7f7b78a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74113ab0 con 0x7f7b74109c20 2024-09-21T00:19:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.952+0000 7f7b71e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b741092d0 0x7f7b74075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.952+0000 7f7b72800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b74103990 0x7f7b74075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.952+0000 7f7b71e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b741092d0 0x7f7b74075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46298/0 (socket says 172.21.15.27:46298) 2024-09-21T00:19:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.952+0000 7f7b71e00640 1 -- 172.21.15.27:0/494827078 learned_addr learned my addr 172.21.15.27:0/494827078 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.952+0000 7f7b73200640 1 --2- 172.21.15.27:0/494827078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b74109c20 0x7f7b74076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.952+0000 7f7b71e00640 1 -- 172.21.15.27:0/494827078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b74109c20 msgr2=0x7f7b74076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.952+0000 7f7b71e00640 1 --2- 172.21.15.27:0/494827078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b74109c20 0x7f7b74076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.952+0000 7f7b71e00640 1 -- 172.21.15.27:0/494827078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b74103990 msgr2=0x7f7b74075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.952+0000 7f7b71e00640 1 --2- 172.21.15.27:0/494827078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b74103990 0x7f7b74075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.952+0000 7f7b71e00640 1 -- 172.21.15.27:0/494827078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b741a8620 con 0x7f7b741092d0 2024-09-21T00:19:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.952+0000 7f7b72800640 1 --2- 172.21.15.27:0/494827078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b74103990 0x7f7b74075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:19:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.952+0000 7f7b71e00640 1 --2- 172.21.15.27:0/494827078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b741092d0 0x7f7b74075c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000e970 tx=0x7f7b6000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.953+0000 7f7b6b400640 1 -- 172.21.15.27:0/494827078 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000cd80 con 0x7f7b741092d0 2024-09-21T00:19:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.953+0000 7f7b78a00640 1 -- 172.21.15.27:0/494827078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b741a8910 con 0x7f7b741092d0 2024-09-21T00:19:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.953+0000 7f7b6b400640 1 -- 172.21.15.27:0/494827078 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b600035f0 con 0x7f7b741092d0 2024-09-21T00:19:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.953+0000 7f7b6b400640 1 -- 172.21.15.27:0/494827078 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60010690 con 0x7f7b741092d0 2024-09-21T00:19:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.953+0000 7f7b78a00640 1 -- 172.21.15.27:0/494827078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b741a8e70 con 0x7f7b741092d0 2024-09-21T00:19:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.954+0000 7f7b78a00640 1 -- 172.21.15.27:0/494827078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b74103da0 con 0x7f7b741092d0 2024-09-21T00:19:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.957+0000 7f7b6b400640 1 -- 172.21.15.27:0/494827078 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f7b60003aa0 con 0x7f7b741092d0 2024-09-21T00:19:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.959+0000 7f7b6b400640 1 --2- 172.21.15.27:0/494827078 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7b580b65e0 0x7f7b580b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.959+0000 7f7b6b400640 1 -- 172.21.15.27:0/494827078 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f7b600e7900 con 0x7f7b741092d0 2024-09-21T00:19:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.959+0000 7f7b72800640 1 --2- 172.21.15.27:0/494827078 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7b580b65e0 0x7f7b580b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.960+0000 7f7b72800640 1 --2- 172.21.15.27:0/494827078 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7b580b65e0 0x7f7b580b8af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b64005ab0 tx=0x7f7b64005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:01.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:01.963+0000 7f7b6b400640 1 -- 172.21.15.27:0/494827078 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7b600ade20 con 0x7f7b741092d0 2024-09-21T00:19:02.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:01 smithi027 ceph-mon[30745]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:02.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:02.205+0000 7f7b78a00640 1 -- 172.21.15.27:0/494827078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b7402e640 con 0x7f7b741092d0 2024-09-21T00:19:02.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:02.206+0000 7f7b6b400640 1 -- 172.21.15.27:0/494827078 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11821 (secure 0 0 0) 0x7f7b600a7e20 con 0x7f7b741092d0 2024-09-21T00:19:02.208 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:19:02.208 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"btime":"2024-09-21T00:18:53:726209+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24703,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24715,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24722,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24727,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:18:52.860449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24761,"qdb_cluster":[24761]},"id":1}]} 2024-09-21T00:19:02.208 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 16 2024-09-21T00:19:02.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:02.209+0000 7f7b78a00640 1 -- 172.21.15.27:0/494827078 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7b580b65e0 msgr2=0x7f7b580b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:02.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:02.209+0000 7f7b78a00640 1 --2- 172.21.15.27:0/494827078 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7b580b65e0 0x7f7b580b8af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b64005ab0 tx=0x7f7b64005a20 comp rx=0 tx=0).stop 2024-09-21T00:19:02.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:02.209+0000 7f7b78a00640 1 -- 172.21.15.27:0/494827078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b741092d0 msgr2=0x7f7b74075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:02.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:02.209+0000 7f7b78a00640 1 --2- 172.21.15.27:0/494827078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b741092d0 0x7f7b74075c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000e970 tx=0x7f7b6000ee30 comp rx=0 tx=0).stop 2024-09-21T00:19:02.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:02.209+0000 7f7b78a00640 1 -- 172.21.15.27:0/494827078 shutdown_connections 2024-09-21T00:19:02.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:02.209+0000 7f7b78a00640 1 --2- 172.21.15.27:0/494827078 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7b580b65e0 0x7f7b580b8af0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:02.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:02.209+0000 7f7b78a00640 1 --2- 172.21.15.27:0/494827078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b74109c20 0x7f7b74076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:02.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:02.209+0000 7f7b78a00640 1 --2- 172.21.15.27:0/494827078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b741092d0 0x7f7b74075c50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:02.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:02.209+0000 7f7b78a00640 1 --2- 172.21.15.27:0/494827078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b74103990 0x7f7b74075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:02.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:02.209+0000 7f7b78a00640 1 -- 172.21.15.27:0/494827078 >> 172.21.15.27:0/494827078 conn(0x7f7b740ff3e0 msgr2=0x7f7b74101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:02.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:02.209+0000 7f7b78a00640 1 -- 172.21.15.27:0/494827078 shutdown_connections 2024-09-21T00:19:02.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:02.209+0000 7f7b78a00640 1 -- 172.21.15.27:0/494827078 wait complete. 2024-09-21T00:19:02.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:01 smithi089 ceph-mon[34481]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:02.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:01 smithi098 ceph-mon[34725]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:02.998 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:02 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/494827078' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:19:03.037 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T00:19:03.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:02 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/494827078' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:19:03.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:02 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/494827078' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:19:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:03.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:03 smithi027 ceph-mon[30745]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:04.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.082+0000 7f6584e00640 1 -- 172.21.15.27:0/2305828948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6580110140 msgr2=0x7f6580112560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:04.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.082+0000 7f6584e00640 1 --2- 172.21.15.27:0/2305828948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6580110140 0x7f6580112560 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6578009ca0 tx=0x7f657802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:19:04.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.082+0000 7f6584e00640 1 -- 172.21.15.27:0/2305828948 shutdown_connections 2024-09-21T00:19:04.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.082+0000 7f6584e00640 1 --2- 172.21.15.27:0/2305828948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6580110140 0x7f6580112560 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:04.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.082+0000 7f6584e00640 1 --2- 172.21.15.27:0/2305828948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6580076080 0x7f658010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:04.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.082+0000 7f6584e00640 1 --2- 172.21.15.27:0/2305828948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6580075730 0x7f6580075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:04.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.082+0000 7f6584e00640 1 -- 172.21.15.27:0/2305828948 >> 172.21.15.27:0/2305828948 conn(0x7f658007b850 msgr2=0x7f658007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:04.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.082+0000 7f6584e00640 1 -- 172.21.15.27:0/2305828948 shutdown_connections 2024-09-21T00:19:04.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.082+0000 7f6584e00640 1 -- 172.21.15.27:0/2305828948 wait complete. 2024-09-21T00:19:04.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.083+0000 7f6584e00640 1 Processor -- start 2024-09-21T00:19:04.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.084+0000 7f6584e00640 1 -- start start 2024-09-21T00:19:04.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.084+0000 7f6584e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6580075730 0x7f658019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:04.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.084+0000 7f6584e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6580076080 0x7f658019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:04.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.084+0000 7f6584e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6580110140 0x7f65801a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:04.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.084+0000 7f6584e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6580115ca0 con 0x7f6580075730 2024-09-21T00:19:04.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.084+0000 7f6584e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6580115b20 con 0x7f6580110140 2024-09-21T00:19:04.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.084+0000 7f6584e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6580115e20 con 0x7f6580076080 2024-09-21T00:19:04.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.084+0000 7f657de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6580075730 0x7f658019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:04.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.085+0000 7f657de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6580075730 0x7f658019c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39152/0 (socket says 172.21.15.27:39152) 2024-09-21T00:19:04.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.085+0000 7f657de00640 1 -- 172.21.15.27:0/1085411083 learned_addr learned my addr 172.21.15.27:0/1085411083 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:04.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.085+0000 7f657e800640 1 --2- 172.21.15.27:0/1085411083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6580110140 0x7f65801a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:04.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.085+0000 7f6577e00640 1 --2- 172.21.15.27:0/1085411083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6580076080 0x7f658019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:04.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.085+0000 7f657de00640 1 -- 172.21.15.27:0/1085411083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6580076080 msgr2=0x7f658019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:04.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.085+0000 7f657de00640 1 --2- 172.21.15.27:0/1085411083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6580076080 0x7f658019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:04.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.085+0000 7f657de00640 1 -- 172.21.15.27:0/1085411083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6580110140 msgr2=0x7f65801a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:04.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.085+0000 7f657de00640 1 --2- 172.21.15.27:0/1085411083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6580110140 0x7f65801a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:04.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.085+0000 7f657de00640 1 -- 172.21.15.27:0/1085411083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65801a25c0 con 0x7f6580075730 2024-09-21T00:19:04.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.085+0000 7f657de00640 1 --2- 172.21.15.27:0/1085411083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6580075730 0x7f658019c650 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f657000e9f0 tx=0x7f657000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:04.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.086+0000 7f6576a00640 1 -- 172.21.15.27:0/1085411083 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657000cd80 con 0x7f6580075730 2024-09-21T00:19:04.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.086+0000 7f6576a00640 1 -- 172.21.15.27:0/1085411083 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65700108e0 con 0x7f6580075730 2024-09-21T00:19:04.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.086+0000 7f6584e00640 1 -- 172.21.15.27:0/1085411083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65801a28b0 con 0x7f6580075730 2024-09-21T00:19:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.086+0000 7f6576a00640 1 -- 172.21.15.27:0/1085411083 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570018980 con 0x7f6580075730 2024-09-21T00:19:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.086+0000 7f6584e00640 1 -- 172.21.15.27:0/1085411083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65801a9010 con 0x7f6580075730 2024-09-21T00:19:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.087+0000 7f6584e00640 1 -- 172.21.15.27:0/1085411083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6580075b40 con 0x7f6580075730 2024-09-21T00:19:04.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.089+0000 7f6576a00640 1 -- 172.21.15.27:0/1085411083 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f6570010430 con 0x7f6580075730 2024-09-21T00:19:04.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.091+0000 7f6576a00640 1 --2- 172.21.15.27:0/1085411083 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f65640b6560 0x7f65640b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:04.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.091+0000 7f6576a00640 1 -- 172.21.15.27:0/1085411083 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f65700e7e90 con 0x7f6580075730 2024-09-21T00:19:04.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.091+0000 7f6577e00640 1 --2- 172.21.15.27:0/1085411083 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f65640b6560 0x7f65640b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:04.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.092+0000 7f6577e00640 1 --2- 172.21.15.27:0/1085411083 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f65640b6560 0x7f65640b8a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6580111e40 tx=0x7f6568005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:04.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.093+0000 7f6576a00640 1 -- 172.21.15.27:0/1085411083 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f65700b2790 con 0x7f6580075730 2024-09-21T00:19:04.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:03 smithi089 ceph-mon[34481]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:04.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:03 smithi098 ceph-mon[34725]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:04.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.295+0000 7f6584e00640 1 -- 172.21.15.27:0/1085411083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65801a92f0 con 0x7f6580075730 2024-09-21T00:19:04.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.296+0000 7f6576a00640 1 -- 172.21.15.27:0/1085411083 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24427 (secure 0 0 0) 0x7f65700b76d0 con 0x7f6580075730 2024-09-21T00:19:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:19:04.298 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":86,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","created":"2024-09-21T00:09:20.149938+0000","modified":"2024-09-21T00:18:52.860270+0000","last_up_change":"2024-09-21T00:17:43.978785+0000","last_in_change":"2024-09-21T00:17:21.325177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:13:32.718543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:18:33.992414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:18:36.817691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:18:46.707770+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63ccc094-62d5-446f-844d-1c60d41fda1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6803","nonce":354370090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6805","nonce":354370090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6809","nonce":354370090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6807","nonce":354370090}]},"public_addr":"172.21.15.27:6803/354370090","cluster_addr":"172.21.15.27:6805/354370090","heartbeat_back_addr":"172.21.15.27:6809/354370090","heartbeat_front_addr":"172.21.15.27:6807/354370090","state":["exists","up"]},{"osd":1,"uuid":"e59a8ed4-3a37-4477-a60a-4abc33d49b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6801","nonce":922528621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6803","nonce":922528621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6807","nonce":922528621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6805","nonce":922528621}]},"public_addr":"172.21.15.89:6801/922528621","cluster_addr":"172.21.15.89:6803/922528621","heartbeat_back_addr":"172.21.15.89:6807/922528621","heartbeat_front_addr":"172.21.15.89:6805/922528621","state":["exists","up"]},{"osd":2,"uuid":"6058a875-d5b2-4459-9407-2fd190cb34a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6801","nonce":257636259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6803","nonce":257636259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6807","nonce":257636259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6805","nonce":257636259}]},"public_addr":"172.21.15.98:6801/257636259","cluster_addr":"172.21.15.98:6803/257636259","heartbeat_back_addr":"172.21.15.98:6807/257636259","heartbeat_front_addr":"172.21.15.98:6805/257636259","state":["exists","up"]},{"osd":3,"uuid":"dfcdaede-e51c-4985-92af-3e296435a36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6811","nonce":821088824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6813","nonce":821088824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6817","nonce":821088824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6815","nonce":821088824}]},"public_addr":"172.21.15.27:6811/821088824","cluster_addr":"172.21.15.27:6813/821088824","heartbeat_back_addr":"172.21.15.27:6817/821088824","heartbeat_front_addr":"172.21.15.27:6815/821088824","state":["exists","up"]},{"osd":4,"uuid":"7c4af037-8ab6-47c0-b62b-76bd775b3491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6809","nonce":4001038690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6811","nonce":4001038690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6815","nonce":4001038690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6813","nonce":4001038690}]},"public_addr":"172.21.15.89:6809/4001038690","cluster_addr":"172.21.15.89:6811/4001038690","heartbeat_back_addr":"172.21.15.89:6815/4001038690","heartbeat_front_addr":"172.21.15.89:6813/4001038690","state":["exists","up"]},{"osd":5,"uuid":"e996ed0d-88ad-4aa7-891f-dfade4e09cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6809","nonce":3029903562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6811","nonce":3029903562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6815","nonce":3029903562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6813","nonce":3029903562}]},"public_addr":"172.21.15.98:6809/3029903562","cluster_addr":"172.21.15.98:6811/3029903562","heartbeat_back_addr":"172.21.15.98:6815/3029903562","heartbeat_front_addr":"172.21.15.98:6813/3029903562","state":["exists","up"]},{"osd":6,"uuid":"cf080497-57c2-42d0-8c8f-1eed0110702d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6819","nonce":1286808857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6821","nonce":1286808857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6825","nonce":1286808857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6823","nonce":1286808857}]},"public_addr":"172.21.15.27:6819/1286808857","cluster_addr":"172.21.15.27:6821/1286808857","heartbeat_back_addr":"172.21.15.27:6825/1286808857","heartbeat_front_addr":"172.21.15.27:6823/1286808857","state":["exists","up"]},{"osd":7,"uuid":"ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6817","nonce":2479349546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6819","nonce":2479349546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6823","nonce":2479349546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6821","nonce":2479349546}]},"public_addr":"172.21.15.89:6817/2479349546","cluster_addr":"172.21.15.89:6819/2479349546","heartbeat_back_addr":"172.21.15.89:6823/2479349546","heartbeat_front_addr":"172.21.15.89:6821/2479349546","state":["exists","up"]},{"osd":8,"uuid":"0b558458-fc2d-4ed1-b72a-e2ea8cf811b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6817","nonce":2830381485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6819","nonce":2830381485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6823","nonce":2830381485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6821","nonce":2830381485}]},"public_addr":"172.21.15.98:6817/2830381485","cluster_addr":"172.21.15.98:6819/2830381485","heartbeat_back_addr":"172.21.15.98:6823/2830381485","heartbeat_front_addr":"172.21.15.98:6821/2830381485","state":["exists","up"]},{"osd":9,"uuid":"1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6827","nonce":3033977113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6829","nonce":3033977113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6833","nonce":3033977113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6831","nonce":3033977113}]},"public_addr":"172.21.15.27:6827/3033977113","cluster_addr":"172.21.15.27:6829/3033977113","heartbeat_back_addr":"172.21.15.27:6833/3033977113","heartbeat_front_addr":"172.21.15.27:6831/3033977113","state":["exists","up"]},{"osd":10,"uuid":"324dd6ce-68fe-4c76-959b-1c9fb0ae92a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6825","nonce":2370332225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6827","nonce":2370332225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6831","nonce":2370332225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6829","nonce":2370332225}]},"public_addr":"172.21.15.89:6825/2370332225","cluster_addr":"172.21.15.89:6827/2370332225","heartbeat_back_addr":"172.21.15.89:6831/2370332225","heartbeat_front_addr":"172.21.15.89:6829/2370332225","state":["exists","up"]},{"osd":11,"uuid":"5e34686e-350a-491e-b9d7-59174ea90944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6825","nonce":2795999064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6827","nonce":2795999064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6831","nonce":2795999064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6829","nonce":2795999064}]},"public_addr":"172.21.15.98:6825/2795999064","cluster_addr":"172.21.15.98:6827/2795999064","heartbeat_back_addr":"172.21.15.98:6831/2795999064","heartbeat_front_addr":"172.21.15.98:6829/2795999064","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:12:39.841958+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:06.684417+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:31.041378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:01.612108+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:28.592326+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:54.303714+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:25.523475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:52.523483+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:19.430503+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:50.599120+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:17.520452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:42.725550+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/502883064":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/831048090":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/3880378905":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/793991452":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/2077956479":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6800/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6801/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/1098432608":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6800/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/528019907":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/2173601508":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:6800/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/764776233":"2024-09-22T00:11:24.231101+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:19:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.299+0000 7f6584e00640 1 -- 172.21.15.27:0/1085411083 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f65640b6560 msgr2=0x7f65640b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.299+0000 7f6584e00640 1 --2- 172.21.15.27:0/1085411083 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f65640b6560 0x7f65640b8a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6580111e40 tx=0x7f6568005f90 comp rx=0 tx=0).stop 2024-09-21T00:19:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.299+0000 7f6584e00640 1 -- 172.21.15.27:0/1085411083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6580075730 msgr2=0x7f658019c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.299+0000 7f6584e00640 1 --2- 172.21.15.27:0/1085411083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6580075730 0x7f658019c650 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f657000e9f0 tx=0x7f657000eeb0 comp rx=0 tx=0).stop 2024-09-21T00:19:04.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.299+0000 7f6584e00640 1 -- 172.21.15.27:0/1085411083 shutdown_connections 2024-09-21T00:19:04.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.299+0000 7f6584e00640 1 --2- 172.21.15.27:0/1085411083 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f65640b6560 0x7f65640b8a70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:04.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.299+0000 7f6584e00640 1 --2- 172.21.15.27:0/1085411083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6580110140 0x7f65801a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:04.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.299+0000 7f6584e00640 1 --2- 172.21.15.27:0/1085411083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6580076080 0x7f658019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:04.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.299+0000 7f6584e00640 1 --2- 172.21.15.27:0/1085411083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6580075730 0x7f658019c650 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:04.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.299+0000 7f6584e00640 1 -- 172.21.15.27:0/1085411083 >> 172.21.15.27:0/1085411083 conn(0x7f658007b850 msgr2=0x7f6580109c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:04.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.299+0000 7f6584e00640 1 -- 172.21.15.27:0/1085411083 shutdown_connections 2024-09-21T00:19:04.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:04.300+0000 7f6584e00640 1 -- 172.21.15.27:0/1085411083 wait complete. 2024-09-21T00:19:05.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:04 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1085411083' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:19:05.164 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:18:39.809850+0000', 'modified': '2024-09-21T00:18:52.860449+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24761}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24761': {'gid': 24761, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.89:6839/2432693964', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 2432693964}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 2432693964}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24761, 'qdb_cluster': [24761]} 2024-09-21T00:19:05.165 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T00:19:05.165 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T00:19:05.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:04 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1085411083' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:19:05.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:04 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1085411083' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:19:05.548 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:05.978 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:05 smithi027 ceph-mon[30745]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:06.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.161+0000 7f5e81e00640 1 -- 172.21.15.27:0/1060270786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e7c105d80 msgr2=0x7f5e7c106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:06.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.161+0000 7f5e81e00640 1 --2- 172.21.15.27:0/1060270786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e7c105d80 0x7f5e7c106190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009900 tx=0x7f5e6c02a410 comp rx=0 tx=0).stop 2024-09-21T00:19:06.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.161+0000 7f5e81e00640 1 -- 172.21.15.27:0/1060270786 shutdown_connections 2024-09-21T00:19:06.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.161+0000 7f5e81e00640 1 --2- 172.21.15.27:0/1060270786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e7c10c010 0x7f5e7c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.161+0000 7f5e81e00640 1 --2- 172.21.15.27:0/1060270786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e7c10b6c0 0x7f5e7c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.161+0000 7f5e81e00640 1 --2- 172.21.15.27:0/1060270786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e7c105d80 0x7f5e7c106190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.161+0000 7f5e81e00640 1 -- 172.21.15.27:0/1060270786 >> 172.21.15.27:0/1060270786 conn(0x7f5e7c101810 msgr2=0x7f5e7c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:06.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.161+0000 7f5e81e00640 1 -- 172.21.15.27:0/1060270786 shutdown_connections 2024-09-21T00:19:06.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.161+0000 7f5e81e00640 1 -- 172.21.15.27:0/1060270786 wait complete. 2024-09-21T00:19:06.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.162+0000 7f5e81e00640 1 Processor -- start 2024-09-21T00:19:06.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.162+0000 7f5e81e00640 1 -- start start 2024-09-21T00:19:06.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.163+0000 7f5e81e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e7c105d80 0x7f5e7c19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.163+0000 7f5e81e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e7c10b6c0 0x7f5e7c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.163+0000 7f5e81e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e7c10c010 0x7f5e7c1a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.163+0000 7f5e81e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c115cd0 con 0x7f5e7c10b6c0 2024-09-21T00:19:06.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.163+0000 7f5e81e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c115b50 con 0x7f5e7c10c010 2024-09-21T00:19:06.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.163+0000 7f5e81e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c115e50 con 0x7f5e7c105d80 2024-09-21T00:19:06.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.163+0000 7f5e72000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e7c10b6c0 0x7f5e7c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.163+0000 7f5e7aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e7c105d80 0x7f5e7c19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.163+0000 7f5e72000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e7c10b6c0 0x7f5e7c19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39166/0 (socket says 172.21.15.27:39166) 2024-09-21T00:19:06.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.163+0000 7f5e72000640 1 -- 172.21.15.27:0/2343945156 learned_addr learned my addr 172.21.15.27:0/2343945156 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:06.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.163+0000 7f5e7b400640 1 --2- 172.21.15.27:0/2343945156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e7c10c010 0x7f5e7c1a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.164+0000 7f5e72000640 1 -- 172.21.15.27:0/2343945156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e7c105d80 msgr2=0x7f5e7c19c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:06.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.164+0000 7f5e72000640 1 --2- 172.21.15.27:0/2343945156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e7c105d80 0x7f5e7c19c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.164+0000 7f5e72000640 1 -- 172.21.15.27:0/2343945156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e7c10c010 msgr2=0x7f5e7c1a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:06.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.164+0000 7f5e72000640 1 --2- 172.21.15.27:0/2343945156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e7c10c010 0x7f5e7c1a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.164+0000 7f5e72000640 1 -- 172.21.15.27:0/2343945156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7c1a2650 con 0x7f5e7c10b6c0 2024-09-21T00:19:06.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.164+0000 7f5e72000640 1 --2- 172.21.15.27:0/2343945156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e7c10b6c0 0x7f5e7c19cc80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400e9c0 tx=0x7f5e6400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:06.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.164+0000 7f5e80a00640 1 -- 172.21.15.27:0/2343945156 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6400ccc0 con 0x7f5e7c10b6c0 2024-09-21T00:19:06.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.165+0000 7f5e80a00640 1 -- 172.21.15.27:0/2343945156 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e640054d0 con 0x7f5e7c10b6c0 2024-09-21T00:19:06.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.165+0000 7f5e81e00640 1 -- 172.21.15.27:0/2343945156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7c1a2940 con 0x7f5e7c10b6c0 2024-09-21T00:19:06.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.165+0000 7f5e80a00640 1 -- 172.21.15.27:0/2343945156 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64010430 con 0x7f5e7c10b6c0 2024-09-21T00:19:06.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.165+0000 7f5e81e00640 1 -- 172.21.15.27:0/2343945156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e7c1a90a0 con 0x7f5e7c10b6c0 2024-09-21T00:19:06.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.168+0000 7f5e88be1740 1 Processor -- start 2024-09-21T00:19:06.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.168+0000 7f5e88be1740 1 -- start start 2024-09-21T00:19:06.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.169+0000 7f5e80a00640 1 -- 172.21.15.27:0/2343945156 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f5e64005020 con 0x7f5e7c10b6c0 2024-09-21T00:19:06.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.169+0000 7f5e88be1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62736f60 0x55cb62add490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.169+0000 7f5e88be1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb6270e0a0 0x55cb62add9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.169+0000 7f5e88be1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62716280 0x55cb62addf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.169+0000 7f5e88be1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cb62a54d50 con 0x55cb62736f60 2024-09-21T00:19:06.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.169+0000 7f5e88be1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55cb62a44180 con 0x55cb62716280 2024-09-21T00:19:06.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.169+0000 7f5e88be1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cb62a47f90 con 0x55cb6270e0a0 2024-09-21T00:19:06.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.169+0000 7f5e7b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62736f60 0x55cb62add490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.169+0000 7f5e7b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62736f60 0x55cb62add490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39168/0 (socket says 172.21.15.27:39168) 2024-09-21T00:19:06.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.169+0000 7f5e7b400640 1 -- 172.21.15.27:0/2925582730 learned_addr learned my addr 172.21.15.27:0/2925582730 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:06.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.169+0000 7f5e72000640 1 --2- 172.21.15.27:0/2925582730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62716280 0x55cb62addf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.169+0000 7f5e7aa00640 1 --2- 172.21.15.27:0/2925582730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb6270e0a0 0x55cb62add9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.170+0000 7f5e7b400640 1 -- 172.21.15.27:0/2925582730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb6270e0a0 msgr2=0x55cb62add9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:19:06.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.170+0000 7f5e7b400640 1 --2- 172.21.15.27:0/2925582730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb6270e0a0 0x55cb62add9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.170+0000 7f5e7b400640 1 -- 172.21.15.27:0/2925582730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62716280 msgr2=0x55cb62addf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:19:06.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.170+0000 7f5e7b400640 1 --2- 172.21.15.27:0/2925582730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62716280 0x55cb62addf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.170+0000 7f5e7b400640 1 -- 172.21.15.27:0/2925582730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb62a0ea70 con 0x55cb62736f60 2024-09-21T00:19:06.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.170+0000 7f5e72000640 1 --2- 172.21.15.27:0/2925582730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62716280 0x55cb62addf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:19:06.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.170+0000 7f5e7b400640 1 --2- 172.21.15.27:0/2925582730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62736f60 0x55cb62add490 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009fd0 tx=0x7f5e740118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ddc2575a905a87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:06.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.171+0000 7f5e6be00640 1 -- 172.21.15.27:0/2925582730 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74004d20 con 0x55cb62736f60 2024-09-21T00:19:06.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.171+0000 7f5e6be00640 1 -- 172.21.15.27:0/2925582730 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e74004050 con 0x55cb62736f60 2024-09-21T00:19:06.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.171+0000 7f5e6be00640 1 -- 172.21.15.27:0/2925582730 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74005280 con 0x55cb62736f60 2024-09-21T00:19:06.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.171+0000 7f5e80a00640 1 --2- 172.21.15.27:0/2343945156 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5e600b65e0 0x7f5e600b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.171+0000 7f5e80a00640 1 -- 172.21.15.27:0/2343945156 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f5e640e79c0 con 0x7f5e7c10b6c0 2024-09-21T00:19:06.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.171+0000 7f5e7aa00640 1 --2- 172.21.15.27:0/2343945156 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5e600b65e0 0x7f5e600b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.172+0000 7f5e88be1740 1 -- 172.21.15.27:0/2925582730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62736f60 msgr2=0x55cb62add490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:06.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.172+0000 7f5e88be1740 1 --2- 172.21.15.27:0/2925582730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62736f60 0x55cb62add490 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009fd0 tx=0x7f5e740118d0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.172+0000 7f5e88be1740 1 -- 172.21.15.27:0/2925582730 shutdown_connections 2024-09-21T00:19:06.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.172+0000 7f5e88be1740 1 --2- 172.21.15.27:0/2925582730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62716280 0x55cb62addf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.172+0000 7f5e88be1740 1 --2- 172.21.15.27:0/2925582730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb6270e0a0 0x55cb62add9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.172+0000 7f5e88be1740 1 --2- 172.21.15.27:0/2925582730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62736f60 0x55cb62add490 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.172+0000 7f5e88be1740 1 -- 172.21.15.27:0/2925582730 >> 172.21.15.27:0/2925582730 conn(0x55cb6272b350 msgr2=0x55cb62adb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:06.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.172+0000 7f5e88be1740 1 -- 172.21.15.27:0/2925582730 shutdown_connections 2024-09-21T00:19:06.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.172+0000 7f5e7aa00640 1 --2- 172.21.15.27:0/2343945156 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5e600b65e0 0x7f5e600b8af0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c031440 tx=0x7f5e6c009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:06.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.172+0000 7f5e88be1740 1 -- 172.21.15.27:0/2925582730 wait complete. 2024-09-21T00:19:06.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.173+0000 7f5e88be1740 1 -- start start 2024-09-21T00:19:06.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.173+0000 7f5e88be1740 1 Processor -- start 2024-09-21T00:19:06.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.173+0000 7f5e88be1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb6270e0a0 0x55cb62ae9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.173+0000 7f5e88be1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62716280 0x55cb62ae9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.173+0000 7f5e88be1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62736f60 0x55cb62af4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.173+0000 7f5e88be1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cb628a55f0 con 0x55cb62716280 2024-09-21T00:19:06.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.173+0000 7f5e88be1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55cb629df460 con 0x55cb62736f60 2024-09-21T00:19:06.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.173+0000 7f5e88be1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cb62aebc70 con 0x55cb6270e0a0 2024-09-21T00:19:06.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.174+0000 7f5e88be1740 10 client.0 _ll_register_callbacks cb 0x55cb629ef210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:19:06.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.174+0000 7f5e7aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb6270e0a0 0x55cb62ae9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.174+0000 7f5e72000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62716280 0x55cb62ae9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.174+0000 7f5e7aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb6270e0a0 0x55cb62ae9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39152/0 (socket says 172.21.15.27:39152) 2024-09-21T00:19:06.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.174+0000 7f5e7aa00640 1 -- 172.21.15.27:0/611067454 learned_addr learned my addr 172.21.15.27:0/611067454 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:06.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.174+0000 7f5e7b400640 1 --2- 172.21.15.27:0/611067454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62736f60 0x55cb62af4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.174+0000 7f5e7aa00640 1 -- 172.21.15.27:0/611067454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62736f60 msgr2=0x55cb62af4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:19:06.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.174+0000 7f5e7aa00640 1 --2- 172.21.15.27:0/611067454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62736f60 0x55cb62af4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.174+0000 7f5e7aa00640 1 -- 172.21.15.27:0/611067454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62716280 msgr2=0x55cb62ae9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:19:06.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.174+0000 7f5e7aa00640 1 --2- 172.21.15.27:0/611067454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62716280 0x55cb62ae9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.174+0000 7f5e7aa00640 1 -- 172.21.15.27:0/611067454 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb627a2540 con 0x55cb6270e0a0 2024-09-21T00:19:06.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.175+0000 7f5e7aa00640 1 --2- 172.21.15.27:0/611067454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb6270e0a0 0x55cb62ae9030 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c033000 tx=0x7f5e6c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b7749789b159ccf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:06.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.175+0000 7f5e68c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:19:06.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.175+0000 7f5e68c00640 1 -- 172.21.15.27:0/611067454 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c04d070 con 0x55cb6270e0a0 2024-09-21T00:19:06.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.175+0000 7f5e68c00640 1 -- 172.21.15.27:0/611067454 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f5e6c005030 con 0x55cb6270e0a0 2024-09-21T00:19:06.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.175+0000 7f5e88be1740 1 -- 172.21.15.27:0/611067454 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cb629e1a70 con 0x55cb6270e0a0 2024-09-21T00:19:06.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.175+0000 7f5e68c00640 1 -- 172.21.15.27:0/611067454 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f5e6c059050 con 0x55cb6270e0a0 2024-09-21T00:19:06.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.175+0000 7f5e88be1740 10 client.24847 fetch_fsmap learned FSMap version 16 2024-09-21T00:19:06.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.175+0000 7f5e88be1740 1 -- 172.21.15.27:0/611067454 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x55cb629fc6f0 con 0x55cb6270e0a0 2024-09-21T00:19:06.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.176+0000 7f5e68c00640 1 -- 172.21.15.27:0/611067454 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cb629e1a70 con 0x55cb6270e0a0 2024-09-21T00:19:06.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.176+0000 7f5e68c00640 1 -- 172.21.15.27:0/611067454 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f5e6c004ce0 con 0x55cb6270e0a0 2024-09-21T00:19:06.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.176+0000 7f5e88be1740 10 client.24847 fetch_fsmap finished waiting for FSMap version 16 2024-09-21T00:19:06.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.176+0000 7f5e88be1740 10 client.24847 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T00:19:06.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.176+0000 7f5e88be1740 20 client.24847 populate_metadata read hostname 'smithi027' 2024-09-21T00:19:06.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.176+0000 7f5e88be1740 1 --2- 172.21.15.27:0/611067454 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55cb626f58e0 0x55cb62afb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.176+0000 7f5e88be1740 4 client.24847 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:19:06.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.176+0000 7f5e88be1740 1 -- 172.21.15.27:0/611067454 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb62a56ca0 con 0x55cb626f58e0 2024-09-21T00:19:06.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.176+0000 7f5e72000640 1 --2- 172.21.15.27:0/611067454 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55cb626f58e0 0x55cb62afb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.177+0000 7f5e72000640 1 --2- 172.21.15.27:0/611067454 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55cb626f58e0 0x55cb62afb500 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9692f10db7d63e0b server_cookie=c61a73b14cdc5fdb in_seq=0 out_seq=0 2024-09-21T00:19:06.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.177+0000 7f5e68c00640 10 client.24847 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:19:06.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.178+0000 7f5e68c00640 1 -- 172.21.15.27:0/611067454 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cb62a56ca0 con 0x55cb626f58e0 2024-09-21T00:19:06.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.178+0000 7f5e68c00640 10 client.24847 handle_command_reply: tid=0 2024-09-21T00:19:06.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.178+0000 7f5e88be1740 1 client.24847 shutdown 2024-09-21T00:19:06.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.178+0000 7f5e88be1740 1 -- 172.21.15.27:0/611067454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb6270e0a0 msgr2=0x55cb62ae9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:06.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.178+0000 7f5e88be1740 1 --2- 172.21.15.27:0/611067454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb6270e0a0 0x55cb62ae9030 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c033000 tx=0x7f5e6c009260 comp rx=0 tx=0).stop 2024-09-21T00:19:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.178+0000 7f5e88be1740 1 -- 172.21.15.27:0/611067454 shutdown_connections 2024-09-21T00:19:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.178+0000 7f5e88be1740 1 --2- 172.21.15.27:0/611067454 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55cb626f58e0 0x55cb62afb500 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.178+0000 7f5e88be1740 1 --2- 172.21.15.27:0/611067454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62736f60 0x55cb62af4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.178+0000 7f5e88be1740 1 --2- 172.21.15.27:0/611067454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62716280 0x55cb62ae9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.178+0000 7f5e88be1740 1 --2- 172.21.15.27:0/611067454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb6270e0a0 0x55cb62ae9030 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.178+0000 7f5e68c00640 0 client.24847 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:19:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.178+0000 7f5e88be1740 1 -- 172.21.15.27:0/611067454 >> 172.21.15.27:0/611067454 conn(0x55cb6272b350 msgr2=0x55cb62ae0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.178+0000 7f5e88be1740 1 -- 172.21.15.27:0/611067454 shutdown_connections 2024-09-21T00:19:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.178+0000 7f5e88be1740 1 -- 172.21.15.27:0/611067454 wait complete. 2024-09-21T00:19:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.179+0000 7f5e88be1740 20 client.24847 trim_cache size 0 max 16384 2024-09-21T00:19:06.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.198+0000 7f5e88be1740 1 Processor -- start 2024-09-21T00:19:06.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.198+0000 7f5e88be1740 1 -- start start 2024-09-21T00:19:06.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.198+0000 7f5e88be1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb6270e0a0 0x55cb62af5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.198+0000 7f5e88be1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb62716280 0x55cb62ae3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.198+0000 7f5e88be1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62736f60 0x55cb62ae44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.198+0000 7f5e88be1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cb62af9840 con 0x55cb6270e0a0 2024-09-21T00:19:06.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.198+0000 7f5e88be1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55cb62af9f10 con 0x55cb62736f60 2024-09-21T00:19:06.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.198+0000 7f5e88be1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cb62afa5e0 con 0x55cb62716280 2024-09-21T00:19:06.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.198+0000 7f5e7aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb6270e0a0 0x55cb62af5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.198+0000 7f5e7aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb6270e0a0 0x55cb62af5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39184/0 (socket says 172.21.15.27:39184) 2024-09-21T00:19:06.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.198+0000 7f5e7aa00640 1 -- 172.21.15.27:0/3036743127 learned_addr learned my addr 172.21.15.27:0/3036743127 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:06.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.198+0000 7f5e72000640 1 --2- 172.21.15.27:0/3036743127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb62716280 0x55cb62ae3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.199+0000 7f5e7b400640 1 --2- 172.21.15.27:0/3036743127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62736f60 0x55cb62ae44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.199+0000 7f5e7aa00640 1 -- 172.21.15.27:0/3036743127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb62716280 msgr2=0x55cb62ae3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:19:06.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.199+0000 7f5e7aa00640 1 --2- 172.21.15.27:0/3036743127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb62716280 0x55cb62ae3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.199+0000 7f5e7aa00640 1 -- 172.21.15.27:0/3036743127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62736f60 msgr2=0x55cb62ae44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:19:06.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.199+0000 7f5e7aa00640 1 --2- 172.21.15.27:0/3036743127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62736f60 0x55cb62ae44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.199+0000 7f5e7aa00640 1 -- 172.21.15.27:0/3036743127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb62adf780 con 0x55cb6270e0a0 2024-09-21T00:19:06.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.199+0000 7f5e7aa00640 1 --2- 172.21.15.27:0/3036743127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb6270e0a0 0x55cb62af5110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c006010 tx=0x7f5e6c032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2dff6f2302f9e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:06.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.199+0000 7f5e6b400640 1 -- 172.21.15.27:0/3036743127 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c04d070 con 0x55cb6270e0a0 2024-09-21T00:19:06.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.199+0000 7f5e6b400640 1 -- 172.21.15.27:0/3036743127 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e6c003d90 con 0x55cb6270e0a0 2024-09-21T00:19:06.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.199+0000 7f5e6b400640 1 -- 172.21.15.27:0/3036743127 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c035040 con 0x55cb6270e0a0 2024-09-21T00:19:06.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.200+0000 7f5e88be1740 1 -- 172.21.15.27:0/3036743127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb6270e0a0 msgr2=0x55cb62af5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:06.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.200+0000 7f5e88be1740 1 --2- 172.21.15.27:0/3036743127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb6270e0a0 0x55cb62af5110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c006010 tx=0x7f5e6c032e60 comp rx=0 tx=0).stop 2024-09-21T00:19:06.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.200+0000 7f5e88be1740 1 -- 172.21.15.27:0/3036743127 shutdown_connections 2024-09-21T00:19:06.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.200+0000 7f5e88be1740 1 --2- 172.21.15.27:0/3036743127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb62736f60 0x55cb62ae44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.200+0000 7f5e88be1740 1 --2- 172.21.15.27:0/3036743127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb62716280 0x55cb62ae3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.200+0000 7f5e88be1740 1 --2- 172.21.15.27:0/3036743127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb6270e0a0 0x55cb62af5110 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.200+0000 7f5e88be1740 1 -- 172.21.15.27:0/3036743127 >> 172.21.15.27:0/3036743127 conn(0x55cb6272b350 msgr2=0x55cb62af8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:06.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.200+0000 7f5e88be1740 1 -- 172.21.15.27:0/3036743127 shutdown_connections 2024-09-21T00:19:06.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.200+0000 7f5e88be1740 1 -- 172.21.15.27:0/3036743127 wait complete. 2024-09-21T00:19:06.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.201+0000 7f5e88be1740 1 -- start start 2024-09-21T00:19:06.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.201+0000 7f5e88be1740 1 Processor -- start 2024-09-21T00:19:06.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.201+0000 7f5e88be1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb6270e0a0 0x55cb62b04000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.201+0000 7f5e88be1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62716280 0x55cb62afd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e88be1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb62736f60 0x55cb62afda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e88be1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cb62afb380 con 0x55cb62716280 2024-09-21T00:19:06.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e88be1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55cb62afacb0 con 0x55cb6270e0a0 2024-09-21T00:19:06.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e88be1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cb62afba50 con 0x55cb62736f60 2024-09-21T00:19:06.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e88be1740 10 client.0 _ll_register_callbacks cb 0x7f5e6c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:19:06.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e72000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62716280 0x55cb62afd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e72000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62716280 0x55cb62afd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39188/0 (socket says 172.21.15.27:39188) 2024-09-21T00:19:06.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e72000640 1 -- 172.21.15.27:0/691303337 learned_addr learned my addr 172.21.15.27:0/691303337 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:06.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e7aa00640 1 --2- 172.21.15.27:0/691303337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb6270e0a0 0x55cb62b04000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e7b400640 1 --2- 172.21.15.27:0/691303337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb62736f60 0x55cb62afda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e72000640 1 -- 172.21.15.27:0/691303337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb62736f60 msgr2=0x55cb62afda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:19:06.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e72000640 1 --2- 172.21.15.27:0/691303337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb62736f60 0x55cb62afda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e72000640 1 -- 172.21.15.27:0/691303337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb6270e0a0 msgr2=0x55cb62b04000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:19:06.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e72000640 1 --2- 172.21.15.27:0/691303337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb6270e0a0 0x55cb62b04000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e72000640 1 -- 172.21.15.27:0/691303337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb62afe200 con 0x55cb62716280 2024-09-21T00:19:06.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.202+0000 7f5e7aa00640 1 --2- 172.21.15.27:0/691303337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb6270e0a0 0x55cb62b04000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:19:06.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.203+0000 7f5e72000640 1 --2- 172.21.15.27:0/691303337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62716280 0x55cb62afd4b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5e640ad770 tx=0x7f5e64004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc9bcc84debed1c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:06.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.203+0000 7f5e68c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:19:06.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.203+0000 7f5e68c00640 1 -- 172.21.15.27:0/691303337 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64005310 con 0x55cb62716280 2024-09-21T00:19:06.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.203+0000 7f5e68c00640 1 -- 172.21.15.27:0/691303337 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f5e640aca40 con 0x55cb62716280 2024-09-21T00:19:06.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.203+0000 7f5e88be1740 1 -- 172.21.15.27:0/691303337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e6c003d90 con 0x55cb62716280 2024-09-21T00:19:06.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.203+0000 7f5e68c00640 1 -- 172.21.15.27:0/691303337 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f5e64005680 con 0x55cb62716280 2024-09-21T00:19:06.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.203+0000 7f5e88be1740 10 client.14853 fetch_fsmap learned FSMap version 16 2024-09-21T00:19:06.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.203+0000 7f5e88be1740 1 -- 172.21.15.27:0/691303337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x55cb62b66a30 con 0x55cb62716280 2024-09-21T00:19:06.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.203+0000 7f5e68c00640 1 -- 172.21.15.27:0/691303337 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e6c003d90 con 0x55cb62716280 2024-09-21T00:19:06.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.204+0000 7f5e68c00640 1 -- 172.21.15.27:0/691303337 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f5e640a9130 con 0x55cb62716280 2024-09-21T00:19:06.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.204+0000 7f5e88be1740 10 client.14853 fetch_fsmap finished waiting for FSMap version 16 2024-09-21T00:19:06.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.204+0000 7f5e88be1740 10 client.14853 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T00:19:06.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.204+0000 7f5e88be1740 20 client.14853 populate_metadata read hostname 'smithi027' 2024-09-21T00:19:06.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.204+0000 7f5e88be1740 1 --2- 172.21.15.27:0/691303337 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55cb626cac90 0x55cb62b6a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:06.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.204+0000 7f5e88be1740 4 client.14853 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:19:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.204+0000 7f5e88be1740 1 -- 172.21.15.27:0/691303337 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "status"}) -- 0x55cb62a52750 con 0x55cb626cac90 2024-09-21T00:19:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.204+0000 7f5e7aa00640 1 --2- 172.21.15.27:0/691303337 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55cb626cac90 0x55cb62b6a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.205+0000 7f5e7aa00640 1 --2- 172.21.15.27:0/691303337 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55cb626cac90 0x55cb62b6a2d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e93a09ea1ddd5d02 server_cookie=fef4795cccc9838a in_seq=0 out_seq=0 2024-09-21T00:19:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.205+0000 7f5e68c00640 10 client.14853 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:19:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.205+0000 7f5e68c00640 1 -- 172.21.15.27:0/691303337 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55cb62a52750 con 0x55cb626cac90 2024-09-21T00:19:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.205+0000 7f5e68c00640 10 client.14853 handle_command_reply: tid=0 2024-09-21T00:19:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.205+0000 7f5e88be1740 1 client.14853 shutdown 2024-09-21T00:19:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.205+0000 7f5e88be1740 1 -- 172.21.15.27:0/691303337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62716280 msgr2=0x55cb62afd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.205+0000 7f5e88be1740 1 --2- 172.21.15.27:0/691303337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62716280 0x55cb62afd4b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5e640ad770 tx=0x7f5e64004a20 comp rx=0 tx=0).stop 2024-09-21T00:19:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.205+0000 7f5e88be1740 1 -- 172.21.15.27:0/691303337 shutdown_connections 2024-09-21T00:19:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.205+0000 7f5e88be1740 1 --2- 172.21.15.27:0/691303337 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55cb626cac90 0x55cb62b6a2d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.205+0000 7f5e88be1740 1 --2- 172.21.15.27:0/691303337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb62736f60 0x55cb62afda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.205+0000 7f5e68c00640 0 client.14853 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:19:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.205+0000 7f5e88be1740 1 --2- 172.21.15.27:0/691303337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cb62716280 0x55cb62afd4b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.205+0000 7f5e88be1740 1 --2- 172.21.15.27:0/691303337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb6270e0a0 0x55cb62b04000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.205+0000 7f5e88be1740 1 -- 172.21.15.27:0/691303337 >> 172.21.15.27:0/691303337 conn(0x55cb6272b350 msgr2=0x55cb62af5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:06.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.205+0000 7f5e88be1740 1 -- 172.21.15.27:0/691303337 shutdown_connections 2024-09-21T00:19:06.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.206+0000 7f5e88be1740 1 -- 172.21.15.27:0/691303337 wait complete. 2024-09-21T00:19:06.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.206+0000 7f5e88be1740 20 client.14853 trim_cache size 0 max 16384 2024-09-21T00:19:06.211 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:19:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "9c21de84-77ad-11ef-baf5-efdc52797490", 2024-09-21T00:19:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-21T00:19:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24761, 2024-09-21T00:19:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T00:19:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T00:19:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T00:19:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 25.542993468999999, 2024-09-21T00:19:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 16, 2024-09-21T00:19:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 86, 2024-09-21T00:19:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 86, 2024-09-21T00:19:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 39.010246735000003 2024-09-21T00:19:06.212 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:19:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.206+0000 7f5e81e00640 1 -- 172.21.15.27:0/2343945156 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5e600b65e0 msgr2=0x7f5e600b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.206+0000 7f5e81e00640 1 --2- 172.21.15.27:0/2343945156 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5e600b65e0 0x7f5e600b8af0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c031440 tx=0x7f5e6c009560 comp rx=0 tx=0).stop 2024-09-21T00:19:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.206+0000 7f5e81e00640 1 -- 172.21.15.27:0/2343945156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e7c10b6c0 msgr2=0x7f5e7c19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.206+0000 7f5e81e00640 1 --2- 172.21.15.27:0/2343945156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e7c10b6c0 0x7f5e7c19cc80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400e9c0 tx=0x7f5e6400ee80 comp rx=0 tx=0).stop 2024-09-21T00:19:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.206+0000 7f5e81e00640 1 -- 172.21.15.27:0/2343945156 shutdown_connections 2024-09-21T00:19:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.206+0000 7f5e81e00640 1 --2- 172.21.15.27:0/2343945156 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5e600b65e0 0x7f5e600b8af0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.206+0000 7f5e81e00640 1 --2- 172.21.15.27:0/2343945156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e7c10c010 0x7f5e7c1a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.206+0000 7f5e81e00640 1 --2- 172.21.15.27:0/2343945156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e7c10b6c0 0x7f5e7c19cc80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.206+0000 7f5e81e00640 1 --2- 172.21.15.27:0/2343945156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e7c105d80 0x7f5e7c19c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.207+0000 7f5e81e00640 1 -- 172.21.15.27:0/2343945156 >> 172.21.15.27:0/2343945156 conn(0x7f5e7c101810 msgr2=0x7f5e7c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.207+0000 7f5e81e00640 1 -- 172.21.15.27:0/2343945156 shutdown_connections 2024-09-21T00:19:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:06.207+0000 7f5e81e00640 1 -- 172.21.15.27:0/2343945156 wait complete. 2024-09-21T00:19:06.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:05 smithi089 ceph-mon[34481]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:06.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:05 smithi098 ceph-mon[34725]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:07.104 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T00:19:07.104 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:19:07.104 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T00:19:07.489 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:07.954 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:07 smithi027 ceph-mon[30745]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:08.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.136+0000 7f6a0d200640 1 -- 172.21.15.27:0/3263165878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a08105db0 msgr2=0x7f6a081061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:08.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.136+0000 7f6a0d200640 1 --2- 172.21.15.27:0/3263165878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a08105db0 0x7f6a081061c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009900 tx=0x7f69f802a410 comp rx=0 tx=0).stop 2024-09-21T00:19:08.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.136+0000 7f6a0d200640 1 -- 172.21.15.27:0/3263165878 shutdown_connections 2024-09-21T00:19:08.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.136+0000 7f6a0d200640 1 --2- 172.21.15.27:0/3263165878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a0810c040 0x7f6a08112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:08.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.136+0000 7f6a0d200640 1 --2- 172.21.15.27:0/3263165878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0810b6f0 0x7f6a0810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:08.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.136+0000 7f6a0d200640 1 --2- 172.21.15.27:0/3263165878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a08105db0 0x7f6a081061c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:08.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.136+0000 7f6a0d200640 1 -- 172.21.15.27:0/3263165878 >> 172.21.15.27:0/3263165878 conn(0x7f6a081017e0 msgr2=0x7f6a08103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:08.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.136+0000 7f6a0d200640 1 -- 172.21.15.27:0/3263165878 shutdown_connections 2024-09-21T00:19:08.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.136+0000 7f6a0d200640 1 -- 172.21.15.27:0/3263165878 wait complete. 2024-09-21T00:19:08.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.137+0000 7f6a0d200640 1 Processor -- start 2024-09-21T00:19:08.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.138+0000 7f6a0d200640 1 -- start start 2024-09-21T00:19:08.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.138+0000 7f6a0d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a08105db0 0x7f6a0819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:08.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.138+0000 7f6a0d200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0810b6f0 0x7f6a0819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:08.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.138+0000 7f6a0d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a0810c040 0x7f6a081a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:08.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.138+0000 7f6a0d200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6a08115d00 con 0x7f6a08105db0 2024-09-21T00:19:08.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.138+0000 7f6a0d200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6a08115b80 con 0x7f6a0810c040 2024-09-21T00:19:08.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.138+0000 7f6a0d200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6a08115e80 con 0x7f6a0810b6f0 2024-09-21T00:19:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.138+0000 7f6a05e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a08105db0 0x7f6a0819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.139+0000 7f6a05e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a08105db0 0x7f6a0819c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39202/0 (socket says 172.21.15.27:39202) 2024-09-21T00:19:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.139+0000 7f6a05e00640 1 -- 172.21.15.27:0/1405506841 learned_addr learned my addr 172.21.15.27:0/1405506841 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.139+0000 7f69ffe00640 1 --2- 172.21.15.27:0/1405506841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0810b6f0 0x7f6a0819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.139+0000 7f6a06800640 1 --2- 172.21.15.27:0/1405506841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a0810c040 0x7f6a081a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.139+0000 7f6a05e00640 1 -- 172.21.15.27:0/1405506841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0810b6f0 msgr2=0x7f6a0819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.139+0000 7f6a05e00640 1 --2- 172.21.15.27:0/1405506841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0810b6f0 0x7f6a0819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.139+0000 7f6a05e00640 1 -- 172.21.15.27:0/1405506841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a0810c040 msgr2=0x7f6a081a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.139+0000 7f6a05e00640 1 --2- 172.21.15.27:0/1405506841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a0810c040 0x7f6a081a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.139+0000 7f6a05e00640 1 -- 172.21.15.27:0/1405506841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a081a2640 con 0x7f6a08105db0 2024-09-21T00:19:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.139+0000 7f6a06800640 1 --2- 172.21.15.27:0/1405506841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a0810c040 0x7f6a081a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:19:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.139+0000 7f6a05e00640 1 --2- 172.21.15.27:0/1405506841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a08105db0 0x7f6a0819c6d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f69f8005940 tx=0x7f69f80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.140+0000 7f69fea00640 1 -- 172.21.15.27:0/1405506841 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f803f030 con 0x7f6a08105db0 2024-09-21T00:19:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.140+0000 7f69fea00640 1 -- 172.21.15.27:0/1405506841 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f80044c0 con 0x7f6a08105db0 2024-09-21T00:19:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.140+0000 7f6a0d200640 1 -- 172.21.15.27:0/1405506841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a081a28d0 con 0x7f6a08105db0 2024-09-21T00:19:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.140+0000 7f69fea00640 1 -- 172.21.15.27:0/1405506841 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8035070 con 0x7f6a08105db0 2024-09-21T00:19:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.140+0000 7f6a0d200640 1 -- 172.21.15.27:0/1405506841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a081a9090 con 0x7f6a08105db0 2024-09-21T00:19:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.141+0000 7f6a0d200640 1 -- 172.21.15.27:0/1405506841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a081061c0 con 0x7f6a08105db0 2024-09-21T00:19:08.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.143+0000 7f69fea00640 1 -- 172.21.15.27:0/1405506841 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f69f8042030 con 0x7f6a08105db0 2024-09-21T00:19:08.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.145+0000 7f69fea00640 1 --2- 172.21.15.27:0/1405506841 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f69ec0b65e0 0x7f69ec0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:08.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.145+0000 7f69fea00640 1 -- 172.21.15.27:0/1405506841 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f69f8107cd0 con 0x7f6a08105db0 2024-09-21T00:19:08.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.145+0000 7f69ffe00640 1 --2- 172.21.15.27:0/1405506841 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f69ec0b65e0 0x7f69ec0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:08.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.146+0000 7f69ffe00640 1 --2- 172.21.15.27:0/1405506841 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f69ec0b65e0 0x7f69ec0b8af0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6a08105050 tx=0x7f69f0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.147+0000 7f69fea00640 1 -- 172.21.15.27:0/1405506841 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f69f80ce1f0 con 0x7f6a08105db0 2024-09-21T00:19:08.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:07 smithi089 ceph-mon[34481]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:08.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:07 smithi098 ceph-mon[34725]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:08.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.389+0000 7f6a0d200640 1 -- 172.21.15.27:0/1405506841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6a0802e640 con 0x7f6a08105db0 2024-09-21T00:19:08.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.390+0000 7f69fea00640 1 -- 172.21.15.27:0/1405506841 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7f69f80d25d0 con 0x7f6a08105db0 2024-09-21T00:19:08.391 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T00:19:08.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.393+0000 7f6a0d200640 1 -- 172.21.15.27:0/1405506841 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f69ec0b65e0 msgr2=0x7f69ec0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:08.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.393+0000 7f6a0d200640 1 --2- 172.21.15.27:0/1405506841 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f69ec0b65e0 0x7f69ec0b8af0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6a08105050 tx=0x7f69f0005f90 comp rx=0 tx=0).stop 2024-09-21T00:19:08.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.393+0000 7f6a0d200640 1 -- 172.21.15.27:0/1405506841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a08105db0 msgr2=0x7f6a0819c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:08.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.393+0000 7f6a0d200640 1 --2- 172.21.15.27:0/1405506841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a08105db0 0x7f6a0819c6d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f69f8005940 tx=0x7f69f80040c0 comp rx=0 tx=0).stop 2024-09-21T00:19:08.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.393+0000 7f6a0d200640 1 -- 172.21.15.27:0/1405506841 shutdown_connections 2024-09-21T00:19:08.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.393+0000 7f6a0d200640 1 --2- 172.21.15.27:0/1405506841 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f69ec0b65e0 0x7f69ec0b8af0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:08.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.393+0000 7f6a0d200640 1 --2- 172.21.15.27:0/1405506841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a0810c040 0x7f6a081a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:08.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.393+0000 7f6a0d200640 1 --2- 172.21.15.27:0/1405506841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0810b6f0 0x7f6a0819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:08.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.393+0000 7f6a0d200640 1 --2- 172.21.15.27:0/1405506841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a08105db0 0x7f6a0819c6d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:08.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.393+0000 7f6a0d200640 1 -- 172.21.15.27:0/1405506841 >> 172.21.15.27:0/1405506841 conn(0x7f6a081017e0 msgr2=0x7f6a0810adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:08.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.393+0000 7f6a0d200640 1 -- 172.21.15.27:0/1405506841 shutdown_connections 2024-09-21T00:19:08.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:08.393+0000 7f6a0d200640 1 -- 172.21.15.27:0/1405506841 wait complete. 2024-09-21T00:19:09.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:08 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1405506841' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T00:19:09.115 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:19:09.115 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:19:09.115 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-21T00:19:09.116 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-21T00:19:09.116 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:19:09.116 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:19:09.116 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T00:19:09.116 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:19:09.116 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T00:19:09.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:19:09.116 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-21T00:19:09.152 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:19:09.152 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:19:09.153 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:19:09.153 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:19:09.153 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-21T00:19:09.153 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:19:09.153 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:19:09.153 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:19:09.153 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-21T00:19:09.153 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:19:09.153 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:19:09.153 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-21T00:19:09.153 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:19:09.153 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-21T00:19:09.153 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-21T00:19:09.153 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T00:19:09.153 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42133sec preferred_lft 42133sec 2024-09-21T00:19:09.154 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-21T00:19:09.154 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:19:09.154 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:19:09.154 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-21T00:19:09.154 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-21T00:19:09.155 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T00:19:09.155 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:19:09.155 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T00:19:09.155 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2024-09-21T00:19:09.155 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2024-09-21T00:19:09.155 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2024-09-21T00:19:09.155 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T00:19:09.156 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T00:19:09.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:08 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1405506841' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T00:19:09.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:08 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1405506841' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T00:19:09.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:19:09.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:19:09.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:19:09.597 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:19:09.655 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-21T00:19:09.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:19:09.659 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-21T00:19:09.709 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-21T00:19:09.709 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:19:09.709 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T00:19:09.709 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T00:19:09.709 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:19:09.710 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:19:09.710 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T00:19:09.710 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T00:19:09.710 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T00:19:09.710 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T00:19:09.710 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T00:19:09.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:19:10.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:09 smithi027 ceph-mon[30745]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:10.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:09 smithi089 ceph-mon[34481]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:10.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:09 smithi098 ceph-mon[34725]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:10.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:19:10.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:19:10.325 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-21T00:19:10.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:19:10.362 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-09-21T00:19:10.439 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:19:10.439 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T00:19:10.439 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T00:19:10.439 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-21T00:19:10.439 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T00:19:10.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:19:10.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:19:10.623 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-21T00:19:10.623 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:19:10.623 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T00:19:10.623 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-21T00:19:10.623 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T00:19:10.624 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-21T00:19:10.624 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-21T00:19:10.625 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-21T00:19:10.625 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T00:19:10.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:19:10.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:19:10.876 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:19:10.876 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T00:19:10.876 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-21T00:19:10.876 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T00:19:10.876 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T00:19:10.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:19:11.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:19:11.047 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-21T00:19:11.048 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:19:11.048 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:19:11.083 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T00:19:11.084 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:19:11.084 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:19:11.159 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-09-21T00:19:11.258 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T00:19:11.297 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-09-21T00:19:11.297 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-09-21T00:19:11.297 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-09-21T00:19:11.297 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-09-21T00:19:11.297 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-09-21T00:19:11.297 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-09-21T00:19:11.297 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-09-21T00:19:11.297 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-09-21T00:19:11.297 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-09-21T00:19:11.297 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-09-21T00:19:11.297 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-09-21T00:19:11.297 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-09-21T00:19:11.298 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T00:19:11.298 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-09-21T00:19:11.298 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-09-21T00:19:11.298 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-09-21T00:19:11.298 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-09-21T00:19:11.298 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-09-21T00:19:11.298 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-09-21T00:19:11.298 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T00:19:11.298 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T00:19:11.298 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T00:19:11.298 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T00:19:11.298 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-09-21T00:19:11.298 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-09-21T00:19:11.298 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-09-21T00:19:11.299 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/08e374f91e3f80cd3ea8f340f5c395fc83d29ceb6575cdaefd2bd7164e506973/merged 2024-09-21T00:19:11.299 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/0135fc2a8b9604a3a6fc96c258e086ef85db81c8efb5732cb389891fc55a73ca/merged 2024-09-21T00:19:11.299 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-09-21T00:19:11.299 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T00:19:11.299 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/327bb41e128901be0655a8ce441b4e7f8360cec8b75cf9e890c7c1126d7c2cbf/merged 2024-09-21T00:19:11.299 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/1c95e4c60ca6987d8aad22098d0facb8e8838870f491f4d733050b748fe2e622/merged 2024-09-21T00:19:11.299 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/77ce322d2ad31f9eb611d80507ea7f1c267ace32f9178d902bb93e63af7261ed/merged 2024-09-21T00:19:11.299 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/f9d6a1d830c0955bb6d884ad76b133e2510f4c8e616143327b04c4311fba0e20/merged 2024-09-21T00:19:11.299 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/9f75a3971628c70b5b7e428d0b8bb0064c0b2ecc0c3fa9d480e38c37bea26e9f/merged 2024-09-21T00:19:11.299 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/b1f640be5d83b9acd502e859a77673f3709b56228a30b5dc4119376f7eb719ae/merged 2024-09-21T00:19:11.299 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/4773807c6467d64ec9277792e5e9184ebd979b705d11a9ae8bd8c66db0299733/merged 2024-09-21T00:19:11.299 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/65ca6b378fc3e89ea660b3fbab808e96b574eeb9b8394121d68789e879b536ca/merged 2024-09-21T00:19:11.299 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-09-21T00:19:11.299 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T00:19:11.300 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T00:19:11.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:19:11.300 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-09-21T00:19:11.376 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T00:19:11.377 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-21T00:19:11.421 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-09-21T00:19:11.480 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T00:19:11.518 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-09-21T00:19:11.518 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-09-21T00:19:11.518 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-09-21T00:19:11.518 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-09-21T00:19:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-09-21T00:19:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-09-21T00:19:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-09-21T00:19:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-09-21T00:19:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-09-21T00:19:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-09-21T00:19:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-09-21T00:19:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-09-21T00:19:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T00:19:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-09-21T00:19:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-09-21T00:19:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-09-21T00:19:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-09-21T00:19:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-09-21T00:19:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-09-21T00:19:11.520 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T00:19:11.520 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T00:19:11.520 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T00:19:11.520 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T00:19:11.520 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-09-21T00:19:11.520 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-09-21T00:19:11.520 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-09-21T00:19:11.520 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/08e374f91e3f80cd3ea8f340f5c395fc83d29ceb6575cdaefd2bd7164e506973/merged 2024-09-21T00:19:11.520 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/0135fc2a8b9604a3a6fc96c258e086ef85db81c8efb5732cb389891fc55a73ca/merged 2024-09-21T00:19:11.520 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-09-21T00:19:11.520 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T00:19:11.520 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/327bb41e128901be0655a8ce441b4e7f8360cec8b75cf9e890c7c1126d7c2cbf/merged 2024-09-21T00:19:11.520 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/1c95e4c60ca6987d8aad22098d0facb8e8838870f491f4d733050b748fe2e622/merged 2024-09-21T00:19:11.521 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/77ce322d2ad31f9eb611d80507ea7f1c267ace32f9178d902bb93e63af7261ed/merged 2024-09-21T00:19:11.521 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/f9d6a1d830c0955bb6d884ad76b133e2510f4c8e616143327b04c4311fba0e20/merged 2024-09-21T00:19:11.521 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/9f75a3971628c70b5b7e428d0b8bb0064c0b2ecc0c3fa9d480e38c37bea26e9f/merged 2024-09-21T00:19:11.521 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/b1f640be5d83b9acd502e859a77673f3709b56228a30b5dc4119376f7eb719ae/merged 2024-09-21T00:19:11.521 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/4773807c6467d64ec9277792e5e9184ebd979b705d11a9ae8bd8c66db0299733/merged 2024-09-21T00:19:11.521 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/65ca6b378fc3e89ea660b3fbab808e96b574eeb9b8394121d68789e879b536ca/merged 2024-09-21T00:19:11.521 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-09-21T00:19:11.521 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T00:19:11.521 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T00:19:11.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:19:11.521 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-09-21T00:19:11.686 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:2024-09-21T00:19:11.685+0000 7ff38da24480 -1 init, newargv = 0x7ff378004ba0 newargc=15 2024-09-21T00:19:11.687 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[73795]: starting ceph client 2024-09-21T00:19:11.697 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[73795]: starting fuse 2024-09-21T00:19:12.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:11 smithi027 ceph-mon[30745]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:12.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:11 smithi089 ceph-mon[34481]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:12.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:11 smithi098 ceph-mon[34725]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:12.696 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-09-21T00:19:12.766 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T00:19:12.806 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-09-21T00:19:12.806 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-09-21T00:19:12.806 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-09-21T00:19:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-09-21T00:19:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-09-21T00:19:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-09-21T00:19:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-09-21T00:19:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-09-21T00:19:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-09-21T00:19:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-09-21T00:19:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-09-21T00:19:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-09-21T00:19:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T00:19:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-09-21T00:19:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-09-21T00:19:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-09-21T00:19:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-09-21T00:19:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-09-21T00:19:12.808 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-09-21T00:19:12.808 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T00:19:12.808 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T00:19:12.808 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T00:19:12.808 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T00:19:12.808 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-09-21T00:19:12.808 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-09-21T00:19:12.808 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-09-21T00:19:12.808 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/08e374f91e3f80cd3ea8f340f5c395fc83d29ceb6575cdaefd2bd7164e506973/merged 2024-09-21T00:19:12.808 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/0135fc2a8b9604a3a6fc96c258e086ef85db81c8efb5732cb389891fc55a73ca/merged 2024-09-21T00:19:12.808 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-09-21T00:19:12.808 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T00:19:12.808 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/327bb41e128901be0655a8ce441b4e7f8360cec8b75cf9e890c7c1126d7c2cbf/merged 2024-09-21T00:19:12.809 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/1c95e4c60ca6987d8aad22098d0facb8e8838870f491f4d733050b748fe2e622/merged 2024-09-21T00:19:12.809 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/77ce322d2ad31f9eb611d80507ea7f1c267ace32f9178d902bb93e63af7261ed/merged 2024-09-21T00:19:12.809 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/f9d6a1d830c0955bb6d884ad76b133e2510f4c8e616143327b04c4311fba0e20/merged 2024-09-21T00:19:12.809 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/9f75a3971628c70b5b7e428d0b8bb0064c0b2ecc0c3fa9d480e38c37bea26e9f/merged 2024-09-21T00:19:12.809 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/b1f640be5d83b9acd502e859a77673f3709b56228a30b5dc4119376f7eb719ae/merged 2024-09-21T00:19:12.809 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/4773807c6467d64ec9277792e5e9184ebd979b705d11a9ae8bd8c66db0299733/merged 2024-09-21T00:19:12.809 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/65ca6b378fc3e89ea660b3fbab808e96b574eeb9b8394121d68789e879b536ca/merged 2024-09-21T00:19:12.809 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-09-21T00:19:12.809 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T00:19:12.809 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T00:19:12.809 INFO:teuthology.orchestra.run.smithi027.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-21T00:19:12.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:19:12.810 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-09-21T00:19:12.888 INFO:teuthology.orchestra.run.smithi027.stdout:84 2024-09-21T00:19:12.888 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [84] 2024-09-21T00:19:12.889 DEBUG:teuthology.orchestra.run.smithi027:> sudo stdin-killer -- python3 -c ' 2024-09-21T00:19:12.889 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2024-09-21T00:19:12.889 DEBUG:teuthology.orchestra.run.smithi027:> import re 2024-09-21T00:19:12.889 DEBUG:teuthology.orchestra.run.smithi027:> import os 2024-09-21T00:19:12.889 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2024-09-21T00:19:12.889 DEBUG:teuthology.orchestra.run.smithi027:> 2024-09-21T00:19:12.889 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2024-09-21T00:19:12.889 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-21T00:19:12.889 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2024-09-21T00:19:12.889 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-21T00:19:12.889 DEBUG:teuthology.orchestra.run.smithi027:> 2024-09-21T00:19:12.889 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2024-09-21T00:19:12.889 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2024-09-21T00:19:12.889 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2024-09-21T00:19:12.889 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2024-09-21T00:19:12.890 DEBUG:teuthology.orchestra.run.smithi027:> 2024-09-21T00:19:12.890 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2024-09-21T00:19:12.890 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T00:19:12.890 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T00:19:12.890 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T00:19:12.890 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2024-09-21T00:19:12.890 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2024-09-21T00:19:12.890 DEBUG:teuthology.orchestra.run.smithi027:> return f 2024-09-21T00:19:12.890 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T00:19:12.890 DEBUG:teuthology.orchestra.run.smithi027:> 2024-09-21T00:19:12.890 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.admin")) 2024-09-21T00:19:12.890 DEBUG:teuthology.orchestra.run.smithi027:> ' 2024-09-21T00:19:13.056 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.admin.73795.asok 2024-09-21T00:19:13.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:19:13.071 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73795.asok 2024-09-21T00:19:13.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:19:13.071 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73795.asok status 2024-09-21T00:19:13.328 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:19:13.328 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2024-09-21T00:19:13.329 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T00:19:13.329 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T00:19:13.329 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "admin", 2024-09-21T00:19:13.329 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2024-09-21T00:19:13.329 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-21T00:19:13.329 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "73795", 2024-09-21T00:19:13.329 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2024-09-21T00:19:13.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:19:13.329 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2024-09-21T00:19:13.329 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2024-09-21T00:19:13.329 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 14865, 2024-09-21T00:19:13.329 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2024-09-21T00:19:13.329 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2024-09-21T00:19:13.329 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2024-09-21T00:19:13.330 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 14865 2024-09-21T00:19:13.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:19:13.330 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2024-09-21T00:19:13.330 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-21T00:19:13.330 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2024-09-21T00:19:13.330 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 356555085 2024-09-21T00:19:13.330 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:19:13.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:19:13.330 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2024-09-21T00:19:13.330 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-21T00:19:13.330 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2024-09-21T00:19:13.330 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 356555085 2024-09-21T00:19:13.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:19:13.330 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.14865 192.168.0.1:0/356555085", 2024-09-21T00:19:13.330 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/356555085", 2024-09-21T00:19:13.331 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2024-09-21T00:19:13.331 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 16, 2024-09-21T00:19:13.331 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 86, 2024-09-21T00:19:13.331 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2024-09-21T00:19:13.331 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2024-09-21T00:19:13.331 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2024-09-21T00:19:13.331 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:19:13.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:19:13.347 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2024-09-21T00:19:13.347 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:19:13.384 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2024-09-21T00:19:13.385 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:19:13.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:19:13.385 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:19:13.484 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-21T00:19:13.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:19:13.592 INFO:teuthology.orchestra.run.smithi027.stdout:# file: . 2024-09-21T00:19:13.593 INFO:teuthology.orchestra.run.smithi027.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-21T00:19:13.593 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:19:13.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:19:13.601 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:19:13.601 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:19:13.637 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T00:19:14.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:13 smithi027 ceph-mon[30745]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:14.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:13 smithi089 ceph-mon[34481]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:14.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:13 smithi098 ceph-mon[34725]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:14.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.692+0000 7fb201600640 1 -- 172.21.15.27:0/1400388234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc105d80 msgr2=0x7fb1fc106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:14.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.692+0000 7fb201600640 1 --2- 172.21.15.27:0/1400388234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc105d80 0x7fb1fc106190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009900 tx=0x7fb1ec02a400 comp rx=0 tx=0).stop 2024-09-21T00:19:14.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.692+0000 7fb201600640 1 -- 172.21.15.27:0/1400388234 shutdown_connections 2024-09-21T00:19:14.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.692+0000 7fb201600640 1 --2- 172.21.15.27:0/1400388234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1fc10c010 0x7fb1fc112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:14.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.692+0000 7fb201600640 1 --2- 172.21.15.27:0/1400388234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1fc10b6c0 0x7fb1fc10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:14.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.692+0000 7fb201600640 1 --2- 172.21.15.27:0/1400388234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc105d80 0x7fb1fc106190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:14.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.692+0000 7fb201600640 1 -- 172.21.15.27:0/1400388234 >> 172.21.15.27:0/1400388234 conn(0x7fb1fc101810 msgr2=0x7fb1fc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:14.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.692+0000 7fb201600640 1 -- 172.21.15.27:0/1400388234 shutdown_connections 2024-09-21T00:19:14.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.692+0000 7fb201600640 1 -- 172.21.15.27:0/1400388234 wait complete. 2024-09-21T00:19:14.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.693+0000 7fb201600640 1 Processor -- start 2024-09-21T00:19:14.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.693+0000 7fb201600640 1 -- start start 2024-09-21T00:19:14.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.694+0000 7fb201600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1fc105d80 0x7fb1fc19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:14.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.694+0000 7fb201600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc10b6c0 0x7fb1fc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:14.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.694+0000 7fb201600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1fc10c010 0x7fb1fc1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:14.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.694+0000 7fb201600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc115cd0 con 0x7fb1fc10b6c0 2024-09-21T00:19:14.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.694+0000 7fb201600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc115b50 con 0x7fb1fc10c010 2024-09-21T00:19:14.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.694+0000 7fb201600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc115e50 con 0x7fb1fc105d80 2024-09-21T00:19:14.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.694+0000 7fb1faa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1fc105d80 0x7fb1fc19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:14.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.694+0000 7fb1f2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc10b6c0 0x7fb1fc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:14.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.694+0000 7fb1f2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc10b6c0 0x7fb1fc19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49760/0 (socket says 172.21.15.27:49760) 2024-09-21T00:19:14.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.694+0000 7fb1f2000640 1 -- 172.21.15.27:0/512159694 learned_addr learned my addr 172.21.15.27:0/512159694 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:14.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.694+0000 7fb1faa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1fc105d80 0x7fb1fc19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:36014/0 (socket says 172.21.15.27:36014) 2024-09-21T00:19:14.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.694+0000 7fb1fb400640 1 --2- 172.21.15.27:0/512159694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1fc10c010 0x7fb1fc1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:14.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.695+0000 7fb1f2000640 1 -- 172.21.15.27:0/512159694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1fc105d80 msgr2=0x7fb1fc19c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:14.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.695+0000 7fb1f2000640 1 --2- 172.21.15.27:0/512159694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1fc105d80 0x7fb1fc19c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:14.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.695+0000 7fb1f2000640 1 -- 172.21.15.27:0/512159694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1fc10c010 msgr2=0x7fb1fc1a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:14.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.695+0000 7fb1f2000640 1 --2- 172.21.15.27:0/512159694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1fc10c010 0x7fb1fc1a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:14.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.695+0000 7fb1f2000640 1 -- 172.21.15.27:0/512159694 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc1a2690 con 0x7fb1fc10b6c0 2024-09-21T00:19:14.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.695+0000 7fb1f2000640 1 --2- 172.21.15.27:0/512159694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc10b6c0 0x7fb1fc19ccc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400e9f0 tx=0x7fb1e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:14.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.695+0000 7fb1f3e00640 1 -- 172.21.15.27:0/512159694 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e400ccc0 con 0x7fb1fc10b6c0 2024-09-21T00:19:14.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.695+0000 7fb1f3e00640 1 -- 172.21.15.27:0/512159694 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e40108e0 con 0x7fb1fc10b6c0 2024-09-21T00:19:14.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.695+0000 7fb201600640 1 -- 172.21.15.27:0/512159694 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1fc1a2980 con 0x7fb1fc10b6c0 2024-09-21T00:19:14.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.696+0000 7fb1f3e00640 1 -- 172.21.15.27:0/512159694 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4018860 con 0x7fb1fc10b6c0 2024-09-21T00:19:14.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.696+0000 7fb201600640 1 -- 172.21.15.27:0/512159694 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1fc1a90e0 con 0x7fb1fc10b6c0 2024-09-21T00:19:14.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.697+0000 7fb201600640 1 -- 172.21.15.27:0/512159694 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1fc106190 con 0x7fb1fc10b6c0 2024-09-21T00:19:14.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.701+0000 7fb1f3e00640 1 -- 172.21.15.27:0/512159694 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fb1e4010430 con 0x7fb1fc10b6c0 2024-09-21T00:19:14.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.702+0000 7fb1f3e00640 1 --2- 172.21.15.27:0/512159694 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb1e00b6560 0x7fb1e00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:14.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.703+0000 7fb1f3e00640 1 -- 172.21.15.27:0/512159694 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7fb1e40e7e30 con 0x7fb1fc10b6c0 2024-09-21T00:19:14.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.703+0000 7fb1faa00640 1 --2- 172.21.15.27:0/512159694 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb1e00b6560 0x7fb1e00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:14.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.704+0000 7fb1faa00640 1 --2- 172.21.15.27:0/512159694 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb1e00b6560 0x7fb1e00b8a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec031440 tx=0x7fb1ec009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:14.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.705+0000 7fb1f3e00640 1 -- 172.21.15.27:0/512159694 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb1e40b2730 con 0x7fb1fc10b6c0 2024-09-21T00:19:14.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.922+0000 7fb201600640 1 -- 172.21.15.27:0/512159694 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb1fc19d850 con 0x7fb1fc10b6c0 2024-09-21T00:19:14.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.922+0000 7fb1f3e00640 1 -- 172.21.15.27:0/512159694 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v86) ==== 81+0+881 (secure 0 0 0) 0x7fb1fc19d850 con 0x7fb1fc10b6c0 2024-09-21T00:19:14.924 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/502883064 2024-09-22T00:11:24.231101+0000 2024-09-21T00:19:14.924 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/831048090 2024-09-22T00:11:24.231101+0000 2024-09-21T00:19:14.924 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3880378905 2024-09-22T00:09:46.276169+0000 2024-09-21T00:19:14.924 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/793991452 2024-09-22T00:09:46.276169+0000 2024-09-21T00:19:14.924 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2077956479 2024-09-22T00:09:46.276169+0000 2024-09-21T00:19:14.924 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/3752372220 2024-09-22T00:09:46.276169+0000 2024-09-21T00:19:14.924 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/3752372220 2024-09-22T00:09:46.276169+0000 2024-09-21T00:19:14.924 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1098432608 2024-09-22T00:10:16.413167+0000 2024-09-21T00:19:14.924 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/2344239267 2024-09-22T00:10:16.413167+0000 2024-09-21T00:19:14.924 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/2344239267 2024-09-22T00:10:16.413167+0000 2024-09-21T00:19:14.924 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/528019907 2024-09-22T00:10:16.413167+0000 2024-09-21T00:19:14.924 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2173601508 2024-09-22T00:10:16.413167+0000 2024-09-21T00:19:14.924 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/827586431 2024-09-22T00:11:24.231101+0000 2024-09-21T00:19:14.924 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/827586431 2024-09-22T00:11:24.231101+0000 2024-09-21T00:19:14.925 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/764776233 2024-09-22T00:11:24.231101+0000 2024-09-21T00:19:14.925 INFO:teuthology.orchestra.run.smithi027.stderr:listed 15 entries 2024-09-21T00:19:14.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.925+0000 7fb201600640 1 -- 172.21.15.27:0/512159694 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb1e00b6560 msgr2=0x7fb1e00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:14.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.925+0000 7fb201600640 1 --2- 172.21.15.27:0/512159694 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb1e00b6560 0x7fb1e00b8a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec031440 tx=0x7fb1ec009560 comp rx=0 tx=0).stop 2024-09-21T00:19:14.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.925+0000 7fb201600640 1 -- 172.21.15.27:0/512159694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc10b6c0 msgr2=0x7fb1fc19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:14.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.925+0000 7fb201600640 1 --2- 172.21.15.27:0/512159694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc10b6c0 0x7fb1fc19ccc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400e9f0 tx=0x7fb1e400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:19:14.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.925+0000 7fb201600640 1 -- 172.21.15.27:0/512159694 shutdown_connections 2024-09-21T00:19:14.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.925+0000 7fb201600640 1 --2- 172.21.15.27:0/512159694 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb1e00b6560 0x7fb1e00b8a70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:14.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.925+0000 7fb201600640 1 --2- 172.21.15.27:0/512159694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1fc10c010 0x7fb1fc1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:14.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.925+0000 7fb201600640 1 --2- 172.21.15.27:0/512159694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc10b6c0 0x7fb1fc19ccc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:14.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.925+0000 7fb201600640 1 --2- 172.21.15.27:0/512159694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1fc105d80 0x7fb1fc19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:14.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.925+0000 7fb201600640 1 -- 172.21.15.27:0/512159694 >> 172.21.15.27:0/512159694 conn(0x7fb1fc101810 msgr2=0x7fb1fc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:14.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.925+0000 7fb201600640 1 -- 172.21.15.27:0/512159694 shutdown_connections 2024-09-21T00:19:14.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:14.925+0000 7fb201600640 1 -- 172.21.15.27:0/512159694 wait complete. 2024-09-21T00:19:15.823 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:19:15.823 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:19:15.861 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T00:19:16.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:15 smithi027 ceph-mon[30745]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:16.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:15 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/512159694' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T00:19:16.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:15 smithi089 ceph-mon[34481]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:16.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:15 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/512159694' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T00:19:16.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:15 smithi098 ceph-mon[34725]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:16.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:15 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/512159694' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T00:19:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.913+0000 7ff190200640 1 -- 172.21.15.27:0/3438547971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff18810b6d0 msgr2=0x7ff18810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.913+0000 7ff190200640 1 --2- 172.21.15.27:0/3438547971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff18810b6d0 0x7ff18810bae0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff174009900 tx=0x7ff17402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:19:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.913+0000 7ff190200640 1 -- 172.21.15.27:0/3438547971 shutdown_connections 2024-09-21T00:19:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.913+0000 7ff190200640 1 --2- 172.21.15.27:0/3438547971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff18810c020 0x7ff188112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.913+0000 7ff190200640 1 --2- 172.21.15.27:0/3438547971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff18810b6d0 0x7ff18810bae0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.913+0000 7ff190200640 1 --2- 172.21.15.27:0/3438547971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff188105d90 0x7ff1881061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.913+0000 7ff190200640 1 -- 172.21.15.27:0/3438547971 >> 172.21.15.27:0/3438547971 conn(0x7ff188101860 msgr2=0x7ff188103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.913+0000 7ff190200640 1 -- 172.21.15.27:0/3438547971 shutdown_connections 2024-09-21T00:19:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.913+0000 7ff190200640 1 -- 172.21.15.27:0/3438547971 wait complete. 2024-09-21T00:19:16.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.915+0000 7ff190200640 1 Processor -- start 2024-09-21T00:19:16.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.915+0000 7ff190200640 1 -- start start 2024-09-21T00:19:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.915+0000 7ff190200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff188105d90 0x7ff18819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.915+0000 7ff190200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff18810b6d0 0x7ff18819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.915+0000 7ff190200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff18810c020 0x7ff1881a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.915+0000 7ff190200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff188115ce0 con 0x7ff18810b6d0 2024-09-21T00:19:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.915+0000 7ff190200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff188115b60 con 0x7ff188105d90 2024-09-21T00:19:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.915+0000 7ff190200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff188115e60 con 0x7ff18810c020 2024-09-21T00:19:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.916+0000 7ff18d800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff188105d90 0x7ff18819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.916+0000 7ff18e200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff18810c020 0x7ff1881a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.916+0000 7ff18ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff18810b6d0 0x7ff18819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.916+0000 7ff18d800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff188105d90 0x7ff18819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:51432/0 (socket says 172.21.15.27:51432) 2024-09-21T00:19:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.916+0000 7ff18d800640 1 -- 172.21.15.27:0/4149063531 learned_addr learned my addr 172.21.15.27:0/4149063531 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.916+0000 7ff18e200640 1 -- 172.21.15.27:0/4149063531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff188105d90 msgr2=0x7ff18819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.916+0000 7ff18e200640 1 --2- 172.21.15.27:0/4149063531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff188105d90 0x7ff18819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.916+0000 7ff18e200640 1 -- 172.21.15.27:0/4149063531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff18810b6d0 msgr2=0x7ff18819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.916+0000 7ff18e200640 1 --2- 172.21.15.27:0/4149063531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff18810b6d0 0x7ff18819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:16.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.916+0000 7ff18e200640 1 -- 172.21.15.27:0/4149063531 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1881a2660 con 0x7ff18810c020 2024-09-21T00:19:16.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.916+0000 7ff18ce00640 1 --2- 172.21.15.27:0/4149063531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff18810b6d0 0x7ff18819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:19:16.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.917+0000 7ff18e200640 1 --2- 172.21.15.27:0/4149063531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff18810c020 0x7ff1881a1f30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff18000da10 tx=0x7ff18000ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:16.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.917+0000 7ff186000640 1 -- 172.21.15.27:0/4149063531 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18000bd70 con 0x7ff18810c020 2024-09-21T00:19:16.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.917+0000 7ff190200640 1 -- 172.21.15.27:0/4149063531 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1881a2950 con 0x7ff18810c020 2024-09-21T00:19:16.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.917+0000 7ff186000640 1 -- 172.21.15.27:0/4149063531 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff180003e20 con 0x7ff18810c020 2024-09-21T00:19:16.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.917+0000 7ff186000640 1 -- 172.21.15.27:0/4149063531 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1800106c0 con 0x7ff18810c020 2024-09-21T00:19:16.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.917+0000 7ff190200640 1 -- 172.21.15.27:0/4149063531 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1881a90b0 con 0x7ff18810c020 2024-09-21T00:19:16.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.918+0000 7ff190200640 1 -- 172.21.15.27:0/4149063531 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1881061a0 con 0x7ff18810c020 2024-09-21T00:19:16.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.921+0000 7ff186000640 1 -- 172.21.15.27:0/4149063531 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7ff180004250 con 0x7ff18810c020 2024-09-21T00:19:16.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.922+0000 7ff186000640 1 --2- 172.21.15.27:0/4149063531 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff1700b65e0 0x7ff1700b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:16.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.922+0000 7ff186000640 1 -- 172.21.15.27:0/4149063531 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7ff1800e7350 con 0x7ff18810c020 2024-09-21T00:19:16.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.922+0000 7ff18d800640 1 --2- 172.21.15.27:0/4149063531 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff1700b65e0 0x7ff1700b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:16.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.923+0000 7ff18d800640 1 --2- 172.21.15.27:0/4149063531 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff1700b65e0 0x7ff1700b8af0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff17c005fd0 tx=0x7ff17c00b480 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:16.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:16.926+0000 7ff186000640 1 -- 172.21.15.27:0/4149063531 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff180021020 con 0x7ff18810c020 2024-09-21T00:19:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:17.143+0000 7ff190200640 1 -- 172.21.15.27:0/4149063531 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff18802e640 con 0x7ff18810c020 2024-09-21T00:19:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:17.144+0000 7ff186000640 1 -- 172.21.15.27:0/4149063531 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v86) ==== 81+0+881 (secure 0 0 0) 0x7ff1800ad870 con 0x7ff18810c020 2024-09-21T00:19:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:listed 15 entries 2024-09-21T00:19:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/502883064 2024-09-22T00:11:24.231101+0000 2024-09-21T00:19:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/831048090 2024-09-22T00:11:24.231101+0000 2024-09-21T00:19:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3880378905 2024-09-22T00:09:46.276169+0000 2024-09-21T00:19:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/793991452 2024-09-22T00:09:46.276169+0000 2024-09-21T00:19:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2077956479 2024-09-22T00:09:46.276169+0000 2024-09-21T00:19:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/3752372220 2024-09-22T00:09:46.276169+0000 2024-09-21T00:19:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/3752372220 2024-09-22T00:09:46.276169+0000 2024-09-21T00:19:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1098432608 2024-09-22T00:10:16.413167+0000 2024-09-21T00:19:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/2344239267 2024-09-22T00:10:16.413167+0000 2024-09-21T00:19:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/2344239267 2024-09-22T00:10:16.413167+0000 2024-09-21T00:19:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/528019907 2024-09-22T00:10:16.413167+0000 2024-09-21T00:19:17.147 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2173601508 2024-09-22T00:10:16.413167+0000 2024-09-21T00:19:17.147 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/827586431 2024-09-22T00:11:24.231101+0000 2024-09-21T00:19:17.147 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/827586431 2024-09-22T00:11:24.231101+0000 2024-09-21T00:19:17.147 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/764776233 2024-09-22T00:11:24.231101+0000 2024-09-21T00:19:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:17.146+0000 7ff190200640 1 -- 172.21.15.27:0/4149063531 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff1700b65e0 msgr2=0x7ff1700b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:17.146+0000 7ff190200640 1 --2- 172.21.15.27:0/4149063531 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff1700b65e0 0x7ff1700b8af0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff17c005fd0 tx=0x7ff17c00b480 comp rx=0 tx=0).stop 2024-09-21T00:19:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:17.146+0000 7ff190200640 1 -- 172.21.15.27:0/4149063531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff18810c020 msgr2=0x7ff1881a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:17.146+0000 7ff190200640 1 --2- 172.21.15.27:0/4149063531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff18810c020 0x7ff1881a1f30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff18000da10 tx=0x7ff18000ded0 comp rx=0 tx=0).stop 2024-09-21T00:19:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:17.147+0000 7ff190200640 1 -- 172.21.15.27:0/4149063531 shutdown_connections 2024-09-21T00:19:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:17.147+0000 7ff190200640 1 --2- 172.21.15.27:0/4149063531 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff1700b65e0 0x7ff1700b8af0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:17.147+0000 7ff190200640 1 --2- 172.21.15.27:0/4149063531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff18810c020 0x7ff1881a1f30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:17.147+0000 7ff190200640 1 --2- 172.21.15.27:0/4149063531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff18810b6d0 0x7ff18819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:17.147+0000 7ff190200640 1 --2- 172.21.15.27:0/4149063531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff188105d90 0x7ff18819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:17.147+0000 7ff190200640 1 -- 172.21.15.27:0/4149063531 >> 172.21.15.27:0/4149063531 conn(0x7ff188101860 msgr2=0x7ff188103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:17.147+0000 7ff190200640 1 -- 172.21.15.27:0/4149063531 shutdown_connections 2024-09-21T00:19:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:17.147+0000 7ff190200640 1 -- 172.21.15.27:0/4149063531 wait complete. 2024-09-21T00:19:17.908 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2024-09-21T00:19:17.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:19:17.908 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:19:17.965 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[73795]: fuse finished with error 0 and tester_r 0 2024-09-21T00:19:17.969 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:19:18.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:17 smithi027 ceph-mon[30745]: pgmap v310: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T00:19:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:17 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/4149063531' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T00:19:18.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:17 smithi089 ceph-mon[34481]: pgmap v310: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T00:19:18.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:17 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/4149063531' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T00:19:18.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:17 smithi098 ceph-mon[34725]: pgmap v310: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T00:19:18.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:17 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/4149063531' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T00:19:20.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:19 smithi027 ceph-mon[30745]: pgmap v311: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T00:19:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:19 smithi089 ceph-mon[34481]: pgmap v311: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T00:19:20.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:19 smithi098 ceph-mon[34725]: pgmap v311: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T00:19:22.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:21 smithi027 ceph-mon[30745]: pgmap v312: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T00:19:22.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:21 smithi089 ceph-mon[34481]: pgmap v312: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T00:19:22.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:21 smithi098 ceph-mon[34725]: pgmap v312: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T00:19:24.071 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T00:19:24.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:19:24.071 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-21T00:19:24.117 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T00:19:24.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:23 smithi027 ceph-mon[30745]: pgmap v313: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T00:19:24.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:23 smithi089 ceph-mon[34481]: pgmap v313: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T00:19:24.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:23 smithi098 ceph-mon[34725]: pgmap v313: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T00:19:24.552 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:25.012 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-21T00:19:25.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-21T00:19:25.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-09-21T00:19:25.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-21T00:19:25.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-09-21T00:19:25.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]} : dispatch 2024-09-21T00:19:25.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]} : dispatch 2024-09-21T00:19:25.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]} : dispatch 2024-09-21T00:19:25.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-21T00:19:25.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-09-21T00:19:25.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-21T00:19:25.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-09-21T00:19:25.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-09-21T00:19:25.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-09-21T00:19:25.014 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]} : dispatch 2024-09-21T00:19:25.014 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]} : dispatch 2024-09-21T00:19:25.014 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]} : dispatch 2024-09-21T00:19:25.014 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-21T00:19:25.014 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-09-21T00:19:25.014 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:24 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-09-21T00:19:25.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.195+0000 7f92ed400640 1 -- 172.21.15.27:0/3083413351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92e8078290 msgr2=0x7f92e8114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:25.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.195+0000 7f92ed400640 1 --2- 172.21.15.27:0/3083413351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92e8078290 0x7f92e8114200 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009900 tx=0x7f92d402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:19:25.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.195+0000 7f92ed400640 1 -- 172.21.15.27:0/3083413351 shutdown_connections 2024-09-21T00:19:25.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.195+0000 7f92ed400640 1 --2- 172.21.15.27:0/3083413351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92e8114740 0x7f92e8116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:25.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.195+0000 7f92ed400640 1 --2- 172.21.15.27:0/3083413351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92e8078290 0x7f92e8114200 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:25.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.195+0000 7f92ed400640 1 --2- 172.21.15.27:0/3083413351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92e8077940 0x7f92e8077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:25.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.195+0000 7f92ed400640 1 -- 172.21.15.27:0/3083413351 >> 172.21.15.27:0/3083413351 conn(0x7f92e8101810 msgr2=0x7f92e8103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:25.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.195+0000 7f92ed400640 1 -- 172.21.15.27:0/3083413351 shutdown_connections 2024-09-21T00:19:25.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.195+0000 7f92ed400640 1 -- 172.21.15.27:0/3083413351 wait complete. 2024-09-21T00:19:25.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.197+0000 7f92ed400640 1 Processor -- start 2024-09-21T00:19:25.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.197+0000 7f92ed400640 1 -- start start 2024-09-21T00:19:25.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.197+0000 7f92ed400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92e8077940 0x7f92e81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:25.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.197+0000 7f92ed400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92e8078290 0x7f92e81a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:25.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.197+0000 7f92ed400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92e8114740 0x7f92e81a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:25.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.197+0000 7f92ed400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f92e811a070 con 0x7f92e8114740 2024-09-21T00:19:25.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.197+0000 7f92ed400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f92e8119ef0 con 0x7f92e8078290 2024-09-21T00:19:25.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.197+0000 7f92ed400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f92e811a1f0 con 0x7f92e8077940 2024-09-21T00:19:25.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.198+0000 7f92e7200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92e8114740 0x7f92e81a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:25.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.198+0000 7f92e7200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92e8114740 0x7f92e81a63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57618/0 (socket says 172.21.15.27:57618) 2024-09-21T00:19:25.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.198+0000 7f92e7200640 1 -- 172.21.15.27:0/2215368699 learned_addr learned my addr 172.21.15.27:0/2215368699 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:25.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.198+0000 7f92e5e00640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92e8078290 0x7f92e81a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:25.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.198+0000 7f92e6800640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92e8077940 0x7f92e81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:25.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.198+0000 7f92e7200640 1 -- 172.21.15.27:0/2215368699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92e8077940 msgr2=0x7f92e81a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:25.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.198+0000 7f92e7200640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92e8077940 0x7f92e81a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:25.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.198+0000 7f92e7200640 1 -- 172.21.15.27:0/2215368699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92e8078290 msgr2=0x7f92e81a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:25.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.198+0000 7f92e7200640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92e8078290 0x7f92e81a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:25.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.198+0000 7f92e7200640 1 -- 172.21.15.27:0/2215368699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e81a6af0 con 0x7f92e8114740 2024-09-21T00:19:25.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.198+0000 7f92e5e00640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92e8078290 0x7f92e81a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:19:25.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.198+0000 7f92e6800640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92e8077940 0x7f92e81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:19:25.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.199+0000 7f92e7200640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92e8114740 0x7f92e81a63c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f92e000efd0 tx=0x7f92e000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:25.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.199+0000 7f92df400640 1 -- 172.21.15.27:0/2215368699 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e0003b70 con 0x7f92e8114740 2024-09-21T00:19:25.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.199+0000 7f92df400640 1 -- 172.21.15.27:0/2215368699 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e0003d10 con 0x7f92e8114740 2024-09-21T00:19:25.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.199+0000 7f92ed400640 1 -- 172.21.15.27:0/2215368699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e81a6de0 con 0x7f92e8114740 2024-09-21T00:19:25.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.199+0000 7f92df400640 1 -- 172.21.15.27:0/2215368699 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e00108a0 con 0x7f92e8114740 2024-09-21T00:19:25.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.199+0000 7f92ed400640 1 -- 172.21.15.27:0/2215368699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e81ad540 con 0x7f92e8114740 2024-09-21T00:19:25.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.200+0000 7f92ed400640 1 -- 172.21.15.27:0/2215368699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e8077d50 con 0x7f92e8114740 2024-09-21T00:19:25.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.203+0000 7f92df400640 1 -- 172.21.15.27:0/2215368699 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f92e0004b40 con 0x7f92e8114740 2024-09-21T00:19:25.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.204+0000 7f92df400640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f92cc0b65e0 0x7f92cc0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:25.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.204+0000 7f92df400640 1 -- 172.21.15.27:0/2215368699 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9307+0+0 (secure 0 0 0) 0x7f92e00e7f20 con 0x7f92e8114740 2024-09-21T00:19:25.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.205+0000 7f92e6800640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f92cc0b65e0 0x7f92cc0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:25.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.205+0000 7f92e6800640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f92cc0b65e0 0x7f92cc0b8af0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009fc0 tx=0x7f92d8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:25.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.206+0000 7f92df400640 1 -- 172.21.15.27:0/2215368699 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f92e00ae320 con 0x7f92e8114740 2024-09-21T00:19:25.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-21T00:19:25.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-21T00:19:25.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-09-21T00:19:25.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-21T00:19:25.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-09-21T00:19:25.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]} : dispatch 2024-09-21T00:19:25.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]} : dispatch 2024-09-21T00:19:25.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]} : dispatch 2024-09-21T00:19:25.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-21T00:19:25.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-09-21T00:19:25.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-21T00:19:25.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-09-21T00:19:25.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-09-21T00:19:25.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-09-21T00:19:25.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]} : dispatch 2024-09-21T00:19:25.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]} : dispatch 2024-09-21T00:19:25.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]} : dispatch 2024-09-21T00:19:25.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-21T00:19:25.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-09-21T00:19:25.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:24 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-09-21T00:19:25.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-21T00:19:25.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-21T00:19:25.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-09-21T00:19:25.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-21T00:19:25.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-09-21T00:19:25.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]} : dispatch 2024-09-21T00:19:25.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]} : dispatch 2024-09-21T00:19:25.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]} : dispatch 2024-09-21T00:19:25.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-21T00:19:25.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-09-21T00:19:25.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-21T00:19:25.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-09-21T00:19:25.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-09-21T00:19:25.248 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-09-21T00:19:25.248 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]} : dispatch 2024-09-21T00:19:25.248 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]} : dispatch 2024-09-21T00:19:25.248 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]} : dispatch 2024-09-21T00:19:25.248 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-21T00:19:25.248 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-09-21T00:19:25.248 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:24 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-09-21T00:19:25.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.406+0000 7f92ed400640 1 -- 172.21.15.27:0/2215368699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92e81a1cb0 con 0x7f92e8114740 2024-09-21T00:19:25.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.407+0000 7f92df400640 1 -- 172.21.15.27:0/2215368699 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24953 (secure 0 0 0) 0x7f92e81a1cb0 con 0x7f92e8114740 2024-09-21T00:19:25.410 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:19:25.410 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":87,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","created":"2024-09-21T00:09:20.149938+0000","modified":"2024-09-21T00:19:24.817877+0000","last_up_change":"2024-09-21T00:17:43.978785+0000","last_in_change":"2024-09-21T00:17:21.325177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:13:32.718543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:18:33.992414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:18:36.817691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:18:46.707770+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63ccc094-62d5-446f-844d-1c60d41fda1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6803","nonce":354370090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6805","nonce":354370090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6809","nonce":354370090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6807","nonce":354370090}]},"public_addr":"172.21.15.27:6803/354370090","cluster_addr":"172.21.15.27:6805/354370090","heartbeat_back_addr":"172.21.15.27:6809/354370090","heartbeat_front_addr":"172.21.15.27:6807/354370090","state":["exists","up"]},{"osd":1,"uuid":"e59a8ed4-3a37-4477-a60a-4abc33d49b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6801","nonce":922528621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6803","nonce":922528621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6807","nonce":922528621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6805","nonce":922528621}]},"public_addr":"172.21.15.89:6801/922528621","cluster_addr":"172.21.15.89:6803/922528621","heartbeat_back_addr":"172.21.15.89:6807/922528621","heartbeat_front_addr":"172.21.15.89:6805/922528621","state":["exists","up"]},{"osd":2,"uuid":"6058a875-d5b2-4459-9407-2fd190cb34a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6801","nonce":257636259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6803","nonce":257636259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6807","nonce":257636259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6805","nonce":257636259}]},"public_addr":"172.21.15.98:6801/257636259","cluster_addr":"172.21.15.98:6803/257636259","heartbeat_back_addr":"172.21.15.98:6807/257636259","heartbeat_front_addr":"172.21.15.98:6805/257636259","state":["exists","up"]},{"osd":3,"uuid":"dfcdaede-e51c-4985-92af-3e296435a36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6811","nonce":821088824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6813","nonce":821088824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6817","nonce":821088824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6815","nonce":821088824}]},"public_addr":"172.21.15.27:6811/821088824","cluster_addr":"172.21.15.27:6813/821088824","heartbeat_back_addr":"172.21.15.27:6817/821088824","heartbeat_front_addr":"172.21.15.27:6815/821088824","state":["exists","up"]},{"osd":4,"uuid":"7c4af037-8ab6-47c0-b62b-76bd775b3491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6809","nonce":4001038690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6811","nonce":4001038690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6815","nonce":4001038690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6813","nonce":4001038690}]},"public_addr":"172.21.15.89:6809/4001038690","cluster_addr":"172.21.15.89:6811/4001038690","heartbeat_back_addr":"172.21.15.89:6815/4001038690","heartbeat_front_addr":"172.21.15.89:6813/4001038690","state":["exists","up"]},{"osd":5,"uuid":"e996ed0d-88ad-4aa7-891f-dfade4e09cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6809","nonce":3029903562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6811","nonce":3029903562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6815","nonce":3029903562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6813","nonce":3029903562}]},"public_addr":"172.21.15.98:6809/3029903562","cluster_addr":"172.21.15.98:6811/3029903562","heartbeat_back_addr":"172.21.15.98:6815/3029903562","heartbeat_front_addr":"172.21.15.98:6813/3029903562","state":["exists","up"]},{"osd":6,"uuid":"cf080497-57c2-42d0-8c8f-1eed0110702d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6819","nonce":1286808857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6821","nonce":1286808857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6825","nonce":1286808857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6823","nonce":1286808857}]},"public_addr":"172.21.15.27:6819/1286808857","cluster_addr":"172.21.15.27:6821/1286808857","heartbeat_back_addr":"172.21.15.27:6825/1286808857","heartbeat_front_addr":"172.21.15.27:6823/1286808857","state":["exists","up"]},{"osd":7,"uuid":"ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6817","nonce":2479349546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6819","nonce":2479349546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6823","nonce":2479349546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6821","nonce":2479349546}]},"public_addr":"172.21.15.89:6817/2479349546","cluster_addr":"172.21.15.89:6819/2479349546","heartbeat_back_addr":"172.21.15.89:6823/2479349546","heartbeat_front_addr":"172.21.15.89:6821/2479349546","state":["exists","up"]},{"osd":8,"uuid":"0b558458-fc2d-4ed1-b72a-e2ea8cf811b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6817","nonce":2830381485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6819","nonce":2830381485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6823","nonce":2830381485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6821","nonce":2830381485}]},"public_addr":"172.21.15.98:6817/2830381485","cluster_addr":"172.21.15.98:6819/2830381485","heartbeat_back_addr":"172.21.15.98:6823/2830381485","heartbeat_front_addr":"172.21.15.98:6821/2830381485","state":["exists","up"]},{"osd":9,"uuid":"1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6827","nonce":3033977113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6829","nonce":3033977113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6833","nonce":3033977113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6831","nonce":3033977113}]},"public_addr":"172.21.15.27:6827/3033977113","cluster_addr":"172.21.15.27:6829/3033977113","heartbeat_back_addr":"172.21.15.27:6833/3033977113","heartbeat_front_addr":"172.21.15.27:6831/3033977113","state":["exists","up"]},{"osd":10,"uuid":"324dd6ce-68fe-4c76-959b-1c9fb0ae92a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6825","nonce":2370332225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6827","nonce":2370332225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6831","nonce":2370332225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6829","nonce":2370332225}]},"public_addr":"172.21.15.89:6825/2370332225","cluster_addr":"172.21.15.89:6827/2370332225","heartbeat_back_addr":"172.21.15.89:6831/2370332225","heartbeat_front_addr":"172.21.15.89:6829/2370332225","state":["exists","up"]},{"osd":11,"uuid":"5e34686e-350a-491e-b9d7-59174ea90944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6825","nonce":2795999064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6827","nonce":2795999064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6831","nonce":2795999064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6829","nonce":2795999064}]},"public_addr":"172.21.15.98:6825/2795999064","cluster_addr":"172.21.15.98:6827/2795999064","heartbeat_back_addr":"172.21.15.98:6831/2795999064","heartbeat_front_addr":"172.21.15.98:6829/2795999064","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:12:39.841958+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:06.684417+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:31.041378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:01.612108+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:28.592326+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:54.303714+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:25.523475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:52.523483+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:19.430503+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:50.599120+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:17.520452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:42.725550+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/502883064":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/831048090":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/3880378905":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/793991452":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/2077956479":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6800/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6801/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/1098432608":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6800/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/528019907":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/2173601508":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:6800/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/764776233":"2024-09-22T00:11:24.231101+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:19:25.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.410+0000 7f92ed400640 1 -- 172.21.15.27:0/2215368699 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f92cc0b65e0 msgr2=0x7f92cc0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:25.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.410+0000 7f92ed400640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f92cc0b65e0 0x7f92cc0b8af0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009fc0 tx=0x7f92d8005f90 comp rx=0 tx=0).stop 2024-09-21T00:19:25.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.410+0000 7f92ed400640 1 -- 172.21.15.27:0/2215368699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92e8114740 msgr2=0x7f92e81a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:25.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.410+0000 7f92ed400640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92e8114740 0x7f92e81a63c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f92e000efd0 tx=0x7f92e000c6a0 comp rx=0 tx=0).stop 2024-09-21T00:19:25.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.410+0000 7f92ed400640 1 -- 172.21.15.27:0/2215368699 shutdown_connections 2024-09-21T00:19:25.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.410+0000 7f92ed400640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f92cc0b65e0 0x7f92cc0b8af0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:25.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.410+0000 7f92ed400640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92e8114740 0x7f92e81a63c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.410+0000 7f92ed400640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92e8078290 0x7f92e81a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.410+0000 7f92ed400640 1 --2- 172.21.15.27:0/2215368699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92e8077940 0x7f92e81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.410+0000 7f92ed400640 1 -- 172.21.15.27:0/2215368699 >> 172.21.15.27:0/2215368699 conn(0x7f92e8101810 msgr2=0x7f92e8116840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.410+0000 7f92ed400640 1 -- 172.21.15.27:0/2215368699 shutdown_connections 2024-09-21T00:19:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:25.410+0000 7f92ed400640 1 -- 172.21.15.27:0/2215368699 wait complete. 2024-09-21T00:19:26.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:25 smithi089 ceph-mon[34481]: pgmap v314: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T00:19:26.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-21T00:19:26.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-21T00:19:26.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-21T00:19:26.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-21T00:19:26.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-21T00:19:26.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-21T00:19:26.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-21T00:19:26.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-21T00:19:26.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-21T00:19:26.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-21T00:19:26.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:25 smithi089 ceph-mon[34481]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T00:19:26.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:25 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2215368699' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:19:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:25 smithi098 ceph-mon[34725]: pgmap v314: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T00:19:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-21T00:19:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-21T00:19:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-21T00:19:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-21T00:19:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-21T00:19:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-21T00:19:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-21T00:19:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-21T00:19:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-21T00:19:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-21T00:19:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:25 smithi098 ceph-mon[34725]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T00:19:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:25 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2215368699' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:19:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:25 smithi027 ceph-mon[30745]: pgmap v314: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T00:19:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-21T00:19:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-21T00:19:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-21T00:19:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-21T00:19:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-21T00:19:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-21T00:19:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-21T00:19:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-21T00:19:26.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-21T00:19:26.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-21T00:19:26.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:25 smithi027 ceph-mon[30745]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T00:19:26.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:25 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2215368699' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:19:26.378 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T00:19:26.769 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:27.196 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:26 smithi027 ceph-mon[30745]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T00:19:27.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:26 smithi089 ceph-mon[34481]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T00:19:27.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:26 smithi098 ceph-mon[34725]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T00:19:27.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.378+0000 7fd708a00640 1 -- 172.21.15.27:0/2147622154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7041039b0 msgr2=0x7fd704103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:27.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.378+0000 7fd708a00640 1 --2- 172.21.15.27:0/2147622154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7041039b0 0x7fd704103dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009900 tx=0x7fd6f402a410 comp rx=0 tx=0).stop 2024-09-21T00:19:27.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.378+0000 7fd708a00640 1 -- 172.21.15.27:0/2147622154 shutdown_connections 2024-09-21T00:19:27.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.378+0000 7fd708a00640 1 --2- 172.21.15.27:0/2147622154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd704109c40 0x7fd704110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:27.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.378+0000 7fd708a00640 1 --2- 172.21.15.27:0/2147622154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7041092f0 0x7fd704109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:27.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.378+0000 7fd708a00640 1 --2- 172.21.15.27:0/2147622154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7041039b0 0x7fd704103dc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:27.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.379+0000 7fd708a00640 1 -- 172.21.15.27:0/2147622154 >> 172.21.15.27:0/2147622154 conn(0x7fd7040ff440 msgr2=0x7fd704101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:27.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.379+0000 7fd708a00640 1 -- 172.21.15.27:0/2147622154 shutdown_connections 2024-09-21T00:19:27.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.379+0000 7fd708a00640 1 -- 172.21.15.27:0/2147622154 wait complete. 2024-09-21T00:19:27.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.380+0000 7fd708a00640 1 Processor -- start 2024-09-21T00:19:27.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.380+0000 7fd708a00640 1 -- start start 2024-09-21T00:19:27.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.380+0000 7fd708a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7041039b0 0x7fd70419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:27.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.380+0000 7fd708a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7041092f0 0x7fd70419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:27.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.381+0000 7fd708a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd704109c40 0x7fd70419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:27.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.381+0000 7fd708a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd704113950 con 0x7fd7041039b0 2024-09-21T00:19:27.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.381+0000 7fd708a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd7041137d0 con 0x7fd7041092f0 2024-09-21T00:19:27.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.381+0000 7fd708a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd704113ad0 con 0x7fd704109c40 2024-09-21T00:19:27.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.381+0000 7fd702800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7041039b0 0x7fd70419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:27.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.381+0000 7fd702800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7041039b0 0x7fd70419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57638/0 (socket says 172.21.15.27:57638) 2024-09-21T00:19:27.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.381+0000 7fd702800640 1 -- 172.21.15.27:0/772117656 learned_addr learned my addr 172.21.15.27:0/772117656 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:27.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.381+0000 7fd6fbe00640 1 --2- 172.21.15.27:0/772117656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7041092f0 0x7fd70419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:27.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.381+0000 7fd703200640 1 --2- 172.21.15.27:0/772117656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd704109c40 0x7fd70419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:27.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.381+0000 7fd6fbe00640 1 -- 172.21.15.27:0/772117656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd704109c40 msgr2=0x7fd70419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:27.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.381+0000 7fd6fbe00640 1 --2- 172.21.15.27:0/772117656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd704109c40 0x7fd70419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:27.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.381+0000 7fd6fbe00640 1 -- 172.21.15.27:0/772117656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7041039b0 msgr2=0x7fd70419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:27.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.381+0000 7fd6fbe00640 1 --2- 172.21.15.27:0/772117656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7041039b0 0x7fd70419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:27.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.381+0000 7fd6fbe00640 1 -- 172.21.15.27:0/772117656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7041a0330 con 0x7fd7041092f0 2024-09-21T00:19:27.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.381+0000 7fd702800640 1 --2- 172.21.15.27:0/772117656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7041039b0 0x7fd70419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:19:27.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.382+0000 7fd6fbe00640 1 --2- 172.21.15.27:0/772117656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7041092f0 0x7fd70419a960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd70406b990 tx=0x7fd6ec00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:27.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.382+0000 7fd6fb400640 1 -- 172.21.15.27:0/772117656 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00cc40 con 0x7fd7041092f0 2024-09-21T00:19:27.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.382+0000 7fd708a00640 1 -- 172.21.15.27:0/772117656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7041a0620 con 0x7fd7041092f0 2024-09-21T00:19:27.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.382+0000 7fd6fb400640 1 -- 172.21.15.27:0/772117656 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6ec00cde0 con 0x7fd7041092f0 2024-09-21T00:19:27.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.382+0000 7fd6fb400640 1 -- 172.21.15.27:0/772117656 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec004a00 con 0x7fd7041092f0 2024-09-21T00:19:27.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.382+0000 7fd708a00640 1 -- 172.21.15.27:0/772117656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7041a6d80 con 0x7fd7041092f0 2024-09-21T00:19:27.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.383+0000 7fd708a00640 1 -- 172.21.15.27:0/772117656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd704103dc0 con 0x7fd7041092f0 2024-09-21T00:19:27.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.386+0000 7fd6fb400640 1 -- 172.21.15.27:0/772117656 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fd6ec004ba0 con 0x7fd7041092f0 2024-09-21T00:19:27.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.387+0000 7fd6fb400640 1 --2- 172.21.15.27:0/772117656 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd6e80b6560 0x7fd6e80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:27.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.388+0000 7fd6fb400640 1 -- 172.21.15.27:0/772117656 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7fd6ec0e77f0 con 0x7fd7041092f0 2024-09-21T00:19:27.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.388+0000 7fd702800640 1 --2- 172.21.15.27:0/772117656 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd6e80b6560 0x7fd6e80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:27.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.388+0000 7fd702800640 1 --2- 172.21.15.27:0/772117656 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd6e80b6560 0x7fd6e80b8a70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40096c0 tx=0x7fd6f4005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:27.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.389+0000 7fd6fb400640 1 -- 172.21.15.27:0/772117656 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd6ec0adc80 con 0x7fd7041092f0 2024-09-21T00:19:27.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.613+0000 7fd708a00640 1 -- 172.21.15.27:0/772117656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd70419b4f0 con 0x7fd7041092f0 2024-09-21T00:19:27.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.614+0000 7fd6fb400640 1 -- 172.21.15.27:0/772117656 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24953 (secure 0 0 0) 0x7fd70419b4f0 con 0x7fd7041092f0 2024-09-21T00:19:27.617 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:19:27.618 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":88,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","created":"2024-09-21T00:09:20.149938+0000","modified":"2024-09-21T00:19:25.824021+0000","last_up_change":"2024-09-21T00:17:43.978785+0000","last_in_change":"2024-09-21T00:17:21.325177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:13:32.718543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:18:33.992414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:18:36.817691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:18:46.707770+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63ccc094-62d5-446f-844d-1c60d41fda1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6803","nonce":354370090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6805","nonce":354370090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6809","nonce":354370090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6807","nonce":354370090}]},"public_addr":"172.21.15.27:6803/354370090","cluster_addr":"172.21.15.27:6805/354370090","heartbeat_back_addr":"172.21.15.27:6809/354370090","heartbeat_front_addr":"172.21.15.27:6807/354370090","state":["exists","up"]},{"osd":1,"uuid":"e59a8ed4-3a37-4477-a60a-4abc33d49b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6801","nonce":922528621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6803","nonce":922528621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6807","nonce":922528621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6805","nonce":922528621}]},"public_addr":"172.21.15.89:6801/922528621","cluster_addr":"172.21.15.89:6803/922528621","heartbeat_back_addr":"172.21.15.89:6807/922528621","heartbeat_front_addr":"172.21.15.89:6805/922528621","state":["exists","up"]},{"osd":2,"uuid":"6058a875-d5b2-4459-9407-2fd190cb34a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6801","nonce":257636259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6803","nonce":257636259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6807","nonce":257636259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6805","nonce":257636259}]},"public_addr":"172.21.15.98:6801/257636259","cluster_addr":"172.21.15.98:6803/257636259","heartbeat_back_addr":"172.21.15.98:6807/257636259","heartbeat_front_addr":"172.21.15.98:6805/257636259","state":["exists","up"]},{"osd":3,"uuid":"dfcdaede-e51c-4985-92af-3e296435a36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6811","nonce":821088824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6813","nonce":821088824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6817","nonce":821088824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6815","nonce":821088824}]},"public_addr":"172.21.15.27:6811/821088824","cluster_addr":"172.21.15.27:6813/821088824","heartbeat_back_addr":"172.21.15.27:6817/821088824","heartbeat_front_addr":"172.21.15.27:6815/821088824","state":["exists","up"]},{"osd":4,"uuid":"7c4af037-8ab6-47c0-b62b-76bd775b3491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6809","nonce":4001038690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6811","nonce":4001038690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6815","nonce":4001038690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6813","nonce":4001038690}]},"public_addr":"172.21.15.89:6809/4001038690","cluster_addr":"172.21.15.89:6811/4001038690","heartbeat_back_addr":"172.21.15.89:6815/4001038690","heartbeat_front_addr":"172.21.15.89:6813/4001038690","state":["exists","up"]},{"osd":5,"uuid":"e996ed0d-88ad-4aa7-891f-dfade4e09cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6809","nonce":3029903562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6811","nonce":3029903562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6815","nonce":3029903562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6813","nonce":3029903562}]},"public_addr":"172.21.15.98:6809/3029903562","cluster_addr":"172.21.15.98:6811/3029903562","heartbeat_back_addr":"172.21.15.98:6815/3029903562","heartbeat_front_addr":"172.21.15.98:6813/3029903562","state":["exists","up"]},{"osd":6,"uuid":"cf080497-57c2-42d0-8c8f-1eed0110702d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6819","nonce":1286808857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6821","nonce":1286808857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6825","nonce":1286808857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6823","nonce":1286808857}]},"public_addr":"172.21.15.27:6819/1286808857","cluster_addr":"172.21.15.27:6821/1286808857","heartbeat_back_addr":"172.21.15.27:6825/1286808857","heartbeat_front_addr":"172.21.15.27:6823/1286808857","state":["exists","up"]},{"osd":7,"uuid":"ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6817","nonce":2479349546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6819","nonce":2479349546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6823","nonce":2479349546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6821","nonce":2479349546}]},"public_addr":"172.21.15.89:6817/2479349546","cluster_addr":"172.21.15.89:6819/2479349546","heartbeat_back_addr":"172.21.15.89:6823/2479349546","heartbeat_front_addr":"172.21.15.89:6821/2479349546","state":["exists","up"]},{"osd":8,"uuid":"0b558458-fc2d-4ed1-b72a-e2ea8cf811b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6817","nonce":2830381485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6819","nonce":2830381485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6823","nonce":2830381485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6821","nonce":2830381485}]},"public_addr":"172.21.15.98:6817/2830381485","cluster_addr":"172.21.15.98:6819/2830381485","heartbeat_back_addr":"172.21.15.98:6823/2830381485","heartbeat_front_addr":"172.21.15.98:6821/2830381485","state":["exists","up"]},{"osd":9,"uuid":"1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6827","nonce":3033977113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6829","nonce":3033977113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6833","nonce":3033977113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6831","nonce":3033977113}]},"public_addr":"172.21.15.27:6827/3033977113","cluster_addr":"172.21.15.27:6829/3033977113","heartbeat_back_addr":"172.21.15.27:6833/3033977113","heartbeat_front_addr":"172.21.15.27:6831/3033977113","state":["exists","up"]},{"osd":10,"uuid":"324dd6ce-68fe-4c76-959b-1c9fb0ae92a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6825","nonce":2370332225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6827","nonce":2370332225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6831","nonce":2370332225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6829","nonce":2370332225}]},"public_addr":"172.21.15.89:6825/2370332225","cluster_addr":"172.21.15.89:6827/2370332225","heartbeat_back_addr":"172.21.15.89:6831/2370332225","heartbeat_front_addr":"172.21.15.89:6829/2370332225","state":["exists","up"]},{"osd":11,"uuid":"5e34686e-350a-491e-b9d7-59174ea90944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6825","nonce":2795999064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6827","nonce":2795999064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6831","nonce":2795999064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6829","nonce":2795999064}]},"public_addr":"172.21.15.98:6825/2795999064","cluster_addr":"172.21.15.98:6827/2795999064","heartbeat_back_addr":"172.21.15.98:6831/2795999064","heartbeat_front_addr":"172.21.15.98:6829/2795999064","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:12:39.841958+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:06.684417+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:31.041378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:01.612108+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:28.592326+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:54.303714+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:25.523475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:52.523483+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:19.430503+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:50.599120+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:17.520452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:42.725550+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/502883064":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/831048090":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/3880378905":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/793991452":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/2077956479":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6800/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6801/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/1098432608":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6800/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/528019907":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/2173601508":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:6800/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/764776233":"2024-09-22T00:11:24.231101+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:19:27.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.617+0000 7fd708a00640 1 -- 172.21.15.27:0/772117656 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd6e80b6560 msgr2=0x7fd6e80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:27.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.617+0000 7fd708a00640 1 --2- 172.21.15.27:0/772117656 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd6e80b6560 0x7fd6e80b8a70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40096c0 tx=0x7fd6f4005a20 comp rx=0 tx=0).stop 2024-09-21T00:19:27.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.617+0000 7fd708a00640 1 -- 172.21.15.27:0/772117656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7041092f0 msgr2=0x7fd70419a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:27.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.617+0000 7fd708a00640 1 --2- 172.21.15.27:0/772117656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7041092f0 0x7fd70419a960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd70406b990 tx=0x7fd6ec00ee50 comp rx=0 tx=0).stop 2024-09-21T00:19:27.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.617+0000 7fd708a00640 1 -- 172.21.15.27:0/772117656 shutdown_connections 2024-09-21T00:19:27.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.617+0000 7fd708a00640 1 --2- 172.21.15.27:0/772117656 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd6e80b6560 0x7fd6e80b8a70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:27.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.617+0000 7fd708a00640 1 --2- 172.21.15.27:0/772117656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd704109c40 0x7fd70419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.617+0000 7fd708a00640 1 --2- 172.21.15.27:0/772117656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7041092f0 0x7fd70419a960 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.617+0000 7fd708a00640 1 --2- 172.21.15.27:0/772117656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7041039b0 0x7fd70419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.617+0000 7fd708a00640 1 -- 172.21.15.27:0/772117656 >> 172.21.15.27:0/772117656 conn(0x7fd7040ff440 msgr2=0x7fd704101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.617+0000 7fd708a00640 1 -- 172.21.15.27:0/772117656 shutdown_connections 2024-09-21T00:19:27.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:27.617+0000 7fd708a00640 1 -- 172.21.15.27:0/772117656 wait complete. 2024-09-21T00:19:28.019 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:27 smithi027 ceph-mon[30745]: pgmap v317: 129 pgs: 5 peering, 124 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-21T00:19:28.019 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:27 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/772117656' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:19:28.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:27 smithi089 ceph-mon[34481]: pgmap v317: 129 pgs: 5 peering, 124 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-21T00:19:28.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:27 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/772117656' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:19:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:27 smithi098 ceph-mon[34725]: pgmap v317: 129 pgs: 5 peering, 124 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-21T00:19:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:27 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/772117656' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:19:28.526 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T00:19:28.915 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.563+0000 7fc3b4200640 1 -- 172.21.15.27:0/266508158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac0757b0 msgr2=0x7fc3ac075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.563+0000 7fc3b4200640 1 --2- 172.21.15.27:0/266508158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac0757b0 0x7fc3ac075bc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009900 tx=0x7fc3a002a410 comp rx=0 tx=0).stop 2024-09-21T00:19:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.563+0000 7fc3b4200640 1 -- 172.21.15.27:0/266508158 shutdown_connections 2024-09-21T00:19:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.563+0000 7fc3b4200640 1 --2- 172.21.15.27:0/266508158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3ac1100d0 0x7fc3ac112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.563+0000 7fc3b4200640 1 --2- 172.21.15.27:0/266508158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ac076100 0x7fc3ac10fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.563+0000 7fc3b4200640 1 --2- 172.21.15.27:0/266508158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac0757b0 0x7fc3ac075bc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.563+0000 7fc3b4200640 1 -- 172.21.15.27:0/266508158 >> 172.21.15.27:0/266508158 conn(0x7fc3ac0ff5e0 msgr2=0x7fc3ac101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.564+0000 7fc3b4200640 1 -- 172.21.15.27:0/266508158 shutdown_connections 2024-09-21T00:19:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.564+0000 7fc3b4200640 1 -- 172.21.15.27:0/266508158 wait complete. 2024-09-21T00:19:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.565+0000 7fc3b4200640 1 Processor -- start 2024-09-21T00:19:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.565+0000 7fc3b4200640 1 -- start start 2024-09-21T00:19:29.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.566+0000 7fc3b4200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3ac0757b0 0x7fc3ac19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:29.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.566+0000 7fc3b4200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ac076100 0x7fc3ac19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:29.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.566+0000 7fc3b4200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac1100d0 0x7fc3ac1a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.566+0000 7fc3b4200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac115cd0 con 0x7fc3ac1100d0 2024-09-21T00:19:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.566+0000 7fc3b4200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac115b50 con 0x7fc3ac076100 2024-09-21T00:19:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.566+0000 7fc3b4200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac115e50 con 0x7fc3ac0757b0 2024-09-21T00:19:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.566+0000 7fc3b1800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3ac0757b0 0x7fc3ac19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.566+0000 7fc3b1800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3ac0757b0 0x7fc3ac19c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:54962/0 (socket says 172.21.15.27:54962) 2024-09-21T00:19:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.566+0000 7fc3b1800640 1 -- 172.21.15.27:0/2275393991 learned_addr learned my addr 172.21.15.27:0/2275393991 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.566+0000 7fc3b0e00640 1 --2- 172.21.15.27:0/2275393991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ac076100 0x7fc3ac19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.566+0000 7fc3b2200640 1 --2- 172.21.15.27:0/2275393991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac1100d0 0x7fc3ac1a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.567+0000 7fc3b1800640 1 -- 172.21.15.27:0/2275393991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ac076100 msgr2=0x7fc3ac19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.567+0000 7fc3b1800640 1 --2- 172.21.15.27:0/2275393991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ac076100 0x7fc3ac19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.567+0000 7fc3b1800640 1 -- 172.21.15.27:0/2275393991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac1100d0 msgr2=0x7fc3ac1a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.567+0000 7fc3b1800640 1 --2- 172.21.15.27:0/2275393991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac1100d0 0x7fc3ac1a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:29.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.567+0000 7fc3b1800640 1 -- 172.21.15.27:0/2275393991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac1a26b0 con 0x7fc3ac0757b0 2024-09-21T00:19:29.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.567+0000 7fc3b1800640 1 --2- 172.21.15.27:0/2275393991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3ac0757b0 0x7fc3ac19c740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0005ab0 tx=0x7fc3a0003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:29.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.567+0000 7fc3aa000640 1 -- 172.21.15.27:0/2275393991 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a003f070 con 0x7fc3ac0757b0 2024-09-21T00:19:29.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.568+0000 7fc3b4200640 1 -- 172.21.15.27:0/2275393991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ac1a2940 con 0x7fc3ac0757b0 2024-09-21T00:19:29.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.568+0000 7fc3aa000640 1 -- 172.21.15.27:0/2275393991 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3a0031d60 con 0x7fc3ac0757b0 2024-09-21T00:19:29.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.568+0000 7fc3aa000640 1 -- 172.21.15.27:0/2275393991 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0035030 con 0x7fc3ac0757b0 2024-09-21T00:19:29.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.568+0000 7fc3b4200640 1 -- 172.21.15.27:0/2275393991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ac1a9100 con 0x7fc3ac0757b0 2024-09-21T00:19:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.569+0000 7fc3b4200640 1 -- 172.21.15.27:0/2275393991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ac075bc0 con 0x7fc3ac0757b0 2024-09-21T00:19:29.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.572+0000 7fc3aa000640 1 -- 172.21.15.27:0/2275393991 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fc3a0042070 con 0x7fc3ac0757b0 2024-09-21T00:19:29.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.573+0000 7fc3aa000640 1 --2- 172.21.15.27:0/2275393991 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc3940b6560 0x7fc3940b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:29.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.573+0000 7fc3aa000640 1 -- 172.21.15.27:0/2275393991 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7fc3a01085d0 con 0x7fc3ac0757b0 2024-09-21T00:19:29.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.573+0000 7fc3b0e00640 1 --2- 172.21.15.27:0/2275393991 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc3940b6560 0x7fc3940b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:29.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.574+0000 7fc3b0e00640 1 --2- 172.21.15.27:0/2275393991 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc3940b6560 0x7fc3940b8a70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac1005a0 tx=0x7fc39800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:29.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.575+0000 7fc3aa000640 1 -- 172.21.15.27:0/2275393991 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3a00d2db0 con 0x7fc3ac0757b0 2024-09-21T00:19:29.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.813+0000 7fc3b4200640 1 -- 172.21.15.27:0/2275393991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc3ac19d870 con 0x7fc3ac0757b0 2024-09-21T00:19:29.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.855+0000 7fc3aa000640 1 -- 172.21.15.27:0/2275393991 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fc3a00d7cf0 con 0x7fc3ac0757b0 2024-09-21T00:19:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.858+0000 7fc3b4200640 1 -- 172.21.15.27:0/2275393991 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc3940b6560 msgr2=0x7fc3940b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.858+0000 7fc3b4200640 1 --2- 172.21.15.27:0/2275393991 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc3940b6560 0x7fc3940b8a70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac1005a0 tx=0x7fc39800b390 comp rx=0 tx=0).stop 2024-09-21T00:19:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.858+0000 7fc3b4200640 1 -- 172.21.15.27:0/2275393991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3ac0757b0 msgr2=0x7fc3ac19c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.858+0000 7fc3b4200640 1 --2- 172.21.15.27:0/2275393991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3ac0757b0 0x7fc3ac19c740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0005ab0 tx=0x7fc3a0003cd0 comp rx=0 tx=0).stop 2024-09-21T00:19:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.858+0000 7fc3b4200640 1 -- 172.21.15.27:0/2275393991 shutdown_connections 2024-09-21T00:19:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.858+0000 7fc3b4200640 1 --2- 172.21.15.27:0/2275393991 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc3940b6560 0x7fc3940b8a70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.858+0000 7fc3b4200640 1 --2- 172.21.15.27:0/2275393991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac1100d0 0x7fc3ac1a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.858+0000 7fc3b4200640 1 --2- 172.21.15.27:0/2275393991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ac076100 0x7fc3ac19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.858+0000 7fc3b4200640 1 --2- 172.21.15.27:0/2275393991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3ac0757b0 0x7fc3ac19c740 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.858+0000 7fc3b4200640 1 -- 172.21.15.27:0/2275393991 >> 172.21.15.27:0/2275393991 conn(0x7fc3ac0ff5e0 msgr2=0x7fc3ac109c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.858+0000 7fc3b4200640 1 -- 172.21.15.27:0/2275393991 shutdown_connections 2024-09-21T00:19:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:29.858+0000 7fc3b4200640 1 -- 172.21.15.27:0/2275393991 wait complete. 2024-09-21T00:19:30.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:29 smithi027 ceph-mon[30745]: pgmap v318: 129 pgs: 5 peering, 124 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:30.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:29 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2275393991' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T00:19:30.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:29 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T00:19:30.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:29 smithi089 ceph-mon[34481]: pgmap v318: 129 pgs: 5 peering, 124 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:30.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:29 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2275393991' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T00:19:30.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:29 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T00:19:30.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:29 smithi098 ceph-mon[34725]: pgmap v318: 129 pgs: 5 peering, 124 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:19:30.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:29 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2275393991' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T00:19:30.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:29 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T00:19:30.741 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-21T00:19:30.741 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs set cephfs max_mds 3 2024-09-21T00:19:31.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:30 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T00:19:31.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:30 smithi027 ceph-mon[30745]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T00:19:31.131 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:31.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:30 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T00:19:31.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:30 smithi089 ceph-mon[34481]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T00:19:31.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:30 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T00:19:31.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:30 smithi098 ceph-mon[34725]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T00:19:31.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.734+0000 7fbb9fc00640 1 -- 172.21.15.27:0/2638561071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb9810c010 msgr2=0x7fbb98112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:31.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.734+0000 7fbb9fc00640 1 --2- 172.21.15.27:0/2638561071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb9810c010 0x7fbb98112840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009ca0 tx=0x7fbb9002a8c0 comp rx=0 tx=0).stop 2024-09-21T00:19:31.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.734+0000 7fbb9fc00640 1 -- 172.21.15.27:0/2638561071 shutdown_connections 2024-09-21T00:19:31.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.735+0000 7fbb9fc00640 1 --2- 172.21.15.27:0/2638561071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb9810c010 0x7fbb98112840 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:31.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.735+0000 7fbb9fc00640 1 --2- 172.21.15.27:0/2638561071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb9810b6c0 0x7fbb9810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:31.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.735+0000 7fbb9fc00640 1 --2- 172.21.15.27:0/2638561071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb98105d80 0x7fbb98106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:31.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.735+0000 7fbb9fc00640 1 -- 172.21.15.27:0/2638561071 >> 172.21.15.27:0/2638561071 conn(0x7fbb98101810 msgr2=0x7fbb98103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:31.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.735+0000 7fbb9fc00640 1 -- 172.21.15.27:0/2638561071 shutdown_connections 2024-09-21T00:19:31.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.735+0000 7fbb9fc00640 1 -- 172.21.15.27:0/2638561071 wait complete. 2024-09-21T00:19:31.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.736+0000 7fbb9fc00640 1 Processor -- start 2024-09-21T00:19:31.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.736+0000 7fbb9fc00640 1 -- start start 2024-09-21T00:19:31.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.737+0000 7fbb9fc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb98105d80 0x7fbb9819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:31.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.737+0000 7fbb9fc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb9810b6c0 0x7fbb9819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:31.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.737+0000 7fbb9fc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb9810c010 0x7fbb981a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:31.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.737+0000 7fbb9fc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbb98115d20 con 0x7fbb9810c010 2024-09-21T00:19:31.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.737+0000 7fbb9fc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbb98115ba0 con 0x7fbb98105d80 2024-09-21T00:19:31.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.737+0000 7fbb9fc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbb98115ea0 con 0x7fbb9810b6c0 2024-09-21T00:19:31.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.737+0000 7fbb9d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb98105d80 0x7fbb9819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:31.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.737+0000 7fbb9d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb98105d80 0x7fbb9819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:59802/0 (socket says 172.21.15.27:59802) 2024-09-21T00:19:31.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.737+0000 7fbb9dc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb9810c010 0x7fbb981a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:31.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.737+0000 7fbb9d200640 1 -- 172.21.15.27:0/4117608720 learned_addr learned my addr 172.21.15.27:0/4117608720 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:31.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.737+0000 7fbb8fe00640 1 --2- 172.21.15.27:0/4117608720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb9810b6c0 0x7fbb9819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:31.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.738+0000 7fbb9d200640 1 -- 172.21.15.27:0/4117608720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb9810b6c0 msgr2=0x7fbb9819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:31.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.738+0000 7fbb9d200640 1 --2- 172.21.15.27:0/4117608720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb9810b6c0 0x7fbb9819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:31.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.738+0000 7fbb9d200640 1 -- 172.21.15.27:0/4117608720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb9810c010 msgr2=0x7fbb981a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:31.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.738+0000 7fbb9d200640 1 --2- 172.21.15.27:0/4117608720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb9810c010 0x7fbb981a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:31.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.738+0000 7fbb9d200640 1 -- 172.21.15.27:0/4117608720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb981a25e0 con 0x7fbb98105d80 2024-09-21T00:19:31.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.738+0000 7fbb9d200640 1 --2- 172.21.15.27:0/4117608720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb98105d80 0x7fbb9819c670 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800e9f0 tx=0x7fbb8800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:31.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.738+0000 7fbb96000640 1 -- 172.21.15.27:0/4117608720 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8800cd80 con 0x7fbb98105d80 2024-09-21T00:19:31.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.738+0000 7fbb96000640 1 -- 172.21.15.27:0/4117608720 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb880054d0 con 0x7fbb98105d80 2024-09-21T00:19:31.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.738+0000 7fbb96000640 1 -- 172.21.15.27:0/4117608720 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88010690 con 0x7fbb98105d80 2024-09-21T00:19:31.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.738+0000 7fbb9fc00640 1 -- 172.21.15.27:0/4117608720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb981a28d0 con 0x7fbb98105d80 2024-09-21T00:19:31.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.738+0000 7fbb9fc00640 1 -- 172.21.15.27:0/4117608720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb981a9030 con 0x7fbb98105d80 2024-09-21T00:19:31.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.739+0000 7fbb9fc00640 1 -- 172.21.15.27:0/4117608720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb98106190 con 0x7fbb98105d80 2024-09-21T00:19:31.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.742+0000 7fbb96000640 1 -- 172.21.15.27:0/4117608720 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fbb88004e50 con 0x7fbb98105d80 2024-09-21T00:19:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.744+0000 7fbb96000640 1 --2- 172.21.15.27:0/4117608720 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbb7c0b65e0 0x7fbb7c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.744+0000 7fbb96000640 1 -- 172.21.15.27:0/4117608720 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7fbb880e7d50 con 0x7fbb98105d80 2024-09-21T00:19:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.744+0000 7fbb8fe00640 1 --2- 172.21.15.27:0/4117608720 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbb7c0b65e0 0x7fbb7c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:31.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.745+0000 7fbb8fe00640 1 --2- 172.21.15.27:0/4117608720 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbb7c0b65e0 0x7fbb7c0b8af0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbb981050c0 tx=0x7fbb80005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:31.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.746+0000 7fbb96000640 1 -- 172.21.15.27:0/4117608720 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbb880ae150 con 0x7fbb98105d80 2024-09-21T00:19:31.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:31.988+0000 7fbb9fc00640 1 -- 172.21.15.27:0/4117608720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fbb9802e640 con 0x7fbb98105d80 2024-09-21T00:19:32.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:31 smithi089 ceph-mon[34481]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 482 B/s, 0 objects/s recovering 2024-09-21T00:19:32.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:31 smithi098 ceph-mon[34725]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 482 B/s, 0 objects/s recovering 2024-09-21T00:19:32.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:31 smithi027 ceph-mon[30745]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 482 B/s, 0 objects/s recovering 2024-09-21T00:19:32.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:32.872+0000 7fbb96000640 1 -- 172.21.15.27:0/4117608720 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fbb880b2530 con 0x7fbb98105d80 2024-09-21T00:19:32.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:32.876+0000 7fbb9fc00640 1 -- 172.21.15.27:0/4117608720 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbb7c0b65e0 msgr2=0x7fbb7c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:32.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:32.876+0000 7fbb9fc00640 1 --2- 172.21.15.27:0/4117608720 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbb7c0b65e0 0x7fbb7c0b8af0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbb981050c0 tx=0x7fbb80005f90 comp rx=0 tx=0).stop 2024-09-21T00:19:32.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:32.876+0000 7fbb9fc00640 1 -- 172.21.15.27:0/4117608720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb98105d80 msgr2=0x7fbb9819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:32.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:32.876+0000 7fbb9fc00640 1 --2- 172.21.15.27:0/4117608720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb98105d80 0x7fbb9819c670 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800e9f0 tx=0x7fbb8800edf0 comp rx=0 tx=0).stop 2024-09-21T00:19:32.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:32.876+0000 7fbb9fc00640 1 -- 172.21.15.27:0/4117608720 shutdown_connections 2024-09-21T00:19:32.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:32.876+0000 7fbb9fc00640 1 --2- 172.21.15.27:0/4117608720 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbb7c0b65e0 0x7fbb7c0b8af0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:32.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:32.876+0000 7fbb9fc00640 1 --2- 172.21.15.27:0/4117608720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb9810c010 0x7fbb981a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:32.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:32.876+0000 7fbb9fc00640 1 --2- 172.21.15.27:0/4117608720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb9810b6c0 0x7fbb9819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:32.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:32.876+0000 7fbb9fc00640 1 --2- 172.21.15.27:0/4117608720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb98105d80 0x7fbb9819c670 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:32.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:32.876+0000 7fbb9fc00640 1 -- 172.21.15.27:0/4117608720 >> 172.21.15.27:0/4117608720 conn(0x7fbb98101810 msgr2=0x7fbb98108c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:32.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:32.877+0000 7fbb9fc00640 1 -- 172.21.15.27:0/4117608720 shutdown_connections 2024-09-21T00:19:32.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:32.877+0000 7fbb9fc00640 1 -- 172.21.15.27:0/4117608720 wait complete. 2024-09-21T00:19:33.228 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:32 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/4117608720' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T00:19:33.228 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:32 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T00:19:33.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:32 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/4117608720' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T00:19:33.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:32 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T00:19:33.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:32 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/4117608720' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T00:19:33.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:32 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T00:19:33.636 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T00:19:34.030 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:34.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:33 smithi027 ceph-mon[30745]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 482 B/s, 0 objects/s recovering 2024-09-21T00:19:34.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:33 smithi027 ceph-mon[30745]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T00:19:34.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:33 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-21T00:19:34.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:33 smithi027 ceph-mon[30745]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T00:19:34.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:33 smithi027 ceph-mon[30745]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T00:19:34.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:33 smithi027 ceph-mon[30745]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-09-21T00:19:34.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:33 smithi027 ceph-mon[30745]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-09-21T00:19:34.228 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:19:33 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-b[63110]: 2024-09-21T00:19:33.901+0000 7f986b756640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T00:19:34.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:33 smithi089 ceph-mon[34481]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 482 B/s, 0 objects/s recovering 2024-09-21T00:19:34.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:33 smithi089 ceph-mon[34481]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T00:19:34.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:33 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-21T00:19:34.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:33 smithi089 ceph-mon[34481]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T00:19:34.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:33 smithi089 ceph-mon[34481]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T00:19:34.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:33 smithi089 ceph-mon[34481]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-09-21T00:19:34.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:33 smithi089 ceph-mon[34481]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-09-21T00:19:34.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:33 smithi098 ceph-mon[34725]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 482 B/s, 0 objects/s recovering 2024-09-21T00:19:34.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:33 smithi098 ceph-mon[34725]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T00:19:34.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:33 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-21T00:19:34.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:33 smithi098 ceph-mon[34725]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T00:19:34.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:33 smithi098 ceph-mon[34725]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T00:19:34.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:33 smithi098 ceph-mon[34725]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-09-21T00:19:34.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:33 smithi098 ceph-mon[34725]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-09-21T00:19:34.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.675+0000 7f88e3200640 1 -- 172.21.15.27:0/3613824440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e4105110 msgr2=0x7f88e410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:34.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.675+0000 7f88e3200640 1 --2- 172.21.15.27:0/3613824440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e4105110 0x7f88e410d660 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009900 tx=0x7f88d002a400 comp rx=0 tx=0).stop 2024-09-21T00:19:34.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.675+0000 7f88e3200640 1 -- 172.21.15.27:0/3613824440 shutdown_connections 2024-09-21T00:19:34.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.675+0000 7f88e3200640 1 --2- 172.21.15.27:0/3613824440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88e410dc20 0x7f88e4110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:34.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.675+0000 7f88e3200640 1 --2- 172.21.15.27:0/3613824440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e4105110 0x7f88e410d660 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:34.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.675+0000 7f88e3200640 1 --2- 172.21.15.27:0/3613824440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88e41047c0 0x7f88e4104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:34.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.675+0000 7f88e3200640 1 -- 172.21.15.27:0/3613824440 >> 172.21.15.27:0/3613824440 conn(0x7f88e40fe2c0 msgr2=0x7f88e4100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:34.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.675+0000 7f88e3200640 1 -- 172.21.15.27:0/3613824440 shutdown_connections 2024-09-21T00:19:34.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.675+0000 7f88e3200640 1 -- 172.21.15.27:0/3613824440 wait complete. 2024-09-21T00:19:34.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.676+0000 7f88e3200640 1 Processor -- start 2024-09-21T00:19:34.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.676+0000 7f88e3200640 1 -- start start 2024-09-21T00:19:34.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.676+0000 7f88e3200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88e41047c0 0x7f88e419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:34.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.676+0000 7f88e3200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e4105110 0x7f88e419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:34.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e3200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88e410dc20 0x7f88e419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:34.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e3200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f88e4113840 con 0x7f88e4105110 2024-09-21T00:19:34.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e3200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41136c0 con 0x7f88e410dc20 2024-09-21T00:19:34.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e3200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41139c0 con 0x7f88e41047c0 2024-09-21T00:19:34.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e4105110 0x7f88e419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:34.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e1e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88e41047c0 0x7f88e419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:34.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e2800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88e410dc20 0x7f88e419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:34.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e1e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88e41047c0 0x7f88e419a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56812/0 (socket says 172.21.15.27:56812) 2024-09-21T00:19:34.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e2800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88e410dc20 0x7f88e419fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:34108/0 (socket says 172.21.15.27:34108) 2024-09-21T00:19:34.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e1e00640 1 -- 172.21.15.27:0/162499137 learned_addr learned my addr 172.21.15.27:0/162499137 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:34.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e2800640 1 -- 172.21.15.27:0/162499137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88e41047c0 msgr2=0x7f88e419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:34.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e2800640 1 --2- 172.21.15.27:0/162499137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88e41047c0 0x7f88e419a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:34.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e2800640 1 -- 172.21.15.27:0/162499137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e4105110 msgr2=0x7f88e419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:34.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e2800640 1 --2- 172.21.15.27:0/162499137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e4105110 0x7f88e419a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e2800640 1 -- 172.21.15.27:0/162499137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e41a02e0 con 0x7f88e410dc20 2024-09-21T00:19:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.677+0000 7f88e1e00640 1 --2- 172.21.15.27:0/162499137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88e41047c0 0x7f88e419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:19:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.678+0000 7f88e2800640 1 --2- 172.21.15.27:0/162499137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88e410dc20 0x7f88e419fbb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00da10 tx=0x7f88dc00ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.678+0000 7f88daa00640 1 -- 172.21.15.27:0/162499137 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc00bd70 con 0x7f88e410dc20 2024-09-21T00:19:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.678+0000 7f88daa00640 1 -- 172.21.15.27:0/162499137 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88dc003e20 con 0x7f88e410dc20 2024-09-21T00:19:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.678+0000 7f88e3200640 1 -- 172.21.15.27:0/162499137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e41a05d0 con 0x7f88e410dc20 2024-09-21T00:19:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.678+0000 7f88daa00640 1 -- 172.21.15.27:0/162499137 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc0106c0 con 0x7f88e410dc20 2024-09-21T00:19:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.678+0000 7f88e3200640 1 -- 172.21.15.27:0/162499137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e41a6d30 con 0x7f88e410dc20 2024-09-21T00:19:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.680+0000 7f88e3200640 1 -- 172.21.15.27:0/162499137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e4104bd0 con 0x7f88e410dc20 2024-09-21T00:19:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.682+0000 7f88daa00640 1 -- 172.21.15.27:0/162499137 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f88dc004270 con 0x7f88e410dc20 2024-09-21T00:19:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.683+0000 7f88daa00640 1 --2- 172.21.15.27:0/162499137 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f88c80b65e0 0x7f88c80b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.684+0000 7f88daa00640 1 -- 172.21.15.27:0/162499137 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7f88dc0e7b50 con 0x7f88e410dc20 2024-09-21T00:19:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.684+0000 7f88e1e00640 1 --2- 172.21.15.27:0/162499137 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f88c80b65e0 0x7f88c80b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:34.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.684+0000 7f88e1e00640 1 --2- 172.21.15.27:0/162499137 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f88c80b65e0 0x7f88c80b8af0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009cf0 tx=0x7f88d4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:34.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.686+0000 7f88daa00640 1 -- 172.21.15.27:0/162499137 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f88dc0adfd0 con 0x7f88e410dc20 2024-09-21T00:19:34.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:34.952+0000 7f88e3200640 1 -- 172.21.15.27:0/162499137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f88e419b4a0 con 0x7f88e410dc20 2024-09-21T00:19:35.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:34 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] up:active 2024-09-21T00:19:35.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:34 smithi027 ceph-mon[30745]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T00:19:35.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:34 smithi027 ceph-mon[30745]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T00:19:35.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:34 smithi027 ceph-mon[30745]: Cluster is now healthy 2024-09-21T00:19:35.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:34 smithi027 ceph-mon[30745]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-09-21T00:19:35.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:34 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-09-21T00:19:35.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:34 smithi027 ceph-mon[30745]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T00:19:35.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:34 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:19:35.196 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:34 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] up:active 2024-09-21T00:19:35.197 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:34 smithi098 ceph-mon[34725]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T00:19:35.197 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:34 smithi098 ceph-mon[34725]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T00:19:35.197 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:34 smithi098 ceph-mon[34725]: Cluster is now healthy 2024-09-21T00:19:35.197 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:34 smithi098 ceph-mon[34725]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-09-21T00:19:35.197 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:34 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-09-21T00:19:35.197 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:34 smithi098 ceph-mon[34725]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T00:19:35.197 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:34 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:19:35.209 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:34 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] up:active 2024-09-21T00:19:35.209 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:34 smithi089 ceph-mon[34481]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T00:19:35.210 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:34 smithi089 ceph-mon[34481]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T00:19:35.210 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:34 smithi089 ceph-mon[34481]: Cluster is now healthy 2024-09-21T00:19:35.210 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:34 smithi089 ceph-mon[34481]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-09-21T00:19:35.210 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:34 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-09-21T00:19:35.210 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:34 smithi089 ceph-mon[34481]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T00:19:35.210 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:34 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:19:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:35.913+0000 7f88daa00640 1 -- 172.21.15.27:0/162499137 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f88e419b4a0 con 0x7f88e410dc20 2024-09-21T00:19:35.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:35.916+0000 7f88e3200640 1 -- 172.21.15.27:0/162499137 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f88c80b65e0 msgr2=0x7f88c80b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:35.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:35.916+0000 7f88e3200640 1 --2- 172.21.15.27:0/162499137 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f88c80b65e0 0x7f88c80b8af0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009cf0 tx=0x7f88d4005f90 comp rx=0 tx=0).stop 2024-09-21T00:19:35.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:35.916+0000 7f88e3200640 1 -- 172.21.15.27:0/162499137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88e410dc20 msgr2=0x7f88e419fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:35.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:35.916+0000 7f88e3200640 1 --2- 172.21.15.27:0/162499137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88e410dc20 0x7f88e419fbb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00da10 tx=0x7f88dc00ded0 comp rx=0 tx=0).stop 2024-09-21T00:19:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:35.917+0000 7f88e3200640 1 -- 172.21.15.27:0/162499137 shutdown_connections 2024-09-21T00:19:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:35.917+0000 7f88e3200640 1 --2- 172.21.15.27:0/162499137 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f88c80b65e0 0x7f88c80b8af0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:35.917+0000 7f88e3200640 1 --2- 172.21.15.27:0/162499137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88e410dc20 0x7f88e419fbb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:35.917+0000 7f88e3200640 1 --2- 172.21.15.27:0/162499137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e4105110 0x7f88e419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:35.917+0000 7f88e3200640 1 --2- 172.21.15.27:0/162499137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88e41047c0 0x7f88e419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:35.917+0000 7f88e3200640 1 -- 172.21.15.27:0/162499137 >> 172.21.15.27:0/162499137 conn(0x7f88e40fe2c0 msgr2=0x7f88e40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:35.917+0000 7f88e3200640 1 -- 172.21.15.27:0/162499137 shutdown_connections 2024-09-21T00:19:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:35.917+0000 7f88e3200640 1 -- 172.21.15.27:0/162499137 wait complete. 2024-09-21T00:19:36.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:35 smithi089 ceph-mon[34481]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s; 406 B/s, 0 objects/s recovering 2024-09-21T00:19:36.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:35 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] up:active 2024-09-21T00:19:36.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:35 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-09-21T00:19:36.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:35 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/162499137' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T00:19:36.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:35 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T00:19:36.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:35 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:36.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:35 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:36.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:35 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:36.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:35 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:36.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:35 smithi098 ceph-mon[34725]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s; 406 B/s, 0 objects/s recovering 2024-09-21T00:19:36.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:35 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] up:active 2024-09-21T00:19:36.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:35 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-09-21T00:19:36.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:35 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/162499137' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T00:19:36.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:35 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T00:19:36.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:35 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:36.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:35 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:36.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:35 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:36.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:35 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:36.261 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:35 smithi027 ceph-mon[30745]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s; 406 B/s, 0 objects/s recovering 2024-09-21T00:19:36.262 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:35 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] up:active 2024-09-21T00:19:36.262 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:35 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-09-21T00:19:36.262 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:35 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/162499137' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T00:19:36.262 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:35 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T00:19:36.262 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:35 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:36.262 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:35 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:36.262 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:35 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:36.262 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:35 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:36.598 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T00:19:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:37.173 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:36 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T00:19:37.173 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:36 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-09-21T00:19:37.173 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:36 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:19:37.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:36 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T00:19:37.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:36 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-09-21T00:19:37.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:36 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:19:37.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:36 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T00:19:37.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:36 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-09-21T00:19:37.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:36 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:19:37.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.697+0000 7f6394e00640 1 -- 172.21.15.27:0/3246350987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6390103b80 msgr2=0x7f6390103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:37.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.697+0000 7f6394e00640 1 --2- 172.21.15.27:0/3246350987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6390103b80 0x7f6390103f90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6380009900 tx=0x7f638002a3f0 comp rx=0 tx=0).stop 2024-09-21T00:19:37.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.697+0000 7f6394e00640 1 -- 172.21.15.27:0/3246350987 shutdown_connections 2024-09-21T00:19:37.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.697+0000 7f6394e00640 1 --2- 172.21.15.27:0/3246350987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6390109e10 0x7f6390110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:37.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.697+0000 7f6394e00640 1 --2- 172.21.15.27:0/3246350987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63901094c0 0x7f63901098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:37.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.697+0000 7f6394e00640 1 --2- 172.21.15.27:0/3246350987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6390103b80 0x7f6390103f90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:37.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.697+0000 7f6394e00640 1 -- 172.21.15.27:0/3246350987 >> 172.21.15.27:0/3246350987 conn(0x7f63900ff590 msgr2=0x7f63901019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:37.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.697+0000 7f6394e00640 1 -- 172.21.15.27:0/3246350987 shutdown_connections 2024-09-21T00:19:37.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.697+0000 7f6394e00640 1 -- 172.21.15.27:0/3246350987 wait complete. 2024-09-21T00:19:37.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.698+0000 7f6394e00640 1 Processor -- start 2024-09-21T00:19:37.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.699+0000 7f6394e00640 1 -- start start 2024-09-21T00:19:37.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.699+0000 7f6394e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6390103b80 0x7f639019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:37.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.699+0000 7f6394e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63901094c0 0x7f639019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:37.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.699+0000 7f6394e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6390109e10 0x7f639019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:37.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.699+0000 7f6394e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6390113ad0 con 0x7f63901094c0 2024-09-21T00:19:37.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.699+0000 7f6394e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6390113950 con 0x7f6390109e10 2024-09-21T00:19:37.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.699+0000 7f6394e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6390113c50 con 0x7f6390103b80 2024-09-21T00:19:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.699+0000 7f638de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6390103b80 0x7f639019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.699+0000 7f638d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63901094c0 0x7f639019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.699+0000 7f638de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6390103b80 0x7f639019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56832/0 (socket says 172.21.15.27:56832) 2024-09-21T00:19:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.699+0000 7f638e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6390109e10 0x7f639019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.699+0000 7f638de00640 1 -- 172.21.15.27:0/479926431 learned_addr learned my addr 172.21.15.27:0/479926431 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.700+0000 7f638de00640 1 -- 172.21.15.27:0/479926431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6390109e10 msgr2=0x7f639019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.700+0000 7f638de00640 1 --2- 172.21.15.27:0/479926431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6390109e10 0x7f639019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.700+0000 7f638de00640 1 -- 172.21.15.27:0/479926431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63901094c0 msgr2=0x7f639019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.700+0000 7f638de00640 1 --2- 172.21.15.27:0/479926431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63901094c0 0x7f639019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.700+0000 7f638de00640 1 -- 172.21.15.27:0/479926431 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63901a0510 con 0x7f6390103b80 2024-09-21T00:19:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.700+0000 7f638e800640 1 --2- 172.21.15.27:0/479926431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6390109e10 0x7f639019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:19:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.701+0000 7f638de00640 1 --2- 172.21.15.27:0/479926431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6390103b80 0x7f639019a5a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6380005940 tx=0x7f63800040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.701+0000 7f6386a00640 1 -- 172.21.15.27:0/479926431 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638003f030 con 0x7f6390103b80 2024-09-21T00:19:37.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.701+0000 7f6386a00640 1 -- 172.21.15.27:0/479926431 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63800044c0 con 0x7f6390103b80 2024-09-21T00:19:37.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.701+0000 7f6394e00640 1 -- 172.21.15.27:0/479926431 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63901a07a0 con 0x7f6390103b80 2024-09-21T00:19:37.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.701+0000 7f6386a00640 1 -- 172.21.15.27:0/479926431 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380035070 con 0x7f6390103b80 2024-09-21T00:19:37.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.701+0000 7f6394e00640 1 -- 172.21.15.27:0/479926431 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63901a6f60 con 0x7f6390103b80 2024-09-21T00:19:37.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.702+0000 7f6394e00640 1 -- 172.21.15.27:0/479926431 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6390104e00 con 0x7f6390103b80 2024-09-21T00:19:37.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.706+0000 7f6386a00640 1 -- 172.21.15.27:0/479926431 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f6380042030 con 0x7f6390103b80 2024-09-21T00:19:37.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.708+0000 7f6386a00640 1 --2- 172.21.15.27:0/479926431 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f63740b6560 0x7f63740b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:37.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.708+0000 7f6386a00640 1 -- 172.21.15.27:0/479926431 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7f6380107b70 con 0x7f6390103b80 2024-09-21T00:19:37.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.708+0000 7f638d400640 1 --2- 172.21.15.27:0/479926431 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f63740b6560 0x7f63740b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:37.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.709+0000 7f638d400640 1 --2- 172.21.15.27:0/479926431 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f63740b6560 0x7f63740b8a70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6390101c80 tx=0x7f637c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:37.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.712+0000 7f6386a00640 1 -- 172.21.15.27:0/479926431 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f63800cdf70 con 0x7f6390103b80 2024-09-21T00:19:37.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:37.957+0000 7f6394e00640 1 -- 172.21.15.27:0/479926431 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f63901a7160 con 0x7f6390103b80 2024-09-21T00:19:38.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:38 smithi027 ceph-mon[30745]: pgmap v322: 129 pgs: 129 active+clean; 583 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1.9 KiB/s wr, 5 op/s; 367 B/s, 0 objects/s recovering 2024-09-21T00:19:38.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:38 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:38.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:38 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:38.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:38 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/479926431' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T00:19:38.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:38 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T00:19:38.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:38.409+0000 7f6386a00640 1 -- 172.21.15.27:0/479926431 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f63800d2350 con 0x7f6390103b80 2024-09-21T00:19:38.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:38.412+0000 7f6394e00640 1 -- 172.21.15.27:0/479926431 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f63740b6560 msgr2=0x7f63740b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:38.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:38.412+0000 7f6394e00640 1 --2- 172.21.15.27:0/479926431 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f63740b6560 0x7f63740b8a70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6390101c80 tx=0x7f637c005f90 comp rx=0 tx=0).stop 2024-09-21T00:19:38.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:38.412+0000 7f6394e00640 1 -- 172.21.15.27:0/479926431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6390103b80 msgr2=0x7f639019a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:38.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:38.412+0000 7f6394e00640 1 --2- 172.21.15.27:0/479926431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6390103b80 0x7f639019a5a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6380005940 tx=0x7f63800040c0 comp rx=0 tx=0).stop 2024-09-21T00:19:38.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:38.412+0000 7f6394e00640 1 -- 172.21.15.27:0/479926431 shutdown_connections 2024-09-21T00:19:38.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:38.412+0000 7f6394e00640 1 --2- 172.21.15.27:0/479926431 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f63740b6560 0x7f63740b8a70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:38.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:38.412+0000 7f6394e00640 1 --2- 172.21.15.27:0/479926431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6390109e10 0x7f639019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:38.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:38.412+0000 7f6394e00640 1 --2- 172.21.15.27:0/479926431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63901094c0 0x7f639019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:38.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:38.412+0000 7f6394e00640 1 --2- 172.21.15.27:0/479926431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6390103b80 0x7f639019a5a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:38.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:38.412+0000 7f6394e00640 1 -- 172.21.15.27:0/479926431 >> 172.21.15.27:0/479926431 conn(0x7f63900ff590 msgr2=0x7f6390107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:38.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:38.412+0000 7f6394e00640 1 -- 172.21.15.27:0/479926431 shutdown_connections 2024-09-21T00:19:38.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:38.412+0000 7f6394e00640 1 -- 172.21.15.27:0/479926431 wait complete. 2024-09-21T00:19:38.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:38 smithi089 ceph-mon[34481]: pgmap v322: 129 pgs: 129 active+clean; 583 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1.9 KiB/s wr, 5 op/s; 367 B/s, 0 objects/s recovering 2024-09-21T00:19:38.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:38 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:38.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:38 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:38.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:38 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/479926431' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T00:19:38.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:38 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T00:19:38.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:38 smithi098 ceph-mon[34725]: pgmap v322: 129 pgs: 129 active+clean; 583 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1.9 KiB/s wr, 5 op/s; 367 B/s, 0 objects/s recovering 2024-09-21T00:19:38.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:38 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:38.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:38 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:38.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:38 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/479926431' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T00:19:38.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:38 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T00:19:39.161 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-21T00:19:39.162 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T00:19:39.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:39 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:19:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:39 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:19:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:39 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:39 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T00:19:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:39 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:19:39.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:39 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:19:39.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:39 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:19:39.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:39 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:39.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:39 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T00:19:39.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:39 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:19:39.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:39 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:19:39.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:39 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:19:39.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:39 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:19:39.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:39 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T00:19:39.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:39 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:19:39.549 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:40.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:40 smithi027 ceph-mon[30745]: pgmap v323: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s; 321 B/s, 0 objects/s recovering 2024-09-21T00:19:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.296+0000 7f652c800640 1 -- 172.21.15.27:0/2708864910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f652410b6d0 msgr2=0x7f652410bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.296+0000 7f652c800640 1 --2- 172.21.15.27:0/2708864910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f652410b6d0 0x7f652410bae0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6514009900 tx=0x7f651402a410 comp rx=0 tx=0).stop 2024-09-21T00:19:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.297+0000 7f652c800640 1 -- 172.21.15.27:0/2708864910 shutdown_connections 2024-09-21T00:19:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.297+0000 7f652c800640 1 --2- 172.21.15.27:0/2708864910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f652410c020 0x7f6524112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.297+0000 7f652c800640 1 --2- 172.21.15.27:0/2708864910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f652410b6d0 0x7f652410bae0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.297+0000 7f652c800640 1 --2- 172.21.15.27:0/2708864910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6524105d90 0x7f65241061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.297+0000 7f652c800640 1 -- 172.21.15.27:0/2708864910 >> 172.21.15.27:0/2708864910 conn(0x7f6524101860 msgr2=0x7f6524103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.297+0000 7f652c800640 1 -- 172.21.15.27:0/2708864910 shutdown_connections 2024-09-21T00:19:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.297+0000 7f652c800640 1 -- 172.21.15.27:0/2708864910 wait complete. 2024-09-21T00:19:40.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.298+0000 7f652c800640 1 Processor -- start 2024-09-21T00:19:40.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.298+0000 7f652c800640 1 -- start start 2024-09-21T00:19:40.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.298+0000 7f652c800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6524105d90 0x7f652419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:40.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.298+0000 7f652c800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f652410b6d0 0x7f652419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:40.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.298+0000 7f652c800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f652410c020 0x7f65241a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:40.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.298+0000 7f652c800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6524115ce0 con 0x7f652410c020 2024-09-21T00:19:40.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.298+0000 7f652c800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6524115b60 con 0x7f652410b6d0 2024-09-21T00:19:40.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.298+0000 7f652c800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6524115e60 con 0x7f6524105d90 2024-09-21T00:19:40.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.299+0000 7f6529e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6524105d90 0x7f652419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:40.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.299+0000 7f652a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f652410c020 0x7f65241a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:40.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.299+0000 7f6529e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6524105d90 0x7f652419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56860/0 (socket says 172.21.15.27:56860) 2024-09-21T00:19:40.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.299+0000 7f6529e00640 1 -- 172.21.15.27:0/3274687458 learned_addr learned my addr 172.21.15.27:0/3274687458 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:40.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.299+0000 7f6529400640 1 --2- 172.21.15.27:0/3274687458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f652410b6d0 0x7f652419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:40.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.299+0000 7f6529e00640 1 -- 172.21.15.27:0/3274687458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f652410b6d0 msgr2=0x7f652419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:40.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.299+0000 7f6529e00640 1 --2- 172.21.15.27:0/3274687458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f652410b6d0 0x7f652419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:40.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.299+0000 7f6529e00640 1 -- 172.21.15.27:0/3274687458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f652410c020 msgr2=0x7f65241a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:40.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.299+0000 7f6529e00640 1 --2- 172.21.15.27:0/3274687458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f652410c020 0x7f65241a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:40.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.299+0000 7f6529e00640 1 -- 172.21.15.27:0/3274687458 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65241a2660 con 0x7f6524105d90 2024-09-21T00:19:40.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.300+0000 7f6529e00640 1 --2- 172.21.15.27:0/3274687458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6524105d90 0x7f652419c6f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f651800e9f0 tx=0x7f651800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:40.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.300+0000 7f651ea00640 1 -- 172.21.15.27:0/3274687458 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651800cd80 con 0x7f6524105d90 2024-09-21T00:19:40.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.300+0000 7f651ea00640 1 -- 172.21.15.27:0/3274687458 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65180054d0 con 0x7f6524105d90 2024-09-21T00:19:40.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.300+0000 7f652c800640 1 -- 172.21.15.27:0/3274687458 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65241a2950 con 0x7f6524105d90 2024-09-21T00:19:40.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.300+0000 7f651ea00640 1 -- 172.21.15.27:0/3274687458 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518010690 con 0x7f6524105d90 2024-09-21T00:19:40.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.300+0000 7f652c800640 1 -- 172.21.15.27:0/3274687458 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65241a90b0 con 0x7f6524105d90 2024-09-21T00:19:40.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.301+0000 7f652c800640 1 -- 172.21.15.27:0/3274687458 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65080052c0 con 0x7f6524105d90 2024-09-21T00:19:40.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.304+0000 7f651ea00640 1 -- 172.21.15.27:0/3274687458 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f65180046c0 con 0x7f6524105d90 2024-09-21T00:19:40.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.305+0000 7f651ea00640 1 --2- 172.21.15.27:0/3274687458 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f650c0b65e0 0x7f650c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:40.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.305+0000 7f651ea00640 1 -- 172.21.15.27:0/3274687458 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7f65180e7c50 con 0x7f6524105d90 2024-09-21T00:19:40.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.305+0000 7f6529400640 1 --2- 172.21.15.27:0/3274687458 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f650c0b65e0 0x7f650c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:40.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.306+0000 7f6529400640 1 --2- 172.21.15.27:0/3274687458 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f650c0b65e0 0x7f650c0b8af0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6514009230 tx=0x7f6514031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:40.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.307+0000 7f651ea00640 1 -- 172.21.15.27:0/3274687458 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f65180ae050 con 0x7f6524105d90 2024-09-21T00:19:40.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:40 smithi089 ceph-mon[34481]: pgmap v323: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s; 321 B/s, 0 objects/s recovering 2024-09-21T00:19:40.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:40 smithi098 ceph-mon[34725]: pgmap v323: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s; 321 B/s, 0 objects/s recovering 2024-09-21T00:19:40.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.530+0000 7f652c800640 1 -- 172.21.15.27:0/3274687458 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6508000bf0 con 0x7f650c0b65e0 2024-09-21T00:19:40.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.561+0000 7f651ea00640 1 -- 172.21.15.27:0/3274687458 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6508000bf0 con 0x7f650c0b65e0 2024-09-21T00:19:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.563+0000 7f652c800640 1 -- 172.21.15.27:0/3274687458 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f650c0b65e0 msgr2=0x7f650c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.563+0000 7f652c800640 1 --2- 172.21.15.27:0/3274687458 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f650c0b65e0 0x7f650c0b8af0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6514009230 tx=0x7f6514031400 comp rx=0 tx=0).stop 2024-09-21T00:19:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.563+0000 7f652c800640 1 -- 172.21.15.27:0/3274687458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6524105d90 msgr2=0x7f652419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.563+0000 7f652c800640 1 --2- 172.21.15.27:0/3274687458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6524105d90 0x7f652419c6f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f651800e9f0 tx=0x7f651800edf0 comp rx=0 tx=0).stop 2024-09-21T00:19:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.563+0000 7f652c800640 1 -- 172.21.15.27:0/3274687458 shutdown_connections 2024-09-21T00:19:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.563+0000 7f652c800640 1 --2- 172.21.15.27:0/3274687458 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f650c0b65e0 0x7f650c0b8af0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.563+0000 7f652c800640 1 --2- 172.21.15.27:0/3274687458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f652410c020 0x7f65241a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.563+0000 7f652c800640 1 --2- 172.21.15.27:0/3274687458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f652410b6d0 0x7f652419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.563+0000 7f652c800640 1 --2- 172.21.15.27:0/3274687458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6524105d90 0x7f652419c6f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.563+0000 7f652c800640 1 -- 172.21.15.27:0/3274687458 >> 172.21.15.27:0/3274687458 conn(0x7f6524101860 msgr2=0x7f6524103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.563+0000 7f652c800640 1 -- 172.21.15.27:0/3274687458 shutdown_connections 2024-09-21T00:19:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:40.563+0000 7f652c800640 1 -- 172.21.15.27:0/3274687458 wait complete. 2024-09-21T00:19:40.817 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:19:40 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:19:40.538+0000 7fbf05e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T00:19:40.817 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:19:40 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:19:40.538+0000 7fbf05e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T00:19:40.817 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:19:40 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:19:40.538+0000 7fbf05e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T00:19:40.817 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:19:40 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:19:40.538+0000 7fbf05e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T00:19:40.817 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:19:40 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:19:40.538+0000 7fbf05e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T00:19:41.315 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T00:19:41.700 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:42.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.309+0000 7fe9d7e00640 1 -- 172.21.15.27:0/113808537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d8109c20 msgr2=0x7fe9d8110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:42.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.309+0000 7fe9d7e00640 1 --2- 172.21.15.27:0/113808537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d8109c20 0x7fe9d8110450 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009ca0 tx=0x7fe9d002a8c0 comp rx=0 tx=0).stop 2024-09-21T00:19:42.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.309+0000 7fe9d7e00640 1 -- 172.21.15.27:0/113808537 shutdown_connections 2024-09-21T00:19:42.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.309+0000 7fe9d7e00640 1 --2- 172.21.15.27:0/113808537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d8109c20 0x7fe9d8110450 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:42.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.309+0000 7fe9d7e00640 1 --2- 172.21.15.27:0/113808537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9d81092d0 0x7fe9d81096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:42.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.309+0000 7fe9d7e00640 1 --2- 172.21.15.27:0/113808537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9d8103990 0x7fe9d8103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:42.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.309+0000 7fe9d7e00640 1 -- 172.21.15.27:0/113808537 >> 172.21.15.27:0/113808537 conn(0x7fe9d80ff440 msgr2=0x7fe9d8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:42.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.309+0000 7fe9d7e00640 1 -- 172.21.15.27:0/113808537 shutdown_connections 2024-09-21T00:19:42.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.309+0000 7fe9d7e00640 1 -- 172.21.15.27:0/113808537 wait complete. 2024-09-21T00:19:42.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.310+0000 7fe9d7e00640 1 Processor -- start 2024-09-21T00:19:42.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.310+0000 7fe9d7e00640 1 -- start start 2024-09-21T00:19:42.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.311+0000 7fe9d7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d8103990 0x7fe9d819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.311+0000 7fe9d7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9d81092d0 0x7fe9d819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.311+0000 7fe9d7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9d8109c20 0x7fe9d81a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.311+0000 7fe9d7e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81138e0 con 0x7fe9d8103990 2024-09-21T00:19:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.311+0000 7fe9d7e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d8113760 con 0x7fe9d8109c20 2024-09-21T00:19:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.311+0000 7fe9d7e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d8113a60 con 0x7fe9d81092d0 2024-09-21T00:19:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.311+0000 7fe9d6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d8103990 0x7fe9d819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.311+0000 7fe9ce000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9d81092d0 0x7fe9d819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.311+0000 7fe9d6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d8103990 0x7fe9d819c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51056/0 (socket says 172.21.15.27:51056) 2024-09-21T00:19:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.311+0000 7fe9d6a00640 1 -- 172.21.15.27:0/1053134047 learned_addr learned my addr 172.21.15.27:0/1053134047 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.311+0000 7fe9d7400640 1 --2- 172.21.15.27:0/1053134047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9d8109c20 0x7fe9d81a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.312+0000 7fe9d6a00640 1 -- 172.21.15.27:0/1053134047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9d81092d0 msgr2=0x7fe9d819c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.312+0000 7fe9d6a00640 1 --2- 172.21.15.27:0/1053134047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9d81092d0 0x7fe9d819c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.312+0000 7fe9d6a00640 1 -- 172.21.15.27:0/1053134047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9d8109c20 msgr2=0x7fe9d81a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.312+0000 7fe9d6a00640 1 --2- 172.21.15.27:0/1053134047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9d8109c20 0x7fe9d81a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.312+0000 7fe9d6a00640 1 -- 172.21.15.27:0/1053134047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d81a23c0 con 0x7fe9d8103990 2024-09-21T00:19:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.312+0000 7fe9d6a00640 1 --2- 172.21.15.27:0/1053134047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d8103990 0x7fe9d819c450 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800e9f0 tx=0x7fe9c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.312+0000 7fe9cfe00640 1 -- 172.21.15.27:0/1053134047 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c800cd80 con 0x7fe9d8103990 2024-09-21T00:19:42.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.312+0000 7fe9cfe00640 1 -- 172.21.15.27:0/1053134047 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9c80108e0 con 0x7fe9d8103990 2024-09-21T00:19:42.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.312+0000 7fe9d7e00640 1 -- 172.21.15.27:0/1053134047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d81a26b0 con 0x7fe9d8103990 2024-09-21T00:19:42.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.313+0000 7fe9cfe00640 1 -- 172.21.15.27:0/1053134047 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8018980 con 0x7fe9d8103990 2024-09-21T00:19:42.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.313+0000 7fe9d7e00640 1 -- 172.21.15.27:0/1053134047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d81a8e10 con 0x7fe9d8103990 2024-09-21T00:19:42.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.314+0000 7fe9d7e00640 1 -- 172.21.15.27:0/1053134047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d8103da0 con 0x7fe9d8103990 2024-09-21T00:19:42.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.316+0000 7fe9cfe00640 1 -- 172.21.15.27:0/1053134047 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fe9c8010430 con 0x7fe9d8103990 2024-09-21T00:19:42.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.319+0000 7fe9cfe00640 1 --2- 172.21.15.27:0/1053134047 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9bc0b6560 0x7fe9bc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:42.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.319+0000 7fe9cfe00640 1 -- 172.21.15.27:0/1053134047 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7fe9c80e7fb0 con 0x7fe9d8103990 2024-09-21T00:19:42.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.319+0000 7fe9ce000640 1 --2- 172.21.15.27:0/1053134047 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9bc0b6560 0x7fe9bc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:42.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.320+0000 7fe9ce000640 1 --2- 172.21.15.27:0/1053134047 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9bc0b6560 0x7fe9bc0b8a70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8102c50 tx=0x7fe9c0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:42.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.321+0000 7fe9cfe00640 1 -- 172.21.15.27:0/1053134047 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe9c80b2790 con 0x7fe9d8103990 2024-09-21T00:19:42.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:42 smithi089 ceph-mon[34481]: pgmap v324: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s; 321 B/s, 0 objects/s recovering 2024-09-21T00:19:42.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:42 smithi089 ceph-mon[34481]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:19:42.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:42 smithi098 ceph-mon[34725]: pgmap v324: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s; 321 B/s, 0 objects/s recovering 2024-09-21T00:19:42.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:42 smithi098 ceph-mon[34725]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:19:42.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.550+0000 7fe9d7e00640 1 -- 172.21.15.27:0/1053134047 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe9d810f720 con 0x7fe9bc0b6560 2024-09-21T00:19:42.552 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:42 smithi027 ceph-mon[30745]: pgmap v324: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s; 321 B/s, 0 objects/s recovering 2024-09-21T00:19:42.552 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:42 smithi027 ceph-mon[30745]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:19:42.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.580+0000 7fe9cfe00640 1 -- 172.21.15.27:0/1053134047 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe9d810f720 con 0x7fe9bc0b6560 2024-09-21T00:19:42.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.582+0000 7fe9d7e00640 1 -- 172.21.15.27:0/1053134047 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9bc0b6560 msgr2=0x7fe9bc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:42.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.582+0000 7fe9d7e00640 1 --2- 172.21.15.27:0/1053134047 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9bc0b6560 0x7fe9bc0b8a70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8102c50 tx=0x7fe9c0005f90 comp rx=0 tx=0).stop 2024-09-21T00:19:42.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.583+0000 7fe9d7e00640 1 -- 172.21.15.27:0/1053134047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d8103990 msgr2=0x7fe9d819c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:42.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.583+0000 7fe9d7e00640 1 --2- 172.21.15.27:0/1053134047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d8103990 0x7fe9d819c450 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800e9f0 tx=0x7fe9c800eeb0 comp rx=0 tx=0).stop 2024-09-21T00:19:42.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.583+0000 7fe9d7e00640 1 -- 172.21.15.27:0/1053134047 shutdown_connections 2024-09-21T00:19:42.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.583+0000 7fe9d7e00640 1 --2- 172.21.15.27:0/1053134047 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9bc0b6560 0x7fe9bc0b8a70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:42.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.583+0000 7fe9d7e00640 1 --2- 172.21.15.27:0/1053134047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9d8109c20 0x7fe9d81a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:42.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.583+0000 7fe9d7e00640 1 --2- 172.21.15.27:0/1053134047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9d81092d0 0x7fe9d819c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:42.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.583+0000 7fe9d7e00640 1 --2- 172.21.15.27:0/1053134047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d8103990 0x7fe9d819c450 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:42.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.583+0000 7fe9d7e00640 1 -- 172.21.15.27:0/1053134047 >> 172.21.15.27:0/1053134047 conn(0x7fe9d80ff440 msgr2=0x7fe9d8108930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:42.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.583+0000 7fe9d7e00640 1 -- 172.21.15.27:0/1053134047 shutdown_connections 2024-09-21T00:19:42.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:42.583+0000 7fe9d7e00640 1 -- 172.21.15.27:0/1053134047 wait complete. 2024-09-21T00:19:43.307 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T00:19:43.692 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.327+0000 7f1116a00640 1 -- 172.21.15.27:0/618285332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f111010b4f0 msgr2=0x7f111010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.327+0000 7f1116a00640 1 --2- 172.21.15.27:0/618285332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f111010b4f0 0x7f111010b900 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009900 tx=0x7f10fc02a3f0 comp rx=0 tx=0).stop 2024-09-21T00:19:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.327+0000 7f1116a00640 1 -- 172.21.15.27:0/618285332 shutdown_connections 2024-09-21T00:19:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.327+0000 7f1116a00640 1 --2- 172.21.15.27:0/618285332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111010be40 0x7f1110112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.327+0000 7f1116a00640 1 --2- 172.21.15.27:0/618285332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f111010b4f0 0x7f111010b900 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.327+0000 7f1116a00640 1 --2- 172.21.15.27:0/618285332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1110105bb0 0x7f1110105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.327+0000 7f1116a00640 1 -- 172.21.15.27:0/618285332 >> 172.21.15.27:0/618285332 conn(0x7f1110101680 msgr2=0x7f1110103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.327+0000 7f1116a00640 1 -- 172.21.15.27:0/618285332 shutdown_connections 2024-09-21T00:19:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.327+0000 7f1116a00640 1 -- 172.21.15.27:0/618285332 wait complete. 2024-09-21T00:19:44.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.328+0000 7f1116a00640 1 Processor -- start 2024-09-21T00:19:44.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.328+0000 7f1116a00640 1 -- start start 2024-09-21T00:19:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.329+0000 7f1116a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1110105bb0 0x7f111019c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.329+0000 7f1116a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f111010b4f0 0x7f111019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.329+0000 7f1116a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111010be40 0x7f11101a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.329+0000 7f1116a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1110115b00 con 0x7f111010b4f0 2024-09-21T00:19:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.329+0000 7f1116a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1110115980 con 0x7f111010be40 2024-09-21T00:19:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.329+0000 7f1116a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1110115c80 con 0x7f1110105bb0 2024-09-21T00:19:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.329+0000 7f1115600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1110105bb0 0x7f111019c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.329+0000 7f1115600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1110105bb0 0x7f111019c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:33644/0 (socket says 172.21.15.27:33644) 2024-09-21T00:19:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.329+0000 7f1115600640 1 -- 172.21.15.27:0/2014919196 learned_addr learned my addr 172.21.15.27:0/2014919196 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.329+0000 7f1114c00640 1 --2- 172.21.15.27:0/2014919196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f111010b4f0 0x7f111019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.329+0000 7f1116000640 1 --2- 172.21.15.27:0/2014919196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111010be40 0x7f11101a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.330+0000 7f1115600640 1 -- 172.21.15.27:0/2014919196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111010be40 msgr2=0x7f11101a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.330+0000 7f1115600640 1 --2- 172.21.15.27:0/2014919196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111010be40 0x7f11101a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.330+0000 7f1115600640 1 -- 172.21.15.27:0/2014919196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f111010b4f0 msgr2=0x7f111019ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:44.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.330+0000 7f1115600640 1 --2- 172.21.15.27:0/2014919196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f111010b4f0 0x7f111019ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:44.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.330+0000 7f1115600640 1 -- 172.21.15.27:0/2014919196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11101a2440 con 0x7f1110105bb0 2024-09-21T00:19:44.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.330+0000 7f1115600640 1 --2- 172.21.15.27:0/2014919196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1110105bb0 0x7f111019c4d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f110400e9f0 tx=0x7f110400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:44.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.330+0000 7f110e000640 1 -- 172.21.15.27:0/2014919196 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110400cd80 con 0x7f1110105bb0 2024-09-21T00:19:44.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.330+0000 7f110e000640 1 -- 172.21.15.27:0/2014919196 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11040108e0 con 0x7f1110105bb0 2024-09-21T00:19:44.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.330+0000 7f1116a00640 1 -- 172.21.15.27:0/2014919196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11101a2730 con 0x7f1110105bb0 2024-09-21T00:19:44.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.330+0000 7f110e000640 1 -- 172.21.15.27:0/2014919196 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104018980 con 0x7f1110105bb0 2024-09-21T00:19:44.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.330+0000 7f1116a00640 1 -- 172.21.15.27:0/2014919196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11101a8e90 con 0x7f1110105bb0 2024-09-21T00:19:44.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.331+0000 7f1116a00640 1 -- 172.21.15.27:0/2014919196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1110105fc0 con 0x7f1110105bb0 2024-09-21T00:19:44.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.334+0000 7f110e000640 1 -- 172.21.15.27:0/2014919196 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f1104010430 con 0x7f1110105bb0 2024-09-21T00:19:44.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.335+0000 7f110e000640 1 --2- 172.21.15.27:0/2014919196 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f10f80b6670 0x7f10f80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:44.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.336+0000 7f110e000640 1 -- 172.21.15.27:0/2014919196 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7f11040e7c40 con 0x7f1110105bb0 2024-09-21T00:19:44.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.336+0000 7f1114c00640 1 --2- 172.21.15.27:0/2014919196 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f10f80b6670 0x7f10f80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:44.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.336+0000 7f1114c00640 1 --2- 172.21.15.27:0/2014919196 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f10f80b6670 0x7f10f80b8b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009230 tx=0x7f10fc031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:44.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.339+0000 7f110e000640 1 -- 172.21.15.27:0/2014919196 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f11040b22c0 con 0x7f1110105bb0 2024-09-21T00:19:44.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:44 smithi089 ceph-mon[34481]: pgmap v325: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-21T00:19:44.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:44 smithi089 ceph-mon[34481]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:19:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:44 smithi089 ceph-mon[34481]: mgrmap e21: x(active, since 8m), standbys: z, y 2024-09-21T00:19:44.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:44 smithi098 ceph-mon[34725]: pgmap v325: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-21T00:19:44.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:44 smithi098 ceph-mon[34725]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:19:44.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:44 smithi098 ceph-mon[34725]: mgrmap e21: x(active, since 8m), standbys: z, y 2024-09-21T00:19:44.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:44 smithi027 ceph-mon[30745]: pgmap v325: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-21T00:19:44.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:44 smithi027 ceph-mon[30745]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:19:44.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:44 smithi027 ceph-mon[30745]: mgrmap e21: x(active, since 8m), standbys: z, y 2024-09-21T00:19:44.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.569+0000 7f1116a00640 1 -- 172.21.15.27:0/2014919196 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f1110063800 con 0x7f10f80b6670 2024-09-21T00:19:44.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.575+0000 7f110e000640 1 -- 172.21.15.27:0/2014919196 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1110063800 con 0x7f10f80b6670 2024-09-21T00:19:44.576 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/qa/sv_0/b5b9bcbe-08d3-410a-adbb-71ec4ab68c21 2024-09-21T00:19:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.578+0000 7f1116a00640 1 -- 172.21.15.27:0/2014919196 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f10f80b6670 msgr2=0x7f10f80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.578+0000 7f1116a00640 1 --2- 172.21.15.27:0/2014919196 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f10f80b6670 0x7f10f80b8b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009230 tx=0x7f10fc031400 comp rx=0 tx=0).stop 2024-09-21T00:19:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.578+0000 7f1116a00640 1 -- 172.21.15.27:0/2014919196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1110105bb0 msgr2=0x7f111019c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.578+0000 7f1116a00640 1 --2- 172.21.15.27:0/2014919196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1110105bb0 0x7f111019c4d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f110400e9f0 tx=0x7f110400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:19:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.578+0000 7f1116a00640 1 -- 172.21.15.27:0/2014919196 shutdown_connections 2024-09-21T00:19:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.578+0000 7f1116a00640 1 --2- 172.21.15.27:0/2014919196 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f10f80b6670 0x7f10f80b8b80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.578+0000 7f1116a00640 1 --2- 172.21.15.27:0/2014919196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111010be40 0x7f11101a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.578+0000 7f1116a00640 1 --2- 172.21.15.27:0/2014919196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f111010b4f0 0x7f111019ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.578+0000 7f1116a00640 1 --2- 172.21.15.27:0/2014919196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1110105bb0 0x7f111019c4d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.578+0000 7f1116a00640 1 -- 172.21.15.27:0/2014919196 >> 172.21.15.27:0/2014919196 conn(0x7f1110101680 msgr2=0x7f1110108b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.578+0000 7f1116a00640 1 -- 172.21.15.27:0/2014919196 shutdown_connections 2024-09-21T00:19:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:44.578+0000 7f1116a00640 1 -- 172.21.15.27:0/2014919196 wait complete. 2024-09-21T00:19:45.535 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T00:19:45.921 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:46.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:46 smithi027 ceph-mon[30745]: pgmap v326: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-09-21T00:19:46.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:46 smithi027 ceph-mon[30745]: from='client.24937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:19:46.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:46 smithi089 ceph-mon[34481]: pgmap v326: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-09-21T00:19:46.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:46 smithi089 ceph-mon[34481]: from='client.24937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:19:46.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:46 smithi098 ceph-mon[34725]: pgmap v326: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-09-21T00:19:46.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:46 smithi098 ceph-mon[34725]: from='client.24937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:19:46.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.623+0000 7fd0ae400640 1 -- 172.21.15.27:0/4185253208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a8109e10 msgr2=0x7fd0a8110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:46.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.623+0000 7fd0ae400640 1 --2- 172.21.15.27:0/4185253208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a8109e10 0x7fd0a8110640 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009ca0 tx=0x7fd0a002a8c0 comp rx=0 tx=0).stop 2024-09-21T00:19:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.623+0000 7fd0ae400640 1 -- 172.21.15.27:0/4185253208 shutdown_connections 2024-09-21T00:19:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.623+0000 7fd0ae400640 1 --2- 172.21.15.27:0/4185253208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a8109e10 0x7fd0a8110640 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.623+0000 7fd0ae400640 1 --2- 172.21.15.27:0/4185253208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0a81094c0 0x7fd0a81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.623+0000 7fd0ae400640 1 --2- 172.21.15.27:0/4185253208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0a8103b80 0x7fd0a8103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.623+0000 7fd0ae400640 1 -- 172.21.15.27:0/4185253208 >> 172.21.15.27:0/4185253208 conn(0x7fd0a80ff590 msgr2=0x7fd0a81019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.623+0000 7fd0ae400640 1 -- 172.21.15.27:0/4185253208 shutdown_connections 2024-09-21T00:19:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.623+0000 7fd0ae400640 1 -- 172.21.15.27:0/4185253208 wait complete. 2024-09-21T00:19:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.624+0000 7fd0ae400640 1 Processor -- start 2024-09-21T00:19:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.624+0000 7fd0ae400640 1 -- start start 2024-09-21T00:19:46.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.625+0000 7fd0ae400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0a8103b80 0x7fd0a819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:46.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.625+0000 7fd0ae400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0a81094c0 0x7fd0a819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:46.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.625+0000 7fd0ae400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a8109e10 0x7fd0a819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:46.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.625+0000 7fd0ae400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a8113ad0 con 0x7fd0a8109e10 2024-09-21T00:19:46.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.625+0000 7fd0ae400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a8113950 con 0x7fd0a8103b80 2024-09-21T00:19:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.625+0000 7fd0ae400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a8113c50 con 0x7fd0a81094c0 2024-09-21T00:19:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.625+0000 7fd0a7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a8109e10 0x7fd0a819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.625+0000 7fd0a7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0a8103b80 0x7fd0a819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.625+0000 7fd0a7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0a8103b80 0x7fd0a819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38306/0 (socket says 172.21.15.27:38306) 2024-09-21T00:19:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.625+0000 7fd0a7400640 1 -- 172.21.15.27:0/1307712887 learned_addr learned my addr 172.21.15.27:0/1307712887 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.625+0000 7fd0a6a00640 1 --2- 172.21.15.27:0/1307712887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0a81094c0 0x7fd0a819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:46.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.626+0000 7fd0a7400640 1 -- 172.21.15.27:0/1307712887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0a81094c0 msgr2=0x7fd0a819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:46.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.626+0000 7fd0a7400640 1 --2- 172.21.15.27:0/1307712887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0a81094c0 0x7fd0a819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:46.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.626+0000 7fd0a7400640 1 -- 172.21.15.27:0/1307712887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a8109e10 msgr2=0x7fd0a819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:46.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.626+0000 7fd0a7400640 1 --2- 172.21.15.27:0/1307712887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a8109e10 0x7fd0a819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:46.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.626+0000 7fd0a7400640 1 -- 172.21.15.27:0/1307712887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a81a03b0 con 0x7fd0a8103b80 2024-09-21T00:19:46.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.626+0000 7fd0a7e00640 1 --2- 172.21.15.27:0/1307712887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a8109e10 0x7fd0a819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:19:46.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.626+0000 7fd0a7400640 1 --2- 172.21.15.27:0/1307712887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0a8103b80 0x7fd0a819a440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd09800d950 tx=0x7fd09800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:46.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.627+0000 7fd09fe00640 1 -- 172.21.15.27:0/1307712887 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098017070 con 0x7fd0a8103b80 2024-09-21T00:19:46.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.627+0000 7fd0ae400640 1 -- 172.21.15.27:0/1307712887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a81a06a0 con 0x7fd0a8103b80 2024-09-21T00:19:46.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.627+0000 7fd09fe00640 1 -- 172.21.15.27:0/1307712887 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd098009c30 con 0x7fd0a8103b80 2024-09-21T00:19:46.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.627+0000 7fd09fe00640 1 -- 172.21.15.27:0/1307712887 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09800f620 con 0x7fd0a8103b80 2024-09-21T00:19:46.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.627+0000 7fd0ae400640 1 -- 172.21.15.27:0/1307712887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a80755f0 con 0x7fd0a8103b80 2024-09-21T00:19:46.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.628+0000 7fd0ae400640 1 -- 172.21.15.27:0/1307712887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a8103f90 con 0x7fd0a8103b80 2024-09-21T00:19:46.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.631+0000 7fd09fe00640 1 -- 172.21.15.27:0/1307712887 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fd098005030 con 0x7fd0a8103b80 2024-09-21T00:19:46.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.632+0000 7fd09fe00640 1 --2- 172.21.15.27:0/1307712887 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd08c0b6670 0x7fd08c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:46.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.633+0000 7fd09fe00640 1 -- 172.21.15.27:0/1307712887 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7fd0980e7450 con 0x7fd0a8103b80 2024-09-21T00:19:46.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.633+0000 7fd0a6a00640 1 --2- 172.21.15.27:0/1307712887 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd08c0b6670 0x7fd08c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:46.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.634+0000 7fd0a6a00640 1 --2- 172.21.15.27:0/1307712887 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd08c0b6670 0x7fd08c0b8b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8101dd0 tx=0x7fd094005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:46.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.634+0000 7fd09fe00640 1 -- 172.21.15.27:0/1307712887 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd098009de0 con 0x7fd0a8103b80 2024-09-21T00:19:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.862+0000 7fd0ae400640 1 -- 172.21.15.27:0/1307712887 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd0a8063800 con 0x7fd08c0b6670 2024-09-21T00:19:46.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.893+0000 7fd09fe00640 1 -- 172.21.15.27:0/1307712887 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd0a8063800 con 0x7fd08c0b6670 2024-09-21T00:19:46.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.896+0000 7fd0ae400640 1 -- 172.21.15.27:0/1307712887 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd08c0b6670 msgr2=0x7fd08c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:46.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.896+0000 7fd0ae400640 1 --2- 172.21.15.27:0/1307712887 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd08c0b6670 0x7fd08c0b8b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8101dd0 tx=0x7fd094005f90 comp rx=0 tx=0).stop 2024-09-21T00:19:46.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.896+0000 7fd0ae400640 1 -- 172.21.15.27:0/1307712887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0a8103b80 msgr2=0x7fd0a819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:46.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.896+0000 7fd0ae400640 1 --2- 172.21.15.27:0/1307712887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0a8103b80 0x7fd0a819a440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd09800d950 tx=0x7fd09800de10 comp rx=0 tx=0).stop 2024-09-21T00:19:46.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.896+0000 7fd0ae400640 1 -- 172.21.15.27:0/1307712887 shutdown_connections 2024-09-21T00:19:46.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.896+0000 7fd0ae400640 1 --2- 172.21.15.27:0/1307712887 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd08c0b6670 0x7fd08c0b8b80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:46.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.896+0000 7fd0ae400640 1 --2- 172.21.15.27:0/1307712887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0a8109e10 0x7fd0a819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:46.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.896+0000 7fd0ae400640 1 --2- 172.21.15.27:0/1307712887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0a81094c0 0x7fd0a819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:46.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.896+0000 7fd0ae400640 1 --2- 172.21.15.27:0/1307712887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0a8103b80 0x7fd0a819a440 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:46.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.896+0000 7fd0ae400640 1 -- 172.21.15.27:0/1307712887 >> 172.21.15.27:0/1307712887 conn(0x7fd0a80ff590 msgr2=0x7fd0a81029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:46.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.896+0000 7fd0ae400640 1 -- 172.21.15.27:0/1307712887 shutdown_connections 2024-09-21T00:19:46.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:46.896+0000 7fd0ae400640 1 -- 172.21.15.27:0/1307712887 wait complete. 2024-09-21T00:19:47.800 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T00:19:48.191 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:48.443 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:48 smithi027 ceph-mon[30745]: pgmap v327: 129 pgs: 129 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-09-21T00:19:48.443 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:48 smithi027 ceph-mon[30745]: from='client.24953 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:19:48.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:48 smithi089 ceph-mon[34481]: pgmap v327: 129 pgs: 129 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-09-21T00:19:48.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:48 smithi089 ceph-mon[34481]: from='client.24953 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:19:48.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:48 smithi098 ceph-mon[34725]: pgmap v327: 129 pgs: 129 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-09-21T00:19:48.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:48 smithi098 ceph-mon[34725]: from='client.24953 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:19:48.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.809+0000 7f9e6e800640 1 -- 172.21.15.27:0/3946569552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e6810be40 msgr2=0x7f9e68112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:48.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.809+0000 7f9e6e800640 1 --2- 172.21.15.27:0/3946569552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e6810be40 0x7f9e68112670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009ca0 tx=0x7f9e6002a8c0 comp rx=0 tx=0).stop 2024-09-21T00:19:48.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.809+0000 7f9e6e800640 1 -- 172.21.15.27:0/3946569552 shutdown_connections 2024-09-21T00:19:48.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.809+0000 7f9e6e800640 1 --2- 172.21.15.27:0/3946569552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e6810be40 0x7f9e68112670 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:48.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.809+0000 7f9e6e800640 1 --2- 172.21.15.27:0/3946569552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e6810b4f0 0x7f9e6810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:48.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.809+0000 7f9e6e800640 1 --2- 172.21.15.27:0/3946569552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e68105bb0 0x7f9e68105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:48.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.809+0000 7f9e6e800640 1 -- 172.21.15.27:0/3946569552 >> 172.21.15.27:0/3946569552 conn(0x7f9e68101680 msgr2=0x7f9e68103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:48.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.809+0000 7f9e6e800640 1 -- 172.21.15.27:0/3946569552 shutdown_connections 2024-09-21T00:19:48.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.809+0000 7f9e6e800640 1 -- 172.21.15.27:0/3946569552 wait complete. 2024-09-21T00:19:48.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.811+0000 7f9e6e800640 1 Processor -- start 2024-09-21T00:19:48.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.811+0000 7f9e6e800640 1 -- start start 2024-09-21T00:19:48.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.811+0000 7f9e6e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e68105bb0 0x7f9e6819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:48.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.811+0000 7f9e6e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e6810b4f0 0x7f9e6819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.811+0000 7f9e6e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e6810be40 0x7f9e681a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.811+0000 7f9e6e800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9e68115b00 con 0x7f9e6810be40 2024-09-21T00:19:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.811+0000 7f9e6e800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9e68115980 con 0x7f9e68105bb0 2024-09-21T00:19:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.811+0000 7f9e6e800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9e68115c80 con 0x7f9e6810b4f0 2024-09-21T00:19:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.812+0000 7f9e6d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e68105bb0 0x7f9e6819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.812+0000 7f9e6d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e68105bb0 0x7f9e6819c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38326/0 (socket says 172.21.15.27:38326) 2024-09-21T00:19:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.812+0000 7f9e6d400640 1 -- 172.21.15.27:0/2321033565 learned_addr learned my addr 172.21.15.27:0/2321033565 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.812+0000 7f9e6ca00640 1 --2- 172.21.15.27:0/2321033565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e6810b4f0 0x7f9e6819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.812+0000 7f9e6de00640 1 --2- 172.21.15.27:0/2321033565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e6810be40 0x7f9e681a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.812+0000 7f9e6d400640 1 -- 172.21.15.27:0/2321033565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e6810b4f0 msgr2=0x7f9e6819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.812+0000 7f9e6d400640 1 --2- 172.21.15.27:0/2321033565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e6810b4f0 0x7f9e6819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.812+0000 7f9e6d400640 1 -- 172.21.15.27:0/2321033565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e6810be40 msgr2=0x7f9e681a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.812+0000 7f9e6d400640 1 --2- 172.21.15.27:0/2321033565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e6810be40 0x7f9e681a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:48.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.812+0000 7f9e6d400640 1 -- 172.21.15.27:0/2321033565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e681a2420 con 0x7f9e68105bb0 2024-09-21T00:19:48.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.813+0000 7f9e6d400640 1 --2- 172.21.15.27:0/2321033565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e68105bb0 0x7f9e6819c4b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00e9f0 tx=0x7f9e5c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:48.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.813+0000 7f9e66000640 1 -- 172.21.15.27:0/2321033565 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00cd80 con 0x7f9e68105bb0 2024-09-21T00:19:48.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.813+0000 7f9e66000640 1 -- 172.21.15.27:0/2321033565 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e5c0108e0 con 0x7f9e68105bb0 2024-09-21T00:19:48.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.813+0000 7f9e66000640 1 -- 172.21.15.27:0/2321033565 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c018980 con 0x7f9e68105bb0 2024-09-21T00:19:48.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.813+0000 7f9e6e800640 1 -- 172.21.15.27:0/2321033565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e681a2710 con 0x7f9e68105bb0 2024-09-21T00:19:48.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.813+0000 7f9e6e800640 1 -- 172.21.15.27:0/2321033565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e681a8e70 con 0x7f9e68105bb0 2024-09-21T00:19:48.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.814+0000 7f9e6e800640 1 -- 172.21.15.27:0/2321033565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e68105fc0 con 0x7f9e68105bb0 2024-09-21T00:19:48.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.817+0000 7f9e66000640 1 -- 172.21.15.27:0/2321033565 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f9e5c010430 con 0x7f9e68105bb0 2024-09-21T00:19:48.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.818+0000 7f9e66000640 1 --2- 172.21.15.27:0/2321033565 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9e500b6670 0x7f9e500b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:48.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.819+0000 7f9e66000640 1 -- 172.21.15.27:0/2321033565 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7f9e5c0e8090 con 0x7f9e68105bb0 2024-09-21T00:19:48.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.819+0000 7f9e6ca00640 1 --2- 172.21.15.27:0/2321033565 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9e500b6670 0x7f9e500b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:48.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.819+0000 7f9e6ca00640 1 --2- 172.21.15.27:0/2321033565 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9e500b6670 0x7f9e500b8b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9e68104f30 tx=0x7f9e54005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:48.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:48.820+0000 7f9e66000640 1 -- 172.21.15.27:0/2321033565 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9e5c0b2710 con 0x7f9e68105bb0 2024-09-21T00:19:49.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:49.048+0000 7f9e6e800640 1 -- 172.21.15.27:0/2321033565 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f9e68063800 con 0x7f9e500b6670 2024-09-21T00:19:49.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:49.055+0000 7f9e66000640 1 -- 172.21.15.27:0/2321033565 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9e68063800 con 0x7f9e500b6670 2024-09-21T00:19:49.056 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/qa/sv_1/494a9dee-140a-4837-be70-199fec6a493d 2024-09-21T00:19:49.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:49.057+0000 7f9e6e800640 1 -- 172.21.15.27:0/2321033565 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9e500b6670 msgr2=0x7f9e500b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:49.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:49.057+0000 7f9e6e800640 1 --2- 172.21.15.27:0/2321033565 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9e500b6670 0x7f9e500b8b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9e68104f30 tx=0x7f9e54005f90 comp rx=0 tx=0).stop 2024-09-21T00:19:49.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:49.057+0000 7f9e6e800640 1 -- 172.21.15.27:0/2321033565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e68105bb0 msgr2=0x7f9e6819c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:49.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:49.057+0000 7f9e6e800640 1 --2- 172.21.15.27:0/2321033565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e68105bb0 0x7f9e6819c4b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00e9f0 tx=0x7f9e5c00eeb0 comp rx=0 tx=0).stop 2024-09-21T00:19:49.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:49.057+0000 7f9e6e800640 1 -- 172.21.15.27:0/2321033565 shutdown_connections 2024-09-21T00:19:49.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:49.057+0000 7f9e6e800640 1 --2- 172.21.15.27:0/2321033565 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9e500b6670 0x7f9e500b8b80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:49.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:49.057+0000 7f9e6e800640 1 --2- 172.21.15.27:0/2321033565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e6810be40 0x7f9e681a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:49.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:49.057+0000 7f9e6e800640 1 --2- 172.21.15.27:0/2321033565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e6810b4f0 0x7f9e6819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:49.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:49.057+0000 7f9e6e800640 1 --2- 172.21.15.27:0/2321033565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e68105bb0 0x7f9e6819c4b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:49.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:49.058+0000 7f9e6e800640 1 -- 172.21.15.27:0/2321033565 >> 172.21.15.27:0/2321033565 conn(0x7f9e68101680 msgr2=0x7f9e6810ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:49.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:49.058+0000 7f9e6e800640 1 -- 172.21.15.27:0/2321033565 shutdown_connections 2024-09-21T00:19:49.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:49.058+0000 7f9e6e800640 1 -- 172.21.15.27:0/2321033565 wait complete. 2024-09-21T00:19:49.966 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:19:50.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:50 smithi027 ceph-mon[30745]: pgmap v328: 129 pgs: 129 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-21T00:19:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:50 smithi027 ceph-mon[30745]: from='client.24959 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:19:50.352 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:50.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:50 smithi089 ceph-mon[34481]: pgmap v328: 129 pgs: 129 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-21T00:19:50.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:50 smithi089 ceph-mon[34481]: from='client.24959 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:19:50.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:50 smithi098 ceph-mon[34725]: pgmap v328: 129 pgs: 129 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-21T00:19:50.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:50 smithi098 ceph-mon[34725]: from='client.24959 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:19:50.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.956+0000 7f8bc3800640 1 -- 172.21.15.27:0/3468206554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bbc10b6c0 msgr2=0x7f8bbc10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:50.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.956+0000 7f8bc3800640 1 --2- 172.21.15.27:0/3468206554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bbc10b6c0 0x7f8bbc10bad0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009900 tx=0x7f8ba802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:19:50.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.956+0000 7f8bc3800640 1 -- 172.21.15.27:0/3468206554 shutdown_connections 2024-09-21T00:19:50.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.956+0000 7f8bc3800640 1 --2- 172.21.15.27:0/3468206554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8bbc10c010 0x7f8bbc112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:50.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.956+0000 7f8bc3800640 1 --2- 172.21.15.27:0/3468206554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bbc10b6c0 0x7f8bbc10bad0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:50.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.956+0000 7f8bc3800640 1 --2- 172.21.15.27:0/3468206554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bbc105d80 0x7f8bbc106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:50.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.956+0000 7f8bc3800640 1 -- 172.21.15.27:0/3468206554 >> 172.21.15.27:0/3468206554 conn(0x7f8bbc101810 msgr2=0x7f8bbc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:50.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.956+0000 7f8bc3800640 1 -- 172.21.15.27:0/3468206554 shutdown_connections 2024-09-21T00:19:50.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.956+0000 7f8bc3800640 1 -- 172.21.15.27:0/3468206554 wait complete. 2024-09-21T00:19:50.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.957+0000 7f8bc3800640 1 Processor -- start 2024-09-21T00:19:50.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.958+0000 7f8bc3800640 1 -- start start 2024-09-21T00:19:50.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.958+0000 7f8bc3800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8bbc105d80 0x7f8bbc19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:50.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.958+0000 7f8bc3800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bbc10b6c0 0x7f8bbc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:50.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.958+0000 7f8bc3800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bbc10c010 0x7f8bbc1a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:50.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.958+0000 7f8bc3800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc115cd0 con 0x7f8bbc10c010 2024-09-21T00:19:50.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.958+0000 7f8bc3800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc115b50 con 0x7f8bbc105d80 2024-09-21T00:19:50.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.958+0000 7f8bc3800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc115e50 con 0x7f8bbc10b6c0 2024-09-21T00:19:50.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.958+0000 7f8bc1800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bbc10c010 0x7f8bbc1a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:50.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.958+0000 7f8bbbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bbc10b6c0 0x7f8bbc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:50.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.958+0000 7f8bc0e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8bbc105d80 0x7f8bbc19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:50.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.958+0000 7f8bc1800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bbc10c010 0x7f8bbc1a1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40054/0 (socket says 172.21.15.27:40054) 2024-09-21T00:19:50.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.958+0000 7f8bbbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bbc10b6c0 0x7f8bbc19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:33692/0 (socket says 172.21.15.27:33692) 2024-09-21T00:19:50.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.958+0000 7f8bc1800640 1 -- 172.21.15.27:0/1760912172 learned_addr learned my addr 172.21.15.27:0/1760912172 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:50.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.959+0000 7f8bc1800640 1 -- 172.21.15.27:0/1760912172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bbc10b6c0 msgr2=0x7f8bbc19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:50.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.959+0000 7f8bc1800640 1 --2- 172.21.15.27:0/1760912172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bbc10b6c0 0x7f8bbc19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:50.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.959+0000 7f8bc1800640 1 -- 172.21.15.27:0/1760912172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8bbc105d80 msgr2=0x7f8bbc19c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:50.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.959+0000 7f8bc1800640 1 --2- 172.21.15.27:0/1760912172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8bbc105d80 0x7f8bbc19c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:50.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.959+0000 7f8bc1800640 1 -- 172.21.15.27:0/1760912172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bbc1a2670 con 0x7f8bbc10c010 2024-09-21T00:19:50.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.959+0000 7f8bc0e00640 1 --2- 172.21.15.27:0/1760912172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8bbc105d80 0x7f8bbc19c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:19:50.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.959+0000 7f8bc1800640 1 --2- 172.21.15.27:0/1760912172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bbc10c010 0x7f8bbc1a1f40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400efd0 tx=0x7f8bb400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:50.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.959+0000 7f8bb9600640 1 -- 172.21.15.27:0/1760912172 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4003b70 con 0x7f8bbc10c010 2024-09-21T00:19:50.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.960+0000 7f8bb9600640 1 -- 172.21.15.27:0/1760912172 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb4003d10 con 0x7f8bbc10c010 2024-09-21T00:19:50.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.960+0000 7f8bb9600640 1 -- 172.21.15.27:0/1760912172 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40108a0 con 0x7f8bbc10c010 2024-09-21T00:19:50.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.960+0000 7f8bc3800640 1 -- 172.21.15.27:0/1760912172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bbc1a2960 con 0x7f8bbc10c010 2024-09-21T00:19:50.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.960+0000 7f8bc3800640 1 -- 172.21.15.27:0/1760912172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bbc1a90c0 con 0x7f8bbc10c010 2024-09-21T00:19:50.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.961+0000 7f8bc3800640 1 -- 172.21.15.27:0/1760912172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bbc107000 con 0x7f8bbc10c010 2024-09-21T00:19:50.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.963+0000 7f8bb9600640 1 -- 172.21.15.27:0/1760912172 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f8bb4004b40 con 0x7f8bbc10c010 2024-09-21T00:19:50.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.966+0000 7f8bb9600640 1 --2- 172.21.15.27:0/1760912172 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8ba40b6670 0x7f8ba40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:50.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.966+0000 7f8bb9600640 1 -- 172.21.15.27:0/1760912172 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7f8bb40e80e0 con 0x7f8bbc10c010 2024-09-21T00:19:50.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.966+0000 7f8bc0e00640 1 --2- 172.21.15.27:0/1760912172 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8ba40b6670 0x7f8ba40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:50.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.967+0000 7f8bc0e00640 1 --2- 172.21.15.27:0/1760912172 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8ba40b6670 0x7f8ba40b8b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009fc0 tx=0x7f8bb0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:50.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:50.968+0000 7f8bb9600640 1 -- 172.21.15.27:0/1760912172 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8bb40ae380 con 0x7f8bbc10c010 2024-09-21T00:19:51.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:51.205+0000 7f8bc3800640 1 -- 172.21.15.27:0/1760912172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bbc106190 con 0x7f8bbc10c010 2024-09-21T00:19:51.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:51.206+0000 7f8bb9600640 1 -- 172.21.15.27:0/1760912172 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11893 (secure 0 0 0) 0x7f8bbc106190 con 0x7f8bbc10c010 2024-09-21T00:19:51.207 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:19:51.207 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"btime":"2024-09-21T00:19:38:403694+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:19:37.959611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24751,24727]},"id":1}]} 2024-09-21T00:19:51.208 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-09-21T00:19:51.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:51.209+0000 7f8bc3800640 1 -- 172.21.15.27:0/1760912172 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8ba40b6670 msgr2=0x7f8ba40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:51.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:51.209+0000 7f8bc3800640 1 --2- 172.21.15.27:0/1760912172 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8ba40b6670 0x7f8ba40b8b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009fc0 tx=0x7f8bb0005f90 comp rx=0 tx=0).stop 2024-09-21T00:19:51.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:51.209+0000 7f8bc3800640 1 -- 172.21.15.27:0/1760912172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bbc10c010 msgr2=0x7f8bbc1a1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:51.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:51.209+0000 7f8bc3800640 1 --2- 172.21.15.27:0/1760912172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bbc10c010 0x7f8bbc1a1f40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400efd0 tx=0x7f8bb400c6a0 comp rx=0 tx=0).stop 2024-09-21T00:19:51.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:51.209+0000 7f8bc3800640 1 -- 172.21.15.27:0/1760912172 shutdown_connections 2024-09-21T00:19:51.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:51.209+0000 7f8bc3800640 1 --2- 172.21.15.27:0/1760912172 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8ba40b6670 0x7f8ba40b8b80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:51.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:51.209+0000 7f8bc3800640 1 --2- 172.21.15.27:0/1760912172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bbc10c010 0x7f8bbc1a1f40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:51.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:51.209+0000 7f8bc3800640 1 --2- 172.21.15.27:0/1760912172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bbc10b6c0 0x7f8bbc19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:51.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:51.209+0000 7f8bc3800640 1 --2- 172.21.15.27:0/1760912172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8bbc105d80 0x7f8bbc19c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:51.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:51.209+0000 7f8bc3800640 1 -- 172.21.15.27:0/1760912172 >> 172.21.15.27:0/1760912172 conn(0x7f8bbc101810 msgr2=0x7f8bbc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:51.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:51.209+0000 7f8bc3800640 1 -- 172.21.15.27:0/1760912172 shutdown_connections 2024-09-21T00:19:51.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:51.209+0000 7f8bc3800640 1 -- 172.21.15.27:0/1760912172 wait complete. 2024-09-21T00:19:52.125 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T00:19:52.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:52 smithi089 ceph-mon[34481]: pgmap v329: 129 pgs: 129 active+clean; 633 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-09-21T00:19:52.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:52 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1760912172' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:19:52.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:52 smithi098 ceph-mon[34725]: pgmap v329: 129 pgs: 129 active+clean; 633 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-09-21T00:19:52.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:52 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1760912172' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:19:52.510 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:52.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:52 smithi027 ceph-mon[30745]: pgmap v329: 129 pgs: 129 active+clean; 633 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-09-21T00:19:52.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:52 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1760912172' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:19:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.173+0000 7f5a9ca00640 1 -- 172.21.15.27:0/2576542037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a98105d80 msgr2=0x7f5a98106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.173+0000 7f5a9ca00640 1 --2- 172.21.15.27:0/2576542037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a98105d80 0x7f5a98106190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009900 tx=0x7f5a8802a410 comp rx=0 tx=0).stop 2024-09-21T00:19:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.173+0000 7f5a9ca00640 1 -- 172.21.15.27:0/2576542037 shutdown_connections 2024-09-21T00:19:53.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.173+0000 7f5a9ca00640 1 --2- 172.21.15.27:0/2576542037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a9810c010 0x7f5a98112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:53.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.173+0000 7f5a9ca00640 1 --2- 172.21.15.27:0/2576542037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9810b6c0 0x7f5a9810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:53.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.173+0000 7f5a9ca00640 1 --2- 172.21.15.27:0/2576542037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a98105d80 0x7f5a98106190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:53.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.173+0000 7f5a9ca00640 1 -- 172.21.15.27:0/2576542037 >> 172.21.15.27:0/2576542037 conn(0x7f5a98101810 msgr2=0x7f5a98103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:53.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.173+0000 7f5a9ca00640 1 -- 172.21.15.27:0/2576542037 shutdown_connections 2024-09-21T00:19:53.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.173+0000 7f5a9ca00640 1 -- 172.21.15.27:0/2576542037 wait complete. 2024-09-21T00:19:53.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.174+0000 7f5a9ca00640 1 Processor -- start 2024-09-21T00:19:53.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.174+0000 7f5a9ca00640 1 -- start start 2024-09-21T00:19:53.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.175+0000 7f5a9ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a98105d80 0x7f5a9819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:53.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.175+0000 7f5a9ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9810b6c0 0x7f5a9819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:53.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.175+0000 7f5a9ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a9810c010 0x7f5a981a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:53.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.175+0000 7f5a9ca00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a98115d20 con 0x7f5a9810c010 2024-09-21T00:19:53.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.175+0000 7f5a9ca00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5a98115ba0 con 0x7f5a9810b6c0 2024-09-21T00:19:53.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.175+0000 7f5a9ca00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5a98115ea0 con 0x7f5a98105d80 2024-09-21T00:19:53.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.175+0000 7f5a96800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a9810c010 0x7f5a981a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:53.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.175+0000 7f5a95e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a98105d80 0x7f5a9819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:53.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.175+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9810b6c0 0x7f5a9819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:53.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.175+0000 7f5a96800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a9810c010 0x7f5a981a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46346/0 (socket says 172.21.15.27:46346) 2024-09-21T00:19:53.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.175+0000 7f5a96800640 1 -- 172.21.15.27:0/3612824256 learned_addr learned my addr 172.21.15.27:0/3612824256 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.175+0000 7f5a96800640 1 -- 172.21.15.27:0/3612824256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a98105d80 msgr2=0x7f5a9819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.175+0000 7f5a96800640 1 --2- 172.21.15.27:0/3612824256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a98105d80 0x7f5a9819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.176+0000 7f5a96800640 1 -- 172.21.15.27:0/3612824256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9810b6c0 msgr2=0x7f5a9819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.176+0000 7f5a96800640 1 --2- 172.21.15.27:0/3612824256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9810b6c0 0x7f5a9819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.176+0000 7f5a96800640 1 -- 172.21.15.27:0/3612824256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a981a2640 con 0x7f5a9810c010 2024-09-21T00:19:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.176+0000 7f5a96800640 1 --2- 172.21.15.27:0/3612824256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a9810c010 0x7f5a981a1f10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000eb10 tx=0x7f5a9000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.176+0000 7f5a8ea00640 1 -- 172.21.15.27:0/3612824256 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90004b20 con 0x7f5a9810c010 2024-09-21T00:19:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.176+0000 7f5a8ea00640 1 -- 172.21.15.27:0/3612824256 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a90004cc0 con 0x7f5a9810c010 2024-09-21T00:19:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.176+0000 7f5a8ea00640 1 -- 172.21.15.27:0/3612824256 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90010670 con 0x7f5a9810c010 2024-09-21T00:19:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.176+0000 7f5a9ca00640 1 -- 172.21.15.27:0/3612824256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a981a2930 con 0x7f5a9810c010 2024-09-21T00:19:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.176+0000 7f5a9ca00640 1 -- 172.21.15.27:0/3612824256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a981a9090 con 0x7f5a9810c010 2024-09-21T00:19:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.177+0000 7f5a9ca00640 1 -- 172.21.15.27:0/3612824256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a98106190 con 0x7f5a9810c010 2024-09-21T00:19:53.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.180+0000 7f5a8ea00640 1 -- 172.21.15.27:0/3612824256 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f5a90005000 con 0x7f5a9810c010 2024-09-21T00:19:53.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.181+0000 7f5a8ea00640 1 --2- 172.21.15.27:0/3612824256 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5a7c0b6670 0x7f5a7c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:53.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.181+0000 7f5a8ea00640 1 -- 172.21.15.27:0/3612824256 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7f5a900e8110 con 0x7f5a9810c010 2024-09-21T00:19:53.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.182+0000 7f5a95e00640 1 --2- 172.21.15.27:0/3612824256 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5a7c0b6670 0x7f5a7c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:53.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.183+0000 7f5a95e00640 1 --2- 172.21.15.27:0/3612824256 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5a7c0b6670 0x7f5a7c0b8b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5a88005ab0 tx=0x7f5a88005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:53.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.183+0000 7f5a8ea00640 1 -- 172.21.15.27:0/3612824256 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5a90056710 con 0x7f5a9810c010 2024-09-21T00:19:53.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.383+0000 7f5a9ca00640 1 -- 172.21.15.27:0/3612824256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a9819d800 con 0x7f5a9810c010 2024-09-21T00:19:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.384+0000 7f5a8ea00640 1 -- 172.21.15.27:0/3612824256 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24953 (secure 0 0 0) 0x7f5a9819d800 con 0x7f5a9810c010 2024-09-21T00:19:53.386 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:19:53.386 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":88,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","created":"2024-09-21T00:09:20.149938+0000","modified":"2024-09-21T00:19:25.824021+0000","last_up_change":"2024-09-21T00:17:43.978785+0000","last_in_change":"2024-09-21T00:17:21.325177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:13:32.718543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:18:33.992414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:18:36.817691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:18:46.707770+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63ccc094-62d5-446f-844d-1c60d41fda1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6803","nonce":354370090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6805","nonce":354370090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6809","nonce":354370090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6807","nonce":354370090}]},"public_addr":"172.21.15.27:6803/354370090","cluster_addr":"172.21.15.27:6805/354370090","heartbeat_back_addr":"172.21.15.27:6809/354370090","heartbeat_front_addr":"172.21.15.27:6807/354370090","state":["exists","up"]},{"osd":1,"uuid":"e59a8ed4-3a37-4477-a60a-4abc33d49b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6801","nonce":922528621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6803","nonce":922528621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6807","nonce":922528621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6805","nonce":922528621}]},"public_addr":"172.21.15.89:6801/922528621","cluster_addr":"172.21.15.89:6803/922528621","heartbeat_back_addr":"172.21.15.89:6807/922528621","heartbeat_front_addr":"172.21.15.89:6805/922528621","state":["exists","up"]},{"osd":2,"uuid":"6058a875-d5b2-4459-9407-2fd190cb34a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6801","nonce":257636259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6803","nonce":257636259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6807","nonce":257636259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6805","nonce":257636259}]},"public_addr":"172.21.15.98:6801/257636259","cluster_addr":"172.21.15.98:6803/257636259","heartbeat_back_addr":"172.21.15.98:6807/257636259","heartbeat_front_addr":"172.21.15.98:6805/257636259","state":["exists","up"]},{"osd":3,"uuid":"dfcdaede-e51c-4985-92af-3e296435a36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6811","nonce":821088824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6813","nonce":821088824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6817","nonce":821088824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6815","nonce":821088824}]},"public_addr":"172.21.15.27:6811/821088824","cluster_addr":"172.21.15.27:6813/821088824","heartbeat_back_addr":"172.21.15.27:6817/821088824","heartbeat_front_addr":"172.21.15.27:6815/821088824","state":["exists","up"]},{"osd":4,"uuid":"7c4af037-8ab6-47c0-b62b-76bd775b3491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6809","nonce":4001038690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6811","nonce":4001038690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6815","nonce":4001038690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6813","nonce":4001038690}]},"public_addr":"172.21.15.89:6809/4001038690","cluster_addr":"172.21.15.89:6811/4001038690","heartbeat_back_addr":"172.21.15.89:6815/4001038690","heartbeat_front_addr":"172.21.15.89:6813/4001038690","state":["exists","up"]},{"osd":5,"uuid":"e996ed0d-88ad-4aa7-891f-dfade4e09cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6809","nonce":3029903562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6811","nonce":3029903562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6815","nonce":3029903562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6813","nonce":3029903562}]},"public_addr":"172.21.15.98:6809/3029903562","cluster_addr":"172.21.15.98:6811/3029903562","heartbeat_back_addr":"172.21.15.98:6815/3029903562","heartbeat_front_addr":"172.21.15.98:6813/3029903562","state":["exists","up"]},{"osd":6,"uuid":"cf080497-57c2-42d0-8c8f-1eed0110702d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6819","nonce":1286808857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6821","nonce":1286808857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6825","nonce":1286808857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6823","nonce":1286808857}]},"public_addr":"172.21.15.27:6819/1286808857","cluster_addr":"172.21.15.27:6821/1286808857","heartbeat_back_addr":"172.21.15.27:6825/1286808857","heartbeat_front_addr":"172.21.15.27:6823/1286808857","state":["exists","up"]},{"osd":7,"uuid":"ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6817","nonce":2479349546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6819","nonce":2479349546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6823","nonce":2479349546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6821","nonce":2479349546}]},"public_addr":"172.21.15.89:6817/2479349546","cluster_addr":"172.21.15.89:6819/2479349546","heartbeat_back_addr":"172.21.15.89:6823/2479349546","heartbeat_front_addr":"172.21.15.89:6821/2479349546","state":["exists","up"]},{"osd":8,"uuid":"0b558458-fc2d-4ed1-b72a-e2ea8cf811b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6817","nonce":2830381485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6819","nonce":2830381485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6823","nonce":2830381485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6821","nonce":2830381485}]},"public_addr":"172.21.15.98:6817/2830381485","cluster_addr":"172.21.15.98:6819/2830381485","heartbeat_back_addr":"172.21.15.98:6823/2830381485","heartbeat_front_addr":"172.21.15.98:6821/2830381485","state":["exists","up"]},{"osd":9,"uuid":"1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6827","nonce":3033977113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6829","nonce":3033977113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6833","nonce":3033977113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6831","nonce":3033977113}]},"public_addr":"172.21.15.27:6827/3033977113","cluster_addr":"172.21.15.27:6829/3033977113","heartbeat_back_addr":"172.21.15.27:6833/3033977113","heartbeat_front_addr":"172.21.15.27:6831/3033977113","state":["exists","up"]},{"osd":10,"uuid":"324dd6ce-68fe-4c76-959b-1c9fb0ae92a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6825","nonce":2370332225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6827","nonce":2370332225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6831","nonce":2370332225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6829","nonce":2370332225}]},"public_addr":"172.21.15.89:6825/2370332225","cluster_addr":"172.21.15.89:6827/2370332225","heartbeat_back_addr":"172.21.15.89:6831/2370332225","heartbeat_front_addr":"172.21.15.89:6829/2370332225","state":["exists","up"]},{"osd":11,"uuid":"5e34686e-350a-491e-b9d7-59174ea90944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6825","nonce":2795999064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6827","nonce":2795999064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6831","nonce":2795999064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6829","nonce":2795999064}]},"public_addr":"172.21.15.98:6825/2795999064","cluster_addr":"172.21.15.98:6827/2795999064","heartbeat_back_addr":"172.21.15.98:6831/2795999064","heartbeat_front_addr":"172.21.15.98:6829/2795999064","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:12:39.841958+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:06.684417+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:31.041378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:01.612108+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:28.592326+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:54.303714+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:25.523475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:52.523483+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:19.430503+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:50.599120+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:17.520452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:42.725550+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/502883064":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/831048090":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/3880378905":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/793991452":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/2077956479":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6800/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6801/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/1098432608":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6800/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/528019907":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/2173601508":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:6800/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/764776233":"2024-09-22T00:11:24.231101+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:19:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.387+0000 7f5a9ca00640 1 -- 172.21.15.27:0/3612824256 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5a7c0b6670 msgr2=0x7f5a7c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.387+0000 7f5a9ca00640 1 --2- 172.21.15.27:0/3612824256 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5a7c0b6670 0x7f5a7c0b8b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5a88005ab0 tx=0x7f5a88005a20 comp rx=0 tx=0).stop 2024-09-21T00:19:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.387+0000 7f5a9ca00640 1 -- 172.21.15.27:0/3612824256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a9810c010 msgr2=0x7f5a981a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.387+0000 7f5a9ca00640 1 --2- 172.21.15.27:0/3612824256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a9810c010 0x7f5a981a1f10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000eb10 tx=0x7f5a9000c2d0 comp rx=0 tx=0).stop 2024-09-21T00:19:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.387+0000 7f5a9ca00640 1 -- 172.21.15.27:0/3612824256 shutdown_connections 2024-09-21T00:19:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.387+0000 7f5a9ca00640 1 --2- 172.21.15.27:0/3612824256 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5a7c0b6670 0x7f5a7c0b8b80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.387+0000 7f5a9ca00640 1 --2- 172.21.15.27:0/3612824256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a9810c010 0x7f5a981a1f10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.387+0000 7f5a9ca00640 1 --2- 172.21.15.27:0/3612824256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9810b6c0 0x7f5a9819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.387+0000 7f5a9ca00640 1 --2- 172.21.15.27:0/3612824256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a98105d80 0x7f5a9819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.387+0000 7f5a9ca00640 1 -- 172.21.15.27:0/3612824256 >> 172.21.15.27:0/3612824256 conn(0x7f5a98101810 msgr2=0x7f5a9810ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.387+0000 7f5a9ca00640 1 -- 172.21.15.27:0/3612824256 shutdown_connections 2024-09-21T00:19:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:53.387+0000 7f5a9ca00640 1 -- 172.21.15.27:0/3612824256 wait complete. 2024-09-21T00:19:54.275 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T00:19:54.275 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T00:19:54.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:54 smithi089 ceph-mon[34481]: pgmap v330: 129 pgs: 129 active+clean; 633 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-09-21T00:19:54.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:54 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3612824256' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:19:54.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:54 smithi098 ceph-mon[34725]: pgmap v330: 129 pgs: 129 active+clean; 633 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-09-21T00:19:54.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:54 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3612824256' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:19:54.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:54 smithi027 ceph-mon[30745]: pgmap v330: 129 pgs: 129 active+clean; 633 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-09-21T00:19:54.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:54 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3612824256' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:19:54.661 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:55.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.284+0000 7f0236000640 1 -- 172.21.15.27:0/1378878280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f023010de20 msgr2=0x7f0230110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:55.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.284+0000 7f0236000640 1 --2- 172.21.15.27:0/1378878280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f023010de20 0x7f0230110310 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0228009ca0 tx=0x7f022802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:19:55.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.284+0000 7f0236000640 1 -- 172.21.15.27:0/1378878280 shutdown_connections 2024-09-21T00:19:55.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.284+0000 7f0236000640 1 --2- 172.21.15.27:0/1378878280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f023010de20 0x7f0230110310 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:55.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.284+0000 7f0236000640 1 --2- 172.21.15.27:0/1378878280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0230105310 0x7f023010d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:55.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.284+0000 7f0236000640 1 --2- 172.21.15.27:0/1378878280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02301049c0 0x7f0230104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:55.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.284+0000 7f0236000640 1 -- 172.21.15.27:0/1378878280 >> 172.21.15.27:0/1378878280 conn(0x7f0230079730 msgr2=0x7f0230079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:55.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.284+0000 7f0236000640 1 -- 172.21.15.27:0/1378878280 shutdown_connections 2024-09-21T00:19:55.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.284+0000 7f0236000640 1 -- 172.21.15.27:0/1378878280 wait complete. 2024-09-21T00:19:55.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.286+0000 7f0236000640 1 Processor -- start 2024-09-21T00:19:55.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.286+0000 7f0236000640 1 -- start start 2024-09-21T00:19:55.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.286+0000 7f0236000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02301049c0 0x7f0230102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:55.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.286+0000 7f0236000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0230105310 0x7f0230102c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:55.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.286+0000 7f0236000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f023010de20 0x7f0230103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:55.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.286+0000 7f0236000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0230113aa0 con 0x7f02301049c0 2024-09-21T00:19:55.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.286+0000 7f0236000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0230113920 con 0x7f023010de20 2024-09-21T00:19:55.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.286+0000 7f0236000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0230113c20 con 0x7f0230105310 2024-09-21T00:19:55.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.287+0000 7f022f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02301049c0 0x7f0230102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:55.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.287+0000 7f022f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02301049c0 0x7f0230102700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46366/0 (socket says 172.21.15.27:46366) 2024-09-21T00:19:55.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.287+0000 7f022f400640 1 -- 172.21.15.27:0/3732103219 learned_addr learned my addr 172.21.15.27:0/3732103219 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:55.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.287+0000 7f0226a00640 1 --2- 172.21.15.27:0/3732103219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0230105310 0x7f0230102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:55.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.287+0000 7f022fe00640 1 --2- 172.21.15.27:0/3732103219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f023010de20 0x7f0230103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:55.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.287+0000 7f022f400640 1 -- 172.21.15.27:0/3732103219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0230105310 msgr2=0x7f0230102c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:55.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.287+0000 7f022f400640 1 --2- 172.21.15.27:0/3732103219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0230105310 0x7f0230102c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:55.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.287+0000 7f022f400640 1 -- 172.21.15.27:0/3732103219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f023010de20 msgr2=0x7f0230103180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:55.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.287+0000 7f022f400640 1 --2- 172.21.15.27:0/3732103219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f023010de20 0x7f0230103180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:55.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.287+0000 7f022f400640 1 -- 172.21.15.27:0/3732103219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02301a8810 con 0x7f02301049c0 2024-09-21T00:19:55.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.287+0000 7f0226a00640 1 --2- 172.21.15.27:0/3732103219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0230105310 0x7f0230102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:19:55.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.288+0000 7f022f400640 1 --2- 172.21.15.27:0/3732103219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02301049c0 0x7f0230102700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f022000e9c0 tx=0x7f022000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:55.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.288+0000 7f022cc00640 1 -- 172.21.15.27:0/3732103219 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022000cd80 con 0x7f02301049c0 2024-09-21T00:19:55.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.288+0000 7f022cc00640 1 -- 172.21.15.27:0/3732103219 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02200035f0 con 0x7f02301049c0 2024-09-21T00:19:55.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.288+0000 7f0236000640 1 -- 172.21.15.27:0/3732103219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02301a8b00 con 0x7f02301049c0 2024-09-21T00:19:55.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.288+0000 7f022cc00640 1 -- 172.21.15.27:0/3732103219 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220010690 con 0x7f02301049c0 2024-09-21T00:19:55.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.288+0000 7f0236000640 1 -- 172.21.15.27:0/3732103219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02301a9040 con 0x7f02301049c0 2024-09-21T00:19:55.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.289+0000 7f0236000640 1 -- 172.21.15.27:0/3732103219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f023006c1e0 con 0x7f02301049c0 2024-09-21T00:19:55.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.292+0000 7f022cc00640 1 -- 172.21.15.27:0/3732103219 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f0220004000 con 0x7f02301049c0 2024-09-21T00:19:55.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.293+0000 7f022cc00640 1 --2- 172.21.15.27:0/3732103219 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f02140b6670 0x7f02140b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:55.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.293+0000 7f022cc00640 1 -- 172.21.15.27:0/3732103219 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7f02200e7f30 con 0x7f02301049c0 2024-09-21T00:19:55.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.294+0000 7f0226a00640 1 --2- 172.21.15.27:0/3732103219 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f02140b6670 0x7f02140b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:55.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.294+0000 7f0226a00640 1 --2- 172.21.15.27:0/3732103219 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f02140b6670 0x7f02140b8b80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f023010faa0 tx=0x7f0218005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:55.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.295+0000 7f022cc00640 1 -- 172.21.15.27:0/3732103219 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f02200ae1d0 con 0x7f02301049c0 2024-09-21T00:19:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.495+0000 7f0236000640 1 -- 172.21.15.27:0/3732103219 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0230108050 con 0x7f02140b6670 2024-09-21T00:19:55.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.504+0000 7f022cc00640 1 -- 172.21.15.27:0/3732103219 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340356 (secure 0 0 0) 0x7f0230108050 con 0x7f02140b6670 2024-09-21T00:19:55.509 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:19:55.510 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T00:19:55.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.509+0000 7f0236000640 1 -- 172.21.15.27:0/3732103219 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f02140b6670 msgr2=0x7f02140b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:55.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.509+0000 7f0236000640 1 --2- 172.21.15.27:0/3732103219 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f02140b6670 0x7f02140b8b80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f023010faa0 tx=0x7f0218005f90 comp rx=0 tx=0).stop 2024-09-21T00:19:55.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.509+0000 7f0236000640 1 -- 172.21.15.27:0/3732103219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02301049c0 msgr2=0x7f0230102700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:55.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.509+0000 7f0236000640 1 --2- 172.21.15.27:0/3732103219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02301049c0 0x7f0230102700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f022000e9c0 tx=0x7f022000ee80 comp rx=0 tx=0).stop 2024-09-21T00:19:55.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.509+0000 7f0236000640 1 -- 172.21.15.27:0/3732103219 shutdown_connections 2024-09-21T00:19:55.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.509+0000 7f0236000640 1 --2- 172.21.15.27:0/3732103219 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f02140b6670 0x7f02140b8b80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:55.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.509+0000 7f0236000640 1 --2- 172.21.15.27:0/3732103219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f023010de20 0x7f0230103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:55.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.509+0000 7f0236000640 1 --2- 172.21.15.27:0/3732103219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0230105310 0x7f0230102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:55.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.509+0000 7f0236000640 1 --2- 172.21.15.27:0/3732103219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02301049c0 0x7f0230102700 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:55.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.509+0000 7f0236000640 1 -- 172.21.15.27:0/3732103219 >> 172.21.15.27:0/3732103219 conn(0x7f0230079730 msgr2=0x7f0230107940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:55.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.509+0000 7f0236000640 1 -- 172.21.15.27:0/3732103219 shutdown_connections 2024-09-21T00:19:55.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:55.509+0000 7f0236000640 1 -- 172.21.15.27:0/3732103219 wait complete. 2024-09-21T00:19:56.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:56 smithi089 ceph-mon[34481]: pgmap v331: 129 pgs: 129 active+clean; 633 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-21T00:19:56.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:56 smithi098 ceph-mon[34725]: pgmap v331: 129 pgs: 129 active+clean; 633 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-21T00:19:56.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:56 smithi027 ceph-mon[30745]: pgmap v331: 129 pgs: 129 active+clean; 633 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-21T00:19:56.621 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-21T00:19:54.346367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647837,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":201,"num_read_kb":191,"num_write":281,"num_write_kb":2254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2955704,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375604,"kb_used_data":13876,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566220,"statfs":{"total":1151940427776,"available":1151555809280,"internally_reserved":0,"allocated":14209024,"data_stored":8119421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49626,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":85,"num_write":28,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003349"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.907198+0000","last_change":"2024-09-21T00:18:35.627618+0000","last_active":"2024-09-21T00:19:33.907198+0000","last_peered":"2024-09-21T00:19:33.907198+0000","last_clean":"2024-09-21T00:19:33.907198+0000","last_became_active":"2024-09-21T00:18:35.626475+0000","last_became_peered":"2024-09-21T00:18:35.626475+0000","last_unstale":"2024-09-21T00:19:33.907198+0000","last_undegraded":"2024-09-21T00:19:33.907198+0000","last_fullsized":"2024-09-21T00:19:33.907198+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:50.379225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.841178+0000","last_change":"2024-09-21T00:18:38.665891+0000","last_active":"2024-09-21T00:19:25.841178+0000","last_peered":"2024-09-21T00:19:25.841178+0000","last_clean":"2024-09-21T00:19:25.841178+0000","last_became_active":"2024-09-21T00:18:38.665472+0000","last_became_peered":"2024-09-21T00:18:38.665472+0000","last_unstale":"2024-09-21T00:19:25.841178+0000","last_undegraded":"2024-09-21T00:19:25.841178+0000","last_fullsized":"2024-09-21T00:19:25.841178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:34:32.490828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.837644+0000","last_change":"2024-09-21T00:18:48.730206+0000","last_active":"2024-09-21T00:19:25.837644+0000","last_peered":"2024-09-21T00:19:25.837644+0000","last_clean":"2024-09-21T00:19:25.837644+0000","last_became_active":"2024-09-21T00:18:48.729729+0000","last_became_peered":"2024-09-21T00:18:48.729729+0000","last_unstale":"2024-09-21T00:19:25.837644+0000","last_undegraded":"2024-09-21T00:19:25.837644+0000","last_fullsized":"2024-09-21T00:19:25.837644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:08:17.019877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.841063+0000","last_change":"2024-09-21T00:18:35.768554+0000","last_active":"2024-09-21T00:19:25.841063+0000","last_peered":"2024-09-21T00:19:25.841063+0000","last_clean":"2024-09-21T00:19:25.841063+0000","last_became_active":"2024-09-21T00:18:35.767642+0000","last_became_peered":"2024-09-21T00:18:35.767642+0000","last_unstale":"2024-09-21T00:19:25.841063+0000","last_undegraded":"2024-09-21T00:19:25.841063+0000","last_fullsized":"2024-09-21T00:19:25.841063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:10.442974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838928+0000","last_change":"2024-09-21T00:18:38.677997+0000","last_active":"2024-09-21T00:19:25.838928+0000","last_peered":"2024-09-21T00:19:25.838928+0000","last_clean":"2024-09-21T00:19:25.838928+0000","last_became_active":"2024-09-21T00:18:38.676698+0000","last_became_peered":"2024-09-21T00:18:38.676698+0000","last_unstale":"2024-09-21T00:19:25.838928+0000","last_undegraded":"2024-09-21T00:19:25.838928+0000","last_fullsized":"2024-09-21T00:19:25.838928+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:11.662765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838302+0000","last_change":"2024-09-21T00:18:48.739563+0000","last_active":"2024-09-21T00:19:25.838302+0000","last_peered":"2024-09-21T00:19:25.838302+0000","last_clean":"2024-09-21T00:19:25.838302+0000","last_became_active":"2024-09-21T00:18:48.739092+0000","last_became_peered":"2024-09-21T00:18:48.739092+0000","last_unstale":"2024-09-21T00:19:25.838302+0000","last_undegraded":"2024-09-21T00:19:25.838302+0000","last_fullsized":"2024-09-21T00:19:25.838302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:44.185556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.901582+0000","last_change":"2024-09-21T00:18:35.626742+0000","last_active":"2024-09-21T00:19:33.901582+0000","last_peered":"2024-09-21T00:19:33.901582+0000","last_clean":"2024-09-21T00:19:33.901582+0000","last_became_active":"2024-09-21T00:18:35.626359+0000","last_became_peered":"2024-09-21T00:18:35.626359+0000","last_unstale":"2024-09-21T00:19:33.901582+0000","last_undegraded":"2024-09-21T00:19:33.901582+0000","last_fullsized":"2024-09-21T00:19:33.901582+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:38.860184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838207+0000","last_change":"2024-09-21T00:18:38.666003+0000","last_active":"2024-09-21T00:19:25.838207+0000","last_peered":"2024-09-21T00:19:25.838207+0000","last_clean":"2024-09-21T00:19:25.838207+0000","last_became_active":"2024-09-21T00:18:38.665425+0000","last_became_peered":"2024-09-21T00:18:38.665425+0000","last_unstale":"2024-09-21T00:19:25.838207+0000","last_undegraded":"2024-09-21T00:19:25.838207+0000","last_fullsized":"2024-09-21T00:19:25.838207+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:14.121864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838743+0000","last_change":"2024-09-21T00:18:48.729604+0000","last_active":"2024-09-21T00:19:25.838743+0000","last_peered":"2024-09-21T00:19:25.838743+0000","last_clean":"2024-09-21T00:19:25.838743+0000","last_became_active":"2024-09-21T00:18:48.729026+0000","last_became_peered":"2024-09-21T00:18:48.729026+0000","last_unstale":"2024-09-21T00:19:25.838743+0000","last_undegraded":"2024-09-21T00:19:25.838743+0000","last_fullsized":"2024-09-21T00:19:25.838743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:18.704373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840011+0000","last_change":"2024-09-21T00:18:35.628091+0000","last_active":"2024-09-21T00:19:25.840011+0000","last_peered":"2024-09-21T00:19:25.840011+0000","last_clean":"2024-09-21T00:19:25.840011+0000","last_became_active":"2024-09-21T00:18:35.627654+0000","last_became_peered":"2024-09-21T00:18:35.627654+0000","last_unstale":"2024-09-21T00:19:25.840011+0000","last_undegraded":"2024-09-21T00:19:25.840011+0000","last_fullsized":"2024-09-21T00:19:25.840011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:05.229036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.844254+0000","last_change":"2024-09-21T00:19:25.844116+0000","last_active":"2024-09-21T00:19:25.844254+0000","last_peered":"2024-09-21T00:19:25.844254+0000","last_clean":"2024-09-21T00:19:25.844254+0000","last_became_active":"2024-09-21T00:19:25.842745+0000","last_became_peered":"2024-09-21T00:19:25.842745+0000","last_unstale":"2024-09-21T00:19:25.844254+0000","last_undegraded":"2024-09-21T00:19:25.844254+0000","last_fullsized":"2024-09-21T00:19:25.844254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:42.651251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838220+0000","last_change":"2024-09-21T00:18:48.740106+0000","last_active":"2024-09-21T00:19:25.838220+0000","last_peered":"2024-09-21T00:19:25.838220+0000","last_clean":"2024-09-21T00:19:25.838220+0000","last_became_active":"2024-09-21T00:18:48.739299+0000","last_became_peered":"2024-09-21T00:18:48.739299+0000","last_unstale":"2024-09-21T00:19:25.838220+0000","last_undegraded":"2024-09-21T00:19:25.838220+0000","last_fullsized":"2024-09-21T00:19:25.838220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:59.058333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.832963+0000","last_change":"2024-09-21T00:18:38.667331+0000","last_active":"2024-09-21T00:19:25.832963+0000","last_peered":"2024-09-21T00:19:25.832963+0000","last_clean":"2024-09-21T00:19:25.832963+0000","last_became_active":"2024-09-21T00:18:38.666960+0000","last_became_peered":"2024-09-21T00:18:38.666960+0000","last_unstale":"2024-09-21T00:19:25.832963+0000","last_undegraded":"2024-09-21T00:19:25.832963+0000","last_fullsized":"2024-09-21T00:19:25.832963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:57.569460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.894046+0000","last_change":"2024-09-21T00:18:35.625570+0000","last_active":"2024-09-21T00:19:32.894046+0000","last_peered":"2024-09-21T00:19:32.894046+0000","last_clean":"2024-09-21T00:19:32.894046+0000","last_became_active":"2024-09-21T00:18:35.625170+0000","last_became_peered":"2024-09-21T00:18:35.625170+0000","last_unstale":"2024-09-21T00:19:32.894046+0000","last_undegraded":"2024-09-21T00:19:32.894046+0000","last_fullsized":"2024-09-21T00:19:32.894046+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:58.614258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838489+0000","last_change":"2024-09-21T00:18:48.727390+0000","last_active":"2024-09-21T00:19:25.838489+0000","last_peered":"2024-09-21T00:19:25.838489+0000","last_clean":"2024-09-21T00:19:25.838489+0000","last_became_active":"2024-09-21T00:18:48.726864+0000","last_became_peered":"2024-09-21T00:18:48.726864+0000","last_unstale":"2024-09-21T00:19:25.838489+0000","last_undegraded":"2024-09-21T00:19:25.838489+0000","last_fullsized":"2024-09-21T00:19:25.838489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:10.917069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840691+0000","last_change":"2024-09-21T00:18:38.664072+0000","last_active":"2024-09-21T00:19:25.840691+0000","last_peered":"2024-09-21T00:19:25.840691+0000","last_clean":"2024-09-21T00:19:25.840691+0000","last_became_active":"2024-09-21T00:18:38.662562+0000","last_became_peered":"2024-09-21T00:18:38.662562+0000","last_unstale":"2024-09-21T00:19:25.840691+0000","last_undegraded":"2024-09-21T00:19:25.840691+0000","last_fullsized":"2024-09-21T00:19:25.840691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.021174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.844315+0000","last_change":"2024-09-21T00:19:25.843599+0000","last_active":"2024-09-21T00:19:25.844315+0000","last_peered":"2024-09-21T00:19:25.844315+0000","last_clean":"2024-09-21T00:19:25.844315+0000","last_became_active":"2024-09-21T00:19:25.843178+0000","last_became_peered":"2024-09-21T00:19:25.843178+0000","last_unstale":"2024-09-21T00:19:25.844315+0000","last_undegraded":"2024-09-21T00:19:25.844315+0000","last_fullsized":"2024-09-21T00:19:25.844315+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:18.809857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.839617+0000","last_change":"2024-09-21T00:18:48.729940+0000","last_active":"2024-09-21T00:19:25.839617+0000","last_peered":"2024-09-21T00:19:25.839617+0000","last_clean":"2024-09-21T00:19:25.839617+0000","last_became_active":"2024-09-21T00:18:48.729465+0000","last_became_peered":"2024-09-21T00:18:48.729465+0000","last_unstale":"2024-09-21T00:19:25.839617+0000","last_undegraded":"2024-09-21T00:19:25.839617+0000","last_fullsized":"2024-09-21T00:19:25.839617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:44.566386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834875+0000","last_change":"2024-09-21T00:18:38.662510+0000","last_active":"2024-09-21T00:19:25.834875+0000","last_peered":"2024-09-21T00:19:25.834875+0000","last_clean":"2024-09-21T00:19:25.834875+0000","last_became_active":"2024-09-21T00:18:38.659788+0000","last_became_peered":"2024-09-21T00:18:38.659788+0000","last_unstale":"2024-09-21T00:19:25.834875+0000","last_undegraded":"2024-09-21T00:19:25.834875+0000","last_fullsized":"2024-09-21T00:19:25.834875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:14.532084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.897505+0000","last_change":"2024-09-21T00:18:35.622949+0000","last_active":"2024-09-21T00:19:32.897505+0000","last_peered":"2024-09-21T00:19:32.897505+0000","last_clean":"2024-09-21T00:19:32.897505+0000","last_became_active":"2024-09-21T00:18:35.621787+0000","last_became_peered":"2024-09-21T00:18:35.621787+0000","last_unstale":"2024-09-21T00:19:32.897505+0000","last_undegraded":"2024-09-21T00:19:32.897505+0000","last_fullsized":"2024-09-21T00:19:32.897505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:39:15.060123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.830466+0000","last_change":"2024-09-21T00:18:48.730711+0000","last_active":"2024-09-21T00:19:25.830466+0000","last_peered":"2024-09-21T00:19:25.830466+0000","last_clean":"2024-09-21T00:19:25.830466+0000","last_became_active":"2024-09-21T00:18:48.730287+0000","last_became_peered":"2024-09-21T00:18:48.730287+0000","last_unstale":"2024-09-21T00:19:25.830466+0000","last_undegraded":"2024-09-21T00:19:25.830466+0000","last_fullsized":"2024-09-21T00:19:25.830466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:34.444747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.839078+0000","last_change":"2024-09-21T00:18:38.669381+0000","last_active":"2024-09-21T00:19:25.839078+0000","last_peered":"2024-09-21T00:19:25.839078+0000","last_clean":"2024-09-21T00:19:25.839078+0000","last_became_active":"2024-09-21T00:18:38.668757+0000","last_became_peered":"2024-09-21T00:18:38.668757+0000","last_unstale":"2024-09-21T00:19:25.839078+0000","last_undegraded":"2024-09-21T00:19:25.839078+0000","last_fullsized":"2024-09-21T00:19:25.839078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:52.475872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.894147+0000","last_change":"2024-09-21T00:18:35.767700+0000","last_active":"2024-09-21T00:19:32.894147+0000","last_peered":"2024-09-21T00:19:32.894147+0000","last_clean":"2024-09-21T00:19:32.894147+0000","last_became_active":"2024-09-21T00:18:35.767369+0000","last_became_peered":"2024-09-21T00:18:35.767369+0000","last_unstale":"2024-09-21T00:19:32.894147+0000","last_undegraded":"2024-09-21T00:19:32.894147+0000","last_fullsized":"2024-09-21T00:19:32.894147+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:44.955659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838625+0000","last_change":"2024-09-21T00:18:48.737411+0000","last_active":"2024-09-21T00:19:25.838625+0000","last_peered":"2024-09-21T00:19:25.838625+0000","last_clean":"2024-09-21T00:19:25.838625+0000","last_became_active":"2024-09-21T00:18:48.736715+0000","last_became_peered":"2024-09-21T00:18:48.736715+0000","last_unstale":"2024-09-21T00:19:25.838625+0000","last_undegraded":"2024-09-21T00:19:25.838625+0000","last_fullsized":"2024-09-21T00:19:25.838625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:01.045318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.842485+0000","last_change":"2024-09-21T00:19:25.842355+0000","last_active":"2024-09-21T00:19:25.842485+0000","last_peered":"2024-09-21T00:19:25.842485+0000","last_clean":"2024-09-21T00:19:25.842485+0000","last_became_active":"2024-09-21T00:19:25.841961+0000","last_became_peered":"2024-09-21T00:19:25.841961+0000","last_unstale":"2024-09-21T00:19:25.842485+0000","last_undegraded":"2024-09-21T00:19:25.842485+0000","last_fullsized":"2024-09-21T00:19:25.842485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:30.490992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.895609+0000","last_change":"2024-09-21T00:18:35.919003+0000","last_active":"2024-09-21T00:19:32.895609+0000","last_peered":"2024-09-21T00:19:32.895609+0000","last_clean":"2024-09-21T00:19:32.895609+0000","last_became_active":"2024-09-21T00:18:35.918593+0000","last_became_peered":"2024-09-21T00:18:35.918593+0000","last_unstale":"2024-09-21T00:19:32.895609+0000","last_undegraded":"2024-09-21T00:19:32.895609+0000","last_fullsized":"2024-09-21T00:19:32.895609+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:35.982726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.837887+0000","last_change":"2024-09-21T00:18:48.725906+0000","last_active":"2024-09-21T00:19:25.837887+0000","last_peered":"2024-09-21T00:19:25.837887+0000","last_clean":"2024-09-21T00:19:25.837887+0000","last_became_active":"2024-09-21T00:18:48.725450+0000","last_became_peered":"2024-09-21T00:18:48.725450+0000","last_unstale":"2024-09-21T00:19:25.837887+0000","last_undegraded":"2024-09-21T00:19:25.837887+0000","last_fullsized":"2024-09-21T00:19:25.837887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:35.163208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840643+0000","last_change":"2024-09-21T00:18:38.666442+0000","last_active":"2024-09-21T00:19:25.840643+0000","last_peered":"2024-09-21T00:19:25.840643+0000","last_clean":"2024-09-21T00:19:25.840643+0000","last_became_active":"2024-09-21T00:18:38.666083+0000","last_became_peered":"2024-09-21T00:18:38.666083+0000","last_unstale":"2024-09-21T00:19:25.840643+0000","last_undegraded":"2024-09-21T00:19:25.840643+0000","last_fullsized":"2024-09-21T00:19:25.840643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:39.986440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:50.677868+0000","last_change":"2024-09-21T00:18:35.624883+0000","last_active":"2024-09-21T00:19:50.677868+0000","last_peered":"2024-09-21T00:19:50.677868+0000","last_clean":"2024-09-21T00:19:50.677868+0000","last_became_active":"2024-09-21T00:18:35.624382+0000","last_became_peered":"2024-09-21T00:18:35.624382+0000","last_unstale":"2024-09-21T00:19:50.677868+0000","last_undegraded":"2024-09-21T00:19:50.677868+0000","last_fullsized":"2024-09-21T00:19:50.677868+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:23.475521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840164+0000","last_change":"2024-09-21T00:18:48.719850+0000","last_active":"2024-09-21T00:19:25.840164+0000","last_peered":"2024-09-21T00:19:25.840164+0000","last_clean":"2024-09-21T00:19:25.840164+0000","last_became_active":"2024-09-21T00:18:48.719414+0000","last_became_peered":"2024-09-21T00:18:48.719414+0000","last_unstale":"2024-09-21T00:19:25.840164+0000","last_undegraded":"2024-09-21T00:19:25.840164+0000","last_fullsized":"2024-09-21T00:19:25.840164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:05.978977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831411+0000","last_change":"2024-09-21T00:18:38.671920+0000","last_active":"2024-09-21T00:19:25.831411+0000","last_peered":"2024-09-21T00:19:25.831411+0000","last_clean":"2024-09-21T00:19:25.831411+0000","last_became_active":"2024-09-21T00:18:38.671592+0000","last_became_peered":"2024-09-21T00:18:38.671592+0000","last_unstale":"2024-09-21T00:19:25.831411+0000","last_undegraded":"2024-09-21T00:19:25.831411+0000","last_fullsized":"2024-09-21T00:19:25.831411+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:40:48.257086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.894290+0000","last_change":"2024-09-21T00:18:35.769197+0000","last_active":"2024-09-21T00:19:32.894290+0000","last_peered":"2024-09-21T00:19:32.894290+0000","last_clean":"2024-09-21T00:19:32.894290+0000","last_became_active":"2024-09-21T00:18:35.768740+0000","last_became_peered":"2024-09-21T00:18:35.768740+0000","last_unstale":"2024-09-21T00:19:32.894290+0000","last_undegraded":"2024-09-21T00:19:32.894290+0000","last_fullsized":"2024-09-21T00:19:32.894290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:32.253006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834262+0000","last_change":"2024-09-21T00:18:48.740189+0000","last_active":"2024-09-21T00:19:25.834262+0000","last_peered":"2024-09-21T00:19:25.834262+0000","last_clean":"2024-09-21T00:19:25.834262+0000","last_became_active":"2024-09-21T00:18:48.739700+0000","last_became_peered":"2024-09-21T00:18:48.739700+0000","last_unstale":"2024-09-21T00:19:25.834262+0000","last_undegraded":"2024-09-21T00:19:25.834262+0000","last_fullsized":"2024-09-21T00:19:25.834262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:06.312160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834262+0000","last_change":"2024-09-21T00:18:38.665635+0000","last_active":"2024-09-21T00:19:25.834262+0000","last_peered":"2024-09-21T00:19:25.834262+0000","last_clean":"2024-09-21T00:19:25.834262+0000","last_became_active":"2024-09-21T00:18:38.665236+0000","last_became_peered":"2024-09-21T00:18:38.665236+0000","last_unstale":"2024-09-21T00:19:25.834262+0000","last_undegraded":"2024-09-21T00:19:25.834262+0000","last_fullsized":"2024-09-21T00:19:25.834262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:52.186480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.830142+0000","last_change":"2024-09-21T00:18:35.771465+0000","last_active":"2024-09-21T00:19:25.830142+0000","last_peered":"2024-09-21T00:19:25.830142+0000","last_clean":"2024-09-21T00:19:25.830142+0000","last_became_active":"2024-09-21T00:18:35.770550+0000","last_became_peered":"2024-09-21T00:18:35.770550+0000","last_unstale":"2024-09-21T00:19:25.830142+0000","last_undegraded":"2024-09-21T00:19:25.830142+0000","last_fullsized":"2024-09-21T00:19:25.830142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:43.042563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.839574+0000","last_change":"2024-09-21T00:18:48.736334+0000","last_active":"2024-09-21T00:19:25.839574+0000","last_peered":"2024-09-21T00:19:25.839574+0000","last_clean":"2024-09-21T00:19:25.839574+0000","last_became_active":"2024-09-21T00:18:48.735784+0000","last_became_peered":"2024-09-21T00:18:48.735784+0000","last_unstale":"2024-09-21T00:19:25.839574+0000","last_undegraded":"2024-09-21T00:19:25.839574+0000","last_fullsized":"2024-09-21T00:19:25.839574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:29.637335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833846+0000","last_change":"2024-09-21T00:18:38.674181+0000","last_active":"2024-09-21T00:19:25.833846+0000","last_peered":"2024-09-21T00:19:25.833846+0000","last_clean":"2024-09-21T00:19:25.833846+0000","last_became_active":"2024-09-21T00:18:38.673784+0000","last_became_peered":"2024-09-21T00:18:38.673784+0000","last_unstale":"2024-09-21T00:19:25.833846+0000","last_undegraded":"2024-09-21T00:19:25.833846+0000","last_fullsized":"2024-09-21T00:19:25.833846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:33.471328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833078+0000","last_change":"2024-09-21T00:18:35.628149+0000","last_active":"2024-09-21T00:19:25.833078+0000","last_peered":"2024-09-21T00:19:25.833078+0000","last_clean":"2024-09-21T00:19:25.833078+0000","last_became_active":"2024-09-21T00:18:35.627674+0000","last_became_peered":"2024-09-21T00:18:35.627674+0000","last_unstale":"2024-09-21T00:19:25.833078+0000","last_undegraded":"2024-09-21T00:19:25.833078+0000","last_fullsized":"2024-09-21T00:19:25.833078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:28:47.244288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831279+0000","last_change":"2024-09-21T00:18:48.740283+0000","last_active":"2024-09-21T00:19:25.831279+0000","last_peered":"2024-09-21T00:19:25.831279+0000","last_clean":"2024-09-21T00:19:25.831279+0000","last_became_active":"2024-09-21T00:18:48.739026+0000","last_became_peered":"2024-09-21T00:18:48.739026+0000","last_unstale":"2024-09-21T00:19:25.831279+0000","last_undegraded":"2024-09-21T00:19:25.831279+0000","last_fullsized":"2024-09-21T00:19:25.831279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:50.898116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833942+0000","last_change":"2024-09-21T00:18:38.667206+0000","last_active":"2024-09-21T00:19:25.833942+0000","last_peered":"2024-09-21T00:19:25.833942+0000","last_clean":"2024-09-21T00:19:25.833942+0000","last_became_active":"2024-09-21T00:18:38.666770+0000","last_became_peered":"2024-09-21T00:18:38.666770+0000","last_unstale":"2024-09-21T00:19:25.833942+0000","last_undegraded":"2024-09-21T00:19:25.833942+0000","last_fullsized":"2024-09-21T00:19:25.833942+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:56.332071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":62,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:51.951671+0000","last_change":"2024-09-21T00:18:35.768012+0000","last_active":"2024-09-21T00:19:51.951671+0000","last_peered":"2024-09-21T00:19:51.951671+0000","last_clean":"2024-09-21T00:19:51.951671+0000","last_became_active":"2024-09-21T00:18:35.767438+0000","last_became_peered":"2024-09-21T00:18:35.767438+0000","last_unstale":"2024-09-21T00:19:51.951671+0000","last_undegraded":"2024-09-21T00:19:51.951671+0000","last_fullsized":"2024-09-21T00:19:51.951671+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:35:40.606871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831461+0000","last_change":"2024-09-21T00:18:48.738228+0000","last_active":"2024-09-21T00:19:25.831461+0000","last_peered":"2024-09-21T00:19:25.831461+0000","last_clean":"2024-09-21T00:19:25.831461+0000","last_became_active":"2024-09-21T00:18:48.737786+0000","last_became_peered":"2024-09-21T00:18:48.737786+0000","last_unstale":"2024-09-21T00:19:25.831461+0000","last_undegraded":"2024-09-21T00:19:25.831461+0000","last_fullsized":"2024-09-21T00:19:25.831461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:52.329677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.830816+0000","last_change":"2024-09-21T00:18:38.678126+0000","last_active":"2024-09-21T00:19:25.830816+0000","last_peered":"2024-09-21T00:19:25.830816+0000","last_clean":"2024-09-21T00:19:25.830816+0000","last_became_active":"2024-09-21T00:18:38.677757+0000","last_became_peered":"2024-09-21T00:18:38.677757+0000","last_unstale":"2024-09-21T00:19:25.830816+0000","last_undegraded":"2024-09-21T00:19:25.830816+0000","last_fullsized":"2024-09-21T00:19:25.830816+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:26.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.902707+0000","last_change":"2024-09-21T00:18:35.625955+0000","last_active":"2024-09-21T00:19:33.902707+0000","last_peered":"2024-09-21T00:19:33.902707+0000","last_clean":"2024-09-21T00:19:33.902707+0000","last_became_active":"2024-09-21T00:18:35.625548+0000","last_became_peered":"2024-09-21T00:18:35.625548+0000","last_unstale":"2024-09-21T00:19:33.902707+0000","last_undegraded":"2024-09-21T00:19:33.902707+0000","last_fullsized":"2024-09-21T00:19:33.902707+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:04.163421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.835153+0000","last_change":"2024-09-21T00:18:48.740765+0000","last_active":"2024-09-21T00:19:25.835153+0000","last_peered":"2024-09-21T00:19:25.835153+0000","last_clean":"2024-09-21T00:19:25.835153+0000","last_became_active":"2024-09-21T00:18:48.740081+0000","last_became_peered":"2024-09-21T00:18:48.740081+0000","last_unstale":"2024-09-21T00:19:25.835153+0000","last_undegraded":"2024-09-21T00:19:25.835153+0000","last_fullsized":"2024-09-21T00:19:25.835153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:38.251475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840450+0000","last_change":"2024-09-21T00:18:38.667581+0000","last_active":"2024-09-21T00:19:25.840450+0000","last_peered":"2024-09-21T00:19:25.840450+0000","last_clean":"2024-09-21T00:19:25.840450+0000","last_became_active":"2024-09-21T00:18:38.665959+0000","last_became_peered":"2024-09-21T00:18:38.665959+0000","last_unstale":"2024-09-21T00:19:25.840450+0000","last_undegraded":"2024-09-21T00:19:25.840450+0000","last_fullsized":"2024-09-21T00:19:25.840450+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:52.321654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:35.920303+0000","last_change":"2024-09-21T00:18:35.624291+0000","last_active":"2024-09-21T00:19:35.920303+0000","last_peered":"2024-09-21T00:19:35.920303+0000","last_clean":"2024-09-21T00:19:35.920303+0000","last_became_active":"2024-09-21T00:18:35.623297+0000","last_became_peered":"2024-09-21T00:18:35.623297+0000","last_unstale":"2024-09-21T00:19:35.920303+0000","last_undegraded":"2024-09-21T00:19:35.920303+0000","last_fullsized":"2024-09-21T00:19:35.920303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:44.824409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":116,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.832837+0000","last_change":"2024-09-21T00:17:21.233627+0000","last_active":"2024-09-21T00:19:25.832837+0000","last_peered":"2024-09-21T00:19:25.832837+0000","last_clean":"2024-09-21T00:19:25.832837+0000","last_became_active":"2024-09-21T00:17:21.227897+0000","last_became_peered":"2024-09-21T00:17:21.227897+0000","last_unstale":"2024-09-21T00:19:25.832837+0000","last_undegraded":"2024-09-21T00:19:25.832837+0000","last_fullsized":"2024-09-21T00:19:25.832837+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:13:33.120691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:13:33.120691+0000","last_clean_scrub_stamp":"2024-09-21T00:13:33.120691+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:20.198660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833211+0000","last_change":"2024-09-21T00:18:48.730107+0000","last_active":"2024-09-21T00:19:25.833211+0000","last_peered":"2024-09-21T00:19:25.833211+0000","last_clean":"2024-09-21T00:19:25.833211+0000","last_became_active":"2024-09-21T00:18:48.729463+0000","last_became_peered":"2024-09-21T00:18:48.729463+0000","last_unstale":"2024-09-21T00:19:25.833211+0000","last_undegraded":"2024-09-21T00:19:25.833211+0000","last_fullsized":"2024-09-21T00:19:25.833211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:55.352870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833585+0000","last_change":"2024-09-21T00:18:38.671244+0000","last_active":"2024-09-21T00:19:25.833585+0000","last_peered":"2024-09-21T00:19:25.833585+0000","last_clean":"2024-09-21T00:19:25.833585+0000","last_became_active":"2024-09-21T00:18:38.670630+0000","last_became_peered":"2024-09-21T00:18:38.670630+0000","last_unstale":"2024-09-21T00:19:25.833585+0000","last_undegraded":"2024-09-21T00:19:25.833585+0000","last_fullsized":"2024-09-21T00:19:25.833585+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:23.695660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.902828+0000","last_change":"2024-09-21T00:19:25.843900+0000","last_active":"2024-09-21T00:19:33.902828+0000","last_peered":"2024-09-21T00:19:33.902828+0000","last_clean":"2024-09-21T00:19:33.902828+0000","last_became_active":"2024-09-21T00:19:25.843438+0000","last_became_peered":"2024-09-21T00:19:25.843438+0000","last_unstale":"2024-09-21T00:19:33.902828+0000","last_undegraded":"2024-09-21T00:19:33.902828+0000","last_fullsized":"2024-09-21T00:19:33.902828+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:17.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.837836+0000","last_change":"2024-09-21T00:18:48.739640+0000","last_active":"2024-09-21T00:19:25.837836+0000","last_peered":"2024-09-21T00:19:25.837836+0000","last_clean":"2024-09-21T00:19:25.837836+0000","last_became_active":"2024-09-21T00:18:48.739118+0000","last_became_peered":"2024-09-21T00:18:48.739118+0000","last_unstale":"2024-09-21T00:19:25.837836+0000","last_undegraded":"2024-09-21T00:19:25.837836+0000","last_fullsized":"2024-09-21T00:19:25.837836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:03.093561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833270+0000","last_change":"2024-09-21T00:18:38.673020+0000","last_active":"2024-09-21T00:19:25.833270+0000","last_peered":"2024-09-21T00:19:25.833270+0000","last_clean":"2024-09-21T00:19:25.833270+0000","last_became_active":"2024-09-21T00:18:38.672391+0000","last_became_peered":"2024-09-21T00:18:38.672391+0000","last_unstale":"2024-09-21T00:19:25.833270+0000","last_undegraded":"2024-09-21T00:19:25.833270+0000","last_fullsized":"2024-09-21T00:19:25.833270+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:13.866576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.835916+0000","last_change":"2024-09-21T00:18:35.615844+0000","last_active":"2024-09-21T00:19:25.835916+0000","last_peered":"2024-09-21T00:19:25.835916+0000","last_clean":"2024-09-21T00:19:25.835916+0000","last_became_active":"2024-09-21T00:18:35.615385+0000","last_became_peered":"2024-09-21T00:18:35.615385+0000","last_unstale":"2024-09-21T00:19:25.835916+0000","last_undegraded":"2024-09-21T00:19:25.835916+0000","last_fullsized":"2024-09-21T00:19:25.835916+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:03.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838419+0000","last_change":"2024-09-21T00:18:48.732919+0000","last_active":"2024-09-21T00:19:25.838419+0000","last_peered":"2024-09-21T00:19:25.838419+0000","last_clean":"2024-09-21T00:19:25.838419+0000","last_became_active":"2024-09-21T00:18:48.732407+0000","last_became_peered":"2024-09-21T00:18:48.732407+0000","last_unstale":"2024-09-21T00:19:25.838419+0000","last_undegraded":"2024-09-21T00:19:25.838419+0000","last_fullsized":"2024-09-21T00:19:25.838419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:24:37.299373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833450+0000","last_change":"2024-09-21T00:18:38.671280+0000","last_active":"2024-09-21T00:19:25.833450+0000","last_peered":"2024-09-21T00:19:25.833450+0000","last_clean":"2024-09-21T00:19:25.833450+0000","last_became_active":"2024-09-21T00:18:38.670770+0000","last_became_peered":"2024-09-21T00:18:38.670770+0000","last_unstale":"2024-09-21T00:19:25.833450+0000","last_undegraded":"2024-09-21T00:19:25.833450+0000","last_fullsized":"2024-09-21T00:19:25.833450+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:06.809450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831651+0000","last_change":"2024-09-21T00:18:35.626071+0000","last_active":"2024-09-21T00:19:25.831651+0000","last_peered":"2024-09-21T00:19:25.831651+0000","last_clean":"2024-09-21T00:19:25.831651+0000","last_became_active":"2024-09-21T00:18:35.625649+0000","last_became_peered":"2024-09-21T00:18:35.625649+0000","last_unstale":"2024-09-21T00:19:25.831651+0000","last_undegraded":"2024-09-21T00:19:25.831651+0000","last_fullsized":"2024-09-21T00:19:25.831651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:16:27.516303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833806+0000","last_change":"2024-09-21T00:18:48.728451+0000","last_active":"2024-09-21T00:19:25.833806+0000","last_peered":"2024-09-21T00:19:25.833806+0000","last_clean":"2024-09-21T00:19:25.833806+0000","last_became_active":"2024-09-21T00:18:48.727840+0000","last_became_peered":"2024-09-21T00:18:48.727840+0000","last_unstale":"2024-09-21T00:19:25.833806+0000","last_undegraded":"2024-09-21T00:19:25.833806+0000","last_fullsized":"2024-09-21T00:19:25.833806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:13.457138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.835153+0000","last_change":"2024-09-21T00:18:38.664090+0000","last_active":"2024-09-21T00:19:25.835153+0000","last_peered":"2024-09-21T00:19:25.835153+0000","last_clean":"2024-09-21T00:19:25.835153+0000","last_became_active":"2024-09-21T00:18:38.663728+0000","last_became_peered":"2024-09-21T00:18:38.663728+0000","last_unstale":"2024-09-21T00:19:25.835153+0000","last_undegraded":"2024-09-21T00:19:25.835153+0000","last_fullsized":"2024-09-21T00:19:25.835153+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:46.367766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:35.897937+0000","last_change":"2024-09-21T00:19:25.847280+0000","last_active":"2024-09-21T00:19:35.897937+0000","last_peered":"2024-09-21T00:19:35.897937+0000","last_clean":"2024-09-21T00:19:35.897937+0000","last_became_active":"2024-09-21T00:19:25.840919+0000","last_became_peered":"2024-09-21T00:19:25.840919+0000","last_unstale":"2024-09-21T00:19:35.897937+0000","last_undegraded":"2024-09-21T00:19:35.897937+0000","last_fullsized":"2024-09-21T00:19:35.897937+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:15.411674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834153+0000","last_change":"2024-09-21T00:18:48.732427+0000","last_active":"2024-09-21T00:19:25.834153+0000","last_peered":"2024-09-21T00:19:25.834153+0000","last_clean":"2024-09-21T00:19:25.834153+0000","last_became_active":"2024-09-21T00:18:48.732078+0000","last_became_peered":"2024-09-21T00:18:48.732078+0000","last_unstale":"2024-09-21T00:19:25.834153+0000","last_undegraded":"2024-09-21T00:19:25.834153+0000","last_fullsized":"2024-09-21T00:19:25.834153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:35.055680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834003+0000","last_change":"2024-09-21T00:18:38.675022+0000","last_active":"2024-09-21T00:19:25.834003+0000","last_peered":"2024-09-21T00:19:25.834003+0000","last_clean":"2024-09-21T00:19:25.834003+0000","last_became_active":"2024-09-21T00:18:38.674650+0000","last_became_peered":"2024-09-21T00:18:38.674650+0000","last_unstale":"2024-09-21T00:19:25.834003+0000","last_undegraded":"2024-09-21T00:19:25.834003+0000","last_fullsized":"2024-09-21T00:19:25.834003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:05.191404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.904121+0000","last_change":"2024-09-21T00:18:35.769555+0000","last_active":"2024-09-21T00:19:33.904121+0000","last_peered":"2024-09-21T00:19:33.904121+0000","last_clean":"2024-09-21T00:19:33.904121+0000","last_became_active":"2024-09-21T00:18:35.769080+0000","last_became_peered":"2024-09-21T00:18:35.769080+0000","last_unstale":"2024-09-21T00:19:33.904121+0000","last_undegraded":"2024-09-21T00:19:33.904121+0000","last_fullsized":"2024-09-21T00:19:33.904121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:19.306830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831650+0000","last_change":"2024-09-21T00:18:48.742957+0000","last_active":"2024-09-21T00:19:25.831650+0000","last_peered":"2024-09-21T00:19:25.831650+0000","last_clean":"2024-09-21T00:19:25.831650+0000","last_became_active":"2024-09-21T00:18:48.742511+0000","last_became_peered":"2024-09-21T00:18:48.742511+0000","last_unstale":"2024-09-21T00:19:25.831650+0000","last_undegraded":"2024-09-21T00:19:25.831650+0000","last_fullsized":"2024-09-21T00:19:25.831650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:15.231189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.832619+0000","last_change":"2024-09-21T00:18:38.666576+0000","last_active":"2024-09-21T00:19:25.832619+0000","last_peered":"2024-09-21T00:19:25.832619+0000","last_clean":"2024-09-21T00:19:25.832619+0000","last_became_active":"2024-09-21T00:18:38.665643+0000","last_became_peered":"2024-09-21T00:18:38.665643+0000","last_unstale":"2024-09-21T00:19:25.832619+0000","last_undegraded":"2024-09-21T00:19:25.832619+0000","last_fullsized":"2024-09-21T00:19:25.832619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:35.397847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":62,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:49.948942+0000","last_change":"2024-09-21T00:18:35.623143+0000","last_active":"2024-09-21T00:19:49.948942+0000","last_peered":"2024-09-21T00:19:49.948942+0000","last_clean":"2024-09-21T00:19:49.948942+0000","last_became_active":"2024-09-21T00:18:35.622693+0000","last_became_peered":"2024-09-21T00:18:35.622693+0000","last_unstale":"2024-09-21T00:19:49.948942+0000","last_undegraded":"2024-09-21T00:19:49.948942+0000","last_fullsized":"2024-09-21T00:19:49.948942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:01.331665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.830550+0000","last_change":"2024-09-21T00:18:48.723877+0000","last_active":"2024-09-21T00:19:25.830550+0000","last_peered":"2024-09-21T00:19:25.830550+0000","last_clean":"2024-09-21T00:19:25.830550+0000","last_became_active":"2024-09-21T00:18:48.723360+0000","last_became_peered":"2024-09-21T00:18:48.723360+0000","last_unstale":"2024-09-21T00:19:25.830550+0000","last_undegraded":"2024-09-21T00:19:25.830550+0000","last_fullsized":"2024-09-21T00:19:25.830550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:07:20.442867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833264+0000","last_change":"2024-09-21T00:18:38.669643+0000","last_active":"2024-09-21T00:19:25.833264+0000","last_peered":"2024-09-21T00:19:25.833264+0000","last_clean":"2024-09-21T00:19:25.833264+0000","last_became_active":"2024-09-21T00:18:38.669245+0000","last_became_peered":"2024-09-21T00:18:38.669245+0000","last_unstale":"2024-09-21T00:19:25.833264+0000","last_undegraded":"2024-09-21T00:19:25.833264+0000","last_fullsized":"2024-09-21T00:19:25.833264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:17.789291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.897675+0000","last_change":"2024-09-21T00:18:35.623189+0000","last_active":"2024-09-21T00:19:32.897675+0000","last_peered":"2024-09-21T00:19:32.897675+0000","last_clean":"2024-09-21T00:19:32.897675+0000","last_became_active":"2024-09-21T00:18:35.622748+0000","last_became_peered":"2024-09-21T00:18:35.622748+0000","last_unstale":"2024-09-21T00:19:32.897675+0000","last_undegraded":"2024-09-21T00:19:32.897675+0000","last_fullsized":"2024-09-21T00:19:32.897675+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:15.570789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834463+0000","last_change":"2024-09-21T00:18:48.739612+0000","last_active":"2024-09-21T00:19:25.834463+0000","last_peered":"2024-09-21T00:19:25.834463+0000","last_clean":"2024-09-21T00:19:25.834463+0000","last_became_active":"2024-09-21T00:18:48.739173+0000","last_became_peered":"2024-09-21T00:18:48.739173+0000","last_unstale":"2024-09-21T00:19:25.834463+0000","last_undegraded":"2024-09-21T00:19:25.834463+0000","last_fullsized":"2024-09-21T00:19:25.834463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:26.327178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831104+0000","last_change":"2024-09-21T00:18:38.670876+0000","last_active":"2024-09-21T00:19:25.831104+0000","last_peered":"2024-09-21T00:19:25.831104+0000","last_clean":"2024-09-21T00:19:25.831104+0000","last_became_active":"2024-09-21T00:18:38.670371+0000","last_became_peered":"2024-09-21T00:18:38.670371+0000","last_unstale":"2024-09-21T00:19:25.831104+0000","last_undegraded":"2024-09-21T00:19:25.831104+0000","last_fullsized":"2024-09-21T00:19:25.831104+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:48.641071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.904808+0000","last_change":"2024-09-21T00:18:35.622919+0000","last_active":"2024-09-21T00:19:33.904808+0000","last_peered":"2024-09-21T00:19:33.904808+0000","last_clean":"2024-09-21T00:19:33.904808+0000","last_became_active":"2024-09-21T00:18:35.622435+0000","last_became_peered":"2024-09-21T00:18:35.622435+0000","last_unstale":"2024-09-21T00:19:33.904808+0000","last_undegraded":"2024-09-21T00:19:33.904808+0000","last_fullsized":"2024-09-21T00:19:33.904808+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:22.113871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.835031+0000","last_change":"2024-09-21T00:18:48.741736+0000","last_active":"2024-09-21T00:19:25.835031+0000","last_peered":"2024-09-21T00:19:25.835031+0000","last_clean":"2024-09-21T00:19:25.835031+0000","last_became_active":"2024-09-21T00:18:48.741135+0000","last_became_peered":"2024-09-21T00:18:48.741135+0000","last_unstale":"2024-09-21T00:19:25.835031+0000","last_undegraded":"2024-09-21T00:19:25.835031+0000","last_fullsized":"2024-09-21T00:19:25.835031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:50.492351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840391+0000","last_change":"2024-09-21T00:18:38.667377+0000","last_active":"2024-09-21T00:19:25.840391+0000","last_peered":"2024-09-21T00:19:25.840391+0000","last_clean":"2024-09-21T00:19:25.840391+0000","last_became_active":"2024-09-21T00:18:38.667010+0000","last_became_peered":"2024-09-21T00:18:38.667010+0000","last_unstale":"2024-09-21T00:19:25.840391+0000","last_undegraded":"2024-09-21T00:19:25.840391+0000","last_fullsized":"2024-09-21T00:19:25.840391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:06.634767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'26","reported_seq":120,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:49.949704+0000","last_change":"2024-09-21T00:19:25.853603+0000","last_active":"2024-09-21T00:19:49.949704+0000","last_peered":"2024-09-21T00:19:49.949704+0000","last_clean":"2024-09-21T00:19:49.949704+0000","last_became_active":"2024-09-21T00:19:25.841126+0000","last_became_peered":"2024-09-21T00:19:25.841126+0000","last_unstale":"2024-09-21T00:19:49.949704+0000","last_undegraded":"2024-09-21T00:19:49.949704+0000","last_fullsized":"2024-09-21T00:19:49.949704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:27:47.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53234,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":70,"num_write":27,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.829810+0000","last_change":"2024-09-21T00:18:48.731004+0000","last_active":"2024-09-21T00:19:25.829810+0000","last_peered":"2024-09-21T00:19:25.829810+0000","last_clean":"2024-09-21T00:19:25.829810+0000","last_became_active":"2024-09-21T00:18:48.730549+0000","last_became_peered":"2024-09-21T00:18:48.730549+0000","last_unstale":"2024-09-21T00:19:25.829810+0000","last_undegraded":"2024-09-21T00:19:25.829810+0000","last_fullsized":"2024-09-21T00:19:25.829810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:30.988300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.843618+0000","last_change":"2024-09-21T00:19:25.843532+0000","last_active":"2024-09-21T00:19:25.843618+0000","last_peered":"2024-09-21T00:19:25.843618+0000","last_clean":"2024-09-21T00:19:25.843618+0000","last_became_active":"2024-09-21T00:19:25.843065+0000","last_became_peered":"2024-09-21T00:19:25.843065+0000","last_unstale":"2024-09-21T00:19:25.843618+0000","last_undegraded":"2024-09-21T00:19:25.843618+0000","last_fullsized":"2024-09-21T00:19:25.843618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:36:26.488692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.894355+0000","last_change":"2024-09-21T00:18:35.770494+0000","last_active":"2024-09-21T00:19:32.894355+0000","last_peered":"2024-09-21T00:19:32.894355+0000","last_clean":"2024-09-21T00:19:32.894355+0000","last_became_active":"2024-09-21T00:18:35.770122+0000","last_became_peered":"2024-09-21T00:18:35.770122+0000","last_unstale":"2024-09-21T00:19:32.894355+0000","last_undegraded":"2024-09-21T00:19:32.894355+0000","last_fullsized":"2024-09-21T00:19:32.894355+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:41.415904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838481+0000","last_change":"2024-09-21T00:18:48.736802+0000","last_active":"2024-09-21T00:19:25.838481+0000","last_peered":"2024-09-21T00:19:25.838481+0000","last_clean":"2024-09-21T00:19:25.838481+0000","last_became_active":"2024-09-21T00:18:48.736008+0000","last_became_peered":"2024-09-21T00:18:48.736008+0000","last_unstale":"2024-09-21T00:19:25.838481+0000","last_undegraded":"2024-09-21T00:19:25.838481+0000","last_fullsized":"2024-09-21T00:19:25.838481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:07.995614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838989+0000","last_change":"2024-09-21T00:18:38.678385+0000","last_active":"2024-09-21T00:19:25.838989+0000","last_peered":"2024-09-21T00:19:25.838989+0000","last_clean":"2024-09-21T00:19:25.838989+0000","last_became_active":"2024-09-21T00:18:38.677174+0000","last_became_peered":"2024-09-21T00:18:38.677174+0000","last_unstale":"2024-09-21T00:19:25.838989+0000","last_undegraded":"2024-09-21T00:19:25.838989+0000","last_fullsized":"2024-09-21T00:19:25.838989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:23.486222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.893717+0000","last_change":"2024-09-21T00:18:35.627703+0000","last_active":"2024-09-21T00:19:32.893717+0000","last_peered":"2024-09-21T00:19:32.893717+0000","last_clean":"2024-09-21T00:19:32.893717+0000","last_became_active":"2024-09-21T00:18:35.627274+0000","last_became_peered":"2024-09-21T00:18:35.627274+0000","last_unstale":"2024-09-21T00:19:32.893717+0000","last_undegraded":"2024-09-21T00:19:32.893717+0000","last_fullsized":"2024-09-21T00:19:32.893717+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:14.326810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.829532+0000","last_change":"2024-09-21T00:18:48.729926+0000","last_active":"2024-09-21T00:19:25.829532+0000","last_peered":"2024-09-21T00:19:25.829532+0000","last_clean":"2024-09-21T00:19:25.829532+0000","last_became_active":"2024-09-21T00:18:48.728366+0000","last_became_peered":"2024-09-21T00:18:48.728366+0000","last_unstale":"2024-09-21T00:19:25.829532+0000","last_undegraded":"2024-09-21T00:19:25.829532+0000","last_fullsized":"2024-09-21T00:19:25.829532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:48:50.005748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838880+0000","last_change":"2024-09-21T00:18:38.664173+0000","last_active":"2024-09-21T00:19:25.838880+0000","last_peered":"2024-09-21T00:19:25.838880+0000","last_clean":"2024-09-21T00:19:25.838880+0000","last_became_active":"2024-09-21T00:18:38.663358+0000","last_became_peered":"2024-09-21T00:18:38.663358+0000","last_unstale":"2024-09-21T00:19:25.838880+0000","last_undegraded":"2024-09-21T00:19:25.838880+0000","last_fullsized":"2024-09-21T00:19:25.838880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:34:31.203621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":59,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:48.945189+0000","last_change":"2024-09-21T00:18:35.622133+0000","last_active":"2024-09-21T00:19:48.945189+0000","last_peered":"2024-09-21T00:19:48.945189+0000","last_clean":"2024-09-21T00:19:48.945189+0000","last_became_active":"2024-09-21T00:18:35.621032+0000","last_became_peered":"2024-09-21T00:18:35.621032+0000","last_unstale":"2024-09-21T00:19:48.945189+0000","last_undegraded":"2024-09-21T00:19:48.945189+0000","last_fullsized":"2024-09-21T00:19:48.945189+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:24.121665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840153+0000","last_change":"2024-09-21T00:18:48.733339+0000","last_active":"2024-09-21T00:19:25.840153+0000","last_peered":"2024-09-21T00:19:25.840153+0000","last_clean":"2024-09-21T00:19:25.840153+0000","last_became_active":"2024-09-21T00:18:48.731843+0000","last_became_peered":"2024-09-21T00:18:48.731843+0000","last_unstale":"2024-09-21T00:19:25.840153+0000","last_undegraded":"2024-09-21T00:19:25.840153+0000","last_fullsized":"2024-09-21T00:19:25.840153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:42:55.474197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838543+0000","last_change":"2024-09-21T00:18:48.740108+0000","last_active":"2024-09-21T00:19:25.838543+0000","last_peered":"2024-09-21T00:19:25.838543+0000","last_clean":"2024-09-21T00:19:25.838543+0000","last_became_active":"2024-09-21T00:18:48.739330+0000","last_became_peered":"2024-09-21T00:18:48.739330+0000","last_unstale":"2024-09-21T00:19:25.838543+0000","last_undegraded":"2024-09-21T00:19:25.838543+0000","last_fullsized":"2024-09-21T00:19:25.838543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:00.103502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.829715+0000","last_change":"2024-09-21T00:18:38.660469+0000","last_active":"2024-09-21T00:19:25.829715+0000","last_peered":"2024-09-21T00:19:25.829715+0000","last_clean":"2024-09-21T00:19:25.829715+0000","last_became_active":"2024-09-21T00:18:38.660138+0000","last_became_peered":"2024-09-21T00:18:38.660138+0000","last_unstale":"2024-09-21T00:19:25.829715+0000","last_undegraded":"2024-09-21T00:19:25.829715+0000","last_fullsized":"2024-09-21T00:19:25.829715+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:01.079112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834134+0000","last_change":"2024-09-21T00:18:35.624528+0000","last_active":"2024-09-21T00:19:25.834134+0000","last_peered":"2024-09-21T00:19:25.834134+0000","last_clean":"2024-09-21T00:19:25.834134+0000","last_became_active":"2024-09-21T00:18:35.624170+0000","last_became_peered":"2024-09-21T00:18:35.624170+0000","last_unstale":"2024-09-21T00:19:25.834134+0000","last_undegraded":"2024-09-21T00:19:25.834134+0000","last_fullsized":"2024-09-21T00:19:25.834134+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:07.914521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834318+0000","last_change":"2024-09-21T00:18:48.719277+0000","last_active":"2024-09-21T00:19:25.834318+0000","last_peered":"2024-09-21T00:19:25.834318+0000","last_clean":"2024-09-21T00:19:25.834318+0000","last_became_active":"2024-09-21T00:18:48.718412+0000","last_became_peered":"2024-09-21T00:18:48.718412+0000","last_unstale":"2024-09-21T00:19:25.834318+0000","last_undegraded":"2024-09-21T00:19:25.834318+0000","last_fullsized":"2024-09-21T00:19:25.834318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:33.741043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.841082+0000","last_change":"2024-09-21T00:18:38.666928+0000","last_active":"2024-09-21T00:19:25.841082+0000","last_peered":"2024-09-21T00:19:25.841082+0000","last_clean":"2024-09-21T00:19:25.841082+0000","last_became_active":"2024-09-21T00:18:38.666236+0000","last_became_peered":"2024-09-21T00:18:38.666236+0000","last_unstale":"2024-09-21T00:19:25.841082+0000","last_undegraded":"2024-09-21T00:19:25.841082+0000","last_fullsized":"2024-09-21T00:19:25.841082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:21.043778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.906573+0000","last_change":"2024-09-21T00:18:35.624514+0000","last_active":"2024-09-21T00:19:33.906573+0000","last_peered":"2024-09-21T00:19:33.906573+0000","last_clean":"2024-09-21T00:19:33.906573+0000","last_became_active":"2024-09-21T00:18:35.624098+0000","last_became_peered":"2024-09-21T00:18:35.624098+0000","last_unstale":"2024-09-21T00:19:33.906573+0000","last_undegraded":"2024-09-21T00:19:33.906573+0000","last_fullsized":"2024-09-21T00:19:33.906573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:21.777568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.905229+0000","last_change":"2024-09-21T00:19:25.841232+0000","last_active":"2024-09-21T00:19:33.905229+0000","last_peered":"2024-09-21T00:19:33.905229+0000","last_clean":"2024-09-21T00:19:33.905229+0000","last_became_active":"2024-09-21T00:19:25.840655+0000","last_became_peered":"2024-09-21T00:19:25.840655+0000","last_unstale":"2024-09-21T00:19:33.905229+0000","last_undegraded":"2024-09-21T00:19:33.905229+0000","last_fullsized":"2024-09-21T00:19:33.905229+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:05.846222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834868+0000","last_change":"2024-09-21T00:18:38.667466+0000","last_active":"2024-09-21T00:19:25.834868+0000","last_peered":"2024-09-21T00:19:25.834868+0000","last_clean":"2024-09-21T00:19:25.834868+0000","last_became_active":"2024-09-21T00:18:38.666960+0000","last_became_peered":"2024-09-21T00:18:38.666960+0000","last_unstale":"2024-09-21T00:19:25.834868+0000","last_undegraded":"2024-09-21T00:19:25.834868+0000","last_fullsized":"2024-09-21T00:19:25.834868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:29:56.614766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.837614+0000","last_change":"2024-09-21T00:18:48.739656+0000","last_active":"2024-09-21T00:19:25.837614+0000","last_peered":"2024-09-21T00:19:25.837614+0000","last_clean":"2024-09-21T00:19:25.837614+0000","last_became_active":"2024-09-21T00:18:48.739228+0000","last_became_peered":"2024-09-21T00:18:48.739228+0000","last_unstale":"2024-09-21T00:19:25.837614+0000","last_undegraded":"2024-09-21T00:19:25.837614+0000","last_fullsized":"2024-09-21T00:19:25.837614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:11.427595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:49.947580+0000","last_change":"2024-09-21T00:18:35.624108+0000","last_active":"2024-09-21T00:19:49.947580+0000","last_peered":"2024-09-21T00:19:49.947580+0000","last_clean":"2024-09-21T00:19:49.947580+0000","last_became_active":"2024-09-21T00:18:35.623708+0000","last_became_peered":"2024-09-21T00:18:35.623708+0000","last_unstale":"2024-09-21T00:19:49.947580+0000","last_undegraded":"2024-09-21T00:19:49.947580+0000","last_fullsized":"2024-09-21T00:19:49.947580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:50.020662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.839902+0000","last_change":"2024-09-21T00:18:38.665402+0000","last_active":"2024-09-21T00:19:25.839902+0000","last_peered":"2024-09-21T00:19:25.839902+0000","last_clean":"2024-09-21T00:19:25.839902+0000","last_became_active":"2024-09-21T00:18:38.665034+0000","last_became_peered":"2024-09-21T00:18:38.665034+0000","last_unstale":"2024-09-21T00:19:25.839902+0000","last_undegraded":"2024-09-21T00:19:25.839902+0000","last_fullsized":"2024-09-21T00:19:25.839902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:33:10.824705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838413+0000","last_change":"2024-09-21T00:18:48.729928+0000","last_active":"2024-09-21T00:19:25.838413+0000","last_peered":"2024-09-21T00:19:25.838413+0000","last_clean":"2024-09-21T00:19:25.838413+0000","last_became_active":"2024-09-21T00:18:48.729498+0000","last_became_peered":"2024-09-21T00:18:48.729498+0000","last_unstale":"2024-09-21T00:19:25.838413+0000","last_undegraded":"2024-09-21T00:19:25.838413+0000","last_fullsized":"2024-09-21T00:19:25.838413+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:10:27.140012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834147+0000","last_change":"2024-09-21T00:18:38.662221+0000","last_active":"2024-09-21T00:19:25.834147+0000","last_peered":"2024-09-21T00:19:25.834147+0000","last_clean":"2024-09-21T00:19:25.834147+0000","last_became_active":"2024-09-21T00:18:38.660543+0000","last_became_peered":"2024-09-21T00:18:38.660543+0000","last_unstale":"2024-09-21T00:19:25.834147+0000","last_undegraded":"2024-09-21T00:19:25.834147+0000","last_fullsized":"2024-09-21T00:19:25.834147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:43.159699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838253+0000","last_change":"2024-09-21T00:18:38.672696+0000","last_active":"2024-09-21T00:19:25.838253+0000","last_peered":"2024-09-21T00:19:25.838253+0000","last_clean":"2024-09-21T00:19:25.838253+0000","last_became_active":"2024-09-21T00:18:38.672356+0000","last_became_peered":"2024-09-21T00:18:38.672356+0000","last_unstale":"2024-09-21T00:19:25.838253+0000","last_undegraded":"2024-09-21T00:19:25.838253+0000","last_fullsized":"2024-09-21T00:19:25.838253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.549939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838037+0000","last_change":"2024-09-21T00:18:38.667017+0000","last_active":"2024-09-21T00:19:25.838037+0000","last_peered":"2024-09-21T00:19:25.838037+0000","last_clean":"2024-09-21T00:19:25.838037+0000","last_became_active":"2024-09-21T00:18:38.666643+0000","last_became_peered":"2024-09-21T00:18:38.666643+0000","last_unstale":"2024-09-21T00:19:25.838037+0000","last_undegraded":"2024-09-21T00:19:25.838037+0000","last_fullsized":"2024-09-21T00:19:25.838037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:30.496339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838471+0000","last_change":"2024-09-21T00:18:38.667705+0000","last_active":"2024-09-21T00:19:25.838471+0000","last_peered":"2024-09-21T00:19:25.838471+0000","last_clean":"2024-09-21T00:19:25.838471+0000","last_became_active":"2024-09-21T00:18:38.665141+0000","last_became_peered":"2024-09-21T00:18:38.665141+0000","last_unstale":"2024-09-21T00:19:25.838471+0000","last_undegraded":"2024-09-21T00:19:25.838471+0000","last_fullsized":"2024-09-21T00:19:25.838471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:17.624035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831393+0000","last_change":"2024-09-21T00:18:38.681364+0000","last_active":"2024-09-21T00:19:25.831393+0000","last_peered":"2024-09-21T00:19:25.831393+0000","last_clean":"2024-09-21T00:19:25.831393+0000","last_became_active":"2024-09-21T00:18:38.680937+0000","last_became_peered":"2024-09-21T00:18:38.680937+0000","last_unstale":"2024-09-21T00:19:25.831393+0000","last_undegraded":"2024-09-21T00:19:25.831393+0000","last_fullsized":"2024-09-21T00:19:25.831393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:34.206201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.843202+0000","last_change":"2024-09-21T00:19:25.843091+0000","last_active":"2024-09-21T00:19:25.843202+0000","last_peered":"2024-09-21T00:19:25.843202+0000","last_clean":"2024-09-21T00:19:25.843202+0000","last_became_active":"2024-09-21T00:19:25.842600+0000","last_became_peered":"2024-09-21T00:19:25.842600+0000","last_unstale":"2024-09-21T00:19:25.843202+0000","last_undegraded":"2024-09-21T00:19:25.843202+0000","last_fullsized":"2024-09-21T00:19:25.843202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:10.256474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838662+0000","last_change":"2024-09-21T00:18:38.668656+0000","last_active":"2024-09-21T00:19:25.838662+0000","last_peered":"2024-09-21T00:19:25.838662+0000","last_clean":"2024-09-21T00:19:25.838662+0000","last_became_active":"2024-09-21T00:18:38.668153+0000","last_became_peered":"2024-09-21T00:18:38.668153+0000","last_unstale":"2024-09-21T00:19:25.838662+0000","last_undegraded":"2024-09-21T00:19:25.838662+0000","last_fullsized":"2024-09-21T00:19:25.838662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:03.763609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840542+0000","last_change":"2024-09-21T00:18:38.675945+0000","last_active":"2024-09-21T00:19:25.840542+0000","last_peered":"2024-09-21T00:19:25.840542+0000","last_clean":"2024-09-21T00:19:25.840542+0000","last_became_active":"2024-09-21T00:18:38.675397+0000","last_became_peered":"2024-09-21T00:18:38.675397+0000","last_unstale":"2024-09-21T00:19:25.840542+0000","last_undegraded":"2024-09-21T00:19:25.840542+0000","last_fullsized":"2024-09-21T00:19:25.840542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:56.082314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833904+0000","last_change":"2024-09-21T00:18:38.662854+0000","last_active":"2024-09-21T00:19:25.833904+0000","last_peered":"2024-09-21T00:19:25.833904+0000","last_clean":"2024-09-21T00:19:25.833904+0000","last_became_active":"2024-09-21T00:18:38.662094+0000","last_became_peered":"2024-09-21T00:18:38.662094+0000","last_unstale":"2024-09-21T00:19:25.833904+0000","last_undegraded":"2024-09-21T00:19:25.833904+0000","last_fullsized":"2024-09-21T00:19:25.833904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:45.756641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840363+0000","last_change":"2024-09-21T00:18:38.663282+0000","last_active":"2024-09-21T00:19:25.840363+0000","last_peered":"2024-09-21T00:19:25.840363+0000","last_clean":"2024-09-21T00:19:25.840363+0000","last_became_active":"2024-09-21T00:18:38.662148+0000","last_became_peered":"2024-09-21T00:18:38.662148+0000","last_unstale":"2024-09-21T00:19:25.840363+0000","last_undegraded":"2024-09-21T00:19:25.840363+0000","last_fullsized":"2024-09-21T00:19:25.840363+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:51.090591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838054+0000","last_change":"2024-09-21T00:18:38.669184+0000","last_active":"2024-09-21T00:19:25.838054+0000","last_peered":"2024-09-21T00:19:25.838054+0000","last_clean":"2024-09-21T00:19:25.838054+0000","last_became_active":"2024-09-21T00:18:38.668799+0000","last_became_peered":"2024-09-21T00:18:38.668799+0000","last_unstale":"2024-09-21T00:19:25.838054+0000","last_undegraded":"2024-09-21T00:19:25.838054+0000","last_fullsized":"2024-09-21T00:19:25.838054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:49.642568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831320+0000","last_change":"2024-09-21T00:18:38.679501+0000","last_active":"2024-09-21T00:19:25.831320+0000","last_peered":"2024-09-21T00:19:25.831320+0000","last_clean":"2024-09-21T00:19:25.831320+0000","last_became_active":"2024-09-21T00:18:38.678582+0000","last_became_peered":"2024-09-21T00:18:38.678582+0000","last_unstale":"2024-09-21T00:19:25.831320+0000","last_undegraded":"2024-09-21T00:19:25.831320+0000","last_fullsized":"2024-09-21T00:19:25.831320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:30.277979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834494+0000","last_change":"2024-09-21T00:18:38.659657+0000","last_active":"2024-09-21T00:19:25.834494+0000","last_peered":"2024-09-21T00:19:25.834494+0000","last_clean":"2024-09-21T00:19:25.834494+0000","last_became_active":"2024-09-21T00:18:38.659257+0000","last_became_peered":"2024-09-21T00:18:38.659257+0000","last_unstale":"2024-09-21T00:19:25.834494+0000","last_undegraded":"2024-09-21T00:19:25.834494+0000","last_fullsized":"2024-09-21T00:19:25.834494+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:08.653729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838542+0000","last_change":"2024-09-21T00:18:38.671817+0000","last_active":"2024-09-21T00:19:25.838542+0000","last_peered":"2024-09-21T00:19:25.838542+0000","last_clean":"2024-09-21T00:19:25.838542+0000","last_became_active":"2024-09-21T00:18:38.671469+0000","last_became_peered":"2024-09-21T00:18:38.671469+0000","last_unstale":"2024-09-21T00:19:25.838542+0000","last_undegraded":"2024-09-21T00:19:25.838542+0000","last_fullsized":"2024-09-21T00:19:25.838542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:58:15.036623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.830413+0000","last_change":"2024-09-21T00:18:38.664481+0000","last_active":"2024-09-21T00:19:25.830413+0000","last_peered":"2024-09-21T00:19:25.830413+0000","last_clean":"2024-09-21T00:19:25.830413+0000","last_became_active":"2024-09-21T00:18:38.664098+0000","last_became_peered":"2024-09-21T00:18:38.664098+0000","last_unstale":"2024-09-21T00:19:25.830413+0000","last_undegraded":"2024-09-21T00:19:25.830413+0000","last_fullsized":"2024-09-21T00:19:25.830413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:09.271251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.835199+0000","last_change":"2024-09-21T00:18:38.662416+0000","last_active":"2024-09-21T00:19:25.835199+0000","last_peered":"2024-09-21T00:19:25.835199+0000","last_clean":"2024-09-21T00:19:25.835199+0000","last_became_active":"2024-09-21T00:18:38.662041+0000","last_became_peered":"2024-09-21T00:18:38.662041+0000","last_unstale":"2024-09-21T00:19:25.835199+0000","last_undegraded":"2024-09-21T00:19:25.835199+0000","last_fullsized":"2024-09-21T00:19:25.835199+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:03.150179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.837470+0000","last_change":"2024-09-21T00:18:38.670797+0000","last_active":"2024-09-21T00:19:25.837470+0000","last_peered":"2024-09-21T00:19:25.837470+0000","last_clean":"2024-09-21T00:19:25.837470+0000","last_became_active":"2024-09-21T00:18:38.670258+0000","last_became_peered":"2024-09-21T00:18:38.670258+0000","last_unstale":"2024-09-21T00:19:25.837470+0000","last_undegraded":"2024-09-21T00:19:25.837470+0000","last_fullsized":"2024-09-21T00:19:25.837470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:55.669836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838051+0000","last_change":"2024-09-21T00:18:38.665594+0000","last_active":"2024-09-21T00:19:25.838051+0000","last_peered":"2024-09-21T00:19:25.838051+0000","last_clean":"2024-09-21T00:19:25.838051+0000","last_became_active":"2024-09-21T00:18:38.665260+0000","last_became_peered":"2024-09-21T00:18:38.665260+0000","last_unstale":"2024-09-21T00:19:25.838051+0000","last_undegraded":"2024-09-21T00:19:25.838051+0000","last_fullsized":"2024-09-21T00:19:25.838051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:52:09.014327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838178+0000","last_change":"2024-09-21T00:18:38.679294+0000","last_active":"2024-09-21T00:19:25.838178+0000","last_peered":"2024-09-21T00:19:25.838178+0000","last_clean":"2024-09-21T00:19:25.838178+0000","last_became_active":"2024-09-21T00:18:38.673765+0000","last_became_peered":"2024-09-21T00:18:38.673765+0000","last_unstale":"2024-09-21T00:19:25.838178+0000","last_undegraded":"2024-09-21T00:19:25.838178+0000","last_fullsized":"2024-09-21T00:19:25.838178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:32.076299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831190+0000","last_change":"2024-09-21T00:18:38.667869+0000","last_active":"2024-09-21T00:19:25.831190+0000","last_peered":"2024-09-21T00:19:25.831190+0000","last_clean":"2024-09-21T00:19:25.831190+0000","last_became_active":"2024-09-21T00:18:38.667499+0000","last_became_peered":"2024-09-21T00:18:38.667499+0000","last_unstale":"2024-09-21T00:19:25.831190+0000","last_undegraded":"2024-09-21T00:19:25.831190+0000","last_fullsized":"2024-09-21T00:19:25.831190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:13.501043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.839877+0000","last_change":"2024-09-21T00:18:38.663107+0000","last_active":"2024-09-21T00:19:25.839877+0000","last_peered":"2024-09-21T00:19:25.839877+0000","last_clean":"2024-09-21T00:19:25.839877+0000","last_became_active":"2024-09-21T00:18:38.662680+0000","last_became_peered":"2024-09-21T00:18:38.662680+0000","last_unstale":"2024-09-21T00:19:25.839877+0000","last_undegraded":"2024-09-21T00:19:25.839877+0000","last_fullsized":"2024-09-21T00:19:25.839877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:48.154705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833535+0000","last_change":"2024-09-21T00:18:38.668167+0000","last_active":"2024-09-21T00:19:25.833535+0000","last_peered":"2024-09-21T00:19:25.833535+0000","last_clean":"2024-09-21T00:19:25.833535+0000","last_became_active":"2024-09-21T00:18:38.667842+0000","last_became_peered":"2024-09-21T00:18:38.667842+0000","last_unstale":"2024-09-21T00:19:25.833535+0000","last_undegraded":"2024-09-21T00:19:25.833535+0000","last_fullsized":"2024-09-21T00:19:25.833535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:55.358351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834667+0000","last_change":"2024-09-21T00:18:38.666580+0000","last_active":"2024-09-21T00:19:25.834667+0000","last_peered":"2024-09-21T00:19:25.834667+0000","last_clean":"2024-09-21T00:19:25.834667+0000","last_became_active":"2024-09-21T00:18:38.666231+0000","last_became_peered":"2024-09-21T00:18:38.666231+0000","last_unstale":"2024-09-21T00:19:25.834667+0000","last_undegraded":"2024-09-21T00:19:25.834667+0000","last_fullsized":"2024-09-21T00:19:25.834667+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:07.362516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838508+0000","last_change":"2024-09-21T00:18:38.667210+0000","last_active":"2024-09-21T00:19:25.838508+0000","last_peered":"2024-09-21T00:19:25.838508+0000","last_clean":"2024-09-21T00:19:25.838508+0000","last_became_active":"2024-09-21T00:18:38.666728+0000","last_became_peered":"2024-09-21T00:18:38.666728+0000","last_unstale":"2024-09-21T00:19:25.838508+0000","last_undegraded":"2024-09-21T00:19:25.838508+0000","last_fullsized":"2024-09-21T00:19:25.838508+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:14.712393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840980+0000","last_change":"2024-09-21T00:18:38.666580+0000","last_active":"2024-09-21T00:19:25.840980+0000","last_peered":"2024-09-21T00:19:25.840980+0000","last_clean":"2024-09-21T00:19:25.840980+0000","last_became_active":"2024-09-21T00:18:38.665698+0000","last_became_peered":"2024-09-21T00:18:38.665698+0000","last_unstale":"2024-09-21T00:19:25.840980+0000","last_undegraded":"2024-09-21T00:19:25.840980+0000","last_fullsized":"2024-09-21T00:19:25.840980+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:45.627667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.832742+0000","last_change":"2024-09-21T00:18:38.670729+0000","last_active":"2024-09-21T00:19:25.832742+0000","last_peered":"2024-09-21T00:19:25.832742+0000","last_clean":"2024-09-21T00:19:25.832742+0000","last_became_active":"2024-09-21T00:18:38.670337+0000","last_became_peered":"2024-09-21T00:18:38.670337+0000","last_unstale":"2024-09-21T00:19:25.832742+0000","last_undegraded":"2024-09-21T00:19:25.832742+0000","last_fullsized":"2024-09-21T00:19:25.832742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:49.880689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834131+0000","last_change":"2024-09-21T00:18:38.664644+0000","last_active":"2024-09-21T00:19:25.834131+0000","last_peered":"2024-09-21T00:19:25.834131+0000","last_clean":"2024-09-21T00:19:25.834131+0000","last_became_active":"2024-09-21T00:18:38.664220+0000","last_became_peered":"2024-09-21T00:18:38.664220+0000","last_unstale":"2024-09-21T00:19:25.834131+0000","last_undegraded":"2024-09-21T00:19:25.834131+0000","last_fullsized":"2024-09-21T00:19:25.834131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:09.610962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.839526+0000","last_change":"2024-09-21T00:18:38.671592+0000","last_active":"2024-09-21T00:19:25.839526+0000","last_peered":"2024-09-21T00:19:25.839526+0000","last_clean":"2024-09-21T00:19:25.839526+0000","last_became_active":"2024-09-21T00:18:38.671228+0000","last_became_peered":"2024-09-21T00:18:38.671228+0000","last_unstale":"2024-09-21T00:19:25.839526+0000","last_undegraded":"2024-09-21T00:19:25.839526+0000","last_fullsized":"2024-09-21T00:19:25.839526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:51.906058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.845109+0000","last_change":"2024-09-21T00:19:25.843696+0000","last_active":"2024-09-21T00:19:25.845109+0000","last_peered":"2024-09-21T00:19:25.845109+0000","last_clean":"2024-09-21T00:19:25.845109+0000","last_became_active":"2024-09-21T00:19:25.843287+0000","last_became_peered":"2024-09-21T00:19:25.843287+0000","last_unstale":"2024-09-21T00:19:25.845109+0000","last_undegraded":"2024-09-21T00:19:25.845109+0000","last_fullsized":"2024-09-21T00:19:25.845109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:21.024555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838751+0000","last_change":"2024-09-21T00:18:38.671820+0000","last_active":"2024-09-21T00:19:25.838751+0000","last_peered":"2024-09-21T00:19:25.838751+0000","last_clean":"2024-09-21T00:19:25.838751+0000","last_became_active":"2024-09-21T00:18:38.667107+0000","last_became_peered":"2024-09-21T00:18:38.667107+0000","last_unstale":"2024-09-21T00:19:25.838751+0000","last_undegraded":"2024-09-21T00:19:25.838751+0000","last_fullsized":"2024-09-21T00:19:25.838751+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:58.099750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838713+0000","last_change":"2024-09-21T00:18:38.671271+0000","last_active":"2024-09-21T00:19:25.838713+0000","last_peered":"2024-09-21T00:19:25.838713+0000","last_clean":"2024-09-21T00:19:25.838713+0000","last_became_active":"2024-09-21T00:18:38.670826+0000","last_became_peered":"2024-09-21T00:18:38.670826+0000","last_unstale":"2024-09-21T00:19:25.838713+0000","last_undegraded":"2024-09-21T00:19:25.838713+0000","last_fullsized":"2024-09-21T00:19:25.838713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:57:01.723120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.835068+0000","last_change":"2024-09-21T00:18:38.662851+0000","last_active":"2024-09-21T00:19:25.835068+0000","last_peered":"2024-09-21T00:19:25.835068+0000","last_clean":"2024-09-21T00:19:25.835068+0000","last_became_active":"2024-09-21T00:18:38.661259+0000","last_became_peered":"2024-09-21T00:18:38.661259+0000","last_unstale":"2024-09-21T00:19:25.835068+0000","last_undegraded":"2024-09-21T00:19:25.835068+0000","last_fullsized":"2024-09-21T00:19:25.835068+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:10.508980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57235,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":120,"num_read_kb":122,"num_write":156,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3864,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":179865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678044,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":516007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874273,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36992,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708160,"statfs":{"total":95995035648,"available":95957155840,"internally_reserved":0,"allocated":1687552,"data_stored":1156487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037798,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":520317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234028,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":516119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430257,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1617920,"data_stored":1112050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593783,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":511397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790014,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1107638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953539,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":569863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149767,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":511591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378703,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":512466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542228,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":572701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705753,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":512785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":54460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:19:56.626 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T00:19:57.012 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:57.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:57 smithi089 ceph-mon[34481]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:19:57.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:57 smithi098 ceph-mon[34725]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:19:57.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:57 smithi027 ceph-mon[30745]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:19:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.768+0000 7ff891600640 1 -- 172.21.15.27:0/1114903145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff88c105db0 msgr2=0x7ff88c1061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.768+0000 7ff891600640 1 --2- 172.21.15.27:0/1114903145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff88c105db0 0x7ff88c1061c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009900 tx=0x7ff87c02a3f0 comp rx=0 tx=0).stop 2024-09-21T00:19:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.768+0000 7ff891600640 1 -- 172.21.15.27:0/1114903145 shutdown_connections 2024-09-21T00:19:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.768+0000 7ff891600640 1 --2- 172.21.15.27:0/1114903145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff88c10c040 0x7ff88c112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.768+0000 7ff891600640 1 --2- 172.21.15.27:0/1114903145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c10b6f0 0x7ff88c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.768+0000 7ff891600640 1 --2- 172.21.15.27:0/1114903145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff88c105db0 0x7ff88c1061c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.768+0000 7ff891600640 1 -- 172.21.15.27:0/1114903145 >> 172.21.15.27:0/1114903145 conn(0x7ff88c1017e0 msgr2=0x7ff88c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.768+0000 7ff891600640 1 -- 172.21.15.27:0/1114903145 shutdown_connections 2024-09-21T00:19:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.769+0000 7ff891600640 1 -- 172.21.15.27:0/1114903145 wait complete. 2024-09-21T00:19:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.770+0000 7ff891600640 1 Processor -- start 2024-09-21T00:19:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.770+0000 7ff891600640 1 -- start start 2024-09-21T00:19:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.770+0000 7ff891600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c105db0 0x7ff88c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.770+0000 7ff891600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff88c10b6f0 0x7ff88c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.770+0000 7ff891600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff88c10c040 0x7ff88c1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.770+0000 7ff891600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c115d50 con 0x7ff88c10b6f0 2024-09-21T00:19:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.770+0000 7ff891600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c115bd0 con 0x7ff88c105db0 2024-09-21T00:19:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.770+0000 7ff891600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c115ed0 con 0x7ff88c10c040 2024-09-21T00:19:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.771+0000 7ff88aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c105db0 0x7ff88c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.771+0000 7ff883e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff88c10b6f0 0x7ff88c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.771+0000 7ff88aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c105db0 0x7ff88c19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39856/0 (socket says 172.21.15.27:39856) 2024-09-21T00:19:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.771+0000 7ff88aa00640 1 -- 172.21.15.27:0/582207865 learned_addr learned my addr 172.21.15.27:0/582207865 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:19:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.771+0000 7ff88b400640 1 --2- 172.21.15.27:0/582207865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff88c10c040 0x7ff88c1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.771+0000 7ff883e00640 1 -- 172.21.15.27:0/582207865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff88c10c040 msgr2=0x7ff88c1a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.771+0000 7ff883e00640 1 --2- 172.21.15.27:0/582207865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff88c10c040 0x7ff88c1a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.771+0000 7ff883e00640 1 -- 172.21.15.27:0/582207865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c105db0 msgr2=0x7ff88c19c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.771+0000 7ff883e00640 1 --2- 172.21.15.27:0/582207865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c105db0 0x7ff88c19c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.771+0000 7ff883e00640 1 -- 172.21.15.27:0/582207865 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c1a2690 con 0x7ff88c10b6f0 2024-09-21T00:19:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.771+0000 7ff88aa00640 1 --2- 172.21.15.27:0/582207865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c105db0 0x7ff88c19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:19:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.772+0000 7ff883e00640 1 --2- 172.21.15.27:0/582207865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff88c10b6f0 0x7ff88c19ccc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff88c06ca20 tx=0x7ff87400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.772+0000 7ff883400640 1 -- 172.21.15.27:0/582207865 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87400cc40 con 0x7ff88c10b6f0 2024-09-21T00:19:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.772+0000 7ff891600640 1 -- 172.21.15.27:0/582207865 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88c1a2980 con 0x7ff88c10b6f0 2024-09-21T00:19:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.772+0000 7ff883400640 1 -- 172.21.15.27:0/582207865 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff87400cde0 con 0x7ff88c10b6f0 2024-09-21T00:19:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.772+0000 7ff883400640 1 -- 172.21.15.27:0/582207865 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874018780 con 0x7ff88c10b6f0 2024-09-21T00:19:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.772+0000 7ff891600640 1 -- 172.21.15.27:0/582207865 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88c1a90e0 con 0x7ff88c10b6f0 2024-09-21T00:19:57.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.773+0000 7ff891600640 1 -- 172.21.15.27:0/582207865 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff88c1061c0 con 0x7ff88c10b6f0 2024-09-21T00:19:57.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.776+0000 7ff883400640 1 -- 172.21.15.27:0/582207865 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7ff874010a90 con 0x7ff88c10b6f0 2024-09-21T00:19:57.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.777+0000 7ff883400640 1 --2- 172.21.15.27:0/582207865 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff8700b6670 0x7ff8700b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:19:57.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.777+0000 7ff883400640 1 -- 172.21.15.27:0/582207865 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7ff8740e8060 con 0x7ff88c10b6f0 2024-09-21T00:19:57.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.778+0000 7ff88aa00640 1 --2- 172.21.15.27:0/582207865 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff8700b6670 0x7ff8700b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:19:57.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.778+0000 7ff88aa00640 1 --2- 172.21.15.27:0/582207865 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff8700b6670 0x7ff8700b8b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0096c0 tx=0x7ff87c005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:19:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.779+0000 7ff883400640 1 -- 172.21.15.27:0/582207865 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff8740b26e0 con 0x7ff88c10b6f0 2024-09-21T00:19:57.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.976+0000 7ff891600640 1 -- 172.21.15.27:0/582207865 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff88c063800 con 0x7ff8700b6670 2024-09-21T00:19:57.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.983+0000 7ff883400640 1 -- 172.21.15.27:0/582207865 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340357 (secure 0 0 0) 0x7ff88c063800 con 0x7ff8700b6670 2024-09-21T00:19:57.986 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:19:57.989 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T00:19:57.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.989+0000 7ff891600640 1 -- 172.21.15.27:0/582207865 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff8700b6670 msgr2=0x7ff8700b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:57.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.989+0000 7ff891600640 1 --2- 172.21.15.27:0/582207865 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff8700b6670 0x7ff8700b8b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0096c0 tx=0x7ff87c005a20 comp rx=0 tx=0).stop 2024-09-21T00:19:57.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.989+0000 7ff891600640 1 -- 172.21.15.27:0/582207865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff88c10b6f0 msgr2=0x7ff88c19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:19:57.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.989+0000 7ff891600640 1 --2- 172.21.15.27:0/582207865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff88c10b6f0 0x7ff88c19ccc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff88c06ca20 tx=0x7ff87400ee50 comp rx=0 tx=0).stop 2024-09-21T00:19:57.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.989+0000 7ff891600640 1 -- 172.21.15.27:0/582207865 shutdown_connections 2024-09-21T00:19:57.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.989+0000 7ff891600640 1 --2- 172.21.15.27:0/582207865 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff8700b6670 0x7ff8700b8b80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:57.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.989+0000 7ff891600640 1 --2- 172.21.15.27:0/582207865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff88c10c040 0x7ff88c1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:57.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.989+0000 7ff891600640 1 --2- 172.21.15.27:0/582207865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff88c10b6f0 0x7ff88c19ccc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:57.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.989+0000 7ff891600640 1 --2- 172.21.15.27:0/582207865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c105db0 0x7ff88c19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:19:57.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.989+0000 7ff891600640 1 -- 172.21.15.27:0/582207865 >> 172.21.15.27:0/582207865 conn(0x7ff88c1017e0 msgr2=0x7ff88c10ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:19:57.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.991+0000 7ff891600640 1 -- 172.21.15.27:0/582207865 shutdown_connections 2024-09-21T00:19:57.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:19:57.991+0000 7ff891600640 1 -- 172.21.15.27:0/582207865 wait complete. 2024-09-21T00:19:58.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:58 smithi027 ceph-mon[30745]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-09-21T00:19:58.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:58 smithi089 ceph-mon[34481]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-09-21T00:19:58.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:58 smithi098 ceph-mon[34725]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-09-21T00:19:58.863 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-21T00:19:56.347078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655997,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":222,"num_read_kb":209,"num_write":284,"num_write_kb":2264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2955704,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375620,"kb_used_data":13892,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566204,"statfs":{"total":1151940427776,"available":1151555792896,"internally_reserved":0,"allocated":14225408,"data_stored":8135741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":57669,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":85,"num_read_kb":98,"num_write":26,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003561"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.907198+0000","last_change":"2024-09-21T00:18:35.627618+0000","last_active":"2024-09-21T00:19:33.907198+0000","last_peered":"2024-09-21T00:19:33.907198+0000","last_clean":"2024-09-21T00:19:33.907198+0000","last_became_active":"2024-09-21T00:18:35.626475+0000","last_became_peered":"2024-09-21T00:18:35.626475+0000","last_unstale":"2024-09-21T00:19:33.907198+0000","last_undegraded":"2024-09-21T00:19:33.907198+0000","last_fullsized":"2024-09-21T00:19:33.907198+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:50.379225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.841178+0000","last_change":"2024-09-21T00:18:38.665891+0000","last_active":"2024-09-21T00:19:25.841178+0000","last_peered":"2024-09-21T00:19:25.841178+0000","last_clean":"2024-09-21T00:19:25.841178+0000","last_became_active":"2024-09-21T00:18:38.665472+0000","last_became_peered":"2024-09-21T00:18:38.665472+0000","last_unstale":"2024-09-21T00:19:25.841178+0000","last_undegraded":"2024-09-21T00:19:25.841178+0000","last_fullsized":"2024-09-21T00:19:25.841178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:34:32.490828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.837644+0000","last_change":"2024-09-21T00:18:48.730206+0000","last_active":"2024-09-21T00:19:25.837644+0000","last_peered":"2024-09-21T00:19:25.837644+0000","last_clean":"2024-09-21T00:19:25.837644+0000","last_became_active":"2024-09-21T00:18:48.729729+0000","last_became_peered":"2024-09-21T00:18:48.729729+0000","last_unstale":"2024-09-21T00:19:25.837644+0000","last_undegraded":"2024-09-21T00:19:25.837644+0000","last_fullsized":"2024-09-21T00:19:25.837644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:08:17.019877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.841063+0000","last_change":"2024-09-21T00:18:35.768554+0000","last_active":"2024-09-21T00:19:25.841063+0000","last_peered":"2024-09-21T00:19:25.841063+0000","last_clean":"2024-09-21T00:19:25.841063+0000","last_became_active":"2024-09-21T00:18:35.767642+0000","last_became_peered":"2024-09-21T00:18:35.767642+0000","last_unstale":"2024-09-21T00:19:25.841063+0000","last_undegraded":"2024-09-21T00:19:25.841063+0000","last_fullsized":"2024-09-21T00:19:25.841063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:10.442974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838928+0000","last_change":"2024-09-21T00:18:38.677997+0000","last_active":"2024-09-21T00:19:25.838928+0000","last_peered":"2024-09-21T00:19:25.838928+0000","last_clean":"2024-09-21T00:19:25.838928+0000","last_became_active":"2024-09-21T00:18:38.676698+0000","last_became_peered":"2024-09-21T00:18:38.676698+0000","last_unstale":"2024-09-21T00:19:25.838928+0000","last_undegraded":"2024-09-21T00:19:25.838928+0000","last_fullsized":"2024-09-21T00:19:25.838928+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:11.662765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838302+0000","last_change":"2024-09-21T00:18:48.739563+0000","last_active":"2024-09-21T00:19:25.838302+0000","last_peered":"2024-09-21T00:19:25.838302+0000","last_clean":"2024-09-21T00:19:25.838302+0000","last_became_active":"2024-09-21T00:18:48.739092+0000","last_became_peered":"2024-09-21T00:18:48.739092+0000","last_unstale":"2024-09-21T00:19:25.838302+0000","last_undegraded":"2024-09-21T00:19:25.838302+0000","last_fullsized":"2024-09-21T00:19:25.838302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:44.185556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.901582+0000","last_change":"2024-09-21T00:18:35.626742+0000","last_active":"2024-09-21T00:19:33.901582+0000","last_peered":"2024-09-21T00:19:33.901582+0000","last_clean":"2024-09-21T00:19:33.901582+0000","last_became_active":"2024-09-21T00:18:35.626359+0000","last_became_peered":"2024-09-21T00:18:35.626359+0000","last_unstale":"2024-09-21T00:19:33.901582+0000","last_undegraded":"2024-09-21T00:19:33.901582+0000","last_fullsized":"2024-09-21T00:19:33.901582+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:38.860184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838207+0000","last_change":"2024-09-21T00:18:38.666003+0000","last_active":"2024-09-21T00:19:25.838207+0000","last_peered":"2024-09-21T00:19:25.838207+0000","last_clean":"2024-09-21T00:19:25.838207+0000","last_became_active":"2024-09-21T00:18:38.665425+0000","last_became_peered":"2024-09-21T00:18:38.665425+0000","last_unstale":"2024-09-21T00:19:25.838207+0000","last_undegraded":"2024-09-21T00:19:25.838207+0000","last_fullsized":"2024-09-21T00:19:25.838207+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:14.121864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838743+0000","last_change":"2024-09-21T00:18:48.729604+0000","last_active":"2024-09-21T00:19:25.838743+0000","last_peered":"2024-09-21T00:19:25.838743+0000","last_clean":"2024-09-21T00:19:25.838743+0000","last_became_active":"2024-09-21T00:18:48.729026+0000","last_became_peered":"2024-09-21T00:18:48.729026+0000","last_unstale":"2024-09-21T00:19:25.838743+0000","last_undegraded":"2024-09-21T00:19:25.838743+0000","last_fullsized":"2024-09-21T00:19:25.838743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:18.704373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840011+0000","last_change":"2024-09-21T00:18:35.628091+0000","last_active":"2024-09-21T00:19:25.840011+0000","last_peered":"2024-09-21T00:19:25.840011+0000","last_clean":"2024-09-21T00:19:25.840011+0000","last_became_active":"2024-09-21T00:18:35.627654+0000","last_became_peered":"2024-09-21T00:18:35.627654+0000","last_unstale":"2024-09-21T00:19:25.840011+0000","last_undegraded":"2024-09-21T00:19:25.840011+0000","last_fullsized":"2024-09-21T00:19:25.840011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:05.229036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.844254+0000","last_change":"2024-09-21T00:19:25.844116+0000","last_active":"2024-09-21T00:19:25.844254+0000","last_peered":"2024-09-21T00:19:25.844254+0000","last_clean":"2024-09-21T00:19:25.844254+0000","last_became_active":"2024-09-21T00:19:25.842745+0000","last_became_peered":"2024-09-21T00:19:25.842745+0000","last_unstale":"2024-09-21T00:19:25.844254+0000","last_undegraded":"2024-09-21T00:19:25.844254+0000","last_fullsized":"2024-09-21T00:19:25.844254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:42.651251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838220+0000","last_change":"2024-09-21T00:18:48.740106+0000","last_active":"2024-09-21T00:19:25.838220+0000","last_peered":"2024-09-21T00:19:25.838220+0000","last_clean":"2024-09-21T00:19:25.838220+0000","last_became_active":"2024-09-21T00:18:48.739299+0000","last_became_peered":"2024-09-21T00:18:48.739299+0000","last_unstale":"2024-09-21T00:19:25.838220+0000","last_undegraded":"2024-09-21T00:19:25.838220+0000","last_fullsized":"2024-09-21T00:19:25.838220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:59.058333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.832963+0000","last_change":"2024-09-21T00:18:38.667331+0000","last_active":"2024-09-21T00:19:25.832963+0000","last_peered":"2024-09-21T00:19:25.832963+0000","last_clean":"2024-09-21T00:19:25.832963+0000","last_became_active":"2024-09-21T00:18:38.666960+0000","last_became_peered":"2024-09-21T00:18:38.666960+0000","last_unstale":"2024-09-21T00:19:25.832963+0000","last_undegraded":"2024-09-21T00:19:25.832963+0000","last_fullsized":"2024-09-21T00:19:25.832963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:57.569460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.894046+0000","last_change":"2024-09-21T00:18:35.625570+0000","last_active":"2024-09-21T00:19:32.894046+0000","last_peered":"2024-09-21T00:19:32.894046+0000","last_clean":"2024-09-21T00:19:32.894046+0000","last_became_active":"2024-09-21T00:18:35.625170+0000","last_became_peered":"2024-09-21T00:18:35.625170+0000","last_unstale":"2024-09-21T00:19:32.894046+0000","last_undegraded":"2024-09-21T00:19:32.894046+0000","last_fullsized":"2024-09-21T00:19:32.894046+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:58.614258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838489+0000","last_change":"2024-09-21T00:18:48.727390+0000","last_active":"2024-09-21T00:19:25.838489+0000","last_peered":"2024-09-21T00:19:25.838489+0000","last_clean":"2024-09-21T00:19:25.838489+0000","last_became_active":"2024-09-21T00:18:48.726864+0000","last_became_peered":"2024-09-21T00:18:48.726864+0000","last_unstale":"2024-09-21T00:19:25.838489+0000","last_undegraded":"2024-09-21T00:19:25.838489+0000","last_fullsized":"2024-09-21T00:19:25.838489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:10.917069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840691+0000","last_change":"2024-09-21T00:18:38.664072+0000","last_active":"2024-09-21T00:19:25.840691+0000","last_peered":"2024-09-21T00:19:25.840691+0000","last_clean":"2024-09-21T00:19:25.840691+0000","last_became_active":"2024-09-21T00:18:38.662562+0000","last_became_peered":"2024-09-21T00:18:38.662562+0000","last_unstale":"2024-09-21T00:19:25.840691+0000","last_undegraded":"2024-09-21T00:19:25.840691+0000","last_fullsized":"2024-09-21T00:19:25.840691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.021174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.844315+0000","last_change":"2024-09-21T00:19:25.843599+0000","last_active":"2024-09-21T00:19:25.844315+0000","last_peered":"2024-09-21T00:19:25.844315+0000","last_clean":"2024-09-21T00:19:25.844315+0000","last_became_active":"2024-09-21T00:19:25.843178+0000","last_became_peered":"2024-09-21T00:19:25.843178+0000","last_unstale":"2024-09-21T00:19:25.844315+0000","last_undegraded":"2024-09-21T00:19:25.844315+0000","last_fullsized":"2024-09-21T00:19:25.844315+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:18.809857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.839617+0000","last_change":"2024-09-21T00:18:48.729940+0000","last_active":"2024-09-21T00:19:25.839617+0000","last_peered":"2024-09-21T00:19:25.839617+0000","last_clean":"2024-09-21T00:19:25.839617+0000","last_became_active":"2024-09-21T00:18:48.729465+0000","last_became_peered":"2024-09-21T00:18:48.729465+0000","last_unstale":"2024-09-21T00:19:25.839617+0000","last_undegraded":"2024-09-21T00:19:25.839617+0000","last_fullsized":"2024-09-21T00:19:25.839617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:44.566386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834875+0000","last_change":"2024-09-21T00:18:38.662510+0000","last_active":"2024-09-21T00:19:25.834875+0000","last_peered":"2024-09-21T00:19:25.834875+0000","last_clean":"2024-09-21T00:19:25.834875+0000","last_became_active":"2024-09-21T00:18:38.659788+0000","last_became_peered":"2024-09-21T00:18:38.659788+0000","last_unstale":"2024-09-21T00:19:25.834875+0000","last_undegraded":"2024-09-21T00:19:25.834875+0000","last_fullsized":"2024-09-21T00:19:25.834875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:14.532084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.897505+0000","last_change":"2024-09-21T00:18:35.622949+0000","last_active":"2024-09-21T00:19:32.897505+0000","last_peered":"2024-09-21T00:19:32.897505+0000","last_clean":"2024-09-21T00:19:32.897505+0000","last_became_active":"2024-09-21T00:18:35.621787+0000","last_became_peered":"2024-09-21T00:18:35.621787+0000","last_unstale":"2024-09-21T00:19:32.897505+0000","last_undegraded":"2024-09-21T00:19:32.897505+0000","last_fullsized":"2024-09-21T00:19:32.897505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:39:15.060123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.830466+0000","last_change":"2024-09-21T00:18:48.730711+0000","last_active":"2024-09-21T00:19:25.830466+0000","last_peered":"2024-09-21T00:19:25.830466+0000","last_clean":"2024-09-21T00:19:25.830466+0000","last_became_active":"2024-09-21T00:18:48.730287+0000","last_became_peered":"2024-09-21T00:18:48.730287+0000","last_unstale":"2024-09-21T00:19:25.830466+0000","last_undegraded":"2024-09-21T00:19:25.830466+0000","last_fullsized":"2024-09-21T00:19:25.830466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:34.444747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.839078+0000","last_change":"2024-09-21T00:18:38.669381+0000","last_active":"2024-09-21T00:19:25.839078+0000","last_peered":"2024-09-21T00:19:25.839078+0000","last_clean":"2024-09-21T00:19:25.839078+0000","last_became_active":"2024-09-21T00:18:38.668757+0000","last_became_peered":"2024-09-21T00:18:38.668757+0000","last_unstale":"2024-09-21T00:19:25.839078+0000","last_undegraded":"2024-09-21T00:19:25.839078+0000","last_fullsized":"2024-09-21T00:19:25.839078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:52.475872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.894147+0000","last_change":"2024-09-21T00:18:35.767700+0000","last_active":"2024-09-21T00:19:32.894147+0000","last_peered":"2024-09-21T00:19:32.894147+0000","last_clean":"2024-09-21T00:19:32.894147+0000","last_became_active":"2024-09-21T00:18:35.767369+0000","last_became_peered":"2024-09-21T00:18:35.767369+0000","last_unstale":"2024-09-21T00:19:32.894147+0000","last_undegraded":"2024-09-21T00:19:32.894147+0000","last_fullsized":"2024-09-21T00:19:32.894147+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:44.955659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838625+0000","last_change":"2024-09-21T00:18:48.737411+0000","last_active":"2024-09-21T00:19:25.838625+0000","last_peered":"2024-09-21T00:19:25.838625+0000","last_clean":"2024-09-21T00:19:25.838625+0000","last_became_active":"2024-09-21T00:18:48.736715+0000","last_became_peered":"2024-09-21T00:18:48.736715+0000","last_unstale":"2024-09-21T00:19:25.838625+0000","last_undegraded":"2024-09-21T00:19:25.838625+0000","last_fullsized":"2024-09-21T00:19:25.838625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:01.045318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.842485+0000","last_change":"2024-09-21T00:19:25.842355+0000","last_active":"2024-09-21T00:19:25.842485+0000","last_peered":"2024-09-21T00:19:25.842485+0000","last_clean":"2024-09-21T00:19:25.842485+0000","last_became_active":"2024-09-21T00:19:25.841961+0000","last_became_peered":"2024-09-21T00:19:25.841961+0000","last_unstale":"2024-09-21T00:19:25.842485+0000","last_undegraded":"2024-09-21T00:19:25.842485+0000","last_fullsized":"2024-09-21T00:19:25.842485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:30.490992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.895609+0000","last_change":"2024-09-21T00:18:35.919003+0000","last_active":"2024-09-21T00:19:32.895609+0000","last_peered":"2024-09-21T00:19:32.895609+0000","last_clean":"2024-09-21T00:19:32.895609+0000","last_became_active":"2024-09-21T00:18:35.918593+0000","last_became_peered":"2024-09-21T00:18:35.918593+0000","last_unstale":"2024-09-21T00:19:32.895609+0000","last_undegraded":"2024-09-21T00:19:32.895609+0000","last_fullsized":"2024-09-21T00:19:32.895609+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:35.982726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.837887+0000","last_change":"2024-09-21T00:18:48.725906+0000","last_active":"2024-09-21T00:19:25.837887+0000","last_peered":"2024-09-21T00:19:25.837887+0000","last_clean":"2024-09-21T00:19:25.837887+0000","last_became_active":"2024-09-21T00:18:48.725450+0000","last_became_peered":"2024-09-21T00:18:48.725450+0000","last_unstale":"2024-09-21T00:19:25.837887+0000","last_undegraded":"2024-09-21T00:19:25.837887+0000","last_fullsized":"2024-09-21T00:19:25.837887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:35.163208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840643+0000","last_change":"2024-09-21T00:18:38.666442+0000","last_active":"2024-09-21T00:19:25.840643+0000","last_peered":"2024-09-21T00:19:25.840643+0000","last_clean":"2024-09-21T00:19:25.840643+0000","last_became_active":"2024-09-21T00:18:38.666083+0000","last_became_peered":"2024-09-21T00:18:38.666083+0000","last_unstale":"2024-09-21T00:19:25.840643+0000","last_undegraded":"2024-09-21T00:19:25.840643+0000","last_fullsized":"2024-09-21T00:19:25.840643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:39.986440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:50.677868+0000","last_change":"2024-09-21T00:18:35.624883+0000","last_active":"2024-09-21T00:19:50.677868+0000","last_peered":"2024-09-21T00:19:50.677868+0000","last_clean":"2024-09-21T00:19:50.677868+0000","last_became_active":"2024-09-21T00:18:35.624382+0000","last_became_peered":"2024-09-21T00:18:35.624382+0000","last_unstale":"2024-09-21T00:19:50.677868+0000","last_undegraded":"2024-09-21T00:19:50.677868+0000","last_fullsized":"2024-09-21T00:19:50.677868+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:23.475521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840164+0000","last_change":"2024-09-21T00:18:48.719850+0000","last_active":"2024-09-21T00:19:25.840164+0000","last_peered":"2024-09-21T00:19:25.840164+0000","last_clean":"2024-09-21T00:19:25.840164+0000","last_became_active":"2024-09-21T00:18:48.719414+0000","last_became_peered":"2024-09-21T00:18:48.719414+0000","last_unstale":"2024-09-21T00:19:25.840164+0000","last_undegraded":"2024-09-21T00:19:25.840164+0000","last_fullsized":"2024-09-21T00:19:25.840164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:05.978977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831411+0000","last_change":"2024-09-21T00:18:38.671920+0000","last_active":"2024-09-21T00:19:25.831411+0000","last_peered":"2024-09-21T00:19:25.831411+0000","last_clean":"2024-09-21T00:19:25.831411+0000","last_became_active":"2024-09-21T00:18:38.671592+0000","last_became_peered":"2024-09-21T00:18:38.671592+0000","last_unstale":"2024-09-21T00:19:25.831411+0000","last_undegraded":"2024-09-21T00:19:25.831411+0000","last_fullsized":"2024-09-21T00:19:25.831411+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:40:48.257086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.894290+0000","last_change":"2024-09-21T00:18:35.769197+0000","last_active":"2024-09-21T00:19:32.894290+0000","last_peered":"2024-09-21T00:19:32.894290+0000","last_clean":"2024-09-21T00:19:32.894290+0000","last_became_active":"2024-09-21T00:18:35.768740+0000","last_became_peered":"2024-09-21T00:18:35.768740+0000","last_unstale":"2024-09-21T00:19:32.894290+0000","last_undegraded":"2024-09-21T00:19:32.894290+0000","last_fullsized":"2024-09-21T00:19:32.894290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:32.253006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834262+0000","last_change":"2024-09-21T00:18:48.740189+0000","last_active":"2024-09-21T00:19:25.834262+0000","last_peered":"2024-09-21T00:19:25.834262+0000","last_clean":"2024-09-21T00:19:25.834262+0000","last_became_active":"2024-09-21T00:18:48.739700+0000","last_became_peered":"2024-09-21T00:18:48.739700+0000","last_unstale":"2024-09-21T00:19:25.834262+0000","last_undegraded":"2024-09-21T00:19:25.834262+0000","last_fullsized":"2024-09-21T00:19:25.834262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:06.312160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834262+0000","last_change":"2024-09-21T00:18:38.665635+0000","last_active":"2024-09-21T00:19:25.834262+0000","last_peered":"2024-09-21T00:19:25.834262+0000","last_clean":"2024-09-21T00:19:25.834262+0000","last_became_active":"2024-09-21T00:18:38.665236+0000","last_became_peered":"2024-09-21T00:18:38.665236+0000","last_unstale":"2024-09-21T00:19:25.834262+0000","last_undegraded":"2024-09-21T00:19:25.834262+0000","last_fullsized":"2024-09-21T00:19:25.834262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:52.186480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.830142+0000","last_change":"2024-09-21T00:18:35.771465+0000","last_active":"2024-09-21T00:19:25.830142+0000","last_peered":"2024-09-21T00:19:25.830142+0000","last_clean":"2024-09-21T00:19:25.830142+0000","last_became_active":"2024-09-21T00:18:35.770550+0000","last_became_peered":"2024-09-21T00:18:35.770550+0000","last_unstale":"2024-09-21T00:19:25.830142+0000","last_undegraded":"2024-09-21T00:19:25.830142+0000","last_fullsized":"2024-09-21T00:19:25.830142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:43.042563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.839574+0000","last_change":"2024-09-21T00:18:48.736334+0000","last_active":"2024-09-21T00:19:25.839574+0000","last_peered":"2024-09-21T00:19:25.839574+0000","last_clean":"2024-09-21T00:19:25.839574+0000","last_became_active":"2024-09-21T00:18:48.735784+0000","last_became_peered":"2024-09-21T00:18:48.735784+0000","last_unstale":"2024-09-21T00:19:25.839574+0000","last_undegraded":"2024-09-21T00:19:25.839574+0000","last_fullsized":"2024-09-21T00:19:25.839574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:29.637335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833846+0000","last_change":"2024-09-21T00:18:38.674181+0000","last_active":"2024-09-21T00:19:25.833846+0000","last_peered":"2024-09-21T00:19:25.833846+0000","last_clean":"2024-09-21T00:19:25.833846+0000","last_became_active":"2024-09-21T00:18:38.673784+0000","last_became_peered":"2024-09-21T00:18:38.673784+0000","last_unstale":"2024-09-21T00:19:25.833846+0000","last_undegraded":"2024-09-21T00:19:25.833846+0000","last_fullsized":"2024-09-21T00:19:25.833846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:33.471328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833078+0000","last_change":"2024-09-21T00:18:35.628149+0000","last_active":"2024-09-21T00:19:25.833078+0000","last_peered":"2024-09-21T00:19:25.833078+0000","last_clean":"2024-09-21T00:19:25.833078+0000","last_became_active":"2024-09-21T00:18:35.627674+0000","last_became_peered":"2024-09-21T00:18:35.627674+0000","last_unstale":"2024-09-21T00:19:25.833078+0000","last_undegraded":"2024-09-21T00:19:25.833078+0000","last_fullsized":"2024-09-21T00:19:25.833078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:28:47.244288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831279+0000","last_change":"2024-09-21T00:18:48.740283+0000","last_active":"2024-09-21T00:19:25.831279+0000","last_peered":"2024-09-21T00:19:25.831279+0000","last_clean":"2024-09-21T00:19:25.831279+0000","last_became_active":"2024-09-21T00:18:48.739026+0000","last_became_peered":"2024-09-21T00:18:48.739026+0000","last_unstale":"2024-09-21T00:19:25.831279+0000","last_undegraded":"2024-09-21T00:19:25.831279+0000","last_fullsized":"2024-09-21T00:19:25.831279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:50.898116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833942+0000","last_change":"2024-09-21T00:18:38.667206+0000","last_active":"2024-09-21T00:19:25.833942+0000","last_peered":"2024-09-21T00:19:25.833942+0000","last_clean":"2024-09-21T00:19:25.833942+0000","last_became_active":"2024-09-21T00:18:38.666770+0000","last_became_peered":"2024-09-21T00:18:38.666770+0000","last_unstale":"2024-09-21T00:19:25.833942+0000","last_undegraded":"2024-09-21T00:19:25.833942+0000","last_fullsized":"2024-09-21T00:19:25.833942+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:56.332071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":62,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:51.951671+0000","last_change":"2024-09-21T00:18:35.768012+0000","last_active":"2024-09-21T00:19:51.951671+0000","last_peered":"2024-09-21T00:19:51.951671+0000","last_clean":"2024-09-21T00:19:51.951671+0000","last_became_active":"2024-09-21T00:18:35.767438+0000","last_became_peered":"2024-09-21T00:18:35.767438+0000","last_unstale":"2024-09-21T00:19:51.951671+0000","last_undegraded":"2024-09-21T00:19:51.951671+0000","last_fullsized":"2024-09-21T00:19:51.951671+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:35:40.606871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831461+0000","last_change":"2024-09-21T00:18:48.738228+0000","last_active":"2024-09-21T00:19:25.831461+0000","last_peered":"2024-09-21T00:19:25.831461+0000","last_clean":"2024-09-21T00:19:25.831461+0000","last_became_active":"2024-09-21T00:18:48.737786+0000","last_became_peered":"2024-09-21T00:18:48.737786+0000","last_unstale":"2024-09-21T00:19:25.831461+0000","last_undegraded":"2024-09-21T00:19:25.831461+0000","last_fullsized":"2024-09-21T00:19:25.831461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:52.329677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.830816+0000","last_change":"2024-09-21T00:18:38.678126+0000","last_active":"2024-09-21T00:19:25.830816+0000","last_peered":"2024-09-21T00:19:25.830816+0000","last_clean":"2024-09-21T00:19:25.830816+0000","last_became_active":"2024-09-21T00:18:38.677757+0000","last_became_peered":"2024-09-21T00:18:38.677757+0000","last_unstale":"2024-09-21T00:19:25.830816+0000","last_undegraded":"2024-09-21T00:19:25.830816+0000","last_fullsized":"2024-09-21T00:19:25.830816+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:26.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.902707+0000","last_change":"2024-09-21T00:18:35.625955+0000","last_active":"2024-09-21T00:19:33.902707+0000","last_peered":"2024-09-21T00:19:33.902707+0000","last_clean":"2024-09-21T00:19:33.902707+0000","last_became_active":"2024-09-21T00:18:35.625548+0000","last_became_peered":"2024-09-21T00:18:35.625548+0000","last_unstale":"2024-09-21T00:19:33.902707+0000","last_undegraded":"2024-09-21T00:19:33.902707+0000","last_fullsized":"2024-09-21T00:19:33.902707+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:04.163421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.835153+0000","last_change":"2024-09-21T00:18:48.740765+0000","last_active":"2024-09-21T00:19:25.835153+0000","last_peered":"2024-09-21T00:19:25.835153+0000","last_clean":"2024-09-21T00:19:25.835153+0000","last_became_active":"2024-09-21T00:18:48.740081+0000","last_became_peered":"2024-09-21T00:18:48.740081+0000","last_unstale":"2024-09-21T00:19:25.835153+0000","last_undegraded":"2024-09-21T00:19:25.835153+0000","last_fullsized":"2024-09-21T00:19:25.835153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:38.251475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840450+0000","last_change":"2024-09-21T00:18:38.667581+0000","last_active":"2024-09-21T00:19:25.840450+0000","last_peered":"2024-09-21T00:19:25.840450+0000","last_clean":"2024-09-21T00:19:25.840450+0000","last_became_active":"2024-09-21T00:18:38.665959+0000","last_became_peered":"2024-09-21T00:18:38.665959+0000","last_unstale":"2024-09-21T00:19:25.840450+0000","last_undegraded":"2024-09-21T00:19:25.840450+0000","last_fullsized":"2024-09-21T00:19:25.840450+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:52.321654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:35.920303+0000","last_change":"2024-09-21T00:18:35.624291+0000","last_active":"2024-09-21T00:19:35.920303+0000","last_peered":"2024-09-21T00:19:35.920303+0000","last_clean":"2024-09-21T00:19:35.920303+0000","last_became_active":"2024-09-21T00:18:35.623297+0000","last_became_peered":"2024-09-21T00:18:35.623297+0000","last_unstale":"2024-09-21T00:19:35.920303+0000","last_undegraded":"2024-09-21T00:19:35.920303+0000","last_fullsized":"2024-09-21T00:19:35.920303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:44.824409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":116,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.832837+0000","last_change":"2024-09-21T00:17:21.233627+0000","last_active":"2024-09-21T00:19:25.832837+0000","last_peered":"2024-09-21T00:19:25.832837+0000","last_clean":"2024-09-21T00:19:25.832837+0000","last_became_active":"2024-09-21T00:17:21.227897+0000","last_became_peered":"2024-09-21T00:17:21.227897+0000","last_unstale":"2024-09-21T00:19:25.832837+0000","last_undegraded":"2024-09-21T00:19:25.832837+0000","last_fullsized":"2024-09-21T00:19:25.832837+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:13:33.120691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:13:33.120691+0000","last_clean_scrub_stamp":"2024-09-21T00:13:33.120691+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:20.198660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833211+0000","last_change":"2024-09-21T00:18:48.730107+0000","last_active":"2024-09-21T00:19:25.833211+0000","last_peered":"2024-09-21T00:19:25.833211+0000","last_clean":"2024-09-21T00:19:25.833211+0000","last_became_active":"2024-09-21T00:18:48.729463+0000","last_became_peered":"2024-09-21T00:18:48.729463+0000","last_unstale":"2024-09-21T00:19:25.833211+0000","last_undegraded":"2024-09-21T00:19:25.833211+0000","last_fullsized":"2024-09-21T00:19:25.833211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:55.352870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833585+0000","last_change":"2024-09-21T00:18:38.671244+0000","last_active":"2024-09-21T00:19:25.833585+0000","last_peered":"2024-09-21T00:19:25.833585+0000","last_clean":"2024-09-21T00:19:25.833585+0000","last_became_active":"2024-09-21T00:18:38.670630+0000","last_became_peered":"2024-09-21T00:18:38.670630+0000","last_unstale":"2024-09-21T00:19:25.833585+0000","last_undegraded":"2024-09-21T00:19:25.833585+0000","last_fullsized":"2024-09-21T00:19:25.833585+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:23.695660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.902828+0000","last_change":"2024-09-21T00:19:25.843900+0000","last_active":"2024-09-21T00:19:33.902828+0000","last_peered":"2024-09-21T00:19:33.902828+0000","last_clean":"2024-09-21T00:19:33.902828+0000","last_became_active":"2024-09-21T00:19:25.843438+0000","last_became_peered":"2024-09-21T00:19:25.843438+0000","last_unstale":"2024-09-21T00:19:33.902828+0000","last_undegraded":"2024-09-21T00:19:33.902828+0000","last_fullsized":"2024-09-21T00:19:33.902828+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:17.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.837836+0000","last_change":"2024-09-21T00:18:48.739640+0000","last_active":"2024-09-21T00:19:25.837836+0000","last_peered":"2024-09-21T00:19:25.837836+0000","last_clean":"2024-09-21T00:19:25.837836+0000","last_became_active":"2024-09-21T00:18:48.739118+0000","last_became_peered":"2024-09-21T00:18:48.739118+0000","last_unstale":"2024-09-21T00:19:25.837836+0000","last_undegraded":"2024-09-21T00:19:25.837836+0000","last_fullsized":"2024-09-21T00:19:25.837836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:03.093561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833270+0000","last_change":"2024-09-21T00:18:38.673020+0000","last_active":"2024-09-21T00:19:25.833270+0000","last_peered":"2024-09-21T00:19:25.833270+0000","last_clean":"2024-09-21T00:19:25.833270+0000","last_became_active":"2024-09-21T00:18:38.672391+0000","last_became_peered":"2024-09-21T00:18:38.672391+0000","last_unstale":"2024-09-21T00:19:25.833270+0000","last_undegraded":"2024-09-21T00:19:25.833270+0000","last_fullsized":"2024-09-21T00:19:25.833270+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:13.866576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.835916+0000","last_change":"2024-09-21T00:18:35.615844+0000","last_active":"2024-09-21T00:19:25.835916+0000","last_peered":"2024-09-21T00:19:25.835916+0000","last_clean":"2024-09-21T00:19:25.835916+0000","last_became_active":"2024-09-21T00:18:35.615385+0000","last_became_peered":"2024-09-21T00:18:35.615385+0000","last_unstale":"2024-09-21T00:19:25.835916+0000","last_undegraded":"2024-09-21T00:19:25.835916+0000","last_fullsized":"2024-09-21T00:19:25.835916+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:03.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838419+0000","last_change":"2024-09-21T00:18:48.732919+0000","last_active":"2024-09-21T00:19:25.838419+0000","last_peered":"2024-09-21T00:19:25.838419+0000","last_clean":"2024-09-21T00:19:25.838419+0000","last_became_active":"2024-09-21T00:18:48.732407+0000","last_became_peered":"2024-09-21T00:18:48.732407+0000","last_unstale":"2024-09-21T00:19:25.838419+0000","last_undegraded":"2024-09-21T00:19:25.838419+0000","last_fullsized":"2024-09-21T00:19:25.838419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:24:37.299373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833450+0000","last_change":"2024-09-21T00:18:38.671280+0000","last_active":"2024-09-21T00:19:25.833450+0000","last_peered":"2024-09-21T00:19:25.833450+0000","last_clean":"2024-09-21T00:19:25.833450+0000","last_became_active":"2024-09-21T00:18:38.670770+0000","last_became_peered":"2024-09-21T00:18:38.670770+0000","last_unstale":"2024-09-21T00:19:25.833450+0000","last_undegraded":"2024-09-21T00:19:25.833450+0000","last_fullsized":"2024-09-21T00:19:25.833450+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:06.809450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831651+0000","last_change":"2024-09-21T00:18:35.626071+0000","last_active":"2024-09-21T00:19:25.831651+0000","last_peered":"2024-09-21T00:19:25.831651+0000","last_clean":"2024-09-21T00:19:25.831651+0000","last_became_active":"2024-09-21T00:18:35.625649+0000","last_became_peered":"2024-09-21T00:18:35.625649+0000","last_unstale":"2024-09-21T00:19:25.831651+0000","last_undegraded":"2024-09-21T00:19:25.831651+0000","last_fullsized":"2024-09-21T00:19:25.831651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:16:27.516303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833806+0000","last_change":"2024-09-21T00:18:48.728451+0000","last_active":"2024-09-21T00:19:25.833806+0000","last_peered":"2024-09-21T00:19:25.833806+0000","last_clean":"2024-09-21T00:19:25.833806+0000","last_became_active":"2024-09-21T00:18:48.727840+0000","last_became_peered":"2024-09-21T00:18:48.727840+0000","last_unstale":"2024-09-21T00:19:25.833806+0000","last_undegraded":"2024-09-21T00:19:25.833806+0000","last_fullsized":"2024-09-21T00:19:25.833806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:13.457138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.835153+0000","last_change":"2024-09-21T00:18:38.664090+0000","last_active":"2024-09-21T00:19:25.835153+0000","last_peered":"2024-09-21T00:19:25.835153+0000","last_clean":"2024-09-21T00:19:25.835153+0000","last_became_active":"2024-09-21T00:18:38.663728+0000","last_became_peered":"2024-09-21T00:18:38.663728+0000","last_unstale":"2024-09-21T00:19:25.835153+0000","last_undegraded":"2024-09-21T00:19:25.835153+0000","last_fullsized":"2024-09-21T00:19:25.835153+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:46.367766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:35.897937+0000","last_change":"2024-09-21T00:19:25.847280+0000","last_active":"2024-09-21T00:19:35.897937+0000","last_peered":"2024-09-21T00:19:35.897937+0000","last_clean":"2024-09-21T00:19:35.897937+0000","last_became_active":"2024-09-21T00:19:25.840919+0000","last_became_peered":"2024-09-21T00:19:25.840919+0000","last_unstale":"2024-09-21T00:19:35.897937+0000","last_undegraded":"2024-09-21T00:19:35.897937+0000","last_fullsized":"2024-09-21T00:19:35.897937+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:15.411674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834153+0000","last_change":"2024-09-21T00:18:48.732427+0000","last_active":"2024-09-21T00:19:25.834153+0000","last_peered":"2024-09-21T00:19:25.834153+0000","last_clean":"2024-09-21T00:19:25.834153+0000","last_became_active":"2024-09-21T00:18:48.732078+0000","last_became_peered":"2024-09-21T00:18:48.732078+0000","last_unstale":"2024-09-21T00:19:25.834153+0000","last_undegraded":"2024-09-21T00:19:25.834153+0000","last_fullsized":"2024-09-21T00:19:25.834153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:35.055680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834003+0000","last_change":"2024-09-21T00:18:38.675022+0000","last_active":"2024-09-21T00:19:25.834003+0000","last_peered":"2024-09-21T00:19:25.834003+0000","last_clean":"2024-09-21T00:19:25.834003+0000","last_became_active":"2024-09-21T00:18:38.674650+0000","last_became_peered":"2024-09-21T00:18:38.674650+0000","last_unstale":"2024-09-21T00:19:25.834003+0000","last_undegraded":"2024-09-21T00:19:25.834003+0000","last_fullsized":"2024-09-21T00:19:25.834003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:05.191404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.904121+0000","last_change":"2024-09-21T00:18:35.769555+0000","last_active":"2024-09-21T00:19:33.904121+0000","last_peered":"2024-09-21T00:19:33.904121+0000","last_clean":"2024-09-21T00:19:33.904121+0000","last_became_active":"2024-09-21T00:18:35.769080+0000","last_became_peered":"2024-09-21T00:18:35.769080+0000","last_unstale":"2024-09-21T00:19:33.904121+0000","last_undegraded":"2024-09-21T00:19:33.904121+0000","last_fullsized":"2024-09-21T00:19:33.904121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:19.306830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831650+0000","last_change":"2024-09-21T00:18:48.742957+0000","last_active":"2024-09-21T00:19:25.831650+0000","last_peered":"2024-09-21T00:19:25.831650+0000","last_clean":"2024-09-21T00:19:25.831650+0000","last_became_active":"2024-09-21T00:18:48.742511+0000","last_became_peered":"2024-09-21T00:18:48.742511+0000","last_unstale":"2024-09-21T00:19:25.831650+0000","last_undegraded":"2024-09-21T00:19:25.831650+0000","last_fullsized":"2024-09-21T00:19:25.831650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:15.231189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.832619+0000","last_change":"2024-09-21T00:18:38.666576+0000","last_active":"2024-09-21T00:19:25.832619+0000","last_peered":"2024-09-21T00:19:25.832619+0000","last_clean":"2024-09-21T00:19:25.832619+0000","last_became_active":"2024-09-21T00:18:38.665643+0000","last_became_peered":"2024-09-21T00:18:38.665643+0000","last_unstale":"2024-09-21T00:19:25.832619+0000","last_undegraded":"2024-09-21T00:19:25.832619+0000","last_fullsized":"2024-09-21T00:19:25.832619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:35.397847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":67,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:54.956951+0000","last_change":"2024-09-21T00:18:35.623143+0000","last_active":"2024-09-21T00:19:54.956951+0000","last_peered":"2024-09-21T00:19:54.956951+0000","last_clean":"2024-09-21T00:19:54.956951+0000","last_became_active":"2024-09-21T00:18:35.622693+0000","last_became_peered":"2024-09-21T00:18:35.622693+0000","last_unstale":"2024-09-21T00:19:54.956951+0000","last_undegraded":"2024-09-21T00:19:54.956951+0000","last_fullsized":"2024-09-21T00:19:54.956951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:01.331665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.830550+0000","last_change":"2024-09-21T00:18:48.723877+0000","last_active":"2024-09-21T00:19:25.830550+0000","last_peered":"2024-09-21T00:19:25.830550+0000","last_clean":"2024-09-21T00:19:25.830550+0000","last_became_active":"2024-09-21T00:18:48.723360+0000","last_became_peered":"2024-09-21T00:18:48.723360+0000","last_unstale":"2024-09-21T00:19:25.830550+0000","last_undegraded":"2024-09-21T00:19:25.830550+0000","last_fullsized":"2024-09-21T00:19:25.830550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:07:20.442867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833264+0000","last_change":"2024-09-21T00:18:38.669643+0000","last_active":"2024-09-21T00:19:25.833264+0000","last_peered":"2024-09-21T00:19:25.833264+0000","last_clean":"2024-09-21T00:19:25.833264+0000","last_became_active":"2024-09-21T00:18:38.669245+0000","last_became_peered":"2024-09-21T00:18:38.669245+0000","last_unstale":"2024-09-21T00:19:25.833264+0000","last_undegraded":"2024-09-21T00:19:25.833264+0000","last_fullsized":"2024-09-21T00:19:25.833264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:17.789291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.897675+0000","last_change":"2024-09-21T00:18:35.623189+0000","last_active":"2024-09-21T00:19:32.897675+0000","last_peered":"2024-09-21T00:19:32.897675+0000","last_clean":"2024-09-21T00:19:32.897675+0000","last_became_active":"2024-09-21T00:18:35.622748+0000","last_became_peered":"2024-09-21T00:18:35.622748+0000","last_unstale":"2024-09-21T00:19:32.897675+0000","last_undegraded":"2024-09-21T00:19:32.897675+0000","last_fullsized":"2024-09-21T00:19:32.897675+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:15.570789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834463+0000","last_change":"2024-09-21T00:18:48.739612+0000","last_active":"2024-09-21T00:19:25.834463+0000","last_peered":"2024-09-21T00:19:25.834463+0000","last_clean":"2024-09-21T00:19:25.834463+0000","last_became_active":"2024-09-21T00:18:48.739173+0000","last_became_peered":"2024-09-21T00:18:48.739173+0000","last_unstale":"2024-09-21T00:19:25.834463+0000","last_undegraded":"2024-09-21T00:19:25.834463+0000","last_fullsized":"2024-09-21T00:19:25.834463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:26.327178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831104+0000","last_change":"2024-09-21T00:18:38.670876+0000","last_active":"2024-09-21T00:19:25.831104+0000","last_peered":"2024-09-21T00:19:25.831104+0000","last_clean":"2024-09-21T00:19:25.831104+0000","last_became_active":"2024-09-21T00:18:38.670371+0000","last_became_peered":"2024-09-21T00:18:38.670371+0000","last_unstale":"2024-09-21T00:19:25.831104+0000","last_undegraded":"2024-09-21T00:19:25.831104+0000","last_fullsized":"2024-09-21T00:19:25.831104+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:48.641071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.904808+0000","last_change":"2024-09-21T00:18:35.622919+0000","last_active":"2024-09-21T00:19:33.904808+0000","last_peered":"2024-09-21T00:19:33.904808+0000","last_clean":"2024-09-21T00:19:33.904808+0000","last_became_active":"2024-09-21T00:18:35.622435+0000","last_became_peered":"2024-09-21T00:18:35.622435+0000","last_unstale":"2024-09-21T00:19:33.904808+0000","last_undegraded":"2024-09-21T00:19:33.904808+0000","last_fullsized":"2024-09-21T00:19:33.904808+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:22.113871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.835031+0000","last_change":"2024-09-21T00:18:48.741736+0000","last_active":"2024-09-21T00:19:25.835031+0000","last_peered":"2024-09-21T00:19:25.835031+0000","last_clean":"2024-09-21T00:19:25.835031+0000","last_became_active":"2024-09-21T00:18:48.741135+0000","last_became_peered":"2024-09-21T00:18:48.741135+0000","last_unstale":"2024-09-21T00:19:25.835031+0000","last_undegraded":"2024-09-21T00:19:25.835031+0000","last_fullsized":"2024-09-21T00:19:25.835031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:50.492351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840391+0000","last_change":"2024-09-21T00:18:38.667377+0000","last_active":"2024-09-21T00:19:25.840391+0000","last_peered":"2024-09-21T00:19:25.840391+0000","last_clean":"2024-09-21T00:19:25.840391+0000","last_became_active":"2024-09-21T00:18:38.667010+0000","last_became_peered":"2024-09-21T00:18:38.667010+0000","last_unstale":"2024-09-21T00:19:25.840391+0000","last_undegraded":"2024-09-21T00:19:25.840391+0000","last_fullsized":"2024-09-21T00:19:25.840391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:06.634767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'29","reported_seq":134,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:54.956648+0000","last_change":"2024-09-21T00:19:25.853603+0000","last_active":"2024-09-21T00:19:54.956648+0000","last_peered":"2024-09-21T00:19:54.956648+0000","last_clean":"2024-09-21T00:19:54.956648+0000","last_became_active":"2024-09-21T00:19:25.841126+0000","last_became_peered":"2024-09-21T00:19:25.841126+0000","last_unstale":"2024-09-21T00:19:54.956648+0000","last_undegraded":"2024-09-21T00:19:54.956648+0000","last_fullsized":"2024-09-21T00:19:54.956648+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:27:47.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61394,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":47,"num_read_kb":83,"num_write":30,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.829810+0000","last_change":"2024-09-21T00:18:48.731004+0000","last_active":"2024-09-21T00:19:25.829810+0000","last_peered":"2024-09-21T00:19:25.829810+0000","last_clean":"2024-09-21T00:19:25.829810+0000","last_became_active":"2024-09-21T00:18:48.730549+0000","last_became_peered":"2024-09-21T00:18:48.730549+0000","last_unstale":"2024-09-21T00:19:25.829810+0000","last_undegraded":"2024-09-21T00:19:25.829810+0000","last_fullsized":"2024-09-21T00:19:25.829810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:30.988300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.843618+0000","last_change":"2024-09-21T00:19:25.843532+0000","last_active":"2024-09-21T00:19:25.843618+0000","last_peered":"2024-09-21T00:19:25.843618+0000","last_clean":"2024-09-21T00:19:25.843618+0000","last_became_active":"2024-09-21T00:19:25.843065+0000","last_became_peered":"2024-09-21T00:19:25.843065+0000","last_unstale":"2024-09-21T00:19:25.843618+0000","last_undegraded":"2024-09-21T00:19:25.843618+0000","last_fullsized":"2024-09-21T00:19:25.843618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:36:26.488692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.894355+0000","last_change":"2024-09-21T00:18:35.770494+0000","last_active":"2024-09-21T00:19:32.894355+0000","last_peered":"2024-09-21T00:19:32.894355+0000","last_clean":"2024-09-21T00:19:32.894355+0000","last_became_active":"2024-09-21T00:18:35.770122+0000","last_became_peered":"2024-09-21T00:18:35.770122+0000","last_unstale":"2024-09-21T00:19:32.894355+0000","last_undegraded":"2024-09-21T00:19:32.894355+0000","last_fullsized":"2024-09-21T00:19:32.894355+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:41.415904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838481+0000","last_change":"2024-09-21T00:18:48.736802+0000","last_active":"2024-09-21T00:19:25.838481+0000","last_peered":"2024-09-21T00:19:25.838481+0000","last_clean":"2024-09-21T00:19:25.838481+0000","last_became_active":"2024-09-21T00:18:48.736008+0000","last_became_peered":"2024-09-21T00:18:48.736008+0000","last_unstale":"2024-09-21T00:19:25.838481+0000","last_undegraded":"2024-09-21T00:19:25.838481+0000","last_fullsized":"2024-09-21T00:19:25.838481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:07.995614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838989+0000","last_change":"2024-09-21T00:18:38.678385+0000","last_active":"2024-09-21T00:19:25.838989+0000","last_peered":"2024-09-21T00:19:25.838989+0000","last_clean":"2024-09-21T00:19:25.838989+0000","last_became_active":"2024-09-21T00:18:38.677174+0000","last_became_peered":"2024-09-21T00:18:38.677174+0000","last_unstale":"2024-09-21T00:19:25.838989+0000","last_undegraded":"2024-09-21T00:19:25.838989+0000","last_fullsized":"2024-09-21T00:19:25.838989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:23.486222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:32.893717+0000","last_change":"2024-09-21T00:18:35.627703+0000","last_active":"2024-09-21T00:19:32.893717+0000","last_peered":"2024-09-21T00:19:32.893717+0000","last_clean":"2024-09-21T00:19:32.893717+0000","last_became_active":"2024-09-21T00:18:35.627274+0000","last_became_peered":"2024-09-21T00:18:35.627274+0000","last_unstale":"2024-09-21T00:19:32.893717+0000","last_undegraded":"2024-09-21T00:19:32.893717+0000","last_fullsized":"2024-09-21T00:19:32.893717+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:14.326810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.829532+0000","last_change":"2024-09-21T00:18:48.729926+0000","last_active":"2024-09-21T00:19:25.829532+0000","last_peered":"2024-09-21T00:19:25.829532+0000","last_clean":"2024-09-21T00:19:25.829532+0000","last_became_active":"2024-09-21T00:18:48.728366+0000","last_became_peered":"2024-09-21T00:18:48.728366+0000","last_unstale":"2024-09-21T00:19:25.829532+0000","last_undegraded":"2024-09-21T00:19:25.829532+0000","last_fullsized":"2024-09-21T00:19:25.829532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:48:50.005748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838880+0000","last_change":"2024-09-21T00:18:38.664173+0000","last_active":"2024-09-21T00:19:25.838880+0000","last_peered":"2024-09-21T00:19:25.838880+0000","last_clean":"2024-09-21T00:19:25.838880+0000","last_became_active":"2024-09-21T00:18:38.663358+0000","last_became_peered":"2024-09-21T00:18:38.663358+0000","last_unstale":"2024-09-21T00:19:25.838880+0000","last_undegraded":"2024-09-21T00:19:25.838880+0000","last_fullsized":"2024-09-21T00:19:25.838880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:34:31.203621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":64,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:53.953851+0000","last_change":"2024-09-21T00:18:35.622133+0000","last_active":"2024-09-21T00:19:53.953851+0000","last_peered":"2024-09-21T00:19:53.953851+0000","last_clean":"2024-09-21T00:19:53.953851+0000","last_became_active":"2024-09-21T00:18:35.621032+0000","last_became_peered":"2024-09-21T00:18:35.621032+0000","last_unstale":"2024-09-21T00:19:53.953851+0000","last_undegraded":"2024-09-21T00:19:53.953851+0000","last_fullsized":"2024-09-21T00:19:53.953851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:24.121665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840153+0000","last_change":"2024-09-21T00:18:48.733339+0000","last_active":"2024-09-21T00:19:25.840153+0000","last_peered":"2024-09-21T00:19:25.840153+0000","last_clean":"2024-09-21T00:19:25.840153+0000","last_became_active":"2024-09-21T00:18:48.731843+0000","last_became_peered":"2024-09-21T00:18:48.731843+0000","last_unstale":"2024-09-21T00:19:25.840153+0000","last_undegraded":"2024-09-21T00:19:25.840153+0000","last_fullsized":"2024-09-21T00:19:25.840153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:42:55.474197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838543+0000","last_change":"2024-09-21T00:18:48.740108+0000","last_active":"2024-09-21T00:19:25.838543+0000","last_peered":"2024-09-21T00:19:25.838543+0000","last_clean":"2024-09-21T00:19:25.838543+0000","last_became_active":"2024-09-21T00:18:48.739330+0000","last_became_peered":"2024-09-21T00:18:48.739330+0000","last_unstale":"2024-09-21T00:19:25.838543+0000","last_undegraded":"2024-09-21T00:19:25.838543+0000","last_fullsized":"2024-09-21T00:19:25.838543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:00.103502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.829715+0000","last_change":"2024-09-21T00:18:38.660469+0000","last_active":"2024-09-21T00:19:25.829715+0000","last_peered":"2024-09-21T00:19:25.829715+0000","last_clean":"2024-09-21T00:19:25.829715+0000","last_became_active":"2024-09-21T00:18:38.660138+0000","last_became_peered":"2024-09-21T00:18:38.660138+0000","last_unstale":"2024-09-21T00:19:25.829715+0000","last_undegraded":"2024-09-21T00:19:25.829715+0000","last_fullsized":"2024-09-21T00:19:25.829715+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:01.079112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834134+0000","last_change":"2024-09-21T00:18:35.624528+0000","last_active":"2024-09-21T00:19:25.834134+0000","last_peered":"2024-09-21T00:19:25.834134+0000","last_clean":"2024-09-21T00:19:25.834134+0000","last_became_active":"2024-09-21T00:18:35.624170+0000","last_became_peered":"2024-09-21T00:18:35.624170+0000","last_unstale":"2024-09-21T00:19:25.834134+0000","last_undegraded":"2024-09-21T00:19:25.834134+0000","last_fullsized":"2024-09-21T00:19:25.834134+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:07.914521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834318+0000","last_change":"2024-09-21T00:18:48.719277+0000","last_active":"2024-09-21T00:19:25.834318+0000","last_peered":"2024-09-21T00:19:25.834318+0000","last_clean":"2024-09-21T00:19:25.834318+0000","last_became_active":"2024-09-21T00:18:48.718412+0000","last_became_peered":"2024-09-21T00:18:48.718412+0000","last_unstale":"2024-09-21T00:19:25.834318+0000","last_undegraded":"2024-09-21T00:19:25.834318+0000","last_fullsized":"2024-09-21T00:19:25.834318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:33.741043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.841082+0000","last_change":"2024-09-21T00:18:38.666928+0000","last_active":"2024-09-21T00:19:25.841082+0000","last_peered":"2024-09-21T00:19:25.841082+0000","last_clean":"2024-09-21T00:19:25.841082+0000","last_became_active":"2024-09-21T00:18:38.666236+0000","last_became_peered":"2024-09-21T00:18:38.666236+0000","last_unstale":"2024-09-21T00:19:25.841082+0000","last_undegraded":"2024-09-21T00:19:25.841082+0000","last_fullsized":"2024-09-21T00:19:25.841082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:21.043778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.906573+0000","last_change":"2024-09-21T00:18:35.624514+0000","last_active":"2024-09-21T00:19:33.906573+0000","last_peered":"2024-09-21T00:19:33.906573+0000","last_clean":"2024-09-21T00:19:33.906573+0000","last_became_active":"2024-09-21T00:18:35.624098+0000","last_became_peered":"2024-09-21T00:18:35.624098+0000","last_unstale":"2024-09-21T00:19:33.906573+0000","last_undegraded":"2024-09-21T00:19:33.906573+0000","last_fullsized":"2024-09-21T00:19:33.906573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:21.777568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:33.905229+0000","last_change":"2024-09-21T00:19:25.841232+0000","last_active":"2024-09-21T00:19:33.905229+0000","last_peered":"2024-09-21T00:19:33.905229+0000","last_clean":"2024-09-21T00:19:33.905229+0000","last_became_active":"2024-09-21T00:19:25.840655+0000","last_became_peered":"2024-09-21T00:19:25.840655+0000","last_unstale":"2024-09-21T00:19:33.905229+0000","last_undegraded":"2024-09-21T00:19:33.905229+0000","last_fullsized":"2024-09-21T00:19:33.905229+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:05.846222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834868+0000","last_change":"2024-09-21T00:18:38.667466+0000","last_active":"2024-09-21T00:19:25.834868+0000","last_peered":"2024-09-21T00:19:25.834868+0000","last_clean":"2024-09-21T00:19:25.834868+0000","last_became_active":"2024-09-21T00:18:38.666960+0000","last_became_peered":"2024-09-21T00:18:38.666960+0000","last_unstale":"2024-09-21T00:19:25.834868+0000","last_undegraded":"2024-09-21T00:19:25.834868+0000","last_fullsized":"2024-09-21T00:19:25.834868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:29:56.614766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.837614+0000","last_change":"2024-09-21T00:18:48.739656+0000","last_active":"2024-09-21T00:19:25.837614+0000","last_peered":"2024-09-21T00:19:25.837614+0000","last_clean":"2024-09-21T00:19:25.837614+0000","last_became_active":"2024-09-21T00:18:48.739228+0000","last_became_peered":"2024-09-21T00:18:48.739228+0000","last_unstale":"2024-09-21T00:19:25.837614+0000","last_undegraded":"2024-09-21T00:19:25.837614+0000","last_fullsized":"2024-09-21T00:19:25.837614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:11.427595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:49.947580+0000","last_change":"2024-09-21T00:18:35.624108+0000","last_active":"2024-09-21T00:19:49.947580+0000","last_peered":"2024-09-21T00:19:49.947580+0000","last_clean":"2024-09-21T00:19:49.947580+0000","last_became_active":"2024-09-21T00:18:35.623708+0000","last_became_peered":"2024-09-21T00:18:35.623708+0000","last_unstale":"2024-09-21T00:19:49.947580+0000","last_undegraded":"2024-09-21T00:19:49.947580+0000","last_fullsized":"2024-09-21T00:19:49.947580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:50.020662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.839902+0000","last_change":"2024-09-21T00:18:38.665402+0000","last_active":"2024-09-21T00:19:25.839902+0000","last_peered":"2024-09-21T00:19:25.839902+0000","last_clean":"2024-09-21T00:19:25.839902+0000","last_became_active":"2024-09-21T00:18:38.665034+0000","last_became_peered":"2024-09-21T00:18:38.665034+0000","last_unstale":"2024-09-21T00:19:25.839902+0000","last_undegraded":"2024-09-21T00:19:25.839902+0000","last_fullsized":"2024-09-21T00:19:25.839902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:33:10.824705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838413+0000","last_change":"2024-09-21T00:18:48.729928+0000","last_active":"2024-09-21T00:19:25.838413+0000","last_peered":"2024-09-21T00:19:25.838413+0000","last_clean":"2024-09-21T00:19:25.838413+0000","last_became_active":"2024-09-21T00:18:48.729498+0000","last_became_peered":"2024-09-21T00:18:48.729498+0000","last_unstale":"2024-09-21T00:19:25.838413+0000","last_undegraded":"2024-09-21T00:19:25.838413+0000","last_fullsized":"2024-09-21T00:19:25.838413+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:10:27.140012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834147+0000","last_change":"2024-09-21T00:18:38.662221+0000","last_active":"2024-09-21T00:19:25.834147+0000","last_peered":"2024-09-21T00:19:25.834147+0000","last_clean":"2024-09-21T00:19:25.834147+0000","last_became_active":"2024-09-21T00:18:38.660543+0000","last_became_peered":"2024-09-21T00:18:38.660543+0000","last_unstale":"2024-09-21T00:19:25.834147+0000","last_undegraded":"2024-09-21T00:19:25.834147+0000","last_fullsized":"2024-09-21T00:19:25.834147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:43.159699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838253+0000","last_change":"2024-09-21T00:18:38.672696+0000","last_active":"2024-09-21T00:19:25.838253+0000","last_peered":"2024-09-21T00:19:25.838253+0000","last_clean":"2024-09-21T00:19:25.838253+0000","last_became_active":"2024-09-21T00:18:38.672356+0000","last_became_peered":"2024-09-21T00:18:38.672356+0000","last_unstale":"2024-09-21T00:19:25.838253+0000","last_undegraded":"2024-09-21T00:19:25.838253+0000","last_fullsized":"2024-09-21T00:19:25.838253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.549939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838037+0000","last_change":"2024-09-21T00:18:38.667017+0000","last_active":"2024-09-21T00:19:25.838037+0000","last_peered":"2024-09-21T00:19:25.838037+0000","last_clean":"2024-09-21T00:19:25.838037+0000","last_became_active":"2024-09-21T00:18:38.666643+0000","last_became_peered":"2024-09-21T00:18:38.666643+0000","last_unstale":"2024-09-21T00:19:25.838037+0000","last_undegraded":"2024-09-21T00:19:25.838037+0000","last_fullsized":"2024-09-21T00:19:25.838037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:30.496339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838471+0000","last_change":"2024-09-21T00:18:38.667705+0000","last_active":"2024-09-21T00:19:25.838471+0000","last_peered":"2024-09-21T00:19:25.838471+0000","last_clean":"2024-09-21T00:19:25.838471+0000","last_became_active":"2024-09-21T00:18:38.665141+0000","last_became_peered":"2024-09-21T00:18:38.665141+0000","last_unstale":"2024-09-21T00:19:25.838471+0000","last_undegraded":"2024-09-21T00:19:25.838471+0000","last_fullsized":"2024-09-21T00:19:25.838471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:17.624035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831393+0000","last_change":"2024-09-21T00:18:38.681364+0000","last_active":"2024-09-21T00:19:25.831393+0000","last_peered":"2024-09-21T00:19:25.831393+0000","last_clean":"2024-09-21T00:19:25.831393+0000","last_became_active":"2024-09-21T00:18:38.680937+0000","last_became_peered":"2024-09-21T00:18:38.680937+0000","last_unstale":"2024-09-21T00:19:25.831393+0000","last_undegraded":"2024-09-21T00:19:25.831393+0000","last_fullsized":"2024-09-21T00:19:25.831393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:34.206201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.843202+0000","last_change":"2024-09-21T00:19:25.843091+0000","last_active":"2024-09-21T00:19:25.843202+0000","last_peered":"2024-09-21T00:19:25.843202+0000","last_clean":"2024-09-21T00:19:25.843202+0000","last_became_active":"2024-09-21T00:19:25.842600+0000","last_became_peered":"2024-09-21T00:19:25.842600+0000","last_unstale":"2024-09-21T00:19:25.843202+0000","last_undegraded":"2024-09-21T00:19:25.843202+0000","last_fullsized":"2024-09-21T00:19:25.843202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:10.256474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838662+0000","last_change":"2024-09-21T00:18:38.668656+0000","last_active":"2024-09-21T00:19:25.838662+0000","last_peered":"2024-09-21T00:19:25.838662+0000","last_clean":"2024-09-21T00:19:25.838662+0000","last_became_active":"2024-09-21T00:18:38.668153+0000","last_became_peered":"2024-09-21T00:18:38.668153+0000","last_unstale":"2024-09-21T00:19:25.838662+0000","last_undegraded":"2024-09-21T00:19:25.838662+0000","last_fullsized":"2024-09-21T00:19:25.838662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:03.763609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840542+0000","last_change":"2024-09-21T00:18:38.675945+0000","last_active":"2024-09-21T00:19:25.840542+0000","last_peered":"2024-09-21T00:19:25.840542+0000","last_clean":"2024-09-21T00:19:25.840542+0000","last_became_active":"2024-09-21T00:18:38.675397+0000","last_became_peered":"2024-09-21T00:18:38.675397+0000","last_unstale":"2024-09-21T00:19:25.840542+0000","last_undegraded":"2024-09-21T00:19:25.840542+0000","last_fullsized":"2024-09-21T00:19:25.840542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:56.082314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833904+0000","last_change":"2024-09-21T00:18:38.662854+0000","last_active":"2024-09-21T00:19:25.833904+0000","last_peered":"2024-09-21T00:19:25.833904+0000","last_clean":"2024-09-21T00:19:25.833904+0000","last_became_active":"2024-09-21T00:18:38.662094+0000","last_became_peered":"2024-09-21T00:18:38.662094+0000","last_unstale":"2024-09-21T00:19:25.833904+0000","last_undegraded":"2024-09-21T00:19:25.833904+0000","last_fullsized":"2024-09-21T00:19:25.833904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:45.756641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840363+0000","last_change":"2024-09-21T00:18:38.663282+0000","last_active":"2024-09-21T00:19:25.840363+0000","last_peered":"2024-09-21T00:19:25.840363+0000","last_clean":"2024-09-21T00:19:25.840363+0000","last_became_active":"2024-09-21T00:18:38.662148+0000","last_became_peered":"2024-09-21T00:18:38.662148+0000","last_unstale":"2024-09-21T00:19:25.840363+0000","last_undegraded":"2024-09-21T00:19:25.840363+0000","last_fullsized":"2024-09-21T00:19:25.840363+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:51.090591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838054+0000","last_change":"2024-09-21T00:18:38.669184+0000","last_active":"2024-09-21T00:19:25.838054+0000","last_peered":"2024-09-21T00:19:25.838054+0000","last_clean":"2024-09-21T00:19:25.838054+0000","last_became_active":"2024-09-21T00:18:38.668799+0000","last_became_peered":"2024-09-21T00:18:38.668799+0000","last_unstale":"2024-09-21T00:19:25.838054+0000","last_undegraded":"2024-09-21T00:19:25.838054+0000","last_fullsized":"2024-09-21T00:19:25.838054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:49.642568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831320+0000","last_change":"2024-09-21T00:18:38.679501+0000","last_active":"2024-09-21T00:19:25.831320+0000","last_peered":"2024-09-21T00:19:25.831320+0000","last_clean":"2024-09-21T00:19:25.831320+0000","last_became_active":"2024-09-21T00:18:38.678582+0000","last_became_peered":"2024-09-21T00:18:38.678582+0000","last_unstale":"2024-09-21T00:19:25.831320+0000","last_undegraded":"2024-09-21T00:19:25.831320+0000","last_fullsized":"2024-09-21T00:19:25.831320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:30.277979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834494+0000","last_change":"2024-09-21T00:18:38.659657+0000","last_active":"2024-09-21T00:19:25.834494+0000","last_peered":"2024-09-21T00:19:25.834494+0000","last_clean":"2024-09-21T00:19:25.834494+0000","last_became_active":"2024-09-21T00:18:38.659257+0000","last_became_peered":"2024-09-21T00:18:38.659257+0000","last_unstale":"2024-09-21T00:19:25.834494+0000","last_undegraded":"2024-09-21T00:19:25.834494+0000","last_fullsized":"2024-09-21T00:19:25.834494+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:08.653729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838542+0000","last_change":"2024-09-21T00:18:38.671817+0000","last_active":"2024-09-21T00:19:25.838542+0000","last_peered":"2024-09-21T00:19:25.838542+0000","last_clean":"2024-09-21T00:19:25.838542+0000","last_became_active":"2024-09-21T00:18:38.671469+0000","last_became_peered":"2024-09-21T00:18:38.671469+0000","last_unstale":"2024-09-21T00:19:25.838542+0000","last_undegraded":"2024-09-21T00:19:25.838542+0000","last_fullsized":"2024-09-21T00:19:25.838542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:58:15.036623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.830413+0000","last_change":"2024-09-21T00:18:38.664481+0000","last_active":"2024-09-21T00:19:25.830413+0000","last_peered":"2024-09-21T00:19:25.830413+0000","last_clean":"2024-09-21T00:19:25.830413+0000","last_became_active":"2024-09-21T00:18:38.664098+0000","last_became_peered":"2024-09-21T00:18:38.664098+0000","last_unstale":"2024-09-21T00:19:25.830413+0000","last_undegraded":"2024-09-21T00:19:25.830413+0000","last_fullsized":"2024-09-21T00:19:25.830413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:09.271251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.835199+0000","last_change":"2024-09-21T00:18:38.662416+0000","last_active":"2024-09-21T00:19:25.835199+0000","last_peered":"2024-09-21T00:19:25.835199+0000","last_clean":"2024-09-21T00:19:25.835199+0000","last_became_active":"2024-09-21T00:18:38.662041+0000","last_became_peered":"2024-09-21T00:18:38.662041+0000","last_unstale":"2024-09-21T00:19:25.835199+0000","last_undegraded":"2024-09-21T00:19:25.835199+0000","last_fullsized":"2024-09-21T00:19:25.835199+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:03.150179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.837470+0000","last_change":"2024-09-21T00:18:38.670797+0000","last_active":"2024-09-21T00:19:25.837470+0000","last_peered":"2024-09-21T00:19:25.837470+0000","last_clean":"2024-09-21T00:19:25.837470+0000","last_became_active":"2024-09-21T00:18:38.670258+0000","last_became_peered":"2024-09-21T00:18:38.670258+0000","last_unstale":"2024-09-21T00:19:25.837470+0000","last_undegraded":"2024-09-21T00:19:25.837470+0000","last_fullsized":"2024-09-21T00:19:25.837470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:55.669836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838051+0000","last_change":"2024-09-21T00:18:38.665594+0000","last_active":"2024-09-21T00:19:25.838051+0000","last_peered":"2024-09-21T00:19:25.838051+0000","last_clean":"2024-09-21T00:19:25.838051+0000","last_became_active":"2024-09-21T00:18:38.665260+0000","last_became_peered":"2024-09-21T00:18:38.665260+0000","last_unstale":"2024-09-21T00:19:25.838051+0000","last_undegraded":"2024-09-21T00:19:25.838051+0000","last_fullsized":"2024-09-21T00:19:25.838051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:52:09.014327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838178+0000","last_change":"2024-09-21T00:18:38.679294+0000","last_active":"2024-09-21T00:19:25.838178+0000","last_peered":"2024-09-21T00:19:25.838178+0000","last_clean":"2024-09-21T00:19:25.838178+0000","last_became_active":"2024-09-21T00:18:38.673765+0000","last_became_peered":"2024-09-21T00:18:38.673765+0000","last_unstale":"2024-09-21T00:19:25.838178+0000","last_undegraded":"2024-09-21T00:19:25.838178+0000","last_fullsized":"2024-09-21T00:19:25.838178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:32.076299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.831190+0000","last_change":"2024-09-21T00:18:38.667869+0000","last_active":"2024-09-21T00:19:25.831190+0000","last_peered":"2024-09-21T00:19:25.831190+0000","last_clean":"2024-09-21T00:19:25.831190+0000","last_became_active":"2024-09-21T00:18:38.667499+0000","last_became_peered":"2024-09-21T00:18:38.667499+0000","last_unstale":"2024-09-21T00:19:25.831190+0000","last_undegraded":"2024-09-21T00:19:25.831190+0000","last_fullsized":"2024-09-21T00:19:25.831190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:13.501043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.839877+0000","last_change":"2024-09-21T00:18:38.663107+0000","last_active":"2024-09-21T00:19:25.839877+0000","last_peered":"2024-09-21T00:19:25.839877+0000","last_clean":"2024-09-21T00:19:25.839877+0000","last_became_active":"2024-09-21T00:18:38.662680+0000","last_became_peered":"2024-09-21T00:18:38.662680+0000","last_unstale":"2024-09-21T00:19:25.839877+0000","last_undegraded":"2024-09-21T00:19:25.839877+0000","last_fullsized":"2024-09-21T00:19:25.839877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:48.154705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.833535+0000","last_change":"2024-09-21T00:18:38.668167+0000","last_active":"2024-09-21T00:19:25.833535+0000","last_peered":"2024-09-21T00:19:25.833535+0000","last_clean":"2024-09-21T00:19:25.833535+0000","last_became_active":"2024-09-21T00:18:38.667842+0000","last_became_peered":"2024-09-21T00:18:38.667842+0000","last_unstale":"2024-09-21T00:19:25.833535+0000","last_undegraded":"2024-09-21T00:19:25.833535+0000","last_fullsized":"2024-09-21T00:19:25.833535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:55.358351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834667+0000","last_change":"2024-09-21T00:18:38.666580+0000","last_active":"2024-09-21T00:19:25.834667+0000","last_peered":"2024-09-21T00:19:25.834667+0000","last_clean":"2024-09-21T00:19:25.834667+0000","last_became_active":"2024-09-21T00:18:38.666231+0000","last_became_peered":"2024-09-21T00:18:38.666231+0000","last_unstale":"2024-09-21T00:19:25.834667+0000","last_undegraded":"2024-09-21T00:19:25.834667+0000","last_fullsized":"2024-09-21T00:19:25.834667+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:07.362516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838508+0000","last_change":"2024-09-21T00:18:38.667210+0000","last_active":"2024-09-21T00:19:25.838508+0000","last_peered":"2024-09-21T00:19:25.838508+0000","last_clean":"2024-09-21T00:19:25.838508+0000","last_became_active":"2024-09-21T00:18:38.666728+0000","last_became_peered":"2024-09-21T00:18:38.666728+0000","last_unstale":"2024-09-21T00:19:25.838508+0000","last_undegraded":"2024-09-21T00:19:25.838508+0000","last_fullsized":"2024-09-21T00:19:25.838508+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:14.712393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.840980+0000","last_change":"2024-09-21T00:18:38.666580+0000","last_active":"2024-09-21T00:19:25.840980+0000","last_peered":"2024-09-21T00:19:25.840980+0000","last_clean":"2024-09-21T00:19:25.840980+0000","last_became_active":"2024-09-21T00:18:38.665698+0000","last_became_peered":"2024-09-21T00:18:38.665698+0000","last_unstale":"2024-09-21T00:19:25.840980+0000","last_undegraded":"2024-09-21T00:19:25.840980+0000","last_fullsized":"2024-09-21T00:19:25.840980+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:45.627667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.832742+0000","last_change":"2024-09-21T00:18:38.670729+0000","last_active":"2024-09-21T00:19:25.832742+0000","last_peered":"2024-09-21T00:19:25.832742+0000","last_clean":"2024-09-21T00:19:25.832742+0000","last_became_active":"2024-09-21T00:18:38.670337+0000","last_became_peered":"2024-09-21T00:18:38.670337+0000","last_unstale":"2024-09-21T00:19:25.832742+0000","last_undegraded":"2024-09-21T00:19:25.832742+0000","last_fullsized":"2024-09-21T00:19:25.832742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:49.880689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.834131+0000","last_change":"2024-09-21T00:18:38.664644+0000","last_active":"2024-09-21T00:19:25.834131+0000","last_peered":"2024-09-21T00:19:25.834131+0000","last_clean":"2024-09-21T00:19:25.834131+0000","last_became_active":"2024-09-21T00:18:38.664220+0000","last_became_peered":"2024-09-21T00:18:38.664220+0000","last_unstale":"2024-09-21T00:19:25.834131+0000","last_undegraded":"2024-09-21T00:19:25.834131+0000","last_fullsized":"2024-09-21T00:19:25.834131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:09.610962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.839526+0000","last_change":"2024-09-21T00:18:38.671592+0000","last_active":"2024-09-21T00:19:25.839526+0000","last_peered":"2024-09-21T00:19:25.839526+0000","last_clean":"2024-09-21T00:19:25.839526+0000","last_became_active":"2024-09-21T00:18:38.671228+0000","last_became_peered":"2024-09-21T00:18:38.671228+0000","last_unstale":"2024-09-21T00:19:25.839526+0000","last_undegraded":"2024-09-21T00:19:25.839526+0000","last_fullsized":"2024-09-21T00:19:25.839526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:51.906058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.845109+0000","last_change":"2024-09-21T00:19:25.843696+0000","last_active":"2024-09-21T00:19:25.845109+0000","last_peered":"2024-09-21T00:19:25.845109+0000","last_clean":"2024-09-21T00:19:25.845109+0000","last_became_active":"2024-09-21T00:19:25.843287+0000","last_became_peered":"2024-09-21T00:19:25.843287+0000","last_unstale":"2024-09-21T00:19:25.845109+0000","last_undegraded":"2024-09-21T00:19:25.845109+0000","last_fullsized":"2024-09-21T00:19:25.845109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:21.024555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838751+0000","last_change":"2024-09-21T00:18:38.671820+0000","last_active":"2024-09-21T00:19:25.838751+0000","last_peered":"2024-09-21T00:19:25.838751+0000","last_clean":"2024-09-21T00:19:25.838751+0000","last_became_active":"2024-09-21T00:18:38.667107+0000","last_became_peered":"2024-09-21T00:18:38.667107+0000","last_unstale":"2024-09-21T00:19:25.838751+0000","last_undegraded":"2024-09-21T00:19:25.838751+0000","last_fullsized":"2024-09-21T00:19:25.838751+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:58.099750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.838713+0000","last_change":"2024-09-21T00:18:38.671271+0000","last_active":"2024-09-21T00:19:25.838713+0000","last_peered":"2024-09-21T00:19:25.838713+0000","last_clean":"2024-09-21T00:19:25.838713+0000","last_became_active":"2024-09-21T00:18:38.670826+0000","last_became_peered":"2024-09-21T00:18:38.670826+0000","last_unstale":"2024-09-21T00:19:25.838713+0000","last_undegraded":"2024-09-21T00:19:25.838713+0000","last_fullsized":"2024-09-21T00:19:25.838713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:57:01.723120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T00:19:25.835068+0000","last_change":"2024-09-21T00:18:38.662851+0000","last_active":"2024-09-21T00:19:25.835068+0000","last_peered":"2024-09-21T00:19:25.835068+0000","last_clean":"2024-09-21T00:19:25.835068+0000","last_became_active":"2024-09-21T00:18:38.661259+0000","last_became_peered":"2024-09-21T00:18:38.661259+0000","last_unstale":"2024-09-21T00:19:25.835068+0000","last_undegraded":"2024-09-21T00:19:25.835068+0000","last_fullsized":"2024-09-21T00:19:25.835068+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:10.508980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65395,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":141,"num_read_kb":140,"num_write":159,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3864,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678045,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":516007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874274,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37000,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708152,"statfs":{"total":95995035648,"available":95957147648,"internally_reserved":0,"allocated":1695744,"data_stored":1164647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037798,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":520317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234029,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":516119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430257,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1617920,"data_stored":1112050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593783,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":511397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790014,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1107638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953540,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":578023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":511591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378703,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":512466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542228,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":572701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705753,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":512785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:19:58.871 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T00:19:58.871 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T00:19:58.871 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T00:19:59.291 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:19:59.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:19:59 smithi027 ceph-mon[30745]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:19:59.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:19:59 smithi089 ceph-mon[34481]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:19:59.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:19:59 smithi098 ceph-mon[34725]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:20:00.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.013+0000 7fb863200640 1 -- 172.21.15.27:0/784783785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb86410b4f0 msgr2=0x7fb86410b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:00.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.013+0000 7fb863200640 1 --2- 172.21.15.27:0/784783785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb86410b4f0 0x7fb86410b900 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009900 tx=0x7fb84c02a410 comp rx=0 tx=0).stop 2024-09-21T00:20:00.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.013+0000 7fb863200640 1 -- 172.21.15.27:0/784783785 shutdown_connections 2024-09-21T00:20:00.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.013+0000 7fb863200640 1 --2- 172.21.15.27:0/784783785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb86410be40 0x7fb864112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:00.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.013+0000 7fb863200640 1 --2- 172.21.15.27:0/784783785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb86410b4f0 0x7fb86410b900 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:00.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.013+0000 7fb863200640 1 --2- 172.21.15.27:0/784783785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb864105bb0 0x7fb864105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:00.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.013+0000 7fb863200640 1 -- 172.21.15.27:0/784783785 >> 172.21.15.27:0/784783785 conn(0x7fb864101640 msgr2=0x7fb864103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:00.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.013+0000 7fb863200640 1 -- 172.21.15.27:0/784783785 shutdown_connections 2024-09-21T00:20:00.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.013+0000 7fb863200640 1 -- 172.21.15.27:0/784783785 wait complete. 2024-09-21T00:20:00.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.015+0000 7fb863200640 1 Processor -- start 2024-09-21T00:20:00.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.015+0000 7fb863200640 1 -- start start 2024-09-21T00:20:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.015+0000 7fb863200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb864105bb0 0x7fb86419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.015+0000 7fb863200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb86410b4f0 0x7fb86419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.016+0000 7fb863200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86410be40 0x7fb8641a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.016+0000 7fb863200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb864115b50 con 0x7fb864105bb0 2024-09-21T00:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.016+0000 7fb863200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641159d0 con 0x7fb86410b4f0 2024-09-21T00:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.016+0000 7fb863200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb864115cd0 con 0x7fb86410be40 2024-09-21T00:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.016+0000 7fb861e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb864105bb0 0x7fb86419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.016+0000 7fb861e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb864105bb0 0x7fb86419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46410/0 (socket says 172.21.15.27:46410) 2024-09-21T00:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.016+0000 7fb861e00640 1 -- 172.21.15.27:0/2447891220 learned_addr learned my addr 172.21.15.27:0/2447891220 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.016+0000 7fb85be00640 1 --2- 172.21.15.27:0/2447891220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb86410b4f0 0x7fb86419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.016+0000 7fb862800640 1 --2- 172.21.15.27:0/2447891220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86410be40 0x7fb8641a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.016+0000 7fb862800640 1 -- 172.21.15.27:0/2447891220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb86410b4f0 msgr2=0x7fb86419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.016+0000 7fb862800640 1 --2- 172.21.15.27:0/2447891220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb86410b4f0 0x7fb86419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:00.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.016+0000 7fb862800640 1 -- 172.21.15.27:0/2447891220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb864105bb0 msgr2=0x7fb86419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:00.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.016+0000 7fb862800640 1 --2- 172.21.15.27:0/2447891220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb864105bb0 0x7fb86419c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:00.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.016+0000 7fb862800640 1 -- 172.21.15.27:0/2447891220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8641a24d0 con 0x7fb86410be40 2024-09-21T00:20:00.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.016+0000 7fb85be00640 1 --2- 172.21.15.27:0/2447891220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb86410b4f0 0x7fb86419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:00.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.017+0000 7fb862800640 1 --2- 172.21.15.27:0/2447891220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86410be40 0x7fb8641a1da0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009fd0 tx=0x7fb85c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:00.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.017+0000 7fb85aa00640 1 -- 172.21.15.27:0/2447891220 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c00ce20 con 0x7fb86410be40 2024-09-21T00:20:00.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.017+0000 7fb85aa00640 1 -- 172.21.15.27:0/2447891220 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb85c005020 con 0x7fb86410be40 2024-09-21T00:20:00.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.017+0000 7fb863200640 1 -- 172.21.15.27:0/2447891220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8641a27c0 con 0x7fb86410be40 2024-09-21T00:20:00.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.017+0000 7fb85aa00640 1 -- 172.21.15.27:0/2447891220 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c010690 con 0x7fb86410be40 2024-09-21T00:20:00.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.017+0000 7fb863200640 1 -- 172.21.15.27:0/2447891220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8641a8f20 con 0x7fb86410be40 2024-09-21T00:20:00.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.018+0000 7fb863200640 1 -- 172.21.15.27:0/2447891220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb864105fc0 con 0x7fb86410be40 2024-09-21T00:20:00.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.021+0000 7fb85aa00640 1 -- 172.21.15.27:0/2447891220 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fb85c0054d0 con 0x7fb86410be40 2024-09-21T00:20:00.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.022+0000 7fb85aa00640 1 --2- 172.21.15.27:0/2447891220 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb8480b65e0 0x7fb8480b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:00.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.023+0000 7fb85aa00640 1 -- 172.21.15.27:0/2447891220 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7fb85c0e7f80 con 0x7fb86410be40 2024-09-21T00:20:00.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.023+0000 7fb861e00640 1 --2- 172.21.15.27:0/2447891220 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb8480b65e0 0x7fb8480b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:00.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.023+0000 7fb861e00640 1 --2- 172.21.15.27:0/2447891220 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb8480b65e0 0x7fb8480b8af0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb854009cd0 tx=0x7fb854005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.024+0000 7fb85aa00640 1 -- 172.21.15.27:0/2447891220 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb85c0ae220 con 0x7fb86410be40 2024-09-21T00:20:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.296+0000 7fb863200640 1 -- 172.21.15.27:0/2447891220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fb86410dc10 con 0x7fb86410be40 2024-09-21T00:20:00.297 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:00 smithi027 ceph-mon[30745]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-09-21T00:20:00.298 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:00 smithi027 ceph-mon[30745]: overall HEALTH_OK 2024-09-21T00:20:00.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.305+0000 7fb85aa00640 1 -- 172.21.15.27:0/2447891220 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fb85c0b2600 con 0x7fb86410be40 2024-09-21T00:20:00.306 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-21T00:20:00.306 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQAwEe5mVl3IERAA6eR4WkyLEQ/NT6483iGOFg== 2024-09-21T00:20:00.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.307+0000 7fb863200640 1 -- 172.21.15.27:0/2447891220 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb8480b65e0 msgr2=0x7fb8480b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:00.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.307+0000 7fb863200640 1 --2- 172.21.15.27:0/2447891220 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb8480b65e0 0x7fb8480b8af0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb854009cd0 tx=0x7fb854005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:00.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.307+0000 7fb863200640 1 -- 172.21.15.27:0/2447891220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86410be40 msgr2=0x7fb8641a1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:00.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.307+0000 7fb863200640 1 --2- 172.21.15.27:0/2447891220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86410be40 0x7fb8641a1da0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009fd0 tx=0x7fb85c00ef10 comp rx=0 tx=0).stop 2024-09-21T00:20:00.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.308+0000 7fb863200640 1 -- 172.21.15.27:0/2447891220 shutdown_connections 2024-09-21T00:20:00.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.308+0000 7fb863200640 1 --2- 172.21.15.27:0/2447891220 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb8480b65e0 0x7fb8480b8af0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:00.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.308+0000 7fb863200640 1 --2- 172.21.15.27:0/2447891220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86410be40 0x7fb8641a1da0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:00.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.308+0000 7fb863200640 1 --2- 172.21.15.27:0/2447891220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb86410b4f0 0x7fb86419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:00.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.308+0000 7fb863200640 1 --2- 172.21.15.27:0/2447891220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb864105bb0 0x7fb86419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:00.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.308+0000 7fb863200640 1 -- 172.21.15.27:0/2447891220 >> 172.21.15.27:0/2447891220 conn(0x7fb864101640 msgr2=0x7fb864103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:00.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.308+0000 7fb863200640 1 -- 172.21.15.27:0/2447891220 shutdown_connections 2024-09-21T00:20:00.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:00.308+0000 7fb863200640 1 -- 172.21.15.27:0/2447891220 wait complete. 2024-09-21T00:20:00.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:00 smithi089 ceph-mon[34481]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-09-21T00:20:00.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:00 smithi089 ceph-mon[34481]: overall HEALTH_OK 2024-09-21T00:20:00.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:00 smithi098 ceph-mon[34725]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-09-21T00:20:00.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:00 smithi098 ceph-mon[34725]: overall HEALTH_OK 2024-09-21T00:20:01.119 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:20:01.119 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T00:20:01.119 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T00:20:01.204 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T00:20:01.293 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:01 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2447891220' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T00:20:01.293 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:01 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T00:20:01.293 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:01 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T00:20:01.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:01 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2447891220' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T00:20:01.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:01 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T00:20:01.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:01 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T00:20:01.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:01 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2447891220' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T00:20:01.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:01 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T00:20:01.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:01 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T00:20:01.637 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:02.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.281+0000 7fb75ee00640 1 -- 172.21.15.27:0/1737264630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb758114580 msgr2=0x7fb7581169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:02.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.281+0000 7fb75ee00640 1 --2- 172.21.15.27:0/1737264630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb758114580 0x7fb7581169a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb750009ca0 tx=0x7fb75002a8c0 comp rx=0 tx=0).stop 2024-09-21T00:20:02.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.281+0000 7fb75ee00640 1 -- 172.21.15.27:0/1737264630 shutdown_connections 2024-09-21T00:20:02.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.281+0000 7fb75ee00640 1 --2- 172.21.15.27:0/1737264630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb758114580 0x7fb7581169a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:02.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.281+0000 7fb75ee00640 1 --2- 172.21.15.27:0/1737264630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb758078290 0x7fb758114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:02.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.281+0000 7fb75ee00640 1 --2- 172.21.15.27:0/1737264630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb758077940 0x7fb758077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:02.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.281+0000 7fb75ee00640 1 -- 172.21.15.27:0/1737264630 >> 172.21.15.27:0/1737264630 conn(0x7fb758101670 msgr2=0x7fb758103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:02.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.281+0000 7fb75ee00640 1 -- 172.21.15.27:0/1737264630 shutdown_connections 2024-09-21T00:20:02.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.281+0000 7fb75ee00640 1 -- 172.21.15.27:0/1737264630 wait complete. 2024-09-21T00:20:02.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.282+0000 7fb75ee00640 1 Processor -- start 2024-09-21T00:20:02.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.283+0000 7fb75ee00640 1 -- start start 2024-09-21T00:20:02.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.283+0000 7fb75ee00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb758077940 0x7fb7581a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:02.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.283+0000 7fb75ee00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb758078290 0x7fb7581a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:02.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.283+0000 7fb75ee00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb758114580 0x7fb7581a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:02.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.283+0000 7fb75ee00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb758119f30 con 0x7fb758114580 2024-09-21T00:20:02.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.283+0000 7fb75ee00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb758119db0 con 0x7fb758078290 2024-09-21T00:20:02.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.283+0000 7fb75ee00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb75811a0b0 con 0x7fb758077940 2024-09-21T00:20:02.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.283+0000 7fb75da00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb758077940 0x7fb7581a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:02.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.283+0000 7fb75e400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb758114580 0x7fb7581a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.283+0000 7fb75da00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb758077940 0x7fb7581a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:53122/0 (socket says 172.21.15.27:53122) 2024-09-21T00:20:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.283+0000 7fb75da00640 1 -- 172.21.15.27:0/3219257402 learned_addr learned my addr 172.21.15.27:0/3219257402 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.283+0000 7fb75d000640 1 --2- 172.21.15.27:0/3219257402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb758078290 0x7fb7581a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.284+0000 7fb75da00640 1 -- 172.21.15.27:0/3219257402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb758078290 msgr2=0x7fb7581a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.284+0000 7fb75da00640 1 --2- 172.21.15.27:0/3219257402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb758078290 0x7fb7581a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.284+0000 7fb75da00640 1 -- 172.21.15.27:0/3219257402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb758114580 msgr2=0x7fb7581a60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.284+0000 7fb75da00640 1 --2- 172.21.15.27:0/3219257402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb758114580 0x7fb7581a60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.284+0000 7fb75da00640 1 -- 172.21.15.27:0/3219257402 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7581a67f0 con 0x7fb758077940 2024-09-21T00:20:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.284+0000 7fb75da00640 1 --2- 172.21.15.27:0/3219257402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb758077940 0x7fb7581a0880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00e9f0 tx=0x7fb74c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.284+0000 7fb756000640 1 -- 172.21.15.27:0/3219257402 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c00cd80 con 0x7fb758077940 2024-09-21T00:20:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.284+0000 7fb756000640 1 -- 172.21.15.27:0/3219257402 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb74c0108e0 con 0x7fb758077940 2024-09-21T00:20:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.284+0000 7fb756000640 1 -- 172.21.15.27:0/3219257402 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c018980 con 0x7fb758077940 2024-09-21T00:20:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.284+0000 7fb75ee00640 1 -- 172.21.15.27:0/3219257402 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7581a6ae0 con 0x7fb758077940 2024-09-21T00:20:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.285+0000 7fb75ee00640 1 -- 172.21.15.27:0/3219257402 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7581ad240 con 0x7fb758077940 2024-09-21T00:20:02.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.286+0000 7fb75ee00640 1 -- 172.21.15.27:0/3219257402 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb758077d50 con 0x7fb758077940 2024-09-21T00:20:02.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.288+0000 7fb756000640 1 -- 172.21.15.27:0/3219257402 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fb74c010430 con 0x7fb758077940 2024-09-21T00:20:02.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.290+0000 7fb756000640 1 --2- 172.21.15.27:0/3219257402 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb7400b6670 0x7fb7400b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:02.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.290+0000 7fb756000640 1 -- 172.21.15.27:0/3219257402 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7fb74c0e80c0 con 0x7fb758077940 2024-09-21T00:20:02.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.290+0000 7fb75d000640 1 --2- 172.21.15.27:0/3219257402 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb7400b6670 0x7fb7400b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:02.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.291+0000 7fb75d000640 1 --2- 172.21.15.27:0/3219257402 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb7400b6670 0x7fb7400b8b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb758079ac0 tx=0x7fb744005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:02.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.291+0000 7fb756000640 1 -- 172.21.15.27:0/3219257402 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb74c0b2740 con 0x7fb758077940 2024-09-21T00:20:02.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:02 smithi089 ceph-mon[34481]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-09-21T00:20:02.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.492+0000 7fb75ee00640 1 -- 172.21.15.27:0/3219257402 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fb75802e640 con 0x7fb758077940 2024-09-21T00:20:02.495 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:02 smithi027 ceph-mon[30745]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-09-21T00:20:02.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:02 smithi098 ceph-mon[34725]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-09-21T00:20:02.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.493+0000 7fb756000640 1 -- 172.21.15.27:0/3219257402 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v56) ==== 86+0+133 (secure 0 0 0) 0x7fb74c0b7680 con 0x7fb758077940 2024-09-21T00:20:02.496 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:20:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.496+0000 7fb75ee00640 1 -- 172.21.15.27:0/3219257402 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb7400b6670 msgr2=0x7fb7400b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.496+0000 7fb75ee00640 1 --2- 172.21.15.27:0/3219257402 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb7400b6670 0x7fb7400b8b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb758079ac0 tx=0x7fb744005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.496+0000 7fb75ee00640 1 -- 172.21.15.27:0/3219257402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb758077940 msgr2=0x7fb7581a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.496+0000 7fb75ee00640 1 --2- 172.21.15.27:0/3219257402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb758077940 0x7fb7581a0880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00e9f0 tx=0x7fb74c00eeb0 comp rx=0 tx=0).stop 2024-09-21T00:20:02.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.496+0000 7fb75ee00640 1 -- 172.21.15.27:0/3219257402 shutdown_connections 2024-09-21T00:20:02.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.496+0000 7fb75ee00640 1 --2- 172.21.15.27:0/3219257402 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb7400b6670 0x7fb7400b8b80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:02.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.496+0000 7fb75ee00640 1 --2- 172.21.15.27:0/3219257402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb758114580 0x7fb7581a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:02.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.496+0000 7fb75ee00640 1 --2- 172.21.15.27:0/3219257402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb758078290 0x7fb7581a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:02.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.496+0000 7fb75ee00640 1 --2- 172.21.15.27:0/3219257402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb758077940 0x7fb7581a0880 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:02.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.496+0000 7fb75ee00640 1 -- 172.21.15.27:0/3219257402 >> 172.21.15.27:0/3219257402 conn(0x7fb758101670 msgr2=0x7fb7581165b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:02.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.496+0000 7fb75ee00640 1 -- 172.21.15.27:0/3219257402 shutdown_connections 2024-09-21T00:20:02.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:02.496+0000 7fb75ee00640 1 -- 172.21.15.27:0/3219257402 wait complete. 2024-09-21T00:20:03.256 INFO:teuthology.orchestra.run.smithi027.stdout:[{"version":56,"timestamp":"2024-09-21T00:18:27.332705+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T00:20:03.257 INFO:tasks.ceph_manager:config epoch is 56 2024-09-21T00:20:03.257 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T00:20:03.257 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T00:20:03.257 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T00:20:03.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:03 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3219257402' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T00:20:03.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:03 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3219257402' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T00:20:03.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:03 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3219257402' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T00:20:03.641 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:04.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.297+0000 7fb9e1400640 1 -- 172.21.15.27:0/3552115821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9dc1100c0 msgr2=0x7fb9dc112500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.297+0000 7fb9e1400640 1 --2- 172.21.15.27:0/3552115821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9dc1100c0 0x7fb9dc112500 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009ca0 tx=0x7fb9d402a8c0 comp rx=0 tx=0).stop 2024-09-21T00:20:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.297+0000 7fb9e1400640 1 -- 172.21.15.27:0/3552115821 shutdown_connections 2024-09-21T00:20:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.297+0000 7fb9e1400640 1 --2- 172.21.15.27:0/3552115821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9dc1100c0 0x7fb9dc112500 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.297+0000 7fb9e1400640 1 --2- 172.21.15.27:0/3552115821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9dc076100 0x7fb9dc10fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.297+0000 7fb9e1400640 1 --2- 172.21.15.27:0/3552115821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9dc0757b0 0x7fb9dc075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.297+0000 7fb9e1400640 1 -- 172.21.15.27:0/3552115821 >> 172.21.15.27:0/3552115821 conn(0x7fb9dc07b850 msgr2=0x7fb9dc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.297+0000 7fb9e1400640 1 -- 172.21.15.27:0/3552115821 shutdown_connections 2024-09-21T00:20:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.297+0000 7fb9e1400640 1 -- 172.21.15.27:0/3552115821 wait complete. 2024-09-21T00:20:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.298+0000 7fb9e1400640 1 Processor -- start 2024-09-21T00:20:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.298+0000 7fb9e1400640 1 -- start start 2024-09-21T00:20:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.299+0000 7fb9e1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9dc0757b0 0x7fb9dc19c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.299+0000 7fb9e1400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9dc076100 0x7fb9dc19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.299+0000 7fb9e1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9dc1100c0 0x7fb9dc1a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.299+0000 7fb9e1400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc115cc0 con 0x7fb9dc0757b0 2024-09-21T00:20:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.299+0000 7fb9e1400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc115b40 con 0x7fb9dc1100c0 2024-09-21T00:20:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.299+0000 7fb9e1400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc115e40 con 0x7fb9dc076100 2024-09-21T00:20:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.299+0000 7fb9da800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9dc0757b0 0x7fb9dc19c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.299+0000 7fb9d3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9dc076100 0x7fb9dc19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.299+0000 7fb9da800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9dc0757b0 0x7fb9dc19c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48120/0 (socket says 172.21.15.27:48120) 2024-09-21T00:20:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.299+0000 7fb9da800640 1 -- 172.21.15.27:0/764369744 learned_addr learned my addr 172.21.15.27:0/764369744 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.299+0000 7fb9db200640 1 --2- 172.21.15.27:0/764369744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9dc1100c0 0x7fb9dc1a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:04.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.300+0000 7fb9d3e00640 1 -- 172.21.15.27:0/764369744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9dc1100c0 msgr2=0x7fb9dc1a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:04.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.300+0000 7fb9d3e00640 1 --2- 172.21.15.27:0/764369744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9dc1100c0 0x7fb9dc1a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:04.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.300+0000 7fb9d3e00640 1 -- 172.21.15.27:0/764369744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9dc0757b0 msgr2=0x7fb9dc19c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:04.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.300+0000 7fb9d3e00640 1 --2- 172.21.15.27:0/764369744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9dc0757b0 0x7fb9dc19c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:04.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.300+0000 7fb9d3e00640 1 -- 172.21.15.27:0/764369744 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc1a2560 con 0x7fb9dc076100 2024-09-21T00:20:04.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.300+0000 7fb9da800640 1 --2- 172.21.15.27:0/764369744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9dc0757b0 0x7fb9dc19c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:04.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.300+0000 7fb9d3e00640 1 --2- 172.21.15.27:0/764369744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9dc076100 0x7fb9dc19cb90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400e9c0 tx=0x7fb9c400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:04.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.300+0000 7fb9d3400640 1 -- 172.21.15.27:0/764369744 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c400ccc0 con 0x7fb9dc076100 2024-09-21T00:20:04.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.300+0000 7fb9d3400640 1 -- 172.21.15.27:0/764369744 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c40035f0 con 0x7fb9dc076100 2024-09-21T00:20:04.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.300+0000 7fb9d3400640 1 -- 172.21.15.27:0/764369744 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c400ce60 con 0x7fb9dc076100 2024-09-21T00:20:04.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.300+0000 7fb9e1400640 1 -- 172.21.15.27:0/764369744 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9dc1a2850 con 0x7fb9dc076100 2024-09-21T00:20:04.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.301+0000 7fb9e1400640 1 -- 172.21.15.27:0/764369744 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9dc1a8fb0 con 0x7fb9dc076100 2024-09-21T00:20:04.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.302+0000 7fb9e1400640 1 -- 172.21.15.27:0/764369744 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9dc075bc0 con 0x7fb9dc076100 2024-09-21T00:20:04.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.304+0000 7fb9d3400640 1 -- 172.21.15.27:0/764369744 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fb9c4004000 con 0x7fb9dc076100 2024-09-21T00:20:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.306+0000 7fb9d3400640 1 --2- 172.21.15.27:0/764369744 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb9c00b6670 0x7fb9c00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.306+0000 7fb9d3400640 1 -- 172.21.15.27:0/764369744 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7fb9c40e80e0 con 0x7fb9dc076100 2024-09-21T00:20:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.306+0000 7fb9da800640 1 --2- 172.21.15.27:0/764369744 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb9c00b6670 0x7fb9c00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.307+0000 7fb9da800640 1 --2- 172.21.15.27:0/764369744 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb9c00b6670 0x7fb9c00b8b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc011500 tx=0x7fb9cc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.307+0000 7fb9d3400640 1 -- 172.21.15.27:0/764369744 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb9c40ae380 con 0x7fb9dc076100 2024-09-21T00:20:04.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:04 smithi089 ceph-mon[34481]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-21T00:20:04.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:04 smithi098 ceph-mon[34725]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-21T00:20:04.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.557+0000 7fb9e1400640 1 -- 172.21.15.27:0/764369744 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb9dc19d720 con 0x7fb9dc076100 2024-09-21T00:20:04.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:04 smithi027 ceph-mon[30745]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-21T00:20:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.562+0000 7fb9d3400640 1 -- 172.21.15.27:0/764369744 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+296453 (secure 0 0 0) 0x7fb9c40b2760 con 0x7fb9dc076100 2024-09-21T00:20:04.570 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:20:04.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.568+0000 7fb9e1400640 1 -- 172.21.15.27:0/764369744 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb9c00b6670 msgr2=0x7fb9c00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:04.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.568+0000 7fb9e1400640 1 --2- 172.21.15.27:0/764369744 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb9c00b6670 0x7fb9c00b8b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc011500 tx=0x7fb9cc005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:04.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.568+0000 7fb9e1400640 1 -- 172.21.15.27:0/764369744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9dc076100 msgr2=0x7fb9dc19cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.568+0000 7fb9e1400640 1 --2- 172.21.15.27:0/764369744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9dc076100 0x7fb9dc19cb90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400e9c0 tx=0x7fb9c400ee80 comp rx=0 tx=0).stop 2024-09-21T00:20:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.568+0000 7fb9e1400640 1 -- 172.21.15.27:0/764369744 shutdown_connections 2024-09-21T00:20:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.568+0000 7fb9e1400640 1 --2- 172.21.15.27:0/764369744 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb9c00b6670 0x7fb9c00b8b80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.568+0000 7fb9e1400640 1 --2- 172.21.15.27:0/764369744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9dc1100c0 0x7fb9dc1a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.568+0000 7fb9e1400640 1 --2- 172.21.15.27:0/764369744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9dc076100 0x7fb9dc19cb90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.568+0000 7fb9e1400640 1 --2- 172.21.15.27:0/764369744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9dc0757b0 0x7fb9dc19c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.568+0000 7fb9e1400640 1 -- 172.21.15.27:0/764369744 >> 172.21.15.27:0/764369744 conn(0x7fb9dc07b850 msgr2=0x7fb9dc1085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.568+0000 7fb9e1400640 1 -- 172.21.15.27:0/764369744 shutdown_connections 2024-09-21T00:20:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:04.568+0000 7fb9e1400640 1 -- 172.21.15.27:0/764369744 wait complete. 2024-09-21T00:20:05.400 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":21,"flags":0,"active_gid":14226,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":2382590111},{"type":"v1","addr":"172.21.15.27:6801","nonce":2382590111}]},"active_addr":"172.21.15.27:6801/2382590111","active_change":"2024-09-21T00:11:24.231237+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24166,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24182,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.27:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":2210567748}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":4278830807}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":663091323}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":2644715933}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":3213896773}]}]} 2024-09-21T00:20:05.408 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T00:20:05.408 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T00:20:05.409 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T00:20:05.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:05 smithi027 ceph-mon[30745]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-21T00:20:05.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:05 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/764369744' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T00:20:05.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:05 smithi089 ceph-mon[34481]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-21T00:20:05.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:05 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/764369744' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T00:20:05.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:05 smithi098 ceph-mon[34725]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-21T00:20:05.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:05 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/764369744' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T00:20:05.795 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:06.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.454+0000 7fda85e00640 1 -- 172.21.15.27:0/2725330350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda8010c010 msgr2=0x7fda80112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:06.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.454+0000 7fda85e00640 1 --2- 172.21.15.27:0/2725330350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda8010c010 0x7fda80112840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fda78009900 tx=0x7fda7802a410 comp rx=0 tx=0).stop 2024-09-21T00:20:06.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.454+0000 7fda85e00640 1 -- 172.21.15.27:0/2725330350 shutdown_connections 2024-09-21T00:20:06.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.454+0000 7fda85e00640 1 --2- 172.21.15.27:0/2725330350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda8010c010 0x7fda80112840 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:06.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.454+0000 7fda85e00640 1 --2- 172.21.15.27:0/2725330350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda8010b6c0 0x7fda8010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:06.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.454+0000 7fda85e00640 1 --2- 172.21.15.27:0/2725330350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda80105d80 0x7fda80106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:06.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.454+0000 7fda85e00640 1 -- 172.21.15.27:0/2725330350 >> 172.21.15.27:0/2725330350 conn(0x7fda80101810 msgr2=0x7fda80103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:06.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.454+0000 7fda85e00640 1 -- 172.21.15.27:0/2725330350 shutdown_connections 2024-09-21T00:20:06.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.455+0000 7fda85e00640 1 -- 172.21.15.27:0/2725330350 wait complete. 2024-09-21T00:20:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.456+0000 7fda85e00640 1 Processor -- start 2024-09-21T00:20:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.456+0000 7fda85e00640 1 -- start start 2024-09-21T00:20:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.456+0000 7fda85e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda80105d80 0x7fda8007b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.456+0000 7fda85e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda8010b6c0 0x7fda80077920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.456+0000 7fda85e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda8010c010 0x7fda80078040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.456+0000 7fda85e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda80115d20 con 0x7fda8010b6c0 2024-09-21T00:20:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.456+0000 7fda85e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fda80115ba0 con 0x7fda8010c010 2024-09-21T00:20:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.456+0000 7fda85e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fda80115ea0 con 0x7fda80105d80 2024-09-21T00:20:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.457+0000 7fda76000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda8010b6c0 0x7fda80077920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:06.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.457+0000 7fda7ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda80105d80 0x7fda8007b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:06.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.457+0000 7fda76000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda8010b6c0 0x7fda80077920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48136/0 (socket says 172.21.15.27:48136) 2024-09-21T00:20:06.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.457+0000 7fda76000640 1 -- 172.21.15.27:0/2003251574 learned_addr learned my addr 172.21.15.27:0/2003251574 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:06.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.457+0000 7fda7f400640 1 --2- 172.21.15.27:0/2003251574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda8010c010 0x7fda80078040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.457+0000 7fda7ea00640 1 -- 172.21.15.27:0/2003251574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda8010c010 msgr2=0x7fda80078040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.457+0000 7fda7ea00640 1 --2- 172.21.15.27:0/2003251574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda8010c010 0x7fda80078040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.457+0000 7fda7ea00640 1 -- 172.21.15.27:0/2003251574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda8010b6c0 msgr2=0x7fda80077920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.457+0000 7fda7ea00640 1 --2- 172.21.15.27:0/2003251574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda8010b6c0 0x7fda80077920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.457+0000 7fda7ea00640 1 -- 172.21.15.27:0/2003251574 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda801acc80 con 0x7fda80105d80 2024-09-21T00:20:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.458+0000 7fda7ea00640 1 --2- 172.21.15.27:0/2003251574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda80105d80 0x7fda8007b640 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fda70009fd0 tx=0x7fda7000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.458+0000 7fda84a00640 1 -- 172.21.15.27:0/2003251574 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70004b20 con 0x7fda80105d80 2024-09-21T00:20:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.458+0000 7fda84a00640 1 -- 172.21.15.27:0/2003251574 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda70004cc0 con 0x7fda80105d80 2024-09-21T00:20:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.458+0000 7fda84a00640 1 -- 172.21.15.27:0/2003251574 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70010720 con 0x7fda80105d80 2024-09-21T00:20:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.458+0000 7fda85e00640 1 -- 172.21.15.27:0/2003251574 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda801aceb0 con 0x7fda80105d80 2024-09-21T00:20:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.458+0000 7fda85e00640 1 -- 172.21.15.27:0/2003251574 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda801ad300 con 0x7fda80105d80 2024-09-21T00:20:06.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.459+0000 7fda85e00640 1 -- 172.21.15.27:0/2003251574 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda80106190 con 0x7fda80105d80 2024-09-21T00:20:06.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.461+0000 7fda84a00640 1 -- 172.21.15.27:0/2003251574 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fda700108c0 con 0x7fda80105d80 2024-09-21T00:20:06.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.463+0000 7fda84a00640 1 --2- 172.21.15.27:0/2003251574 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda640b6670 0x7fda640b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:06.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.463+0000 7fda84a00640 1 -- 172.21.15.27:0/2003251574 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7fda700e8090 con 0x7fda80105d80 2024-09-21T00:20:06.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.463+0000 7fda76000640 1 --2- 172.21.15.27:0/2003251574 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda640b6670 0x7fda640b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:06.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.464+0000 7fda76000640 1 --2- 172.21.15.27:0/2003251574 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda640b6670 0x7fda640b8b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fda80105130 tx=0x7fda68005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:06.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.465+0000 7fda84a00640 1 -- 172.21.15.27:0/2003251574 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fda700ae330 con 0x7fda80105d80 2024-09-21T00:20:06.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.667+0000 7fda85e00640 1 -- 172.21.15.27:0/2003251574 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda8002e640 con 0x7fda80105d80 2024-09-21T00:20:06.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.669+0000 7fda84a00640 1 -- 172.21.15.27:0/2003251574 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24953 (secure 0 0 0) 0x7fda700b2710 con 0x7fda80105d80 2024-09-21T00:20:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:20:06.672 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":88,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","created":"2024-09-21T00:09:20.149938+0000","modified":"2024-09-21T00:19:25.824021+0000","last_up_change":"2024-09-21T00:17:43.978785+0000","last_in_change":"2024-09-21T00:17:21.325177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:13:32.718543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:18:33.992414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:18:36.817691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:18:46.707770+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63ccc094-62d5-446f-844d-1c60d41fda1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6803","nonce":354370090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6805","nonce":354370090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6809","nonce":354370090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6807","nonce":354370090}]},"public_addr":"172.21.15.27:6803/354370090","cluster_addr":"172.21.15.27:6805/354370090","heartbeat_back_addr":"172.21.15.27:6809/354370090","heartbeat_front_addr":"172.21.15.27:6807/354370090","state":["exists","up"]},{"osd":1,"uuid":"e59a8ed4-3a37-4477-a60a-4abc33d49b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6801","nonce":922528621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6803","nonce":922528621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6807","nonce":922528621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6805","nonce":922528621}]},"public_addr":"172.21.15.89:6801/922528621","cluster_addr":"172.21.15.89:6803/922528621","heartbeat_back_addr":"172.21.15.89:6807/922528621","heartbeat_front_addr":"172.21.15.89:6805/922528621","state":["exists","up"]},{"osd":2,"uuid":"6058a875-d5b2-4459-9407-2fd190cb34a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6801","nonce":257636259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6803","nonce":257636259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6807","nonce":257636259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6805","nonce":257636259}]},"public_addr":"172.21.15.98:6801/257636259","cluster_addr":"172.21.15.98:6803/257636259","heartbeat_back_addr":"172.21.15.98:6807/257636259","heartbeat_front_addr":"172.21.15.98:6805/257636259","state":["exists","up"]},{"osd":3,"uuid":"dfcdaede-e51c-4985-92af-3e296435a36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6811","nonce":821088824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6813","nonce":821088824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6817","nonce":821088824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6815","nonce":821088824}]},"public_addr":"172.21.15.27:6811/821088824","cluster_addr":"172.21.15.27:6813/821088824","heartbeat_back_addr":"172.21.15.27:6817/821088824","heartbeat_front_addr":"172.21.15.27:6815/821088824","state":["exists","up"]},{"osd":4,"uuid":"7c4af037-8ab6-47c0-b62b-76bd775b3491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6809","nonce":4001038690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6811","nonce":4001038690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6815","nonce":4001038690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6813","nonce":4001038690}]},"public_addr":"172.21.15.89:6809/4001038690","cluster_addr":"172.21.15.89:6811/4001038690","heartbeat_back_addr":"172.21.15.89:6815/4001038690","heartbeat_front_addr":"172.21.15.89:6813/4001038690","state":["exists","up"]},{"osd":5,"uuid":"e996ed0d-88ad-4aa7-891f-dfade4e09cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6809","nonce":3029903562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6811","nonce":3029903562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6815","nonce":3029903562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6813","nonce":3029903562}]},"public_addr":"172.21.15.98:6809/3029903562","cluster_addr":"172.21.15.98:6811/3029903562","heartbeat_back_addr":"172.21.15.98:6815/3029903562","heartbeat_front_addr":"172.21.15.98:6813/3029903562","state":["exists","up"]},{"osd":6,"uuid":"cf080497-57c2-42d0-8c8f-1eed0110702d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6819","nonce":1286808857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6821","nonce":1286808857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6825","nonce":1286808857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6823","nonce":1286808857}]},"public_addr":"172.21.15.27:6819/1286808857","cluster_addr":"172.21.15.27:6821/1286808857","heartbeat_back_addr":"172.21.15.27:6825/1286808857","heartbeat_front_addr":"172.21.15.27:6823/1286808857","state":["exists","up"]},{"osd":7,"uuid":"ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6817","nonce":2479349546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6819","nonce":2479349546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6823","nonce":2479349546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6821","nonce":2479349546}]},"public_addr":"172.21.15.89:6817/2479349546","cluster_addr":"172.21.15.89:6819/2479349546","heartbeat_back_addr":"172.21.15.89:6823/2479349546","heartbeat_front_addr":"172.21.15.89:6821/2479349546","state":["exists","up"]},{"osd":8,"uuid":"0b558458-fc2d-4ed1-b72a-e2ea8cf811b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6817","nonce":2830381485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6819","nonce":2830381485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6823","nonce":2830381485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6821","nonce":2830381485}]},"public_addr":"172.21.15.98:6817/2830381485","cluster_addr":"172.21.15.98:6819/2830381485","heartbeat_back_addr":"172.21.15.98:6823/2830381485","heartbeat_front_addr":"172.21.15.98:6821/2830381485","state":["exists","up"]},{"osd":9,"uuid":"1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6827","nonce":3033977113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6829","nonce":3033977113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6833","nonce":3033977113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6831","nonce":3033977113}]},"public_addr":"172.21.15.27:6827/3033977113","cluster_addr":"172.21.15.27:6829/3033977113","heartbeat_back_addr":"172.21.15.27:6833/3033977113","heartbeat_front_addr":"172.21.15.27:6831/3033977113","state":["exists","up"]},{"osd":10,"uuid":"324dd6ce-68fe-4c76-959b-1c9fb0ae92a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6825","nonce":2370332225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6827","nonce":2370332225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6831","nonce":2370332225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6829","nonce":2370332225}]},"public_addr":"172.21.15.89:6825/2370332225","cluster_addr":"172.21.15.89:6827/2370332225","heartbeat_back_addr":"172.21.15.89:6831/2370332225","heartbeat_front_addr":"172.21.15.89:6829/2370332225","state":["exists","up"]},{"osd":11,"uuid":"5e34686e-350a-491e-b9d7-59174ea90944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6825","nonce":2795999064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6827","nonce":2795999064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6831","nonce":2795999064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6829","nonce":2795999064}]},"public_addr":"172.21.15.98:6825/2795999064","cluster_addr":"172.21.15.98:6827/2795999064","heartbeat_back_addr":"172.21.15.98:6831/2795999064","heartbeat_front_addr":"172.21.15.98:6829/2795999064","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:12:39.841958+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:06.684417+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:31.041378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:01.612108+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:28.592326+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:54.303714+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:25.523475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:52.523483+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:19.430503+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:50.599120+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:17.520452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:42.725550+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/502883064":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/831048090":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/3880378905":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/793991452":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/2077956479":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6800/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6801/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/1098432608":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6800/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/528019907":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/2173601508":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:6800/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/764776233":"2024-09-22T00:11:24.231101+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:20:06.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.672+0000 7fda85e00640 1 -- 172.21.15.27:0/2003251574 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda640b6670 msgr2=0x7fda640b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:06.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.672+0000 7fda85e00640 1 --2- 172.21.15.27:0/2003251574 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda640b6670 0x7fda640b8b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fda80105130 tx=0x7fda68005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:06.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.672+0000 7fda85e00640 1 -- 172.21.15.27:0/2003251574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda80105d80 msgr2=0x7fda8007b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:06.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.672+0000 7fda85e00640 1 --2- 172.21.15.27:0/2003251574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda80105d80 0x7fda8007b640 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fda70009fd0 tx=0x7fda7000ef90 comp rx=0 tx=0).stop 2024-09-21T00:20:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.672+0000 7fda85e00640 1 -- 172.21.15.27:0/2003251574 shutdown_connections 2024-09-21T00:20:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.672+0000 7fda85e00640 1 --2- 172.21.15.27:0/2003251574 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda640b6670 0x7fda640b8b80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.672+0000 7fda85e00640 1 --2- 172.21.15.27:0/2003251574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda8010c010 0x7fda80078040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.672+0000 7fda85e00640 1 --2- 172.21.15.27:0/2003251574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda8010b6c0 0x7fda80077920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.672+0000 7fda85e00640 1 --2- 172.21.15.27:0/2003251574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda80105d80 0x7fda8007b640 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.672+0000 7fda85e00640 1 -- 172.21.15.27:0/2003251574 >> 172.21.15.27:0/2003251574 conn(0x7fda80101810 msgr2=0x7fda8010acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.672+0000 7fda85e00640 1 -- 172.21.15.27:0/2003251574 shutdown_connections 2024-09-21T00:20:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:06.672+0000 7fda85e00640 1 -- 172.21.15.27:0/2003251574 wait complete. 2024-09-21T00:20:07.524 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T00:20:07.524 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T00:20:07.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:07 smithi089 ceph-mon[34481]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-21T00:20:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:07 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2003251574' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:20:07.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:07 smithi098 ceph-mon[34725]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-21T00:20:07.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:07 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2003251574' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:20:07.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:07 smithi027 ceph-mon[30745]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-21T00:20:07.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:07 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2003251574' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:20:07.908 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.557+0000 7f6e95e00640 1 -- 172.21.15.27:0/2093644715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e90109e10 msgr2=0x7f6e90110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.557+0000 7f6e95e00640 1 --2- 172.21.15.27:0/2093644715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e90109e10 0x7f6e90110640 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009ca0 tx=0x7f6e8802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:20:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.557+0000 7f6e95e00640 1 -- 172.21.15.27:0/2093644715 shutdown_connections 2024-09-21T00:20:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.557+0000 7f6e95e00640 1 --2- 172.21.15.27:0/2093644715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e90109e10 0x7f6e90110640 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.557+0000 7f6e95e00640 1 --2- 172.21.15.27:0/2093644715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e901094c0 0x7f6e901098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.557+0000 7f6e95e00640 1 --2- 172.21.15.27:0/2093644715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e90103b80 0x7f6e90103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.557+0000 7f6e95e00640 1 -- 172.21.15.27:0/2093644715 >> 172.21.15.27:0/2093644715 conn(0x7f6e900ff5d0 msgr2=0x7f6e90101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.557+0000 7f6e95e00640 1 -- 172.21.15.27:0/2093644715 shutdown_connections 2024-09-21T00:20:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.557+0000 7f6e95e00640 1 -- 172.21.15.27:0/2093644715 wait complete. 2024-09-21T00:20:08.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.559+0000 7f6e95e00640 1 Processor -- start 2024-09-21T00:20:08.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.559+0000 7f6e95e00640 1 -- start start 2024-09-21T00:20:08.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.559+0000 7f6e95e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e90103b80 0x7f6e90075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:08.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.559+0000 7f6e95e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e901094c0 0x7f6e90075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:08.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.559+0000 7f6e95e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e90109e10 0x7f6e90076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:08.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.560+0000 7f6e95e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6e90113b20 con 0x7f6e90109e10 2024-09-21T00:20:08.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.560+0000 7f6e95e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6e901139a0 con 0x7f6e90103b80 2024-09-21T00:20:08.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.560+0000 7f6e95e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6e90113ca0 con 0x7f6e901094c0 2024-09-21T00:20:08.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.560+0000 7f6e8f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e90109e10 0x7f6e90076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:08.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.560+0000 7f6e8ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e90103b80 0x7f6e90075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:08.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.560+0000 7f6e8ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e90103b80 0x7f6e90075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57416/0 (socket says 172.21.15.27:57416) 2024-09-21T00:20:08.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.560+0000 7f6e8f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e90109e10 0x7f6e90076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48158/0 (socket says 172.21.15.27:48158) 2024-09-21T00:20:08.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.560+0000 7f6e8ea00640 1 -- 172.21.15.27:0/4092828072 learned_addr learned my addr 172.21.15.27:0/4092828072 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:08.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.560+0000 7f6e87e00640 1 --2- 172.21.15.27:0/4092828072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e901094c0 0x7f6e90075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:08.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.560+0000 7f6e8ea00640 1 -- 172.21.15.27:0/4092828072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e901094c0 msgr2=0x7f6e90075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:08.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.560+0000 7f6e8ea00640 1 --2- 172.21.15.27:0/4092828072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e901094c0 0x7f6e90075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:08.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.560+0000 7f6e8ea00640 1 -- 172.21.15.27:0/4092828072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e90109e10 msgr2=0x7f6e90076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:08.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.560+0000 7f6e8ea00640 1 --2- 172.21.15.27:0/4092828072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e90109e10 0x7f6e90076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:08.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.560+0000 7f6e8ea00640 1 -- 172.21.15.27:0/4092828072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e901a8710 con 0x7f6e90103b80 2024-09-21T00:20:08.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.561+0000 7f6e8ea00640 1 --2- 172.21.15.27:0/4092828072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e90103b80 0x7f6e90075710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000e9f0 tx=0x7f6e8000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:08.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.561+0000 7f6e94a00640 1 -- 172.21.15.27:0/4092828072 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8000cd80 con 0x7f6e90103b80 2024-09-21T00:20:08.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.561+0000 7f6e95e00640 1 -- 172.21.15.27:0/4092828072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e901a8a00 con 0x7f6e90103b80 2024-09-21T00:20:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.561+0000 7f6e94a00640 1 -- 172.21.15.27:0/4092828072 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e800054d0 con 0x7f6e90103b80 2024-09-21T00:20:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.561+0000 7f6e94a00640 1 -- 172.21.15.27:0/4092828072 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80010690 con 0x7f6e90103b80 2024-09-21T00:20:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.561+0000 7f6e95e00640 1 -- 172.21.15.27:0/4092828072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e901a8f40 con 0x7f6e90103b80 2024-09-21T00:20:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.562+0000 7f6e95e00640 1 -- 172.21.15.27:0/4092828072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e90103f90 con 0x7f6e90103b80 2024-09-21T00:20:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.565+0000 7f6e94a00640 1 -- 172.21.15.27:0/4092828072 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f6e80004e90 con 0x7f6e90103b80 2024-09-21T00:20:08.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.566+0000 7f6e94a00640 1 --2- 172.21.15.27:0/4092828072 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6e740b6670 0x7f6e740b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:08.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.566+0000 7f6e94a00640 1 -- 172.21.15.27:0/4092828072 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7f6e800e7fe0 con 0x7f6e90103b80 2024-09-21T00:20:08.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.566+0000 7f6e87e00640 1 --2- 172.21.15.27:0/4092828072 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6e740b6670 0x7f6e740b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:08.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.567+0000 7f6e87e00640 1 --2- 172.21.15.27:0/4092828072 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6e740b6670 0x7f6e740b8b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6e90102e20 tx=0x7f6e78005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:08.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.568+0000 7f6e94a00640 1 -- 172.21.15.27:0/4092828072 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6e800ae280 con 0x7f6e90103b80 2024-09-21T00:20:08.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.775+0000 7f6e95e00640 1 -- 172.21.15.27:0/4092828072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e90076a10 con 0x7f6e90103b80 2024-09-21T00:20:08.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.777+0000 7f6e94a00640 1 -- 172.21.15.27:0/4092828072 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24953 (secure 0 0 0) 0x7f6e90076a10 con 0x7f6e90103b80 2024-09-21T00:20:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:20:08.779 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":88,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","created":"2024-09-21T00:09:20.149938+0000","modified":"2024-09-21T00:19:25.824021+0000","last_up_change":"2024-09-21T00:17:43.978785+0000","last_in_change":"2024-09-21T00:17:21.325177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:13:32.718543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:18:33.992414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:18:36.817691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:18:46.707770+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63ccc094-62d5-446f-844d-1c60d41fda1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6803","nonce":354370090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6805","nonce":354370090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6809","nonce":354370090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6807","nonce":354370090}]},"public_addr":"172.21.15.27:6803/354370090","cluster_addr":"172.21.15.27:6805/354370090","heartbeat_back_addr":"172.21.15.27:6809/354370090","heartbeat_front_addr":"172.21.15.27:6807/354370090","state":["exists","up"]},{"osd":1,"uuid":"e59a8ed4-3a37-4477-a60a-4abc33d49b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6801","nonce":922528621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6803","nonce":922528621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6807","nonce":922528621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6805","nonce":922528621}]},"public_addr":"172.21.15.89:6801/922528621","cluster_addr":"172.21.15.89:6803/922528621","heartbeat_back_addr":"172.21.15.89:6807/922528621","heartbeat_front_addr":"172.21.15.89:6805/922528621","state":["exists","up"]},{"osd":2,"uuid":"6058a875-d5b2-4459-9407-2fd190cb34a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6801","nonce":257636259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6803","nonce":257636259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6807","nonce":257636259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6805","nonce":257636259}]},"public_addr":"172.21.15.98:6801/257636259","cluster_addr":"172.21.15.98:6803/257636259","heartbeat_back_addr":"172.21.15.98:6807/257636259","heartbeat_front_addr":"172.21.15.98:6805/257636259","state":["exists","up"]},{"osd":3,"uuid":"dfcdaede-e51c-4985-92af-3e296435a36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6811","nonce":821088824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6813","nonce":821088824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6817","nonce":821088824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6815","nonce":821088824}]},"public_addr":"172.21.15.27:6811/821088824","cluster_addr":"172.21.15.27:6813/821088824","heartbeat_back_addr":"172.21.15.27:6817/821088824","heartbeat_front_addr":"172.21.15.27:6815/821088824","state":["exists","up"]},{"osd":4,"uuid":"7c4af037-8ab6-47c0-b62b-76bd775b3491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6809","nonce":4001038690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6811","nonce":4001038690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6815","nonce":4001038690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6813","nonce":4001038690}]},"public_addr":"172.21.15.89:6809/4001038690","cluster_addr":"172.21.15.89:6811/4001038690","heartbeat_back_addr":"172.21.15.89:6815/4001038690","heartbeat_front_addr":"172.21.15.89:6813/4001038690","state":["exists","up"]},{"osd":5,"uuid":"e996ed0d-88ad-4aa7-891f-dfade4e09cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6809","nonce":3029903562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6811","nonce":3029903562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6815","nonce":3029903562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6813","nonce":3029903562}]},"public_addr":"172.21.15.98:6809/3029903562","cluster_addr":"172.21.15.98:6811/3029903562","heartbeat_back_addr":"172.21.15.98:6815/3029903562","heartbeat_front_addr":"172.21.15.98:6813/3029903562","state":["exists","up"]},{"osd":6,"uuid":"cf080497-57c2-42d0-8c8f-1eed0110702d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6819","nonce":1286808857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6821","nonce":1286808857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6825","nonce":1286808857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6823","nonce":1286808857}]},"public_addr":"172.21.15.27:6819/1286808857","cluster_addr":"172.21.15.27:6821/1286808857","heartbeat_back_addr":"172.21.15.27:6825/1286808857","heartbeat_front_addr":"172.21.15.27:6823/1286808857","state":["exists","up"]},{"osd":7,"uuid":"ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6817","nonce":2479349546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6819","nonce":2479349546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6823","nonce":2479349546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6821","nonce":2479349546}]},"public_addr":"172.21.15.89:6817/2479349546","cluster_addr":"172.21.15.89:6819/2479349546","heartbeat_back_addr":"172.21.15.89:6823/2479349546","heartbeat_front_addr":"172.21.15.89:6821/2479349546","state":["exists","up"]},{"osd":8,"uuid":"0b558458-fc2d-4ed1-b72a-e2ea8cf811b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6817","nonce":2830381485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6819","nonce":2830381485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6823","nonce":2830381485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6821","nonce":2830381485}]},"public_addr":"172.21.15.98:6817/2830381485","cluster_addr":"172.21.15.98:6819/2830381485","heartbeat_back_addr":"172.21.15.98:6823/2830381485","heartbeat_front_addr":"172.21.15.98:6821/2830381485","state":["exists","up"]},{"osd":9,"uuid":"1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6827","nonce":3033977113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6829","nonce":3033977113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6833","nonce":3033977113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6831","nonce":3033977113}]},"public_addr":"172.21.15.27:6827/3033977113","cluster_addr":"172.21.15.27:6829/3033977113","heartbeat_back_addr":"172.21.15.27:6833/3033977113","heartbeat_front_addr":"172.21.15.27:6831/3033977113","state":["exists","up"]},{"osd":10,"uuid":"324dd6ce-68fe-4c76-959b-1c9fb0ae92a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6825","nonce":2370332225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6827","nonce":2370332225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6831","nonce":2370332225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6829","nonce":2370332225}]},"public_addr":"172.21.15.89:6825/2370332225","cluster_addr":"172.21.15.89:6827/2370332225","heartbeat_back_addr":"172.21.15.89:6831/2370332225","heartbeat_front_addr":"172.21.15.89:6829/2370332225","state":["exists","up"]},{"osd":11,"uuid":"5e34686e-350a-491e-b9d7-59174ea90944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6825","nonce":2795999064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6827","nonce":2795999064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6831","nonce":2795999064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6829","nonce":2795999064}]},"public_addr":"172.21.15.98:6825/2795999064","cluster_addr":"172.21.15.98:6827/2795999064","heartbeat_back_addr":"172.21.15.98:6831/2795999064","heartbeat_front_addr":"172.21.15.98:6829/2795999064","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:12:39.841958+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:06.684417+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:31.041378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:01.612108+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:28.592326+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:54.303714+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:25.523475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:52.523483+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:19.430503+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:50.599120+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:17.520452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:42.725550+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/502883064":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/831048090":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/3880378905":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/793991452":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/2077956479":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6800/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6801/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/1098432608":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6800/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/528019907":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/2173601508":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:6800/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/764776233":"2024-09-22T00:11:24.231101+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:20:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.779+0000 7f6e95e00640 1 -- 172.21.15.27:0/4092828072 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6e740b6670 msgr2=0x7f6e740b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.779+0000 7f6e95e00640 1 --2- 172.21.15.27:0/4092828072 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6e740b6670 0x7f6e740b8b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6e90102e20 tx=0x7f6e78005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.779+0000 7f6e95e00640 1 -- 172.21.15.27:0/4092828072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e90103b80 msgr2=0x7f6e90075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.779+0000 7f6e95e00640 1 --2- 172.21.15.27:0/4092828072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e90103b80 0x7f6e90075710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000e9f0 tx=0x7f6e8000edf0 comp rx=0 tx=0).stop 2024-09-21T00:20:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.779+0000 7f6e95e00640 1 -- 172.21.15.27:0/4092828072 shutdown_connections 2024-09-21T00:20:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.779+0000 7f6e95e00640 1 --2- 172.21.15.27:0/4092828072 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6e740b6670 0x7f6e740b8b80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.779+0000 7f6e95e00640 1 --2- 172.21.15.27:0/4092828072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e90109e10 0x7f6e90076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.779+0000 7f6e95e00640 1 --2- 172.21.15.27:0/4092828072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e901094c0 0x7f6e90075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.779+0000 7f6e95e00640 1 --2- 172.21.15.27:0/4092828072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e90103b80 0x7f6e90075710 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.779+0000 7f6e95e00640 1 -- 172.21.15.27:0/4092828072 >> 172.21.15.27:0/4092828072 conn(0x7f6e900ff5d0 msgr2=0x7f6e90106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.780+0000 7f6e95e00640 1 -- 172.21.15.27:0/4092828072 shutdown_connections 2024-09-21T00:20:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:08.780+0000 7f6e95e00640 1 -- 172.21.15.27:0/4092828072 wait complete. 2024-09-21T00:20:09.628 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T00:20:09.628 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T00:20:09.629 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T00:20:09.629 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T00:20:09.630 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T00:20:09.630 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T00:20:09.630 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T00:20:09.631 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T00:20:09.631 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T00:20:09.632 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T00:20:09.632 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T00:20:09.633 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T00:20:09.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:09 smithi089 ceph-mon[34481]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-21T00:20:09.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:09 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/4092828072' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:20:09.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:09 smithi027 ceph-mon[30745]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-21T00:20:09.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:09 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/4092828072' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:20:09.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:09 smithi098 ceph-mon[34725]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-21T00:20:09.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:09 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/4092828072' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:20:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:10.750 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:10.758 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:10.764 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:10.765 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:10.767 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:10.774 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:11.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:11 smithi027 ceph-mon[30745]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-21T00:20:11.605 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:11.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:11 smithi089 ceph-mon[34481]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-21T00:20:11.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:11 smithi098 ceph-mon[34725]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-21T00:20:11.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.922+0000 7fd334c00640 1 -- 172.21.15.27:0/3930477450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3300757b0 msgr2=0x7fd330075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.922+0000 7fd334c00640 1 --2- 172.21.15.27:0/3930477450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3300757b0 0x7fd330075bc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd320009900 tx=0x7fd32002a410 comp rx=0 tx=0).stop 2024-09-21T00:20:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.922+0000 7fd334c00640 1 -- 172.21.15.27:0/3930477450 shutdown_connections 2024-09-21T00:20:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.922+0000 7fd334c00640 1 --2- 172.21.15.27:0/3930477450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd33010ff10 0x7fd330112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.922+0000 7fd334c00640 1 --2- 172.21.15.27:0/3930477450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd330076100 0x7fd33010f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.922+0000 7fd334c00640 1 --2- 172.21.15.27:0/3930477450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3300757b0 0x7fd330075bc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.922+0000 7fd334c00640 1 -- 172.21.15.27:0/3930477450 >> 172.21.15.27:0/3930477450 conn(0x7fd3300ff440 msgr2=0x7fd330101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.922+0000 7fd334c00640 1 -- 172.21.15.27:0/3930477450 shutdown_connections 2024-09-21T00:20:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.922+0000 7fd334c00640 1 -- 172.21.15.27:0/3930477450 wait complete. 2024-09-21T00:20:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.924+0000 7fd334c00640 1 Processor -- start 2024-09-21T00:20:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.924+0000 7fd334c00640 1 -- start start 2024-09-21T00:20:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.924+0000 7fd334c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3300757b0 0x7fd33019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.924+0000 7fd334c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd330076100 0x7fd33019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.924+0000 7fd334c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd33010ff10 0x7fd3301a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.924+0000 7fd334c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd330115b10 con 0x7fd3300757b0 2024-09-21T00:20:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.924+0000 7fd334c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd330115990 con 0x7fd330076100 2024-09-21T00:20:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.924+0000 7fd334c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd330115c90 con 0x7fd33010ff10 2024-09-21T00:20:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.924+0000 7fd32f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3300757b0 0x7fd33019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.924+0000 7fd32ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd330076100 0x7fd33019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.924+0000 7fd32f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3300757b0 0x7fd33019c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48172/0 (socket says 172.21.15.27:48172) 2024-09-21T00:20:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.924+0000 7fd32f400640 1 -- 172.21.15.27:0/4123810142 learned_addr learned my addr 172.21.15.27:0/4123810142 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.924+0000 7fd32ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd330076100 0x7fd33019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57432/0 (socket says 172.21.15.27:57432) 2024-09-21T00:20:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.924+0000 7fd32fe00640 1 --2- 172.21.15.27:0/4123810142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd33010ff10 0x7fd3301a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.925+0000 7fd32ea00640 1 -- 172.21.15.27:0/4123810142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd33010ff10 msgr2=0x7fd3301a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.925+0000 7fd32ea00640 1 --2- 172.21.15.27:0/4123810142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd33010ff10 0x7fd3301a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.925+0000 7fd32ea00640 1 -- 172.21.15.27:0/4123810142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3300757b0 msgr2=0x7fd33019c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.925+0000 7fd32ea00640 1 --2- 172.21.15.27:0/4123810142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3300757b0 0x7fd33019c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.925+0000 7fd32ea00640 1 -- 172.21.15.27:0/4123810142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3301a2430 con 0x7fd330076100 2024-09-21T00:20:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.925+0000 7fd32ea00640 1 --2- 172.21.15.27:0/4123810142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd330076100 0x7fd33019ca60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd33006b700 tx=0x7fd31c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.925+0000 7fd327e00640 1 -- 172.21.15.27:0/4123810142 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c017070 con 0x7fd330076100 2024-09-21T00:20:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.925+0000 7fd334c00640 1 -- 172.21.15.27:0/4123810142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3301a2720 con 0x7fd330076100 2024-09-21T00:20:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.926+0000 7fd334c00640 1 -- 172.21.15.27:0/4123810142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3301a8e80 con 0x7fd330076100 2024-09-21T00:20:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.926+0000 7fd327e00640 1 -- 172.21.15.27:0/4123810142 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd31c009c30 con 0x7fd330076100 2024-09-21T00:20:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.926+0000 7fd327e00640 1 -- 172.21.15.27:0/4123810142 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c021060 con 0x7fd330076100 2024-09-21T00:20:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.927+0000 7fd334c00640 1 -- 172.21.15.27:0/4123810142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd310001060 con 0x7fd330076100 2024-09-21T00:20:11.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.933+0000 7fd327e00640 1 -- 172.21.15.27:0/4123810142 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fd31c01a070 con 0x7fd330076100 2024-09-21T00:20:11.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.934+0000 7fd327e00640 1 --2- 172.21.15.27:0/4123810142 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd3140b6670 0x7fd3140b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.934+0000 7fd327e00640 1 -- 172.21.15.27:0/4123810142 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7fd31c0e75a0 con 0x7fd330076100 2024-09-21T00:20:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.935+0000 7fd32f400640 1 --2- 172.21.15.27:0/4123810142 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd3140b6670 0x7fd3140b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.935+0000 7fd327e00640 1 --2- 172.21.15.27:0/4123810142 >> [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824] conn(0x7fd3140c1a00 0x7fd3140c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.935+0000 7fd327e00640 1 -- 172.21.15.27:0/4123810142 --> [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd31c016ec0 con 0x7fd3140c1a00 2024-09-21T00:20:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.935+0000 7fd327e00640 1 -- 172.21.15.27:0/4123810142 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fd31c0e79f0 con 0x7fd330076100 2024-09-21T00:20:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.935+0000 7fd32fe00640 1 --2- 172.21.15.27:0/4123810142 >> [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824] conn(0x7fd3140c1a00 0x7fd3140c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:11.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.935+0000 7fd32f400640 1 --2- 172.21.15.27:0/4123810142 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd3140b6670 0x7fd3140b8b80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd3200096c0 tx=0x7fd320005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:11.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.936+0000 7fd32fe00640 1 --2- 172.21.15.27:0/4123810142 >> [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824] conn(0x7fd3140c1a00 0x7fd3140c3ef0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:11.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.937+0000 7fd327e00640 1 -- 172.21.15.27:0/4123810142 <== osd.3 v2:172.21.15.27:6810/821088824 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd31c016ec0 con 0x7fd3140c1a00 2024-09-21T00:20:11.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.952+0000 7fd334c00640 1 -- 172.21.15.27:0/4123810142 --> [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd310002e00 con 0x7fd3140c1a00 2024-09-21T00:20:11.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.953+0000 7fd327e00640 1 -- 172.21.15.27:0/4123810142 <== osd.3 v2:172.21.15.27:6810/821088824 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd310002e00 con 0x7fd3140c1a00 2024-09-21T00:20:11.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.954+0000 7fd334c00640 1 -- 172.21.15.27:0/4123810142 >> [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824] conn(0x7fd3140c1a00 msgr2=0x7fd3140c3ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:11.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.954+0000 7fd334c00640 1 --2- 172.21.15.27:0/4123810142 >> [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824] conn(0x7fd3140c1a00 0x7fd3140c3ef0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:11.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.954+0000 7fd334c00640 1 -- 172.21.15.27:0/4123810142 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd3140b6670 msgr2=0x7fd3140b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:11.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.954+0000 7fd334c00640 1 --2- 172.21.15.27:0/4123810142 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd3140b6670 0x7fd3140b8b80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd3200096c0 tx=0x7fd320005a20 comp rx=0 tx=0).stop 2024-09-21T00:20:11.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.954+0000 7fd334c00640 1 -- 172.21.15.27:0/4123810142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd330076100 msgr2=0x7fd33019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:11.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.954+0000 7fd334c00640 1 --2- 172.21.15.27:0/4123810142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd330076100 0x7fd33019ca60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd33006b700 tx=0x7fd31c00dde0 comp rx=0 tx=0).stop 2024-09-21T00:20:11.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.954+0000 7fd32fe00640 1 -- 172.21.15.27:0/4123810142 reap_dead start 2024-09-21T00:20:11.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.954+0000 7fd334c00640 1 -- 172.21.15.27:0/4123810142 shutdown_connections 2024-09-21T00:20:11.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.954+0000 7fd334c00640 1 -- 172.21.15.27:0/4123810142 >> 172.21.15.27:0/4123810142 conn(0x7fd3300ff440 msgr2=0x7fd330109a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:11.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.954+0000 7fd334c00640 1 -- 172.21.15.27:0/4123810142 shutdown_connections 2024-09-21T00:20:11.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:11.954+0000 7fd334c00640 1 -- 172.21.15.27:0/4123810142 wait complete. 2024-09-21T00:20:13.040 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:13.644 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:13 smithi027 ceph-mon[30745]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:20:13.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:13 smithi089 ceph-mon[34481]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:20:13.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:13 smithi098 ceph-mon[34725]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:20:14.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.023+0000 7ff680c00640 1 -- 172.21.15.27:0/724680242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff67c109c20 msgr2=0x7ff67c110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:14.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.023+0000 7ff680c00640 1 --2- 172.21.15.27:0/724680242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff67c109c20 0x7ff67c110450 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff674009ca0 tx=0x7ff67402a8c0 comp rx=0 tx=0).stop 2024-09-21T00:20:14.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.023+0000 7ff680c00640 1 -- 172.21.15.27:0/724680242 shutdown_connections 2024-09-21T00:20:14.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.023+0000 7ff680c00640 1 --2- 172.21.15.27:0/724680242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff67c109c20 0x7ff67c110450 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:14.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.023+0000 7ff680c00640 1 --2- 172.21.15.27:0/724680242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff67c1092d0 0x7ff67c1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:14.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.023+0000 7ff680c00640 1 --2- 172.21.15.27:0/724680242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67c103990 0x7ff67c103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:14.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.023+0000 7ff680c00640 1 -- 172.21.15.27:0/724680242 >> 172.21.15.27:0/724680242 conn(0x7ff67c0ff440 msgr2=0x7ff67c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:14.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.023+0000 7ff680c00640 1 -- 172.21.15.27:0/724680242 shutdown_connections 2024-09-21T00:20:14.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.023+0000 7ff680c00640 1 -- 172.21.15.27:0/724680242 wait complete. 2024-09-21T00:20:14.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.024+0000 7ff680c00640 1 Processor -- start 2024-09-21T00:20:14.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.024+0000 7ff680c00640 1 -- start start 2024-09-21T00:20:14.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.025+0000 7ff680c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67c103990 0x7ff67c19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:14.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.025+0000 7ff680c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff67c1092d0 0x7ff67c19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:14.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.025+0000 7ff680c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff67c109c20 0x7ff67c19fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:14.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.025+0000 7ff680c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c113930 con 0x7ff67c1092d0 2024-09-21T00:20:14.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.025+0000 7ff680c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c1137b0 con 0x7ff67c103990 2024-09-21T00:20:14.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.025+0000 7ff680c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c113ab0 con 0x7ff67c109c20 2024-09-21T00:20:14.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.025+0000 7ff673e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff67c1092d0 0x7ff67c19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:14.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.025+0000 7ff67b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67c103990 0x7ff67c19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:14.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.025+0000 7ff673e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff67c1092d0 0x7ff67c19a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33766/0 (socket says 172.21.15.27:33766) 2024-09-21T00:20:14.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.025+0000 7ff673e00640 1 -- 172.21.15.27:0/3029465533 learned_addr learned my addr 172.21.15.27:0/3029465533 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:14.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.025+0000 7ff67be00640 1 --2- 172.21.15.27:0/3029465533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff67c109c20 0x7ff67c19fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:14.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.026+0000 7ff673e00640 1 -- 172.21.15.27:0/3029465533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff67c109c20 msgr2=0x7ff67c19fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:14.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.026+0000 7ff673e00640 1 --2- 172.21.15.27:0/3029465533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff67c109c20 0x7ff67c19fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:14.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.026+0000 7ff673e00640 1 -- 172.21.15.27:0/3029465533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67c103990 msgr2=0x7ff67c19a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:14.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.026+0000 7ff673e00640 1 --2- 172.21.15.27:0/3029465533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67c103990 0x7ff67c19a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:14.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.026+0000 7ff673e00640 1 -- 172.21.15.27:0/3029465533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67c1a0210 con 0x7ff67c1092d0 2024-09-21T00:20:14.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.026+0000 7ff67b400640 1 --2- 172.21.15.27:0/3029465533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67c103990 0x7ff67c19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:14.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.026+0000 7ff673e00640 1 --2- 172.21.15.27:0/3029465533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff67c1092d0 0x7ff67c19a840 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff66400e9f0 tx=0x7ff66400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:14.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.026+0000 7ff678c00640 1 -- 172.21.15.27:0/3029465533 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66400ccc0 con 0x7ff67c1092d0 2024-09-21T00:20:14.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.026+0000 7ff678c00640 1 -- 172.21.15.27:0/3029465533 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6640108e0 con 0x7ff67c1092d0 2024-09-21T00:20:14.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.026+0000 7ff678c00640 1 -- 172.21.15.27:0/3029465533 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664018860 con 0x7ff67c1092d0 2024-09-21T00:20:14.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.026+0000 7ff680c00640 1 -- 172.21.15.27:0/3029465533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff67c1a0500 con 0x7ff67c1092d0 2024-09-21T00:20:14.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.026+0000 7ff680c00640 1 -- 172.21.15.27:0/3029465533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff67c1a6c60 con 0x7ff67c1092d0 2024-09-21T00:20:14.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.027+0000 7ff680c00640 1 -- 172.21.15.27:0/3029465533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff67c063b10 con 0x7ff67c1092d0 2024-09-21T00:20:14.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.031+0000 7ff678c00640 1 -- 172.21.15.27:0/3029465533 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7ff664010430 con 0x7ff67c1092d0 2024-09-21T00:20:14.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.032+0000 7ff678c00640 1 --2- 172.21.15.27:0/3029465533 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff6600b6670 0x7ff6600b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:14.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.032+0000 7ff678c00640 1 -- 172.21.15.27:0/3029465533 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7ff6640e82f0 con 0x7ff67c1092d0 2024-09-21T00:20:14.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.032+0000 7ff67b400640 1 --2- 172.21.15.27:0/3029465533 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff6600b6670 0x7ff6600b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:14.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.033+0000 7ff678c00640 1 --2- 172.21.15.27:0/3029465533 >> [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259] conn(0x7ff6600c1a00 0x7ff6600c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:14.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.033+0000 7ff678c00640 1 -- 172.21.15.27:0/3029465533 --> [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff66401ce70 con 0x7ff6600c1a00 2024-09-21T00:20:14.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.033+0000 7ff678c00640 1 -- 172.21.15.27:0/3029465533 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7ff6640e86a0 con 0x7ff67c1092d0 2024-09-21T00:20:14.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.033+0000 7ff67be00640 1 --2- 172.21.15.27:0/3029465533 >> [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259] conn(0x7ff6600c1a00 0x7ff6600c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:14.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.033+0000 7ff67b400640 1 --2- 172.21.15.27:0/3029465533 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff6600b6670 0x7ff6600b8b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0095f0 tx=0x7ff66c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:14.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.034+0000 7ff67be00640 1 --2- 172.21.15.27:0/3029465533 >> [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259] conn(0x7ff6600c1a00 0x7ff6600c3ef0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:14.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.035+0000 7ff678c00640 1 -- 172.21.15.27:0/3029465533 <== osd.2 v2:172.21.15.98:6800/257636259 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff66401ce70 con 0x7ff6600c1a00 2024-09-21T00:20:14.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.049+0000 7ff680c00640 1 -- 172.21.15.27:0/3029465533 --> [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff67c063fe0 con 0x7ff6600c1a00 2024-09-21T00:20:14.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.050+0000 7ff678c00640 1 -- 172.21.15.27:0/3029465533 <== osd.2 v2:172.21.15.98:6800/257636259 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff67c063fe0 con 0x7ff6600c1a00 2024-09-21T00:20:14.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.050+0000 7ff680c00640 1 -- 172.21.15.27:0/3029465533 >> [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259] conn(0x7ff6600c1a00 msgr2=0x7ff6600c3ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.050+0000 7ff680c00640 1 --2- 172.21.15.27:0/3029465533 >> [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259] conn(0x7ff6600c1a00 0x7ff6600c3ef0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.050+0000 7ff680c00640 1 -- 172.21.15.27:0/3029465533 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff6600b6670 msgr2=0x7ff6600b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.050+0000 7ff680c00640 1 --2- 172.21.15.27:0/3029465533 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff6600b6670 0x7ff6600b8b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0095f0 tx=0x7ff66c005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.050+0000 7ff680c00640 1 -- 172.21.15.27:0/3029465533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff67c1092d0 msgr2=0x7ff67c19a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.050+0000 7ff680c00640 1 --2- 172.21.15.27:0/3029465533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff67c1092d0 0x7ff67c19a840 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff66400e9f0 tx=0x7ff66400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:20:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.050+0000 7ff67be00640 1 -- 172.21.15.27:0/3029465533 reap_dead start 2024-09-21T00:20:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.050+0000 7ff680c00640 1 -- 172.21.15.27:0/3029465533 shutdown_connections 2024-09-21T00:20:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.050+0000 7ff680c00640 1 -- 172.21.15.27:0/3029465533 >> 172.21.15.27:0/3029465533 conn(0x7ff67c0ff440 msgr2=0x7ff67c1088e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.051+0000 7ff680c00640 1 -- 172.21.15.27:0/3029465533 shutdown_connections 2024-09-21T00:20:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.051+0000 7ff680c00640 1 -- 172.21.15.27:0/3029465533 wait complete. 2024-09-21T00:20:14.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.277+0000 7fc045000640 1 -- 172.21.15.27:0/2287830179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0401092d0 msgr2=0x7fc0401096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:14.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.277+0000 7fc045000640 1 --2- 172.21.15.27:0/2287830179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0401092d0 0x7fc0401096e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009900 tx=0x7fc02c02a410 comp rx=0 tx=0).stop 2024-09-21T00:20:14.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.277+0000 7fc045000640 1 -- 172.21.15.27:0/2287830179 shutdown_connections 2024-09-21T00:20:14.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.277+0000 7fc045000640 1 --2- 172.21.15.27:0/2287830179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc040109c20 0x7fc040110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:14.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.277+0000 7fc045000640 1 --2- 172.21.15.27:0/2287830179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0401092d0 0x7fc0401096e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:14.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.277+0000 7fc045000640 1 --2- 172.21.15.27:0/2287830179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc040103990 0x7fc040103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:14.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.277+0000 7fc045000640 1 -- 172.21.15.27:0/2287830179 >> 172.21.15.27:0/2287830179 conn(0x7fc0400ff3e0 msgr2=0x7fc040101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:14.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.278+0000 7fc045000640 1 -- 172.21.15.27:0/2287830179 shutdown_connections 2024-09-21T00:20:14.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.278+0000 7fc045000640 1 -- 172.21.15.27:0/2287830179 wait complete. 2024-09-21T00:20:14.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.279+0000 7fc045000640 1 Processor -- start 2024-09-21T00:20:14.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.279+0000 7fc045000640 1 -- start start 2024-09-21T00:20:14.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.279+0000 7fc045000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc040103990 0x7fc04019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:14.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.279+0000 7fc045000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0401092d0 0x7fc04019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.279+0000 7fc045000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc040109c20 0x7fc04019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.279+0000 7fc045000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc0401138e0 con 0x7fc040109c20 2024-09-21T00:20:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.279+0000 7fc045000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc040113760 con 0x7fc040103990 2024-09-21T00:20:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.279+0000 7fc045000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc040113a60 con 0x7fc0401092d0 2024-09-21T00:20:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.279+0000 7fc03fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc040109c20 0x7fc04019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.279+0000 7fc03fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc040109c20 0x7fc04019fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33782/0 (socket says 172.21.15.27:33782) 2024-09-21T00:20:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.279+0000 7fc03fe00640 1 -- 172.21.15.27:0/2093286147 learned_addr learned my addr 172.21.15.27:0/2093286147 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.280+0000 7fc03f400640 1 --2- 172.21.15.27:0/2093286147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc040103990 0x7fc04019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.280+0000 7fc03ea00640 1 --2- 172.21.15.27:0/2093286147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0401092d0 0x7fc04019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.280+0000 7fc03fe00640 1 -- 172.21.15.27:0/2093286147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0401092d0 msgr2=0x7fc04019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.280+0000 7fc03fe00640 1 --2- 172.21.15.27:0/2093286147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0401092d0 0x7fc04019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.280+0000 7fc03fe00640 1 -- 172.21.15.27:0/2093286147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc040103990 msgr2=0x7fc04019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.280+0000 7fc03fe00640 1 --2- 172.21.15.27:0/2093286147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc040103990 0x7fc04019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.280+0000 7fc03fe00640 1 -- 172.21.15.27:0/2093286147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0401a0260 con 0x7fc040109c20 2024-09-21T00:20:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.280+0000 7fc03f400640 1 --2- 172.21.15.27:0/2093286147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc040103990 0x7fc04019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.280+0000 7fc03fe00640 1 --2- 172.21.15.27:0/2093286147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc040109c20 0x7fc04019fb30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc03800eb10 tx=0x7fc03800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.281+0000 7fc037e00640 1 -- 172.21.15.27:0/2093286147 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038004b20 con 0x7fc040109c20 2024-09-21T00:20:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.281+0000 7fc037e00640 1 -- 172.21.15.27:0/2093286147 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc038004cc0 con 0x7fc040109c20 2024-09-21T00:20:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.281+0000 7fc045000640 1 -- 172.21.15.27:0/2093286147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0401a0550 con 0x7fc040109c20 2024-09-21T00:20:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.281+0000 7fc037e00640 1 -- 172.21.15.27:0/2093286147 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038010670 con 0x7fc040109c20 2024-09-21T00:20:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.281+0000 7fc045000640 1 -- 172.21.15.27:0/2093286147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0401a6cb0 con 0x7fc040109c20 2024-09-21T00:20:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.282+0000 7fc045000640 1 -- 172.21.15.27:0/2093286147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc040063b50 con 0x7fc040109c20 2024-09-21T00:20:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.284+0000 7fc037e00640 1 -- 172.21.15.27:0/2093286147 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fc038005000 con 0x7fc040109c20 2024-09-21T00:20:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.286+0000 7fc037e00640 1 --2- 172.21.15.27:0/2093286147 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc0240b6670 0x7fc0240b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.286+0000 7fc037e00640 1 -- 172.21.15.27:0/2093286147 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7fc0380e82a0 con 0x7fc040109c20 2024-09-21T00:20:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.286+0000 7fc03f400640 1 --2- 172.21.15.27:0/2093286147 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc0240b6670 0x7fc0240b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.286+0000 7fc037e00640 1 --2- 172.21.15.27:0/2093286147 >> [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090] conn(0x7fc0240c1a00 0x7fc0240c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.286+0000 7fc037e00640 1 -- 172.21.15.27:0/2093286147 --> [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc038015e80 con 0x7fc0240c1a00 2024-09-21T00:20:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.286+0000 7fc037e00640 1 -- 172.21.15.27:0/2093286147 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fc0380e8650 con 0x7fc040109c20 2024-09-21T00:20:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.287+0000 7fc03ea00640 1 --2- 172.21.15.27:0/2093286147 >> [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090] conn(0x7fc0240c1a00 0x7fc0240c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.287+0000 7fc03f400640 1 --2- 172.21.15.27:0/2093286147 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc0240b6670 0x7fc0240b8b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc03000bfd0 tx=0x7fc03000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.287+0000 7fc03ea00640 1 --2- 172.21.15.27:0/2093286147 >> [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090] conn(0x7fc0240c1a00 0x7fc0240c3ef0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.288+0000 7fc037e00640 1 -- 172.21.15.27:0/2093286147 <== osd.0 v2:172.21.15.27:6802/354370090 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc038015e80 con 0x7fc0240c1a00 2024-09-21T00:20:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.303+0000 7fc045000640 1 -- 172.21.15.27:0/2093286147 --> [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc040063fd0 con 0x7fc0240c1a00 2024-09-21T00:20:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.304+0000 7fc037e00640 1 -- 172.21.15.27:0/2093286147 <== osd.0 v2:172.21.15.27:6802/354370090 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc040063fd0 con 0x7fc0240c1a00 2024-09-21T00:20:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.304+0000 7fc045000640 1 -- 172.21.15.27:0/2093286147 >> [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090] conn(0x7fc0240c1a00 msgr2=0x7fc0240c3ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.304+0000 7fc045000640 1 --2- 172.21.15.27:0/2093286147 >> [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090] conn(0x7fc0240c1a00 0x7fc0240c3ef0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.305+0000 7fc045000640 1 -- 172.21.15.27:0/2093286147 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc0240b6670 msgr2=0x7fc0240b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.305+0000 7fc045000640 1 --2- 172.21.15.27:0/2093286147 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc0240b6670 0x7fc0240b8b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc03000bfd0 tx=0x7fc03000b390 comp rx=0 tx=0).stop 2024-09-21T00:20:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.305+0000 7fc045000640 1 -- 172.21.15.27:0/2093286147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc040109c20 msgr2=0x7fc04019fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.305+0000 7fc045000640 1 --2- 172.21.15.27:0/2093286147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc040109c20 0x7fc04019fb30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc03800eb10 tx=0x7fc03800c2d0 comp rx=0 tx=0).stop 2024-09-21T00:20:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.305+0000 7fc03fe00640 1 -- 172.21.15.27:0/2093286147 reap_dead start 2024-09-21T00:20:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.305+0000 7fc045000640 1 -- 172.21.15.27:0/2093286147 shutdown_connections 2024-09-21T00:20:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.305+0000 7fc045000640 1 -- 172.21.15.27:0/2093286147 >> 172.21.15.27:0/2093286147 conn(0x7fc0400ff3e0 msgr2=0x7fc040101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.305+0000 7fc045000640 1 -- 172.21.15.27:0/2093286147 shutdown_connections 2024-09-21T00:20:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.305+0000 7fc045000640 1 -- 172.21.15.27:0/2093286147 wait complete. 2024-09-21T00:20:15.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.710+0000 7f3fcb200640 1 -- 172.21.15.27:0/3167186881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fc41039b0 msgr2=0x7f3fc4103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:15.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.710+0000 7f3fcb200640 1 --2- 172.21.15.27:0/3167186881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fc41039b0 0x7f3fc4103dc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009860 tx=0x7f3fb802a340 comp rx=0 tx=0).stop 2024-09-21T00:20:15.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.710+0000 7f3fcb200640 1 -- 172.21.15.27:0/3167186881 shutdown_connections 2024-09-21T00:20:15.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.710+0000 7f3fcb200640 1 --2- 172.21.15.27:0/3167186881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fc4109c40 0x7f3fc4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:15.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.710+0000 7f3fcb200640 1 --2- 172.21.15.27:0/3167186881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fc41092f0 0x7f3fc4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:15.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.710+0000 7f3fcb200640 1 --2- 172.21.15.27:0/3167186881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fc41039b0 0x7f3fc4103dc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.710+0000 7f3fcb200640 1 -- 172.21.15.27:0/3167186881 >> 172.21.15.27:0/3167186881 conn(0x7f3fc40ff440 msgr2=0x7f3fc4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.710+0000 7f3fcb200640 1 -- 172.21.15.27:0/3167186881 shutdown_connections 2024-09-21T00:20:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.710+0000 7f3fcb200640 1 -- 172.21.15.27:0/3167186881 wait complete. 2024-09-21T00:20:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.712+0000 7f3fcb200640 1 Processor -- start 2024-09-21T00:20:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.712+0000 7f3fcb200640 1 -- start start 2024-09-21T00:20:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.712+0000 7f3fcb200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fc41039b0 0x7f3fc419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.712+0000 7f3fcb200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fc41092f0 0x7f3fc419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.712+0000 7f3fcb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fc4109c40 0x7f3fc419fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.712+0000 7f3fcb200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc4113900 con 0x7f3fc4109c40 2024-09-21T00:20:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.712+0000 7f3fcb200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc4113780 con 0x7f3fc41092f0 2024-09-21T00:20:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.712+0000 7f3fcb200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc4113a80 con 0x7f3fc41039b0 2024-09-21T00:20:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.713+0000 7f3fc9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fc41039b0 0x7f3fc419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.713+0000 7f3fca800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fc4109c40 0x7f3fc419fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.713+0000 7f3fc9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fc41039b0 0x7f3fc419a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39668/0 (socket says 172.21.15.27:39668) 2024-09-21T00:20:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.713+0000 7f3fc9e00640 1 -- 172.21.15.27:0/244456953 learned_addr learned my addr 172.21.15.27:0/244456953 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.713+0000 7f3fc9400640 1 --2- 172.21.15.27:0/244456953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fc41092f0 0x7f3fc419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.713+0000 7f3fc9e00640 1 -- 172.21.15.27:0/244456953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fc41092f0 msgr2=0x7f3fc419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.713+0000 7f3fc9e00640 1 --2- 172.21.15.27:0/244456953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fc41092f0 0x7f3fc419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.713+0000 7f3fc9e00640 1 -- 172.21.15.27:0/244456953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fc4109c40 msgr2=0x7f3fc419fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.713+0000 7f3fc9e00640 1 --2- 172.21.15.27:0/244456953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fc4109c40 0x7f3fc419fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.713+0000 7f3fc9e00640 1 -- 172.21.15.27:0/244456953 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc41a0240 con 0x7f3fc41039b0 2024-09-21T00:20:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.714+0000 7f3fc9e00640 1 --2- 172.21.15.27:0/244456953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fc41039b0 0x7f3fc419a2d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80303f0 tx=0x7f3fb8032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.714+0000 7f3fbea00640 1 -- 172.21.15.27:0/244456953 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8040020 con 0x7f3fc41039b0 2024-09-21T00:20:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.714+0000 7f3fbea00640 1 -- 172.21.15.27:0/244456953 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fb8030dd0 con 0x7f3fc41039b0 2024-09-21T00:20:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.714+0000 7f3fcb200640 1 -- 172.21.15.27:0/244456953 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc41a04d0 con 0x7f3fc41039b0 2024-09-21T00:20:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.714+0000 7f3fbea00640 1 -- 172.21.15.27:0/244456953 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8037040 con 0x7f3fc41039b0 2024-09-21T00:20:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.714+0000 7f3fcb200640 1 -- 172.21.15.27:0/244456953 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc40755f0 con 0x7f3fc41039b0 2024-09-21T00:20:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.715+0000 7f3fcb200640 1 -- 172.21.15.27:0/244456953 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3fc4063b10 con 0x7f3fc41039b0 2024-09-21T00:20:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.718+0000 7f3fbea00640 1 -- 172.21.15.27:0/244456953 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f3fb8043020 con 0x7f3fc41039b0 2024-09-21T00:20:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.720+0000 7f3fbea00640 1 --2- 172.21.15.27:0/244456953 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3fac0b65f0 0x7f3fac0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.720+0000 7f3fbea00640 1 -- 172.21.15.27:0/244456953 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7f3fb81078a0 con 0x7f3fc41039b0 2024-09-21T00:20:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.720+0000 7f3fc9400640 1 --2- 172.21.15.27:0/244456953 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3fac0b65f0 0x7f3fac0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.720+0000 7f3fbea00640 1 --2- 172.21.15.27:0/244456953 >> [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113] conn(0x7f3fac0c1980 0x7f3fac0c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.720+0000 7f3fbea00640 1 -- 172.21.15.27:0/244456953 --> [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3fac0c4540 con 0x7f3fac0c1980 2024-09-21T00:20:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.720+0000 7f3fbea00640 1 -- 172.21.15.27:0/244456953 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f3fb8107c50 con 0x7f3fc41039b0 2024-09-21T00:20:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.720+0000 7f3fca800640 1 --2- 172.21.15.27:0/244456953 >> [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113] conn(0x7f3fac0c1980 0x7f3fac0c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.721+0000 7f3fc9400640 1 --2- 172.21.15.27:0/244456953 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3fac0b65f0 0x7f3fac0b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4108e40 tx=0x7f3fb4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:15.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.721+0000 7f3fca800640 1 --2- 172.21.15.27:0/244456953 >> [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113] conn(0x7f3fac0c1980 0x7f3fac0c3e70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:15.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.722+0000 7f3fbea00640 1 -- 172.21.15.27:0/244456953 <== osd.9 v2:172.21.15.27:6826/3033977113 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3fac0c4540 con 0x7f3fac0c1980 2024-09-21T00:20:15.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.738+0000 7f3fcb200640 1 -- 172.21.15.27:0/244456953 --> [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3fc4063f90 con 0x7f3fac0c1980 2024-09-21T00:20:15.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.739+0000 7f3fbea00640 1 -- 172.21.15.27:0/244456953 <== osd.9 v2:172.21.15.27:6826/3033977113 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3fc4063f90 con 0x7f3fac0c1980 2024-09-21T00:20:15.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.739+0000 7f3fcb200640 1 -- 172.21.15.27:0/244456953 >> [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113] conn(0x7f3fac0c1980 msgr2=0x7f3fac0c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:15.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.739+0000 7f3fcb200640 1 --2- 172.21.15.27:0/244456953 >> [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113] conn(0x7f3fac0c1980 0x7f3fac0c3e70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:15.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.739+0000 7f3fcb200640 1 -- 172.21.15.27:0/244456953 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3fac0b65f0 msgr2=0x7f3fac0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:15.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.739+0000 7f3fcb200640 1 --2- 172.21.15.27:0/244456953 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3fac0b65f0 0x7f3fac0b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4108e40 tx=0x7f3fb4005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:15.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.739+0000 7f3fcb200640 1 -- 172.21.15.27:0/244456953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fc41039b0 msgr2=0x7f3fc419a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:15.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.739+0000 7f3fcb200640 1 --2- 172.21.15.27:0/244456953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fc41039b0 0x7f3fc419a2d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80303f0 tx=0x7f3fb8032da0 comp rx=0 tx=0).stop 2024-09-21T00:20:15.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.739+0000 7f3fca800640 1 -- 172.21.15.27:0/244456953 reap_dead start 2024-09-21T00:20:15.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.739+0000 7f3fcb200640 1 -- 172.21.15.27:0/244456953 shutdown_connections 2024-09-21T00:20:15.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.739+0000 7f3fcb200640 1 -- 172.21.15.27:0/244456953 >> 172.21.15.27:0/244456953 conn(0x7f3fc40ff440 msgr2=0x7f3fc4101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:15.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.740+0000 7f3fcb200640 1 -- 172.21.15.27:0/244456953 shutdown_connections 2024-09-21T00:20:15.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.740+0000 7f3fcb200640 1 -- 172.21.15.27:0/244456953 wait complete. 2024-09-21T00:20:15.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.815+0000 7fda35600640 1 -- 172.21.15.27:0/244705044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda301039b0 msgr2=0x7fda30103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:15.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.815+0000 7fda35600640 1 --2- 172.21.15.27:0/244705044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda301039b0 0x7fda30103dc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fda20009900 tx=0x7fda2002a3f0 comp rx=0 tx=0).stop 2024-09-21T00:20:15.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.815+0000 7fda35600640 1 -- 172.21.15.27:0/244705044 shutdown_connections 2024-09-21T00:20:15.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.815+0000 7fda35600640 1 --2- 172.21.15.27:0/244705044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda30109c40 0x7fda30110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:15.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.815+0000 7fda35600640 1 --2- 172.21.15.27:0/244705044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda301092f0 0x7fda30109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:15.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.815+0000 7fda35600640 1 --2- 172.21.15.27:0/244705044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda301039b0 0x7fda30103dc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:15.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.815+0000 7fda35600640 1 -- 172.21.15.27:0/244705044 >> 172.21.15.27:0/244705044 conn(0x7fda300ff440 msgr2=0x7fda30101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:15.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.815+0000 7fda35600640 1 -- 172.21.15.27:0/244705044 shutdown_connections 2024-09-21T00:20:15.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.815+0000 7fda35600640 1 -- 172.21.15.27:0/244705044 wait complete. 2024-09-21T00:20:15.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.816+0000 7fda35600640 1 Processor -- start 2024-09-21T00:20:15.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.816+0000 7fda35600640 1 -- start start 2024-09-21T00:20:15.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda35600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda301039b0 0x7fda3019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:15.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda35600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301092f0 0x7fda3019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:15.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda35600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda30109c40 0x7fda3019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:15.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda35600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda30113900 con 0x7fda30109c40 2024-09-21T00:20:15.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda35600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fda30113780 con 0x7fda301092f0 2024-09-21T00:20:15.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda35600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fda30113a80 con 0x7fda301039b0 2024-09-21T00:20:15.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda2f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301092f0 0x7fda3019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:15.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda2f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301092f0 0x7fda3019a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:34558/0 (socket says 172.21.15.27:34558) 2024-09-21T00:20:15.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda2f400640 1 -- 172.21.15.27:0/1285185740 learned_addr learned my addr 172.21.15.27:0/1285185740 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:15.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda34c00640 1 --2- 172.21.15.27:0/1285185740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda30109c40 0x7fda3019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:15.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda2fe00640 1 --2- 172.21.15.27:0/1285185740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda301039b0 0x7fda3019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:15.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda2f400640 1 -- 172.21.15.27:0/1285185740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda301039b0 msgr2=0x7fda3019a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:15.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda2f400640 1 --2- 172.21.15.27:0/1285185740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda301039b0 0x7fda3019a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:15.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda2f400640 1 -- 172.21.15.27:0/1285185740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda30109c40 msgr2=0x7fda3019fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:15.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda2f400640 1 --2- 172.21.15.27:0/1285185740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda30109c40 0x7fda3019fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:15.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.817+0000 7fda2f400640 1 -- 172.21.15.27:0/1285185740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda301a0280 con 0x7fda301092f0 2024-09-21T00:20:15.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.818+0000 7fda2f400640 1 --2- 172.21.15.27:0/1285185740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301092f0 0x7fda3019a8b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00e9c0 tx=0x7fda1c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:15.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.818+0000 7fda2cc00640 1 -- 172.21.15.27:0/1285185740 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c00ccc0 con 0x7fda301092f0 2024-09-21T00:20:15.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.818+0000 7fda35600640 1 -- 172.21.15.27:0/1285185740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda301a0570 con 0x7fda301092f0 2024-09-21T00:20:15.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.818+0000 7fda2cc00640 1 -- 172.21.15.27:0/1285185740 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda1c0054d0 con 0x7fda301092f0 2024-09-21T00:20:15.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.818+0000 7fda2cc00640 1 -- 172.21.15.27:0/1285185740 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c010430 con 0x7fda301092f0 2024-09-21T00:20:15.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.818+0000 7fda35600640 1 -- 172.21.15.27:0/1285185740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda301a6cd0 con 0x7fda301092f0 2024-09-21T00:20:15.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.819+0000 7fda35600640 1 -- 172.21.15.27:0/1285185740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fda30063b10 con 0x7fda301092f0 2024-09-21T00:20:15.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.822+0000 7fda2cc00640 1 -- 172.21.15.27:0/1285185740 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fda1c005000 con 0x7fda301092f0 2024-09-21T00:20:15.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.824+0000 7fda2cc00640 1 --2- 172.21.15.27:0/1285185740 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda140b65f0 0x7fda140b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:15.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.824+0000 7fda2cc00640 1 -- 172.21.15.27:0/1285185740 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7fda1c0e80a0 con 0x7fda301092f0 2024-09-21T00:20:15.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.824+0000 7fda2cc00640 1 --2- 172.21.15.27:0/1285185740 >> [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621] conn(0x7fda140c1980 0x7fda140c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:15.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.824+0000 7fda2cc00640 1 -- 172.21.15.27:0/1285185740 --> [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda1c016ec0 con 0x7fda140c1980 2024-09-21T00:20:15.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.824+0000 7fda2cc00640 1 -- 172.21.15.27:0/1285185740 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fda1c0e84e0 con 0x7fda301092f0 2024-09-21T00:20:15.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.824+0000 7fda2fe00640 1 --2- 172.21.15.27:0/1285185740 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda140b65f0 0x7fda140b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:15.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.825+0000 7fda34c00640 1 --2- 172.21.15.27:0/1285185740 >> [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621] conn(0x7fda140c1980 0x7fda140c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:15.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.825+0000 7fda2fe00640 1 --2- 172.21.15.27:0/1285185740 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda140b65f0 0x7fda140b8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fda20031440 tx=0x7fda20009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:15.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.825+0000 7fda34c00640 1 --2- 172.21.15.27:0/1285185740 >> [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621] conn(0x7fda140c1980 0x7fda140c3e70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:15.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.826+0000 7fda2cc00640 1 -- 172.21.15.27:0/1285185740 <== osd.1 v2:172.21.15.89:6800/922528621 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda1c016ec0 con 0x7fda140c1980 2024-09-21T00:20:15.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.841+0000 7fda35600640 1 -- 172.21.15.27:0/1285185740 --> [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fda30063f90 con 0x7fda140c1980 2024-09-21T00:20:15.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.842+0000 7fda2cc00640 1 -- 172.21.15.27:0/1285185740 <== osd.1 v2:172.21.15.89:6800/922528621 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fda30063f90 con 0x7fda140c1980 2024-09-21T00:20:15.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.842+0000 7fda35600640 1 -- 172.21.15.27:0/1285185740 >> [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621] conn(0x7fda140c1980 msgr2=0x7fda140c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:15.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.842+0000 7fda35600640 1 --2- 172.21.15.27:0/1285185740 >> [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621] conn(0x7fda140c1980 0x7fda140c3e70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:15.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.842+0000 7fda35600640 1 -- 172.21.15.27:0/1285185740 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda140b65f0 msgr2=0x7fda140b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:15.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.842+0000 7fda35600640 1 --2- 172.21.15.27:0/1285185740 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda140b65f0 0x7fda140b8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fda20031440 tx=0x7fda20009560 comp rx=0 tx=0).stop 2024-09-21T00:20:15.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.842+0000 7fda35600640 1 -- 172.21.15.27:0/1285185740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301092f0 msgr2=0x7fda3019a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:15.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.842+0000 7fda35600640 1 --2- 172.21.15.27:0/1285185740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301092f0 0x7fda3019a8b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00e9c0 tx=0x7fda1c00ee80 comp rx=0 tx=0).stop 2024-09-21T00:20:15.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.842+0000 7fda34c00640 1 -- 172.21.15.27:0/1285185740 reap_dead start 2024-09-21T00:20:15.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.842+0000 7fda35600640 1 -- 172.21.15.27:0/1285185740 shutdown_connections 2024-09-21T00:20:15.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.842+0000 7fda35600640 1 -- 172.21.15.27:0/1285185740 >> 172.21.15.27:0/1285185740 conn(0x7fda300ff440 msgr2=0x7fda30101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:15.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.842+0000 7fda35600640 1 -- 172.21.15.27:0/1285185740 shutdown_connections 2024-09-21T00:20:15.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:14.842+0000 7fda35600640 1 -- 172.21.15.27:0/1285185740 wait complete. 2024-09-21T00:20:15.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:15 smithi089 ceph-mon[34481]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-21T00:20:15.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:15 smithi098 ceph-mon[34725]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-21T00:20:15.787 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:15 smithi027 ceph-mon[30745]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-21T00:20:16.383 INFO:teuthology.orchestra.run.smithi027.stdout:60129542233 2024-09-21T00:20:16.383 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T00:20:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.894+0000 7f1559e00640 1 -- 172.21.15.27:0/3675277915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1554109e10 msgr2=0x7f1554110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.894+0000 7f1559e00640 1 --2- 172.21.15.27:0/3675277915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1554109e10 0x7f1554110640 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f154c009ca0 tx=0x7f154c02a8c0 comp rx=0 tx=0).stop 2024-09-21T00:20:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.895+0000 7f1559e00640 1 -- 172.21.15.27:0/3675277915 shutdown_connections 2024-09-21T00:20:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.895+0000 7f1559e00640 1 --2- 172.21.15.27:0/3675277915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1554109e10 0x7f1554110640 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.895+0000 7f1559e00640 1 --2- 172.21.15.27:0/3675277915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15541094c0 0x7f15541098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.895+0000 7f1559e00640 1 --2- 172.21.15.27:0/3675277915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1554103b80 0x7f1554103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.895+0000 7f1559e00640 1 -- 172.21.15.27:0/3675277915 >> 172.21.15.27:0/3675277915 conn(0x7f15540ff590 msgr2=0x7f15541019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.895+0000 7f1559e00640 1 -- 172.21.15.27:0/3675277915 shutdown_connections 2024-09-21T00:20:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.895+0000 7f1559e00640 1 -- 172.21.15.27:0/3675277915 wait complete. 2024-09-21T00:20:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.896+0000 7f1559e00640 1 Processor -- start 2024-09-21T00:20:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.896+0000 7f1559e00640 1 -- start start 2024-09-21T00:20:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.896+0000 7f1559e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1554103b80 0x7f155419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.896+0000 7f1559e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15541094c0 0x7f155419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.897+0000 7f1559e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1554109e10 0x7f15541a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.897+0000 7f1559e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1554113b20 con 0x7f15541094c0 2024-09-21T00:20:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.897+0000 7f1559e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f15541139a0 con 0x7f1554103b80 2024-09-21T00:20:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.897+0000 7f1559e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1554113ca0 con 0x7f1554109e10 2024-09-21T00:20:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.897+0000 7f1552a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1554103b80 0x7f155419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.897+0000 7f1552a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1554103b80 0x7f155419c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:34496/0 (socket says 172.21.15.27:34496) 2024-09-21T00:20:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.897+0000 7f1552a00640 1 -- 172.21.15.27:0/2228585454 learned_addr learned my addr 172.21.15.27:0/2228585454 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.897+0000 7f1553400640 1 --2- 172.21.15.27:0/2228585454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1554109e10 0x7f15541a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.897+0000 7f1552a00640 1 -- 172.21.15.27:0/2228585454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1554109e10 msgr2=0x7f15541a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.897+0000 7f1552a00640 1 --2- 172.21.15.27:0/2228585454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1554109e10 0x7f15541a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.897+0000 7f1552a00640 1 -- 172.21.15.27:0/2228585454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15541094c0 msgr2=0x7f155419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.897+0000 7f1552a00640 1 --2- 172.21.15.27:0/2228585454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15541094c0 0x7f155419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.897+0000 7f1552a00640 1 -- 172.21.15.27:0/2228585454 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15541a25c0 con 0x7f1554103b80 2024-09-21T00:20:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.898+0000 7f1552a00640 1 --2- 172.21.15.27:0/2228585454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1554103b80 0x7f155419c650 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f154400ea60 tx=0x7f154400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.898+0000 7f1558a00640 1 -- 172.21.15.27:0/2228585454 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154400cde0 con 0x7f1554103b80 2024-09-21T00:20:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.898+0000 7f1559e00640 1 -- 172.21.15.27:0/2228585454 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15541a28b0 con 0x7f1554103b80 2024-09-21T00:20:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.898+0000 7f1558a00640 1 -- 172.21.15.27:0/2228585454 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15440035f0 con 0x7f1554103b80 2024-09-21T00:20:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.898+0000 7f1558a00640 1 -- 172.21.15.27:0/2228585454 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544010690 con 0x7f1554103b80 2024-09-21T00:20:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.898+0000 7f1559e00640 1 -- 172.21.15.27:0/2228585454 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15541a9010 con 0x7f1554103b80 2024-09-21T00:20:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.899+0000 7f1559e00640 1 -- 172.21.15.27:0/2228585454 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1554063b50 con 0x7f1554103b80 2024-09-21T00:20:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.902+0000 7f1558a00640 1 -- 172.21.15.27:0/2228585454 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f15440037a0 con 0x7f1554103b80 2024-09-21T00:20:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.903+0000 7f1558a00640 1 --2- 172.21.15.27:0/2228585454 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f15380b65f0 0x7f15380b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.904+0000 7f1558a00640 1 -- 172.21.15.27:0/2228585454 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7f15440e8330 con 0x7f1554103b80 2024-09-21T00:20:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.904+0000 7f1552000640 1 --2- 172.21.15.27:0/2228585454 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f15380b65f0 0x7f15380b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.904+0000 7f1558a00640 1 --2- 172.21.15.27:0/2228585454 >> [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225] conn(0x7f15380c1970 0x7f15380c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.904+0000 7f1558a00640 1 -- 172.21.15.27:0/2228585454 --> [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f15380c4530 con 0x7f15380c1970 2024-09-21T00:20:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.904+0000 7f1558a00640 1 -- 172.21.15.27:0/2228585454 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f15440e86e0 con 0x7f1554103b80 2024-09-21T00:20:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.905+0000 7f1553400640 1 --2- 172.21.15.27:0/2228585454 >> [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225] conn(0x7f15380c1970 0x7f15380c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.905+0000 7f1552000640 1 --2- 172.21.15.27:0/2228585454 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f15380b65f0 0x7f15380b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f15400058c0 tx=0x7f1540005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.905+0000 7f1553400640 1 --2- 172.21.15.27:0/2228585454 >> [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225] conn(0x7f15380c1970 0x7f15380c3e60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.906+0000 7f1558a00640 1 -- 172.21.15.27:0/2228585454 <== osd.10 v2:172.21.15.89:6824/2370332225 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f15380c4530 con 0x7f15380c1970 2024-09-21T00:20:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.921+0000 7f1559e00640 1 -- 172.21.15.27:0/2228585454 --> [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1554063f80 con 0x7f15380c1970 2024-09-21T00:20:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.922+0000 7f1558a00640 1 -- 172.21.15.27:0/2228585454 <== osd.10 v2:172.21.15.89:6824/2370332225 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1554063f80 con 0x7f15380c1970 2024-09-21T00:20:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.922+0000 7f1559e00640 1 -- 172.21.15.27:0/2228585454 >> [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225] conn(0x7f15380c1970 msgr2=0x7f15380c3e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.922+0000 7f1559e00640 1 --2- 172.21.15.27:0/2228585454 >> [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225] conn(0x7f15380c1970 0x7f15380c3e60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.922+0000 7f1559e00640 1 -- 172.21.15.27:0/2228585454 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f15380b65f0 msgr2=0x7f15380b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.922+0000 7f1559e00640 1 --2- 172.21.15.27:0/2228585454 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f15380b65f0 0x7f15380b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f15400058c0 tx=0x7f1540005850 comp rx=0 tx=0).stop 2024-09-21T00:20:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.923+0000 7f1559e00640 1 -- 172.21.15.27:0/2228585454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1554103b80 msgr2=0x7f155419c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.923+0000 7f1559e00640 1 --2- 172.21.15.27:0/2228585454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1554103b80 0x7f155419c650 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f154400ea60 tx=0x7f154400ef20 comp rx=0 tx=0).stop 2024-09-21T00:20:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.923+0000 7f1553400640 1 -- 172.21.15.27:0/2228585454 reap_dead start 2024-09-21T00:20:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.923+0000 7f1559e00640 1 -- 172.21.15.27:0/2228585454 shutdown_connections 2024-09-21T00:20:16.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.923+0000 7f1559e00640 1 -- 172.21.15.27:0/2228585454 >> 172.21.15.27:0/2228585454 conn(0x7f15540ff590 msgr2=0x7f15541069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:16.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.923+0000 7f1559e00640 1 -- 172.21.15.27:0/2228585454 shutdown_connections 2024-09-21T00:20:16.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:13.923+0000 7f1559e00640 1 -- 172.21.15.27:0/2228585454 wait complete. 2024-09-21T00:20:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.237+0000 7fa893c00640 1 -- 172.21.15.27:0/2343039922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa88c10b6d0 msgr2=0x7fa88c10bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.237+0000 7fa893c00640 1 --2- 172.21.15.27:0/2343039922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa88c10b6d0 0x7fa88c10bae0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa878009860 tx=0x7fa87802a340 comp rx=0 tx=0).stop 2024-09-21T00:20:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.237+0000 7fa893c00640 1 -- 172.21.15.27:0/2343039922 shutdown_connections 2024-09-21T00:20:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.237+0000 7fa893c00640 1 --2- 172.21.15.27:0/2343039922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c10c020 0x7fa88c112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.237+0000 7fa893c00640 1 --2- 172.21.15.27:0/2343039922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa88c10b6d0 0x7fa88c10bae0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.237+0000 7fa893c00640 1 --2- 172.21.15.27:0/2343039922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa88c105d90 0x7fa88c1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.237+0000 7fa893c00640 1 -- 172.21.15.27:0/2343039922 >> 172.21.15.27:0/2343039922 conn(0x7fa88c101860 msgr2=0x7fa88c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:16.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.237+0000 7fa893c00640 1 -- 172.21.15.27:0/2343039922 shutdown_connections 2024-09-21T00:20:16.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.237+0000 7fa893c00640 1 -- 172.21.15.27:0/2343039922 wait complete. 2024-09-21T00:20:16.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.238+0000 7fa893c00640 1 Processor -- start 2024-09-21T00:20:16.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.239+0000 7fa893c00640 1 -- start start 2024-09-21T00:20:16.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.239+0000 7fa893c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa88c105d90 0x7fa88c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:16.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.239+0000 7fa893c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa88c10b6d0 0x7fa88c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:16.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.239+0000 7fa893c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c10c020 0x7fa88c1a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:16.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.239+0000 7fa893c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c115ce0 con 0x7fa88c10c020 2024-09-21T00:20:16.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.239+0000 7fa893c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c115b60 con 0x7fa88c105d90 2024-09-21T00:20:16.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.239+0000 7fa893c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c115e60 con 0x7fa88c10b6d0 2024-09-21T00:20:16.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.239+0000 7fa891c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c10c020 0x7fa88c1a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:16.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.239+0000 7fa891200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa88c105d90 0x7fa88c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:16.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.239+0000 7fa891c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c10c020 0x7fa88c1a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33842/0 (socket says 172.21.15.27:33842) 2024-09-21T00:20:16.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.239+0000 7fa891c00640 1 -- 172.21.15.27:0/3484468960 learned_addr learned my addr 172.21.15.27:0/3484468960 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:16.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.239+0000 7fa88be00640 1 --2- 172.21.15.27:0/3484468960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa88c10b6d0 0x7fa88c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:16.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.240+0000 7fa891200640 1 -- 172.21.15.27:0/3484468960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa88c10b6d0 msgr2=0x7fa88c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:16.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.240+0000 7fa891200640 1 --2- 172.21.15.27:0/3484468960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa88c10b6d0 0x7fa88c19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:16.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.240+0000 7fa891200640 1 -- 172.21.15.27:0/3484468960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c10c020 msgr2=0x7fa88c1a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:16.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.240+0000 7fa891200640 1 --2- 172.21.15.27:0/3484468960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c10c020 0x7fa88c1a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:16.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.240+0000 7fa891200640 1 -- 172.21.15.27:0/3484468960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88c1a2660 con 0x7fa88c105d90 2024-09-21T00:20:16.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.240+0000 7fa891c00640 1 --2- 172.21.15.27:0/3484468960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c10c020 0x7fa88c1a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:16.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.240+0000 7fa891200640 1 --2- 172.21.15.27:0/3484468960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa88c105d90 0x7fa88c19c6f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa880009fd0 tx=0x7fa88000eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:16.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.240+0000 7fa889600640 1 -- 172.21.15.27:0/3484468960 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88000cde0 con 0x7fa88c105d90 2024-09-21T00:20:16.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.241+0000 7fa893c00640 1 -- 172.21.15.27:0/3484468960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88c1a2950 con 0x7fa88c105d90 2024-09-21T00:20:16.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.241+0000 7fa889600640 1 -- 172.21.15.27:0/3484468960 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa880005020 con 0x7fa88c105d90 2024-09-21T00:20:16.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.241+0000 7fa889600640 1 -- 172.21.15.27:0/3484468960 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880010690 con 0x7fa88c105d90 2024-09-21T00:20:16.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.241+0000 7fa893c00640 1 -- 172.21.15.27:0/3484468960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88c1a90b0 con 0x7fa88c105d90 2024-09-21T00:20:16.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.241+0000 7fa893c00640 1 -- 172.21.15.27:0/3484468960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa88c063b50 con 0x7fa88c105d90 2024-09-21T00:20:16.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.245+0000 7fa889600640 1 -- 172.21.15.27:0/3484468960 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fa8800054d0 con 0x7fa88c105d90 2024-09-21T00:20:16.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.246+0000 7fa889600640 1 --2- 172.21.15.27:0/3484468960 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa8740b65f0 0x7fa8740b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:16.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.246+0000 7fa889600640 1 -- 172.21.15.27:0/3484468960 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7fa8800e82e0 con 0x7fa88c105d90 2024-09-21T00:20:16.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.246+0000 7fa88be00640 1 --2- 172.21.15.27:0/3484468960 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa8740b65f0 0x7fa8740b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:16.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.247+0000 7fa889600640 1 --2- 172.21.15.27:0/3484468960 >> [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064] conn(0x7fa8740c1980 0x7fa8740c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:16.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.247+0000 7fa889600640 1 -- 172.21.15.27:0/3484468960 --> [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8740c4540 con 0x7fa8740c1980 2024-09-21T00:20:16.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.247+0000 7fa889600640 1 -- 172.21.15.27:0/3484468960 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fa8800e8690 con 0x7fa88c105d90 2024-09-21T00:20:16.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.247+0000 7fa891c00640 1 --2- 172.21.15.27:0/3484468960 >> [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064] conn(0x7fa8740c1980 0x7fa8740c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.247+0000 7fa88be00640 1 --2- 172.21.15.27:0/3484468960 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa8740b65f0 0x7fa8740b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa878035040 tx=0x7fa878030340 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.247+0000 7fa891c00640 1 --2- 172.21.15.27:0/3484468960 >> [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064] conn(0x7fa8740c1980 0x7fa8740c3e70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.248+0000 7fa889600640 1 -- 172.21.15.27:0/3484468960 <== osd.11 v2:172.21.15.98:6824/2795999064 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa8740c4540 con 0x7fa8740c1980 2024-09-21T00:20:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.263+0000 7fa893c00640 1 -- 172.21.15.27:0/3484468960 --> [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa88c063fd0 con 0x7fa8740c1980 2024-09-21T00:20:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.264+0000 7fa889600640 1 -- 172.21.15.27:0/3484468960 <== osd.11 v2:172.21.15.98:6824/2795999064 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa88c063fd0 con 0x7fa8740c1980 2024-09-21T00:20:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.264+0000 7fa893c00640 1 -- 172.21.15.27:0/3484468960 >> [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064] conn(0x7fa8740c1980 msgr2=0x7fa8740c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.264+0000 7fa893c00640 1 --2- 172.21.15.27:0/3484468960 >> [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064] conn(0x7fa8740c1980 0x7fa8740c3e70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.264+0000 7fa893c00640 1 -- 172.21.15.27:0/3484468960 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa8740b65f0 msgr2=0x7fa8740b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.264+0000 7fa893c00640 1 --2- 172.21.15.27:0/3484468960 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa8740b65f0 0x7fa8740b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa878035040 tx=0x7fa878030340 comp rx=0 tx=0).stop 2024-09-21T00:20:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.264+0000 7fa893c00640 1 -- 172.21.15.27:0/3484468960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa88c105d90 msgr2=0x7fa88c19c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.264+0000 7fa893c00640 1 --2- 172.21.15.27:0/3484468960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa88c105d90 0x7fa88c19c6f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa880009fd0 tx=0x7fa88000eed0 comp rx=0 tx=0).stop 2024-09-21T00:20:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.264+0000 7fa891c00640 1 -- 172.21.15.27:0/3484468960 reap_dead start 2024-09-21T00:20:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.264+0000 7fa893c00640 1 -- 172.21.15.27:0/3484468960 shutdown_connections 2024-09-21T00:20:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.264+0000 7fa893c00640 1 -- 172.21.15.27:0/3484468960 >> 172.21.15.27:0/3484468960 conn(0x7fa88c101860 msgr2=0x7fa88c103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:16.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.265+0000 7fa893c00640 1 -- 172.21.15.27:0/3484468960 shutdown_connections 2024-09-21T00:20:16.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:15.265+0000 7fa893c00640 1 -- 172.21.15.27:0/3484468960 wait complete. 2024-09-21T00:20:17.528 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:17 smithi027 ceph-mon[30745]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:20:17.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.064+0000 7f88cf000640 1 -- 172.21.15.27:0/45014229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88c80760a0 msgr2=0x7f88c810fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:17.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.064+0000 7f88cf000640 1 --2- 172.21.15.27:0/45014229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88c80760a0 0x7f88c810fbe0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009900 tx=0x7f88b402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:20:17.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.065+0000 7f88cf000640 1 -- 172.21.15.27:0/45014229 shutdown_connections 2024-09-21T00:20:17.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.065+0000 7f88cf000640 1 --2- 172.21.15.27:0/45014229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c8110120 0x7f88c8112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:17.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.065+0000 7f88cf000640 1 --2- 172.21.15.27:0/45014229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88c80760a0 0x7f88c810fbe0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:17.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.065+0000 7f88cf000640 1 --2- 172.21.15.27:0/45014229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88c8075750 0x7f88c8075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:17.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.065+0000 7f88cf000640 1 -- 172.21.15.27:0/45014229 >> 172.21.15.27:0/45014229 conn(0x7f88c807b850 msgr2=0x7f88c807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:17.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.065+0000 7f88cf000640 1 -- 172.21.15.27:0/45014229 shutdown_connections 2024-09-21T00:20:17.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.065+0000 7f88cf000640 1 -- 172.21.15.27:0/45014229 wait complete. 2024-09-21T00:20:17.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.066+0000 7f88cf000640 1 Processor -- start 2024-09-21T00:20:17.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.066+0000 7f88cf000640 1 -- start start 2024-09-21T00:20:17.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.066+0000 7f88cf000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88c8075750 0x7f88c819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:17.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88cf000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88c80760a0 0x7f88c819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:17.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88cf000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c8110120 0x7f88c81a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:17.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88cf000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8115c90 con 0x7f88c80760a0 2024-09-21T00:20:17.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88cf000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8115b10 con 0x7f88c8075750 2024-09-21T00:20:17.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88cf000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8115e10 con 0x7f88c8110120 2024-09-21T00:20:17.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88c7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88c80760a0 0x7f88c819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:17.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88c7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88c8075750 0x7f88c819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:17.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88c7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88c8075750 0x7f88c819c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:34594/0 (socket says 172.21.15.27:34594) 2024-09-21T00:20:17.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88c7e00640 1 -- 172.21.15.27:0/2687147776 learned_addr learned my addr 172.21.15.27:0/2687147776 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:17.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88c7400640 1 --2- 172.21.15.27:0/2687147776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88c80760a0 0x7f88c819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33872/0 (socket says 172.21.15.27:33872) 2024-09-21T00:20:17.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88cd000640 1 --2- 172.21.15.27:0/2687147776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c8110120 0x7f88c81a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:17.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88c7e00640 1 -- 172.21.15.27:0/2687147776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c8110120 msgr2=0x7f88c81a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:17.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88c7e00640 1 --2- 172.21.15.27:0/2687147776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c8110120 0x7f88c81a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:17.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88c7e00640 1 -- 172.21.15.27:0/2687147776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88c80760a0 msgr2=0x7f88c819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:17.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88c7e00640 1 --2- 172.21.15.27:0/2687147776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88c80760a0 0x7f88c819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:17.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88c7e00640 1 -- 172.21.15.27:0/2687147776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c81a2690 con 0x7f88c8075750 2024-09-21T00:20:17.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.067+0000 7f88cd000640 1 --2- 172.21.15.27:0/2687147776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c8110120 0x7f88c81a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:20:17.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.068+0000 7f88c7e00640 1 --2- 172.21.15.27:0/2687147776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88c8075750 0x7f88c819c720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f88b800ea60 tx=0x7f88b800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:17.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.068+0000 7f88c4c00640 1 -- 172.21.15.27:0/2687147776 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b800cde0 con 0x7f88c8075750 2024-09-21T00:20:17.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.068+0000 7f88cf000640 1 -- 172.21.15.27:0/2687147776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c81a2980 con 0x7f88c8075750 2024-09-21T00:20:17.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.068+0000 7f88c4c00640 1 -- 172.21.15.27:0/2687147776 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b80035f0 con 0x7f88c8075750 2024-09-21T00:20:17.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.068+0000 7f88c4c00640 1 -- 172.21.15.27:0/2687147776 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8010690 con 0x7f88c8075750 2024-09-21T00:20:17.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.068+0000 7f88cf000640 1 -- 172.21.15.27:0/2687147776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c81a90e0 con 0x7f88c8075750 2024-09-21T00:20:17.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.069+0000 7f88cf000640 1 -- 172.21.15.27:0/2687147776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f88c810a720 con 0x7f88c8075750 2024-09-21T00:20:17.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.072+0000 7f88c4c00640 1 -- 172.21.15.27:0/2687147776 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f88b80037a0 con 0x7f88c8075750 2024-09-21T00:20:17.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.073+0000 7f88c4c00640 1 --2- 172.21.15.27:0/2687147776 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f88ac0b6670 0x7f88ac0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:17.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.074+0000 7f88c4c00640 1 -- 172.21.15.27:0/2687147776 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7f88b80e8090 con 0x7f88c8075750 2024-09-21T00:20:17.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.074+0000 7f88c7400640 1 --2- 172.21.15.27:0/2687147776 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f88ac0b6670 0x7f88ac0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:17.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.074+0000 7f88c4c00640 1 --2- 172.21.15.27:0/2687147776 >> [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485] conn(0x7f88ac0c1a00 0x7f88ac0c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:17.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.074+0000 7f88c4c00640 1 -- 172.21.15.27:0/2687147776 --> [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f88b8011e50 con 0x7f88ac0c1a00 2024-09-21T00:20:17.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.074+0000 7f88c4c00640 1 -- 172.21.15.27:0/2687147776 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f88b80e8440 con 0x7f88c8075750 2024-09-21T00:20:17.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.074+0000 7f88cd000640 1 --2- 172.21.15.27:0/2687147776 >> [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485] conn(0x7f88ac0c1a00 0x7f88ac0c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:17.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.074+0000 7f88c7400640 1 --2- 172.21.15.27:0/2687147776 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f88ac0b6670 0x7f88ac0b8b80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009230 tx=0x7f88b4031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:17.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.075+0000 7f88cd000640 1 --2- 172.21.15.27:0/2687147776 >> [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485] conn(0x7f88ac0c1a00 0x7f88ac0c3ef0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:17.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.076+0000 7f88c4c00640 1 -- 172.21.15.27:0/2687147776 <== osd.8 v2:172.21.15.98:6816/2830381485 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f88b8011e50 con 0x7f88ac0c1a00 2024-09-21T00:20:17.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.090+0000 7f88cf000640 1 -- 172.21.15.27:0/2687147776 --> [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f88c8076c60 con 0x7f88ac0c1a00 2024-09-21T00:20:17.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.091+0000 7f88c4c00640 1 -- 172.21.15.27:0/2687147776 <== osd.8 v2:172.21.15.98:6816/2830381485 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f88c8076c60 con 0x7f88ac0c1a00 2024-09-21T00:20:17.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.091+0000 7f88cf000640 1 -- 172.21.15.27:0/2687147776 >> [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485] conn(0x7f88ac0c1a00 msgr2=0x7f88ac0c3ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:17.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.091+0000 7f88cf000640 1 --2- 172.21.15.27:0/2687147776 >> [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485] conn(0x7f88ac0c1a00 0x7f88ac0c3ef0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:17.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.091+0000 7f88cf000640 1 -- 172.21.15.27:0/2687147776 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f88ac0b6670 msgr2=0x7f88ac0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:17.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.091+0000 7f88cf000640 1 --2- 172.21.15.27:0/2687147776 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f88ac0b6670 0x7f88ac0b8b80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009230 tx=0x7f88b4031400 comp rx=0 tx=0).stop 2024-09-21T00:20:17.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.091+0000 7f88cf000640 1 -- 172.21.15.27:0/2687147776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88c8075750 msgr2=0x7f88c819c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:17.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.091+0000 7f88cf000640 1 --2- 172.21.15.27:0/2687147776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88c8075750 0x7f88c819c720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f88b800ea60 tx=0x7f88b800ef20 comp rx=0 tx=0).stop 2024-09-21T00:20:17.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.091+0000 7f88cd000640 1 -- 172.21.15.27:0/2687147776 reap_dead start 2024-09-21T00:20:17.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.091+0000 7f88cf000640 1 -- 172.21.15.27:0/2687147776 shutdown_connections 2024-09-21T00:20:17.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.091+0000 7f88cf000640 1 -- 172.21.15.27:0/2687147776 >> 172.21.15.27:0/2687147776 conn(0x7f88c807b850 msgr2=0x7f88c8109cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:17.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.091+0000 7f88cf000640 1 -- 172.21.15.27:0/2687147776 shutdown_connections 2024-09-21T00:20:17.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:16.091+0000 7f88cf000640 1 -- 172.21.15.27:0/2687147776 wait complete. 2024-09-21T00:20:17.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:17 smithi089 ceph-mon[34481]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:20:17.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:17 smithi098 ceph-mon[34725]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:20:18.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.229+0000 7fb83ea00640 1 -- 172.21.15.27:0/3832983909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb838075730 msgr2=0x7fb838075b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:18.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.229+0000 7fb83ea00640 1 --2- 172.21.15.27:0/3832983909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb838075730 0x7fb838075b40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb828009900 tx=0x7fb82802a410 comp rx=0 tx=0).stop 2024-09-21T00:20:18.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.229+0000 7fb83ea00640 1 -- 172.21.15.27:0/3832983909 shutdown_connections 2024-09-21T00:20:18.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.229+0000 7fb83ea00640 1 --2- 172.21.15.27:0/3832983909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb838110140 0x7fb838112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:18.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.229+0000 7fb83ea00640 1 --2- 172.21.15.27:0/3832983909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb838076080 0x7fb83810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:18.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.229+0000 7fb83ea00640 1 --2- 172.21.15.27:0/3832983909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb838075730 0x7fb838075b40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:18.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.229+0000 7fb83ea00640 1 -- 172.21.15.27:0/3832983909 >> 172.21.15.27:0/3832983909 conn(0x7fb83807b850 msgr2=0x7fb83807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:18.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.230+0000 7fb83ea00640 1 -- 172.21.15.27:0/3832983909 shutdown_connections 2024-09-21T00:20:18.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.230+0000 7fb83ea00640 1 -- 172.21.15.27:0/3832983909 wait complete. 2024-09-21T00:20:18.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.232+0000 7fb83ea00640 1 Processor -- start 2024-09-21T00:20:18.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.232+0000 7fb83ea00640 1 -- start start 2024-09-21T00:20:18.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.232+0000 7fb83ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb838075730 0x7fb83819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:18.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.233+0000 7fb83ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb838076080 0x7fb83819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:18.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.233+0000 7fb837e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb838075730 0x7fb83819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:18.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.233+0000 7fb837e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb838075730 0x7fb83819c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:34610/0 (socket says 172.21.15.27:34610) 2024-09-21T00:20:18.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.233+0000 7fb837400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb838076080 0x7fb83819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:18.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.233+0000 7fb837400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb838076080 0x7fb83819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33892/0 (socket says 172.21.15.27:33892) 2024-09-21T00:20:18.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.233+0000 7fb83ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb838110140 0x7fb8381a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:18.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.233+0000 7fb83ea00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb838115cf0 con 0x7fb838076080 2024-09-21T00:20:18.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.233+0000 7fb83ea00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb838115b70 con 0x7fb838075730 2024-09-21T00:20:18.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.233+0000 7fb83ea00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb838115e70 con 0x7fb838110140 2024-09-21T00:20:18.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.233+0000 7fb837e00640 1 -- 172.21.15.27:0/561581906 learned_addr learned my addr 172.21.15.27:0/561581906 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:18.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.234+0000 7fb83ca00640 1 --2- 172.21.15.27:0/561581906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb838110140 0x7fb8381a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:18.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.234+0000 7fb837400640 1 -- 172.21.15.27:0/561581906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb838110140 msgr2=0x7fb8381a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:18.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.234+0000 7fb837400640 1 --2- 172.21.15.27:0/561581906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb838110140 0x7fb8381a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:18.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.234+0000 7fb837400640 1 -- 172.21.15.27:0/561581906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb838075730 msgr2=0x7fb83819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:18.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.234+0000 7fb837400640 1 --2- 172.21.15.27:0/561581906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb838075730 0x7fb83819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:18.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.234+0000 7fb837400640 1 -- 172.21.15.27:0/561581906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8381a2710 con 0x7fb838076080 2024-09-21T00:20:18.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.234+0000 7fb83ca00640 1 --2- 172.21.15.27:0/561581906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb838110140 0x7fb8381a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:20:18.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.234+0000 7fb837e00640 1 --2- 172.21.15.27:0/561581906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb838075730 0x7fb83819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:18.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.234+0000 7fb837400640 1 --2- 172.21.15.27:0/561581906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb838076080 0x7fb83819cd40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb83806c9f0 tx=0x7fb82400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:18.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.235+0000 7fb834c00640 1 -- 172.21.15.27:0/561581906 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82400cc40 con 0x7fb838076080 2024-09-21T00:20:18.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.235+0000 7fb834c00640 1 -- 172.21.15.27:0/561581906 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb82400cde0 con 0x7fb838076080 2024-09-21T00:20:18.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.235+0000 7fb83ea00640 1 -- 172.21.15.27:0/561581906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8381a2a00 con 0x7fb838076080 2024-09-21T00:20:18.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.235+0000 7fb834c00640 1 -- 172.21.15.27:0/561581906 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824004a00 con 0x7fb838076080 2024-09-21T00:20:18.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.235+0000 7fb83ea00640 1 -- 172.21.15.27:0/561581906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8381a9160 con 0x7fb838076080 2024-09-21T00:20:18.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.236+0000 7fb83ea00640 1 -- 172.21.15.27:0/561581906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb8381096c0 con 0x7fb838076080 2024-09-21T00:20:18.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.241+0000 7fb834c00640 1 -- 172.21.15.27:0/561581906 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fb824004ba0 con 0x7fb838076080 2024-09-21T00:20:18.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.242+0000 7fb834c00640 1 --2- 172.21.15.27:0/561581906 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb81c0b6670 0x7fb81c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:18.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.242+0000 7fb834c00640 1 -- 172.21.15.27:0/561581906 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7fb8240e7cf0 con 0x7fb838076080 2024-09-21T00:20:18.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.243+0000 7fb837e00640 1 --2- 172.21.15.27:0/561581906 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb81c0b6670 0x7fb81c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:18.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.243+0000 7fb834c00640 1 --2- 172.21.15.27:0/561581906 >> [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546] conn(0x7fb81c0c1a00 0x7fb81c0c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:18.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.243+0000 7fb834c00640 1 -- 172.21.15.27:0/561581906 --> [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb82400be50 con 0x7fb81c0c1a00 2024-09-21T00:20:18.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.243+0000 7fb834c00640 1 -- 172.21.15.27:0/561581906 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fb8240e8140 con 0x7fb838076080 2024-09-21T00:20:18.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.243+0000 7fb837e00640 1 --2- 172.21.15.27:0/561581906 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb81c0b6670 0x7fb81c0b8b80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb8280096c0 tx=0x7fb828005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:18.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.243+0000 7fb83ca00640 1 --2- 172.21.15.27:0/561581906 >> [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546] conn(0x7fb81c0c1a00 0x7fb81c0c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:18.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.244+0000 7fb83ca00640 1 --2- 172.21.15.27:0/561581906 >> [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546] conn(0x7fb81c0c1a00 0x7fb81c0c3ef0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:18.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.245+0000 7fb834c00640 1 -- 172.21.15.27:0/561581906 <== osd.7 v2:172.21.15.89:6816/2479349546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb82400be50 con 0x7fb81c0c1a00 2024-09-21T00:20:18.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.261+0000 7fb83ea00640 1 -- 172.21.15.27:0/561581906 --> [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb838076c40 con 0x7fb81c0c1a00 2024-09-21T00:20:18.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.261+0000 7fb834c00640 1 -- 172.21.15.27:0/561581906 <== osd.7 v2:172.21.15.89:6816/2479349546 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb838076c40 con 0x7fb81c0c1a00 2024-09-21T00:20:18.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.262+0000 7fb83ea00640 1 -- 172.21.15.27:0/561581906 >> [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546] conn(0x7fb81c0c1a00 msgr2=0x7fb81c0c3ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:18.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.262+0000 7fb83ea00640 1 --2- 172.21.15.27:0/561581906 >> [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546] conn(0x7fb81c0c1a00 0x7fb81c0c3ef0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:18.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.262+0000 7fb83ea00640 1 -- 172.21.15.27:0/561581906 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb81c0b6670 msgr2=0x7fb81c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:18.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.262+0000 7fb83ea00640 1 --2- 172.21.15.27:0/561581906 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb81c0b6670 0x7fb81c0b8b80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb8280096c0 tx=0x7fb828005a20 comp rx=0 tx=0).stop 2024-09-21T00:20:18.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.262+0000 7fb83ea00640 1 -- 172.21.15.27:0/561581906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb838076080 msgr2=0x7fb83819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:18.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.262+0000 7fb83ea00640 1 --2- 172.21.15.27:0/561581906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb838076080 0x7fb83819cd40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb83806c9f0 tx=0x7fb82400ee50 comp rx=0 tx=0).stop 2024-09-21T00:20:18.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.262+0000 7fb83ca00640 1 -- 172.21.15.27:0/561581906 reap_dead start 2024-09-21T00:20:18.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.262+0000 7fb83ea00640 1 -- 172.21.15.27:0/561581906 shutdown_connections 2024-09-21T00:20:18.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.262+0000 7fb83ea00640 1 -- 172.21.15.27:0/561581906 >> 172.21.15.27:0/561581906 conn(0x7fb83807b850 msgr2=0x7fb838108ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:18.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.262+0000 7fb83ea00640 1 -- 172.21.15.27:0/561581906 shutdown_connections 2024-09-21T00:20:18.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:18.262+0000 7fb83ea00640 1 -- 172.21.15.27:0/561581906 wait complete. 2024-09-21T00:20:18.975 INFO:teuthology.orchestra.run.smithi027.stdout:304942678050 2024-09-21T00:20:18.976 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T00:20:19.107 INFO:teuthology.orchestra.run.smithi027.stdout:279172874278 2024-09-21T00:20:19.107 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T00:20:19.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:19 smithi089 ceph-mon[34481]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:19.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:19 smithi027 ceph-mon[30745]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:19.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:19 smithi098 ceph-mon[34725]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.224+0000 7f9195200640 1 -- 172.21.15.27:0/2502516128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f919010be40 msgr2=0x7f9190112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.224+0000 7f9195200640 1 --2- 172.21.15.27:0/2502516128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f919010be40 0x7f9190112670 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9188009be0 tx=0x7f918802a6c0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.224+0000 7f9195200640 1 -- 172.21.15.27:0/2502516128 shutdown_connections 2024-09-21T00:20:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.224+0000 7f9195200640 1 --2- 172.21.15.27:0/2502516128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f919010be40 0x7f9190112670 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.224+0000 7f9195200640 1 --2- 172.21.15.27:0/2502516128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f919010b4f0 0x7f919010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.224+0000 7f9195200640 1 --2- 172.21.15.27:0/2502516128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9190105bb0 0x7f9190105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.224+0000 7f9195200640 1 -- 172.21.15.27:0/2502516128 >> 172.21.15.27:0/2502516128 conn(0x7f9190101680 msgr2=0x7f9190103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.225+0000 7f9195200640 1 -- 172.21.15.27:0/2502516128 shutdown_connections 2024-09-21T00:20:20.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.225+0000 7f9195200640 1 -- 172.21.15.27:0/2502516128 wait complete. 2024-09-21T00:20:20.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.227+0000 7f9195200640 1 Processor -- start 2024-09-21T00:20:20.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.227+0000 7f9195200640 1 -- start start 2024-09-21T00:20:20.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.228+0000 7f9195200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9190105bb0 0x7f919019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:20.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.228+0000 7f9195200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f919010b4f0 0x7f919019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:20.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.228+0000 7f9195200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f919010be40 0x7f91901a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:20.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.228+0000 7f9195200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9190115b00 con 0x7f9190105bb0 2024-09-21T00:20:20.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.228+0000 7f9195200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9190115980 con 0x7f919010be40 2024-09-21T00:20:20.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.228+0000 7f9195200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9190115c80 con 0x7f919010b4f0 2024-09-21T00:20:20.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.228+0000 7f918f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9190105bb0 0x7f919019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:20.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.228+0000 7f918fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f919010be40 0x7f91901a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:20.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.228+0000 7f918f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9190105bb0 0x7f919019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33924/0 (socket says 172.21.15.27:33924) 2024-09-21T00:20:20.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.228+0000 7f918f400640 1 -- 172.21.15.27:0/1850330285 learned_addr learned my addr 172.21.15.27:0/1850330285 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:20.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.229+0000 7f918fe00640 1 -- 172.21.15.27:0/1850330285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f919010b4f0 msgr2=0x7f919019ca50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:20:20.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.229+0000 7f918fe00640 1 --2- 172.21.15.27:0/1850330285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f919010b4f0 0x7f919019ca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.229+0000 7f918fe00640 1 -- 172.21.15.27:0/1850330285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9190105bb0 msgr2=0x7f919019c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.229+0000 7f918fe00640 1 --2- 172.21.15.27:0/1850330285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9190105bb0 0x7f919019c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.229+0000 7f918fe00640 1 -- 172.21.15.27:0/1850330285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91901a2420 con 0x7f919010be40 2024-09-21T00:20:20.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.229+0000 7f918fe00640 1 --2- 172.21.15.27:0/1850330285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f919010be40 0x7f91901a1cf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f918802abe0 tx=0x7f9188003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.230+0000 7f918cc00640 1 -- 172.21.15.27:0/1850330285 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188041020 con 0x7f919010be40 2024-09-21T00:20:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.230+0000 7f9195200640 1 -- 172.21.15.27:0/1850330285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91901a26b0 con 0x7f919010be40 2024-09-21T00:20:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.230+0000 7f918cc00640 1 -- 172.21.15.27:0/1850330285 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9188003e10 con 0x7f919010be40 2024-09-21T00:20:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.230+0000 7f918cc00640 1 -- 172.21.15.27:0/1850330285 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188036040 con 0x7f919010be40 2024-09-21T00:20:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.230+0000 7f9195200640 1 -- 172.21.15.27:0/1850330285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91901a8e70 con 0x7f919010be40 2024-09-21T00:20:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.231+0000 7f9195200640 1 -- 172.21.15.27:0/1850330285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9190063b50 con 0x7f919010be40 2024-09-21T00:20:20.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.234+0000 7f918cc00640 1 -- 172.21.15.27:0/1850330285 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f9188044020 con 0x7f919010be40 2024-09-21T00:20:20.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.236+0000 7f918cc00640 1 --2- 172.21.15.27:0/1850330285 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f91740b6670 0x7f91740b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:20.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.236+0000 7f918cc00640 1 -- 172.21.15.27:0/1850330285 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7f91881089b0 con 0x7f919010be40 2024-09-21T00:20:20.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.236+0000 7f918f400640 1 --2- 172.21.15.27:0/1850330285 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f91740b6670 0x7f91740b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:20.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.236+0000 7f918cc00640 1 --2- 172.21.15.27:0/1850330285 >> [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690] conn(0x7f91740c1a00 0x7f91740c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:20.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.236+0000 7f918cc00640 1 -- 172.21.15.27:0/1850330285 --> [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9188047e50 con 0x7f91740c1a00 2024-09-21T00:20:20.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.236+0000 7f918cc00640 1 -- 172.21.15.27:0/1850330285 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f9188108e00 con 0x7f919010be40 2024-09-21T00:20:20.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.237+0000 7f9186a00640 1 --2- 172.21.15.27:0/1850330285 >> [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690] conn(0x7f91740c1a00 0x7f91740c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:20.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.237+0000 7f918f400640 1 --2- 172.21.15.27:0/1850330285 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f91740b6670 0x7f91740b8b80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9180005c80 tx=0x7f9180009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:20.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.237+0000 7f9186a00640 1 --2- 172.21.15.27:0/1850330285 >> [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690] conn(0x7f91740c1a00 0x7f91740c3ef0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:20.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.238+0000 7f918cc00640 1 -- 172.21.15.27:0/1850330285 <== osd.4 v2:172.21.15.89:6808/4001038690 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9188047e50 con 0x7f91740c1a00 2024-09-21T00:20:20.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.254+0000 7f9195200640 1 -- 172.21.15.27:0/1850330285 --> [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9190064000 con 0x7f91740c1a00 2024-09-21T00:20:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.255+0000 7f918cc00640 1 -- 172.21.15.27:0/1850330285 <== osd.4 v2:172.21.15.89:6808/4001038690 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9190064000 con 0x7f91740c1a00 2024-09-21T00:20:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.255+0000 7f9195200640 1 -- 172.21.15.27:0/1850330285 >> [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690] conn(0x7f91740c1a00 msgr2=0x7f91740c3ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.255+0000 7f9195200640 1 --2- 172.21.15.27:0/1850330285 >> [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690] conn(0x7f91740c1a00 0x7f91740c3ef0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.255+0000 7f9195200640 1 -- 172.21.15.27:0/1850330285 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f91740b6670 msgr2=0x7f91740b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.255+0000 7f9195200640 1 --2- 172.21.15.27:0/1850330285 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f91740b6670 0x7f91740b8b80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9180005c80 tx=0x7f9180009260 comp rx=0 tx=0).stop 2024-09-21T00:20:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.255+0000 7f9195200640 1 -- 172.21.15.27:0/1850330285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f919010be40 msgr2=0x7f91901a1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.255+0000 7f9195200640 1 --2- 172.21.15.27:0/1850330285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f919010be40 0x7f91901a1cf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f918802abe0 tx=0x7f9188003c20 comp rx=0 tx=0).stop 2024-09-21T00:20:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.255+0000 7f918fe00640 1 -- 172.21.15.27:0/1850330285 reap_dead start 2024-09-21T00:20:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.256+0000 7f9195200640 1 -- 172.21.15.27:0/1850330285 shutdown_connections 2024-09-21T00:20:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.256+0000 7f9195200640 1 -- 172.21.15.27:0/1850330285 >> 172.21.15.27:0/1850330285 conn(0x7f9190101680 msgr2=0x7f919010ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.256+0000 7f9195200640 1 -- 172.21.15.27:0/1850330285 shutdown_connections 2024-09-21T00:20:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:20.256+0000 7f9195200640 1 -- 172.21.15.27:0/1850330285 wait complete. 2024-09-21T00:20:20.267 INFO:teuthology.orchestra.run.smithi027.stdout:231928234033 2024-09-21T00:20:20.267 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T00:20:20.421 INFO:teuthology.orchestra.run.smithi027.stdout:257698037803 2024-09-21T00:20:20.422 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T00:20:20.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.737+0000 7facbac00640 1 -- 172.21.15.27:0/2404577133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facb410b4f0 msgr2=0x7facb410b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.737+0000 7facbac00640 1 --2- 172.21.15.27:0/2404577133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facb410b4f0 0x7facb410b900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faca0009860 tx=0x7faca002a340 comp rx=0 tx=0).stop 2024-09-21T00:20:20.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.737+0000 7facbac00640 1 -- 172.21.15.27:0/2404577133 shutdown_connections 2024-09-21T00:20:20.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.737+0000 7facbac00640 1 --2- 172.21.15.27:0/2404577133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facb410be40 0x7facb4112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.737+0000 7facbac00640 1 --2- 172.21.15.27:0/2404577133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facb410b4f0 0x7facb410b900 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.737+0000 7facbac00640 1 --2- 172.21.15.27:0/2404577133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facb4105bb0 0x7facb4105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.737+0000 7facbac00640 1 -- 172.21.15.27:0/2404577133 >> 172.21.15.27:0/2404577133 conn(0x7facb4101680 msgr2=0x7facb4103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:20.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.737+0000 7facbac00640 1 -- 172.21.15.27:0/2404577133 shutdown_connections 2024-09-21T00:20:20.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.737+0000 7facbac00640 1 -- 172.21.15.27:0/2404577133 wait complete. 2024-09-21T00:20:20.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.738+0000 7facbac00640 1 Processor -- start 2024-09-21T00:20:20.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.738+0000 7facbac00640 1 -- start start 2024-09-21T00:20:20.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.739+0000 7facbac00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facb4105bb0 0x7facb419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:20.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.739+0000 7facbac00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facb410b4f0 0x7facb419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:20.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.739+0000 7facbac00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facb410be40 0x7facb41a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:20.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.739+0000 7facbac00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7facb4115b00 con 0x7facb410b4f0 2024-09-21T00:20:20.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.739+0000 7facbac00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7facb4115980 con 0x7facb4105bb0 2024-09-21T00:20:20.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.739+0000 7facbac00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7facb4115c80 con 0x7facb410be40 2024-09-21T00:20:20.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.739+0000 7facb9800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facb4105bb0 0x7facb419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:20.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.739+0000 7facb9800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facb4105bb0 0x7facb419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:34626/0 (socket says 172.21.15.27:34626) 2024-09-21T00:20:20.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.739+0000 7facb9800640 1 -- 172.21.15.27:0/1791308201 learned_addr learned my addr 172.21.15.27:0/1791308201 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:20.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.739+0000 7facb8e00640 1 --2- 172.21.15.27:0/1791308201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facb410b4f0 0x7facb419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:20.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.739+0000 7facba200640 1 --2- 172.21.15.27:0/1791308201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facb410be40 0x7facb41a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:20.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.740+0000 7facb9800640 1 -- 172.21.15.27:0/1791308201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facb410be40 msgr2=0x7facb41a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.740+0000 7facb9800640 1 --2- 172.21.15.27:0/1791308201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facb410be40 0x7facb41a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.740+0000 7facb9800640 1 -- 172.21.15.27:0/1791308201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facb410b4f0 msgr2=0x7facb419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.740+0000 7facb9800640 1 --2- 172.21.15.27:0/1791308201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facb410b4f0 0x7facb419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.740+0000 7facb9800640 1 -- 172.21.15.27:0/1791308201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb41a2460 con 0x7facb4105bb0 2024-09-21T00:20:20.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.740+0000 7facb9800640 1 --2- 172.21.15.27:0/1791308201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facb4105bb0 0x7facb419c4f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faca800e9f0 tx=0x7faca800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:20.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.740+0000 7facb2000640 1 -- 172.21.15.27:0/1791308201 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca800cd80 con 0x7facb4105bb0 2024-09-21T00:20:20.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.740+0000 7facbac00640 1 -- 172.21.15.27:0/1791308201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb41a2750 con 0x7facb4105bb0 2024-09-21T00:20:20.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.741+0000 7facb2000640 1 -- 172.21.15.27:0/1791308201 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca80108e0 con 0x7facb4105bb0 2024-09-21T00:20:20.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.741+0000 7facb2000640 1 -- 172.21.15.27:0/1791308201 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca8018980 con 0x7facb4105bb0 2024-09-21T00:20:20.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.741+0000 7facbac00640 1 -- 172.21.15.27:0/1791308201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb41a8eb0 con 0x7facb4105bb0 2024-09-21T00:20:20.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.741+0000 7facbac00640 1 -- 172.21.15.27:0/1791308201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7facb4063b50 con 0x7facb4105bb0 2024-09-21T00:20:20.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.744+0000 7facb2000640 1 -- 172.21.15.27:0/1791308201 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7faca8010430 con 0x7facb4105bb0 2024-09-21T00:20:20.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.746+0000 7facb2000640 1 --2- 172.21.15.27:0/1791308201 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fac9c0b65f0 0x7fac9c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:20.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.746+0000 7facb2000640 1 -- 172.21.15.27:0/1791308201 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7faca80e83b0 con 0x7facb4105bb0 2024-09-21T00:20:20.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.746+0000 7facb8e00640 1 --2- 172.21.15.27:0/1791308201 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fac9c0b65f0 0x7fac9c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:20.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.746+0000 7facb2000640 1 --2- 172.21.15.27:0/1791308201 >> [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562] conn(0x7fac9c0c1980 0x7fac9c0c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:20.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.746+0000 7facb2000640 1 -- 172.21.15.27:0/1791308201 --> [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faca801ce70 con 0x7fac9c0c1980 2024-09-21T00:20:20.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.746+0000 7facb2000640 1 -- 172.21.15.27:0/1791308201 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7faca80e8760 con 0x7facb4105bb0 2024-09-21T00:20:20.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.747+0000 7facba200640 1 --2- 172.21.15.27:0/1791308201 >> [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562] conn(0x7fac9c0c1980 0x7fac9c0c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:20.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.747+0000 7facb8e00640 1 --2- 172.21.15.27:0/1791308201 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fac9c0b65f0 0x7fac9c0b8b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faca0035040 tx=0x7faca0030340 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:20.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.747+0000 7facba200640 1 --2- 172.21.15.27:0/1791308201 >> [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562] conn(0x7fac9c0c1980 0x7fac9c0c3e70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:20.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.748+0000 7facb2000640 1 -- 172.21.15.27:0/1791308201 <== osd.5 v2:172.21.15.98:6808/3029903562 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faca801ce70 con 0x7fac9c0c1980 2024-09-21T00:20:20.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.763+0000 7facbac00640 1 -- 172.21.15.27:0/1791308201 --> [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7facb4063fd0 con 0x7fac9c0c1980 2024-09-21T00:20:20.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.763+0000 7facb2000640 1 -- 172.21.15.27:0/1791308201 <== osd.5 v2:172.21.15.98:6808/3029903562 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7facb4063fd0 con 0x7fac9c0c1980 2024-09-21T00:20:20.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.764+0000 7facbac00640 1 -- 172.21.15.27:0/1791308201 >> [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562] conn(0x7fac9c0c1980 msgr2=0x7fac9c0c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.764+0000 7facbac00640 1 --2- 172.21.15.27:0/1791308201 >> [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562] conn(0x7fac9c0c1980 0x7fac9c0c3e70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.764+0000 7facbac00640 1 -- 172.21.15.27:0/1791308201 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fac9c0b65f0 msgr2=0x7fac9c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.764+0000 7facbac00640 1 --2- 172.21.15.27:0/1791308201 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fac9c0b65f0 0x7fac9c0b8b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faca0035040 tx=0x7faca0030340 comp rx=0 tx=0).stop 2024-09-21T00:20:20.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.764+0000 7facbac00640 1 -- 172.21.15.27:0/1791308201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facb4105bb0 msgr2=0x7facb419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.764+0000 7facbac00640 1 --2- 172.21.15.27:0/1791308201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facb4105bb0 0x7facb419c4f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faca800e9f0 tx=0x7faca800eeb0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.764+0000 7facba200640 1 -- 172.21.15.27:0/1791308201 reap_dead start 2024-09-21T00:20:20.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.764+0000 7facbac00640 1 -- 172.21.15.27:0/1791308201 shutdown_connections 2024-09-21T00:20:20.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.764+0000 7facbac00640 1 -- 172.21.15.27:0/1791308201 >> 172.21.15.27:0/1791308201 conn(0x7facb4101680 msgr2=0x7facb4103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:20.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.764+0000 7facbac00640 1 -- 172.21.15.27:0/1791308201 shutdown_connections 2024-09-21T00:20:20.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:19.764+0000 7facbac00640 1 -- 172.21.15.27:0/1791308201 wait complete. 2024-09-21T00:20:20.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.346+0000 7fac20a00640 1 -- 172.21.15.27:0/1042212041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac1c10be50 msgr2=0x7fac1c112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.346+0000 7fac20a00640 1 --2- 172.21.15.27:0/1042212041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac1c10be50 0x7fac1c112680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fac14009ca0 tx=0x7fac1402a8c0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.346+0000 7fac20a00640 1 -- 172.21.15.27:0/1042212041 shutdown_connections 2024-09-21T00:20:20.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.346+0000 7fac20a00640 1 --2- 172.21.15.27:0/1042212041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac1c10be50 0x7fac1c112680 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.346+0000 7fac20a00640 1 --2- 172.21.15.27:0/1042212041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1c10b500 0x7fac1c10b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.346+0000 7fac20a00640 1 --2- 172.21.15.27:0/1042212041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac1c105bc0 0x7fac1c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.346+0000 7fac20a00640 1 -- 172.21.15.27:0/1042212041 >> 172.21.15.27:0/1042212041 conn(0x7fac1c101630 msgr2=0x7fac1c103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:20.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.346+0000 7fac20a00640 1 -- 172.21.15.27:0/1042212041 shutdown_connections 2024-09-21T00:20:20.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.346+0000 7fac20a00640 1 -- 172.21.15.27:0/1042212041 wait complete. 2024-09-21T00:20:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.347+0000 7fac20a00640 1 Processor -- start 2024-09-21T00:20:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.347+0000 7fac20a00640 1 -- start start 2024-09-21T00:20:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac20a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac1c105bc0 0x7fac1c19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac20a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac1c10b500 0x7fac1c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac20a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1c10be50 0x7fac1c1a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac20a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c115b60 con 0x7fac1c10be50 2024-09-21T00:20:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac20a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c1159e0 con 0x7fac1c10b500 2024-09-21T00:20:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac20a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c115ce0 con 0x7fac1c105bc0 2024-09-21T00:20:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac13e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac1c10b500 0x7fac1c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac1be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1c10be50 0x7fac1c1a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac13e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac1c10b500 0x7fac1c19ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57454/0 (socket says 172.21.15.27:57454) 2024-09-21T00:20:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac13e00640 1 -- 172.21.15.27:0/2651928160 learned_addr learned my addr 172.21.15.27:0/2651928160 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac1b400640 1 --2- 172.21.15.27:0/2651928160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac1c105bc0 0x7fac1c19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac13e00640 1 -- 172.21.15.27:0/2651928160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac1c105bc0 msgr2=0x7fac1c19c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac13e00640 1 --2- 172.21.15.27:0/2651928160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac1c105bc0 0x7fac1c19c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac13e00640 1 -- 172.21.15.27:0/2651928160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1c10be50 msgr2=0x7fac1c1a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac13e00640 1 --2- 172.21.15.27:0/2651928160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1c10be50 0x7fac1c1a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.348+0000 7fac13e00640 1 -- 172.21.15.27:0/2651928160 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c1a2400 con 0x7fac1c10b500 2024-09-21T00:20:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.349+0000 7fac13e00640 1 --2- 172.21.15.27:0/2651928160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac1c10b500 0x7fac1c19ca30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fac0400e970 tx=0x7fac0400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.349+0000 7fac18c00640 1 -- 172.21.15.27:0/2651928160 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0400cd80 con 0x7fac1c10b500 2024-09-21T00:20:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.349+0000 7fac20a00640 1 -- 172.21.15.27:0/2651928160 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c1a26f0 con 0x7fac1c10b500 2024-09-21T00:20:20.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.349+0000 7fac18c00640 1 -- 172.21.15.27:0/2651928160 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac040035f0 con 0x7fac1c10b500 2024-09-21T00:20:20.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.349+0000 7fac18c00640 1 -- 172.21.15.27:0/2651928160 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04010690 con 0x7fac1c10b500 2024-09-21T00:20:20.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.349+0000 7fac20a00640 1 -- 172.21.15.27:0/2651928160 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c1a8e50 con 0x7fac1c10b500 2024-09-21T00:20:20.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.350+0000 7fac20a00640 1 -- 172.21.15.27:0/2651928160 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac1c063b50 con 0x7fac1c10b500 2024-09-21T00:20:20.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.353+0000 7fac18c00640 1 -- 172.21.15.27:0/2651928160 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fac04003aa0 con 0x7fac1c10b500 2024-09-21T00:20:20.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.356+0000 7fac18c00640 1 --2- 172.21.15.27:0/2651928160 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fac000b65f0 0x7fac000b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:20.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.356+0000 7fac18c00640 1 -- 172.21.15.27:0/2651928160 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9307+0+0 (secure 0 0 0) 0x7fac040e7e30 con 0x7fac1c10b500 2024-09-21T00:20:20.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.356+0000 7fac1b400640 1 --2- 172.21.15.27:0/2651928160 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fac000b65f0 0x7fac000b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:20.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.356+0000 7fac18c00640 1 --2- 172.21.15.27:0/2651928160 >> [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857] conn(0x7fac000c1980 0x7fac000c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:20.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.356+0000 7fac18c00640 1 -- 172.21.15.27:0/2651928160 --> [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac04015e80 con 0x7fac000c1980 2024-09-21T00:20:20.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.356+0000 7fac18c00640 1 -- 172.21.15.27:0/2651928160 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fac040e81e0 con 0x7fac1c10b500 2024-09-21T00:20:20.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.357+0000 7fac1be00640 1 --2- 172.21.15.27:0/2651928160 >> [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857] conn(0x7fac000c1980 0x7fac000c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:20.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.357+0000 7fac1b400640 1 --2- 172.21.15.27:0/2651928160 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fac000b65f0 0x7fac000b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009bd0 tx=0x7fac0c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:20.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.357+0000 7fac1be00640 1 --2- 172.21.15.27:0/2651928160 >> [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857] conn(0x7fac000c1980 0x7fac000c3e70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:20.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.358+0000 7fac18c00640 1 -- 172.21.15.27:0/2651928160 <== osd.6 v2:172.21.15.27:6818/1286808857 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac04015e80 con 0x7fac000c1980 2024-09-21T00:20:20.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.384+0000 7fac20a00640 1 -- 172.21.15.27:0/2651928160 --> [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fac1c063f50 con 0x7fac000c1980 2024-09-21T00:20:20.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.384+0000 7fac18c00640 1 -- 172.21.15.27:0/2651928160 <== osd.6 v2:172.21.15.27:6818/1286808857 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fac1c063f50 con 0x7fac000c1980 2024-09-21T00:20:20.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.385+0000 7fac20a00640 1 -- 172.21.15.27:0/2651928160 >> [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857] conn(0x7fac000c1980 msgr2=0x7fac000c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.385+0000 7fac20a00640 1 --2- 172.21.15.27:0/2651928160 >> [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857] conn(0x7fac000c1980 0x7fac000c3e70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:20.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.385+0000 7fac20a00640 1 -- 172.21.15.27:0/2651928160 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fac000b65f0 msgr2=0x7fac000b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.385+0000 7fac20a00640 1 --2- 172.21.15.27:0/2651928160 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fac000b65f0 0x7fac000b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009bd0 tx=0x7fac0c005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:20.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.385+0000 7fac20a00640 1 -- 172.21.15.27:0/2651928160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac1c10b500 msgr2=0x7fac1c19ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:20.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.385+0000 7fac20a00640 1 --2- 172.21.15.27:0/2651928160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac1c10b500 0x7fac1c19ca30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fac0400e970 tx=0x7fac0400ee30 comp rx=0 tx=0).stop 2024-09-21T00:20:20.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.385+0000 7fac1be00640 1 -- 172.21.15.27:0/2651928160 reap_dead start 2024-09-21T00:20:20.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.385+0000 7fac20a00640 1 -- 172.21.15.27:0/2651928160 shutdown_connections 2024-09-21T00:20:20.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.385+0000 7fac20a00640 1 -- 172.21.15.27:0/2651928160 >> 172.21.15.27:0/2651928160 conn(0x7fac1c101630 msgr2=0x7fac1c108a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:20.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.385+0000 7fac20a00640 1 -- 172.21.15.27:0/2651928160 shutdown_connections 2024-09-21T00:20:20.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:12.385+0000 7fac20a00640 1 -- 172.21.15.27:0/2651928160 wait complete. 2024-09-21T00:20:21.523 INFO:teuthology.orchestra.run.smithi027.stdout:158913790020 2024-09-21T00:20:21.523 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T00:20:21.713 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:21 smithi027 ceph-mon[30745]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:20:21.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:21 smithi089 ceph-mon[34481]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:20:21.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:21 smithi098 ceph-mon[34725]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:20:22.546 INFO:teuthology.orchestra.run.smithi027.stdout:111669149772 2024-09-21T00:20:22.547 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T00:20:22.891 INFO:teuthology.orchestra.run.smithi027.stdout:81604378708 2024-09-21T00:20:22.891 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T00:20:23.597 INFO:teuthology.orchestra.run.smithi027.stdout:184683593787 2024-09-21T00:20:23.598 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T00:20:23.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:23 smithi089 ceph-mon[34481]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:20:23.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:23 smithi098 ceph-mon[34725]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:20:23.815 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:23 smithi027 ceph-mon[30745]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:20:24.657 INFO:teuthology.orchestra.run.smithi027.stdout:206158430262 2024-09-21T00:20:24.658 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T00:20:25.012 INFO:teuthology.orchestra.run.smithi027.stdout:38654705758 2024-09-21T00:20:25.012 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T00:20:25.633 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:25.633 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:25.634 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:25.638 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:25.686 INFO:teuthology.orchestra.run.smithi027.stdout:137438953545 2024-09-21T00:20:25.687 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T00:20:25.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:25 smithi089 ceph-mon[34481]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:25.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:25 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]} : dispatch 2024-09-21T00:20:25.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:25 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]} : dispatch 2024-09-21T00:20:25.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:25 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]} : dispatch 2024-09-21T00:20:25.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]} : dispatch 2024-09-21T00:20:25.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:25 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]} : dispatch 2024-09-21T00:20:25.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]} : dispatch 2024-09-21T00:20:25.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]} : dispatch 2024-09-21T00:20:25.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:25 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]} : dispatch 2024-09-21T00:20:25.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:25 smithi098 ceph-mon[34725]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:25.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:25 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]} : dispatch 2024-09-21T00:20:25.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:25 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]} : dispatch 2024-09-21T00:20:25.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:25 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]} : dispatch 2024-09-21T00:20:25.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]} : dispatch 2024-09-21T00:20:25.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:25 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]} : dispatch 2024-09-21T00:20:25.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]} : dispatch 2024-09-21T00:20:25.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]} : dispatch 2024-09-21T00:20:25.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:25 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]} : dispatch 2024-09-21T00:20:25.747 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:25.788 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:25.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:25 smithi027 ceph-mon[30745]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:25.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:25 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]} : dispatch 2024-09-21T00:20:25.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:25 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]} : dispatch 2024-09-21T00:20:25.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:25 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]} : dispatch 2024-09-21T00:20:25.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]} : dispatch 2024-09-21T00:20:25.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:25 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]} : dispatch 2024-09-21T00:20:25.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]} : dispatch 2024-09-21T00:20:25.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]} : dispatch 2024-09-21T00:20:25.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:25 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]} : dispatch 2024-09-21T00:20:25.802 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:26.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:26 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-09-21T00:20:26.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:26 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-21T00:20:26.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:26 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]': finished 2024-09-21T00:20:26.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:26 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-09-21T00:20:26.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:26 smithi027 ceph-mon[30745]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T00:20:26.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:26 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-09-21T00:20:26.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:26 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-21T00:20:26.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:26 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]': finished 2024-09-21T00:20:26.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:26 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-09-21T00:20:26.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:26 smithi089 ceph-mon[34481]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T00:20:26.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:26 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-09-21T00:20:26.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:26 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-21T00:20:26.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:26 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]': finished 2024-09-21T00:20:26.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:26 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-09-21T00:20:26.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:26 smithi098 ceph-mon[34725]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T00:20:26.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.713+0000 7f607ae00640 1 -- 172.21.15.27:0/2758780558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60741047c0 msgr2=0x7f6074104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.713+0000 7f607ae00640 1 --2- 172.21.15.27:0/2758780558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60741047c0 0x7f6074104bd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6068009900 tx=0x7f606802a410 comp rx=0 tx=0).stop 2024-09-21T00:20:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.713+0000 7f607ae00640 1 -- 172.21.15.27:0/2758780558 shutdown_connections 2024-09-21T00:20:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.713+0000 7f607ae00640 1 --2- 172.21.15.27:0/2758780558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f607410dc20 0x7f6074110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.713+0000 7f607ae00640 1 --2- 172.21.15.27:0/2758780558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6074105110 0x7f607410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.713+0000 7f607ae00640 1 --2- 172.21.15.27:0/2758780558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60741047c0 0x7f6074104bd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.713+0000 7f607ae00640 1 -- 172.21.15.27:0/2758780558 >> 172.21.15.27:0/2758780558 conn(0x7f60740fe2c0 msgr2=0x7f6074100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.713+0000 7f607ae00640 1 -- 172.21.15.27:0/2758780558 shutdown_connections 2024-09-21T00:20:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.713+0000 7f607ae00640 1 -- 172.21.15.27:0/2758780558 wait complete. 2024-09-21T00:20:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.714+0000 7f607ae00640 1 Processor -- start 2024-09-21T00:20:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.714+0000 7f607ae00640 1 -- start start 2024-09-21T00:20:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.714+0000 7f607ae00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60741047c0 0x7f607419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f607ae00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6074105110 0x7f607419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f607ae00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607410dc20 0x7f60741a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f607ae00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6074113880 con 0x7f60741047c0 2024-09-21T00:20:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f607ae00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6074113700 con 0x7f607410dc20 2024-09-21T00:20:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f607ae00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6074113a00 con 0x7f6074105110 2024-09-21T00:20:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f6079a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60741047c0 0x7f607419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f6079a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60741047c0 0x7f607419c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37962/0 (socket says 172.21.15.27:37962) 2024-09-21T00:20:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f6079a00640 1 -- 172.21.15.27:0/677443595 learned_addr learned my addr 172.21.15.27:0/677443595 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f607a400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607410dc20 0x7f60741a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f6079000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6074105110 0x7f607419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f607a400640 1 -- 172.21.15.27:0/677443595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6074105110 msgr2=0x7f607419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f607a400640 1 --2- 172.21.15.27:0/677443595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6074105110 0x7f607419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f607a400640 1 -- 172.21.15.27:0/677443595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60741047c0 msgr2=0x7f607419c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f607a400640 1 --2- 172.21.15.27:0/677443595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60741047c0 0x7f607419c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f607a400640 1 -- 172.21.15.27:0/677443595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60741a2440 con 0x7f607410dc20 2024-09-21T00:20:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.715+0000 7f6079000640 1 --2- 172.21.15.27:0/677443595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6074105110 0x7f607419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.716+0000 7f607a400640 1 --2- 172.21.15.27:0/677443595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607410dc20 0x7f60741a1d10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f606c009fd0 tx=0x7f606c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.716+0000 7f6072000640 1 -- 172.21.15.27:0/677443595 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c00ce20 con 0x7f607410dc20 2024-09-21T00:20:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.716+0000 7f6072000640 1 -- 172.21.15.27:0/677443595 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f606c005020 con 0x7f607410dc20 2024-09-21T00:20:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.716+0000 7f6072000640 1 -- 172.21.15.27:0/677443595 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c010690 con 0x7f607410dc20 2024-09-21T00:20:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.716+0000 7f607ae00640 1 -- 172.21.15.27:0/677443595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60741a2730 con 0x7f607410dc20 2024-09-21T00:20:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.716+0000 7f607ae00640 1 -- 172.21.15.27:0/677443595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60741a8e90 con 0x7f607410dc20 2024-09-21T00:20:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.717+0000 7f607ae00640 1 -- 172.21.15.27:0/677443595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6074104bd0 con 0x7f607410dc20 2024-09-21T00:20:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.720+0000 7f6072000640 1 -- 172.21.15.27:0/677443595 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f606c0054d0 con 0x7f607410dc20 2024-09-21T00:20:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.722+0000 7f6072000640 1 --2- 172.21.15.27:0/677443595 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f605c0b6670 0x7f605c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.722+0000 7f6072000640 1 -- 172.21.15.27:0/677443595 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f606c0e7ff0 con 0x7f607410dc20 2024-09-21T00:20:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.722+0000 7f6079a00640 1 --2- 172.21.15.27:0/677443595 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f605c0b6670 0x7f605c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.723+0000 7f6079a00640 1 --2- 172.21.15.27:0/677443595 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f605c0b6670 0x7f605c0b8b80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6068031440 tx=0x7f60680058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.724+0000 7f6072000640 1 -- 172.21.15.27:0/677443595 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f606c0ae270 con 0x7f607410dc20 2024-09-21T00:20:26.835 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:26.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.950+0000 7f607ae00640 1 -- 172.21.15.27:0/677443595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f607419d600 con 0x7f607410dc20 2024-09-21T00:20:26.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.951+0000 7f6072000640 1 -- 172.21.15.27:0/677443595 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f606c0b2650 con 0x7f607410dc20 2024-09-21T00:20:26.952 INFO:teuthology.orchestra.run.smithi027.stdout:257698037805 2024-09-21T00:20:26.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.953+0000 7f607ae00640 1 -- 172.21.15.27:0/677443595 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f605c0b6670 msgr2=0x7f605c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:26.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.954+0000 7f607ae00640 1 --2- 172.21.15.27:0/677443595 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f605c0b6670 0x7f605c0b8b80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6068031440 tx=0x7f60680058d0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.954+0000 7f607ae00640 1 -- 172.21.15.27:0/677443595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607410dc20 msgr2=0x7f60741a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:26.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.954+0000 7f607ae00640 1 --2- 172.21.15.27:0/677443595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607410dc20 0x7f60741a1d10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f606c009fd0 tx=0x7f606c00ef10 comp rx=0 tx=0).stop 2024-09-21T00:20:26.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.954+0000 7f607ae00640 1 -- 172.21.15.27:0/677443595 shutdown_connections 2024-09-21T00:20:26.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.954+0000 7f607ae00640 1 --2- 172.21.15.27:0/677443595 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f605c0b6670 0x7f605c0b8b80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.954+0000 7f607ae00640 1 --2- 172.21.15.27:0/677443595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607410dc20 0x7f60741a1d10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.954+0000 7f607ae00640 1 --2- 172.21.15.27:0/677443595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6074105110 0x7f607419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.954+0000 7f607ae00640 1 --2- 172.21.15.27:0/677443595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60741047c0 0x7f607419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.954+0000 7f607ae00640 1 -- 172.21.15.27:0/677443595 >> 172.21.15.27:0/677443595 conn(0x7f60740fe2c0 msgr2=0x7f6074102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:26.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.580+0000 7fed59800640 1 -- 172.21.15.27:0/2668741845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed54103b70 msgr2=0x7fed54103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:26.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.580+0000 7fed59800640 1 --2- 172.21.15.27:0/2668741845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed54103b70 0x7fed54103f80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fed44009900 tx=0x7fed4402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.580+0000 7fed59800640 1 -- 172.21.15.27:0/2668741845 shutdown_connections 2024-09-21T00:20:26.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.580+0000 7fed59800640 1 --2- 172.21.15.27:0/2668741845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed54109e00 0x7fed54110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.580+0000 7fed59800640 1 --2- 172.21.15.27:0/2668741845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed541094b0 0x7fed541098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.580+0000 7fed59800640 1 --2- 172.21.15.27:0/2668741845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed54103b70 0x7fed54103f80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.580+0000 7fed59800640 1 -- 172.21.15.27:0/2668741845 >> 172.21.15.27:0/2668741845 conn(0x7fed540ff5e0 msgr2=0x7fed54101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:26.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.580+0000 7fed59800640 1 -- 172.21.15.27:0/2668741845 shutdown_connections 2024-09-21T00:20:26.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.954+0000 7f607ae00640 1 -- 172.21.15.27:0/677443595 shutdown_connections 2024-09-21T00:20:26.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.954+0000 7f607ae00640 1 -- 172.21.15.27:0/677443595 wait complete. 2024-09-21T00:20:26.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.580+0000 7fed59800640 1 -- 172.21.15.27:0/2668741845 wait complete. 2024-09-21T00:20:26.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.581+0000 7fed59800640 1 Processor -- start 2024-09-21T00:20:26.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.581+0000 7fed59800640 1 -- start start 2024-09-21T00:20:26.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.582+0000 7fed59800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed54103b70 0x7fed5419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:26.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.582+0000 7fed59800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed541094b0 0x7fed5419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:26.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.582+0000 7fed59800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed54109e00 0x7fed5419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:26.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.582+0000 7fed59800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fed54113b10 con 0x7fed541094b0 2024-09-21T00:20:26.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.582+0000 7fed59800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fed54113990 con 0x7fed54109e00 2024-09-21T00:20:26.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.582+0000 7fed59800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed54113c90 con 0x7fed54103b70 2024-09-21T00:20:26.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.582+0000 7fed52000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed541094b0 0x7fed5419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:26.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.582+0000 7fed53400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed54109e00 0x7fed5419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:26.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.582+0000 7fed52a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed54103b70 0x7fed5419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:26.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.582+0000 7fed53400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed54109e00 0x7fed5419fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:52252/0 (socket says 172.21.15.27:52252) 2024-09-21T00:20:26.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.582+0000 7fed53400640 1 -- 172.21.15.27:0/3487880969 learned_addr learned my addr 172.21.15.27:0/3487880969 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:26.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.583+0000 7fed52000640 1 -- 172.21.15.27:0/3487880969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed54103b70 msgr2=0x7fed5419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:26.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.583+0000 7fed52000640 1 --2- 172.21.15.27:0/3487880969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed54103b70 0x7fed5419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.583+0000 7fed52000640 1 -- 172.21.15.27:0/3487880969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed54109e00 msgr2=0x7fed5419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:26.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.583+0000 7fed52000640 1 --2- 172.21.15.27:0/3487880969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed54109e00 0x7fed5419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.583+0000 7fed52000640 1 -- 172.21.15.27:0/3487880969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed541a0430 con 0x7fed541094b0 2024-09-21T00:20:26.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.583+0000 7fed52000640 1 --2- 172.21.15.27:0/3487880969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed541094b0 0x7fed5419aa60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fed5406bd40 tx=0x7fed4000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:26.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.583+0000 7fed4b400640 1 -- 172.21.15.27:0/3487880969 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4000cd00 con 0x7fed541094b0 2024-09-21T00:20:26.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.583+0000 7fed59800640 1 -- 172.21.15.27:0/3487880969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed541a0720 con 0x7fed541094b0 2024-09-21T00:20:26.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.583+0000 7fed4b400640 1 -- 172.21.15.27:0/3487880969 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed400035f0 con 0x7fed541094b0 2024-09-21T00:20:26.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.584+0000 7fed4b400640 1 -- 172.21.15.27:0/3487880969 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40010690 con 0x7fed541094b0 2024-09-21T00:20:26.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.584+0000 7fed59800640 1 -- 172.21.15.27:0/3487880969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed541a6e80 con 0x7fed541094b0 2024-09-21T00:20:26.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.585+0000 7fed59800640 1 -- 172.21.15.27:0/3487880969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed54103f80 con 0x7fed541094b0 2024-09-21T00:20:26.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.587+0000 7fed4b400640 1 -- 172.21.15.27:0/3487880969 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fed400037a0 con 0x7fed541094b0 2024-09-21T00:20:26.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.589+0000 7fed4b400640 1 --2- 172.21.15.27:0/3487880969 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fed380b65f0 0x7fed380b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:26.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.589+0000 7fed4b400640 1 -- 172.21.15.27:0/3487880969 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fed400e7df0 con 0x7fed541094b0 2024-09-21T00:20:26.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.589+0000 7fed52a00640 1 --2- 172.21.15.27:0/3487880969 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fed380b65f0 0x7fed380b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:26.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.590+0000 7fed52a00640 1 --2- 172.21.15.27:0/3487880969 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fed380b65f0 0x7fed380b8b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fed44005ab0 tx=0x7fed44005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:26.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.591+0000 7fed4b400640 1 -- 172.21.15.27:0/3487880969 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fed40056710 con 0x7fed541094b0 2024-09-21T00:20:26.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.795+0000 7fed59800640 1 -- 172.21.15.27:0/3487880969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fed5419b5f0 con 0x7fed541094b0 2024-09-21T00:20:26.959 INFO:teuthology.orchestra.run.smithi027.stdout:231928234036 2024-09-21T00:20:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.795+0000 7fed4b400640 1 -- 172.21.15.27:0/3487880969 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fed400ae010 con 0x7fed541094b0 2024-09-21T00:20:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.797+0000 7fed59800640 1 -- 172.21.15.27:0/3487880969 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fed380b65f0 msgr2=0x7fed380b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.797+0000 7fed59800640 1 --2- 172.21.15.27:0/3487880969 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fed380b65f0 0x7fed380b8b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fed44005ab0 tx=0x7fed44005a20 comp rx=0 tx=0).stop 2024-09-21T00:20:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.797+0000 7fed59800640 1 -- 172.21.15.27:0/3487880969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed541094b0 msgr2=0x7fed5419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.797+0000 7fed59800640 1 --2- 172.21.15.27:0/3487880969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed541094b0 0x7fed5419aa60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fed5406bd40 tx=0x7fed4000ee50 comp rx=0 tx=0).stop 2024-09-21T00:20:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.797+0000 7fed59800640 1 -- 172.21.15.27:0/3487880969 shutdown_connections 2024-09-21T00:20:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.797+0000 7fed59800640 1 --2- 172.21.15.27:0/3487880969 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fed380b65f0 0x7fed380b8b00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.797+0000 7fed59800640 1 --2- 172.21.15.27:0/3487880969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed54109e00 0x7fed5419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.797+0000 7fed59800640 1 --2- 172.21.15.27:0/3487880969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed541094b0 0x7fed5419aa60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.797+0000 7fed59800640 1 --2- 172.21.15.27:0/3487880969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed54103b70 0x7fed5419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.797+0000 7fed59800640 1 -- 172.21.15.27:0/3487880969 >> 172.21.15.27:0/3487880969 conn(0x7fed540ff5e0 msgr2=0x7fed54101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.798+0000 7fed59800640 1 -- 172.21.15.27:0/3487880969 shutdown_connections 2024-09-21T00:20:26.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:26.798+0000 7fed59800640 1 -- 172.21.15.27:0/3487880969 wait complete. 2024-09-21T00:20:27.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:27 smithi089 ceph-mon[34481]: pgmap v348: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T00:20:27.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:27 smithi089 ceph-mon[34481]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-21T00:20:27.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:27 smithi089 ceph-mon[34481]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T00:20:27.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:27 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3487880969' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T00:20:27.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:27 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/677443595' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T00:20:27.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:27 smithi098 ceph-mon[34725]: pgmap v348: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T00:20:27.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:27 smithi098 ceph-mon[34725]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-21T00:20:27.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:27 smithi098 ceph-mon[34725]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T00:20:27.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:27 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3487880969' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T00:20:27.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:27 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/677443595' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T00:20:27.754 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:27 smithi027 ceph-mon[30745]: pgmap v348: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T00:20:27.754 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:27 smithi027 ceph-mon[30745]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-21T00:20:27.754 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:27 smithi027 ceph-mon[30745]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T00:20:27.754 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:27 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3487880969' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T00:20:27.754 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:27 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/677443595' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T00:20:28.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.040+0000 7f83a2400640 1 -- 172.21.15.27:0/1952039054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f839c110440 msgr2=0x7f839c116c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:28.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.040+0000 7f83a2400640 1 --2- 172.21.15.27:0/1952039054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f839c110440 0x7f839c116c70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8394009ca0 tx=0x7f839402a8b0 comp rx=0 tx=0).stop 2024-09-21T00:20:28.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.040+0000 7f83a2400640 1 -- 172.21.15.27:0/1952039054 shutdown_connections 2024-09-21T00:20:28.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.040+0000 7f83a2400640 1 --2- 172.21.15.27:0/1952039054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f839c110440 0x7f839c116c70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:28.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.040+0000 7f83a2400640 1 --2- 172.21.15.27:0/1952039054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f839c10faf0 0x7f839c10ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:28.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.040+0000 7f83a2400640 1 --2- 172.21.15.27:0/1952039054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f839c10a1b0 0x7f839c10a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:28.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.040+0000 7f83a2400640 1 -- 172.21.15.27:0/1952039054 >> 172.21.15.27:0/1952039054 conn(0x7f839c0780e0 msgr2=0x7f839c1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:28.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.040+0000 7f83a2400640 1 -- 172.21.15.27:0/1952039054 shutdown_connections 2024-09-21T00:20:28.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.040+0000 7f83a2400640 1 -- 172.21.15.27:0/1952039054 wait complete. 2024-09-21T00:20:28.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.041+0000 7f83a2400640 1 Processor -- start 2024-09-21T00:20:28.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.042+0000 7f83a2400640 1 -- start start 2024-09-21T00:20:28.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.042+0000 7f83a2400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f839c10a1b0 0x7f839c1a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:28.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.042+0000 7f83a2400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f839c10faf0 0x7f839c1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:28.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.042+0000 7f83a2400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f839c110440 0x7f839c1a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:28.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.042+0000 7f83a2400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f839c11a140 con 0x7f839c10a1b0 2024-09-21T00:20:28.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.042+0000 7f83a2400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f839c119fc0 con 0x7f839c10faf0 2024-09-21T00:20:28.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.042+0000 7f83a2400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f839c11a2c0 con 0x7f839c110440 2024-09-21T00:20:28.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.042+0000 7f839b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f839c10a1b0 0x7f839c1a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:28.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.042+0000 7f839b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f839c10a1b0 0x7f839c1a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37982/0 (socket says 172.21.15.27:37982) 2024-09-21T00:20:28.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.042+0000 7f8392a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f839c10faf0 0x7f839c1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:28.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.042+0000 7f839b400640 1 -- 172.21.15.27:0/117488274 learned_addr learned my addr 172.21.15.27:0/117488274 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:28.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.043+0000 7f839be00640 1 --2- 172.21.15.27:0/117488274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f839c110440 0x7f839c1a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:28.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.043+0000 7f8392a00640 1 -- 172.21.15.27:0/117488274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f839c110440 msgr2=0x7f839c1a6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:28.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.043+0000 7f8392a00640 1 --2- 172.21.15.27:0/117488274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f839c110440 0x7f839c1a6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:28.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.043+0000 7f8392a00640 1 -- 172.21.15.27:0/117488274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f839c10a1b0 msgr2=0x7f839c1a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:28.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.043+0000 7f8392a00640 1 --2- 172.21.15.27:0/117488274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f839c10a1b0 0x7f839c1a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:28.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.043+0000 7f8392a00640 1 -- 172.21.15.27:0/117488274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c1a6a60 con 0x7f839c10faf0 2024-09-21T00:20:28.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.043+0000 7f839be00640 1 --2- 172.21.15.27:0/117488274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f839c110440 0x7f839c1a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:20:28.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.043+0000 7f8392a00640 1 --2- 172.21.15.27:0/117488274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f839c10faf0 0x7f839c1a1090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f839c06ca20 tx=0x7f838400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:28.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.044+0000 7f8398c00640 1 -- 172.21.15.27:0/117488274 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838400cd00 con 0x7f839c10faf0 2024-09-21T00:20:28.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.044+0000 7f83a2400640 1 -- 172.21.15.27:0/117488274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c1a6d50 con 0x7f839c10faf0 2024-09-21T00:20:28.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.044+0000 7f8398c00640 1 -- 172.21.15.27:0/117488274 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83840035f0 con 0x7f839c10faf0 2024-09-21T00:20:28.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.044+0000 7f8398c00640 1 -- 172.21.15.27:0/117488274 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384010690 con 0x7f839c10faf0 2024-09-21T00:20:28.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.044+0000 7f83a2400640 1 -- 172.21.15.27:0/117488274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c1ad4b0 con 0x7f839c10faf0 2024-09-21T00:20:28.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.045+0000 7f83a2400640 1 -- 172.21.15.27:0/117488274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839c10a5c0 con 0x7f839c10faf0 2024-09-21T00:20:28.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.048+0000 7f8398c00640 1 -- 172.21.15.27:0/117488274 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f83840037a0 con 0x7f839c10faf0 2024-09-21T00:20:28.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.049+0000 7f8398c00640 1 --2- 172.21.15.27:0/117488274 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f83800b65f0 0x7f83800b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:28.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.049+0000 7f8398c00640 1 -- 172.21.15.27:0/117488274 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f83840e80b0 con 0x7f839c10faf0 2024-09-21T00:20:28.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.049+0000 7f839b400640 1 --2- 172.21.15.27:0/117488274 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f83800b65f0 0x7f83800b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:28.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.050+0000 7f839b400640 1 --2- 172.21.15.27:0/117488274 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f83800b65f0 0x7f83800b8b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f838c0095f0 tx=0x7f838c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:28.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.051+0000 7f8398c00640 1 -- 172.21.15.27:0/117488274 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f83840ae2d0 con 0x7f839c10faf0 2024-09-21T00:20:28.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.256+0000 7f83a2400640 1 -- 172.21.15.27:0/117488274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f839c1a1c20 con 0x7f839c10faf0 2024-09-21T00:20:28.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.257+0000 7f8398c00640 1 -- 172.21.15.27:0/117488274 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f83840b26b0 con 0x7f839c10faf0 2024-09-21T00:20:28.258 INFO:teuthology.orchestra.run.smithi027.stdout:304942678052 2024-09-21T00:20:28.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.260+0000 7f83a2400640 1 -- 172.21.15.27:0/117488274 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f83800b65f0 msgr2=0x7f83800b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:28.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.260+0000 7f83a2400640 1 --2- 172.21.15.27:0/117488274 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f83800b65f0 0x7f83800b8b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f838c0095f0 tx=0x7f838c005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:28.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.260+0000 7f83a2400640 1 -- 172.21.15.27:0/117488274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f839c10faf0 msgr2=0x7f839c1a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:28.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.260+0000 7f83a2400640 1 --2- 172.21.15.27:0/117488274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f839c10faf0 0x7f839c1a1090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f839c06ca20 tx=0x7f838400ee50 comp rx=0 tx=0).stop 2024-09-21T00:20:28.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.260+0000 7f83a2400640 1 -- 172.21.15.27:0/117488274 shutdown_connections 2024-09-21T00:20:28.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.260+0000 7f83a2400640 1 --2- 172.21.15.27:0/117488274 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f83800b65f0 0x7f83800b8b00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:28.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.260+0000 7f83a2400640 1 --2- 172.21.15.27:0/117488274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f839c110440 0x7f839c1a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:28.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.260+0000 7f83a2400640 1 --2- 172.21.15.27:0/117488274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f839c10faf0 0x7f839c1a1090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:28.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.260+0000 7f83a2400640 1 --2- 172.21.15.27:0/117488274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f839c10a1b0 0x7f839c1a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:28.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.260+0000 7f83a2400640 1 -- 172.21.15.27:0/117488274 >> 172.21.15.27:0/117488274 conn(0x7f839c0780e0 msgr2=0x7f839c078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:28.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.260+0000 7f83a2400640 1 -- 172.21.15.27:0/117488274 shutdown_connections 2024-09-21T00:20:28.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:28.260+0000 7f83a2400640 1 -- 172.21.15.27:0/117488274 wait complete. 2024-09-21T00:20:28.559 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:28 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/117488274' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T00:20:28.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:28 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/117488274' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T00:20:28.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:28 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/117488274' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T00:20:29.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.150+0000 7fbea6c00640 1 -- 172.21.15.27:0/3732088833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea01039b0 msgr2=0x7fbea0103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:29.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.150+0000 7fbea6c00640 1 --2- 172.21.15.27:0/3732088833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea01039b0 0x7fbea0103dc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009900 tx=0x7fbe9402a410 comp rx=0 tx=0).stop 2024-09-21T00:20:29.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.150+0000 7fbea6c00640 1 -- 172.21.15.27:0/3732088833 shutdown_connections 2024-09-21T00:20:29.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.150+0000 7fbea6c00640 1 --2- 172.21.15.27:0/3732088833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbea0109c40 0x7fbea0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:29.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.150+0000 7fbea6c00640 1 --2- 172.21.15.27:0/3732088833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbea01092f0 0x7fbea0109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:29.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.150+0000 7fbea6c00640 1 --2- 172.21.15.27:0/3732088833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea01039b0 0x7fbea0103dc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:29.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.150+0000 7fbea6c00640 1 -- 172.21.15.27:0/3732088833 >> 172.21.15.27:0/3732088833 conn(0x7fbea00ff440 msgr2=0x7fbea0101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:29.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.150+0000 7fbea6c00640 1 -- 172.21.15.27:0/3732088833 shutdown_connections 2024-09-21T00:20:29.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.150+0000 7fbea6c00640 1 -- 172.21.15.27:0/3732088833 wait complete. 2024-09-21T00:20:29.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.152+0000 7fbea6c00640 1 Processor -- start 2024-09-21T00:20:29.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.152+0000 7fbea6c00640 1 -- start start 2024-09-21T00:20:29.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.153+0000 7fbea6c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea01039b0 0x7fbea0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:29.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.153+0000 7fbea6c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbea01092f0 0x7fbea0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:29.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.153+0000 7fbea6c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbea0109c40 0x7fbea0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:29.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.153+0000 7fbea6c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbea0113950 con 0x7fbea01039b0 2024-09-21T00:20:29.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.153+0000 7fbea6c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbea01137d0 con 0x7fbea0109c40 2024-09-21T00:20:29.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.153+0000 7fbea6c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbea0113ad0 con 0x7fbea01092f0 2024-09-21T00:20:29.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.153+0000 7fbea5800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea01039b0 0x7fbea0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:29.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.153+0000 7fbea5800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea01039b0 0x7fbea0075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38000/0 (socket says 172.21.15.27:38000) 2024-09-21T00:20:29.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.153+0000 7fbea5800640 1 -- 172.21.15.27:0/3834054438 learned_addr learned my addr 172.21.15.27:0/3834054438 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:29.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.153+0000 7fbea6200640 1 --2- 172.21.15.27:0/3834054438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbea0109c40 0x7fbea0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:29.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.153+0000 7fbea4e00640 1 --2- 172.21.15.27:0/3834054438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbea01092f0 0x7fbea0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:29.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.154+0000 7fbea5800640 1 -- 172.21.15.27:0/3834054438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbea01092f0 msgr2=0x7fbea0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:29.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.154+0000 7fbea5800640 1 --2- 172.21.15.27:0/3834054438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbea01092f0 0x7fbea0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:29.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.154+0000 7fbea5800640 1 -- 172.21.15.27:0/3834054438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbea0109c40 msgr2=0x7fbea0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:29.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.154+0000 7fbea5800640 1 --2- 172.21.15.27:0/3834054438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbea0109c40 0x7fbea0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:29.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.154+0000 7fbea5800640 1 -- 172.21.15.27:0/3834054438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea01a8620 con 0x7fbea01039b0 2024-09-21T00:20:29.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.154+0000 7fbea6200640 1 --2- 172.21.15.27:0/3834054438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbea0109c40 0x7fbea0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:29.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.154+0000 7fbea5800640 1 --2- 172.21.15.27:0/3834054438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea01039b0 0x7fbea0075710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbe94005ab0 tx=0x7fbe94003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:29.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.155+0000 7fbe9e000640 1 -- 172.21.15.27:0/3834054438 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9403f070 con 0x7fbea01039b0 2024-09-21T00:20:29.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.155+0000 7fbea6c00640 1 -- 172.21.15.27:0/3834054438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea01a88b0 con 0x7fbea01039b0 2024-09-21T00:20:29.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.155+0000 7fbe9e000640 1 -- 172.21.15.27:0/3834054438 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe94031b60 con 0x7fbea01039b0 2024-09-21T00:20:29.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.155+0000 7fbe9e000640 1 -- 172.21.15.27:0/3834054438 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94035030 con 0x7fbea01039b0 2024-09-21T00:20:29.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.155+0000 7fbea6c00640 1 -- 172.21.15.27:0/3834054438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea01a8d90 con 0x7fbea01039b0 2024-09-21T00:20:29.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.156+0000 7fbea6c00640 1 -- 172.21.15.27:0/3834054438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea0103dc0 con 0x7fbea01039b0 2024-09-21T00:20:29.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.160+0000 7fbe9e000640 1 -- 172.21.15.27:0/3834054438 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fbe94042070 con 0x7fbea01039b0 2024-09-21T00:20:29.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.162+0000 7fbe9e000640 1 --2- 172.21.15.27:0/3834054438 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbe880b65f0 0x7fbe880b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:29.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.162+0000 7fbe9e000640 1 -- 172.21.15.27:0/3834054438 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fbe94108420 con 0x7fbea01039b0 2024-09-21T00:20:29.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.162+0000 7fbea4e00640 1 --2- 172.21.15.27:0/3834054438 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbe880b65f0 0x7fbe880b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:29.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.162+0000 7fbe9e000640 1 -- 172.21.15.27:0/3834054438 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbe940d2a20 con 0x7fbea01039b0 2024-09-21T00:20:29.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.163+0000 7fbea4e00640 1 --2- 172.21.15.27:0/3834054438 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbe880b65f0 0x7fbe880b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbea0108d70 tx=0x7fbe8c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:29.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.369+0000 7fbea6c00640 1 -- 172.21.15.27:0/3834054438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fbea002e640 con 0x7fbea01039b0 2024-09-21T00:20:29.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.370+0000 7fbe9e000640 1 -- 172.21.15.27:0/3834054438 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbe940d7960 con 0x7fbea01039b0 2024-09-21T00:20:29.371 INFO:teuthology.orchestra.run.smithi027.stdout:60129542235 2024-09-21T00:20:29.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.372+0000 7fbea6c00640 1 -- 172.21.15.27:0/3834054438 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbe880b65f0 msgr2=0x7fbe880b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:29.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.372+0000 7fbea6c00640 1 --2- 172.21.15.27:0/3834054438 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbe880b65f0 0x7fbe880b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbea0108d70 tx=0x7fbe8c005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:29.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.372+0000 7fbea6c00640 1 -- 172.21.15.27:0/3834054438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea01039b0 msgr2=0x7fbea0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:29.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.372+0000 7fbea6c00640 1 --2- 172.21.15.27:0/3834054438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea01039b0 0x7fbea0075710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbe94005ab0 tx=0x7fbe94003c10 comp rx=0 tx=0).stop 2024-09-21T00:20:29.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.372+0000 7fbea6c00640 1 -- 172.21.15.27:0/3834054438 shutdown_connections 2024-09-21T00:20:29.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.372+0000 7fbea6c00640 1 --2- 172.21.15.27:0/3834054438 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbe880b65f0 0x7fbe880b8b00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:29.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.372+0000 7fbea6c00640 1 --2- 172.21.15.27:0/3834054438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbea0109c40 0x7fbea0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:29.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.372+0000 7fbea6c00640 1 --2- 172.21.15.27:0/3834054438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbea01092f0 0x7fbea0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:29.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.372+0000 7fbea6c00640 1 --2- 172.21.15.27:0/3834054438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea01039b0 0x7fbea0075710 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:29.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.372+0000 7fbea6c00640 1 -- 172.21.15.27:0/3834054438 >> 172.21.15.27:0/3834054438 conn(0x7fbea00ff440 msgr2=0x7fbea0101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:29.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.373+0000 7fbea6c00640 1 -- 172.21.15.27:0/3834054438 shutdown_connections 2024-09-21T00:20:29.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:29.373+0000 7fbea6c00640 1 -- 172.21.15.27:0/3834054438 wait complete. 2024-09-21T00:20:29.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:29 smithi089 ceph-mon[34481]: pgmap v350: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T00:20:29.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:29 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3834054438' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T00:20:29.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:29 smithi098 ceph-mon[34725]: pgmap v350: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T00:20:29.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:29 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3834054438' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T00:20:29.764 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:29.768 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:29.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:29 smithi027 ceph-mon[30745]: pgmap v350: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T00:20:29.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:29 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3834054438' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T00:20:29.918 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037803 got 257698037805 for osd.9 2024-09-21T00:20:29.918 DEBUG:teuthology.parallel:result is None 2024-09-21T00:20:30.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.284+0000 7fc0e9800640 1 -- 172.21.15.27:0/3868401780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0e4110440 msgr2=0x7fc0e4116c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.284+0000 7fc0e9800640 1 --2- 172.21.15.27:0/3868401780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0e4110440 0x7fc0e4116c70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009be0 tx=0x7fc0dc02a6c0 comp rx=0 tx=0).stop 2024-09-21T00:20:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.284+0000 7fc0e9800640 1 -- 172.21.15.27:0/3868401780 shutdown_connections 2024-09-21T00:20:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.284+0000 7fc0e9800640 1 --2- 172.21.15.27:0/3868401780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0e4110440 0x7fc0e4116c70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.284+0000 7fc0e9800640 1 --2- 172.21.15.27:0/3868401780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0e410faf0 0x7fc0e410ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.284+0000 7fc0e9800640 1 --2- 172.21.15.27:0/3868401780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e410a1b0 0x7fc0e410a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.284+0000 7fc0e9800640 1 -- 172.21.15.27:0/3868401780 >> 172.21.15.27:0/3868401780 conn(0x7fc0e40780e0 msgr2=0x7fc0e41081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.284+0000 7fc0e9800640 1 -- 172.21.15.27:0/3868401780 shutdown_connections 2024-09-21T00:20:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.284+0000 7fc0e9800640 1 -- 172.21.15.27:0/3868401780 wait complete. 2024-09-21T00:20:30.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.285+0000 7fc0e9800640 1 Processor -- start 2024-09-21T00:20:30.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.286+0000 7fc0e9800640 1 -- start start 2024-09-21T00:20:30.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.286+0000 7fc0e9800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0e410a1b0 0x7fc0e41a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.286+0000 7fc0e9800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0e410faf0 0x7fc0e41a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.286+0000 7fc0e9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e4110440 0x7fc0e41a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.286+0000 7fc0e9800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e411a0f0 con 0x7fc0e4110440 2024-09-21T00:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.286+0000 7fc0e9800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e4119f70 con 0x7fc0e410a1b0 2024-09-21T00:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.286+0000 7fc0e9800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e411a270 con 0x7fc0e410faf0 2024-09-21T00:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.287+0000 7fc0e3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e4110440 0x7fc0e41a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.287+0000 7fc0e2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0e410a1b0 0x7fc0e41a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.287+0000 7fc0da000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0e410faf0 0x7fc0e41a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.287+0000 7fc0e3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e4110440 0x7fc0e41a6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38016/0 (socket says 172.21.15.27:38016) 2024-09-21T00:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.287+0000 7fc0da000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0e410faf0 0x7fc0e41a1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:57976/0 (socket says 172.21.15.27:57976) 2024-09-21T00:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.287+0000 7fc0e3400640 1 -- 172.21.15.27:0/621645362 learned_addr learned my addr 172.21.15.27:0/621645362 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:30.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.287+0000 7fc0e2a00640 1 -- 172.21.15.27:0/621645362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0e410faf0 msgr2=0x7fc0e41a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:30.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.287+0000 7fc0e2a00640 1 --2- 172.21.15.27:0/621645362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0e410faf0 0x7fc0e41a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:30.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.287+0000 7fc0e2a00640 1 -- 172.21.15.27:0/621645362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e4110440 msgr2=0x7fc0e41a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:30.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.287+0000 7fc0e2a00640 1 --2- 172.21.15.27:0/621645362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e4110440 0x7fc0e41a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:30.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.287+0000 7fc0e2a00640 1 -- 172.21.15.27:0/621645362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e41a6a50 con 0x7fc0e410a1b0 2024-09-21T00:20:30.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.287+0000 7fc0e2a00640 1 --2- 172.21.15.27:0/621645362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0e410a1b0 0x7fc0e41a0ae0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400e9f0 tx=0x7fc0d400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:30.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.288+0000 7fc0dbe00640 1 -- 172.21.15.27:0/621645362 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d400ccc0 con 0x7fc0e410a1b0 2024-09-21T00:20:30.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.288+0000 7fc0dbe00640 1 -- 172.21.15.27:0/621645362 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0d40108e0 con 0x7fc0e410a1b0 2024-09-21T00:20:30.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.288+0000 7fc0e9800640 1 -- 172.21.15.27:0/621645362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e41a6d40 con 0x7fc0e410a1b0 2024-09-21T00:20:30.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.288+0000 7fc0dbe00640 1 -- 172.21.15.27:0/621645362 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4018840 con 0x7fc0e410a1b0 2024-09-21T00:20:30.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.288+0000 7fc0e9800640 1 -- 172.21.15.27:0/621645362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e41ad4a0 con 0x7fc0e410a1b0 2024-09-21T00:20:30.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.289+0000 7fc0e9800640 1 -- 172.21.15.27:0/621645362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e410a5c0 con 0x7fc0e410a1b0 2024-09-21T00:20:30.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.292+0000 7fc0dbe00640 1 -- 172.21.15.27:0/621645362 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fc0d4010430 con 0x7fc0e410a1b0 2024-09-21T00:20:30.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.293+0000 7fc0dbe00640 1 --2- 172.21.15.27:0/621645362 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc0c80b6670 0x7fc0c80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:30.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.293+0000 7fc0dbe00640 1 -- 172.21.15.27:0/621645362 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fc0d40e80c0 con 0x7fc0e410a1b0 2024-09-21T00:20:30.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.293+0000 7fc0da000640 1 --2- 172.21.15.27:0/621645362 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc0c80b6670 0x7fc0c80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:30.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.294+0000 7fc0da000640 1 --2- 172.21.15.27:0/621645362 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc0c80b6670 0x7fc0c80b8b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc0e406b150 tx=0x7fc0cc009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:30.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.295+0000 7fc0dbe00640 1 -- 172.21.15.27:0/621645362 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc0d40ae2e0 con 0x7fc0e410a1b0 2024-09-21T00:20:30.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.497+0000 7fc0e9800640 1 -- 172.21.15.27:0/621645362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc0e41a1c10 con 0x7fc0e410a1b0 2024-09-21T00:20:30.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.497+0000 7fc0dbe00640 1 -- 172.21.15.27:0/621645362 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc0d40b26c0 con 0x7fc0e410a1b0 2024-09-21T00:20:30.499 INFO:teuthology.orchestra.run.smithi027.stdout:81604378711 2024-09-21T00:20:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.500+0000 7fc0e9800640 1 -- 172.21.15.27:0/621645362 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc0c80b6670 msgr2=0x7fc0c80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.500+0000 7fc0e9800640 1 --2- 172.21.15.27:0/621645362 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc0c80b6670 0x7fc0c80b8b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc0e406b150 tx=0x7fc0cc009260 comp rx=0 tx=0).stop 2024-09-21T00:20:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.500+0000 7fc0e9800640 1 -- 172.21.15.27:0/621645362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0e410a1b0 msgr2=0x7fc0e41a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.500+0000 7fc0e9800640 1 --2- 172.21.15.27:0/621645362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0e410a1b0 0x7fc0e41a0ae0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400e9f0 tx=0x7fc0d400edf0 comp rx=0 tx=0).stop 2024-09-21T00:20:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.500+0000 7fc0e9800640 1 -- 172.21.15.27:0/621645362 shutdown_connections 2024-09-21T00:20:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.500+0000 7fc0e9800640 1 --2- 172.21.15.27:0/621645362 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc0c80b6670 0x7fc0c80b8b80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.500+0000 7fc0e9800640 1 --2- 172.21.15.27:0/621645362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e4110440 0x7fc0e41a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.500+0000 7fc0e9800640 1 --2- 172.21.15.27:0/621645362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0e410faf0 0x7fc0e41a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:30.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.500+0000 7fc0e9800640 1 --2- 172.21.15.27:0/621645362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0e410a1b0 0x7fc0e41a0ae0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:30.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.500+0000 7fc0e9800640 1 -- 172.21.15.27:0/621645362 >> 172.21.15.27:0/621645362 conn(0x7fc0e40780e0 msgr2=0x7fc0e4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:30.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.500+0000 7fc0e9800640 1 -- 172.21.15.27:0/621645362 shutdown_connections 2024-09-21T00:20:30.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:30.500+0000 7fc0e9800640 1 -- 172.21.15.27:0/621645362 wait complete. 2024-09-21T00:20:31.194 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542233 got 60129542235 for osd.1 2024-09-21T00:20:31.194 DEBUG:teuthology.parallel:result is None 2024-09-21T00:20:31.720 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:31 smithi027 ceph-mon[30745]: pgmap v351: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T00:20:31.720 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:31 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/621645362' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T00:20:31.722 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234033 got 231928234036 for osd.8 2024-09-21T00:20:31.722 DEBUG:teuthology.parallel:result is None 2024-09-21T00:20:31.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:31 smithi089 ceph-mon[34481]: pgmap v351: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T00:20:31.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:31 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/621645362' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T00:20:31.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:31 smithi098 ceph-mon[34725]: pgmap v351: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T00:20:31.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:31 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/621645362' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T00:20:32.333 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678050 got 304942678052 for osd.11 2024-09-21T00:20:32.333 DEBUG:teuthology.parallel:result is None 2024-09-21T00:20:32.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:32 smithi027 ceph-mon[30745]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-21T00:20:32.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:32 smithi027 ceph-mon[30745]: Cluster is now healthy 2024-09-21T00:20:32.774 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:32 smithi098 ceph-mon[34725]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-21T00:20:32.774 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:32 smithi098 ceph-mon[34725]: Cluster is now healthy 2024-09-21T00:20:32.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.848+0000 7fa3bf400640 1 -- 172.21.15.27:0/4091189175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3b8078290 msgr2=0x7fa3b8114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.848+0000 7fa3bf400640 1 --2- 172.21.15.27:0/4091189175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3b8078290 0x7fa3b8114200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009900 tx=0x7fa3a002a410 comp rx=0 tx=0).stop 2024-09-21T00:20:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.848+0000 7fa3bf400640 1 -- 172.21.15.27:0/4091189175 shutdown_connections 2024-09-21T00:20:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.848+0000 7fa3bf400640 1 --2- 172.21.15.27:0/4091189175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3b8114740 0x7fa3b8116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.848+0000 7fa3bf400640 1 --2- 172.21.15.27:0/4091189175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3b8078290 0x7fa3b8114200 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.848+0000 7fa3bf400640 1 --2- 172.21.15.27:0/4091189175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b8077940 0x7fa3b8077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.848+0000 7fa3bf400640 1 -- 172.21.15.27:0/4091189175 >> 172.21.15.27:0/4091189175 conn(0x7fa3b8101810 msgr2=0x7fa3b8103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.848+0000 7fa3bf400640 1 -- 172.21.15.27:0/4091189175 shutdown_connections 2024-09-21T00:20:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.848+0000 7fa3bf400640 1 -- 172.21.15.27:0/4091189175 wait complete. 2024-09-21T00:20:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.849+0000 7fa3bf400640 1 Processor -- start 2024-09-21T00:20:32.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.850+0000 7fa3bf400640 1 -- start start 2024-09-21T00:20:32.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.850+0000 7fa3bf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b8077940 0x7fa3b81a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:32.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.850+0000 7fa3bf400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3b8078290 0x7fa3b81a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.850+0000 7fa3bf400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3b8114740 0x7fa3b81a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.850+0000 7fa3bf400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b811a070 con 0x7fa3b8077940 2024-09-21T00:20:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.850+0000 7fa3bf400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b8119ef0 con 0x7fa3b8078290 2024-09-21T00:20:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.850+0000 7fa3bf400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b811a1f0 con 0x7fa3b8114740 2024-09-21T00:20:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.850+0000 7fa3bca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b8077940 0x7fa3b81a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.850+0000 7fa3afe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3b8078290 0x7fa3b81a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.850+0000 7fa3bca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b8077940 0x7fa3b81a0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38034/0 (socket says 172.21.15.27:38034) 2024-09-21T00:20:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.850+0000 7fa3bca00640 1 -- 172.21.15.27:0/3012119293 learned_addr learned my addr 172.21.15.27:0/3012119293 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.851+0000 7fa3bd400640 1 --2- 172.21.15.27:0/3012119293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3b8114740 0x7fa3b81a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.851+0000 7fa3afe00640 1 -- 172.21.15.27:0/3012119293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3b8114740 msgr2=0x7fa3b81a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.851+0000 7fa3afe00640 1 --2- 172.21.15.27:0/3012119293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3b8114740 0x7fa3b81a62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.851+0000 7fa3afe00640 1 -- 172.21.15.27:0/3012119293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b8077940 msgr2=0x7fa3b81a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.851+0000 7fa3afe00640 1 --2- 172.21.15.27:0/3012119293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b8077940 0x7fa3b81a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.851+0000 7fa3afe00640 1 -- 172.21.15.27:0/3012119293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b81a69f0 con 0x7fa3b8078290 2024-09-21T00:20:32.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.851+0000 7fa3bca00640 1 --2- 172.21.15.27:0/3012119293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b8077940 0x7fa3b81a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:32.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.851+0000 7fa3afe00640 1 --2- 172.21.15.27:0/3012119293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3b8078290 0x7fa3b81a1020 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa3a002a930 tx=0x7fa3a0004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:32.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.852+0000 7fa3b6000640 1 -- 172.21.15.27:0/3012119293 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a003f070 con 0x7fa3b8078290 2024-09-21T00:20:32.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.852+0000 7fa3b6000640 1 -- 172.21.15.27:0/3012119293 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a0003ca0 con 0x7fa3b8078290 2024-09-21T00:20:32.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.852+0000 7fa3bf400640 1 -- 172.21.15.27:0/3012119293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b81a6c80 con 0x7fa3b8078290 2024-09-21T00:20:32.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.852+0000 7fa3b6000640 1 -- 172.21.15.27:0/3012119293 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0035070 con 0x7fa3b8078290 2024-09-21T00:20:32.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.852+0000 7fa3bf400640 1 -- 172.21.15.27:0/3012119293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b81ad440 con 0x7fa3b8078290 2024-09-21T00:20:32.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.853+0000 7fa3bf400640 1 -- 172.21.15.27:0/3012119293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b8078e50 con 0x7fa3b8078290 2024-09-21T00:20:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.856+0000 7fa3b6000640 1 -- 172.21.15.27:0/3012119293 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fa3a003b460 con 0x7fa3b8078290 2024-09-21T00:20:32.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.857+0000 7fa3b6000640 1 --2- 172.21.15.27:0/3012119293 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa39c0b65f0 0x7fa39c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:32.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.857+0000 7fa3b6000640 1 -- 172.21.15.27:0/3012119293 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fa3a01080b0 con 0x7fa3b8078290 2024-09-21T00:20:32.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.857+0000 7fa3bca00640 1 --2- 172.21.15.27:0/3012119293 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa39c0b65f0 0x7fa39c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:32.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.858+0000 7fa3bca00640 1 --2- 172.21.15.27:0/3012119293 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa39c0b65f0 0x7fa39c0b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8011550 tx=0x7fa3a8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:32.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:32.859+0000 7fa3b6000640 1 -- 172.21.15.27:0/3012119293 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa3a00d26b0 con 0x7fa3b8078290 2024-09-21T00:20:32.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:32 smithi089 ceph-mon[34481]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-21T00:20:32.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:32 smithi089 ceph-mon[34481]: Cluster is now healthy 2024-09-21T00:20:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.056+0000 7f2b4e800640 1 -- 172.21.15.27:0/2037301723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b481094c0 msgr2=0x7f2b481098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.056+0000 7f2b4e800640 1 --2- 172.21.15.27:0/2037301723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b481094c0 0x7f2b481098d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009860 tx=0x7f2b3402a340 comp rx=0 tx=0).stop 2024-09-21T00:20:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.056+0000 7f2b4e800640 1 -- 172.21.15.27:0/2037301723 shutdown_connections 2024-09-21T00:20:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.056+0000 7f2b4e800640 1 --2- 172.21.15.27:0/2037301723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b48109e10 0x7f2b48110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.056+0000 7f2b4e800640 1 --2- 172.21.15.27:0/2037301723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b481094c0 0x7f2b481098d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.056+0000 7f2b4e800640 1 --2- 172.21.15.27:0/2037301723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b48103b80 0x7f2b48103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.056+0000 7f2b4e800640 1 -- 172.21.15.27:0/2037301723 >> 172.21.15.27:0/2037301723 conn(0x7f2b480ff610 msgr2=0x7f2b48101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.056+0000 7f2b4e800640 1 -- 172.21.15.27:0/2037301723 shutdown_connections 2024-09-21T00:20:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.056+0000 7f2b4e800640 1 -- 172.21.15.27:0/2037301723 wait complete. 2024-09-21T00:20:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.058+0000 7f2b4e800640 1 Processor -- start 2024-09-21T00:20:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.058+0000 7f2b4e800640 1 -- start start 2024-09-21T00:20:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.058+0000 7f2b4e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b48103b80 0x7f2b48198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.058+0000 7f2b4e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b481094c0 0x7f2b48199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.058+0000 7f2b4e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b48109e10 0x7f2b4819ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:33.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.058+0000 7f2b4e800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2b48113ad0 con 0x7f2b48109e10 2024-09-21T00:20:33.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.058+0000 7f2b4e800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2b48113950 con 0x7f2b481094c0 2024-09-21T00:20:33.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.058+0000 7f2b4e800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2b48113c50 con 0x7f2b48103b80 2024-09-21T00:20:33.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.058+0000 7f2b47400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b48103b80 0x7f2b48198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:33.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.058+0000 7f2b47400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b48103b80 0x7f2b48198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38910/0 (socket says 172.21.15.27:38910) 2024-09-21T00:20:33.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.058+0000 7f2b47400640 1 -- 172.21.15.27:0/1179173328 learned_addr learned my addr 172.21.15.27:0/1179173328 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:33.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.059+0000 7f2b46a00640 1 --2- 172.21.15.27:0/1179173328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b481094c0 0x7f2b48199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:33.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.059+0000 7f2b47e00640 1 --2- 172.21.15.27:0/1179173328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b48109e10 0x7f2b4819ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:33.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.059+0000 7f2b46a00640 1 -- 172.21.15.27:0/1179173328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b48103b80 msgr2=0x7f2b48198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:33.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.059+0000 7f2b46a00640 1 --2- 172.21.15.27:0/1179173328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b48103b80 0x7f2b48198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.059+0000 7f2b46a00640 1 -- 172.21.15.27:0/1179173328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b48109e10 msgr2=0x7f2b4819ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:33.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.059+0000 7f2b46a00640 1 --2- 172.21.15.27:0/1179173328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b48109e10 0x7f2b4819ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.059+0000 7f2b46a00640 1 -- 172.21.15.27:0/1179173328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b481a04e0 con 0x7f2b481094c0 2024-09-21T00:20:33.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.060+0000 7f2b46a00640 1 --2- 172.21.15.27:0/1179173328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b481094c0 0x7f2b48199480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009780 tx=0x7f2b34032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:33.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.060+0000 7f2b3fe00640 1 -- 172.21.15.27:0/1179173328 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34037040 con 0x7f2b481094c0 2024-09-21T00:20:33.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.060+0000 7f2b3fe00640 1 -- 172.21.15.27:0/1179173328 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b34030c10 con 0x7f2b481094c0 2024-09-21T00:20:33.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.060+0000 7f2b3fe00640 1 -- 172.21.15.27:0/1179173328 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b340035f0 con 0x7f2b481094c0 2024-09-21T00:20:33.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.060+0000 7f2b4e800640 1 -- 172.21.15.27:0/1179173328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b481a0770 con 0x7f2b481094c0 2024-09-21T00:20:33.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.060+0000 7f2b4e800640 1 -- 172.21.15.27:0/1179173328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b481a0bc0 con 0x7f2b481094c0 2024-09-21T00:20:33.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.062+0000 7f2b4e800640 1 -- 172.21.15.27:0/1179173328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b48104e00 con 0x7f2b481094c0 2024-09-21T00:20:33.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.065+0000 7f2b3fe00640 1 -- 172.21.15.27:0/1179173328 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f2b34043020 con 0x7f2b481094c0 2024-09-21T00:20:33.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.066+0000 7f2b3fe00640 1 --2- 172.21.15.27:0/1179173328 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2b2c0b65f0 0x7f2b2c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:33.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.066+0000 7f2b3fe00640 1 -- 172.21.15.27:0/1179173328 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f2b34106e00 con 0x7f2b481094c0 2024-09-21T00:20:33.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.066+0000 7f2b47400640 1 --2- 172.21.15.27:0/1179173328 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2b2c0b65f0 0x7f2b2c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:33.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.067+0000 7f2b47400640 1 --2- 172.21.15.27:0/1179173328 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2b2c0b65f0 0x7f2b2c0b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2b48110750 tx=0x7f2b38005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:33.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.068+0000 7f2b3fe00640 1 -- 172.21.15.27:0/1179173328 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2b340cd130 con 0x7f2b481094c0 2024-09-21T00:20:33.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.068+0000 7fa3bf400640 1 -- 172.21.15.27:0/3012119293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fa3b810f8d0 con 0x7fa3b8078290 2024-09-21T00:20:33.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.069+0000 7fa3b6000640 1 -- 172.21.15.27:0/3012119293 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa3a00d75f0 con 0x7fa3b8078290 2024-09-21T00:20:33.070 INFO:teuthology.orchestra.run.smithi027.stdout:279172874282 2024-09-21T00:20:33.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.072+0000 7fa3bf400640 1 -- 172.21.15.27:0/3012119293 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa39c0b65f0 msgr2=0x7fa39c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:33.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.072+0000 7fa3bf400640 1 --2- 172.21.15.27:0/3012119293 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa39c0b65f0 0x7fa39c0b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8011550 tx=0x7fa3a8005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:33.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.072+0000 7fa3bf400640 1 -- 172.21.15.27:0/3012119293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3b8078290 msgr2=0x7fa3b81a1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:33.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.072+0000 7fa3bf400640 1 --2- 172.21.15.27:0/3012119293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3b8078290 0x7fa3b81a1020 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa3a002a930 tx=0x7fa3a0004060 comp rx=0 tx=0).stop 2024-09-21T00:20:33.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.072+0000 7fa3bf400640 1 -- 172.21.15.27:0/3012119293 shutdown_connections 2024-09-21T00:20:33.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.072+0000 7fa3bf400640 1 --2- 172.21.15.27:0/3012119293 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa39c0b65f0 0x7fa39c0b8b00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.072+0000 7fa3bf400640 1 --2- 172.21.15.27:0/3012119293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3b8114740 0x7fa3b81a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.072+0000 7fa3bf400640 1 --2- 172.21.15.27:0/3012119293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3b8078290 0x7fa3b81a1020 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.072+0000 7fa3bf400640 1 --2- 172.21.15.27:0/3012119293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b8077940 0x7fa3b81a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.072+0000 7fa3bf400640 1 -- 172.21.15.27:0/3012119293 >> 172.21.15.27:0/3012119293 conn(0x7fa3b8101810 msgr2=0x7fa3b8116860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:33.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.072+0000 7fa3bf400640 1 -- 172.21.15.27:0/3012119293 shutdown_connections 2024-09-21T00:20:33.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.072+0000 7fa3bf400640 1 -- 172.21.15.27:0/3012119293 wait complete. 2024-09-21T00:20:33.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.270+0000 7f2b4e800640 1 -- 172.21.15.27:0/1179173328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2b48199ce0 con 0x7f2b481094c0 2024-09-21T00:20:33.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.271+0000 7f2b3fe00640 1 -- 172.21.15.27:0/1179173328 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2b34030dc0 con 0x7f2b481094c0 2024-09-21T00:20:33.273 INFO:teuthology.orchestra.run.smithi027.stdout:184683593791 2024-09-21T00:20:33.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.274+0000 7f2b4e800640 1 -- 172.21.15.27:0/1179173328 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2b2c0b65f0 msgr2=0x7f2b2c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:33.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.274+0000 7f2b4e800640 1 --2- 172.21.15.27:0/1179173328 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2b2c0b65f0 0x7f2b2c0b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2b48110750 tx=0x7f2b38005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:33.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.274+0000 7f2b4e800640 1 -- 172.21.15.27:0/1179173328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b481094c0 msgr2=0x7f2b48199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:33.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.274+0000 7f2b4e800640 1 --2- 172.21.15.27:0/1179173328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b481094c0 0x7f2b48199480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009780 tx=0x7f2b34032d50 comp rx=0 tx=0).stop 2024-09-21T00:20:33.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.274+0000 7f2b4e800640 1 -- 172.21.15.27:0/1179173328 shutdown_connections 2024-09-21T00:20:33.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.274+0000 7f2b4e800640 1 --2- 172.21.15.27:0/1179173328 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2b2c0b65f0 0x7f2b2c0b8b00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.274+0000 7f2b4e800640 1 --2- 172.21.15.27:0/1179173328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b48109e10 0x7f2b4819ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.274+0000 7f2b4e800640 1 --2- 172.21.15.27:0/1179173328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b481094c0 0x7f2b48199480 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.274+0000 7f2b4e800640 1 --2- 172.21.15.27:0/1179173328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b48103b80 0x7f2b48198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.274+0000 7f2b4e800640 1 -- 172.21.15.27:0/1179173328 >> 172.21.15.27:0/1179173328 conn(0x7f2b480ff610 msgr2=0x7f2b480ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:33.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.274+0000 7f2b4e800640 1 -- 172.21.15.27:0/1179173328 shutdown_connections 2024-09-21T00:20:33.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.274+0000 7f2b4e800640 1 -- 172.21.15.27:0/1179173328 wait complete. 2024-09-21T00:20:33.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.460+0000 7f9bb8a00640 1 -- 172.21.15.27:0/1442117006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bb410be50 msgr2=0x7f9bb4112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:33.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.460+0000 7f9bb8a00640 1 --2- 172.21.15.27:0/1442117006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bb410be50 0x7f9bb4112680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9bac009be0 tx=0x7f9bac02a6d0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.461+0000 7f9bb8a00640 1 -- 172.21.15.27:0/1442117006 shutdown_connections 2024-09-21T00:20:33.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.461+0000 7f9bb8a00640 1 --2- 172.21.15.27:0/1442117006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bb410be50 0x7f9bb4112680 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.461+0000 7f9bb8a00640 1 --2- 172.21.15.27:0/1442117006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bb410b500 0x7f9bb410b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.461+0000 7f9bb8a00640 1 --2- 172.21.15.27:0/1442117006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bb4105bc0 0x7f9bb4105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.461+0000 7f9bb8a00640 1 -- 172.21.15.27:0/1442117006 >> 172.21.15.27:0/1442117006 conn(0x7f9bb4101670 msgr2=0x7f9bb4103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:33.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.461+0000 7f9bb8a00640 1 -- 172.21.15.27:0/1442117006 shutdown_connections 2024-09-21T00:20:33.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.461+0000 7f9bb8a00640 1 -- 172.21.15.27:0/1442117006 wait complete. 2024-09-21T00:20:33.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.462+0000 7f9bb8a00640 1 Processor -- start 2024-09-21T00:20:33.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.462+0000 7f9bb8a00640 1 -- start start 2024-09-21T00:20:33.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.462+0000 7f9bb8a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bb4105bc0 0x7f9bb419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:33.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.462+0000 7f9bb8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bb410b500 0x7f9bb419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:33.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.462+0000 7f9bb8a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bb410be50 0x7f9bb41a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:33.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.462+0000 7f9bb8a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb4115b10 con 0x7f9bb410b500 2024-09-21T00:20:33.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.462+0000 7f9bb8a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb4115990 con 0x7f9bb4105bc0 2024-09-21T00:20:33.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.462+0000 7f9bb8a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb4115c90 con 0x7f9bb410be50 2024-09-21T00:20:33.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.463+0000 7f9bb2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bb410b500 0x7f9bb419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:33.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.463+0000 7f9bb2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bb410b500 0x7f9bb419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48490/0 (socket says 172.21.15.27:48490) 2024-09-21T00:20:33.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.463+0000 7f9bb3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bb4105bc0 0x7f9bb419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:33.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.463+0000 7f9bb2a00640 1 -- 172.21.15.27:0/297850573 learned_addr learned my addr 172.21.15.27:0/297850573 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:33.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.463+0000 7f9bb3e00640 1 --2- 172.21.15.27:0/297850573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bb410be50 0x7f9bb41a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:33.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.463+0000 7f9bb3400640 1 -- 172.21.15.27:0/297850573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bb410be50 msgr2=0x7f9bb41a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:33.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.463+0000 7f9bb3400640 1 --2- 172.21.15.27:0/297850573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bb410be50 0x7f9bb41a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.463+0000 7f9bb3400640 1 -- 172.21.15.27:0/297850573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bb410b500 msgr2=0x7f9bb419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:33.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.463+0000 7f9bb3400640 1 --2- 172.21.15.27:0/297850573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bb410b500 0x7f9bb419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.463+0000 7f9bb3400640 1 -- 172.21.15.27:0/297850573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb41a2410 con 0x7f9bb4105bc0 2024-09-21T00:20:33.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.464+0000 7f9bb3400640 1 --2- 172.21.15.27:0/297850573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bb4105bc0 0x7f9bb419c4a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400e9f0 tx=0x7f9ba400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:33.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.464+0000 7f9babe00640 1 -- 172.21.15.27:0/297850573 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba400ccc0 con 0x7f9bb4105bc0 2024-09-21T00:20:33.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.464+0000 7f9bb8a00640 1 -- 172.21.15.27:0/297850573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb41a2700 con 0x7f9bb4105bc0 2024-09-21T00:20:33.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.464+0000 7f9babe00640 1 -- 172.21.15.27:0/297850573 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ba4005020 con 0x7f9bb4105bc0 2024-09-21T00:20:33.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.464+0000 7f9babe00640 1 -- 172.21.15.27:0/297850573 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4004a00 con 0x7f9bb4105bc0 2024-09-21T00:20:33.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.464+0000 7f9bb8a00640 1 -- 172.21.15.27:0/297850573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb41a8e60 con 0x7f9bb4105bc0 2024-09-21T00:20:33.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.465+0000 7f9bb8a00640 1 -- 172.21.15.27:0/297850573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb4105fd0 con 0x7f9bb4105bc0 2024-09-21T00:20:33.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.468+0000 7f9babe00640 1 -- 172.21.15.27:0/297850573 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f9ba4004ba0 con 0x7f9bb4105bc0 2024-09-21T00:20:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.469+0000 7f9babe00640 1 --2- 172.21.15.27:0/297850573 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9b980b6670 0x7f9b980b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.470+0000 7f9babe00640 1 -- 172.21.15.27:0/297850573 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f9ba40e7990 con 0x7f9bb4105bc0 2024-09-21T00:20:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.470+0000 7f9bb2a00640 1 --2- 172.21.15.27:0/297850573 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9b980b6670 0x7f9b980b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.470+0000 7f9bb2a00640 1 --2- 172.21.15.27:0/297850573 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9b980b6670 0x7f9b980b8b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4104e50 tx=0x7f9ba0009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.471+0000 7f9babe00640 1 -- 172.21.15.27:0/297850573 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9ba40adc40 con 0x7f9bb4105bc0 2024-09-21T00:20:33.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:33 smithi027 ceph-mon[30745]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-21T00:20:33.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:33 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3012119293' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T00:20:33.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:33 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1179173328' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T00:20:33.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.675+0000 7f9bb8a00640 1 -- 172.21.15.27:0/297850573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9bb419d5d0 con 0x7f9bb4105bc0 2024-09-21T00:20:33.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.676+0000 7f9babe00640 1 -- 172.21.15.27:0/297850573 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9bb419d5d0 con 0x7f9bb4105bc0 2024-09-21T00:20:33.677 INFO:teuthology.orchestra.run.smithi027.stdout:206158430265 2024-09-21T00:20:33.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.678+0000 7f9bb8a00640 1 -- 172.21.15.27:0/297850573 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9b980b6670 msgr2=0x7f9b980b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:33.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.678+0000 7f9bb8a00640 1 --2- 172.21.15.27:0/297850573 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9b980b6670 0x7f9b980b8b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4104e50 tx=0x7f9ba0009260 comp rx=0 tx=0).stop 2024-09-21T00:20:33.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.678+0000 7f9bb8a00640 1 -- 172.21.15.27:0/297850573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bb4105bc0 msgr2=0x7f9bb419c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:33.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.678+0000 7f9bb8a00640 1 --2- 172.21.15.27:0/297850573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bb4105bc0 0x7f9bb419c4a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400e9f0 tx=0x7f9ba400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.678+0000 7f9bb8a00640 1 -- 172.21.15.27:0/297850573 shutdown_connections 2024-09-21T00:20:33.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.678+0000 7f9bb8a00640 1 --2- 172.21.15.27:0/297850573 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9b980b6670 0x7f9b980b8b80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.678+0000 7f9bb8a00640 1 --2- 172.21.15.27:0/297850573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bb410be50 0x7f9bb41a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.678+0000 7f9bb8a00640 1 --2- 172.21.15.27:0/297850573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bb410b500 0x7f9bb419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.678+0000 7f9bb8a00640 1 --2- 172.21.15.27:0/297850573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bb4105bc0 0x7f9bb419c4a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:33.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.678+0000 7f9bb8a00640 1 -- 172.21.15.27:0/297850573 >> 172.21.15.27:0/297850573 conn(0x7f9bb4101670 msgr2=0x7f9bb4108ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:33.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.678+0000 7f9bb8a00640 1 -- 172.21.15.27:0/297850573 shutdown_connections 2024-09-21T00:20:33.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:33.678+0000 7f9bb8a00640 1 -- 172.21.15.27:0/297850573 wait complete. 2024-09-21T00:20:33.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:33 smithi089 ceph-mon[34481]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-21T00:20:33.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:33 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3012119293' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T00:20:33.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:33 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1179173328' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T00:20:33.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:33 smithi098 ceph-mon[34725]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-21T00:20:33.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:33 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3012119293' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T00:20:33.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:33 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1179173328' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T00:20:34.647 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430262 got 206158430265 for osd.7 2024-09-21T00:20:34.647 DEBUG:teuthology.parallel:result is None 2024-09-21T00:20:34.777 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:34 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/297850573' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T00:20:34.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:34 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/297850573' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T00:20:34.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:34 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/297850573' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T00:20:35.265 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378708 got 81604378711 for osd.2 2024-09-21T00:20:35.266 DEBUG:teuthology.parallel:result is None 2024-09-21T00:20:35.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.813+0000 7f6594c00640 1 -- 172.21.15.27:0/1025240831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6590109c40 msgr2=0x7f6590110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:35.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.813+0000 7f6594c00640 1 --2- 172.21.15.27:0/1025240831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6590109c40 0x7f6590110470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6588009ca0 tx=0x7f658802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.813+0000 7f6594c00640 1 -- 172.21.15.27:0/1025240831 shutdown_connections 2024-09-21T00:20:35.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.813+0000 7f6594c00640 1 --2- 172.21.15.27:0/1025240831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6590109c40 0x7f6590110470 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.813+0000 7f6594c00640 1 --2- 172.21.15.27:0/1025240831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65901092f0 0x7f6590109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.813+0000 7f6594c00640 1 --2- 172.21.15.27:0/1025240831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65901039b0 0x7f6590103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.813+0000 7f6594c00640 1 -- 172.21.15.27:0/1025240831 >> 172.21.15.27:0/1025240831 conn(0x7f65900ff440 msgr2=0x7f6590101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:35.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.814+0000 7f6594c00640 1 -- 172.21.15.27:0/1025240831 shutdown_connections 2024-09-21T00:20:35.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.814+0000 7f6594c00640 1 -- 172.21.15.27:0/1025240831 wait complete. 2024-09-21T00:20:35.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:35 smithi027 ceph-mon[30745]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-09-21T00:20:35.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.815+0000 7f6594c00640 1 Processor -- start 2024-09-21T00:20:35.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.815+0000 7f6594c00640 1 -- start start 2024-09-21T00:20:35.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.816+0000 7f6594c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65901039b0 0x7f659019c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:35.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.816+0000 7f6594c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65901092f0 0x7f659019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:35.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.816+0000 7f6594c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6590109c40 0x7f65901a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:35.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.816+0000 7f6594c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6590113900 con 0x7f6590109c40 2024-09-21T00:20:35.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.816+0000 7f6594c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6590113780 con 0x7f65901092f0 2024-09-21T00:20:35.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.816+0000 7f6594c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6590113a80 con 0x7f65901039b0 2024-09-21T00:20:35.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.816+0000 7f658f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65901039b0 0x7f659019c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:35.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.816+0000 7f658ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65901092f0 0x7f659019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:35.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.816+0000 7f658f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65901039b0 0x7f659019c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38968/0 (socket says 172.21.15.27:38968) 2024-09-21T00:20:35.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.816+0000 7f658f400640 1 -- 172.21.15.27:0/2766071849 learned_addr learned my addr 172.21.15.27:0/2766071849 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:35.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.816+0000 7f658fe00640 1 --2- 172.21.15.27:0/2766071849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6590109c40 0x7f65901a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:35.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.817+0000 7f658f400640 1 -- 172.21.15.27:0/2766071849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65901092f0 msgr2=0x7f659019ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:35.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.817+0000 7f658f400640 1 --2- 172.21.15.27:0/2766071849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65901092f0 0x7f659019ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.817+0000 7f658f400640 1 -- 172.21.15.27:0/2766071849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6590109c40 msgr2=0x7f65901a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:35.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.817+0000 7f658f400640 1 --2- 172.21.15.27:0/2766071849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6590109c40 0x7f65901a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.817+0000 7f658f400640 1 -- 172.21.15.27:0/2766071849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65901a2440 con 0x7f65901039b0 2024-09-21T00:20:35.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.817+0000 7f658fe00640 1 --2- 172.21.15.27:0/2766071849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6590109c40 0x7f65901a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:20:35.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.817+0000 7f658f400640 1 --2- 172.21.15.27:0/2766071849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65901039b0 0x7f659019c4d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f658000e9f0 tx=0x7f658000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:35.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.817+0000 7f6587e00640 1 -- 172.21.15.27:0/2766071849 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658000cd80 con 0x7f65901039b0 2024-09-21T00:20:35.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.817+0000 7f6594c00640 1 -- 172.21.15.27:0/2766071849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65901a2730 con 0x7f65901039b0 2024-09-21T00:20:35.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.817+0000 7f6587e00640 1 -- 172.21.15.27:0/2766071849 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65800054d0 con 0x7f65901039b0 2024-09-21T00:20:35.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.817+0000 7f6587e00640 1 -- 172.21.15.27:0/2766071849 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580010690 con 0x7f65901039b0 2024-09-21T00:20:35.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.817+0000 7f6594c00640 1 -- 172.21.15.27:0/2766071849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65901a8e90 con 0x7f65901039b0 2024-09-21T00:20:35.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.818+0000 7f6594c00640 1 -- 172.21.15.27:0/2766071849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6590103dc0 con 0x7f65901039b0 2024-09-21T00:20:35.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.821+0000 7f6587e00640 1 -- 172.21.15.27:0/2766071849 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f6580004e30 con 0x7f65901039b0 2024-09-21T00:20:35.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.823+0000 7f6587e00640 1 --2- 172.21.15.27:0/2766071849 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f65740b6670 0x7f65740b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:35.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.823+0000 7f6587e00640 1 -- 172.21.15.27:0/2766071849 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f65800e7ff0 con 0x7f65901039b0 2024-09-21T00:20:35.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.823+0000 7f658ea00640 1 --2- 172.21.15.27:0/2766071849 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f65740b6670 0x7f65740b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:35.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.824+0000 7f658ea00640 1 --2- 172.21.15.27:0/2766071849 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f65740b6670 0x7f65740b8b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6590106dd0 tx=0x7f657c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:35.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.824+0000 7f6587e00640 1 -- 172.21.15.27:0/2766071849 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6580056710 con 0x7f65901039b0 2024-09-21T00:20:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.694+0000 7f0e26000640 1 -- 172.21.15.27:0/3000285176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e20076080 msgr2=0x7f0e2010fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.694+0000 7f0e26000640 1 --2- 172.21.15.27:0/3000285176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e20076080 0x7f0e2010fc00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009900 tx=0x7f0e0c02a410 comp rx=0 tx=0).stop 2024-09-21T00:20:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.694+0000 7f0e26000640 1 -- 172.21.15.27:0/3000285176 shutdown_connections 2024-09-21T00:20:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.694+0000 7f0e26000640 1 --2- 172.21.15.27:0/3000285176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e20110140 0x7f0e20112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.694+0000 7f0e26000640 1 --2- 172.21.15.27:0/3000285176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e20076080 0x7f0e2010fc00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.694+0000 7f0e26000640 1 --2- 172.21.15.27:0/3000285176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e20075730 0x7f0e20075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.694+0000 7f0e26000640 1 -- 172.21.15.27:0/3000285176 >> 172.21.15.27:0/3000285176 conn(0x7f0e2007b850 msgr2=0x7f0e2007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.695+0000 7f0e26000640 1 -- 172.21.15.27:0/3000285176 shutdown_connections 2024-09-21T00:20:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.695+0000 7f0e26000640 1 -- 172.21.15.27:0/3000285176 wait complete. 2024-09-21T00:20:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.696+0000 7f0e26000640 1 Processor -- start 2024-09-21T00:20:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.696+0000 7f0e26000640 1 -- start start 2024-09-21T00:20:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.696+0000 7f0e26000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e20075730 0x7f0e2019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.696+0000 7f0e26000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e20076080 0x7f0e2019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.696+0000 7f0e26000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e20110140 0x7f0e201a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.696+0000 7f0e26000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20115cd0 con 0x7f0e20110140 2024-09-21T00:20:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.696+0000 7f0e26000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20115b50 con 0x7f0e20076080 2024-09-21T00:20:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.696+0000 7f0e26000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20115e50 con 0x7f0e20075730 2024-09-21T00:20:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.697+0000 7f0e1ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e20076080 0x7f0e2019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.697+0000 7f0e1fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e20110140 0x7f0e201a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.697+0000 7f0e1f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e20075730 0x7f0e2019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.697+0000 7f0e1ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e20076080 0x7f0e2019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:48978/0 (socket says 172.21.15.27:48978) 2024-09-21T00:20:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.697+0000 7f0e1ea00640 1 -- 172.21.15.27:0/3815656309 learned_addr learned my addr 172.21.15.27:0/3815656309 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.697+0000 7f0e1f400640 1 -- 172.21.15.27:0/3815656309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e20076080 msgr2=0x7f0e2019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.697+0000 7f0e1f400640 1 --2- 172.21.15.27:0/3815656309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e20076080 0x7f0e2019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.697+0000 7f0e1f400640 1 -- 172.21.15.27:0/3815656309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e20110140 msgr2=0x7f0e201a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.697+0000 7f0e1f400640 1 --2- 172.21.15.27:0/3815656309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e20110140 0x7f0e201a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.697+0000 7f0e1f400640 1 -- 172.21.15.27:0/3815656309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e201a25f0 con 0x7f0e20075730 2024-09-21T00:20:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.698+0000 7f0e1f400640 1 --2- 172.21.15.27:0/3815656309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e20075730 0x7f0e2019c680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000e9f0 tx=0x7f0e1000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.698+0000 7f0e17e00640 1 -- 172.21.15.27:0/3815656309 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1000ccc0 con 0x7f0e20075730 2024-09-21T00:20:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.698+0000 7f0e26000640 1 -- 172.21.15.27:0/3815656309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e201a28e0 con 0x7f0e20075730 2024-09-21T00:20:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.698+0000 7f0e17e00640 1 -- 172.21.15.27:0/3815656309 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e10005020 con 0x7f0e20075730 2024-09-21T00:20:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.698+0000 7f0e17e00640 1 -- 172.21.15.27:0/3815656309 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10004a00 con 0x7f0e20075730 2024-09-21T00:20:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.698+0000 7f0e26000640 1 -- 172.21.15.27:0/3815656309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e201a9040 con 0x7f0e20075730 2024-09-21T00:20:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.699+0000 7f0e26000640 1 -- 172.21.15.27:0/3815656309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e20075b40 con 0x7f0e20075730 2024-09-21T00:20:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.702+0000 7f0e17e00640 1 -- 172.21.15.27:0/3815656309 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f0e10004ba0 con 0x7f0e20075730 2024-09-21T00:20:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.703+0000 7f0e17e00640 1 --2- 172.21.15.27:0/3815656309 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0e040b65f0 0x7f0e040b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.703+0000 7f0e17e00640 1 -- 172.21.15.27:0/3815656309 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f0e100e7a40 con 0x7f0e20075730 2024-09-21T00:20:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.703+0000 7f0e1ea00640 1 --2- 172.21.15.27:0/3815656309 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0e040b65f0 0x7f0e040b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.704+0000 7f0e1ea00640 1 --2- 172.21.15.27:0/3815656309 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0e040b65f0 0x7f0e040b8b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c006010 tx=0x7f0e0c005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.705+0000 7f0e17e00640 1 -- 172.21.15.27:0/3815656309 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0e100adcf0 con 0x7f0e20075730 2024-09-21T00:20:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:35.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.924+0000 7f0e26000640 1 -- 172.21.15.27:0/3815656309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0e2019d7b0 con 0x7f0e20075730 2024-09-21T00:20:35.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.925+0000 7f0e17e00640 1 -- 172.21.15.27:0/3815656309 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0e100b20d0 con 0x7f0e20075730 2024-09-21T00:20:35.926 INFO:teuthology.orchestra.run.smithi027.stdout:158913790023 2024-09-21T00:20:35.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.928+0000 7f0e26000640 1 -- 172.21.15.27:0/3815656309 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0e040b65f0 msgr2=0x7f0e040b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:35.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.928+0000 7f0e26000640 1 --2- 172.21.15.27:0/3815656309 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0e040b65f0 0x7f0e040b8b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c006010 tx=0x7f0e0c005a20 comp rx=0 tx=0).stop 2024-09-21T00:20:35.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.928+0000 7f0e26000640 1 -- 172.21.15.27:0/3815656309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e20075730 msgr2=0x7f0e2019c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:35.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.928+0000 7f0e26000640 1 --2- 172.21.15.27:0/3815656309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e20075730 0x7f0e2019c680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000e9f0 tx=0x7f0e1000eeb0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.929+0000 7f0e26000640 1 -- 172.21.15.27:0/3815656309 shutdown_connections 2024-09-21T00:20:35.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.929+0000 7f0e26000640 1 --2- 172.21.15.27:0/3815656309 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0e040b65f0 0x7f0e040b8b00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.929+0000 7f0e26000640 1 --2- 172.21.15.27:0/3815656309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e20110140 0x7f0e201a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.929+0000 7f0e26000640 1 --2- 172.21.15.27:0/3815656309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e20076080 0x7f0e2019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.929+0000 7f0e26000640 1 --2- 172.21.15.27:0/3815656309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e20075730 0x7f0e2019c680 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:35.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.929+0000 7f0e26000640 1 -- 172.21.15.27:0/3815656309 >> 172.21.15.27:0/3815656309 conn(0x7f0e2007b850 msgr2=0x7f0e20101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:35.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.929+0000 7f0e26000640 1 -- 172.21.15.27:0/3815656309 shutdown_connections 2024-09-21T00:20:35.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:35.929+0000 7f0e26000640 1 -- 172.21.15.27:0/3815656309 wait complete. 2024-09-21T00:20:35.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:35 smithi089 ceph-mon[34481]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-09-21T00:20:35.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:35 smithi098 ceph-mon[34725]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-09-21T00:20:36.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:36.048+0000 7f6594c00640 1 -- 172.21.15.27:0/2766071849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f659019d600 con 0x7f65901039b0 2024-09-21T00:20:36.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:36.049+0000 7f6587e00640 1 -- 172.21.15.27:0/2766071849 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f65800ae210 con 0x7f65901039b0 2024-09-21T00:20:36.050 INFO:teuthology.orchestra.run.smithi027.stdout:111669149776 2024-09-21T00:20:36.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:36.051+0000 7f6594c00640 1 -- 172.21.15.27:0/2766071849 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f65740b6670 msgr2=0x7f65740b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:36.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:36.051+0000 7f6594c00640 1 --2- 172.21.15.27:0/2766071849 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f65740b6670 0x7f65740b8b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6590106dd0 tx=0x7f657c005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:36.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:36.051+0000 7f6594c00640 1 -- 172.21.15.27:0/2766071849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65901039b0 msgr2=0x7f659019c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:36.051+0000 7f6594c00640 1 --2- 172.21.15.27:0/2766071849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65901039b0 0x7f659019c4d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f658000e9f0 tx=0x7f658000edf0 comp rx=0 tx=0).stop 2024-09-21T00:20:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:36.051+0000 7f6594c00640 1 -- 172.21.15.27:0/2766071849 shutdown_connections 2024-09-21T00:20:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:36.051+0000 7f6594c00640 1 --2- 172.21.15.27:0/2766071849 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f65740b6670 0x7f65740b8b80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:36.051+0000 7f6594c00640 1 --2- 172.21.15.27:0/2766071849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6590109c40 0x7f65901a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:36.051+0000 7f6594c00640 1 --2- 172.21.15.27:0/2766071849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65901092f0 0x7f659019ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:36.051+0000 7f6594c00640 1 --2- 172.21.15.27:0/2766071849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65901039b0 0x7f659019c4d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:36.051+0000 7f6594c00640 1 -- 172.21.15.27:0/2766071849 >> 172.21.15.27:0/2766071849 conn(0x7f65900ff440 msgr2=0x7f659010e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:36.052+0000 7f6594c00640 1 -- 172.21.15.27:0/2766071849 shutdown_connections 2024-09-21T00:20:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:36.052+0000 7f6594c00640 1 -- 172.21.15.27:0/2766071849 wait complete. 2024-09-21T00:20:36.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:36 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3815656309' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T00:20:36.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:36 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2766071849' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T00:20:36.974 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874278 got 279172874282 for osd.10 2024-09-21T00:20:36.974 DEBUG:teuthology.parallel:result is None 2024-09-21T00:20:36.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:36 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3815656309' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T00:20:36.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:36 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2766071849' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T00:20:36.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:36 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3815656309' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T00:20:36.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:36 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2766071849' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T00:20:37.696 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593787 got 184683593791 for osd.6 2024-09-21T00:20:37.696 DEBUG:teuthology.parallel:result is None 2024-09-21T00:20:37.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:37 smithi027 ceph-mon[30745]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T00:20:37.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:37 smithi089 ceph-mon[34481]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T00:20:37.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:37 smithi098 ceph-mon[34725]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T00:20:38.014 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149772 got 111669149776 for osd.3 2024-09-21T00:20:38.014 DEBUG:teuthology.parallel:result is None 2024-09-21T00:20:38.260 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:38.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.289+0000 7f159cc00640 1 -- 172.21.15.27:0/3719804354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1598109c40 msgr2=0x7f1598110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:38.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.289+0000 7f159cc00640 1 --2- 172.21.15.27:0/3719804354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1598109c40 0x7f1598110470 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1590009ca0 tx=0x7f159002a8c0 comp rx=0 tx=0).stop 2024-09-21T00:20:38.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.289+0000 7f159cc00640 1 -- 172.21.15.27:0/3719804354 shutdown_connections 2024-09-21T00:20:38.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.289+0000 7f159cc00640 1 --2- 172.21.15.27:0/3719804354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1598109c40 0x7f1598110470 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:38.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.289+0000 7f159cc00640 1 --2- 172.21.15.27:0/3719804354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15981092f0 0x7f1598109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:38.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.289+0000 7f159cc00640 1 --2- 172.21.15.27:0/3719804354 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15981039b0 0x7f1598103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:38.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.289+0000 7f159cc00640 1 -- 172.21.15.27:0/3719804354 >> 172.21.15.27:0/3719804354 conn(0x7f15980ff440 msgr2=0x7f1598101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:38.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.289+0000 7f159cc00640 1 -- 172.21.15.27:0/3719804354 shutdown_connections 2024-09-21T00:20:38.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.289+0000 7f159cc00640 1 -- 172.21.15.27:0/3719804354 wait complete. 2024-09-21T00:20:38.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.290+0000 7f159cc00640 1 Processor -- start 2024-09-21T00:20:38.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.290+0000 7f159cc00640 1 -- start start 2024-09-21T00:20:38.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.291+0000 7f159cc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15981039b0 0x7f159819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:38.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.291+0000 7f159cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15981092f0 0x7f159819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:38.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.291+0000 7f159cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1598109c40 0x7f15981a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:38.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.291+0000 7f159cc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1598113950 con 0x7f1598109c40 2024-09-21T00:20:38.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.291+0000 7f159cc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f15981137d0 con 0x7f15981092f0 2024-09-21T00:20:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.291+0000 7f159cc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1598113ad0 con 0x7f15981039b0 2024-09-21T00:20:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.291+0000 7f1597400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15981039b0 0x7f159819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.291+0000 7f1596a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15981092f0 0x7f159819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.291+0000 7f1596a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15981092f0 0x7f159819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49028/0 (socket says 172.21.15.27:49028) 2024-09-21T00:20:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.291+0000 7f1596a00640 1 -- 172.21.15.27:0/2431382834 learned_addr learned my addr 172.21.15.27:0/2431382834 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.291+0000 7f1597400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15981039b0 0x7f159819c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38994/0 (socket says 172.21.15.27:38994) 2024-09-21T00:20:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.291+0000 7f1597e00640 1 --2- 172.21.15.27:0/2431382834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1598109c40 0x7f15981a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:38.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.292+0000 7f1596a00640 1 -- 172.21.15.27:0/2431382834 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15981039b0 msgr2=0x7f159819c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:38.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.292+0000 7f1596a00640 1 --2- 172.21.15.27:0/2431382834 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15981039b0 0x7f159819c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:38.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.292+0000 7f1596a00640 1 -- 172.21.15.27:0/2431382834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1598109c40 msgr2=0x7f15981a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:38.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.292+0000 7f1596a00640 1 --2- 172.21.15.27:0/2431382834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1598109c40 0x7f15981a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:38.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.292+0000 7f1596a00640 1 -- 172.21.15.27:0/2431382834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15981a2450 con 0x7f15981092f0 2024-09-21T00:20:38.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.292+0000 7f1596a00640 1 --2- 172.21.15.27:0/2431382834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15981092f0 0x7f159819ca80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f158400e9f0 tx=0x7f158400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:38.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.292+0000 7f158fe00640 1 -- 172.21.15.27:0/2431382834 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158400ccc0 con 0x7f15981092f0 2024-09-21T00:20:38.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.293+0000 7f158fe00640 1 -- 172.21.15.27:0/2431382834 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15840108e0 con 0x7f15981092f0 2024-09-21T00:20:38.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.293+0000 7f158fe00640 1 -- 172.21.15.27:0/2431382834 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584018860 con 0x7f15981092f0 2024-09-21T00:20:38.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.293+0000 7f159cc00640 1 -- 172.21.15.27:0/2431382834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15981a2740 con 0x7f15981092f0 2024-09-21T00:20:38.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.293+0000 7f159cc00640 1 -- 172.21.15.27:0/2431382834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15981a8ea0 con 0x7f15981092f0 2024-09-21T00:20:38.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.294+0000 7f159cc00640 1 -- 172.21.15.27:0/2431382834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1598103dc0 con 0x7f15981092f0 2024-09-21T00:20:38.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.296+0000 7f158fe00640 1 -- 172.21.15.27:0/2431382834 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f1584010430 con 0x7f15981092f0 2024-09-21T00:20:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.298+0000 7f158fe00640 1 --2- 172.21.15.27:0/2431382834 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f157c0b66f0 0x7f157c0b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.298+0000 7f158fe00640 1 -- 172.21.15.27:0/2431382834 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f15840e7c10 con 0x7f15981092f0 2024-09-21T00:20:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.298+0000 7f1597400640 1 --2- 172.21.15.27:0/2431382834 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f157c0b66f0 0x7f157c0b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:38.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.299+0000 7f1597400640 1 --2- 172.21.15.27:0/2431382834 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f157c0b66f0 0x7f157c0b8c00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1588009bd0 tx=0x7f1588005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:38.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.302+0000 7f158fe00640 1 -- 172.21.15.27:0/2431382834 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f15840b2210 con 0x7f15981092f0 2024-09-21T00:20:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.508+0000 7f159cc00640 1 -- 172.21.15.27:0/2431382834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f15981a9180 con 0x7f15981092f0 2024-09-21T00:20:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.509+0000 7f158fe00640 1 -- 172.21.15.27:0/2431382834 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f15840b7150 con 0x7f15981092f0 2024-09-21T00:20:38.510 INFO:teuthology.orchestra.run.smithi027.stdout:38654705762 2024-09-21T00:20:38.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.511+0000 7f159cc00640 1 -- 172.21.15.27:0/2431382834 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f157c0b66f0 msgr2=0x7f157c0b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.511+0000 7f159cc00640 1 --2- 172.21.15.27:0/2431382834 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f157c0b66f0 0x7f157c0b8c00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1588009bd0 tx=0x7f1588005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.511+0000 7f159cc00640 1 -- 172.21.15.27:0/2431382834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15981092f0 msgr2=0x7f159819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.511+0000 7f159cc00640 1 --2- 172.21.15.27:0/2431382834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15981092f0 0x7f159819ca80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f158400e9f0 tx=0x7f158400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:20:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.511+0000 7f159cc00640 1 -- 172.21.15.27:0/2431382834 shutdown_connections 2024-09-21T00:20:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.511+0000 7f159cc00640 1 --2- 172.21.15.27:0/2431382834 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f157c0b66f0 0x7f157c0b8c00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.511+0000 7f159cc00640 1 --2- 172.21.15.27:0/2431382834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1598109c40 0x7f15981a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.511+0000 7f159cc00640 1 --2- 172.21.15.27:0/2431382834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15981092f0 0x7f159819ca80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.511+0000 7f159cc00640 1 --2- 172.21.15.27:0/2431382834 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15981039b0 0x7f159819c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.511+0000 7f159cc00640 1 -- 172.21.15.27:0/2431382834 >> 172.21.15.27:0/2431382834 conn(0x7f15980ff440 msgr2=0x7f1598106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.511+0000 7f159cc00640 1 -- 172.21.15.27:0/2431382834 shutdown_connections 2024-09-21T00:20:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:38.511+0000 7f159cc00640 1 -- 172.21.15.27:0/2431382834 wait complete. 2024-09-21T00:20:38.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:38 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:20:38.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:38 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2431382834' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T00:20:38.834 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790020 got 158913790023 for osd.5 2024-09-21T00:20:38.834 DEBUG:teuthology.parallel:result is None 2024-09-21T00:20:38.895 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:38 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:20:38.896 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:38 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2431382834' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T00:20:38.903 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:38 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:20:38.903 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:38 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2431382834' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T00:20:39.936 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705758 got 38654705762 for osd.0 2024-09-21T00:20:39.937 DEBUG:teuthology.parallel:result is None 2024-09-21T00:20:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.184+0000 7fbd6a400640 1 -- 172.21.15.27:0/2940418977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd64078290 msgr2=0x7fbd64114210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.184+0000 7fbd6a400640 1 --2- 172.21.15.27:0/2940418977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd64078290 0x7fbd64114210 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbd50009900 tx=0x7fbd5002a3f0 comp rx=0 tx=0).stop 2024-09-21T00:20:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.184+0000 7fbd6a400640 1 -- 172.21.15.27:0/2940418977 shutdown_connections 2024-09-21T00:20:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.184+0000 7fbd6a400640 1 --2- 172.21.15.27:0/2940418977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd64114750 0x7fbd64116b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.184+0000 7fbd6a400640 1 --2- 172.21.15.27:0/2940418977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd64078290 0x7fbd64114210 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.184+0000 7fbd6a400640 1 --2- 172.21.15.27:0/2940418977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd64077940 0x7fbd64077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.184+0000 7fbd6a400640 1 -- 172.21.15.27:0/2940418977 >> 172.21.15.27:0/2940418977 conn(0x7fbd64101860 msgr2=0x7fbd64103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.184+0000 7fbd6a400640 1 -- 172.21.15.27:0/2940418977 shutdown_connections 2024-09-21T00:20:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.185+0000 7fbd6a400640 1 -- 172.21.15.27:0/2940418977 wait complete. 2024-09-21T00:20:40.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:39 smithi027 ceph-mon[30745]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:40.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:39 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:39 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:39 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:39 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:39 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:39 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.186+0000 7fbd6a400640 1 Processor -- start 2024-09-21T00:20:40.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.186+0000 7fbd6a400640 1 -- start start 2024-09-21T00:20:40.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.187+0000 7fbd6a400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd64077940 0x7fbd641a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:40.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.187+0000 7fbd6a400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd64078290 0x7fbd641a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:40.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.187+0000 7fbd6a400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd64114750 0x7fbd641a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:40.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.187+0000 7fbd6a400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6411a080 con 0x7fbd64077940 2024-09-21T00:20:40.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.187+0000 7fbd6a400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbd64119f00 con 0x7fbd64078290 2024-09-21T00:20:40.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.187+0000 7fbd6a400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6411a200 con 0x7fbd64114750 2024-09-21T00:20:40.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.187+0000 7fbd63400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd64077940 0x7fbd641a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:40.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.187+0000 7fbd62a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd64078290 0x7fbd641a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.187+0000 7fbd62a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd64078290 0x7fbd641a1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49056/0 (socket says 172.21.15.27:49056) 2024-09-21T00:20:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.187+0000 7fbd62a00640 1 -- 172.21.15.27:0/4232353253 learned_addr learned my addr 172.21.15.27:0/4232353253 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.187+0000 7fbd63e00640 1 --2- 172.21.15.27:0/4232353253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd64114750 0x7fbd641a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.188+0000 7fbd63e00640 1 -- 172.21.15.27:0/4232353253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd64078290 msgr2=0x7fbd641a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.188+0000 7fbd63e00640 1 --2- 172.21.15.27:0/4232353253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd64078290 0x7fbd641a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.188+0000 7fbd63e00640 1 -- 172.21.15.27:0/4232353253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd64077940 msgr2=0x7fbd641a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.188+0000 7fbd63e00640 1 --2- 172.21.15.27:0/4232353253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd64077940 0x7fbd641a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.188+0000 7fbd63e00640 1 -- 172.21.15.27:0/4232353253 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd641a6a20 con 0x7fbd64114750 2024-09-21T00:20:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.188+0000 7fbd62a00640 1 --2- 172.21.15.27:0/4232353253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd64078290 0x7fbd641a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.189+0000 7fbd63e00640 1 --2- 172.21.15.27:0/4232353253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd64114750 0x7fbd641a62f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400d6b0 tx=0x7fbd5400db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:40.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.189+0000 7fbd5be00640 1 -- 172.21.15.27:0/4232353253 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd54004230 con 0x7fbd64114750 2024-09-21T00:20:40.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.189+0000 7fbd6a400640 1 -- 172.21.15.27:0/4232353253 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd641a6d10 con 0x7fbd64114750 2024-09-21T00:20:40.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.189+0000 7fbd5be00640 1 -- 172.21.15.27:0/4232353253 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd540043d0 con 0x7fbd64114750 2024-09-21T00:20:40.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.190+0000 7fbd6a400640 1 -- 172.21.15.27:0/4232353253 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd641ad470 con 0x7fbd64114750 2024-09-21T00:20:40.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.190+0000 7fbd5be00640 1 -- 172.21.15.27:0/4232353253 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd54003de0 con 0x7fbd64114750 2024-09-21T00:20:40.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.191+0000 7fbd6a400640 1 -- 172.21.15.27:0/4232353253 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd440052c0 con 0x7fbd64114750 2024-09-21T00:20:40.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.195+0000 7fbd5be00640 1 -- 172.21.15.27:0/4232353253 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fbd54009820 con 0x7fbd64114750 2024-09-21T00:20:40.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.197+0000 7fbd5be00640 1 --2- 172.21.15.27:0/4232353253 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbd480b65f0 0x7fbd480b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:40.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.198+0000 7fbd5be00640 1 -- 172.21.15.27:0/4232353253 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fbd540e7210 con 0x7fbd64114750 2024-09-21T00:20:40.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.198+0000 7fbd63400640 1 --2- 172.21.15.27:0/4232353253 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbd480b65f0 0x7fbd480b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:40.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.199+0000 7fbd63400640 1 --2- 172.21.15.27:0/4232353253 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbd480b65f0 0x7fbd480b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c005fd0 tx=0x7fbd5c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:40.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.201+0000 7fbd5be00640 1 -- 172.21.15.27:0/4232353253 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbd54017020 con 0x7fbd64114750 2024-09-21T00:20:40.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:39 smithi089 ceph-mon[34481]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:40.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:39 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:39 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:39 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:39 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:39 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:39 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:39 smithi098 ceph-mon[34725]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:40.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:39 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:39 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:39 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:39 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:39 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:39 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:40.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.461+0000 7fbd6a400640 1 -- 172.21.15.27:0/4232353253 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fbd44005880 con 0x7fbd64114750 2024-09-21T00:20:40.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.461+0000 7fbd5be00640 1 -- 172.21.15.27:0/4232353253 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbd540b1870 con 0x7fbd64114750 2024-09-21T00:20:40.462 INFO:teuthology.orchestra.run.smithi027.stdout:137438953549 2024-09-21T00:20:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.464+0000 7fbd6a400640 1 -- 172.21.15.27:0/4232353253 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbd480b65f0 msgr2=0x7fbd480b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.464+0000 7fbd6a400640 1 --2- 172.21.15.27:0/4232353253 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbd480b65f0 0x7fbd480b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c005fd0 tx=0x7fbd5c009500 comp rx=0 tx=0).stop 2024-09-21T00:20:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.464+0000 7fbd6a400640 1 -- 172.21.15.27:0/4232353253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd64114750 msgr2=0x7fbd641a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.464+0000 7fbd6a400640 1 --2- 172.21.15.27:0/4232353253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd64114750 0x7fbd641a62f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400d6b0 tx=0x7fbd5400db70 comp rx=0 tx=0).stop 2024-09-21T00:20:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.464+0000 7fbd6a400640 1 -- 172.21.15.27:0/4232353253 shutdown_connections 2024-09-21T00:20:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.464+0000 7fbd6a400640 1 --2- 172.21.15.27:0/4232353253 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbd480b65f0 0x7fbd480b8b00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.464+0000 7fbd6a400640 1 --2- 172.21.15.27:0/4232353253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd64114750 0x7fbd641a62f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.464+0000 7fbd6a400640 1 --2- 172.21.15.27:0/4232353253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd64078290 0x7fbd641a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.464+0000 7fbd6a400640 1 --2- 172.21.15.27:0/4232353253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd64077940 0x7fbd641a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:40.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.464+0000 7fbd6a400640 1 -- 172.21.15.27:0/4232353253 >> 172.21.15.27:0/4232353253 conn(0x7fbd64101860 msgr2=0x7fbd64116810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:40.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.464+0000 7fbd6a400640 1 -- 172.21.15.27:0/4232353253 shutdown_connections 2024-09-21T00:20:40.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:40.464+0000 7fbd6a400640 1 -- 172.21.15.27:0/4232353253 wait complete. 2024-09-21T00:20:41.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:40 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/4232353253' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T00:20:41.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:40 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/4232353253' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T00:20:41.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:40 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/4232353253' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T00:20:42.016 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953545 got 137438953549 for osd.4 2024-09-21T00:20:42.016 DEBUG:teuthology.parallel:result is None 2024-09-21T00:20:42.016 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T00:20:42.016 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T00:20:42.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:41 smithi089 ceph-mon[34481]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:20:42.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:41 smithi098 ceph-mon[34725]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:20:42.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:41 smithi027 ceph-mon[30745]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:20:42.475 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.109+0000 7fee37600640 1 -- 172.21.15.27:0/3196952324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee30076080 msgr2=0x7fee3010fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.109+0000 7fee37600640 1 --2- 172.21.15.27:0/3196952324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee30076080 0x7fee3010fc00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009900 tx=0x7fee1c02a3f0 comp rx=0 tx=0).stop 2024-09-21T00:20:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.109+0000 7fee37600640 1 -- 172.21.15.27:0/3196952324 shutdown_connections 2024-09-21T00:20:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.109+0000 7fee37600640 1 --2- 172.21.15.27:0/3196952324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee30110140 0x7fee30112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.109+0000 7fee37600640 1 --2- 172.21.15.27:0/3196952324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee30076080 0x7fee3010fc00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.109+0000 7fee37600640 1 --2- 172.21.15.27:0/3196952324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee30075730 0x7fee30075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.109+0000 7fee37600640 1 -- 172.21.15.27:0/3196952324 >> 172.21.15.27:0/3196952324 conn(0x7fee3007b850 msgr2=0x7fee3007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.109+0000 7fee37600640 1 -- 172.21.15.27:0/3196952324 shutdown_connections 2024-09-21T00:20:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.109+0000 7fee37600640 1 -- 172.21.15.27:0/3196952324 wait complete. 2024-09-21T00:20:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.111+0000 7fee37600640 1 Processor -- start 2024-09-21T00:20:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.111+0000 7fee37600640 1 -- start start 2024-09-21T00:20:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.111+0000 7fee37600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee30075730 0x7fee3019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.111+0000 7fee37600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee30076080 0x7fee3019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:43.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.111+0000 7fee37600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee30110140 0x7fee301a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:43.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.111+0000 7fee37600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fee30115c80 con 0x7fee30076080 2024-09-21T00:20:43.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.111+0000 7fee37600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fee30115b00 con 0x7fee30110140 2024-09-21T00:20:43.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.112+0000 7fee37600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fee30115e00 con 0x7fee30075730 2024-09-21T00:20:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.112+0000 7fee2fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee30076080 0x7fee3019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.112+0000 7fee2fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee30076080 0x7fee3019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51330/0 (socket says 172.21.15.27:51330) 2024-09-21T00:20:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.112+0000 7fee2fe00640 1 -- 172.21.15.27:0/1612580675 learned_addr learned my addr 172.21.15.27:0/1612580675 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.112+0000 7fee35600640 1 --2- 172.21.15.27:0/1612580675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee30110140 0x7fee301a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.112+0000 7fee34c00640 1 --2- 172.21.15.27:0/1612580675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee30075730 0x7fee3019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.112+0000 7fee2fe00640 1 -- 172.21.15.27:0/1612580675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee30075730 msgr2=0x7fee3019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.112+0000 7fee2fe00640 1 --2- 172.21.15.27:0/1612580675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee30075730 0x7fee3019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.112+0000 7fee2fe00640 1 -- 172.21.15.27:0/1612580675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee30110140 msgr2=0x7fee301a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.112+0000 7fee2fe00640 1 --2- 172.21.15.27:0/1612580675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee30110140 0x7fee301a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.112+0000 7fee2fe00640 1 -- 172.21.15.27:0/1612580675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee301a25e0 con 0x7fee30076080 2024-09-21T00:20:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.112+0000 7fee35600640 1 --2- 172.21.15.27:0/1612580675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee30110140 0x7fee301a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.113+0000 7fee2fe00640 1 --2- 172.21.15.27:0/1612580675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee30076080 0x7fee3019cc10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fee1c02a910 tx=0x7fee1c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.113+0000 7fee2d600640 1 -- 172.21.15.27:0/1612580675 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c03f030 con 0x7fee30076080 2024-09-21T00:20:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.113+0000 7fee37600640 1 -- 172.21.15.27:0/1612580675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee301a2870 con 0x7fee30076080 2024-09-21T00:20:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.113+0000 7fee2d600640 1 -- 172.21.15.27:0/1612580675 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee1c004490 con 0x7fee30076080 2024-09-21T00:20:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.113+0000 7fee2d600640 1 -- 172.21.15.27:0/1612580675 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c035070 con 0x7fee30076080 2024-09-21T00:20:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.113+0000 7fee37600640 1 -- 172.21.15.27:0/1612580675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee301a9030 con 0x7fee30076080 2024-09-21T00:20:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.115+0000 7fee37600640 1 -- 172.21.15.27:0/1612580675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee30075b40 con 0x7fee30076080 2024-09-21T00:20:43.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.119+0000 7fee2d600640 1 -- 172.21.15.27:0/1612580675 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fee1c042030 con 0x7fee30076080 2024-09-21T00:20:43.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.121+0000 7fee2d600640 1 --2- 172.21.15.27:0/1612580675 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee180b65f0 0x7fee180b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:43.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.121+0000 7fee2d600640 1 -- 172.21.15.27:0/1612580675 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fee1c108110 con 0x7fee30076080 2024-09-21T00:20:43.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.121+0000 7fee34c00640 1 --2- 172.21.15.27:0/1612580675 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee180b65f0 0x7fee180b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:43.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.122+0000 7fee2d600640 1 -- 172.21.15.27:0/1612580675 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fee1c0d2770 con 0x7fee30076080 2024-09-21T00:20:43.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.122+0000 7fee34c00640 1 --2- 172.21.15.27:0/1612580675 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee180b65f0 0x7fee180b8b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fee240095f0 tx=0x7fee24005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:43.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.322+0000 7fee37600640 1 -- 172.21.15.27:0/1612580675 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fee3010b2f0 con 0x7fee180b65f0 2024-09-21T00:20:43.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.328+0000 7fee2d600640 1 -- 172.21.15.27:0/1612580675 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340367 (secure 0 0 0) 0x7fee3010b2f0 con 0x7fee180b65f0 2024-09-21T00:20:43.335 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:20:43.337 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T00:20:43.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.333+0000 7fee37600640 1 -- 172.21.15.27:0/1612580675 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee180b65f0 msgr2=0x7fee180b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:43.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.333+0000 7fee37600640 1 --2- 172.21.15.27:0/1612580675 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee180b65f0 0x7fee180b8b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fee240095f0 tx=0x7fee24005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:43.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.333+0000 7fee37600640 1 -- 172.21.15.27:0/1612580675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee30076080 msgr2=0x7fee3019cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:43.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.333+0000 7fee37600640 1 --2- 172.21.15.27:0/1612580675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee30076080 0x7fee3019cc10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fee1c02a910 tx=0x7fee1c031a20 comp rx=0 tx=0).stop 2024-09-21T00:20:43.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.333+0000 7fee37600640 1 -- 172.21.15.27:0/1612580675 shutdown_connections 2024-09-21T00:20:43.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.333+0000 7fee37600640 1 --2- 172.21.15.27:0/1612580675 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee180b65f0 0x7fee180b8b00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:43.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.333+0000 7fee37600640 1 --2- 172.21.15.27:0/1612580675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee30110140 0x7fee301a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.333+0000 7fee37600640 1 --2- 172.21.15.27:0/1612580675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee30076080 0x7fee3019cc10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.333+0000 7fee37600640 1 --2- 172.21.15.27:0/1612580675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee30075730 0x7fee3019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.333+0000 7fee37600640 1 -- 172.21.15.27:0/1612580675 >> 172.21.15.27:0/1612580675 conn(0x7fee3007b850 msgr2=0x7fee30101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.333+0000 7fee37600640 1 -- 172.21.15.27:0/1612580675 shutdown_connections 2024-09-21T00:20:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:43.334+0000 7fee37600640 1 -- 172.21.15.27:0/1612580675 wait complete. 2024-09-21T00:20:44.086 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-21T00:20:42.362146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660262,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":498,"num_read_kb":349,"num_write":288,"num_write_kb":2270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2955704,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380268,"kb_used_data":14288,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561556,"statfs":{"total":1151940427776,"available":1151551033344,"internally_reserved":0,"allocated":14630912,"data_stored":8381552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003769"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453002+0000","last_change":"2024-09-21T00:18:35.627618+0000","last_active":"2024-09-21T00:20:26.453002+0000","last_peered":"2024-09-21T00:20:26.453002+0000","last_clean":"2024-09-21T00:20:26.453002+0000","last_became_active":"2024-09-21T00:18:35.626475+0000","last_became_peered":"2024-09-21T00:18:35.626475+0000","last_unstale":"2024-09-21T00:20:26.453002+0000","last_undegraded":"2024-09-21T00:20:26.453002+0000","last_fullsized":"2024-09-21T00:20:26.453002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:50.379225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.575104+0000","last_change":"2024-09-21T00:18:38.665891+0000","last_active":"2024-09-21T00:20:26.575104+0000","last_peered":"2024-09-21T00:20:26.575104+0000","last_clean":"2024-09-21T00:20:26.575104+0000","last_became_active":"2024-09-21T00:18:38.665472+0000","last_became_peered":"2024-09-21T00:18:38.665472+0000","last_unstale":"2024-09-21T00:20:26.575104+0000","last_undegraded":"2024-09-21T00:20:26.575104+0000","last_fullsized":"2024-09-21T00:20:26.575104+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:34:32.490828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451354+0000","last_change":"2024-09-21T00:18:48.730206+0000","last_active":"2024-09-21T00:20:26.451354+0000","last_peered":"2024-09-21T00:20:26.451354+0000","last_clean":"2024-09-21T00:20:26.451354+0000","last_became_active":"2024-09-21T00:18:48.729729+0000","last_became_peered":"2024-09-21T00:18:48.729729+0000","last_unstale":"2024-09-21T00:20:26.451354+0000","last_undegraded":"2024-09-21T00:20:26.451354+0000","last_fullsized":"2024-09-21T00:20:26.451354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:08:17.019877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.575433+0000","last_change":"2024-09-21T00:18:35.768554+0000","last_active":"2024-09-21T00:20:26.575433+0000","last_peered":"2024-09-21T00:20:26.575433+0000","last_clean":"2024-09-21T00:20:26.575433+0000","last_became_active":"2024-09-21T00:18:35.767642+0000","last_became_peered":"2024-09-21T00:18:35.767642+0000","last_unstale":"2024-09-21T00:20:26.575433+0000","last_undegraded":"2024-09-21T00:20:26.575433+0000","last_fullsized":"2024-09-21T00:20:26.575433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:10.442974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455757+0000","last_change":"2024-09-21T00:18:38.677997+0000","last_active":"2024-09-21T00:20:26.455757+0000","last_peered":"2024-09-21T00:20:26.455757+0000","last_clean":"2024-09-21T00:20:26.455757+0000","last_became_active":"2024-09-21T00:18:38.676698+0000","last_became_peered":"2024-09-21T00:18:38.676698+0000","last_unstale":"2024-09-21T00:20:26.455757+0000","last_undegraded":"2024-09-21T00:20:26.455757+0000","last_fullsized":"2024-09-21T00:20:26.455757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:11.662765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451608+0000","last_change":"2024-09-21T00:18:48.739563+0000","last_active":"2024-09-21T00:20:26.451608+0000","last_peered":"2024-09-21T00:20:26.451608+0000","last_clean":"2024-09-21T00:20:26.451608+0000","last_became_active":"2024-09-21T00:18:48.739092+0000","last_became_peered":"2024-09-21T00:18:48.739092+0000","last_unstale":"2024-09-21T00:20:26.451608+0000","last_undegraded":"2024-09-21T00:20:26.451608+0000","last_fullsized":"2024-09-21T00:20:26.451608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:44.185556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510905+0000","last_change":"2024-09-21T00:18:35.626742+0000","last_active":"2024-09-21T00:20:26.510905+0000","last_peered":"2024-09-21T00:20:26.510905+0000","last_clean":"2024-09-21T00:20:26.510905+0000","last_became_active":"2024-09-21T00:18:35.626359+0000","last_became_peered":"2024-09-21T00:18:35.626359+0000","last_unstale":"2024-09-21T00:20:26.510905+0000","last_undegraded":"2024-09-21T00:20:26.510905+0000","last_fullsized":"2024-09-21T00:20:26.510905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:38.860184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510679+0000","last_change":"2024-09-21T00:18:38.666003+0000","last_active":"2024-09-21T00:20:26.510679+0000","last_peered":"2024-09-21T00:20:26.510679+0000","last_clean":"2024-09-21T00:20:26.510679+0000","last_became_active":"2024-09-21T00:18:38.665425+0000","last_became_peered":"2024-09-21T00:18:38.665425+0000","last_unstale":"2024-09-21T00:20:26.510679+0000","last_undegraded":"2024-09-21T00:20:26.510679+0000","last_fullsized":"2024-09-21T00:20:26.510679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:14.121864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.576823+0000","last_change":"2024-09-21T00:20:26.576748+0000","last_active":"2024-09-21T00:20:26.576823+0000","last_peered":"2024-09-21T00:20:26.576823+0000","last_clean":"2024-09-21T00:20:26.576823+0000","last_became_active":"2024-09-21T00:20:26.576186+0000","last_became_peered":"2024-09-21T00:20:26.576186+0000","last_unstale":"2024-09-21T00:20:26.576823+0000","last_undegraded":"2024-09-21T00:20:26.576823+0000","last_fullsized":"2024-09-21T00:20:26.576823+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:20.283717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510591+0000","last_change":"2024-09-21T00:18:35.628091+0000","last_active":"2024-09-21T00:20:26.510591+0000","last_peered":"2024-09-21T00:20:26.510591+0000","last_clean":"2024-09-21T00:20:26.510591+0000","last_became_active":"2024-09-21T00:18:35.627654+0000","last_became_peered":"2024-09-21T00:18:35.627654+0000","last_unstale":"2024-09-21T00:20:26.510591+0000","last_undegraded":"2024-09-21T00:20:26.510591+0000","last_fullsized":"2024-09-21T00:20:26.510591+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:05.229036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509097+0000","last_change":"2024-09-21T00:19:25.844116+0000","last_active":"2024-09-21T00:20:26.509097+0000","last_peered":"2024-09-21T00:20:26.509097+0000","last_clean":"2024-09-21T00:20:26.509097+0000","last_became_active":"2024-09-21T00:19:25.842745+0000","last_became_peered":"2024-09-21T00:19:25.842745+0000","last_unstale":"2024-09-21T00:20:26.509097+0000","last_undegraded":"2024-09-21T00:20:26.509097+0000","last_fullsized":"2024-09-21T00:20:26.509097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:42.651251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451746+0000","last_change":"2024-09-21T00:18:48.740106+0000","last_active":"2024-09-21T00:20:26.451746+0000","last_peered":"2024-09-21T00:20:26.451746+0000","last_clean":"2024-09-21T00:20:26.451746+0000","last_became_active":"2024-09-21T00:18:48.739299+0000","last_became_peered":"2024-09-21T00:18:48.739299+0000","last_unstale":"2024-09-21T00:20:26.451746+0000","last_undegraded":"2024-09-21T00:20:26.451746+0000","last_fullsized":"2024-09-21T00:20:26.451746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:59.058333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509916+0000","last_change":"2024-09-21T00:18:38.667331+0000","last_active":"2024-09-21T00:20:26.509916+0000","last_peered":"2024-09-21T00:20:26.509916+0000","last_clean":"2024-09-21T00:20:26.509916+0000","last_became_active":"2024-09-21T00:18:38.666960+0000","last_became_peered":"2024-09-21T00:18:38.666960+0000","last_unstale":"2024-09-21T00:20:26.509916+0000","last_undegraded":"2024-09-21T00:20:26.509916+0000","last_fullsized":"2024-09-21T00:20:26.509916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:57.569460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510258+0000","last_change":"2024-09-21T00:18:35.625570+0000","last_active":"2024-09-21T00:20:26.510258+0000","last_peered":"2024-09-21T00:20:26.510258+0000","last_clean":"2024-09-21T00:20:26.510258+0000","last_became_active":"2024-09-21T00:18:35.625170+0000","last_became_peered":"2024-09-21T00:18:35.625170+0000","last_unstale":"2024-09-21T00:20:26.510258+0000","last_undegraded":"2024-09-21T00:20:26.510258+0000","last_fullsized":"2024-09-21T00:20:26.510258+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:58.614258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452011+0000","last_change":"2024-09-21T00:18:48.727390+0000","last_active":"2024-09-21T00:20:26.452011+0000","last_peered":"2024-09-21T00:20:26.452011+0000","last_clean":"2024-09-21T00:20:26.452011+0000","last_became_active":"2024-09-21T00:18:48.726864+0000","last_became_peered":"2024-09-21T00:18:48.726864+0000","last_unstale":"2024-09-21T00:20:26.452011+0000","last_undegraded":"2024-09-21T00:20:26.452011+0000","last_fullsized":"2024-09-21T00:20:26.452011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:10.917069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574789+0000","last_change":"2024-09-21T00:18:38.664072+0000","last_active":"2024-09-21T00:20:26.574789+0000","last_peered":"2024-09-21T00:20:26.574789+0000","last_clean":"2024-09-21T00:20:26.574789+0000","last_became_active":"2024-09-21T00:18:38.662562+0000","last_became_peered":"2024-09-21T00:18:38.662562+0000","last_unstale":"2024-09-21T00:20:26.574789+0000","last_undegraded":"2024-09-21T00:20:26.574789+0000","last_fullsized":"2024-09-21T00:20:26.574789+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.021174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453693+0000","last_change":"2024-09-21T00:19:25.843599+0000","last_active":"2024-09-21T00:20:26.453693+0000","last_peered":"2024-09-21T00:20:26.453693+0000","last_clean":"2024-09-21T00:20:26.453693+0000","last_became_active":"2024-09-21T00:19:25.843178+0000","last_became_peered":"2024-09-21T00:19:25.843178+0000","last_unstale":"2024-09-21T00:20:26.453693+0000","last_undegraded":"2024-09-21T00:20:26.453693+0000","last_fullsized":"2024-09-21T00:20:26.453693+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:18.809857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452048+0000","last_change":"2024-09-21T00:18:48.729940+0000","last_active":"2024-09-21T00:20:26.452048+0000","last_peered":"2024-09-21T00:20:26.452048+0000","last_clean":"2024-09-21T00:20:26.452048+0000","last_became_active":"2024-09-21T00:18:48.729465+0000","last_became_peered":"2024-09-21T00:18:48.729465+0000","last_unstale":"2024-09-21T00:20:26.452048+0000","last_undegraded":"2024-09-21T00:20:26.452048+0000","last_fullsized":"2024-09-21T00:20:26.452048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:44.566386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453715+0000","last_change":"2024-09-21T00:18:38.662510+0000","last_active":"2024-09-21T00:20:26.453715+0000","last_peered":"2024-09-21T00:20:26.453715+0000","last_clean":"2024-09-21T00:20:26.453715+0000","last_became_active":"2024-09-21T00:18:38.659788+0000","last_became_peered":"2024-09-21T00:18:38.659788+0000","last_unstale":"2024-09-21T00:20:26.453715+0000","last_undegraded":"2024-09-21T00:20:26.453715+0000","last_fullsized":"2024-09-21T00:20:26.453715+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:14.532084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574979+0000","last_change":"2024-09-21T00:18:35.622949+0000","last_active":"2024-09-21T00:20:26.574979+0000","last_peered":"2024-09-21T00:20:26.574979+0000","last_clean":"2024-09-21T00:20:26.574979+0000","last_became_active":"2024-09-21T00:18:35.621787+0000","last_became_peered":"2024-09-21T00:18:35.621787+0000","last_unstale":"2024-09-21T00:20:26.574979+0000","last_undegraded":"2024-09-21T00:20:26.574979+0000","last_fullsized":"2024-09-21T00:20:26.574979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:39:15.060123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452419+0000","last_change":"2024-09-21T00:18:48.730711+0000","last_active":"2024-09-21T00:20:26.452419+0000","last_peered":"2024-09-21T00:20:26.452419+0000","last_clean":"2024-09-21T00:20:26.452419+0000","last_became_active":"2024-09-21T00:18:48.730287+0000","last_became_peered":"2024-09-21T00:18:48.730287+0000","last_unstale":"2024-09-21T00:20:26.452419+0000","last_undegraded":"2024-09-21T00:20:26.452419+0000","last_fullsized":"2024-09-21T00:20:26.452419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:34.444747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510047+0000","last_change":"2024-09-21T00:18:38.669381+0000","last_active":"2024-09-21T00:20:26.510047+0000","last_peered":"2024-09-21T00:20:26.510047+0000","last_clean":"2024-09-21T00:20:26.510047+0000","last_became_active":"2024-09-21T00:18:38.668757+0000","last_became_peered":"2024-09-21T00:18:38.668757+0000","last_unstale":"2024-09-21T00:20:26.510047+0000","last_undegraded":"2024-09-21T00:20:26.510047+0000","last_fullsized":"2024-09-21T00:20:26.510047+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:52.475872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508528+0000","last_change":"2024-09-21T00:18:35.767700+0000","last_active":"2024-09-21T00:20:26.508528+0000","last_peered":"2024-09-21T00:20:26.508528+0000","last_clean":"2024-09-21T00:20:26.508528+0000","last_became_active":"2024-09-21T00:18:35.767369+0000","last_became_peered":"2024-09-21T00:18:35.767369+0000","last_unstale":"2024-09-21T00:20:26.508528+0000","last_undegraded":"2024-09-21T00:20:26.508528+0000","last_fullsized":"2024-09-21T00:20:26.508528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:44.955659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452223+0000","last_change":"2024-09-21T00:18:48.737411+0000","last_active":"2024-09-21T00:20:26.452223+0000","last_peered":"2024-09-21T00:20:26.452223+0000","last_clean":"2024-09-21T00:20:26.452223+0000","last_became_active":"2024-09-21T00:18:48.736715+0000","last_became_peered":"2024-09-21T00:18:48.736715+0000","last_unstale":"2024-09-21T00:20:26.452223+0000","last_undegraded":"2024-09-21T00:20:26.452223+0000","last_fullsized":"2024-09-21T00:20:26.452223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:01.045318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455965+0000","last_change":"2024-09-21T00:19:25.842355+0000","last_active":"2024-09-21T00:20:26.455965+0000","last_peered":"2024-09-21T00:20:26.455965+0000","last_clean":"2024-09-21T00:20:26.455965+0000","last_became_active":"2024-09-21T00:19:25.841961+0000","last_became_peered":"2024-09-21T00:19:25.841961+0000","last_unstale":"2024-09-21T00:20:26.455965+0000","last_undegraded":"2024-09-21T00:20:26.455965+0000","last_fullsized":"2024-09-21T00:20:26.455965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:30.490992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452433+0000","last_change":"2024-09-21T00:18:35.919003+0000","last_active":"2024-09-21T00:20:26.452433+0000","last_peered":"2024-09-21T00:20:26.452433+0000","last_clean":"2024-09-21T00:20:26.452433+0000","last_became_active":"2024-09-21T00:18:35.918593+0000","last_became_peered":"2024-09-21T00:18:35.918593+0000","last_unstale":"2024-09-21T00:20:26.452433+0000","last_undegraded":"2024-09-21T00:20:26.452433+0000","last_fullsized":"2024-09-21T00:20:26.452433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:35.982726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451474+0000","last_change":"2024-09-21T00:18:48.725906+0000","last_active":"2024-09-21T00:20:26.451474+0000","last_peered":"2024-09-21T00:20:26.451474+0000","last_clean":"2024-09-21T00:20:26.451474+0000","last_became_active":"2024-09-21T00:18:48.725450+0000","last_became_peered":"2024-09-21T00:18:48.725450+0000","last_unstale":"2024-09-21T00:20:26.451474+0000","last_undegraded":"2024-09-21T00:20:26.451474+0000","last_fullsized":"2024-09-21T00:20:26.451474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:35.163208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.575296+0000","last_change":"2024-09-21T00:18:38.666442+0000","last_active":"2024-09-21T00:20:26.575296+0000","last_peered":"2024-09-21T00:20:26.575296+0000","last_clean":"2024-09-21T00:20:26.575296+0000","last_became_active":"2024-09-21T00:18:38.666083+0000","last_became_peered":"2024-09-21T00:18:38.666083+0000","last_unstale":"2024-09-21T00:20:26.575296+0000","last_undegraded":"2024-09-21T00:20:26.575296+0000","last_fullsized":"2024-09-21T00:20:26.575296+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:39.986440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451791+0000","last_change":"2024-09-21T00:18:35.624883+0000","last_active":"2024-09-21T00:20:26.451791+0000","last_peered":"2024-09-21T00:20:26.451791+0000","last_clean":"2024-09-21T00:20:26.451791+0000","last_became_active":"2024-09-21T00:18:35.624382+0000","last_became_peered":"2024-09-21T00:18:35.624382+0000","last_unstale":"2024-09-21T00:20:26.451791+0000","last_undegraded":"2024-09-21T00:20:26.451791+0000","last_fullsized":"2024-09-21T00:20:26.451791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:23.475521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574380+0000","last_change":"2024-09-21T00:18:48.719850+0000","last_active":"2024-09-21T00:20:26.574380+0000","last_peered":"2024-09-21T00:20:26.574380+0000","last_clean":"2024-09-21T00:20:26.574380+0000","last_became_active":"2024-09-21T00:18:48.719414+0000","last_became_peered":"2024-09-21T00:18:48.719414+0000","last_unstale":"2024-09-21T00:20:26.574380+0000","last_undegraded":"2024-09-21T00:20:26.574380+0000","last_fullsized":"2024-09-21T00:20:26.574380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:05.978977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509073+0000","last_change":"2024-09-21T00:18:38.671920+0000","last_active":"2024-09-21T00:20:26.509073+0000","last_peered":"2024-09-21T00:20:26.509073+0000","last_clean":"2024-09-21T00:20:26.509073+0000","last_became_active":"2024-09-21T00:18:38.671592+0000","last_became_peered":"2024-09-21T00:18:38.671592+0000","last_unstale":"2024-09-21T00:20:26.509073+0000","last_undegraded":"2024-09-21T00:20:26.509073+0000","last_fullsized":"2024-09-21T00:20:26.509073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:40:48.257086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.511127+0000","last_change":"2024-09-21T00:18:35.769197+0000","last_active":"2024-09-21T00:20:26.511127+0000","last_peered":"2024-09-21T00:20:26.511127+0000","last_clean":"2024-09-21T00:20:26.511127+0000","last_became_active":"2024-09-21T00:18:35.768740+0000","last_became_peered":"2024-09-21T00:18:35.768740+0000","last_unstale":"2024-09-21T00:20:26.511127+0000","last_undegraded":"2024-09-21T00:20:26.511127+0000","last_fullsized":"2024-09-21T00:20:26.511127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:32.253006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454032+0000","last_change":"2024-09-21T00:18:48.740189+0000","last_active":"2024-09-21T00:20:26.454032+0000","last_peered":"2024-09-21T00:20:26.454032+0000","last_clean":"2024-09-21T00:20:26.454032+0000","last_became_active":"2024-09-21T00:18:48.739700+0000","last_became_peered":"2024-09-21T00:18:48.739700+0000","last_unstale":"2024-09-21T00:20:26.454032+0000","last_undegraded":"2024-09-21T00:20:26.454032+0000","last_fullsized":"2024-09-21T00:20:26.454032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:06.312160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509517+0000","last_change":"2024-09-21T00:18:38.665635+0000","last_active":"2024-09-21T00:20:26.509517+0000","last_peered":"2024-09-21T00:20:26.509517+0000","last_clean":"2024-09-21T00:20:26.509517+0000","last_became_active":"2024-09-21T00:18:38.665236+0000","last_became_peered":"2024-09-21T00:18:38.665236+0000","last_unstale":"2024-09-21T00:20:26.509517+0000","last_undegraded":"2024-09-21T00:20:26.509517+0000","last_fullsized":"2024-09-21T00:20:26.509517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:52.186480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452007+0000","last_change":"2024-09-21T00:18:35.771465+0000","last_active":"2024-09-21T00:20:26.452007+0000","last_peered":"2024-09-21T00:20:26.452007+0000","last_clean":"2024-09-21T00:20:26.452007+0000","last_became_active":"2024-09-21T00:18:35.770550+0000","last_became_peered":"2024-09-21T00:18:35.770550+0000","last_unstale":"2024-09-21T00:20:26.452007+0000","last_undegraded":"2024-09-21T00:20:26.452007+0000","last_fullsized":"2024-09-21T00:20:26.452007+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:43.042563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510004+0000","last_change":"2024-09-21T00:18:48.736334+0000","last_active":"2024-09-21T00:20:26.510004+0000","last_peered":"2024-09-21T00:20:26.510004+0000","last_clean":"2024-09-21T00:20:26.510004+0000","last_became_active":"2024-09-21T00:18:48.735784+0000","last_became_peered":"2024-09-21T00:18:48.735784+0000","last_unstale":"2024-09-21T00:20:26.510004+0000","last_undegraded":"2024-09-21T00:20:26.510004+0000","last_fullsized":"2024-09-21T00:20:26.510004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:29.637335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510185+0000","last_change":"2024-09-21T00:18:38.674181+0000","last_active":"2024-09-21T00:20:26.510185+0000","last_peered":"2024-09-21T00:20:26.510185+0000","last_clean":"2024-09-21T00:20:26.510185+0000","last_became_active":"2024-09-21T00:18:38.673784+0000","last_became_peered":"2024-09-21T00:18:38.673784+0000","last_unstale":"2024-09-21T00:20:26.510185+0000","last_undegraded":"2024-09-21T00:20:26.510185+0000","last_fullsized":"2024-09-21T00:20:26.510185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:33.471328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510109+0000","last_change":"2024-09-21T00:18:35.628149+0000","last_active":"2024-09-21T00:20:26.510109+0000","last_peered":"2024-09-21T00:20:26.510109+0000","last_clean":"2024-09-21T00:20:26.510109+0000","last_became_active":"2024-09-21T00:18:35.627674+0000","last_became_peered":"2024-09-21T00:18:35.627674+0000","last_unstale":"2024-09-21T00:20:26.510109+0000","last_undegraded":"2024-09-21T00:20:26.510109+0000","last_fullsized":"2024-09-21T00:20:26.510109+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:28:47.244288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508830+0000","last_change":"2024-09-21T00:18:48.740283+0000","last_active":"2024-09-21T00:20:26.508830+0000","last_peered":"2024-09-21T00:20:26.508830+0000","last_clean":"2024-09-21T00:20:26.508830+0000","last_became_active":"2024-09-21T00:18:48.739026+0000","last_became_peered":"2024-09-21T00:18:48.739026+0000","last_unstale":"2024-09-21T00:20:26.508830+0000","last_undegraded":"2024-09-21T00:20:26.508830+0000","last_fullsized":"2024-09-21T00:20:26.508830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:50.898116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508762+0000","last_change":"2024-09-21T00:18:38.667206+0000","last_active":"2024-09-21T00:20:26.508762+0000","last_peered":"2024-09-21T00:20:26.508762+0000","last_clean":"2024-09-21T00:20:26.508762+0000","last_became_active":"2024-09-21T00:18:38.666770+0000","last_became_peered":"2024-09-21T00:18:38.666770+0000","last_unstale":"2024-09-21T00:20:26.508762+0000","last_undegraded":"2024-09-21T00:20:26.508762+0000","last_fullsized":"2024-09-21T00:20:26.508762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:56.332071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":111,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:37.020018+0000","last_change":"2024-09-21T00:18:35.768012+0000","last_active":"2024-09-21T00:20:37.020018+0000","last_peered":"2024-09-21T00:20:37.020018+0000","last_clean":"2024-09-21T00:20:37.020018+0000","last_became_active":"2024-09-21T00:18:35.767438+0000","last_became_peered":"2024-09-21T00:18:35.767438+0000","last_unstale":"2024-09-21T00:20:37.020018+0000","last_undegraded":"2024-09-21T00:20:37.020018+0000","last_fullsized":"2024-09-21T00:20:37.020018+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:35:40.606871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508575+0000","last_change":"2024-09-21T00:18:48.738228+0000","last_active":"2024-09-21T00:20:26.508575+0000","last_peered":"2024-09-21T00:20:26.508575+0000","last_clean":"2024-09-21T00:20:26.508575+0000","last_became_active":"2024-09-21T00:18:48.737786+0000","last_became_peered":"2024-09-21T00:18:48.737786+0000","last_unstale":"2024-09-21T00:20:26.508575+0000","last_undegraded":"2024-09-21T00:20:26.508575+0000","last_fullsized":"2024-09-21T00:20:26.508575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:52.329677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508254+0000","last_change":"2024-09-21T00:18:38.678126+0000","last_active":"2024-09-21T00:20:26.508254+0000","last_peered":"2024-09-21T00:20:26.508254+0000","last_clean":"2024-09-21T00:20:26.508254+0000","last_became_active":"2024-09-21T00:18:38.677757+0000","last_became_peered":"2024-09-21T00:18:38.677757+0000","last_unstale":"2024-09-21T00:20:26.508254+0000","last_undegraded":"2024-09-21T00:20:26.508254+0000","last_fullsized":"2024-09-21T00:20:26.508254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:26.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508836+0000","last_change":"2024-09-21T00:18:35.625955+0000","last_active":"2024-09-21T00:20:26.508836+0000","last_peered":"2024-09-21T00:20:26.508836+0000","last_clean":"2024-09-21T00:20:26.508836+0000","last_became_active":"2024-09-21T00:18:35.625548+0000","last_became_peered":"2024-09-21T00:18:35.625548+0000","last_unstale":"2024-09-21T00:20:26.508836+0000","last_undegraded":"2024-09-21T00:20:26.508836+0000","last_fullsized":"2024-09-21T00:20:26.508836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:04.163421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509284+0000","last_change":"2024-09-21T00:18:48.740765+0000","last_active":"2024-09-21T00:20:26.509284+0000","last_peered":"2024-09-21T00:20:26.509284+0000","last_clean":"2024-09-21T00:20:26.509284+0000","last_became_active":"2024-09-21T00:18:48.740081+0000","last_became_peered":"2024-09-21T00:18:48.740081+0000","last_unstale":"2024-09-21T00:20:26.509284+0000","last_undegraded":"2024-09-21T00:20:26.509284+0000","last_fullsized":"2024-09-21T00:20:26.509284+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:38.251475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574852+0000","last_change":"2024-09-21T00:18:38.667581+0000","last_active":"2024-09-21T00:20:26.574852+0000","last_peered":"2024-09-21T00:20:26.574852+0000","last_clean":"2024-09-21T00:20:26.574852+0000","last_became_active":"2024-09-21T00:18:38.665959+0000","last_became_peered":"2024-09-21T00:18:38.665959+0000","last_unstale":"2024-09-21T00:20:26.574852+0000","last_undegraded":"2024-09-21T00:20:26.574852+0000","last_fullsized":"2024-09-21T00:20:26.574852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:52.321654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452596+0000","last_change":"2024-09-21T00:18:35.624291+0000","last_active":"2024-09-21T00:20:26.452596+0000","last_peered":"2024-09-21T00:20:26.452596+0000","last_clean":"2024-09-21T00:20:26.452596+0000","last_became_active":"2024-09-21T00:18:35.623297+0000","last_became_peered":"2024-09-21T00:18:35.623297+0000","last_unstale":"2024-09-21T00:20:26.452596+0000","last_undegraded":"2024-09-21T00:20:26.452596+0000","last_fullsized":"2024-09-21T00:20:26.452596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:44.824409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509191+0000","last_change":"2024-09-21T00:17:21.233627+0000","last_active":"2024-09-21T00:20:26.509191+0000","last_peered":"2024-09-21T00:20:26.509191+0000","last_clean":"2024-09-21T00:20:26.509191+0000","last_became_active":"2024-09-21T00:17:21.227897+0000","last_became_peered":"2024-09-21T00:17:21.227897+0000","last_unstale":"2024-09-21T00:20:26.509191+0000","last_undegraded":"2024-09-21T00:20:26.509191+0000","last_fullsized":"2024-09-21T00:20:26.509191+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:13:33.120691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:13:33.120691+0000","last_clean_scrub_stamp":"2024-09-21T00:13:33.120691+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:20.198660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509344+0000","last_change":"2024-09-21T00:18:48.730107+0000","last_active":"2024-09-21T00:20:26.509344+0000","last_peered":"2024-09-21T00:20:26.509344+0000","last_clean":"2024-09-21T00:20:26.509344+0000","last_became_active":"2024-09-21T00:18:48.729463+0000","last_became_peered":"2024-09-21T00:18:48.729463+0000","last_unstale":"2024-09-21T00:20:26.509344+0000","last_undegraded":"2024-09-21T00:20:26.509344+0000","last_fullsized":"2024-09-21T00:20:26.509344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:55.352870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509702+0000","last_change":"2024-09-21T00:18:38.671244+0000","last_active":"2024-09-21T00:20:26.509702+0000","last_peered":"2024-09-21T00:20:26.509702+0000","last_clean":"2024-09-21T00:20:26.509702+0000","last_became_active":"2024-09-21T00:18:38.670630+0000","last_became_peered":"2024-09-21T00:18:38.670630+0000","last_unstale":"2024-09-21T00:20:26.509702+0000","last_undegraded":"2024-09-21T00:20:26.509702+0000","last_fullsized":"2024-09-21T00:20:26.509702+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:23.695660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451789+0000","last_change":"2024-09-21T00:19:25.843900+0000","last_active":"2024-09-21T00:20:26.451789+0000","last_peered":"2024-09-21T00:20:26.451789+0000","last_clean":"2024-09-21T00:20:26.451789+0000","last_became_active":"2024-09-21T00:19:25.843438+0000","last_became_peered":"2024-09-21T00:19:25.843438+0000","last_unstale":"2024-09-21T00:20:26.451789+0000","last_undegraded":"2024-09-21T00:20:26.451789+0000","last_fullsized":"2024-09-21T00:20:26.451789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:17.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509343+0000","last_change":"2024-09-21T00:18:48.739640+0000","last_active":"2024-09-21T00:20:26.509343+0000","last_peered":"2024-09-21T00:20:26.509343+0000","last_clean":"2024-09-21T00:20:26.509343+0000","last_became_active":"2024-09-21T00:18:48.739118+0000","last_became_peered":"2024-09-21T00:18:48.739118+0000","last_unstale":"2024-09-21T00:20:26.509343+0000","last_undegraded":"2024-09-21T00:20:26.509343+0000","last_fullsized":"2024-09-21T00:20:26.509343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:03.093561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509166+0000","last_change":"2024-09-21T00:18:38.673020+0000","last_active":"2024-09-21T00:20:26.509166+0000","last_peered":"2024-09-21T00:20:26.509166+0000","last_clean":"2024-09-21T00:20:26.509166+0000","last_became_active":"2024-09-21T00:18:38.672391+0000","last_became_peered":"2024-09-21T00:18:38.672391+0000","last_unstale":"2024-09-21T00:20:26.509166+0000","last_undegraded":"2024-09-21T00:20:26.509166+0000","last_fullsized":"2024-09-21T00:20:26.509166+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:13.866576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509394+0000","last_change":"2024-09-21T00:18:35.615844+0000","last_active":"2024-09-21T00:20:26.509394+0000","last_peered":"2024-09-21T00:20:26.509394+0000","last_clean":"2024-09-21T00:20:26.509394+0000","last_became_active":"2024-09-21T00:18:35.615385+0000","last_became_peered":"2024-09-21T00:18:35.615385+0000","last_unstale":"2024-09-21T00:20:26.509394+0000","last_undegraded":"2024-09-21T00:20:26.509394+0000","last_fullsized":"2024-09-21T00:20:26.509394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:03.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.512053+0000","last_change":"2024-09-21T00:20:26.511995+0000","last_active":"2024-09-21T00:20:26.512053+0000","last_peered":"2024-09-21T00:20:26.512053+0000","last_clean":"2024-09-21T00:20:26.512053+0000","last_became_active":"2024-09-21T00:20:26.511607+0000","last_became_peered":"2024-09-21T00:20:26.511607+0000","last_unstale":"2024-09-21T00:20:26.512053+0000","last_undegraded":"2024-09-21T00:20:26.512053+0000","last_fullsized":"2024-09-21T00:20:26.512053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:38.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508874+0000","last_change":"2024-09-21T00:18:38.671280+0000","last_active":"2024-09-21T00:20:26.508874+0000","last_peered":"2024-09-21T00:20:26.508874+0000","last_clean":"2024-09-21T00:20:26.508874+0000","last_became_active":"2024-09-21T00:18:38.670770+0000","last_became_peered":"2024-09-21T00:18:38.670770+0000","last_unstale":"2024-09-21T00:20:26.508874+0000","last_undegraded":"2024-09-21T00:20:26.508874+0000","last_fullsized":"2024-09-21T00:20:26.508874+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:06.809450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509562+0000","last_change":"2024-09-21T00:18:35.626071+0000","last_active":"2024-09-21T00:20:26.509562+0000","last_peered":"2024-09-21T00:20:26.509562+0000","last_clean":"2024-09-21T00:20:26.509562+0000","last_became_active":"2024-09-21T00:18:35.625649+0000","last_became_peered":"2024-09-21T00:18:35.625649+0000","last_unstale":"2024-09-21T00:20:26.509562+0000","last_undegraded":"2024-09-21T00:20:26.509562+0000","last_fullsized":"2024-09-21T00:20:26.509562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:16:27.516303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508563+0000","last_change":"2024-09-21T00:18:48.728451+0000","last_active":"2024-09-21T00:20:26.508563+0000","last_peered":"2024-09-21T00:20:26.508563+0000","last_clean":"2024-09-21T00:20:26.508563+0000","last_became_active":"2024-09-21T00:18:48.727840+0000","last_became_peered":"2024-09-21T00:18:48.727840+0000","last_unstale":"2024-09-21T00:20:26.508563+0000","last_undegraded":"2024-09-21T00:20:26.508563+0000","last_fullsized":"2024-09-21T00:20:26.508563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:13.457138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454195+0000","last_change":"2024-09-21T00:18:38.664090+0000","last_active":"2024-09-21T00:20:26.454195+0000","last_peered":"2024-09-21T00:20:26.454195+0000","last_clean":"2024-09-21T00:20:26.454195+0000","last_became_active":"2024-09-21T00:18:38.663728+0000","last_became_peered":"2024-09-21T00:18:38.663728+0000","last_unstale":"2024-09-21T00:20:26.454195+0000","last_undegraded":"2024-09-21T00:20:26.454195+0000","last_fullsized":"2024-09-21T00:20:26.454195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:46.367766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454833+0000","last_change":"2024-09-21T00:19:25.847280+0000","last_active":"2024-09-21T00:20:26.454833+0000","last_peered":"2024-09-21T00:20:26.454833+0000","last_clean":"2024-09-21T00:20:26.454833+0000","last_became_active":"2024-09-21T00:19:25.840919+0000","last_became_peered":"2024-09-21T00:19:25.840919+0000","last_unstale":"2024-09-21T00:20:26.454833+0000","last_undegraded":"2024-09-21T00:20:26.454833+0000","last_fullsized":"2024-09-21T00:20:26.454833+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:15.411674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508729+0000","last_change":"2024-09-21T00:18:48.732427+0000","last_active":"2024-09-21T00:20:26.508729+0000","last_peered":"2024-09-21T00:20:26.508729+0000","last_clean":"2024-09-21T00:20:26.508729+0000","last_became_active":"2024-09-21T00:18:48.732078+0000","last_became_peered":"2024-09-21T00:18:48.732078+0000","last_unstale":"2024-09-21T00:20:26.508729+0000","last_undegraded":"2024-09-21T00:20:26.508729+0000","last_fullsized":"2024-09-21T00:20:26.508729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:35.055680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509565+0000","last_change":"2024-09-21T00:18:38.675022+0000","last_active":"2024-09-21T00:20:26.509565+0000","last_peered":"2024-09-21T00:20:26.509565+0000","last_clean":"2024-09-21T00:20:26.509565+0000","last_became_active":"2024-09-21T00:18:38.674650+0000","last_became_peered":"2024-09-21T00:18:38.674650+0000","last_unstale":"2024-09-21T00:20:26.509565+0000","last_undegraded":"2024-09-21T00:20:26.509565+0000","last_fullsized":"2024-09-21T00:20:26.509565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:05.191404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510081+0000","last_change":"2024-09-21T00:18:35.769555+0000","last_active":"2024-09-21T00:20:26.510081+0000","last_peered":"2024-09-21T00:20:26.510081+0000","last_clean":"2024-09-21T00:20:26.510081+0000","last_became_active":"2024-09-21T00:18:35.769080+0000","last_became_peered":"2024-09-21T00:18:35.769080+0000","last_unstale":"2024-09-21T00:20:26.510081+0000","last_undegraded":"2024-09-21T00:20:26.510081+0000","last_fullsized":"2024-09-21T00:20:26.510081+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:19.306830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509500+0000","last_change":"2024-09-21T00:18:48.742957+0000","last_active":"2024-09-21T00:20:26.509500+0000","last_peered":"2024-09-21T00:20:26.509500+0000","last_clean":"2024-09-21T00:20:26.509500+0000","last_became_active":"2024-09-21T00:18:48.742511+0000","last_became_peered":"2024-09-21T00:18:48.742511+0000","last_unstale":"2024-09-21T00:20:26.509500+0000","last_undegraded":"2024-09-21T00:20:26.509500+0000","last_fullsized":"2024-09-21T00:20:26.509500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:15.231189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508572+0000","last_change":"2024-09-21T00:18:38.666576+0000","last_active":"2024-09-21T00:20:26.508572+0000","last_peered":"2024-09-21T00:20:26.508572+0000","last_clean":"2024-09-21T00:20:26.508572+0000","last_became_active":"2024-09-21T00:18:38.665643+0000","last_became_peered":"2024-09-21T00:18:38.665643+0000","last_unstale":"2024-09-21T00:20:26.508572+0000","last_undegraded":"2024-09-21T00:20:26.508572+0000","last_fullsized":"2024-09-21T00:20:26.508572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:35.397847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:40.024711+0000","last_change":"2024-09-21T00:18:35.623143+0000","last_active":"2024-09-21T00:20:40.024711+0000","last_peered":"2024-09-21T00:20:40.024711+0000","last_clean":"2024-09-21T00:20:40.024711+0000","last_became_active":"2024-09-21T00:18:35.622693+0000","last_became_peered":"2024-09-21T00:18:35.622693+0000","last_unstale":"2024-09-21T00:20:40.024711+0000","last_undegraded":"2024-09-21T00:20:40.024711+0000","last_fullsized":"2024-09-21T00:20:40.024711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:01.331665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452805+0000","last_change":"2024-09-21T00:18:48.723877+0000","last_active":"2024-09-21T00:20:26.452805+0000","last_peered":"2024-09-21T00:20:26.452805+0000","last_clean":"2024-09-21T00:20:26.452805+0000","last_became_active":"2024-09-21T00:18:48.723360+0000","last_became_peered":"2024-09-21T00:18:48.723360+0000","last_unstale":"2024-09-21T00:20:26.452805+0000","last_undegraded":"2024-09-21T00:20:26.452805+0000","last_fullsized":"2024-09-21T00:20:26.452805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:07:20.442867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509000+0000","last_change":"2024-09-21T00:18:38.669643+0000","last_active":"2024-09-21T00:20:26.509000+0000","last_peered":"2024-09-21T00:20:26.509000+0000","last_clean":"2024-09-21T00:20:26.509000+0000","last_became_active":"2024-09-21T00:18:38.669245+0000","last_became_peered":"2024-09-21T00:18:38.669245+0000","last_unstale":"2024-09-21T00:20:26.509000+0000","last_undegraded":"2024-09-21T00:20:26.509000+0000","last_fullsized":"2024-09-21T00:20:26.509000+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:17.789291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509513+0000","last_change":"2024-09-21T00:18:35.623189+0000","last_active":"2024-09-21T00:20:26.509513+0000","last_peered":"2024-09-21T00:20:26.509513+0000","last_clean":"2024-09-21T00:20:26.509513+0000","last_became_active":"2024-09-21T00:18:35.622748+0000","last_became_peered":"2024-09-21T00:18:35.622748+0000","last_unstale":"2024-09-21T00:20:26.509513+0000","last_undegraded":"2024-09-21T00:20:26.509513+0000","last_fullsized":"2024-09-21T00:20:26.509513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:15.570789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453777+0000","last_change":"2024-09-21T00:18:48.739612+0000","last_active":"2024-09-21T00:20:26.453777+0000","last_peered":"2024-09-21T00:20:26.453777+0000","last_clean":"2024-09-21T00:20:26.453777+0000","last_became_active":"2024-09-21T00:18:48.739173+0000","last_became_peered":"2024-09-21T00:18:48.739173+0000","last_unstale":"2024-09-21T00:20:26.453777+0000","last_undegraded":"2024-09-21T00:20:26.453777+0000","last_fullsized":"2024-09-21T00:20:26.453777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:26.327178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508470+0000","last_change":"2024-09-21T00:18:38.670876+0000","last_active":"2024-09-21T00:20:26.508470+0000","last_peered":"2024-09-21T00:20:26.508470+0000","last_clean":"2024-09-21T00:20:26.508470+0000","last_became_active":"2024-09-21T00:18:38.670371+0000","last_became_peered":"2024-09-21T00:18:38.670371+0000","last_unstale":"2024-09-21T00:20:26.508470+0000","last_undegraded":"2024-09-21T00:20:26.508470+0000","last_fullsized":"2024-09-21T00:20:26.508470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:48.641071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451958+0000","last_change":"2024-09-21T00:18:35.622919+0000","last_active":"2024-09-21T00:20:26.451958+0000","last_peered":"2024-09-21T00:20:26.451958+0000","last_clean":"2024-09-21T00:20:26.451958+0000","last_became_active":"2024-09-21T00:18:35.622435+0000","last_became_peered":"2024-09-21T00:18:35.622435+0000","last_unstale":"2024-09-21T00:20:26.451958+0000","last_undegraded":"2024-09-21T00:20:26.451958+0000","last_fullsized":"2024-09-21T00:20:26.451958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:22.113871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508927+0000","last_change":"2024-09-21T00:18:48.741736+0000","last_active":"2024-09-21T00:20:26.508927+0000","last_peered":"2024-09-21T00:20:26.508927+0000","last_clean":"2024-09-21T00:20:26.508927+0000","last_became_active":"2024-09-21T00:18:48.741135+0000","last_became_peered":"2024-09-21T00:18:48.741135+0000","last_unstale":"2024-09-21T00:20:26.508927+0000","last_undegraded":"2024-09-21T00:20:26.508927+0000","last_fullsized":"2024-09-21T00:20:26.508927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:50.492351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510579+0000","last_change":"2024-09-21T00:18:38.667377+0000","last_active":"2024-09-21T00:20:26.510579+0000","last_peered":"2024-09-21T00:20:26.510579+0000","last_clean":"2024-09-21T00:20:26.510579+0000","last_became_active":"2024-09-21T00:18:38.667010+0000","last_became_peered":"2024-09-21T00:18:38.667010+0000","last_unstale":"2024-09-21T00:20:26.510579+0000","last_undegraded":"2024-09-21T00:20:26.510579+0000","last_fullsized":"2024-09-21T00:20:26.510579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:06.634767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":231,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:40.023437+0000","last_change":"2024-09-21T00:19:25.853603+0000","last_active":"2024-09-21T00:20:40.023437+0000","last_peered":"2024-09-21T00:20:40.023437+0000","last_clean":"2024-09-21T00:20:40.023437+0000","last_became_active":"2024-09-21T00:19:25.841126+0000","last_became_peered":"2024-09-21T00:19:25.841126+0000","last_unstale":"2024-09-21T00:20:40.023437+0000","last_undegraded":"2024-09-21T00:20:40.023437+0000","last_fullsized":"2024-09-21T00:20:40.023437+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:27:47.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65659,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":138,"num_read_kb":133,"num_write":32,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451643+0000","last_change":"2024-09-21T00:18:48.731004+0000","last_active":"2024-09-21T00:20:26.451643+0000","last_peered":"2024-09-21T00:20:26.451643+0000","last_clean":"2024-09-21T00:20:26.451643+0000","last_became_active":"2024-09-21T00:18:48.730549+0000","last_became_peered":"2024-09-21T00:18:48.730549+0000","last_unstale":"2024-09-21T00:20:26.451643+0000","last_undegraded":"2024-09-21T00:20:26.451643+0000","last_fullsized":"2024-09-21T00:20:26.451643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:30.988300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510768+0000","last_change":"2024-09-21T00:19:25.843532+0000","last_active":"2024-09-21T00:20:26.510768+0000","last_peered":"2024-09-21T00:20:26.510768+0000","last_clean":"2024-09-21T00:20:26.510768+0000","last_became_active":"2024-09-21T00:19:25.843065+0000","last_became_peered":"2024-09-21T00:19:25.843065+0000","last_unstale":"2024-09-21T00:20:26.510768+0000","last_undegraded":"2024-09-21T00:20:26.510768+0000","last_fullsized":"2024-09-21T00:20:26.510768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:36:26.488692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509235+0000","last_change":"2024-09-21T00:18:35.770494+0000","last_active":"2024-09-21T00:20:26.509235+0000","last_peered":"2024-09-21T00:20:26.509235+0000","last_clean":"2024-09-21T00:20:26.509235+0000","last_became_active":"2024-09-21T00:18:35.770122+0000","last_became_peered":"2024-09-21T00:18:35.770122+0000","last_unstale":"2024-09-21T00:20:26.509235+0000","last_undegraded":"2024-09-21T00:20:26.509235+0000","last_fullsized":"2024-09-21T00:20:26.509235+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:41.415904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451780+0000","last_change":"2024-09-21T00:18:48.736802+0000","last_active":"2024-09-21T00:20:26.451780+0000","last_peered":"2024-09-21T00:20:26.451780+0000","last_clean":"2024-09-21T00:20:26.451780+0000","last_became_active":"2024-09-21T00:18:48.736008+0000","last_became_peered":"2024-09-21T00:18:48.736008+0000","last_unstale":"2024-09-21T00:20:26.451780+0000","last_undegraded":"2024-09-21T00:20:26.451780+0000","last_fullsized":"2024-09-21T00:20:26.451780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:07.995614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455871+0000","last_change":"2024-09-21T00:18:38.678385+0000","last_active":"2024-09-21T00:20:26.455871+0000","last_peered":"2024-09-21T00:20:26.455871+0000","last_clean":"2024-09-21T00:20:26.455871+0000","last_became_active":"2024-09-21T00:18:38.677174+0000","last_became_peered":"2024-09-21T00:18:38.677174+0000","last_unstale":"2024-09-21T00:20:26.455871+0000","last_undegraded":"2024-09-21T00:20:26.455871+0000","last_fullsized":"2024-09-21T00:20:26.455871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:23.486222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509438+0000","last_change":"2024-09-21T00:18:35.627703+0000","last_active":"2024-09-21T00:20:26.509438+0000","last_peered":"2024-09-21T00:20:26.509438+0000","last_clean":"2024-09-21T00:20:26.509438+0000","last_became_active":"2024-09-21T00:18:35.627274+0000","last_became_peered":"2024-09-21T00:18:35.627274+0000","last_unstale":"2024-09-21T00:20:26.509438+0000","last_undegraded":"2024-09-21T00:20:26.509438+0000","last_fullsized":"2024-09-21T00:20:26.509438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:14.326810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451042+0000","last_change":"2024-09-21T00:18:48.729926+0000","last_active":"2024-09-21T00:20:26.451042+0000","last_peered":"2024-09-21T00:20:26.451042+0000","last_clean":"2024-09-21T00:20:26.451042+0000","last_became_active":"2024-09-21T00:18:48.728366+0000","last_became_peered":"2024-09-21T00:18:48.728366+0000","last_unstale":"2024-09-21T00:20:26.451042+0000","last_undegraded":"2024-09-21T00:20:26.451042+0000","last_fullsized":"2024-09-21T00:20:26.451042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:48:50.005748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.511150+0000","last_change":"2024-09-21T00:18:38.664173+0000","last_active":"2024-09-21T00:20:26.511150+0000","last_peered":"2024-09-21T00:20:26.511150+0000","last_clean":"2024-09-21T00:20:26.511150+0000","last_became_active":"2024-09-21T00:18:38.663358+0000","last_became_peered":"2024-09-21T00:18:38.663358+0000","last_unstale":"2024-09-21T00:20:26.511150+0000","last_undegraded":"2024-09-21T00:20:26.511150+0000","last_fullsized":"2024-09-21T00:20:26.511150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:34:31.203621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":113,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:39.022273+0000","last_change":"2024-09-21T00:18:35.622133+0000","last_active":"2024-09-21T00:20:39.022273+0000","last_peered":"2024-09-21T00:20:39.022273+0000","last_clean":"2024-09-21T00:20:39.022273+0000","last_became_active":"2024-09-21T00:18:35.621032+0000","last_became_peered":"2024-09-21T00:18:35.621032+0000","last_unstale":"2024-09-21T00:20:39.022273+0000","last_undegraded":"2024-09-21T00:20:39.022273+0000","last_fullsized":"2024-09-21T00:20:39.022273+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:24.121665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574904+0000","last_change":"2024-09-21T00:18:48.733339+0000","last_active":"2024-09-21T00:20:26.574904+0000","last_peered":"2024-09-21T00:20:26.574904+0000","last_clean":"2024-09-21T00:20:26.574904+0000","last_became_active":"2024-09-21T00:18:48.731843+0000","last_became_peered":"2024-09-21T00:18:48.731843+0000","last_unstale":"2024-09-21T00:20:26.574904+0000","last_undegraded":"2024-09-21T00:20:26.574904+0000","last_fullsized":"2024-09-21T00:20:26.574904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:42:55.474197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455802+0000","last_change":"2024-09-21T00:18:48.740108+0000","last_active":"2024-09-21T00:20:26.455802+0000","last_peered":"2024-09-21T00:20:26.455802+0000","last_clean":"2024-09-21T00:20:26.455802+0000","last_became_active":"2024-09-21T00:18:48.739330+0000","last_became_peered":"2024-09-21T00:18:48.739330+0000","last_unstale":"2024-09-21T00:20:26.455802+0000","last_undegraded":"2024-09-21T00:20:26.455802+0000","last_fullsized":"2024-09-21T00:20:26.455802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:00.103502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451244+0000","last_change":"2024-09-21T00:18:38.660469+0000","last_active":"2024-09-21T00:20:26.451244+0000","last_peered":"2024-09-21T00:20:26.451244+0000","last_clean":"2024-09-21T00:20:26.451244+0000","last_became_active":"2024-09-21T00:18:38.660138+0000","last_became_peered":"2024-09-21T00:18:38.660138+0000","last_unstale":"2024-09-21T00:20:26.451244+0000","last_undegraded":"2024-09-21T00:20:26.451244+0000","last_fullsized":"2024-09-21T00:20:26.451244+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:01.079112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509126+0000","last_change":"2024-09-21T00:18:35.624528+0000","last_active":"2024-09-21T00:20:26.509126+0000","last_peered":"2024-09-21T00:20:26.509126+0000","last_clean":"2024-09-21T00:20:26.509126+0000","last_became_active":"2024-09-21T00:18:35.624170+0000","last_became_peered":"2024-09-21T00:18:35.624170+0000","last_unstale":"2024-09-21T00:20:26.509126+0000","last_undegraded":"2024-09-21T00:20:26.509126+0000","last_fullsized":"2024-09-21T00:20:26.509126+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:07.914521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.511927+0000","last_change":"2024-09-21T00:20:26.511824+0000","last_active":"2024-09-21T00:20:26.511927+0000","last_peered":"2024-09-21T00:20:26.511927+0000","last_clean":"2024-09-21T00:20:26.511927+0000","last_became_active":"2024-09-21T00:20:26.511254+0000","last_became_peered":"2024-09-21T00:20:26.511254+0000","last_unstale":"2024-09-21T00:20:26.511927+0000","last_undegraded":"2024-09-21T00:20:26.511927+0000","last_fullsized":"2024-09-21T00:20:26.511927+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:12.884034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574527+0000","last_change":"2024-09-21T00:18:38.666928+0000","last_active":"2024-09-21T00:20:26.574527+0000","last_peered":"2024-09-21T00:20:26.574527+0000","last_clean":"2024-09-21T00:20:26.574527+0000","last_became_active":"2024-09-21T00:18:38.666236+0000","last_became_peered":"2024-09-21T00:18:38.666236+0000","last_unstale":"2024-09-21T00:20:26.574527+0000","last_undegraded":"2024-09-21T00:20:26.574527+0000","last_fullsized":"2024-09-21T00:20:26.574527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:21.043778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508260+0000","last_change":"2024-09-21T00:18:35.624514+0000","last_active":"2024-09-21T00:20:26.508260+0000","last_peered":"2024-09-21T00:20:26.508260+0000","last_clean":"2024-09-21T00:20:26.508260+0000","last_became_active":"2024-09-21T00:18:35.624098+0000","last_became_peered":"2024-09-21T00:18:35.624098+0000","last_unstale":"2024-09-21T00:20:26.508260+0000","last_undegraded":"2024-09-21T00:20:26.508260+0000","last_fullsized":"2024-09-21T00:20:26.508260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:21.777568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509824+0000","last_change":"2024-09-21T00:19:25.841232+0000","last_active":"2024-09-21T00:20:26.509824+0000","last_peered":"2024-09-21T00:20:26.509824+0000","last_clean":"2024-09-21T00:20:26.509824+0000","last_became_active":"2024-09-21T00:19:25.840655+0000","last_became_peered":"2024-09-21T00:19:25.840655+0000","last_unstale":"2024-09-21T00:20:26.509824+0000","last_undegraded":"2024-09-21T00:20:26.509824+0000","last_fullsized":"2024-09-21T00:20:26.509824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:05.846222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509106+0000","last_change":"2024-09-21T00:18:38.667466+0000","last_active":"2024-09-21T00:20:26.509106+0000","last_peered":"2024-09-21T00:20:26.509106+0000","last_clean":"2024-09-21T00:20:26.509106+0000","last_became_active":"2024-09-21T00:18:38.666960+0000","last_became_peered":"2024-09-21T00:18:38.666960+0000","last_unstale":"2024-09-21T00:20:26.509106+0000","last_undegraded":"2024-09-21T00:20:26.509106+0000","last_fullsized":"2024-09-21T00:20:26.509106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:29:56.614766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451538+0000","last_change":"2024-09-21T00:18:48.739656+0000","last_active":"2024-09-21T00:20:26.451538+0000","last_peered":"2024-09-21T00:20:26.451538+0000","last_clean":"2024-09-21T00:20:26.451538+0000","last_became_active":"2024-09-21T00:18:48.739228+0000","last_became_peered":"2024-09-21T00:18:48.739228+0000","last_unstale":"2024-09-21T00:20:26.451538+0000","last_undegraded":"2024-09-21T00:20:26.451538+0000","last_fullsized":"2024-09-21T00:20:26.451538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:11.427595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:40.024152+0000","last_change":"2024-09-21T00:18:35.624108+0000","last_active":"2024-09-21T00:20:40.024152+0000","last_peered":"2024-09-21T00:20:40.024152+0000","last_clean":"2024-09-21T00:20:40.024152+0000","last_became_active":"2024-09-21T00:18:35.623708+0000","last_became_peered":"2024-09-21T00:18:35.623708+0000","last_unstale":"2024-09-21T00:20:40.024152+0000","last_undegraded":"2024-09-21T00:20:40.024152+0000","last_fullsized":"2024-09-21T00:20:40.024152+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:50.020662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510214+0000","last_change":"2024-09-21T00:18:38.665402+0000","last_active":"2024-09-21T00:20:26.510214+0000","last_peered":"2024-09-21T00:20:26.510214+0000","last_clean":"2024-09-21T00:20:26.510214+0000","last_became_active":"2024-09-21T00:18:38.665034+0000","last_became_peered":"2024-09-21T00:18:38.665034+0000","last_unstale":"2024-09-21T00:20:26.510214+0000","last_undegraded":"2024-09-21T00:20:26.510214+0000","last_fullsized":"2024-09-21T00:20:26.510214+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:33:10.824705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451734+0000","last_change":"2024-09-21T00:18:48.729928+0000","last_active":"2024-09-21T00:20:26.451734+0000","last_peered":"2024-09-21T00:20:26.451734+0000","last_clean":"2024-09-21T00:20:26.451734+0000","last_became_active":"2024-09-21T00:18:48.729498+0000","last_became_peered":"2024-09-21T00:18:48.729498+0000","last_unstale":"2024-09-21T00:20:26.451734+0000","last_undegraded":"2024-09-21T00:20:26.451734+0000","last_fullsized":"2024-09-21T00:20:26.451734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:10:27.140012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453735+0000","last_change":"2024-09-21T00:18:38.662221+0000","last_active":"2024-09-21T00:20:26.453735+0000","last_peered":"2024-09-21T00:20:26.453735+0000","last_clean":"2024-09-21T00:20:26.453735+0000","last_became_active":"2024-09-21T00:18:38.660543+0000","last_became_peered":"2024-09-21T00:18:38.660543+0000","last_unstale":"2024-09-21T00:20:26.453735+0000","last_undegraded":"2024-09-21T00:20:26.453735+0000","last_fullsized":"2024-09-21T00:20:26.453735+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:43.159699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452102+0000","last_change":"2024-09-21T00:18:38.672696+0000","last_active":"2024-09-21T00:20:26.452102+0000","last_peered":"2024-09-21T00:20:26.452102+0000","last_clean":"2024-09-21T00:20:26.452102+0000","last_became_active":"2024-09-21T00:18:38.672356+0000","last_became_peered":"2024-09-21T00:18:38.672356+0000","last_unstale":"2024-09-21T00:20:26.452102+0000","last_undegraded":"2024-09-21T00:20:26.452102+0000","last_fullsized":"2024-09-21T00:20:26.452102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.549939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451409+0000","last_change":"2024-09-21T00:18:38.667017+0000","last_active":"2024-09-21T00:20:26.451409+0000","last_peered":"2024-09-21T00:20:26.451409+0000","last_clean":"2024-09-21T00:20:26.451409+0000","last_became_active":"2024-09-21T00:18:38.666643+0000","last_became_peered":"2024-09-21T00:18:38.666643+0000","last_unstale":"2024-09-21T00:20:26.451409+0000","last_undegraded":"2024-09-21T00:20:26.451409+0000","last_fullsized":"2024-09-21T00:20:26.451409+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:30.496339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454925+0000","last_change":"2024-09-21T00:18:38.667705+0000","last_active":"2024-09-21T00:20:26.454925+0000","last_peered":"2024-09-21T00:20:26.454925+0000","last_clean":"2024-09-21T00:20:26.454925+0000","last_became_active":"2024-09-21T00:18:38.665141+0000","last_became_peered":"2024-09-21T00:18:38.665141+0000","last_unstale":"2024-09-21T00:20:26.454925+0000","last_undegraded":"2024-09-21T00:20:26.454925+0000","last_fullsized":"2024-09-21T00:20:26.454925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:17.624035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509000+0000","last_change":"2024-09-21T00:18:38.681364+0000","last_active":"2024-09-21T00:20:26.509000+0000","last_peered":"2024-09-21T00:20:26.509000+0000","last_clean":"2024-09-21T00:20:26.509000+0000","last_became_active":"2024-09-21T00:18:38.680937+0000","last_became_peered":"2024-09-21T00:18:38.680937+0000","last_unstale":"2024-09-21T00:20:26.509000+0000","last_undegraded":"2024-09-21T00:20:26.509000+0000","last_fullsized":"2024-09-21T00:20:26.509000+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:34.206201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454175+0000","last_change":"2024-09-21T00:19:25.843091+0000","last_active":"2024-09-21T00:20:26.454175+0000","last_peered":"2024-09-21T00:20:26.454175+0000","last_clean":"2024-09-21T00:20:26.454175+0000","last_became_active":"2024-09-21T00:19:25.842600+0000","last_became_peered":"2024-09-21T00:19:25.842600+0000","last_unstale":"2024-09-21T00:20:26.454175+0000","last_undegraded":"2024-09-21T00:20:26.454175+0000","last_fullsized":"2024-09-21T00:20:26.454175+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:10.256474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451827+0000","last_change":"2024-09-21T00:18:38.668656+0000","last_active":"2024-09-21T00:20:26.451827+0000","last_peered":"2024-09-21T00:20:26.451827+0000","last_clean":"2024-09-21T00:20:26.451827+0000","last_became_active":"2024-09-21T00:18:38.668153+0000","last_became_peered":"2024-09-21T00:18:38.668153+0000","last_unstale":"2024-09-21T00:20:26.451827+0000","last_undegraded":"2024-09-21T00:20:26.451827+0000","last_fullsized":"2024-09-21T00:20:26.451827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:03.763609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.575289+0000","last_change":"2024-09-21T00:18:38.675945+0000","last_active":"2024-09-21T00:20:26.575289+0000","last_peered":"2024-09-21T00:20:26.575289+0000","last_clean":"2024-09-21T00:20:26.575289+0000","last_became_active":"2024-09-21T00:18:38.675397+0000","last_became_peered":"2024-09-21T00:18:38.675397+0000","last_unstale":"2024-09-21T00:20:26.575289+0000","last_undegraded":"2024-09-21T00:20:26.575289+0000","last_fullsized":"2024-09-21T00:20:26.575289+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:56.082314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508699+0000","last_change":"2024-09-21T00:18:38.662854+0000","last_active":"2024-09-21T00:20:26.508699+0000","last_peered":"2024-09-21T00:20:26.508699+0000","last_clean":"2024-09-21T00:20:26.508699+0000","last_became_active":"2024-09-21T00:18:38.662094+0000","last_became_peered":"2024-09-21T00:18:38.662094+0000","last_unstale":"2024-09-21T00:20:26.508699+0000","last_undegraded":"2024-09-21T00:20:26.508699+0000","last_fullsized":"2024-09-21T00:20:26.508699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:45.756641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574767+0000","last_change":"2024-09-21T00:18:38.663282+0000","last_active":"2024-09-21T00:20:26.574767+0000","last_peered":"2024-09-21T00:20:26.574767+0000","last_clean":"2024-09-21T00:20:26.574767+0000","last_became_active":"2024-09-21T00:18:38.662148+0000","last_became_peered":"2024-09-21T00:18:38.662148+0000","last_unstale":"2024-09-21T00:20:26.574767+0000","last_undegraded":"2024-09-21T00:20:26.574767+0000","last_fullsized":"2024-09-21T00:20:26.574767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:51.090591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509613+0000","last_change":"2024-09-21T00:18:38.669184+0000","last_active":"2024-09-21T00:20:26.509613+0000","last_peered":"2024-09-21T00:20:26.509613+0000","last_clean":"2024-09-21T00:20:26.509613+0000","last_became_active":"2024-09-21T00:18:38.668799+0000","last_became_peered":"2024-09-21T00:18:38.668799+0000","last_unstale":"2024-09-21T00:20:26.509613+0000","last_undegraded":"2024-09-21T00:20:26.509613+0000","last_fullsized":"2024-09-21T00:20:26.509613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:49.642568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508860+0000","last_change":"2024-09-21T00:18:38.679501+0000","last_active":"2024-09-21T00:20:26.508860+0000","last_peered":"2024-09-21T00:20:26.508860+0000","last_clean":"2024-09-21T00:20:26.508860+0000","last_became_active":"2024-09-21T00:18:38.678582+0000","last_became_peered":"2024-09-21T00:18:38.678582+0000","last_unstale":"2024-09-21T00:20:26.508860+0000","last_undegraded":"2024-09-21T00:20:26.508860+0000","last_fullsized":"2024-09-21T00:20:26.508860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:30.277979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454141+0000","last_change":"2024-09-21T00:18:38.659657+0000","last_active":"2024-09-21T00:20:26.454141+0000","last_peered":"2024-09-21T00:20:26.454141+0000","last_clean":"2024-09-21T00:20:26.454141+0000","last_became_active":"2024-09-21T00:18:38.659257+0000","last_became_peered":"2024-09-21T00:18:38.659257+0000","last_unstale":"2024-09-21T00:20:26.454141+0000","last_undegraded":"2024-09-21T00:20:26.454141+0000","last_fullsized":"2024-09-21T00:20:26.454141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:08.653729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452097+0000","last_change":"2024-09-21T00:18:38.671817+0000","last_active":"2024-09-21T00:20:26.452097+0000","last_peered":"2024-09-21T00:20:26.452097+0000","last_clean":"2024-09-21T00:20:26.452097+0000","last_became_active":"2024-09-21T00:18:38.671469+0000","last_became_peered":"2024-09-21T00:18:38.671469+0000","last_unstale":"2024-09-21T00:20:26.452097+0000","last_undegraded":"2024-09-21T00:20:26.452097+0000","last_fullsized":"2024-09-21T00:20:26.452097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:58:15.036623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451661+0000","last_change":"2024-09-21T00:18:38.664481+0000","last_active":"2024-09-21T00:20:26.451661+0000","last_peered":"2024-09-21T00:20:26.451661+0000","last_clean":"2024-09-21T00:20:26.451661+0000","last_became_active":"2024-09-21T00:18:38.664098+0000","last_became_peered":"2024-09-21T00:18:38.664098+0000","last_unstale":"2024-09-21T00:20:26.451661+0000","last_undegraded":"2024-09-21T00:20:26.451661+0000","last_fullsized":"2024-09-21T00:20:26.451661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:09.271251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455016+0000","last_change":"2024-09-21T00:18:38.662416+0000","last_active":"2024-09-21T00:20:26.455016+0000","last_peered":"2024-09-21T00:20:26.455016+0000","last_clean":"2024-09-21T00:20:26.455016+0000","last_became_active":"2024-09-21T00:18:38.662041+0000","last_became_peered":"2024-09-21T00:18:38.662041+0000","last_unstale":"2024-09-21T00:20:26.455016+0000","last_undegraded":"2024-09-21T00:20:26.455016+0000","last_fullsized":"2024-09-21T00:20:26.455016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:03.150179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508380+0000","last_change":"2024-09-21T00:18:38.670797+0000","last_active":"2024-09-21T00:20:26.508380+0000","last_peered":"2024-09-21T00:20:26.508380+0000","last_clean":"2024-09-21T00:20:26.508380+0000","last_became_active":"2024-09-21T00:18:38.670258+0000","last_became_peered":"2024-09-21T00:18:38.670258+0000","last_unstale":"2024-09-21T00:20:26.508380+0000","last_undegraded":"2024-09-21T00:20:26.508380+0000","last_fullsized":"2024-09-21T00:20:26.508380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:55.669836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454635+0000","last_change":"2024-09-21T00:18:38.665594+0000","last_active":"2024-09-21T00:20:26.454635+0000","last_peered":"2024-09-21T00:20:26.454635+0000","last_clean":"2024-09-21T00:20:26.454635+0000","last_became_active":"2024-09-21T00:18:38.665260+0000","last_became_peered":"2024-09-21T00:18:38.665260+0000","last_unstale":"2024-09-21T00:20:26.454635+0000","last_undegraded":"2024-09-21T00:20:26.454635+0000","last_fullsized":"2024-09-21T00:20:26.454635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:52:09.014327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454704+0000","last_change":"2024-09-21T00:18:38.679294+0000","last_active":"2024-09-21T00:20:26.454704+0000","last_peered":"2024-09-21T00:20:26.454704+0000","last_clean":"2024-09-21T00:20:26.454704+0000","last_became_active":"2024-09-21T00:18:38.673765+0000","last_became_peered":"2024-09-21T00:18:38.673765+0000","last_unstale":"2024-09-21T00:20:26.454704+0000","last_undegraded":"2024-09-21T00:20:26.454704+0000","last_fullsized":"2024-09-21T00:20:26.454704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:32.076299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508744+0000","last_change":"2024-09-21T00:18:38.667869+0000","last_active":"2024-09-21T00:20:26.508744+0000","last_peered":"2024-09-21T00:20:26.508744+0000","last_clean":"2024-09-21T00:20:26.508744+0000","last_became_active":"2024-09-21T00:18:38.667499+0000","last_became_peered":"2024-09-21T00:18:38.667499+0000","last_unstale":"2024-09-21T00:20:26.508744+0000","last_undegraded":"2024-09-21T00:20:26.508744+0000","last_fullsized":"2024-09-21T00:20:26.508744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:13.501043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452298+0000","last_change":"2024-09-21T00:18:38.663107+0000","last_active":"2024-09-21T00:20:26.452298+0000","last_peered":"2024-09-21T00:20:26.452298+0000","last_clean":"2024-09-21T00:20:26.452298+0000","last_became_active":"2024-09-21T00:18:38.662680+0000","last_became_peered":"2024-09-21T00:18:38.662680+0000","last_unstale":"2024-09-21T00:20:26.452298+0000","last_undegraded":"2024-09-21T00:20:26.452298+0000","last_fullsized":"2024-09-21T00:20:26.452298+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:48.154705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509579+0000","last_change":"2024-09-21T00:18:38.668167+0000","last_active":"2024-09-21T00:20:26.509579+0000","last_peered":"2024-09-21T00:20:26.509579+0000","last_clean":"2024-09-21T00:20:26.509579+0000","last_became_active":"2024-09-21T00:18:38.667842+0000","last_became_peered":"2024-09-21T00:18:38.667842+0000","last_unstale":"2024-09-21T00:20:26.509579+0000","last_undegraded":"2024-09-21T00:20:26.509579+0000","last_fullsized":"2024-09-21T00:20:26.509579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:55.358351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510744+0000","last_change":"2024-09-21T00:18:38.666580+0000","last_active":"2024-09-21T00:20:26.510744+0000","last_peered":"2024-09-21T00:20:26.510744+0000","last_clean":"2024-09-21T00:20:26.510744+0000","last_became_active":"2024-09-21T00:18:38.666231+0000","last_became_peered":"2024-09-21T00:18:38.666231+0000","last_unstale":"2024-09-21T00:20:26.510744+0000","last_undegraded":"2024-09-21T00:20:26.510744+0000","last_fullsized":"2024-09-21T00:20:26.510744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:07.362516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.512231+0000","last_change":"2024-09-21T00:20:26.511248+0000","last_active":"2024-09-21T00:20:26.512231+0000","last_peered":"2024-09-21T00:20:26.512231+0000","last_clean":"2024-09-21T00:20:26.512231+0000","last_became_active":"2024-09-21T00:20:26.510743+0000","last_became_peered":"2024-09-21T00:20:26.510743+0000","last_unstale":"2024-09-21T00:20:26.512231+0000","last_undegraded":"2024-09-21T00:20:26.512231+0000","last_fullsized":"2024-09-21T00:20:26.512231+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:41.577950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574484+0000","last_change":"2024-09-21T00:18:38.666580+0000","last_active":"2024-09-21T00:20:26.574484+0000","last_peered":"2024-09-21T00:20:26.574484+0000","last_clean":"2024-09-21T00:20:26.574484+0000","last_became_active":"2024-09-21T00:18:38.665698+0000","last_became_peered":"2024-09-21T00:18:38.665698+0000","last_unstale":"2024-09-21T00:20:26.574484+0000","last_undegraded":"2024-09-21T00:20:26.574484+0000","last_fullsized":"2024-09-21T00:20:26.574484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:45.627667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509070+0000","last_change":"2024-09-21T00:18:38.670729+0000","last_active":"2024-09-21T00:20:26.509070+0000","last_peered":"2024-09-21T00:20:26.509070+0000","last_clean":"2024-09-21T00:20:26.509070+0000","last_became_active":"2024-09-21T00:18:38.670337+0000","last_became_peered":"2024-09-21T00:18:38.670337+0000","last_unstale":"2024-09-21T00:20:26.509070+0000","last_undegraded":"2024-09-21T00:20:26.509070+0000","last_fullsized":"2024-09-21T00:20:26.509070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:49.880689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509695+0000","last_change":"2024-09-21T00:18:38.664644+0000","last_active":"2024-09-21T00:20:26.509695+0000","last_peered":"2024-09-21T00:20:26.509695+0000","last_clean":"2024-09-21T00:20:26.509695+0000","last_became_active":"2024-09-21T00:18:38.664220+0000","last_became_peered":"2024-09-21T00:18:38.664220+0000","last_unstale":"2024-09-21T00:20:26.509695+0000","last_undegraded":"2024-09-21T00:20:26.509695+0000","last_fullsized":"2024-09-21T00:20:26.509695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:09.610962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510095+0000","last_change":"2024-09-21T00:18:38.671592+0000","last_active":"2024-09-21T00:20:26.510095+0000","last_peered":"2024-09-21T00:20:26.510095+0000","last_clean":"2024-09-21T00:20:26.510095+0000","last_became_active":"2024-09-21T00:18:38.671228+0000","last_became_peered":"2024-09-21T00:18:38.671228+0000","last_unstale":"2024-09-21T00:20:26.510095+0000","last_undegraded":"2024-09-21T00:20:26.510095+0000","last_fullsized":"2024-09-21T00:20:26.510095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:51.906058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452615+0000","last_change":"2024-09-21T00:19:25.843696+0000","last_active":"2024-09-21T00:20:26.452615+0000","last_peered":"2024-09-21T00:20:26.452615+0000","last_clean":"2024-09-21T00:20:26.452615+0000","last_became_active":"2024-09-21T00:19:25.843287+0000","last_became_peered":"2024-09-21T00:19:25.843287+0000","last_unstale":"2024-09-21T00:20:26.452615+0000","last_undegraded":"2024-09-21T00:20:26.452615+0000","last_fullsized":"2024-09-21T00:20:26.452615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:21.024555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455481+0000","last_change":"2024-09-21T00:18:38.671820+0000","last_active":"2024-09-21T00:20:26.455481+0000","last_peered":"2024-09-21T00:20:26.455481+0000","last_clean":"2024-09-21T00:20:26.455481+0000","last_became_active":"2024-09-21T00:18:38.667107+0000","last_became_peered":"2024-09-21T00:18:38.667107+0000","last_unstale":"2024-09-21T00:20:26.455481+0000","last_undegraded":"2024-09-21T00:20:26.455481+0000","last_fullsized":"2024-09-21T00:20:26.455481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:58.099750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455903+0000","last_change":"2024-09-21T00:18:38.671271+0000","last_active":"2024-09-21T00:20:26.455903+0000","last_peered":"2024-09-21T00:20:26.455903+0000","last_clean":"2024-09-21T00:20:26.455903+0000","last_became_active":"2024-09-21T00:18:38.670826+0000","last_became_peered":"2024-09-21T00:18:38.670826+0000","last_unstale":"2024-09-21T00:20:26.455903+0000","last_undegraded":"2024-09-21T00:20:26.455903+0000","last_fullsized":"2024-09-21T00:20:26.455903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:57:01.723120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454997+0000","last_change":"2024-09-21T00:18:38.662851+0000","last_active":"2024-09-21T00:20:26.454997+0000","last_peered":"2024-09-21T00:20:26.454997+0000","last_clean":"2024-09-21T00:20:26.454997+0000","last_became_active":"2024-09-21T00:18:38.661259+0000","last_became_peered":"2024-09-21T00:18:38.661259+0000","last_unstale":"2024-09-21T00:20:26.454997+0000","last_undegraded":"2024-09-21T00:20:26.454997+0000","last_fullsized":"2024-09-21T00:20:26.454997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:10.508980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69660,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":417,"num_read_kb":280,"num_write":163,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3864,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":208980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678055,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":535425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874284,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37044,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708108,"statfs":{"total":95995035648,"available":95957102592,"internally_reserved":0,"allocated":1732608,"data_stored":1188330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037808,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1060864,"data_stored":539735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1052672,"data_stored":535537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430267,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1650688,"data_stored":1131468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1036288,"data_stored":530815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1658880,"data_stored":1127056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1126400,"data_stored":601706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1048576,"data_stored":531009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378713,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":531884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542238,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1105920,"data_stored":596384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705763,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1060864,"data_stored":532203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:20:44.092 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T00:20:44.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:44 smithi027 ceph-mon[30745]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:20:44.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:44 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:44.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:44 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:44.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:44 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:20:44.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:44 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:20:44.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:44 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:44.478 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:44 smithi089 ceph-mon[34481]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:20:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:44 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:44 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:44 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:20:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:44 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:20:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:44 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:44.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:44 smithi098 ceph-mon[34725]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:20:44.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:44 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:44.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:44 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:44.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:44 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:20:44.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:44 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:20:44.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:44 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:20:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.142+0000 7f1e58a00640 1 -- 172.21.15.27:0/3458022889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e541094a0 msgr2=0x7f1e541098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:45.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.142+0000 7f1e58a00640 1 --2- 172.21.15.27:0/3458022889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e541094a0 0x7f1e541098b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009900 tx=0x7f1e3c02a410 comp rx=0 tx=0).stop 2024-09-21T00:20:45.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.142+0000 7f1e58a00640 1 -- 172.21.15.27:0/3458022889 shutdown_connections 2024-09-21T00:20:45.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.142+0000 7f1e58a00640 1 --2- 172.21.15.27:0/3458022889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e54109df0 0x7f1e54110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:45.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.142+0000 7f1e58a00640 1 --2- 172.21.15.27:0/3458022889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e541094a0 0x7f1e541098b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:45.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.142+0000 7f1e58a00640 1 --2- 172.21.15.27:0/3458022889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e54103b60 0x7f1e54103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:45.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.142+0000 7f1e58a00640 1 -- 172.21.15.27:0/3458022889 >> 172.21.15.27:0/3458022889 conn(0x7f1e5407b850 msgr2=0x7f1e5407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:45.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.142+0000 7f1e58a00640 1 -- 172.21.15.27:0/3458022889 shutdown_connections 2024-09-21T00:20:45.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.142+0000 7f1e58a00640 1 -- 172.21.15.27:0/3458022889 wait complete. 2024-09-21T00:20:45.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.143+0000 7f1e58a00640 1 Processor -- start 2024-09-21T00:20:45.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.143+0000 7f1e58a00640 1 -- start start 2024-09-21T00:20:45.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.144+0000 7f1e58a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e54103b60 0x7f1e54075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:45.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.144+0000 7f1e58a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e541094a0 0x7f1e54075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:45.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.144+0000 7f1e58a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e54109df0 0x7f1e54076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:45.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.144+0000 7f1e58a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1e54113ab0 con 0x7f1e541094a0 2024-09-21T00:20:45.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.144+0000 7f1e58a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1e54113930 con 0x7f1e54109df0 2024-09-21T00:20:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.144+0000 7f1e58a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1e54113c30 con 0x7f1e54103b60 2024-09-21T00:20:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.144+0000 7f1e4be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e541094a0 0x7f1e54075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.144+0000 7f1e51e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e54103b60 0x7f1e54075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.144+0000 7f1e52800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e54109df0 0x7f1e54076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.144+0000 7f1e52800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e54109df0 0x7f1e54076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54946/0 (socket says 172.21.15.27:54946) 2024-09-21T00:20:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.144+0000 7f1e51e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e54103b60 0x7f1e54075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:50378/0 (socket says 172.21.15.27:50378) 2024-09-21T00:20:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.144+0000 7f1e52800640 1 -- 172.21.15.27:0/2385767294 learned_addr learned my addr 172.21.15.27:0/2385767294 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.145+0000 7f1e52800640 1 -- 172.21.15.27:0/2385767294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e54103b60 msgr2=0x7f1e54075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.145+0000 7f1e52800640 1 --2- 172.21.15.27:0/2385767294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e54103b60 0x7f1e54075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.145+0000 7f1e52800640 1 -- 172.21.15.27:0/2385767294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e541094a0 msgr2=0x7f1e54075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.145+0000 7f1e52800640 1 --2- 172.21.15.27:0/2385767294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e541094a0 0x7f1e54075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.145+0000 7f1e52800640 1 -- 172.21.15.27:0/2385767294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e541a8820 con 0x7f1e54109df0 2024-09-21T00:20:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.145+0000 7f1e52800640 1 --2- 172.21.15.27:0/2385767294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e54109df0 0x7f1e54076190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00eb10 tx=0x7f1e4c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.145+0000 7f1e4aa00640 1 -- 172.21.15.27:0/2385767294 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c004b20 con 0x7f1e54109df0 2024-09-21T00:20:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.145+0000 7f1e58a00640 1 -- 172.21.15.27:0/2385767294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e541a8b10 con 0x7f1e54109df0 2024-09-21T00:20:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.145+0000 7f1e4aa00640 1 -- 172.21.15.27:0/2385767294 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4c004cc0 con 0x7f1e54109df0 2024-09-21T00:20:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.145+0000 7f1e4aa00640 1 -- 172.21.15.27:0/2385767294 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c010670 con 0x7f1e54109df0 2024-09-21T00:20:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.146+0000 7f1e58a00640 1 -- 172.21.15.27:0/2385767294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e541a9020 con 0x7f1e54109df0 2024-09-21T00:20:45.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.147+0000 7f1e58a00640 1 -- 172.21.15.27:0/2385767294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e54103f70 con 0x7f1e54109df0 2024-09-21T00:20:45.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.149+0000 7f1e4aa00640 1 -- 172.21.15.27:0/2385767294 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f1e4c005000 con 0x7f1e54109df0 2024-09-21T00:20:45.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.151+0000 7f1e4aa00640 1 --2- 172.21.15.27:0/2385767294 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1e380b65f0 0x7f1e380b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:45.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.151+0000 7f1e4aa00640 1 -- 172.21.15.27:0/2385767294 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f1e4c0e8120 con 0x7f1e54109df0 2024-09-21T00:20:45.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.151+0000 7f1e51e00640 1 --2- 172.21.15.27:0/2385767294 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1e380b65f0 0x7f1e380b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:45.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.152+0000 7f1e51e00640 1 --2- 172.21.15.27:0/2385767294 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1e380b65f0 0x7f1e380b8b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1e54077160 tx=0x7f1e4400b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:45.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.152+0000 7f1e4aa00640 1 -- 172.21.15.27:0/2385767294 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1e4c056710 con 0x7f1e54109df0 2024-09-21T00:20:45.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:45 smithi027 ceph-mon[30745]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:20:45.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.350+0000 7f1e58a00640 1 -- 172.21.15.27:0/2385767294 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e54063800 con 0x7f1e380b65f0 2024-09-21T00:20:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.356+0000 7f1e4aa00640 1 -- 172.21.15.27:0/2385767294 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340367 (secure 0 0 0) 0x7f1e54063800 con 0x7f1e380b65f0 2024-09-21T00:20:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:20:45.363 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T00:20:45.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.361+0000 7f1e58a00640 1 -- 172.21.15.27:0/2385767294 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1e380b65f0 msgr2=0x7f1e380b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:45.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.361+0000 7f1e58a00640 1 --2- 172.21.15.27:0/2385767294 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1e380b65f0 0x7f1e380b8b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1e54077160 tx=0x7f1e4400b410 comp rx=0 tx=0).stop 2024-09-21T00:20:45.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.361+0000 7f1e58a00640 1 -- 172.21.15.27:0/2385767294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e54109df0 msgr2=0x7f1e54076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:45.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.361+0000 7f1e58a00640 1 --2- 172.21.15.27:0/2385767294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e54109df0 0x7f1e54076190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00eb10 tx=0x7f1e4c00c2d0 comp rx=0 tx=0).stop 2024-09-21T00:20:45.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.361+0000 7f1e58a00640 1 -- 172.21.15.27:0/2385767294 shutdown_connections 2024-09-21T00:20:45.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.361+0000 7f1e58a00640 1 --2- 172.21.15.27:0/2385767294 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1e380b65f0 0x7f1e380b8b00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:45.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.361+0000 7f1e58a00640 1 --2- 172.21.15.27:0/2385767294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e54109df0 0x7f1e54076190 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:45.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.361+0000 7f1e58a00640 1 --2- 172.21.15.27:0/2385767294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e541094a0 0x7f1e54075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:45.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.361+0000 7f1e58a00640 1 --2- 172.21.15.27:0/2385767294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e54103b60 0x7f1e54075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:45.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.361+0000 7f1e58a00640 1 -- 172.21.15.27:0/2385767294 >> 172.21.15.27:0/2385767294 conn(0x7f1e5407b850 msgr2=0x7f1e541075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:45.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.361+0000 7f1e58a00640 1 -- 172.21.15.27:0/2385767294 shutdown_connections 2024-09-21T00:20:45.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:45.361+0000 7f1e58a00640 1 -- 172.21.15.27:0/2385767294 wait complete. 2024-09-21T00:20:45.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:45 smithi089 ceph-mon[34481]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:20:45.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:45 smithi098 ceph-mon[34725]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:20:46.130 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-21T00:20:44.362677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660262,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":503,"num_read_kb":354,"num_write":288,"num_write_kb":2270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2955704,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380268,"kb_used_data":14288,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561556,"statfs":{"total":1151940427776,"available":1151551033344,"internally_reserved":0,"allocated":14630912,"data_stored":8381552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003671"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453054+0000","last_change":"2024-09-21T00:18:35.627618+0000","last_active":"2024-09-21T00:20:26.453054+0000","last_peered":"2024-09-21T00:20:26.453054+0000","last_clean":"2024-09-21T00:20:26.453054+0000","last_became_active":"2024-09-21T00:18:35.626475+0000","last_became_peered":"2024-09-21T00:18:35.626475+0000","last_unstale":"2024-09-21T00:20:26.453054+0000","last_undegraded":"2024-09-21T00:20:26.453054+0000","last_fullsized":"2024-09-21T00:20:26.453054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:50.379225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.575104+0000","last_change":"2024-09-21T00:18:38.665891+0000","last_active":"2024-09-21T00:20:26.575104+0000","last_peered":"2024-09-21T00:20:26.575104+0000","last_clean":"2024-09-21T00:20:26.575104+0000","last_became_active":"2024-09-21T00:18:38.665472+0000","last_became_peered":"2024-09-21T00:18:38.665472+0000","last_unstale":"2024-09-21T00:20:26.575104+0000","last_undegraded":"2024-09-21T00:20:26.575104+0000","last_fullsized":"2024-09-21T00:20:26.575104+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:34:32.490828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451354+0000","last_change":"2024-09-21T00:18:48.730206+0000","last_active":"2024-09-21T00:20:26.451354+0000","last_peered":"2024-09-21T00:20:26.451354+0000","last_clean":"2024-09-21T00:20:26.451354+0000","last_became_active":"2024-09-21T00:18:48.729729+0000","last_became_peered":"2024-09-21T00:18:48.729729+0000","last_unstale":"2024-09-21T00:20:26.451354+0000","last_undegraded":"2024-09-21T00:20:26.451354+0000","last_fullsized":"2024-09-21T00:20:26.451354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:08:17.019877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.575433+0000","last_change":"2024-09-21T00:18:35.768554+0000","last_active":"2024-09-21T00:20:26.575433+0000","last_peered":"2024-09-21T00:20:26.575433+0000","last_clean":"2024-09-21T00:20:26.575433+0000","last_became_active":"2024-09-21T00:18:35.767642+0000","last_became_peered":"2024-09-21T00:18:35.767642+0000","last_unstale":"2024-09-21T00:20:26.575433+0000","last_undegraded":"2024-09-21T00:20:26.575433+0000","last_fullsized":"2024-09-21T00:20:26.575433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:10.442974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455809+0000","last_change":"2024-09-21T00:18:38.677997+0000","last_active":"2024-09-21T00:20:26.455809+0000","last_peered":"2024-09-21T00:20:26.455809+0000","last_clean":"2024-09-21T00:20:26.455809+0000","last_became_active":"2024-09-21T00:18:38.676698+0000","last_became_peered":"2024-09-21T00:18:38.676698+0000","last_unstale":"2024-09-21T00:20:26.455809+0000","last_undegraded":"2024-09-21T00:20:26.455809+0000","last_fullsized":"2024-09-21T00:20:26.455809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:11.662765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451608+0000","last_change":"2024-09-21T00:18:48.739563+0000","last_active":"2024-09-21T00:20:26.451608+0000","last_peered":"2024-09-21T00:20:26.451608+0000","last_clean":"2024-09-21T00:20:26.451608+0000","last_became_active":"2024-09-21T00:18:48.739092+0000","last_became_peered":"2024-09-21T00:18:48.739092+0000","last_unstale":"2024-09-21T00:20:26.451608+0000","last_undegraded":"2024-09-21T00:20:26.451608+0000","last_fullsized":"2024-09-21T00:20:26.451608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:44.185556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510905+0000","last_change":"2024-09-21T00:18:35.626742+0000","last_active":"2024-09-21T00:20:26.510905+0000","last_peered":"2024-09-21T00:20:26.510905+0000","last_clean":"2024-09-21T00:20:26.510905+0000","last_became_active":"2024-09-21T00:18:35.626359+0000","last_became_peered":"2024-09-21T00:18:35.626359+0000","last_unstale":"2024-09-21T00:20:26.510905+0000","last_undegraded":"2024-09-21T00:20:26.510905+0000","last_fullsized":"2024-09-21T00:20:26.510905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:38.860184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510679+0000","last_change":"2024-09-21T00:18:38.666003+0000","last_active":"2024-09-21T00:20:26.510679+0000","last_peered":"2024-09-21T00:20:26.510679+0000","last_clean":"2024-09-21T00:20:26.510679+0000","last_became_active":"2024-09-21T00:18:38.665425+0000","last_became_peered":"2024-09-21T00:18:38.665425+0000","last_unstale":"2024-09-21T00:20:26.510679+0000","last_undegraded":"2024-09-21T00:20:26.510679+0000","last_fullsized":"2024-09-21T00:20:26.510679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:14.121864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.576823+0000","last_change":"2024-09-21T00:20:26.576748+0000","last_active":"2024-09-21T00:20:26.576823+0000","last_peered":"2024-09-21T00:20:26.576823+0000","last_clean":"2024-09-21T00:20:26.576823+0000","last_became_active":"2024-09-21T00:20:26.576186+0000","last_became_peered":"2024-09-21T00:20:26.576186+0000","last_unstale":"2024-09-21T00:20:26.576823+0000","last_undegraded":"2024-09-21T00:20:26.576823+0000","last_fullsized":"2024-09-21T00:20:26.576823+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:20.283717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510591+0000","last_change":"2024-09-21T00:18:35.628091+0000","last_active":"2024-09-21T00:20:26.510591+0000","last_peered":"2024-09-21T00:20:26.510591+0000","last_clean":"2024-09-21T00:20:26.510591+0000","last_became_active":"2024-09-21T00:18:35.627654+0000","last_became_peered":"2024-09-21T00:18:35.627654+0000","last_unstale":"2024-09-21T00:20:26.510591+0000","last_undegraded":"2024-09-21T00:20:26.510591+0000","last_fullsized":"2024-09-21T00:20:26.510591+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:05.229036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509097+0000","last_change":"2024-09-21T00:19:25.844116+0000","last_active":"2024-09-21T00:20:26.509097+0000","last_peered":"2024-09-21T00:20:26.509097+0000","last_clean":"2024-09-21T00:20:26.509097+0000","last_became_active":"2024-09-21T00:19:25.842745+0000","last_became_peered":"2024-09-21T00:19:25.842745+0000","last_unstale":"2024-09-21T00:20:26.509097+0000","last_undegraded":"2024-09-21T00:20:26.509097+0000","last_fullsized":"2024-09-21T00:20:26.509097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:42.651251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451746+0000","last_change":"2024-09-21T00:18:48.740106+0000","last_active":"2024-09-21T00:20:26.451746+0000","last_peered":"2024-09-21T00:20:26.451746+0000","last_clean":"2024-09-21T00:20:26.451746+0000","last_became_active":"2024-09-21T00:18:48.739299+0000","last_became_peered":"2024-09-21T00:18:48.739299+0000","last_unstale":"2024-09-21T00:20:26.451746+0000","last_undegraded":"2024-09-21T00:20:26.451746+0000","last_fullsized":"2024-09-21T00:20:26.451746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:59.058333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509916+0000","last_change":"2024-09-21T00:18:38.667331+0000","last_active":"2024-09-21T00:20:26.509916+0000","last_peered":"2024-09-21T00:20:26.509916+0000","last_clean":"2024-09-21T00:20:26.509916+0000","last_became_active":"2024-09-21T00:18:38.666960+0000","last_became_peered":"2024-09-21T00:18:38.666960+0000","last_unstale":"2024-09-21T00:20:26.509916+0000","last_undegraded":"2024-09-21T00:20:26.509916+0000","last_fullsized":"2024-09-21T00:20:26.509916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:57.569460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510258+0000","last_change":"2024-09-21T00:18:35.625570+0000","last_active":"2024-09-21T00:20:26.510258+0000","last_peered":"2024-09-21T00:20:26.510258+0000","last_clean":"2024-09-21T00:20:26.510258+0000","last_became_active":"2024-09-21T00:18:35.625170+0000","last_became_peered":"2024-09-21T00:18:35.625170+0000","last_unstale":"2024-09-21T00:20:26.510258+0000","last_undegraded":"2024-09-21T00:20:26.510258+0000","last_fullsized":"2024-09-21T00:20:26.510258+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:58.614258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452011+0000","last_change":"2024-09-21T00:18:48.727390+0000","last_active":"2024-09-21T00:20:26.452011+0000","last_peered":"2024-09-21T00:20:26.452011+0000","last_clean":"2024-09-21T00:20:26.452011+0000","last_became_active":"2024-09-21T00:18:48.726864+0000","last_became_peered":"2024-09-21T00:18:48.726864+0000","last_unstale":"2024-09-21T00:20:26.452011+0000","last_undegraded":"2024-09-21T00:20:26.452011+0000","last_fullsized":"2024-09-21T00:20:26.452011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:10.917069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574789+0000","last_change":"2024-09-21T00:18:38.664072+0000","last_active":"2024-09-21T00:20:26.574789+0000","last_peered":"2024-09-21T00:20:26.574789+0000","last_clean":"2024-09-21T00:20:26.574789+0000","last_became_active":"2024-09-21T00:18:38.662562+0000","last_became_peered":"2024-09-21T00:18:38.662562+0000","last_unstale":"2024-09-21T00:20:26.574789+0000","last_undegraded":"2024-09-21T00:20:26.574789+0000","last_fullsized":"2024-09-21T00:20:26.574789+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.021174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453693+0000","last_change":"2024-09-21T00:19:25.843599+0000","last_active":"2024-09-21T00:20:26.453693+0000","last_peered":"2024-09-21T00:20:26.453693+0000","last_clean":"2024-09-21T00:20:26.453693+0000","last_became_active":"2024-09-21T00:19:25.843178+0000","last_became_peered":"2024-09-21T00:19:25.843178+0000","last_unstale":"2024-09-21T00:20:26.453693+0000","last_undegraded":"2024-09-21T00:20:26.453693+0000","last_fullsized":"2024-09-21T00:20:26.453693+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:18.809857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452048+0000","last_change":"2024-09-21T00:18:48.729940+0000","last_active":"2024-09-21T00:20:26.452048+0000","last_peered":"2024-09-21T00:20:26.452048+0000","last_clean":"2024-09-21T00:20:26.452048+0000","last_became_active":"2024-09-21T00:18:48.729465+0000","last_became_peered":"2024-09-21T00:18:48.729465+0000","last_unstale":"2024-09-21T00:20:26.452048+0000","last_undegraded":"2024-09-21T00:20:26.452048+0000","last_fullsized":"2024-09-21T00:20:26.452048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:44.566386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453715+0000","last_change":"2024-09-21T00:18:38.662510+0000","last_active":"2024-09-21T00:20:26.453715+0000","last_peered":"2024-09-21T00:20:26.453715+0000","last_clean":"2024-09-21T00:20:26.453715+0000","last_became_active":"2024-09-21T00:18:38.659788+0000","last_became_peered":"2024-09-21T00:18:38.659788+0000","last_unstale":"2024-09-21T00:20:26.453715+0000","last_undegraded":"2024-09-21T00:20:26.453715+0000","last_fullsized":"2024-09-21T00:20:26.453715+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:14.532084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574979+0000","last_change":"2024-09-21T00:18:35.622949+0000","last_active":"2024-09-21T00:20:26.574979+0000","last_peered":"2024-09-21T00:20:26.574979+0000","last_clean":"2024-09-21T00:20:26.574979+0000","last_became_active":"2024-09-21T00:18:35.621787+0000","last_became_peered":"2024-09-21T00:18:35.621787+0000","last_unstale":"2024-09-21T00:20:26.574979+0000","last_undegraded":"2024-09-21T00:20:26.574979+0000","last_fullsized":"2024-09-21T00:20:26.574979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:39:15.060123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452419+0000","last_change":"2024-09-21T00:18:48.730711+0000","last_active":"2024-09-21T00:20:26.452419+0000","last_peered":"2024-09-21T00:20:26.452419+0000","last_clean":"2024-09-21T00:20:26.452419+0000","last_became_active":"2024-09-21T00:18:48.730287+0000","last_became_peered":"2024-09-21T00:18:48.730287+0000","last_unstale":"2024-09-21T00:20:26.452419+0000","last_undegraded":"2024-09-21T00:20:26.452419+0000","last_fullsized":"2024-09-21T00:20:26.452419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:34.444747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510047+0000","last_change":"2024-09-21T00:18:38.669381+0000","last_active":"2024-09-21T00:20:26.510047+0000","last_peered":"2024-09-21T00:20:26.510047+0000","last_clean":"2024-09-21T00:20:26.510047+0000","last_became_active":"2024-09-21T00:18:38.668757+0000","last_became_peered":"2024-09-21T00:18:38.668757+0000","last_unstale":"2024-09-21T00:20:26.510047+0000","last_undegraded":"2024-09-21T00:20:26.510047+0000","last_fullsized":"2024-09-21T00:20:26.510047+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:52.475872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508528+0000","last_change":"2024-09-21T00:18:35.767700+0000","last_active":"2024-09-21T00:20:26.508528+0000","last_peered":"2024-09-21T00:20:26.508528+0000","last_clean":"2024-09-21T00:20:26.508528+0000","last_became_active":"2024-09-21T00:18:35.767369+0000","last_became_peered":"2024-09-21T00:18:35.767369+0000","last_unstale":"2024-09-21T00:20:26.508528+0000","last_undegraded":"2024-09-21T00:20:26.508528+0000","last_fullsized":"2024-09-21T00:20:26.508528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:44.955659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452223+0000","last_change":"2024-09-21T00:18:48.737411+0000","last_active":"2024-09-21T00:20:26.452223+0000","last_peered":"2024-09-21T00:20:26.452223+0000","last_clean":"2024-09-21T00:20:26.452223+0000","last_became_active":"2024-09-21T00:18:48.736715+0000","last_became_peered":"2024-09-21T00:18:48.736715+0000","last_unstale":"2024-09-21T00:20:26.452223+0000","last_undegraded":"2024-09-21T00:20:26.452223+0000","last_fullsized":"2024-09-21T00:20:26.452223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:01.045318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455965+0000","last_change":"2024-09-21T00:19:25.842355+0000","last_active":"2024-09-21T00:20:26.455965+0000","last_peered":"2024-09-21T00:20:26.455965+0000","last_clean":"2024-09-21T00:20:26.455965+0000","last_became_active":"2024-09-21T00:19:25.841961+0000","last_became_peered":"2024-09-21T00:19:25.841961+0000","last_unstale":"2024-09-21T00:20:26.455965+0000","last_undegraded":"2024-09-21T00:20:26.455965+0000","last_fullsized":"2024-09-21T00:20:26.455965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:30.490992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452433+0000","last_change":"2024-09-21T00:18:35.919003+0000","last_active":"2024-09-21T00:20:26.452433+0000","last_peered":"2024-09-21T00:20:26.452433+0000","last_clean":"2024-09-21T00:20:26.452433+0000","last_became_active":"2024-09-21T00:18:35.918593+0000","last_became_peered":"2024-09-21T00:18:35.918593+0000","last_unstale":"2024-09-21T00:20:26.452433+0000","last_undegraded":"2024-09-21T00:20:26.452433+0000","last_fullsized":"2024-09-21T00:20:26.452433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:35.982726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451474+0000","last_change":"2024-09-21T00:18:48.725906+0000","last_active":"2024-09-21T00:20:26.451474+0000","last_peered":"2024-09-21T00:20:26.451474+0000","last_clean":"2024-09-21T00:20:26.451474+0000","last_became_active":"2024-09-21T00:18:48.725450+0000","last_became_peered":"2024-09-21T00:18:48.725450+0000","last_unstale":"2024-09-21T00:20:26.451474+0000","last_undegraded":"2024-09-21T00:20:26.451474+0000","last_fullsized":"2024-09-21T00:20:26.451474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:35.163208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.575296+0000","last_change":"2024-09-21T00:18:38.666442+0000","last_active":"2024-09-21T00:20:26.575296+0000","last_peered":"2024-09-21T00:20:26.575296+0000","last_clean":"2024-09-21T00:20:26.575296+0000","last_became_active":"2024-09-21T00:18:38.666083+0000","last_became_peered":"2024-09-21T00:18:38.666083+0000","last_unstale":"2024-09-21T00:20:26.575296+0000","last_undegraded":"2024-09-21T00:20:26.575296+0000","last_fullsized":"2024-09-21T00:20:26.575296+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:39.986440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451791+0000","last_change":"2024-09-21T00:18:35.624883+0000","last_active":"2024-09-21T00:20:26.451791+0000","last_peered":"2024-09-21T00:20:26.451791+0000","last_clean":"2024-09-21T00:20:26.451791+0000","last_became_active":"2024-09-21T00:18:35.624382+0000","last_became_peered":"2024-09-21T00:18:35.624382+0000","last_unstale":"2024-09-21T00:20:26.451791+0000","last_undegraded":"2024-09-21T00:20:26.451791+0000","last_fullsized":"2024-09-21T00:20:26.451791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:23.475521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574380+0000","last_change":"2024-09-21T00:18:48.719850+0000","last_active":"2024-09-21T00:20:26.574380+0000","last_peered":"2024-09-21T00:20:26.574380+0000","last_clean":"2024-09-21T00:20:26.574380+0000","last_became_active":"2024-09-21T00:18:48.719414+0000","last_became_peered":"2024-09-21T00:18:48.719414+0000","last_unstale":"2024-09-21T00:20:26.574380+0000","last_undegraded":"2024-09-21T00:20:26.574380+0000","last_fullsized":"2024-09-21T00:20:26.574380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:05.978977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509131+0000","last_change":"2024-09-21T00:18:38.671920+0000","last_active":"2024-09-21T00:20:26.509131+0000","last_peered":"2024-09-21T00:20:26.509131+0000","last_clean":"2024-09-21T00:20:26.509131+0000","last_became_active":"2024-09-21T00:18:38.671592+0000","last_became_peered":"2024-09-21T00:18:38.671592+0000","last_unstale":"2024-09-21T00:20:26.509131+0000","last_undegraded":"2024-09-21T00:20:26.509131+0000","last_fullsized":"2024-09-21T00:20:26.509131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:40:48.257086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.511127+0000","last_change":"2024-09-21T00:18:35.769197+0000","last_active":"2024-09-21T00:20:26.511127+0000","last_peered":"2024-09-21T00:20:26.511127+0000","last_clean":"2024-09-21T00:20:26.511127+0000","last_became_active":"2024-09-21T00:18:35.768740+0000","last_became_peered":"2024-09-21T00:18:35.768740+0000","last_unstale":"2024-09-21T00:20:26.511127+0000","last_undegraded":"2024-09-21T00:20:26.511127+0000","last_fullsized":"2024-09-21T00:20:26.511127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:32.253006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454032+0000","last_change":"2024-09-21T00:18:48.740189+0000","last_active":"2024-09-21T00:20:26.454032+0000","last_peered":"2024-09-21T00:20:26.454032+0000","last_clean":"2024-09-21T00:20:26.454032+0000","last_became_active":"2024-09-21T00:18:48.739700+0000","last_became_peered":"2024-09-21T00:18:48.739700+0000","last_unstale":"2024-09-21T00:20:26.454032+0000","last_undegraded":"2024-09-21T00:20:26.454032+0000","last_fullsized":"2024-09-21T00:20:26.454032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:06.312160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509517+0000","last_change":"2024-09-21T00:18:38.665635+0000","last_active":"2024-09-21T00:20:26.509517+0000","last_peered":"2024-09-21T00:20:26.509517+0000","last_clean":"2024-09-21T00:20:26.509517+0000","last_became_active":"2024-09-21T00:18:38.665236+0000","last_became_peered":"2024-09-21T00:18:38.665236+0000","last_unstale":"2024-09-21T00:20:26.509517+0000","last_undegraded":"2024-09-21T00:20:26.509517+0000","last_fullsized":"2024-09-21T00:20:26.509517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:52.186480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452058+0000","last_change":"2024-09-21T00:18:35.771465+0000","last_active":"2024-09-21T00:20:26.452058+0000","last_peered":"2024-09-21T00:20:26.452058+0000","last_clean":"2024-09-21T00:20:26.452058+0000","last_became_active":"2024-09-21T00:18:35.770550+0000","last_became_peered":"2024-09-21T00:18:35.770550+0000","last_unstale":"2024-09-21T00:20:26.452058+0000","last_undegraded":"2024-09-21T00:20:26.452058+0000","last_fullsized":"2024-09-21T00:20:26.452058+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:43.042563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510004+0000","last_change":"2024-09-21T00:18:48.736334+0000","last_active":"2024-09-21T00:20:26.510004+0000","last_peered":"2024-09-21T00:20:26.510004+0000","last_clean":"2024-09-21T00:20:26.510004+0000","last_became_active":"2024-09-21T00:18:48.735784+0000","last_became_peered":"2024-09-21T00:18:48.735784+0000","last_unstale":"2024-09-21T00:20:26.510004+0000","last_undegraded":"2024-09-21T00:20:26.510004+0000","last_fullsized":"2024-09-21T00:20:26.510004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:29.637335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510185+0000","last_change":"2024-09-21T00:18:38.674181+0000","last_active":"2024-09-21T00:20:26.510185+0000","last_peered":"2024-09-21T00:20:26.510185+0000","last_clean":"2024-09-21T00:20:26.510185+0000","last_became_active":"2024-09-21T00:18:38.673784+0000","last_became_peered":"2024-09-21T00:18:38.673784+0000","last_unstale":"2024-09-21T00:20:26.510185+0000","last_undegraded":"2024-09-21T00:20:26.510185+0000","last_fullsized":"2024-09-21T00:20:26.510185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:33.471328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510165+0000","last_change":"2024-09-21T00:18:35.628149+0000","last_active":"2024-09-21T00:20:26.510165+0000","last_peered":"2024-09-21T00:20:26.510165+0000","last_clean":"2024-09-21T00:20:26.510165+0000","last_became_active":"2024-09-21T00:18:35.627674+0000","last_became_peered":"2024-09-21T00:18:35.627674+0000","last_unstale":"2024-09-21T00:20:26.510165+0000","last_undegraded":"2024-09-21T00:20:26.510165+0000","last_fullsized":"2024-09-21T00:20:26.510165+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:28:47.244288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508830+0000","last_change":"2024-09-21T00:18:48.740283+0000","last_active":"2024-09-21T00:20:26.508830+0000","last_peered":"2024-09-21T00:20:26.508830+0000","last_clean":"2024-09-21T00:20:26.508830+0000","last_became_active":"2024-09-21T00:18:48.739026+0000","last_became_peered":"2024-09-21T00:18:48.739026+0000","last_unstale":"2024-09-21T00:20:26.508830+0000","last_undegraded":"2024-09-21T00:20:26.508830+0000","last_fullsized":"2024-09-21T00:20:26.508830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:50.898116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508762+0000","last_change":"2024-09-21T00:18:38.667206+0000","last_active":"2024-09-21T00:20:26.508762+0000","last_peered":"2024-09-21T00:20:26.508762+0000","last_clean":"2024-09-21T00:20:26.508762+0000","last_became_active":"2024-09-21T00:18:38.666770+0000","last_became_peered":"2024-09-21T00:18:38.666770+0000","last_unstale":"2024-09-21T00:20:26.508762+0000","last_undegraded":"2024-09-21T00:20:26.508762+0000","last_fullsized":"2024-09-21T00:20:26.508762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:56.332071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":117,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:43.028574+0000","last_change":"2024-09-21T00:18:35.768012+0000","last_active":"2024-09-21T00:20:43.028574+0000","last_peered":"2024-09-21T00:20:43.028574+0000","last_clean":"2024-09-21T00:20:43.028574+0000","last_became_active":"2024-09-21T00:18:35.767438+0000","last_became_peered":"2024-09-21T00:18:35.767438+0000","last_unstale":"2024-09-21T00:20:43.028574+0000","last_undegraded":"2024-09-21T00:20:43.028574+0000","last_fullsized":"2024-09-21T00:20:43.028574+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:35:40.606871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508575+0000","last_change":"2024-09-21T00:18:48.738228+0000","last_active":"2024-09-21T00:20:26.508575+0000","last_peered":"2024-09-21T00:20:26.508575+0000","last_clean":"2024-09-21T00:20:26.508575+0000","last_became_active":"2024-09-21T00:18:48.737786+0000","last_became_peered":"2024-09-21T00:18:48.737786+0000","last_unstale":"2024-09-21T00:20:26.508575+0000","last_undegraded":"2024-09-21T00:20:26.508575+0000","last_fullsized":"2024-09-21T00:20:26.508575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:52.329677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508322+0000","last_change":"2024-09-21T00:18:38.678126+0000","last_active":"2024-09-21T00:20:26.508322+0000","last_peered":"2024-09-21T00:20:26.508322+0000","last_clean":"2024-09-21T00:20:26.508322+0000","last_became_active":"2024-09-21T00:18:38.677757+0000","last_became_peered":"2024-09-21T00:18:38.677757+0000","last_unstale":"2024-09-21T00:20:26.508322+0000","last_undegraded":"2024-09-21T00:20:26.508322+0000","last_fullsized":"2024-09-21T00:20:26.508322+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:26.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508836+0000","last_change":"2024-09-21T00:18:35.625955+0000","last_active":"2024-09-21T00:20:26.508836+0000","last_peered":"2024-09-21T00:20:26.508836+0000","last_clean":"2024-09-21T00:20:26.508836+0000","last_became_active":"2024-09-21T00:18:35.625548+0000","last_became_peered":"2024-09-21T00:18:35.625548+0000","last_unstale":"2024-09-21T00:20:26.508836+0000","last_undegraded":"2024-09-21T00:20:26.508836+0000","last_fullsized":"2024-09-21T00:20:26.508836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:04.163421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509284+0000","last_change":"2024-09-21T00:18:48.740765+0000","last_active":"2024-09-21T00:20:26.509284+0000","last_peered":"2024-09-21T00:20:26.509284+0000","last_clean":"2024-09-21T00:20:26.509284+0000","last_became_active":"2024-09-21T00:18:48.740081+0000","last_became_peered":"2024-09-21T00:18:48.740081+0000","last_unstale":"2024-09-21T00:20:26.509284+0000","last_undegraded":"2024-09-21T00:20:26.509284+0000","last_fullsized":"2024-09-21T00:20:26.509284+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:38.251475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574852+0000","last_change":"2024-09-21T00:18:38.667581+0000","last_active":"2024-09-21T00:20:26.574852+0000","last_peered":"2024-09-21T00:20:26.574852+0000","last_clean":"2024-09-21T00:20:26.574852+0000","last_became_active":"2024-09-21T00:18:38.665959+0000","last_became_peered":"2024-09-21T00:18:38.665959+0000","last_unstale":"2024-09-21T00:20:26.574852+0000","last_undegraded":"2024-09-21T00:20:26.574852+0000","last_fullsized":"2024-09-21T00:20:26.574852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:52.321654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452596+0000","last_change":"2024-09-21T00:18:35.624291+0000","last_active":"2024-09-21T00:20:26.452596+0000","last_peered":"2024-09-21T00:20:26.452596+0000","last_clean":"2024-09-21T00:20:26.452596+0000","last_became_active":"2024-09-21T00:18:35.623297+0000","last_became_peered":"2024-09-21T00:18:35.623297+0000","last_unstale":"2024-09-21T00:20:26.452596+0000","last_undegraded":"2024-09-21T00:20:26.452596+0000","last_fullsized":"2024-09-21T00:20:26.452596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:44.824409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509191+0000","last_change":"2024-09-21T00:17:21.233627+0000","last_active":"2024-09-21T00:20:26.509191+0000","last_peered":"2024-09-21T00:20:26.509191+0000","last_clean":"2024-09-21T00:20:26.509191+0000","last_became_active":"2024-09-21T00:17:21.227897+0000","last_became_peered":"2024-09-21T00:17:21.227897+0000","last_unstale":"2024-09-21T00:20:26.509191+0000","last_undegraded":"2024-09-21T00:20:26.509191+0000","last_fullsized":"2024-09-21T00:20:26.509191+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:13:33.120691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:13:33.120691+0000","last_clean_scrub_stamp":"2024-09-21T00:13:33.120691+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:20.198660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509344+0000","last_change":"2024-09-21T00:18:48.730107+0000","last_active":"2024-09-21T00:20:26.509344+0000","last_peered":"2024-09-21T00:20:26.509344+0000","last_clean":"2024-09-21T00:20:26.509344+0000","last_became_active":"2024-09-21T00:18:48.729463+0000","last_became_peered":"2024-09-21T00:18:48.729463+0000","last_unstale":"2024-09-21T00:20:26.509344+0000","last_undegraded":"2024-09-21T00:20:26.509344+0000","last_fullsized":"2024-09-21T00:20:26.509344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:55.352870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509702+0000","last_change":"2024-09-21T00:18:38.671244+0000","last_active":"2024-09-21T00:20:26.509702+0000","last_peered":"2024-09-21T00:20:26.509702+0000","last_clean":"2024-09-21T00:20:26.509702+0000","last_became_active":"2024-09-21T00:18:38.670630+0000","last_became_peered":"2024-09-21T00:18:38.670630+0000","last_unstale":"2024-09-21T00:20:26.509702+0000","last_undegraded":"2024-09-21T00:20:26.509702+0000","last_fullsized":"2024-09-21T00:20:26.509702+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:23.695660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451789+0000","last_change":"2024-09-21T00:19:25.843900+0000","last_active":"2024-09-21T00:20:26.451789+0000","last_peered":"2024-09-21T00:20:26.451789+0000","last_clean":"2024-09-21T00:20:26.451789+0000","last_became_active":"2024-09-21T00:19:25.843438+0000","last_became_peered":"2024-09-21T00:19:25.843438+0000","last_unstale":"2024-09-21T00:20:26.451789+0000","last_undegraded":"2024-09-21T00:20:26.451789+0000","last_fullsized":"2024-09-21T00:20:26.451789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:17.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509343+0000","last_change":"2024-09-21T00:18:48.739640+0000","last_active":"2024-09-21T00:20:26.509343+0000","last_peered":"2024-09-21T00:20:26.509343+0000","last_clean":"2024-09-21T00:20:26.509343+0000","last_became_active":"2024-09-21T00:18:48.739118+0000","last_became_peered":"2024-09-21T00:18:48.739118+0000","last_unstale":"2024-09-21T00:20:26.509343+0000","last_undegraded":"2024-09-21T00:20:26.509343+0000","last_fullsized":"2024-09-21T00:20:26.509343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:03.093561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509166+0000","last_change":"2024-09-21T00:18:38.673020+0000","last_active":"2024-09-21T00:20:26.509166+0000","last_peered":"2024-09-21T00:20:26.509166+0000","last_clean":"2024-09-21T00:20:26.509166+0000","last_became_active":"2024-09-21T00:18:38.672391+0000","last_became_peered":"2024-09-21T00:18:38.672391+0000","last_unstale":"2024-09-21T00:20:26.509166+0000","last_undegraded":"2024-09-21T00:20:26.509166+0000","last_fullsized":"2024-09-21T00:20:26.509166+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:13.866576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509394+0000","last_change":"2024-09-21T00:18:35.615844+0000","last_active":"2024-09-21T00:20:26.509394+0000","last_peered":"2024-09-21T00:20:26.509394+0000","last_clean":"2024-09-21T00:20:26.509394+0000","last_became_active":"2024-09-21T00:18:35.615385+0000","last_became_peered":"2024-09-21T00:18:35.615385+0000","last_unstale":"2024-09-21T00:20:26.509394+0000","last_undegraded":"2024-09-21T00:20:26.509394+0000","last_fullsized":"2024-09-21T00:20:26.509394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:03.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.512053+0000","last_change":"2024-09-21T00:20:26.511995+0000","last_active":"2024-09-21T00:20:26.512053+0000","last_peered":"2024-09-21T00:20:26.512053+0000","last_clean":"2024-09-21T00:20:26.512053+0000","last_became_active":"2024-09-21T00:20:26.511607+0000","last_became_peered":"2024-09-21T00:20:26.511607+0000","last_unstale":"2024-09-21T00:20:26.512053+0000","last_undegraded":"2024-09-21T00:20:26.512053+0000","last_fullsized":"2024-09-21T00:20:26.512053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:38.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508874+0000","last_change":"2024-09-21T00:18:38.671280+0000","last_active":"2024-09-21T00:20:26.508874+0000","last_peered":"2024-09-21T00:20:26.508874+0000","last_clean":"2024-09-21T00:20:26.508874+0000","last_became_active":"2024-09-21T00:18:38.670770+0000","last_became_peered":"2024-09-21T00:18:38.670770+0000","last_unstale":"2024-09-21T00:20:26.508874+0000","last_undegraded":"2024-09-21T00:20:26.508874+0000","last_fullsized":"2024-09-21T00:20:26.508874+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:06.809450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509562+0000","last_change":"2024-09-21T00:18:35.626071+0000","last_active":"2024-09-21T00:20:26.509562+0000","last_peered":"2024-09-21T00:20:26.509562+0000","last_clean":"2024-09-21T00:20:26.509562+0000","last_became_active":"2024-09-21T00:18:35.625649+0000","last_became_peered":"2024-09-21T00:18:35.625649+0000","last_unstale":"2024-09-21T00:20:26.509562+0000","last_undegraded":"2024-09-21T00:20:26.509562+0000","last_fullsized":"2024-09-21T00:20:26.509562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:16:27.516303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508563+0000","last_change":"2024-09-21T00:18:48.728451+0000","last_active":"2024-09-21T00:20:26.508563+0000","last_peered":"2024-09-21T00:20:26.508563+0000","last_clean":"2024-09-21T00:20:26.508563+0000","last_became_active":"2024-09-21T00:18:48.727840+0000","last_became_peered":"2024-09-21T00:18:48.727840+0000","last_unstale":"2024-09-21T00:20:26.508563+0000","last_undegraded":"2024-09-21T00:20:26.508563+0000","last_fullsized":"2024-09-21T00:20:26.508563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:13.457138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454195+0000","last_change":"2024-09-21T00:18:38.664090+0000","last_active":"2024-09-21T00:20:26.454195+0000","last_peered":"2024-09-21T00:20:26.454195+0000","last_clean":"2024-09-21T00:20:26.454195+0000","last_became_active":"2024-09-21T00:18:38.663728+0000","last_became_peered":"2024-09-21T00:18:38.663728+0000","last_unstale":"2024-09-21T00:20:26.454195+0000","last_undegraded":"2024-09-21T00:20:26.454195+0000","last_fullsized":"2024-09-21T00:20:26.454195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:46.367766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454833+0000","last_change":"2024-09-21T00:19:25.847280+0000","last_active":"2024-09-21T00:20:26.454833+0000","last_peered":"2024-09-21T00:20:26.454833+0000","last_clean":"2024-09-21T00:20:26.454833+0000","last_became_active":"2024-09-21T00:19:25.840919+0000","last_became_peered":"2024-09-21T00:19:25.840919+0000","last_unstale":"2024-09-21T00:20:26.454833+0000","last_undegraded":"2024-09-21T00:20:26.454833+0000","last_fullsized":"2024-09-21T00:20:26.454833+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:15.411674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508729+0000","last_change":"2024-09-21T00:18:48.732427+0000","last_active":"2024-09-21T00:20:26.508729+0000","last_peered":"2024-09-21T00:20:26.508729+0000","last_clean":"2024-09-21T00:20:26.508729+0000","last_became_active":"2024-09-21T00:18:48.732078+0000","last_became_peered":"2024-09-21T00:18:48.732078+0000","last_unstale":"2024-09-21T00:20:26.508729+0000","last_undegraded":"2024-09-21T00:20:26.508729+0000","last_fullsized":"2024-09-21T00:20:26.508729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:35.055680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509565+0000","last_change":"2024-09-21T00:18:38.675022+0000","last_active":"2024-09-21T00:20:26.509565+0000","last_peered":"2024-09-21T00:20:26.509565+0000","last_clean":"2024-09-21T00:20:26.509565+0000","last_became_active":"2024-09-21T00:18:38.674650+0000","last_became_peered":"2024-09-21T00:18:38.674650+0000","last_unstale":"2024-09-21T00:20:26.509565+0000","last_undegraded":"2024-09-21T00:20:26.509565+0000","last_fullsized":"2024-09-21T00:20:26.509565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:05.191404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510081+0000","last_change":"2024-09-21T00:18:35.769555+0000","last_active":"2024-09-21T00:20:26.510081+0000","last_peered":"2024-09-21T00:20:26.510081+0000","last_clean":"2024-09-21T00:20:26.510081+0000","last_became_active":"2024-09-21T00:18:35.769080+0000","last_became_peered":"2024-09-21T00:18:35.769080+0000","last_unstale":"2024-09-21T00:20:26.510081+0000","last_undegraded":"2024-09-21T00:20:26.510081+0000","last_fullsized":"2024-09-21T00:20:26.510081+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:19.306830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509500+0000","last_change":"2024-09-21T00:18:48.742957+0000","last_active":"2024-09-21T00:20:26.509500+0000","last_peered":"2024-09-21T00:20:26.509500+0000","last_clean":"2024-09-21T00:20:26.509500+0000","last_became_active":"2024-09-21T00:18:48.742511+0000","last_became_peered":"2024-09-21T00:18:48.742511+0000","last_unstale":"2024-09-21T00:20:26.509500+0000","last_undegraded":"2024-09-21T00:20:26.509500+0000","last_fullsized":"2024-09-21T00:20:26.509500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:15.231189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508572+0000","last_change":"2024-09-21T00:18:38.666576+0000","last_active":"2024-09-21T00:20:26.508572+0000","last_peered":"2024-09-21T00:20:26.508572+0000","last_clean":"2024-09-21T00:20:26.508572+0000","last_became_active":"2024-09-21T00:18:38.665643+0000","last_became_peered":"2024-09-21T00:18:38.665643+0000","last_unstale":"2024-09-21T00:20:26.508572+0000","last_undegraded":"2024-09-21T00:20:26.508572+0000","last_fullsized":"2024-09-21T00:20:26.508572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:35.397847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:40.024711+0000","last_change":"2024-09-21T00:18:35.623143+0000","last_active":"2024-09-21T00:20:40.024711+0000","last_peered":"2024-09-21T00:20:40.024711+0000","last_clean":"2024-09-21T00:20:40.024711+0000","last_became_active":"2024-09-21T00:18:35.622693+0000","last_became_peered":"2024-09-21T00:18:35.622693+0000","last_unstale":"2024-09-21T00:20:40.024711+0000","last_undegraded":"2024-09-21T00:20:40.024711+0000","last_fullsized":"2024-09-21T00:20:40.024711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:01.331665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452805+0000","last_change":"2024-09-21T00:18:48.723877+0000","last_active":"2024-09-21T00:20:26.452805+0000","last_peered":"2024-09-21T00:20:26.452805+0000","last_clean":"2024-09-21T00:20:26.452805+0000","last_became_active":"2024-09-21T00:18:48.723360+0000","last_became_peered":"2024-09-21T00:18:48.723360+0000","last_unstale":"2024-09-21T00:20:26.452805+0000","last_undegraded":"2024-09-21T00:20:26.452805+0000","last_fullsized":"2024-09-21T00:20:26.452805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:07:20.442867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509000+0000","last_change":"2024-09-21T00:18:38.669643+0000","last_active":"2024-09-21T00:20:26.509000+0000","last_peered":"2024-09-21T00:20:26.509000+0000","last_clean":"2024-09-21T00:20:26.509000+0000","last_became_active":"2024-09-21T00:18:38.669245+0000","last_became_peered":"2024-09-21T00:18:38.669245+0000","last_unstale":"2024-09-21T00:20:26.509000+0000","last_undegraded":"2024-09-21T00:20:26.509000+0000","last_fullsized":"2024-09-21T00:20:26.509000+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:17.789291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509513+0000","last_change":"2024-09-21T00:18:35.623189+0000","last_active":"2024-09-21T00:20:26.509513+0000","last_peered":"2024-09-21T00:20:26.509513+0000","last_clean":"2024-09-21T00:20:26.509513+0000","last_became_active":"2024-09-21T00:18:35.622748+0000","last_became_peered":"2024-09-21T00:18:35.622748+0000","last_unstale":"2024-09-21T00:20:26.509513+0000","last_undegraded":"2024-09-21T00:20:26.509513+0000","last_fullsized":"2024-09-21T00:20:26.509513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:15.570789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453777+0000","last_change":"2024-09-21T00:18:48.739612+0000","last_active":"2024-09-21T00:20:26.453777+0000","last_peered":"2024-09-21T00:20:26.453777+0000","last_clean":"2024-09-21T00:20:26.453777+0000","last_became_active":"2024-09-21T00:18:48.739173+0000","last_became_peered":"2024-09-21T00:18:48.739173+0000","last_unstale":"2024-09-21T00:20:26.453777+0000","last_undegraded":"2024-09-21T00:20:26.453777+0000","last_fullsized":"2024-09-21T00:20:26.453777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:26.327178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508470+0000","last_change":"2024-09-21T00:18:38.670876+0000","last_active":"2024-09-21T00:20:26.508470+0000","last_peered":"2024-09-21T00:20:26.508470+0000","last_clean":"2024-09-21T00:20:26.508470+0000","last_became_active":"2024-09-21T00:18:38.670371+0000","last_became_peered":"2024-09-21T00:18:38.670371+0000","last_unstale":"2024-09-21T00:20:26.508470+0000","last_undegraded":"2024-09-21T00:20:26.508470+0000","last_fullsized":"2024-09-21T00:20:26.508470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:48.641071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451958+0000","last_change":"2024-09-21T00:18:35.622919+0000","last_active":"2024-09-21T00:20:26.451958+0000","last_peered":"2024-09-21T00:20:26.451958+0000","last_clean":"2024-09-21T00:20:26.451958+0000","last_became_active":"2024-09-21T00:18:35.622435+0000","last_became_peered":"2024-09-21T00:18:35.622435+0000","last_unstale":"2024-09-21T00:20:26.451958+0000","last_undegraded":"2024-09-21T00:20:26.451958+0000","last_fullsized":"2024-09-21T00:20:26.451958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:22.113871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508927+0000","last_change":"2024-09-21T00:18:48.741736+0000","last_active":"2024-09-21T00:20:26.508927+0000","last_peered":"2024-09-21T00:20:26.508927+0000","last_clean":"2024-09-21T00:20:26.508927+0000","last_became_active":"2024-09-21T00:18:48.741135+0000","last_became_peered":"2024-09-21T00:18:48.741135+0000","last_unstale":"2024-09-21T00:20:26.508927+0000","last_undegraded":"2024-09-21T00:20:26.508927+0000","last_fullsized":"2024-09-21T00:20:26.508927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:50.492351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510579+0000","last_change":"2024-09-21T00:18:38.667377+0000","last_active":"2024-09-21T00:20:26.510579+0000","last_peered":"2024-09-21T00:20:26.510579+0000","last_clean":"2024-09-21T00:20:26.510579+0000","last_became_active":"2024-09-21T00:18:38.667010+0000","last_became_peered":"2024-09-21T00:18:38.667010+0000","last_unstale":"2024-09-21T00:20:26.510579+0000","last_undegraded":"2024-09-21T00:20:26.510579+0000","last_fullsized":"2024-09-21T00:20:26.510579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:06.634767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":231,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:40.023437+0000","last_change":"2024-09-21T00:19:25.853603+0000","last_active":"2024-09-21T00:20:40.023437+0000","last_peered":"2024-09-21T00:20:40.023437+0000","last_clean":"2024-09-21T00:20:40.023437+0000","last_became_active":"2024-09-21T00:19:25.841126+0000","last_became_peered":"2024-09-21T00:19:25.841126+0000","last_unstale":"2024-09-21T00:20:40.023437+0000","last_undegraded":"2024-09-21T00:20:40.023437+0000","last_fullsized":"2024-09-21T00:20:40.023437+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:27:47.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65659,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":138,"num_read_kb":133,"num_write":32,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451643+0000","last_change":"2024-09-21T00:18:48.731004+0000","last_active":"2024-09-21T00:20:26.451643+0000","last_peered":"2024-09-21T00:20:26.451643+0000","last_clean":"2024-09-21T00:20:26.451643+0000","last_became_active":"2024-09-21T00:18:48.730549+0000","last_became_peered":"2024-09-21T00:18:48.730549+0000","last_unstale":"2024-09-21T00:20:26.451643+0000","last_undegraded":"2024-09-21T00:20:26.451643+0000","last_fullsized":"2024-09-21T00:20:26.451643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:30.988300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510768+0000","last_change":"2024-09-21T00:19:25.843532+0000","last_active":"2024-09-21T00:20:26.510768+0000","last_peered":"2024-09-21T00:20:26.510768+0000","last_clean":"2024-09-21T00:20:26.510768+0000","last_became_active":"2024-09-21T00:19:25.843065+0000","last_became_peered":"2024-09-21T00:19:25.843065+0000","last_unstale":"2024-09-21T00:20:26.510768+0000","last_undegraded":"2024-09-21T00:20:26.510768+0000","last_fullsized":"2024-09-21T00:20:26.510768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:36:26.488692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509235+0000","last_change":"2024-09-21T00:18:35.770494+0000","last_active":"2024-09-21T00:20:26.509235+0000","last_peered":"2024-09-21T00:20:26.509235+0000","last_clean":"2024-09-21T00:20:26.509235+0000","last_became_active":"2024-09-21T00:18:35.770122+0000","last_became_peered":"2024-09-21T00:18:35.770122+0000","last_unstale":"2024-09-21T00:20:26.509235+0000","last_undegraded":"2024-09-21T00:20:26.509235+0000","last_fullsized":"2024-09-21T00:20:26.509235+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:41.415904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451780+0000","last_change":"2024-09-21T00:18:48.736802+0000","last_active":"2024-09-21T00:20:26.451780+0000","last_peered":"2024-09-21T00:20:26.451780+0000","last_clean":"2024-09-21T00:20:26.451780+0000","last_became_active":"2024-09-21T00:18:48.736008+0000","last_became_peered":"2024-09-21T00:18:48.736008+0000","last_unstale":"2024-09-21T00:20:26.451780+0000","last_undegraded":"2024-09-21T00:20:26.451780+0000","last_fullsized":"2024-09-21T00:20:26.451780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:07.995614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455871+0000","last_change":"2024-09-21T00:18:38.678385+0000","last_active":"2024-09-21T00:20:26.455871+0000","last_peered":"2024-09-21T00:20:26.455871+0000","last_clean":"2024-09-21T00:20:26.455871+0000","last_became_active":"2024-09-21T00:18:38.677174+0000","last_became_peered":"2024-09-21T00:18:38.677174+0000","last_unstale":"2024-09-21T00:20:26.455871+0000","last_undegraded":"2024-09-21T00:20:26.455871+0000","last_fullsized":"2024-09-21T00:20:26.455871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:23.486222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509438+0000","last_change":"2024-09-21T00:18:35.627703+0000","last_active":"2024-09-21T00:20:26.509438+0000","last_peered":"2024-09-21T00:20:26.509438+0000","last_clean":"2024-09-21T00:20:26.509438+0000","last_became_active":"2024-09-21T00:18:35.627274+0000","last_became_peered":"2024-09-21T00:18:35.627274+0000","last_unstale":"2024-09-21T00:20:26.509438+0000","last_undegraded":"2024-09-21T00:20:26.509438+0000","last_fullsized":"2024-09-21T00:20:26.509438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:14.326810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451042+0000","last_change":"2024-09-21T00:18:48.729926+0000","last_active":"2024-09-21T00:20:26.451042+0000","last_peered":"2024-09-21T00:20:26.451042+0000","last_clean":"2024-09-21T00:20:26.451042+0000","last_became_active":"2024-09-21T00:18:48.728366+0000","last_became_peered":"2024-09-21T00:18:48.728366+0000","last_unstale":"2024-09-21T00:20:26.451042+0000","last_undegraded":"2024-09-21T00:20:26.451042+0000","last_fullsized":"2024-09-21T00:20:26.451042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:48:50.005748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.511150+0000","last_change":"2024-09-21T00:18:38.664173+0000","last_active":"2024-09-21T00:20:26.511150+0000","last_peered":"2024-09-21T00:20:26.511150+0000","last_clean":"2024-09-21T00:20:26.511150+0000","last_became_active":"2024-09-21T00:18:38.663358+0000","last_became_peered":"2024-09-21T00:18:38.663358+0000","last_unstale":"2024-09-21T00:20:26.511150+0000","last_undegraded":"2024-09-21T00:20:26.511150+0000","last_fullsized":"2024-09-21T00:20:26.511150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:34:31.203621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":113,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:39.022273+0000","last_change":"2024-09-21T00:18:35.622133+0000","last_active":"2024-09-21T00:20:39.022273+0000","last_peered":"2024-09-21T00:20:39.022273+0000","last_clean":"2024-09-21T00:20:39.022273+0000","last_became_active":"2024-09-21T00:18:35.621032+0000","last_became_peered":"2024-09-21T00:18:35.621032+0000","last_unstale":"2024-09-21T00:20:39.022273+0000","last_undegraded":"2024-09-21T00:20:39.022273+0000","last_fullsized":"2024-09-21T00:20:39.022273+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:24.121665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574904+0000","last_change":"2024-09-21T00:18:48.733339+0000","last_active":"2024-09-21T00:20:26.574904+0000","last_peered":"2024-09-21T00:20:26.574904+0000","last_clean":"2024-09-21T00:20:26.574904+0000","last_became_active":"2024-09-21T00:18:48.731843+0000","last_became_peered":"2024-09-21T00:18:48.731843+0000","last_unstale":"2024-09-21T00:20:26.574904+0000","last_undegraded":"2024-09-21T00:20:26.574904+0000","last_fullsized":"2024-09-21T00:20:26.574904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:42:55.474197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455802+0000","last_change":"2024-09-21T00:18:48.740108+0000","last_active":"2024-09-21T00:20:26.455802+0000","last_peered":"2024-09-21T00:20:26.455802+0000","last_clean":"2024-09-21T00:20:26.455802+0000","last_became_active":"2024-09-21T00:18:48.739330+0000","last_became_peered":"2024-09-21T00:18:48.739330+0000","last_unstale":"2024-09-21T00:20:26.455802+0000","last_undegraded":"2024-09-21T00:20:26.455802+0000","last_fullsized":"2024-09-21T00:20:26.455802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:00.103502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451244+0000","last_change":"2024-09-21T00:18:38.660469+0000","last_active":"2024-09-21T00:20:26.451244+0000","last_peered":"2024-09-21T00:20:26.451244+0000","last_clean":"2024-09-21T00:20:26.451244+0000","last_became_active":"2024-09-21T00:18:38.660138+0000","last_became_peered":"2024-09-21T00:18:38.660138+0000","last_unstale":"2024-09-21T00:20:26.451244+0000","last_undegraded":"2024-09-21T00:20:26.451244+0000","last_fullsized":"2024-09-21T00:20:26.451244+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:01.079112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509126+0000","last_change":"2024-09-21T00:18:35.624528+0000","last_active":"2024-09-21T00:20:26.509126+0000","last_peered":"2024-09-21T00:20:26.509126+0000","last_clean":"2024-09-21T00:20:26.509126+0000","last_became_active":"2024-09-21T00:18:35.624170+0000","last_became_peered":"2024-09-21T00:18:35.624170+0000","last_unstale":"2024-09-21T00:20:26.509126+0000","last_undegraded":"2024-09-21T00:20:26.509126+0000","last_fullsized":"2024-09-21T00:20:26.509126+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:07.914521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.511927+0000","last_change":"2024-09-21T00:20:26.511824+0000","last_active":"2024-09-21T00:20:26.511927+0000","last_peered":"2024-09-21T00:20:26.511927+0000","last_clean":"2024-09-21T00:20:26.511927+0000","last_became_active":"2024-09-21T00:20:26.511254+0000","last_became_peered":"2024-09-21T00:20:26.511254+0000","last_unstale":"2024-09-21T00:20:26.511927+0000","last_undegraded":"2024-09-21T00:20:26.511927+0000","last_fullsized":"2024-09-21T00:20:26.511927+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:12.884034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574527+0000","last_change":"2024-09-21T00:18:38.666928+0000","last_active":"2024-09-21T00:20:26.574527+0000","last_peered":"2024-09-21T00:20:26.574527+0000","last_clean":"2024-09-21T00:20:26.574527+0000","last_became_active":"2024-09-21T00:18:38.666236+0000","last_became_peered":"2024-09-21T00:18:38.666236+0000","last_unstale":"2024-09-21T00:20:26.574527+0000","last_undegraded":"2024-09-21T00:20:26.574527+0000","last_fullsized":"2024-09-21T00:20:26.574527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:21.043778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508260+0000","last_change":"2024-09-21T00:18:35.624514+0000","last_active":"2024-09-21T00:20:26.508260+0000","last_peered":"2024-09-21T00:20:26.508260+0000","last_clean":"2024-09-21T00:20:26.508260+0000","last_became_active":"2024-09-21T00:18:35.624098+0000","last_became_peered":"2024-09-21T00:18:35.624098+0000","last_unstale":"2024-09-21T00:20:26.508260+0000","last_undegraded":"2024-09-21T00:20:26.508260+0000","last_fullsized":"2024-09-21T00:20:26.508260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:21.777568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509824+0000","last_change":"2024-09-21T00:19:25.841232+0000","last_active":"2024-09-21T00:20:26.509824+0000","last_peered":"2024-09-21T00:20:26.509824+0000","last_clean":"2024-09-21T00:20:26.509824+0000","last_became_active":"2024-09-21T00:19:25.840655+0000","last_became_peered":"2024-09-21T00:19:25.840655+0000","last_unstale":"2024-09-21T00:20:26.509824+0000","last_undegraded":"2024-09-21T00:20:26.509824+0000","last_fullsized":"2024-09-21T00:20:26.509824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:05.846222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509106+0000","last_change":"2024-09-21T00:18:38.667466+0000","last_active":"2024-09-21T00:20:26.509106+0000","last_peered":"2024-09-21T00:20:26.509106+0000","last_clean":"2024-09-21T00:20:26.509106+0000","last_became_active":"2024-09-21T00:18:38.666960+0000","last_became_peered":"2024-09-21T00:18:38.666960+0000","last_unstale":"2024-09-21T00:20:26.509106+0000","last_undegraded":"2024-09-21T00:20:26.509106+0000","last_fullsized":"2024-09-21T00:20:26.509106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:29:56.614766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451538+0000","last_change":"2024-09-21T00:18:48.739656+0000","last_active":"2024-09-21T00:20:26.451538+0000","last_peered":"2024-09-21T00:20:26.451538+0000","last_clean":"2024-09-21T00:20:26.451538+0000","last_became_active":"2024-09-21T00:18:48.739228+0000","last_became_peered":"2024-09-21T00:18:48.739228+0000","last_unstale":"2024-09-21T00:20:26.451538+0000","last_undegraded":"2024-09-21T00:20:26.451538+0000","last_fullsized":"2024-09-21T00:20:26.451538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:11.427595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:40.024152+0000","last_change":"2024-09-21T00:18:35.624108+0000","last_active":"2024-09-21T00:20:40.024152+0000","last_peered":"2024-09-21T00:20:40.024152+0000","last_clean":"2024-09-21T00:20:40.024152+0000","last_became_active":"2024-09-21T00:18:35.623708+0000","last_became_peered":"2024-09-21T00:18:35.623708+0000","last_unstale":"2024-09-21T00:20:40.024152+0000","last_undegraded":"2024-09-21T00:20:40.024152+0000","last_fullsized":"2024-09-21T00:20:40.024152+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:50.020662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510214+0000","last_change":"2024-09-21T00:18:38.665402+0000","last_active":"2024-09-21T00:20:26.510214+0000","last_peered":"2024-09-21T00:20:26.510214+0000","last_clean":"2024-09-21T00:20:26.510214+0000","last_became_active":"2024-09-21T00:18:38.665034+0000","last_became_peered":"2024-09-21T00:18:38.665034+0000","last_unstale":"2024-09-21T00:20:26.510214+0000","last_undegraded":"2024-09-21T00:20:26.510214+0000","last_fullsized":"2024-09-21T00:20:26.510214+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:33:10.824705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451734+0000","last_change":"2024-09-21T00:18:48.729928+0000","last_active":"2024-09-21T00:20:26.451734+0000","last_peered":"2024-09-21T00:20:26.451734+0000","last_clean":"2024-09-21T00:20:26.451734+0000","last_became_active":"2024-09-21T00:18:48.729498+0000","last_became_peered":"2024-09-21T00:18:48.729498+0000","last_unstale":"2024-09-21T00:20:26.451734+0000","last_undegraded":"2024-09-21T00:20:26.451734+0000","last_fullsized":"2024-09-21T00:20:26.451734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:10:27.140012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453735+0000","last_change":"2024-09-21T00:18:38.662221+0000","last_active":"2024-09-21T00:20:26.453735+0000","last_peered":"2024-09-21T00:20:26.453735+0000","last_clean":"2024-09-21T00:20:26.453735+0000","last_became_active":"2024-09-21T00:18:38.660543+0000","last_became_peered":"2024-09-21T00:18:38.660543+0000","last_unstale":"2024-09-21T00:20:26.453735+0000","last_undegraded":"2024-09-21T00:20:26.453735+0000","last_fullsized":"2024-09-21T00:20:26.453735+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:43.159699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452162+0000","last_change":"2024-09-21T00:18:38.672696+0000","last_active":"2024-09-21T00:20:26.452162+0000","last_peered":"2024-09-21T00:20:26.452162+0000","last_clean":"2024-09-21T00:20:26.452162+0000","last_became_active":"2024-09-21T00:18:38.672356+0000","last_became_peered":"2024-09-21T00:18:38.672356+0000","last_unstale":"2024-09-21T00:20:26.452162+0000","last_undegraded":"2024-09-21T00:20:26.452162+0000","last_fullsized":"2024-09-21T00:20:26.452162+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.549939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451529+0000","last_change":"2024-09-21T00:18:38.667017+0000","last_active":"2024-09-21T00:20:26.451529+0000","last_peered":"2024-09-21T00:20:26.451529+0000","last_clean":"2024-09-21T00:20:26.451529+0000","last_became_active":"2024-09-21T00:18:38.666643+0000","last_became_peered":"2024-09-21T00:18:38.666643+0000","last_unstale":"2024-09-21T00:20:26.451529+0000","last_undegraded":"2024-09-21T00:20:26.451529+0000","last_fullsized":"2024-09-21T00:20:26.451529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:30.496339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454976+0000","last_change":"2024-09-21T00:18:38.667705+0000","last_active":"2024-09-21T00:20:26.454976+0000","last_peered":"2024-09-21T00:20:26.454976+0000","last_clean":"2024-09-21T00:20:26.454976+0000","last_became_active":"2024-09-21T00:18:38.665141+0000","last_became_peered":"2024-09-21T00:18:38.665141+0000","last_unstale":"2024-09-21T00:20:26.454976+0000","last_undegraded":"2024-09-21T00:20:26.454976+0000","last_fullsized":"2024-09-21T00:20:26.454976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:17.624035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509000+0000","last_change":"2024-09-21T00:18:38.681364+0000","last_active":"2024-09-21T00:20:26.509000+0000","last_peered":"2024-09-21T00:20:26.509000+0000","last_clean":"2024-09-21T00:20:26.509000+0000","last_became_active":"2024-09-21T00:18:38.680937+0000","last_became_peered":"2024-09-21T00:18:38.680937+0000","last_unstale":"2024-09-21T00:20:26.509000+0000","last_undegraded":"2024-09-21T00:20:26.509000+0000","last_fullsized":"2024-09-21T00:20:26.509000+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:34.206201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454175+0000","last_change":"2024-09-21T00:19:25.843091+0000","last_active":"2024-09-21T00:20:26.454175+0000","last_peered":"2024-09-21T00:20:26.454175+0000","last_clean":"2024-09-21T00:20:26.454175+0000","last_became_active":"2024-09-21T00:19:25.842600+0000","last_became_peered":"2024-09-21T00:19:25.842600+0000","last_unstale":"2024-09-21T00:20:26.454175+0000","last_undegraded":"2024-09-21T00:20:26.454175+0000","last_fullsized":"2024-09-21T00:20:26.454175+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:10.256474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451827+0000","last_change":"2024-09-21T00:18:38.668656+0000","last_active":"2024-09-21T00:20:26.451827+0000","last_peered":"2024-09-21T00:20:26.451827+0000","last_clean":"2024-09-21T00:20:26.451827+0000","last_became_active":"2024-09-21T00:18:38.668153+0000","last_became_peered":"2024-09-21T00:18:38.668153+0000","last_unstale":"2024-09-21T00:20:26.451827+0000","last_undegraded":"2024-09-21T00:20:26.451827+0000","last_fullsized":"2024-09-21T00:20:26.451827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:03.763609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.575289+0000","last_change":"2024-09-21T00:18:38.675945+0000","last_active":"2024-09-21T00:20:26.575289+0000","last_peered":"2024-09-21T00:20:26.575289+0000","last_clean":"2024-09-21T00:20:26.575289+0000","last_became_active":"2024-09-21T00:18:38.675397+0000","last_became_peered":"2024-09-21T00:18:38.675397+0000","last_unstale":"2024-09-21T00:20:26.575289+0000","last_undegraded":"2024-09-21T00:20:26.575289+0000","last_fullsized":"2024-09-21T00:20:26.575289+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:56.082314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508699+0000","last_change":"2024-09-21T00:18:38.662854+0000","last_active":"2024-09-21T00:20:26.508699+0000","last_peered":"2024-09-21T00:20:26.508699+0000","last_clean":"2024-09-21T00:20:26.508699+0000","last_became_active":"2024-09-21T00:18:38.662094+0000","last_became_peered":"2024-09-21T00:18:38.662094+0000","last_unstale":"2024-09-21T00:20:26.508699+0000","last_undegraded":"2024-09-21T00:20:26.508699+0000","last_fullsized":"2024-09-21T00:20:26.508699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:45.756641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574767+0000","last_change":"2024-09-21T00:18:38.663282+0000","last_active":"2024-09-21T00:20:26.574767+0000","last_peered":"2024-09-21T00:20:26.574767+0000","last_clean":"2024-09-21T00:20:26.574767+0000","last_became_active":"2024-09-21T00:18:38.662148+0000","last_became_peered":"2024-09-21T00:18:38.662148+0000","last_unstale":"2024-09-21T00:20:26.574767+0000","last_undegraded":"2024-09-21T00:20:26.574767+0000","last_fullsized":"2024-09-21T00:20:26.574767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:51.090591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509613+0000","last_change":"2024-09-21T00:18:38.669184+0000","last_active":"2024-09-21T00:20:26.509613+0000","last_peered":"2024-09-21T00:20:26.509613+0000","last_clean":"2024-09-21T00:20:26.509613+0000","last_became_active":"2024-09-21T00:18:38.668799+0000","last_became_peered":"2024-09-21T00:18:38.668799+0000","last_unstale":"2024-09-21T00:20:26.509613+0000","last_undegraded":"2024-09-21T00:20:26.509613+0000","last_fullsized":"2024-09-21T00:20:26.509613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:49.642568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508860+0000","last_change":"2024-09-21T00:18:38.679501+0000","last_active":"2024-09-21T00:20:26.508860+0000","last_peered":"2024-09-21T00:20:26.508860+0000","last_clean":"2024-09-21T00:20:26.508860+0000","last_became_active":"2024-09-21T00:18:38.678582+0000","last_became_peered":"2024-09-21T00:18:38.678582+0000","last_unstale":"2024-09-21T00:20:26.508860+0000","last_undegraded":"2024-09-21T00:20:26.508860+0000","last_fullsized":"2024-09-21T00:20:26.508860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:30.277979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454141+0000","last_change":"2024-09-21T00:18:38.659657+0000","last_active":"2024-09-21T00:20:26.454141+0000","last_peered":"2024-09-21T00:20:26.454141+0000","last_clean":"2024-09-21T00:20:26.454141+0000","last_became_active":"2024-09-21T00:18:38.659257+0000","last_became_peered":"2024-09-21T00:18:38.659257+0000","last_unstale":"2024-09-21T00:20:26.454141+0000","last_undegraded":"2024-09-21T00:20:26.454141+0000","last_fullsized":"2024-09-21T00:20:26.454141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:08.653729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452097+0000","last_change":"2024-09-21T00:18:38.671817+0000","last_active":"2024-09-21T00:20:26.452097+0000","last_peered":"2024-09-21T00:20:26.452097+0000","last_clean":"2024-09-21T00:20:26.452097+0000","last_became_active":"2024-09-21T00:18:38.671469+0000","last_became_peered":"2024-09-21T00:18:38.671469+0000","last_unstale":"2024-09-21T00:20:26.452097+0000","last_undegraded":"2024-09-21T00:20:26.452097+0000","last_fullsized":"2024-09-21T00:20:26.452097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:58:15.036623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451661+0000","last_change":"2024-09-21T00:18:38.664481+0000","last_active":"2024-09-21T00:20:26.451661+0000","last_peered":"2024-09-21T00:20:26.451661+0000","last_clean":"2024-09-21T00:20:26.451661+0000","last_became_active":"2024-09-21T00:18:38.664098+0000","last_became_peered":"2024-09-21T00:18:38.664098+0000","last_unstale":"2024-09-21T00:20:26.451661+0000","last_undegraded":"2024-09-21T00:20:26.451661+0000","last_fullsized":"2024-09-21T00:20:26.451661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:09.271251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455016+0000","last_change":"2024-09-21T00:18:38.662416+0000","last_active":"2024-09-21T00:20:26.455016+0000","last_peered":"2024-09-21T00:20:26.455016+0000","last_clean":"2024-09-21T00:20:26.455016+0000","last_became_active":"2024-09-21T00:18:38.662041+0000","last_became_peered":"2024-09-21T00:18:38.662041+0000","last_unstale":"2024-09-21T00:20:26.455016+0000","last_undegraded":"2024-09-21T00:20:26.455016+0000","last_fullsized":"2024-09-21T00:20:26.455016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:03.150179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508380+0000","last_change":"2024-09-21T00:18:38.670797+0000","last_active":"2024-09-21T00:20:26.508380+0000","last_peered":"2024-09-21T00:20:26.508380+0000","last_clean":"2024-09-21T00:20:26.508380+0000","last_became_active":"2024-09-21T00:18:38.670258+0000","last_became_peered":"2024-09-21T00:18:38.670258+0000","last_unstale":"2024-09-21T00:20:26.508380+0000","last_undegraded":"2024-09-21T00:20:26.508380+0000","last_fullsized":"2024-09-21T00:20:26.508380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:55.669836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454635+0000","last_change":"2024-09-21T00:18:38.665594+0000","last_active":"2024-09-21T00:20:26.454635+0000","last_peered":"2024-09-21T00:20:26.454635+0000","last_clean":"2024-09-21T00:20:26.454635+0000","last_became_active":"2024-09-21T00:18:38.665260+0000","last_became_peered":"2024-09-21T00:18:38.665260+0000","last_unstale":"2024-09-21T00:20:26.454635+0000","last_undegraded":"2024-09-21T00:20:26.454635+0000","last_fullsized":"2024-09-21T00:20:26.454635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:52:09.014327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454704+0000","last_change":"2024-09-21T00:18:38.679294+0000","last_active":"2024-09-21T00:20:26.454704+0000","last_peered":"2024-09-21T00:20:26.454704+0000","last_clean":"2024-09-21T00:20:26.454704+0000","last_became_active":"2024-09-21T00:18:38.673765+0000","last_became_peered":"2024-09-21T00:18:38.673765+0000","last_unstale":"2024-09-21T00:20:26.454704+0000","last_undegraded":"2024-09-21T00:20:26.454704+0000","last_fullsized":"2024-09-21T00:20:26.454704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:32.076299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508744+0000","last_change":"2024-09-21T00:18:38.667869+0000","last_active":"2024-09-21T00:20:26.508744+0000","last_peered":"2024-09-21T00:20:26.508744+0000","last_clean":"2024-09-21T00:20:26.508744+0000","last_became_active":"2024-09-21T00:18:38.667499+0000","last_became_peered":"2024-09-21T00:18:38.667499+0000","last_unstale":"2024-09-21T00:20:26.508744+0000","last_undegraded":"2024-09-21T00:20:26.508744+0000","last_fullsized":"2024-09-21T00:20:26.508744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:13.501043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452298+0000","last_change":"2024-09-21T00:18:38.663107+0000","last_active":"2024-09-21T00:20:26.452298+0000","last_peered":"2024-09-21T00:20:26.452298+0000","last_clean":"2024-09-21T00:20:26.452298+0000","last_became_active":"2024-09-21T00:18:38.662680+0000","last_became_peered":"2024-09-21T00:18:38.662680+0000","last_unstale":"2024-09-21T00:20:26.452298+0000","last_undegraded":"2024-09-21T00:20:26.452298+0000","last_fullsized":"2024-09-21T00:20:26.452298+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:48.154705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509579+0000","last_change":"2024-09-21T00:18:38.668167+0000","last_active":"2024-09-21T00:20:26.509579+0000","last_peered":"2024-09-21T00:20:26.509579+0000","last_clean":"2024-09-21T00:20:26.509579+0000","last_became_active":"2024-09-21T00:18:38.667842+0000","last_became_peered":"2024-09-21T00:18:38.667842+0000","last_unstale":"2024-09-21T00:20:26.509579+0000","last_undegraded":"2024-09-21T00:20:26.509579+0000","last_fullsized":"2024-09-21T00:20:26.509579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:55.358351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510744+0000","last_change":"2024-09-21T00:18:38.666580+0000","last_active":"2024-09-21T00:20:26.510744+0000","last_peered":"2024-09-21T00:20:26.510744+0000","last_clean":"2024-09-21T00:20:26.510744+0000","last_became_active":"2024-09-21T00:18:38.666231+0000","last_became_peered":"2024-09-21T00:18:38.666231+0000","last_unstale":"2024-09-21T00:20:26.510744+0000","last_undegraded":"2024-09-21T00:20:26.510744+0000","last_fullsized":"2024-09-21T00:20:26.510744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:07.362516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.512231+0000","last_change":"2024-09-21T00:20:26.511248+0000","last_active":"2024-09-21T00:20:26.512231+0000","last_peered":"2024-09-21T00:20:26.512231+0000","last_clean":"2024-09-21T00:20:26.512231+0000","last_became_active":"2024-09-21T00:20:26.510743+0000","last_became_peered":"2024-09-21T00:20:26.510743+0000","last_unstale":"2024-09-21T00:20:26.512231+0000","last_undegraded":"2024-09-21T00:20:26.512231+0000","last_fullsized":"2024-09-21T00:20:26.512231+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:41.577950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574484+0000","last_change":"2024-09-21T00:18:38.666580+0000","last_active":"2024-09-21T00:20:26.574484+0000","last_peered":"2024-09-21T00:20:26.574484+0000","last_clean":"2024-09-21T00:20:26.574484+0000","last_became_active":"2024-09-21T00:18:38.665698+0000","last_became_peered":"2024-09-21T00:18:38.665698+0000","last_unstale":"2024-09-21T00:20:26.574484+0000","last_undegraded":"2024-09-21T00:20:26.574484+0000","last_fullsized":"2024-09-21T00:20:26.574484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:45.627667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509070+0000","last_change":"2024-09-21T00:18:38.670729+0000","last_active":"2024-09-21T00:20:26.509070+0000","last_peered":"2024-09-21T00:20:26.509070+0000","last_clean":"2024-09-21T00:20:26.509070+0000","last_became_active":"2024-09-21T00:18:38.670337+0000","last_became_peered":"2024-09-21T00:18:38.670337+0000","last_unstale":"2024-09-21T00:20:26.509070+0000","last_undegraded":"2024-09-21T00:20:26.509070+0000","last_fullsized":"2024-09-21T00:20:26.509070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:49.880689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509695+0000","last_change":"2024-09-21T00:18:38.664644+0000","last_active":"2024-09-21T00:20:26.509695+0000","last_peered":"2024-09-21T00:20:26.509695+0000","last_clean":"2024-09-21T00:20:26.509695+0000","last_became_active":"2024-09-21T00:18:38.664220+0000","last_became_peered":"2024-09-21T00:18:38.664220+0000","last_unstale":"2024-09-21T00:20:26.509695+0000","last_undegraded":"2024-09-21T00:20:26.509695+0000","last_fullsized":"2024-09-21T00:20:26.509695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:09.610962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510095+0000","last_change":"2024-09-21T00:18:38.671592+0000","last_active":"2024-09-21T00:20:26.510095+0000","last_peered":"2024-09-21T00:20:26.510095+0000","last_clean":"2024-09-21T00:20:26.510095+0000","last_became_active":"2024-09-21T00:18:38.671228+0000","last_became_peered":"2024-09-21T00:18:38.671228+0000","last_unstale":"2024-09-21T00:20:26.510095+0000","last_undegraded":"2024-09-21T00:20:26.510095+0000","last_fullsized":"2024-09-21T00:20:26.510095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:51.906058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452615+0000","last_change":"2024-09-21T00:19:25.843696+0000","last_active":"2024-09-21T00:20:26.452615+0000","last_peered":"2024-09-21T00:20:26.452615+0000","last_clean":"2024-09-21T00:20:26.452615+0000","last_became_active":"2024-09-21T00:19:25.843287+0000","last_became_peered":"2024-09-21T00:19:25.843287+0000","last_unstale":"2024-09-21T00:20:26.452615+0000","last_undegraded":"2024-09-21T00:20:26.452615+0000","last_fullsized":"2024-09-21T00:20:26.452615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:21.024555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455537+0000","last_change":"2024-09-21T00:18:38.671820+0000","last_active":"2024-09-21T00:20:26.455537+0000","last_peered":"2024-09-21T00:20:26.455537+0000","last_clean":"2024-09-21T00:20:26.455537+0000","last_became_active":"2024-09-21T00:18:38.667107+0000","last_became_peered":"2024-09-21T00:18:38.667107+0000","last_unstale":"2024-09-21T00:20:26.455537+0000","last_undegraded":"2024-09-21T00:20:26.455537+0000","last_fullsized":"2024-09-21T00:20:26.455537+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:58.099750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455932+0000","last_change":"2024-09-21T00:18:38.671271+0000","last_active":"2024-09-21T00:20:26.455932+0000","last_peered":"2024-09-21T00:20:26.455932+0000","last_clean":"2024-09-21T00:20:26.455932+0000","last_became_active":"2024-09-21T00:18:38.670826+0000","last_became_peered":"2024-09-21T00:18:38.670826+0000","last_unstale":"2024-09-21T00:20:26.455932+0000","last_undegraded":"2024-09-21T00:20:26.455932+0000","last_fullsized":"2024-09-21T00:20:26.455932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:57:01.723120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454997+0000","last_change":"2024-09-21T00:18:38.662851+0000","last_active":"2024-09-21T00:20:26.454997+0000","last_peered":"2024-09-21T00:20:26.454997+0000","last_clean":"2024-09-21T00:20:26.454997+0000","last_became_active":"2024-09-21T00:18:38.661259+0000","last_became_peered":"2024-09-21T00:18:38.661259+0000","last_unstale":"2024-09-21T00:20:26.454997+0000","last_undegraded":"2024-09-21T00:20:26.454997+0000","last_fullsized":"2024-09-21T00:20:26.454997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:10.508980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69660,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":422,"num_read_kb":285,"num_write":163,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3864,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":208980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678055,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":535425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874284,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37044,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708108,"statfs":{"total":95995035648,"available":95957102592,"internally_reserved":0,"allocated":1732608,"data_stored":1188330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037809,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1060864,"data_stored":539735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1052672,"data_stored":535537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430268,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1650688,"data_stored":1131468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593794,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1036288,"data_stored":530815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1658880,"data_stored":1127056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1126400,"data_stored":601706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1048576,"data_stored":531009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378714,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":531884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542239,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1105920,"data_stored":596384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705764,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1060864,"data_stored":532203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:20:46.135 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T00:20:46.135 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T00:20:46.136 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T00:20:46.136 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T00:20:46.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:46 smithi089 ceph-mon[34481]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:46.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:46 smithi098 ceph-mon[34725]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:46.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:46 smithi027 ceph-mon[30745]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:47.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.133+0000 7efe38600640 1 -- 172.21.15.27:0/1892923463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe30109e10 msgr2=0x7efe30110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:47.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.133+0000 7efe38600640 1 --2- 172.21.15.27:0/1892923463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe30109e10 0x7efe30110640 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efe28009ca0 tx=0x7efe2802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:20:47.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.133+0000 7efe38600640 1 -- 172.21.15.27:0/1892923463 shutdown_connections 2024-09-21T00:20:47.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.133+0000 7efe38600640 1 --2- 172.21.15.27:0/1892923463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe30109e10 0x7efe30110640 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:47.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.133+0000 7efe38600640 1 --2- 172.21.15.27:0/1892923463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe301094c0 0x7efe301098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:47.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.133+0000 7efe38600640 1 --2- 172.21.15.27:0/1892923463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe30103b80 0x7efe30103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:47.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.133+0000 7efe38600640 1 -- 172.21.15.27:0/1892923463 >> 172.21.15.27:0/1892923463 conn(0x7efe300ff5d0 msgr2=0x7efe30101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:47.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.133+0000 7efe38600640 1 -- 172.21.15.27:0/1892923463 shutdown_connections 2024-09-21T00:20:47.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.133+0000 7efe38600640 1 -- 172.21.15.27:0/1892923463 wait complete. 2024-09-21T00:20:47.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.135+0000 7efe38600640 1 Processor -- start 2024-09-21T00:20:47.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.135+0000 7efe38600640 1 -- start start 2024-09-21T00:20:47.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.135+0000 7efe38600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe30103b80 0x7efe3019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:47.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.135+0000 7efe38600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe301094c0 0x7efe3019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:47.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.135+0000 7efe38600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe30109e10 0x7efe301a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:47.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.135+0000 7efe38600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efe30113ad0 con 0x7efe301094c0 2024-09-21T00:20:47.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.135+0000 7efe38600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efe30113950 con 0x7efe30103b80 2024-09-21T00:20:47.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.135+0000 7efe38600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efe30113c50 con 0x7efe30109e10 2024-09-21T00:20:47.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.136+0000 7efe35200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe301094c0 0x7efe3019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:47.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.136+0000 7efe35200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe301094c0 0x7efe3019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51368/0 (socket says 172.21.15.27:51368) 2024-09-21T00:20:47.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.136+0000 7efe35200640 1 -- 172.21.15.27:0/1247903872 learned_addr learned my addr 172.21.15.27:0/1247903872 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:47.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.136+0000 7efe36600640 1 --2- 172.21.15.27:0/1247903872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe30109e10 0x7efe301a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:47.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.136+0000 7efe35c00640 1 --2- 172.21.15.27:0/1247903872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe30103b80 0x7efe3019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:47.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.136+0000 7efe35200640 1 -- 172.21.15.27:0/1247903872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe30109e10 msgr2=0x7efe301a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:47.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.136+0000 7efe35200640 1 --2- 172.21.15.27:0/1247903872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe30109e10 0x7efe301a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:47.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.136+0000 7efe35200640 1 -- 172.21.15.27:0/1247903872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe30103b80 msgr2=0x7efe3019c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:47.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.136+0000 7efe35200640 1 --2- 172.21.15.27:0/1247903872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe30103b80 0x7efe3019c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:47.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.136+0000 7efe35200640 1 -- 172.21.15.27:0/1247903872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe301a2670 con 0x7efe301094c0 2024-09-21T00:20:47.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.137+0000 7efe35200640 1 --2- 172.21.15.27:0/1247903872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe301094c0 0x7efe3019cca0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00e970 tx=0x7efe1c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:47.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.137+0000 7efe2e000640 1 -- 172.21.15.27:0/1247903872 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c00cd80 con 0x7efe301094c0 2024-09-21T00:20:47.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.137+0000 7efe38600640 1 -- 172.21.15.27:0/1247903872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe301a2960 con 0x7efe301094c0 2024-09-21T00:20:47.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.137+0000 7efe2e000640 1 -- 172.21.15.27:0/1247903872 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe1c0035f0 con 0x7efe301094c0 2024-09-21T00:20:47.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.137+0000 7efe2e000640 1 -- 172.21.15.27:0/1247903872 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c010690 con 0x7efe301094c0 2024-09-21T00:20:47.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.137+0000 7efe38600640 1 -- 172.21.15.27:0/1247903872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe301a90c0 con 0x7efe301094c0 2024-09-21T00:20:47.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.138+0000 7efe38600640 1 -- 172.21.15.27:0/1247903872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe30103f90 con 0x7efe301094c0 2024-09-21T00:20:47.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.141+0000 7efe2e000640 1 -- 172.21.15.27:0/1247903872 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7efe1c003aa0 con 0x7efe301094c0 2024-09-21T00:20:47.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.142+0000 7efe2e000640 1 --2- 172.21.15.27:0/1247903872 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7efe180b65f0 0x7efe180b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:47.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.143+0000 7efe2e000640 1 -- 172.21.15.27:0/1247903872 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7efe1c0e7cd0 con 0x7efe301094c0 2024-09-21T00:20:47.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.143+0000 7efe35c00640 1 --2- 172.21.15.27:0/1247903872 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7efe180b65f0 0x7efe180b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:47.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.143+0000 7efe35c00640 1 --2- 172.21.15.27:0/1247903872 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7efe180b65f0 0x7efe180b8b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efe24009bd0 tx=0x7efe24005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:47.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.144+0000 7efe2e000640 1 -- 172.21.15.27:0/1247903872 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efe1c0adef0 con 0x7efe301094c0 2024-09-21T00:20:47.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:47 smithi027 ceph-mon[30745]: from='client.25169 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:20:47.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.382+0000 7efe38600640 1 -- 172.21.15.27:0/1247903872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efe3019d830 con 0x7efe301094c0 2024-09-21T00:20:47.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.382+0000 7efe2e000640 1 -- 172.21.15.27:0/1247903872 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7efe3019d830 con 0x7efe301094c0 2024-09-21T00:20:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:20:47.385 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T00:20:47.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.385+0000 7efe38600640 1 -- 172.21.15.27:0/1247903872 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7efe180b65f0 msgr2=0x7efe180b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:47.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.385+0000 7efe38600640 1 --2- 172.21.15.27:0/1247903872 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7efe180b65f0 0x7efe180b8b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efe24009bd0 tx=0x7efe24005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:47.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.385+0000 7efe38600640 1 -- 172.21.15.27:0/1247903872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe301094c0 msgr2=0x7efe3019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:47.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.385+0000 7efe38600640 1 --2- 172.21.15.27:0/1247903872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe301094c0 0x7efe3019cca0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00e970 tx=0x7efe1c00ee30 comp rx=0 tx=0).stop 2024-09-21T00:20:47.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.385+0000 7efe38600640 1 -- 172.21.15.27:0/1247903872 shutdown_connections 2024-09-21T00:20:47.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.385+0000 7efe38600640 1 --2- 172.21.15.27:0/1247903872 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7efe180b65f0 0x7efe180b8b00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:47.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.385+0000 7efe38600640 1 --2- 172.21.15.27:0/1247903872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe30109e10 0x7efe301a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:47.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.385+0000 7efe38600640 1 --2- 172.21.15.27:0/1247903872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe301094c0 0x7efe3019cca0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:47.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.385+0000 7efe38600640 1 --2- 172.21.15.27:0/1247903872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe30103b80 0x7efe3019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:47.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.385+0000 7efe38600640 1 -- 172.21.15.27:0/1247903872 >> 172.21.15.27:0/1247903872 conn(0x7efe300ff5d0 msgr2=0x7efe30106a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:47.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.385+0000 7efe38600640 1 -- 172.21.15.27:0/1247903872 shutdown_connections 2024-09-21T00:20:47.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:47.385+0000 7efe38600640 1 -- 172.21.15.27:0/1247903872 wait complete. 2024-09-21T00:20:47.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:47 smithi089 ceph-mon[34481]: from='client.25169 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:20:47.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:47 smithi098 ceph-mon[34725]: from='client.25169 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:20:48.215 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T00:20:48.215 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:20:48.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:48 smithi089 ceph-mon[34481]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:20:48.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:48 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1247903872' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T00:20:48.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:48 smithi098 ceph-mon[34725]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:20:48.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:48 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1247903872' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T00:20:48.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:48 smithi027 ceph-mon[30745]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:20:48.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:48 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1247903872' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T00:20:48.638 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:49.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.338+0000 7f0146800640 1 -- 172.21.15.27:0/2016189938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01400760a0 msgr2=0x7f014010fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:49.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.338+0000 7f0146800640 1 --2- 172.21.15.27:0/2016189938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01400760a0 0x7f014010fa30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f012c009900 tx=0x7f012c02a3f0 comp rx=0 tx=0).stop 2024-09-21T00:20:49.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.338+0000 7f0146800640 1 -- 172.21.15.27:0/2016189938 shutdown_connections 2024-09-21T00:20:49.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.338+0000 7f0146800640 1 --2- 172.21.15.27:0/2016189938 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f014010ff70 0x7f0140112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:49.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.338+0000 7f0146800640 1 --2- 172.21.15.27:0/2016189938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01400760a0 0x7f014010fa30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:49.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.338+0000 7f0146800640 1 --2- 172.21.15.27:0/2016189938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0140075750 0x7f0140075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:49.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.338+0000 7f0146800640 1 -- 172.21.15.27:0/2016189938 >> 172.21.15.27:0/2016189938 conn(0x7f01400ff400 msgr2=0x7f0140101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:49.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.339+0000 7f0146800640 1 -- 172.21.15.27:0/2016189938 shutdown_connections 2024-09-21T00:20:49.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.339+0000 7f0146800640 1 -- 172.21.15.27:0/2016189938 wait complete. 2024-09-21T00:20:49.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.340+0000 7f0146800640 1 Processor -- start 2024-09-21T00:20:49.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.340+0000 7f0146800640 1 -- start start 2024-09-21T00:20:49.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.340+0000 7f0146800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0140075750 0x7f014019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:49.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.340+0000 7f0146800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01400760a0 0x7f014019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:49.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.340+0000 7f0146800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f014010ff70 0x7f01401a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:49.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.340+0000 7f0146800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0140115ae0 con 0x7f0140075750 2024-09-21T00:20:49.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.340+0000 7f0146800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0140115960 con 0x7f014010ff70 2024-09-21T00:20:49.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.340+0000 7f0146800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0140115c60 con 0x7f01400760a0 2024-09-21T00:20:49.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.340+0000 7f0145400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0140075750 0x7f014019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:49.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.341+0000 7f0145400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0140075750 0x7f014019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51392/0 (socket says 172.21.15.27:51392) 2024-09-21T00:20:49.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.341+0000 7f0145400640 1 -- 172.21.15.27:0/3000497496 learned_addr learned my addr 172.21.15.27:0/3000497496 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:49.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.341+0000 7f0144a00640 1 --2- 172.21.15.27:0/3000497496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01400760a0 0x7f014019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:49.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.341+0000 7f0145e00640 1 --2- 172.21.15.27:0/3000497496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f014010ff70 0x7f01401a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:49.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.341+0000 7f0145400640 1 -- 172.21.15.27:0/3000497496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01400760a0 msgr2=0x7f014019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:49.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.341+0000 7f0145400640 1 --2- 172.21.15.27:0/3000497496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01400760a0 0x7f014019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:49.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.341+0000 7f0145400640 1 -- 172.21.15.27:0/3000497496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f014010ff70 msgr2=0x7f01401a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:49.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.341+0000 7f0145400640 1 --2- 172.21.15.27:0/3000497496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f014010ff70 0x7f01401a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:49.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.341+0000 7f0145400640 1 -- 172.21.15.27:0/3000497496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01401a2480 con 0x7f0140075750 2024-09-21T00:20:49.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.341+0000 7f0145400640 1 --2- 172.21.15.27:0/3000497496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0140075750 0x7f014019c510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f013400e9f0 tx=0x7f013400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:49.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.342+0000 7f013e000640 1 -- 172.21.15.27:0/3000497496 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013400cd80 con 0x7f0140075750 2024-09-21T00:20:49.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.342+0000 7f013e000640 1 -- 172.21.15.27:0/3000497496 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01340108e0 con 0x7f0140075750 2024-09-21T00:20:49.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.342+0000 7f0146800640 1 -- 172.21.15.27:0/3000497496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01401a2770 con 0x7f0140075750 2024-09-21T00:20:49.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.342+0000 7f013e000640 1 -- 172.21.15.27:0/3000497496 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134018980 con 0x7f0140075750 2024-09-21T00:20:49.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.342+0000 7f0146800640 1 -- 172.21.15.27:0/3000497496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01401a8ed0 con 0x7f0140075750 2024-09-21T00:20:49.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.343+0000 7f0146800640 1 -- 172.21.15.27:0/3000497496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0140075b60 con 0x7f0140075750 2024-09-21T00:20:49.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.345+0000 7f013e000640 1 -- 172.21.15.27:0/3000497496 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f0134010430 con 0x7f0140075750 2024-09-21T00:20:49.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.347+0000 7f013e000640 1 --2- 172.21.15.27:0/3000497496 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f01280b65f0 0x7f01280b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:49.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.347+0000 7f013e000640 1 -- 172.21.15.27:0/3000497496 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f01340e81c0 con 0x7f0140075750 2024-09-21T00:20:49.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.347+0000 7f0144a00640 1 --2- 172.21.15.27:0/3000497496 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f01280b65f0 0x7f01280b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:49.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.348+0000 7f0144a00640 1 --2- 172.21.15.27:0/3000497496 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f01280b65f0 0x7f01280b8b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f012c009230 tx=0x7f012c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:49.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.349+0000 7f013e000640 1 -- 172.21.15.27:0/3000497496 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f01340b27c0 con 0x7f0140075750 2024-09-21T00:20:49.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.593+0000 7f0146800640 1 -- 172.21.15.27:0/3000497496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01401a9100 con 0x7f0140075750 2024-09-21T00:20:49.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.594+0000 7f013e000640 1 -- 172.21.15.27:0/3000497496 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11893 (secure 0 0 0) 0x7f01340b7700 con 0x7f0140075750 2024-09-21T00:20:49.596 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:20:49.596 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"btime":"2024-09-21T00:19:38:403694+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:19:37.959611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24751,24727]},"id":1}]} 2024-09-21T00:20:49.597 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-09-21T00:20:49.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.597+0000 7f0146800640 1 -- 172.21.15.27:0/3000497496 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f01280b65f0 msgr2=0x7f01280b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:49.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.597+0000 7f0146800640 1 --2- 172.21.15.27:0/3000497496 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f01280b65f0 0x7f01280b8b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f012c009230 tx=0x7f012c031400 comp rx=0 tx=0).stop 2024-09-21T00:20:49.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.597+0000 7f0146800640 1 -- 172.21.15.27:0/3000497496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0140075750 msgr2=0x7f014019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:49.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.597+0000 7f0146800640 1 --2- 172.21.15.27:0/3000497496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0140075750 0x7f014019c510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f013400e9f0 tx=0x7f013400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:20:49.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.597+0000 7f0146800640 1 -- 172.21.15.27:0/3000497496 shutdown_connections 2024-09-21T00:20:49.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.597+0000 7f0146800640 1 --2- 172.21.15.27:0/3000497496 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f01280b65f0 0x7f01280b8b00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:49.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.597+0000 7f0146800640 1 --2- 172.21.15.27:0/3000497496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f014010ff70 0x7f01401a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:49.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.597+0000 7f0146800640 1 --2- 172.21.15.27:0/3000497496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01400760a0 0x7f014019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:49.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.597+0000 7f0146800640 1 --2- 172.21.15.27:0/3000497496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0140075750 0x7f014019c510 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:49.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.597+0000 7f0146800640 1 -- 172.21.15.27:0/3000497496 >> 172.21.15.27:0/3000497496 conn(0x7f01400ff400 msgr2=0x7f0140109b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:49.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.597+0000 7f0146800640 1 -- 172.21.15.27:0/3000497496 shutdown_connections 2024-09-21T00:20:49.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:49.597+0000 7f0146800640 1 -- 172.21.15.27:0/3000497496 wait complete. 2024-09-21T00:20:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:50 smithi027 ceph-mon[30745]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:50 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3000497496' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:20:50.374 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:20:50.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:50 smithi089 ceph-mon[34481]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:50.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:50 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3000497496' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:20:50.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:50 smithi098 ceph-mon[34725]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:50.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:50 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3000497496' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:20:50.762 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.473+0000 7f2af5e00640 1 -- 172.21.15.27:0/414032370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2af0114570 msgr2=0x7f2af0116990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.473+0000 7f2af5e00640 1 --2- 172.21.15.27:0/414032370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2af0114570 0x7f2af0116990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009ca0 tx=0x7f2ae802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:20:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.473+0000 7f2af5e00640 1 -- 172.21.15.27:0/414032370 shutdown_connections 2024-09-21T00:20:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.473+0000 7f2af5e00640 1 --2- 172.21.15.27:0/414032370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2af0114570 0x7f2af0116990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:51.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.473+0000 7f2af5e00640 1 --2- 172.21.15.27:0/414032370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af0078290 0x7f2af0114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:51.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.473+0000 7f2af5e00640 1 --2- 172.21.15.27:0/414032370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2af0077940 0x7f2af0077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:51.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.473+0000 7f2af5e00640 1 -- 172.21.15.27:0/414032370 >> 172.21.15.27:0/414032370 conn(0x7f2af0101680 msgr2=0x7f2af0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:51.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.473+0000 7f2af5e00640 1 -- 172.21.15.27:0/414032370 shutdown_connections 2024-09-21T00:20:51.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.473+0000 7f2af5e00640 1 -- 172.21.15.27:0/414032370 wait complete. 2024-09-21T00:20:51.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.474+0000 7f2af5e00640 1 Processor -- start 2024-09-21T00:20:51.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.474+0000 7f2af5e00640 1 -- start start 2024-09-21T00:20:51.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.474+0000 7f2af5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af0077940 0x7f2af01a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:51.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2af5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2af0078290 0x7f2af01a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:51.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2af5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2af0114570 0x7f2af01a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:51.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2af5e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2af0119ef0 con 0x7f2af0078290 2024-09-21T00:20:51.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2af5e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2af0119d70 con 0x7f2af0077940 2024-09-21T00:20:51.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2af5e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2af011a070 con 0x7f2af0114570 2024-09-21T00:20:51.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2aefe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2af0078290 0x7f2af01a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:51.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2af4a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af0077940 0x7f2af01a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:51.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2af4a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af0077940 0x7f2af01a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54984/0 (socket says 172.21.15.27:54984) 2024-09-21T00:20:51.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2af4a00640 1 -- 172.21.15.27:0/3387479120 learned_addr learned my addr 172.21.15.27:0/3387479120 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:51.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2aefe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2af0078290 0x7f2af01a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51396/0 (socket says 172.21.15.27:51396) 2024-09-21T00:20:51.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2af5400640 1 --2- 172.21.15.27:0/3387479120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2af0114570 0x7f2af01a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:51.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2af4a00640 1 -- 172.21.15.27:0/3387479120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2af0114570 msgr2=0x7f2af01a60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:51.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2af4a00640 1 --2- 172.21.15.27:0/3387479120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2af0114570 0x7f2af01a60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:51.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2af4a00640 1 -- 172.21.15.27:0/3387479120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2af0078290 msgr2=0x7f2af01a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:51.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2af4a00640 1 --2- 172.21.15.27:0/3387479120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2af0078290 0x7f2af01a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:51.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.475+0000 7f2af4a00640 1 -- 172.21.15.27:0/3387479120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af01a6810 con 0x7f2af0077940 2024-09-21T00:20:51.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.476+0000 7f2af4a00640 1 --2- 172.21.15.27:0/3387479120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af0077940 0x7f2af01a08a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400e9f0 tx=0x7f2ae400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:51.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.476+0000 7f2aed600640 1 -- 172.21.15.27:0/3387479120 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae400cd80 con 0x7f2af0077940 2024-09-21T00:20:51.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.476+0000 7f2af5e00640 1 -- 172.21.15.27:0/3387479120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af01a6b00 con 0x7f2af0077940 2024-09-21T00:20:51.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.476+0000 7f2aed600640 1 -- 172.21.15.27:0/3387479120 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ae40054d0 con 0x7f2af0077940 2024-09-21T00:20:51.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.476+0000 7f2aed600640 1 -- 172.21.15.27:0/3387479120 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4010690 con 0x7f2af0077940 2024-09-21T00:20:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.476+0000 7f2af5e00640 1 -- 172.21.15.27:0/3387479120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af01ad260 con 0x7f2af0077940 2024-09-21T00:20:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.477+0000 7f2af5e00640 1 -- 172.21.15.27:0/3387479120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af0077d50 con 0x7f2af0077940 2024-09-21T00:20:51.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.481+0000 7f2aed600640 1 -- 172.21.15.27:0/3387479120 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f2ae4004e90 con 0x7f2af0077940 2024-09-21T00:20:51.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.482+0000 7f2aed600640 1 --2- 172.21.15.27:0/3387479120 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2ad80b65f0 0x7f2ad80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:51.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.482+0000 7f2aed600640 1 -- 172.21.15.27:0/3387479120 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f2ae40e80e0 con 0x7f2af0077940 2024-09-21T00:20:51.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.482+0000 7f2aefe00640 1 --2- 172.21.15.27:0/3387479120 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2ad80b65f0 0x7f2ad80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:51.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.483+0000 7f2aefe00640 1 --2- 172.21.15.27:0/3387479120 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2ad80b65f0 0x7f2ad80b8b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2af007b240 tx=0x7f2adc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:51.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.483+0000 7f2aed600640 1 -- 172.21.15.27:0/3387479120 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2ae40ae300 con 0x7f2af0077940 2024-09-21T00:20:51.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.730+0000 7f2af5e00640 1 -- 172.21.15.27:0/3387479120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2af01a19d0 con 0x7f2af0077940 2024-09-21T00:20:51.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.731+0000 7f2aed600640 1 -- 172.21.15.27:0/3387479120 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11893 (secure 0 0 0) 0x7f2ae40b26e0 con 0x7f2af0077940 2024-09-21T00:20:51.733 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:20:51.733 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"btime":"2024-09-21T00:19:38:403694+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:19:37.959611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24751,24727]},"id":1}]} 2024-09-21T00:20:51.733 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-09-21T00:20:51.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.734+0000 7f2af5e00640 1 -- 172.21.15.27:0/3387479120 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2ad80b65f0 msgr2=0x7f2ad80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:51.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.734+0000 7f2af5e00640 1 --2- 172.21.15.27:0/3387479120 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2ad80b65f0 0x7f2ad80b8b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2af007b240 tx=0x7f2adc005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:51.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.734+0000 7f2af5e00640 1 -- 172.21.15.27:0/3387479120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af0077940 msgr2=0x7f2af01a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:51.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.734+0000 7f2af5e00640 1 --2- 172.21.15.27:0/3387479120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af0077940 0x7f2af01a08a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400e9f0 tx=0x7f2ae400edf0 comp rx=0 tx=0).stop 2024-09-21T00:20:51.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.734+0000 7f2af5e00640 1 -- 172.21.15.27:0/3387479120 shutdown_connections 2024-09-21T00:20:51.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.734+0000 7f2af5e00640 1 --2- 172.21.15.27:0/3387479120 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2ad80b65f0 0x7f2ad80b8b00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:51.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.734+0000 7f2af5e00640 1 --2- 172.21.15.27:0/3387479120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2af0114570 0x7f2af01a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:51.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.734+0000 7f2af5e00640 1 --2- 172.21.15.27:0/3387479120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2af0078290 0x7f2af01a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:51.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.734+0000 7f2af5e00640 1 --2- 172.21.15.27:0/3387479120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af0077940 0x7f2af01a08a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:51.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.734+0000 7f2af5e00640 1 -- 172.21.15.27:0/3387479120 >> 172.21.15.27:0/3387479120 conn(0x7f2af0101680 msgr2=0x7f2af01165a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:51.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.734+0000 7f2af5e00640 1 -- 172.21.15.27:0/3387479120 shutdown_connections 2024-09-21T00:20:51.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:51.735+0000 7f2af5e00640 1 -- 172.21.15.27:0/3387479120 wait complete. 2024-09-21T00:20:52.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:52 smithi089 ceph-mon[34481]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:20:52.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:52 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3387479120' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:20:52.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:52 smithi098 ceph-mon[34725]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:20:52.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:52 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3387479120' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:20:52.496 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:52 smithi027 ceph-mon[30745]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:20:52.496 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:52 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3387479120' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:20:52.534 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T00:20:52.921 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:53.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:53 smithi027 ceph-mon[30745]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:20:53.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:53 smithi089 ceph-mon[34481]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:20:53.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:53 smithi098 ceph-mon[34725]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:20:53.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.565+0000 7fc70d400640 1 -- 172.21.15.27:0/510667637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc70810b500 msgr2=0x7fc70810b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:53.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.565+0000 7fc70d400640 1 --2- 172.21.15.27:0/510667637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc70810b500 0x7fc70810b910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009900 tx=0x7fc6f402a410 comp rx=0 tx=0).stop 2024-09-21T00:20:53.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.565+0000 7fc70d400640 1 -- 172.21.15.27:0/510667637 shutdown_connections 2024-09-21T00:20:53.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.565+0000 7fc70d400640 1 --2- 172.21.15.27:0/510667637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc70810be50 0x7fc708112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:53.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.565+0000 7fc70d400640 1 --2- 172.21.15.27:0/510667637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc70810b500 0x7fc70810b910 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:53.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.565+0000 7fc70d400640 1 --2- 172.21.15.27:0/510667637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc708105bc0 0x7fc708105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:53.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.565+0000 7fc70d400640 1 -- 172.21.15.27:0/510667637 >> 172.21.15.27:0/510667637 conn(0x7fc708101630 msgr2=0x7fc708103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:53.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.565+0000 7fc70d400640 1 -- 172.21.15.27:0/510667637 shutdown_connections 2024-09-21T00:20:53.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.565+0000 7fc70d400640 1 -- 172.21.15.27:0/510667637 wait complete. 2024-09-21T00:20:53.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.567+0000 7fc70d400640 1 Processor -- start 2024-09-21T00:20:53.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.567+0000 7fc70d400640 1 -- start start 2024-09-21T00:20:53.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.567+0000 7fc70d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc708105bc0 0x7fc70819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:53.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.568+0000 7fc70d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70810b500 0x7fc70819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:53.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.568+0000 7fc70d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc70810be50 0x7fc7081a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:53.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.568+0000 7fc70d400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc708115b60 con 0x7fc70810be50 2024-09-21T00:20:53.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.568+0000 7fc70d400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081159e0 con 0x7fc708105bc0 2024-09-21T00:20:53.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.568+0000 7fc70d400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc708115ce0 con 0x7fc70810b500 2024-09-21T00:20:53.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.568+0000 7fc70ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc70810be50 0x7fc7081a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:53.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.568+0000 7fc707400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70810b500 0x7fc70819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:53.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.568+0000 7fc70ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc70810be50 0x7fc7081a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58086/0 (socket says 172.21.15.27:58086) 2024-09-21T00:20:53.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.568+0000 7fc70ca00640 1 -- 172.21.15.27:0/3509752690 learned_addr learned my addr 172.21.15.27:0/3509752690 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:53.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.568+0000 7fc707e00640 1 --2- 172.21.15.27:0/3509752690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc708105bc0 0x7fc70819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:53.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.569+0000 7fc707400640 1 -- 172.21.15.27:0/3509752690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc708105bc0 msgr2=0x7fc70819c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:53.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.569+0000 7fc707400640 1 --2- 172.21.15.27:0/3509752690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc708105bc0 0x7fc70819c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:53.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.569+0000 7fc707400640 1 -- 172.21.15.27:0/3509752690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc70810be50 msgr2=0x7fc7081a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:53.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.569+0000 7fc707400640 1 --2- 172.21.15.27:0/3509752690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc70810be50 0x7fc7081a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:53.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.569+0000 7fc707400640 1 -- 172.21.15.27:0/3509752690 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7081a2460 con 0x7fc70810b500 2024-09-21T00:20:53.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.569+0000 7fc70ca00640 1 --2- 172.21.15.27:0/3509752690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc70810be50 0x7fc7081a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:53.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.569+0000 7fc707400640 1 --2- 172.21.15.27:0/3509752690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70810b500 0x7fc70819ca90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009230 tx=0x7fc6f4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:53.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.569+0000 7fc704c00640 1 -- 172.21.15.27:0/3509752690 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f403f070 con 0x7fc70810b500 2024-09-21T00:20:53.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.569+0000 7fc704c00640 1 -- 172.21.15.27:0/3509752690 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6f4031b60 con 0x7fc70810b500 2024-09-21T00:20:53.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.569+0000 7fc70d400640 1 -- 172.21.15.27:0/3509752690 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7081a26f0 con 0x7fc70810b500 2024-09-21T00:20:53.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.569+0000 7fc704c00640 1 -- 172.21.15.27:0/3509752690 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4035030 con 0x7fc70810b500 2024-09-21T00:20:53.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.570+0000 7fc70d400640 1 -- 172.21.15.27:0/3509752690 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7081a8eb0 con 0x7fc70810b500 2024-09-21T00:20:53.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.571+0000 7fc70d400640 1 -- 172.21.15.27:0/3509752690 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc708105fd0 con 0x7fc70810b500 2024-09-21T00:20:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.573+0000 7fc704c00640 1 -- 172.21.15.27:0/3509752690 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fc6f4042070 con 0x7fc70810b500 2024-09-21T00:20:53.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.575+0000 7fc704c00640 1 --2- 172.21.15.27:0/3509752690 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc6ec0b6670 0x7fc6ec0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:53.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.575+0000 7fc704c00640 1 -- 172.21.15.27:0/3509752690 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fc6f4108720 con 0x7fc70810b500 2024-09-21T00:20:53.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.575+0000 7fc707e00640 1 --2- 172.21.15.27:0/3509752690 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc6ec0b6670 0x7fc6ec0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:53.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.576+0000 7fc707e00640 1 --2- 172.21.15.27:0/3509752690 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc6ec0b6670 0x7fc6ec0b8b80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009bd0 tx=0x7fc6f8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:53.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.577+0000 7fc704c00640 1 -- 172.21.15.27:0/3509752690 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc6f40d2d20 con 0x7fc70810b500 2024-09-21T00:20:53.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.776+0000 7fc70d400640 1 -- 172.21.15.27:0/3509752690 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc70810dc20 con 0x7fc70810b500 2024-09-21T00:20:53.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.778+0000 7fc704c00640 1 -- 172.21.15.27:0/3509752690 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25160 (secure 0 0 0) 0x7fc6f40d7c60 con 0x7fc70810b500 2024-09-21T00:20:53.780 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:20:53.780 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","created":"2024-09-21T00:09:20.149938+0000","modified":"2024-09-21T00:20:26.442227+0000","last_up_change":"2024-09-21T00:17:43.978785+0000","last_in_change":"2024-09-21T00:17:21.325177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:13:32.718543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:18:33.992414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:18:36.817691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:18:46.707770+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63ccc094-62d5-446f-844d-1c60d41fda1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6803","nonce":354370090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6805","nonce":354370090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6809","nonce":354370090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6807","nonce":354370090}]},"public_addr":"172.21.15.27:6803/354370090","cluster_addr":"172.21.15.27:6805/354370090","heartbeat_back_addr":"172.21.15.27:6809/354370090","heartbeat_front_addr":"172.21.15.27:6807/354370090","state":["exists","up"]},{"osd":1,"uuid":"e59a8ed4-3a37-4477-a60a-4abc33d49b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6801","nonce":922528621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6803","nonce":922528621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6807","nonce":922528621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6805","nonce":922528621}]},"public_addr":"172.21.15.89:6801/922528621","cluster_addr":"172.21.15.89:6803/922528621","heartbeat_back_addr":"172.21.15.89:6807/922528621","heartbeat_front_addr":"172.21.15.89:6805/922528621","state":["exists","up"]},{"osd":2,"uuid":"6058a875-d5b2-4459-9407-2fd190cb34a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6801","nonce":257636259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6803","nonce":257636259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6807","nonce":257636259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6805","nonce":257636259}]},"public_addr":"172.21.15.98:6801/257636259","cluster_addr":"172.21.15.98:6803/257636259","heartbeat_back_addr":"172.21.15.98:6807/257636259","heartbeat_front_addr":"172.21.15.98:6805/257636259","state":["exists","up"]},{"osd":3,"uuid":"dfcdaede-e51c-4985-92af-3e296435a36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6811","nonce":821088824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6813","nonce":821088824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6817","nonce":821088824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6815","nonce":821088824}]},"public_addr":"172.21.15.27:6811/821088824","cluster_addr":"172.21.15.27:6813/821088824","heartbeat_back_addr":"172.21.15.27:6817/821088824","heartbeat_front_addr":"172.21.15.27:6815/821088824","state":["exists","up"]},{"osd":4,"uuid":"7c4af037-8ab6-47c0-b62b-76bd775b3491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6809","nonce":4001038690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6811","nonce":4001038690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6815","nonce":4001038690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6813","nonce":4001038690}]},"public_addr":"172.21.15.89:6809/4001038690","cluster_addr":"172.21.15.89:6811/4001038690","heartbeat_back_addr":"172.21.15.89:6815/4001038690","heartbeat_front_addr":"172.21.15.89:6813/4001038690","state":["exists","up"]},{"osd":5,"uuid":"e996ed0d-88ad-4aa7-891f-dfade4e09cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6809","nonce":3029903562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6811","nonce":3029903562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6815","nonce":3029903562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6813","nonce":3029903562}]},"public_addr":"172.21.15.98:6809/3029903562","cluster_addr":"172.21.15.98:6811/3029903562","heartbeat_back_addr":"172.21.15.98:6815/3029903562","heartbeat_front_addr":"172.21.15.98:6813/3029903562","state":["exists","up"]},{"osd":6,"uuid":"cf080497-57c2-42d0-8c8f-1eed0110702d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6819","nonce":1286808857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6821","nonce":1286808857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6825","nonce":1286808857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6823","nonce":1286808857}]},"public_addr":"172.21.15.27:6819/1286808857","cluster_addr":"172.21.15.27:6821/1286808857","heartbeat_back_addr":"172.21.15.27:6825/1286808857","heartbeat_front_addr":"172.21.15.27:6823/1286808857","state":["exists","up"]},{"osd":7,"uuid":"ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6817","nonce":2479349546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6819","nonce":2479349546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6823","nonce":2479349546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6821","nonce":2479349546}]},"public_addr":"172.21.15.89:6817/2479349546","cluster_addr":"172.21.15.89:6819/2479349546","heartbeat_back_addr":"172.21.15.89:6823/2479349546","heartbeat_front_addr":"172.21.15.89:6821/2479349546","state":["exists","up"]},{"osd":8,"uuid":"0b558458-fc2d-4ed1-b72a-e2ea8cf811b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6817","nonce":2830381485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6819","nonce":2830381485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6823","nonce":2830381485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6821","nonce":2830381485}]},"public_addr":"172.21.15.98:6817/2830381485","cluster_addr":"172.21.15.98:6819/2830381485","heartbeat_back_addr":"172.21.15.98:6823/2830381485","heartbeat_front_addr":"172.21.15.98:6821/2830381485","state":["exists","up"]},{"osd":9,"uuid":"1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6827","nonce":3033977113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6829","nonce":3033977113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6833","nonce":3033977113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6831","nonce":3033977113}]},"public_addr":"172.21.15.27:6827/3033977113","cluster_addr":"172.21.15.27:6829/3033977113","heartbeat_back_addr":"172.21.15.27:6833/3033977113","heartbeat_front_addr":"172.21.15.27:6831/3033977113","state":["exists","up"]},{"osd":10,"uuid":"324dd6ce-68fe-4c76-959b-1c9fb0ae92a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6825","nonce":2370332225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6827","nonce":2370332225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6831","nonce":2370332225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6829","nonce":2370332225}]},"public_addr":"172.21.15.89:6825/2370332225","cluster_addr":"172.21.15.89:6827/2370332225","heartbeat_back_addr":"172.21.15.89:6831/2370332225","heartbeat_front_addr":"172.21.15.89:6829/2370332225","state":["exists","up"]},{"osd":11,"uuid":"5e34686e-350a-491e-b9d7-59174ea90944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6825","nonce":2795999064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6827","nonce":2795999064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6831","nonce":2795999064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6829","nonce":2795999064}]},"public_addr":"172.21.15.98:6825/2795999064","cluster_addr":"172.21.15.98:6827/2795999064","heartbeat_back_addr":"172.21.15.98:6831/2795999064","heartbeat_front_addr":"172.21.15.98:6829/2795999064","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:12:39.841958+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:06.684417+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:31.041378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:01.612108+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:28.592326+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:54.303714+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:25.523475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:52.523483+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:19.430503+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:50.599120+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:17.520452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:42.725550+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/502883064":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/831048090":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/3880378905":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/793991452":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/2077956479":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6800/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6801/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/1098432608":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6800/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/528019907":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/2173601508":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:6800/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/764776233":"2024-09-22T00:11:24.231101+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:20:53.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.781+0000 7fc70d400640 1 -- 172.21.15.27:0/3509752690 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc6ec0b6670 msgr2=0x7fc6ec0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:53.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.781+0000 7fc70d400640 1 --2- 172.21.15.27:0/3509752690 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc6ec0b6670 0x7fc6ec0b8b80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009bd0 tx=0x7fc6f8005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:53.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.781+0000 7fc70d400640 1 -- 172.21.15.27:0/3509752690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70810b500 msgr2=0x7fc70819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:53.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.781+0000 7fc70d400640 1 --2- 172.21.15.27:0/3509752690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70810b500 0x7fc70819ca90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009230 tx=0x7fc6f4003c10 comp rx=0 tx=0).stop 2024-09-21T00:20:53.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.781+0000 7fc70d400640 1 -- 172.21.15.27:0/3509752690 shutdown_connections 2024-09-21T00:20:53.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.781+0000 7fc70d400640 1 --2- 172.21.15.27:0/3509752690 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc6ec0b6670 0x7fc6ec0b8b80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:53.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.781+0000 7fc70d400640 1 --2- 172.21.15.27:0/3509752690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc70810be50 0x7fc7081a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:53.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.781+0000 7fc70d400640 1 --2- 172.21.15.27:0/3509752690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70810b500 0x7fc70819ca90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:53.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.781+0000 7fc70d400640 1 --2- 172.21.15.27:0/3509752690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc708105bc0 0x7fc70819c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:53.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.781+0000 7fc70d400640 1 -- 172.21.15.27:0/3509752690 >> 172.21.15.27:0/3509752690 conn(0x7fc708101630 msgr2=0x7fc708103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:53.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.781+0000 7fc70d400640 1 -- 172.21.15.27:0/3509752690 shutdown_connections 2024-09-21T00:20:53.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:53.781+0000 7fc70d400640 1 -- 172.21.15.27:0/3509752690 wait complete. 2024-09-21T00:20:54.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:54 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3509752690' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:20:54.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:54 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3509752690' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:20:54.499 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:54 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3509752690' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:20:54.539 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:20:54.954 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:55.370 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:55 smithi027 ceph-mon[30745]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:55.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:55 smithi089 ceph-mon[34481]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:55.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:55 smithi098 ceph-mon[34725]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:20:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.551+0000 7fd838400640 1 -- 172.21.15.27:0/56394023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd83010c010 msgr2=0x7fd830112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.551+0000 7fd838400640 1 --2- 172.21.15.27:0/56394023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd83010c010 0x7fd830112840 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd828009ca0 tx=0x7fd82802a8b0 comp rx=0 tx=0).stop 2024-09-21T00:20:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.551+0000 7fd838400640 1 -- 172.21.15.27:0/56394023 shutdown_connections 2024-09-21T00:20:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.551+0000 7fd838400640 1 --2- 172.21.15.27:0/56394023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd83010c010 0x7fd830112840 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.551+0000 7fd838400640 1 --2- 172.21.15.27:0/56394023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd83010b6c0 0x7fd83010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.551+0000 7fd838400640 1 --2- 172.21.15.27:0/56394023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd830105d80 0x7fd830106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.551+0000 7fd838400640 1 -- 172.21.15.27:0/56394023 >> 172.21.15.27:0/56394023 conn(0x7fd830101810 msgr2=0x7fd830103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.551+0000 7fd838400640 1 -- 172.21.15.27:0/56394023 shutdown_connections 2024-09-21T00:20:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.551+0000 7fd838400640 1 -- 172.21.15.27:0/56394023 wait complete. 2024-09-21T00:20:55.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.553+0000 7fd838400640 1 Processor -- start 2024-09-21T00:20:55.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.553+0000 7fd838400640 1 -- start start 2024-09-21T00:20:55.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd838400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd830105d80 0x7fd83019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:55.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd838400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd83010b6c0 0x7fd83019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:55.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd838400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd83010c010 0x7fd8301a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:55.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd838400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd830115d20 con 0x7fd83010c010 2024-09-21T00:20:55.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd838400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd830115ba0 con 0x7fd830105d80 2024-09-21T00:20:55.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd838400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd830115ea0 con 0x7fd83010b6c0 2024-09-21T00:20:55.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd835a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd830105d80 0x7fd83019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:55.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd835000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd83010b6c0 0x7fd83019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:55.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd835a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd830105d80 0x7fd83019c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58794/0 (socket says 172.21.15.27:58794) 2024-09-21T00:20:55.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd835000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd83010b6c0 0x7fd83019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55890/0 (socket says 172.21.15.27:55890) 2024-09-21T00:20:55.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd835a00640 1 -- 172.21.15.27:0/2878303708 learned_addr learned my addr 172.21.15.27:0/2878303708 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:55.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd836400640 1 --2- 172.21.15.27:0/2878303708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd83010c010 0x7fd8301a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:55.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd835a00640 1 -- 172.21.15.27:0/2878303708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd83010b6c0 msgr2=0x7fd83019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd835a00640 1 --2- 172.21.15.27:0/2878303708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd83010b6c0 0x7fd83019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd835a00640 1 -- 172.21.15.27:0/2878303708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd83010c010 msgr2=0x7fd8301a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd835a00640 1 --2- 172.21.15.27:0/2878303708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd83010c010 0x7fd8301a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.554+0000 7fd835a00640 1 -- 172.21.15.27:0/2878303708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8301a26a0 con 0x7fd830105d80 2024-09-21T00:20:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.555+0000 7fd835000640 1 --2- 172.21.15.27:0/2878303708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd83010b6c0 0x7fd83019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.555+0000 7fd835a00640 1 --2- 172.21.15.27:0/2878303708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd830105d80 0x7fd83019c730 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd82400e9f0 tx=0x7fd82400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.555+0000 7fd82e000640 1 -- 172.21.15.27:0/2878303708 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82400cd80 con 0x7fd830105d80 2024-09-21T00:20:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.555+0000 7fd82e000640 1 -- 172.21.15.27:0/2878303708 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8240108e0 con 0x7fd830105d80 2024-09-21T00:20:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.555+0000 7fd82e000640 1 -- 172.21.15.27:0/2878303708 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd824018980 con 0x7fd830105d80 2024-09-21T00:20:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.555+0000 7fd838400640 1 -- 172.21.15.27:0/2878303708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8301a2990 con 0x7fd830105d80 2024-09-21T00:20:55.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.555+0000 7fd838400640 1 -- 172.21.15.27:0/2878303708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8301a90f0 con 0x7fd830105d80 2024-09-21T00:20:55.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.556+0000 7fd838400640 1 -- 172.21.15.27:0/2878303708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd830106190 con 0x7fd830105d80 2024-09-21T00:20:55.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.559+0000 7fd82e000640 1 -- 172.21.15.27:0/2878303708 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fd824010430 con 0x7fd830105d80 2024-09-21T00:20:55.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.561+0000 7fd82e000640 1 --2- 172.21.15.27:0/2878303708 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd8180b6670 0x7fd8180b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.561+0000 7fd82e000640 1 -- 172.21.15.27:0/2878303708 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fd8240e8140 con 0x7fd830105d80 2024-09-21T00:20:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.561+0000 7fd835000640 1 --2- 172.21.15.27:0/2878303708 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd8180b6670 0x7fd8180b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.562+0000 7fd835000640 1 --2- 172.21.15.27:0/2878303708 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd8180b6670 0x7fd8180b8b80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd830104f10 tx=0x7fd81c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:55.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.562+0000 7fd82e000640 1 -- 172.21.15.27:0/2878303708 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd8240b2740 con 0x7fd830105d80 2024-09-21T00:20:55.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.799+0000 7fd838400640 1 -- 172.21.15.27:0/2878303708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd83002e640 con 0x7fd830105d80 2024-09-21T00:20:55.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.800+0000 7fd82e000640 1 -- 172.21.15.27:0/2878303708 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11893 (secure 0 0 0) 0x7fd8240b7680 con 0x7fd830105d80 2024-09-21T00:20:55.802 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:20:55.802 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"btime":"2024-09-21T00:19:38:403694+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:19:37.959611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24751,24727]},"id":1}]} 2024-09-21T00:20:55.802 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-09-21T00:20:55.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.803+0000 7fd838400640 1 -- 172.21.15.27:0/2878303708 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd8180b6670 msgr2=0x7fd8180b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:55.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.803+0000 7fd838400640 1 --2- 172.21.15.27:0/2878303708 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd8180b6670 0x7fd8180b8b80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd830104f10 tx=0x7fd81c005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:55.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.803+0000 7fd838400640 1 -- 172.21.15.27:0/2878303708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd830105d80 msgr2=0x7fd83019c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:55.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.803+0000 7fd838400640 1 --2- 172.21.15.27:0/2878303708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd830105d80 0x7fd83019c730 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd82400e9f0 tx=0x7fd82400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:20:55.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.803+0000 7fd838400640 1 -- 172.21.15.27:0/2878303708 shutdown_connections 2024-09-21T00:20:55.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.803+0000 7fd838400640 1 --2- 172.21.15.27:0/2878303708 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd8180b6670 0x7fd8180b8b80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:55.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.803+0000 7fd838400640 1 --2- 172.21.15.27:0/2878303708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd83010c010 0x7fd8301a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:55.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.803+0000 7fd838400640 1 --2- 172.21.15.27:0/2878303708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd83010b6c0 0x7fd83019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:55.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.803+0000 7fd838400640 1 --2- 172.21.15.27:0/2878303708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd830105d80 0x7fd83019c730 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:55.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.803+0000 7fd838400640 1 -- 172.21.15.27:0/2878303708 >> 172.21.15.27:0/2878303708 conn(0x7fd830101810 msgr2=0x7fd83010acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:55.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.803+0000 7fd838400640 1 -- 172.21.15.27:0/2878303708 shutdown_connections 2024-09-21T00:20:55.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:55.803+0000 7fd838400640 1 -- 172.21.15.27:0/2878303708 wait complete. 2024-09-21T00:20:56.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:56 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2878303708' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:20:56.498 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:56 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2878303708' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:20:56.517 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:56 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2878303708' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:20:56.555 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T00:18:39.809850+0000', 'modified': '2024-09-21T00:19:37.959611+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24761, 'mds_1': 24751, 'mds_2': 24727}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24703': {'gid': 24703, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6835/388589971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 388589971}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 388589971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24715': {'gid': 24715, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6837/3858073278', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 3858073278}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 3858073278}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24722': {'gid': 24722, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6837/3044996985', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6836', 'nonce': 3044996985}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 3044996985}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24727': {'gid': 24727, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.98:6839/3074699335', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 3074699335}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 3074699335}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24751': {'gid': 24751, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.27:6841/378707351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6840', 'nonce': 378707351}, {'type': 'v1', 'addr': '172.21.15.27:6841', 'nonce': 378707351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24761': {'gid': 24761, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.89:6839/2432693964', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 2432693964}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 2432693964}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24761, 'qdb_cluster': [24761, 24751, 24727]} 2024-09-21T00:20:56.555 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T00:20:56.556 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T00:20:56.947 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:57.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:57 smithi089 ceph-mon[34481]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-21T00:20:57.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:57 smithi098 ceph-mon[34725]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-21T00:20:57.529 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:57 smithi027 ceph-mon[30745]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-21T00:20:57.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.712+0000 7f5592200640 1 -- 172.21.15.27:0/2627852698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c114740 msgr2=0x7f558c116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:57.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.712+0000 7f5592200640 1 --2- 172.21.15.27:0/2627852698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c114740 0x7f558c116b60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5584009ca0 tx=0x7f558402a8c0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.713+0000 7f5592200640 1 -- 172.21.15.27:0/2627852698 shutdown_connections 2024-09-21T00:20:57.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.713+0000 7f5592200640 1 --2- 172.21.15.27:0/2627852698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c114740 0x7f558c116b60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.713+0000 7f5592200640 1 --2- 172.21.15.27:0/2627852698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f558c078290 0x7f558c114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.713+0000 7f5592200640 1 --2- 172.21.15.27:0/2627852698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f558c077940 0x7f558c077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.713+0000 7f5592200640 1 -- 172.21.15.27:0/2627852698 >> 172.21.15.27:0/2627852698 conn(0x7f558c101810 msgr2=0x7f558c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:57.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.713+0000 7f5592200640 1 -- 172.21.15.27:0/2627852698 shutdown_connections 2024-09-21T00:20:57.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.713+0000 7f5592200640 1 -- 172.21.15.27:0/2627852698 wait complete. 2024-09-21T00:20:57.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.714+0000 7f5592200640 1 Processor -- start 2024-09-21T00:20:57.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.714+0000 7f5592200640 1 -- start start 2024-09-21T00:20:57.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.715+0000 7f5592200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c077940 0x7f558c1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.715+0000 7f5592200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f558c078290 0x7f558c1a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.715+0000 7f5592200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f558c114740 0x7f558c1a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.715+0000 7f5592200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f558c11a0d0 con 0x7f558c114740 2024-09-21T00:20:57.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.715+0000 7f5592200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f558c119f50 con 0x7f558c077940 2024-09-21T00:20:57.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.715+0000 7f5592200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f558c11a250 con 0x7f558c078290 2024-09-21T00:20:57.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.715+0000 7f558b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c077940 0x7f558c1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.715+0000 7f558b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c077940 0x7f558c1a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58808/0 (socket says 172.21.15.27:58808) 2024-09-21T00:20:57.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.715+0000 7f558be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f558c114740 0x7f558c1a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.715+0000 7f558b400640 1 -- 172.21.15.27:0/2666570833 learned_addr learned my addr 172.21.15.27:0/2666570833 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:57.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.715+0000 7f5582a00640 1 --2- 172.21.15.27:0/2666570833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f558c078290 0x7f558c1a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.716+0000 7f5582a00640 1 -- 172.21.15.27:0/2666570833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c077940 msgr2=0x7f558c1a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:57.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.716+0000 7f5582a00640 1 --2- 172.21.15.27:0/2666570833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c077940 0x7f558c1a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.716+0000 7f5582a00640 1 -- 172.21.15.27:0/2666570833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f558c114740 msgr2=0x7f558c1a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:57.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.716+0000 7f5582a00640 1 --2- 172.21.15.27:0/2666570833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f558c114740 0x7f558c1a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.716+0000 7f5582a00640 1 -- 172.21.15.27:0/2666570833 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c1a69d0 con 0x7f558c078290 2024-09-21T00:20:57.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.716+0000 7f5582a00640 1 --2- 172.21.15.27:0/2666570833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f558c078290 0x7f558c1a1000 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f558c06caa0 tx=0x7f557400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:57.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.716+0000 7f5588c00640 1 -- 172.21.15.27:0/2666570833 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557400cc40 con 0x7f558c078290 2024-09-21T00:20:57.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.716+0000 7f5588c00640 1 -- 172.21.15.27:0/2666570833 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f557400cde0 con 0x7f558c078290 2024-09-21T00:20:57.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.716+0000 7f5592200640 1 -- 172.21.15.27:0/2666570833 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c1a6cc0 con 0x7f558c078290 2024-09-21T00:20:57.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.716+0000 7f5588c00640 1 -- 172.21.15.27:0/2666570833 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574004a00 con 0x7f558c078290 2024-09-21T00:20:57.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.717+0000 7f5592200640 1 -- 172.21.15.27:0/2666570833 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c1ad420 con 0x7f558c078290 2024-09-21T00:20:57.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.720+0000 7f5599003740 1 Processor -- start 2024-09-21T00:20:57.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.720+0000 7f5599003740 1 -- start start 2024-09-21T00:20:57.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.720+0000 7f5588c00640 1 -- 172.21.15.27:0/2666570833 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f5574004ba0 con 0x7f558c078290 2024-09-21T00:20:57.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.720+0000 7f5599003740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e71f60 0x559999218490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.720+0000 7f5599003740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e490a0 0x5599992189d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.721+0000 7f5599003740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e51280 0x559999218f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.721+0000 7f5599003740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55999918fd50 con 0x559998e490a0 2024-09-21T00:20:57.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.721+0000 7f5599003740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55999917f180 con 0x559998e71f60 2024-09-21T00:20:57.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.721+0000 7f5599003740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559999182f90 con 0x559998e51280 2024-09-21T00:20:57.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.721+0000 7f558b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e490a0 0x5599992189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.721+0000 7f558be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e71f60 0x559999218490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.721+0000 7f558b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e490a0 0x5599992189d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58118/0 (socket says 172.21.15.27:58118) 2024-09-21T00:20:57.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.721+0000 7f5582a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e51280 0x559999218f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.721+0000 7f558b400640 1 -- 172.21.15.27:0/666303136 learned_addr learned my addr 172.21.15.27:0/666303136 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:57.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.721+0000 7f558be00640 1 -- 172.21.15.27:0/666303136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e51280 msgr2=0x559999218f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:57.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.721+0000 7f558be00640 1 --2- 172.21.15.27:0/666303136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e51280 0x559999218f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.721+0000 7f558be00640 1 -- 172.21.15.27:0/666303136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e490a0 msgr2=0x5599992189d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:57.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.721+0000 7f558be00640 1 --2- 172.21.15.27:0/666303136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e490a0 0x5599992189d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.721+0000 7f558be00640 1 -- 172.21.15.27:0/666303136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559999149a70 con 0x559998e71f60 2024-09-21T00:20:57.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.721+0000 7f558b400640 1 --2- 172.21.15.27:0/666303136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e490a0 0x5599992189d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:57.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.722+0000 7f558be00640 1 --2- 172.21.15.27:0/666303136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e71f60 0x559999218490 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5584005e30 tx=0x7f558402f410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83670535b374344b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:57.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.722+0000 7f557be00640 1 -- 172.21.15.27:0/666303136 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584031cf0 con 0x559998e71f60 2024-09-21T00:20:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.722+0000 7f557be00640 1 -- 172.21.15.27:0/666303136 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5584036030 con 0x559998e71f60 2024-09-21T00:20:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.722+0000 7f557be00640 1 -- 172.21.15.27:0/666303136 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584004040 con 0x559998e71f60 2024-09-21T00:20:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.722+0000 7f5588c00640 1 --2- 172.21.15.27:0/2666570833 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f55700b65e0 0x7f55700b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.723+0000 7f5588c00640 1 -- 172.21.15.27:0/2666570833 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f55740e7740 con 0x7f558c078290 2024-09-21T00:20:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.723+0000 7f558b400640 1 --2- 172.21.15.27:0/2666570833 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f55700b65e0 0x7f55700b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.723+0000 7f5599003740 1 -- 172.21.15.27:0/666303136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e71f60 msgr2=0x559999218490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.723+0000 7f5599003740 1 --2- 172.21.15.27:0/666303136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e71f60 0x559999218490 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5584005e30 tx=0x7f558402f410 comp rx=0 tx=0).stop 2024-09-21T00:20:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.723+0000 7f5599003740 1 -- 172.21.15.27:0/666303136 shutdown_connections 2024-09-21T00:20:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.723+0000 7f5599003740 1 --2- 172.21.15.27:0/666303136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e51280 0x559999218f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.723+0000 7f5599003740 1 --2- 172.21.15.27:0/666303136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e490a0 0x5599992189d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.723+0000 7f5599003740 1 --2- 172.21.15.27:0/666303136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e71f60 0x559999218490 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.723+0000 7f5599003740 1 -- 172.21.15.27:0/666303136 >> 172.21.15.27:0/666303136 conn(0x559998e66350 msgr2=0x559999216380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:57.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.723+0000 7f5599003740 1 -- 172.21.15.27:0/666303136 shutdown_connections 2024-09-21T00:20:57.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.723+0000 7f5599003740 1 -- 172.21.15.27:0/666303136 wait complete. 2024-09-21T00:20:57.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.724+0000 7f558b400640 1 --2- 172.21.15.27:0/2666570833 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f55700b65e0 0x7f55700b8af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x559999217110 tx=0x7f557c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:57.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.724+0000 7f5599003740 1 -- start start 2024-09-21T00:20:57.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.724+0000 7f5599003740 1 Processor -- start 2024-09-21T00:20:57.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.725+0000 7f5599003740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e490a0 0x559999224050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.725+0000 7f5599003740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e51280 0x559999224590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.725+0000 7f5599003740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e71f60 0x55999922fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.725+0000 7f5599003740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559998fe05f0 con 0x559998e51280 2024-09-21T00:20:57.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.725+0000 7f5599003740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55999911a460 con 0x559998e490a0 2024-09-21T00:20:57.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.725+0000 7f5599003740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559999226a90 con 0x559998e71f60 2024-09-21T00:20:57.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.725+0000 7f5599003740 10 client.0 _ll_register_callbacks cb 0x55999912a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:20:57.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.725+0000 7f558b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e490a0 0x559999224050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.725+0000 7f558b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e490a0 0x559999224050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58816/0 (socket says 172.21.15.27:58816) 2024-09-21T00:20:57.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.725+0000 7f558b400640 1 -- 172.21.15.27:0/2674970919 learned_addr learned my addr 172.21.15.27:0/2674970919 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:57.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.725+0000 7f5582a00640 1 --2- 172.21.15.27:0/2674970919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e51280 0x559999224590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.726+0000 7f558be00640 1 --2- 172.21.15.27:0/2674970919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e71f60 0x55999922fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.726+0000 7f558b400640 1 -- 172.21.15.27:0/2674970919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e71f60 msgr2=0x55999922fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:57.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.726+0000 7f558b400640 1 --2- 172.21.15.27:0/2674970919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e71f60 0x55999922fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.726+0000 7f558b400640 1 -- 172.21.15.27:0/2674970919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e51280 msgr2=0x559999224590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:57.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.726+0000 7f558b400640 1 --2- 172.21.15.27:0/2674970919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e51280 0x559999224590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.726+0000 7f558b400640 1 -- 172.21.15.27:0/2674970919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559998edd540 con 0x559998e490a0 2024-09-21T00:20:57.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.726+0000 7f5582a00640 1 --2- 172.21.15.27:0/2674970919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e51280 0x559999224590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:20:57.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.726+0000 7f558b400640 1 --2- 172.21.15.27:0/2674970919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e490a0 0x559999224050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f557c00fa50 tx=0x7f557c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2ee1ddff221bc03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:57.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.726+0000 7f5578c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:20:57.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.726+0000 7f5578c00640 1 -- 172.21.15.27:0/2674970919 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c005020 con 0x559998e490a0 2024-09-21T00:20:57.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.727+0000 7f5599003740 1 -- 172.21.15.27:0/2674970919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55999911ca70 con 0x559998e490a0 2024-09-21T00:20:57.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.727+0000 7f5578c00640 1 -- 172.21.15.27:0/2674970919 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f557c018030 con 0x559998e490a0 2024-09-21T00:20:57.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.727+0000 7f5578c00640 1 -- 172.21.15.27:0/2674970919 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f557c033050 con 0x559998e490a0 2024-09-21T00:20:57.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.727+0000 7f5599003740 10 client.25211 fetch_fsmap learned FSMap version 25 2024-09-21T00:20:57.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.727+0000 7f5599003740 1 -- 172.21.15.27:0/2674970919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5599991376f0 con 0x559998e490a0 2024-09-21T00:20:57.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.727+0000 7f5578c00640 1 -- 172.21.15.27:0/2674970919 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55999911ca70 con 0x559998e490a0 2024-09-21T00:20:57.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.727+0000 7f5578c00640 1 -- 172.21.15.27:0/2674970919 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f557c01d620 con 0x559998e490a0 2024-09-21T00:20:57.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.728+0000 7f5599003740 10 client.25211 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T00:20:57.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.728+0000 7f5599003740 10 client.25211 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T00:20:57.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.728+0000 7f5599003740 20 client.25211 populate_metadata read hostname 'smithi027' 2024-09-21T00:20:57.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.728+0000 7f5599003740 1 --2- 172.21.15.27:0/2674970919 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x559998e308e0 0x559999236360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.728+0000 7f5599003740 4 client.25211 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:20:57.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.728+0000 7f5599003740 1 -- 172.21.15.27:0/2674970919 --> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559999191ca0 con 0x559998e308e0 2024-09-21T00:20:57.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.728+0000 7f5582a00640 1 --2- 172.21.15.27:0/2674970919 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x559998e308e0 0x559999236360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.729+0000 7f5582a00640 1 --2- 172.21.15.27:0/2674970919 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x559998e308e0 0x559999236360 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2476541f17783ac server_cookie=d7e33e8bb0b52668 in_seq=0 out_seq=0 2024-09-21T00:20:57.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.729+0000 7f5578c00640 10 client.25211 ms_handle_connect on v2:172.21.15.98:6838/3074699335 2024-09-21T00:20:57.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.730+0000 7f5578c00640 1 -- 172.21.15.27:0/2674970919 <== mds.2 v2:172.21.15.98:6838/3074699335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559999191ca0 con 0x559998e308e0 2024-09-21T00:20:57.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.730+0000 7f5578c00640 10 client.25211 handle_command_reply: tid=0 2024-09-21T00:20:57.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.730+0000 7f5599003740 1 client.25211 shutdown 2024-09-21T00:20:57.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.730+0000 7f5599003740 1 -- 172.21.15.27:0/2674970919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e490a0 msgr2=0x559999224050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:57.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.730+0000 7f5599003740 1 --2- 172.21.15.27:0/2674970919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e490a0 0x559999224050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f557c00fa50 tx=0x7f557c004730 comp rx=0 tx=0).stop 2024-09-21T00:20:57.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.730+0000 7f5599003740 1 -- 172.21.15.27:0/2674970919 shutdown_connections 2024-09-21T00:20:57.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.730+0000 7f5599003740 1 --2- 172.21.15.27:0/2674970919 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x559998e308e0 0x559999236360 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.730+0000 7f5599003740 1 --2- 172.21.15.27:0/2674970919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e71f60 0x55999922fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.730+0000 7f5599003740 1 --2- 172.21.15.27:0/2674970919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e51280 0x559999224590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.730+0000 7f5578c00640 0 client.25211 ms_handle_reset on v2:172.21.15.98:6838/3074699335 2024-09-21T00:20:57.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.730+0000 7f5599003740 1 --2- 172.21.15.27:0/2674970919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e490a0 0x559999224050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.730+0000 7f5599003740 1 -- 172.21.15.27:0/2674970919 >> 172.21.15.27:0/2674970919 conn(0x559998e66350 msgr2=0x55999921b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:57.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.730+0000 7f5599003740 1 -- 172.21.15.27:0/2674970919 shutdown_connections 2024-09-21T00:20:57.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.730+0000 7f5599003740 1 -- 172.21.15.27:0/2674970919 wait complete. 2024-09-21T00:20:57.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.731+0000 7f5599003740 20 client.25211 trim_cache size 0 max 16384 2024-09-21T00:20:57.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.749+0000 7f5599003740 1 Processor -- start 2024-09-21T00:20:57.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.749+0000 7f5599003740 1 -- start start 2024-09-21T00:20:57.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.750+0000 7f5599003740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e490a0 0x559999233d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.750+0000 7f5599003740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e51280 0x55999921e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.750+0000 7f5599003740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e71f60 0x55999922fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.750+0000 7f5599003740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5599992346a0 con 0x559998e71f60 2024-09-21T00:20:57.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.750+0000 7f5599003740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559999234d70 con 0x559998e51280 2024-09-21T00:20:57.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.750+0000 7f5599003740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559999235440 con 0x559998e490a0 2024-09-21T00:20:57.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.750+0000 7f558be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e71f60 0x55999922fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.750+0000 7f558b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e490a0 0x559999233d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.750+0000 7f558b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e490a0 0x559999233d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55938/0 (socket says 172.21.15.27:55938) 2024-09-21T00:20:57.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.750+0000 7f558b400640 1 -- 172.21.15.27:0/147095233 learned_addr learned my addr 172.21.15.27:0/147095233 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:57.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.750+0000 7f5582a00640 1 --2- 172.21.15.27:0/147095233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e51280 0x55999921e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.751+0000 7f558be00640 1 -- 172.21.15.27:0/147095233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e490a0 msgr2=0x559999233d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:57.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.751+0000 7f558be00640 1 --2- 172.21.15.27:0/147095233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e490a0 0x559999233d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.751+0000 7f558be00640 1 -- 172.21.15.27:0/147095233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e51280 msgr2=0x55999921e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:57.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.751+0000 7f558be00640 1 --2- 172.21.15.27:0/147095233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e51280 0x55999921e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.751+0000 7f558be00640 1 -- 172.21.15.27:0/147095233 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559999225110 con 0x559998e71f60 2024-09-21T00:20:57.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.751+0000 7f558b400640 1 --2- 172.21.15.27:0/147095233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e490a0 0x559999233d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:57.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.751+0000 7f558be00640 1 --2- 172.21.15.27:0/147095233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e71f60 0x55999922fcf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5584006010 tx=0x7f5584000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0a31190812599d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:57.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.751+0000 7f557b400640 1 -- 172.21.15.27:0/147095233 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558404a040 con 0x559998e71f60 2024-09-21T00:20:57.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.751+0000 7f557b400640 1 -- 172.21.15.27:0/147095233 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5584036040 con 0x559998e71f60 2024-09-21T00:20:57.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.752+0000 7f557b400640 1 -- 172.21.15.27:0/147095233 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584004050 con 0x559998e71f60 2024-09-21T00:20:57.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.752+0000 7f5599003740 1 -- 172.21.15.27:0/147095233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e71f60 msgr2=0x55999922fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:57.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.752+0000 7f5599003740 1 --2- 172.21.15.27:0/147095233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e71f60 0x55999922fcf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5584006010 tx=0x7f5584000f40 comp rx=0 tx=0).stop 2024-09-21T00:20:57.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.752+0000 7f5599003740 1 -- 172.21.15.27:0/147095233 shutdown_connections 2024-09-21T00:20:57.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.752+0000 7f5599003740 1 --2- 172.21.15.27:0/147095233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e71f60 0x55999922fcf0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.752+0000 7f5599003740 1 --2- 172.21.15.27:0/147095233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e51280 0x55999921e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.752+0000 7f5599003740 1 --2- 172.21.15.27:0/147095233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e490a0 0x559999233d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.752+0000 7f5599003740 1 -- 172.21.15.27:0/147095233 >> 172.21.15.27:0/147095233 conn(0x559998e66350 msgr2=0x5599992226d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:57.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.752+0000 7f5599003740 1 -- 172.21.15.27:0/147095233 shutdown_connections 2024-09-21T00:20:57.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.752+0000 7f5599003740 1 -- 172.21.15.27:0/147095233 wait complete. 2024-09-21T00:20:57.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.753+0000 7f5599003740 1 -- start start 2024-09-21T00:20:57.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.753+0000 7f5599003740 1 Processor -- start 2024-09-21T00:20:57.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f5599003740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e490a0 0x55999923ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f5599003740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e51280 0x559999238080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f5599003740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e71f60 0x5599992385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f5599003740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5599992361e0 con 0x559998e71f60 2024-09-21T00:20:57.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f5599003740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559999235b10 con 0x559998e490a0 2024-09-21T00:20:57.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f5599003740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5599992368b0 con 0x559998e51280 2024-09-21T00:20:57.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f5599003740 10 client.0 _ll_register_callbacks cb 0x7f557c00f910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:20:57.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f558b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e490a0 0x55999923ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f558be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e71f60 0x5599992385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f558b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e490a0 0x55999923ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58842/0 (socket says 172.21.15.27:58842) 2024-09-21T00:20:57.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f558b400640 1 -- 172.21.15.27:0/2413493716 learned_addr learned my addr 172.21.15.27:0/2413493716 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:57.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f5582a00640 1 --2- 172.21.15.27:0/2413493716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e51280 0x559999238080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f558b400640 1 -- 172.21.15.27:0/2413493716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e51280 msgr2=0x559999238080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:57.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f558b400640 1 --2- 172.21.15.27:0/2413493716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e51280 0x559999238080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f558b400640 1 -- 172.21.15.27:0/2413493716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e71f60 msgr2=0x5599992385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:57.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f558b400640 1 --2- 172.21.15.27:0/2413493716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e71f60 0x5599992385c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.754+0000 7f558b400640 1 -- 172.21.15.27:0/2413493716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559999238b30 con 0x559998e490a0 2024-09-21T00:20:57.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.755+0000 7f558b400640 1 --2- 172.21.15.27:0/2413493716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e490a0 0x55999923ebd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f557c01dc30 tx=0x7f557c01dee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcd6eed8d25d88be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:57.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.755+0000 7f5578c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:20:57.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.755+0000 7f5578c00640 1 -- 172.21.15.27:0/2413493716 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c03b070 con 0x559998e490a0 2024-09-21T00:20:57.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.755+0000 7f5599003740 1 -- 172.21.15.27:0/2413493716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5584036040 con 0x559998e490a0 2024-09-21T00:20:57.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.755+0000 7f5578c00640 1 -- 172.21.15.27:0/2413493716 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f557c005020 con 0x559998e490a0 2024-09-21T00:20:57.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.755+0000 7f5578c00640 1 -- 172.21.15.27:0/2413493716 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f557c044050 con 0x559998e490a0 2024-09-21T00:20:57.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.755+0000 7f5599003740 10 client.25217 fetch_fsmap learned FSMap version 25 2024-09-21T00:20:57.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.755+0000 7f5599003740 1 -- 172.21.15.27:0/2413493716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55999923e5b0 con 0x559998e490a0 2024-09-21T00:20:57.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.756+0000 7f5578c00640 1 -- 172.21.15.27:0/2413493716 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5584036040 con 0x559998e490a0 2024-09-21T00:20:57.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.756+0000 7f5578c00640 1 -- 172.21.15.27:0/2413493716 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f557c03e020 con 0x559998e490a0 2024-09-21T00:20:57.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.756+0000 7f5599003740 10 client.25217 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T00:20:57.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.756+0000 7f5599003740 10 client.25217 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T00:20:57.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.756+0000 7f5599003740 20 client.25217 populate_metadata read hostname 'smithi027' 2024-09-21T00:20:57.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.756+0000 7f5599003740 1 --2- 172.21.15.27:0/2413493716 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x559998e05c90 0x5599992a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:57.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.756+0000 7f5599003740 4 client.25217 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:20:57.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.756+0000 7f5599003740 1 -- 172.21.15.27:0/2413493716 --> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] -- command(tid 0: {"prefix": "status"}) -- 0x5599992344f0 con 0x559998e05c90 2024-09-21T00:20:57.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.757+0000 7f5582a00640 1 --2- 172.21.15.27:0/2413493716 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x559998e05c90 0x5599992a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:57.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.757+0000 7f5582a00640 1 --2- 172.21.15.27:0/2413493716 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x559998e05c90 0x5599992a4f00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a6269b59c8c2455 server_cookie=ea70ae3a6dbe0f06 in_seq=0 out_seq=0 2024-09-21T00:20:57.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.757+0000 7f5578c00640 10 client.25217 ms_handle_connect on v2:172.21.15.98:6838/3074699335 2024-09-21T00:20:57.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.758+0000 7f5578c00640 1 -- 172.21.15.27:0/2413493716 <== mds.2 v2:172.21.15.98:6838/3074699335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5599992344f0 con 0x559998e05c90 2024-09-21T00:20:57.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.758+0000 7f5578c00640 10 client.25217 handle_command_reply: tid=0 2024-09-21T00:20:57.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.758+0000 7f5599003740 1 client.25217 shutdown 2024-09-21T00:20:57.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.758+0000 7f5599003740 1 -- 172.21.15.27:0/2413493716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e490a0 msgr2=0x55999923ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:57.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.758+0000 7f5599003740 1 --2- 172.21.15.27:0/2413493716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e490a0 0x55999923ebd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f557c01dc30 tx=0x7f557c01dee0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.758+0000 7f5599003740 1 -- 172.21.15.27:0/2413493716 shutdown_connections 2024-09-21T00:20:57.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.758+0000 7f5599003740 1 --2- 172.21.15.27:0/2413493716 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x559998e05c90 0x5599992a4f00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.758+0000 7f5599003740 1 --2- 172.21.15.27:0/2413493716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559998e71f60 0x5599992385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.758+0000 7f5599003740 1 --2- 172.21.15.27:0/2413493716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559998e51280 0x559999238080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.758+0000 7f5599003740 1 --2- 172.21.15.27:0/2413493716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559998e490a0 0x55999923ebd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.758+0000 7f5578c00640 0 client.25217 ms_handle_reset on v2:172.21.15.98:6838/3074699335 2024-09-21T00:20:57.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.758+0000 7f5599003740 1 -- 172.21.15.27:0/2413493716 >> 172.21.15.27:0/2413493716 conn(0x559998e66350 msgr2=0x559999224030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:57.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.758+0000 7f5599003740 1 -- 172.21.15.27:0/2413493716 shutdown_connections 2024-09-21T00:20:57.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.758+0000 7f5599003740 1 -- 172.21.15.27:0/2413493716 wait complete. 2024-09-21T00:20:57.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.758+0000 7f5599003740 20 client.25217 trim_cache size 0 max 16384 2024-09-21T00:20:57.762 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:20:57.762 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "9c21de84-77ad-11ef-baf5-efdc52797490", 2024-09-21T00:20:57.762 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-09-21T00:20:57.763 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24727, 2024-09-21T00:20:57.763 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T00:20:57.763 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T00:20:57.763 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T00:20:57.763 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 83.865721061000002, 2024-09-21T00:20:57.763 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 25, 2024-09-21T00:20:57.763 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T00:20:57.763 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T00:20:57.763 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 161.12950552199999 2024-09-21T00:20:57.763 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:20:57.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.759+0000 7f5592200640 1 -- 172.21.15.27:0/2666570833 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f55700b65e0 msgr2=0x7f55700b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:57.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.759+0000 7f5592200640 1 --2- 172.21.15.27:0/2666570833 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f55700b65e0 0x7f55700b8af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x559999217110 tx=0x7f557c005f90 comp rx=0 tx=0).stop 2024-09-21T00:20:57.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.759+0000 7f5592200640 1 -- 172.21.15.27:0/2666570833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f558c078290 msgr2=0x7f558c1a1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:57.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.759+0000 7f5592200640 1 --2- 172.21.15.27:0/2666570833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f558c078290 0x7f558c1a1000 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f558c06caa0 tx=0x7f557400ee50 comp rx=0 tx=0).stop 2024-09-21T00:20:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.759+0000 7f5592200640 1 -- 172.21.15.27:0/2666570833 shutdown_connections 2024-09-21T00:20:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.759+0000 7f5592200640 1 --2- 172.21.15.27:0/2666570833 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f55700b65e0 0x7f55700b8af0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.759+0000 7f5592200640 1 --2- 172.21.15.27:0/2666570833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f558c114740 0x7f558c1a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.759+0000 7f5592200640 1 --2- 172.21.15.27:0/2666570833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f558c078290 0x7f558c1a1000 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.759+0000 7f5592200640 1 --2- 172.21.15.27:0/2666570833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c077940 0x7f558c1a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.759+0000 7f5592200640 1 -- 172.21.15.27:0/2666570833 >> 172.21.15.27:0/2666570833 conn(0x7f558c101810 msgr2=0x7f558c116720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.759+0000 7f5592200640 1 -- 172.21.15.27:0/2666570833 shutdown_connections 2024-09-21T00:20:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:57.759+0000 7f5592200640 1 -- 172.21.15.27:0/2666570833 wait complete. 2024-09-21T00:20:58.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:58 smithi089 ceph-mon[34481]: mgrmap e22: x(active, since 9m), standbys: z, y 2024-09-21T00:20:58.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:58 smithi098 ceph-mon[34725]: mgrmap e22: x(active, since 9m), standbys: z, y 2024-09-21T00:20:58.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:58 smithi027 ceph-mon[30745]: mgrmap e22: x(active, since 9m), standbys: z, y 2024-09-21T00:20:58.665 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell mds.a status 2024-09-21T00:20:59.049 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:20:59.371 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:20:59 smithi027 ceph-mon[30745]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:20:59.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:20:59 smithi089 ceph-mon[34481]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:20:59.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:20:59 smithi098 ceph-mon[34725]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:20:59.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.710+0000 7efd48c00640 1 -- 172.21.15.27:0/3961702827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd441094a0 msgr2=0x7efd441098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:59.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.710+0000 7efd48c00640 1 --2- 172.21.15.27:0/3961702827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd441094a0 0x7efd441098b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009900 tx=0x7efd2c02a410 comp rx=0 tx=0).stop 2024-09-21T00:20:59.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.710+0000 7efd48c00640 1 -- 172.21.15.27:0/3961702827 shutdown_connections 2024-09-21T00:20:59.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.710+0000 7efd48c00640 1 --2- 172.21.15.27:0/3961702827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd44109df0 0x7efd44110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.710+0000 7efd48c00640 1 --2- 172.21.15.27:0/3961702827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd441094a0 0x7efd441098b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.710+0000 7efd48c00640 1 --2- 172.21.15.27:0/3961702827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd44103b60 0x7efd44103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.710+0000 7efd48c00640 1 -- 172.21.15.27:0/3961702827 >> 172.21.15.27:0/3961702827 conn(0x7efd4407b850 msgr2=0x7efd4407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:59.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.710+0000 7efd48c00640 1 -- 172.21.15.27:0/3961702827 shutdown_connections 2024-09-21T00:20:59.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.710+0000 7efd48c00640 1 -- 172.21.15.27:0/3961702827 wait complete. 2024-09-21T00:20:59.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.711+0000 7efd48c00640 1 Processor -- start 2024-09-21T00:20:59.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.712+0000 7efd48c00640 1 -- start start 2024-09-21T00:20:59.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.712+0000 7efd48c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd44103b60 0x7efd4419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.712+0000 7efd48c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd441094a0 0x7efd4419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.712+0000 7efd48c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd44109df0 0x7efd4419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.712+0000 7efd48c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efd44113b00 con 0x7efd44109df0 2024-09-21T00:20:59.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.712+0000 7efd48c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efd44113980 con 0x7efd44103b60 2024-09-21T00:20:59.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.712+0000 7efd48c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efd44113c80 con 0x7efd441094a0 2024-09-21T00:20:59.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.713+0000 7efd41e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd44103b60 0x7efd4419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.713+0000 7efd41e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd44103b60 0x7efd4419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58862/0 (socket says 172.21.15.27:58862) 2024-09-21T00:20:59.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.713+0000 7efd41e00640 1 -- 172.21.15.27:0/983192054 learned_addr learned my addr 172.21.15.27:0/983192054 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:59.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.713+0000 7efd3be00640 1 --2- 172.21.15.27:0/983192054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd441094a0 0x7efd4419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.713+0000 7efd42800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd44109df0 0x7efd4419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.713+0000 7efd41e00640 1 -- 172.21.15.27:0/983192054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd441094a0 msgr2=0x7efd4419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:59.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.713+0000 7efd41e00640 1 --2- 172.21.15.27:0/983192054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd441094a0 0x7efd4419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.713+0000 7efd41e00640 1 -- 172.21.15.27:0/983192054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd44109df0 msgr2=0x7efd4419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:59.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.713+0000 7efd41e00640 1 --2- 172.21.15.27:0/983192054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd44109df0 0x7efd4419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.713+0000 7efd41e00640 1 -- 172.21.15.27:0/983192054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd441a0440 con 0x7efd44103b60 2024-09-21T00:20:59.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.714+0000 7efd41e00640 1 --2- 172.21.15.27:0/983192054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd44103b60 0x7efd4419a4d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efd3400e9f0 tx=0x7efd3400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:59.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.714+0000 7efd3aa00640 1 -- 172.21.15.27:0/983192054 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3400cd80 con 0x7efd44103b60 2024-09-21T00:20:59.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.714+0000 7efd3aa00640 1 -- 172.21.15.27:0/983192054 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd340108e0 con 0x7efd44103b60 2024-09-21T00:20:59.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.714+0000 7efd48c00640 1 -- 172.21.15.27:0/983192054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd441a0730 con 0x7efd44103b60 2024-09-21T00:20:59.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.714+0000 7efd3aa00640 1 -- 172.21.15.27:0/983192054 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34018980 con 0x7efd44103b60 2024-09-21T00:20:59.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.714+0000 7efd48c00640 1 -- 172.21.15.27:0/983192054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd441a6e90 con 0x7efd44103b60 2024-09-21T00:20:59.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.717+0000 7efd4f9fd740 1 Processor -- start 2024-09-21T00:20:59.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.717+0000 7efd4f9fd740 1 -- start start 2024-09-21T00:20:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.718+0000 7efd4f9fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a995af60 0x5565a9d01490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.718+0000 7efd4f9fd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 0x5565a9d019d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.718+0000 7efd4f9fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a993a280 0x5565a9d01f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.718+0000 7efd4f9fd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5565a9c78d50 con 0x5565a99320a0 2024-09-21T00:20:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.718+0000 7efd4f9fd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5565a9c68180 con 0x5565a995af60 2024-09-21T00:20:59.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.718+0000 7efd4f9fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5565a9c6bf90 con 0x5565a993a280 2024-09-21T00:20:59.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.718+0000 7efd3aa00640 1 -- 172.21.15.27:0/983192054 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7efd34010430 con 0x7efd44103b60 2024-09-21T00:20:59.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.718+0000 7efd42800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a995af60 0x5565a9d01490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.718+0000 7efd42800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a995af60 0x5565a9d01490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58866/0 (socket says 172.21.15.27:58866) 2024-09-21T00:20:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.718+0000 7efd42800640 1 -- 172.21.15.27:0/286217226 learned_addr learned my addr 172.21.15.27:0/286217226 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.719+0000 7efd41e00640 1 --2- 172.21.15.27:0/286217226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 0x5565a9d019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.719+0000 7efd3be00640 1 --2- 172.21.15.27:0/286217226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a993a280 0x5565a9d01f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.719+0000 7efd42800640 1 -- 172.21.15.27:0/286217226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a993a280 msgr2=0x5565a9d01f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.719+0000 7efd42800640 1 --2- 172.21.15.27:0/286217226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a993a280 0x5565a9d01f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.719+0000 7efd42800640 1 -- 172.21.15.27:0/286217226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 msgr2=0x5565a9d019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.719+0000 7efd42800640 1 --2- 172.21.15.27:0/286217226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 0x5565a9d019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.719+0000 7efd42800640 1 -- 172.21.15.27:0/286217226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a9c32a70 con 0x5565a995af60 2024-09-21T00:20:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.719+0000 7efd42800640 1 --2- 172.21.15.27:0/286217226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a995af60 0x5565a9d01490 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009fd0 tx=0x7efd3c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=220bfea57ac6f290 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.720+0000 7efd33000640 1 -- 172.21.15.27:0/286217226 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c004d20 con 0x5565a995af60 2024-09-21T00:20:59.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.720+0000 7efd33000640 1 -- 172.21.15.27:0/286217226 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd3c004050 con 0x5565a995af60 2024-09-21T00:20:59.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.720+0000 7efd33000640 1 -- 172.21.15.27:0/286217226 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c005220 con 0x5565a995af60 2024-09-21T00:20:59.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.720+0000 7efd3aa00640 1 --2- 172.21.15.27:0/983192054 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7efd280b6560 0x7efd280b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.720+0000 7efd4f9fd740 1 -- 172.21.15.27:0/286217226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a995af60 msgr2=0x5565a9d01490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.720+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/286217226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a995af60 0x5565a9d01490 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009fd0 tx=0x7efd3c0118d0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.721+0000 7efd4f9fd740 1 -- 172.21.15.27:0/286217226 shutdown_connections 2024-09-21T00:20:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.721+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/286217226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a993a280 0x5565a9d01f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.721+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/286217226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 0x5565a9d019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.721+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/286217226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a995af60 0x5565a9d01490 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.721+0000 7efd4f9fd740 1 -- 172.21.15.27:0/286217226 >> 172.21.15.27:0/286217226 conn(0x5565a994f350 msgr2=0x5565a9cff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.721+0000 7efd3aa00640 1 -- 172.21.15.27:0/983192054 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7efd340e7ba0 con 0x7efd44103b60 2024-09-21T00:20:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.721+0000 7efd4f9fd740 1 -- 172.21.15.27:0/286217226 shutdown_connections 2024-09-21T00:20:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.721+0000 7efd41e00640 1 --2- 172.21.15.27:0/983192054 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7efd280b6560 0x7efd280b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.721+0000 7efd4f9fd740 1 -- 172.21.15.27:0/286217226 wait complete. 2024-09-21T00:20:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.722+0000 7efd4f9fd740 1 -- start start 2024-09-21T00:20:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.722+0000 7efd41e00640 1 --2- 172.21.15.27:0/983192054 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7efd280b6560 0x7efd280b8a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd340a9ea0 tx=0x7efd34018420 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.722+0000 7efd4f9fd740 1 Processor -- start 2024-09-21T00:20:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.723+0000 7efd4f9fd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 0x5565a9d0d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.723+0000 7efd4f9fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d0d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.723+0000 7efd4f9fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a995af60 0x5565a9d1d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.723+0000 7efd4f9fd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5565a9ac95f0 con 0x5565a99320a0 2024-09-21T00:20:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.723+0000 7efd4f9fd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5565a9c03460 con 0x5565a993a280 2024-09-21T00:20:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.723+0000 7efd4f9fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5565a9d0fc70 con 0x5565a995af60 2024-09-21T00:20:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.723+0000 7efd4f9fd740 10 client.0 _ll_register_callbacks cb 0x5565a9c13210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:20:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.723+0000 7efd3be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d0d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.723+0000 7efd3be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d0d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58882/0 (socket says 172.21.15.27:58882) 2024-09-21T00:20:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.723+0000 7efd3be00640 1 -- 172.21.15.27:0/3251061158 learned_addr learned my addr 172.21.15.27:0/3251061158 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.723+0000 7efd42800640 1 --2- 172.21.15.27:0/3251061158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 0x5565a9d0d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.723+0000 7efd42800640 1 --2- 172.21.15.27:0/3251061158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a995af60 0x5565a9d1d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.724+0000 7efd3be00640 1 -- 172.21.15.27:0/3251061158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a995af60 msgr2=0x5565a9d1d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:59.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.724+0000 7efd3be00640 1 --2- 172.21.15.27:0/3251061158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a995af60 0x5565a9d1d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.724+0000 7efd3be00640 1 -- 172.21.15.27:0/3251061158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 msgr2=0x5565a9d0d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:59.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.724+0000 7efd3be00640 1 --2- 172.21.15.27:0/3251061158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 0x5565a9d0d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.724+0000 7efd3be00640 1 -- 172.21.15.27:0/3251061158 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a99c6540 con 0x5565a993a280 2024-09-21T00:20:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.724+0000 7efd3be00640 1 --2- 172.21.15.27:0/3251061158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d0d590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efd2c005ab0 tx=0x7efd2c0042c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d605a9e7c3984116 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.724+0000 7efd27400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:20:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.724+0000 7efd27400640 1 -- 172.21.15.27:0/3251061158 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c035040 con 0x5565a993a280 2024-09-21T00:20:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.724+0000 7efd27400640 1 -- 172.21.15.27:0/3251061158 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7efd2c005450 con 0x5565a993a280 2024-09-21T00:20:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.724+0000 7efd4f9fd740 1 -- 172.21.15.27:0/3251061158 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5565a9c05a70 con 0x5565a993a280 2024-09-21T00:20:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.725+0000 7efd27400640 1 -- 172.21.15.27:0/3251061158 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efd2c04e050 con 0x5565a993a280 2024-09-21T00:20:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.725+0000 7efd4f9fd740 10 client.25229 fetch_fsmap learned FSMap version 25 2024-09-21T00:20:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.725+0000 7efd4f9fd740 1 -- 172.21.15.27:0/3251061158 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565a9c206f0 con 0x5565a993a280 2024-09-21T00:20:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.725+0000 7efd27400640 1 -- 172.21.15.27:0/3251061158 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5565a9c05a70 con 0x5565a993a280 2024-09-21T00:20:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.725+0000 7efd27400640 1 -- 172.21.15.27:0/3251061158 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efd2c003780 con 0x5565a993a280 2024-09-21T00:20:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.725+0000 7efd4f9fd740 10 client.25229 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T00:20:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.726+0000 7efd4f9fd740 10 client.25229 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:20:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.726+0000 7efd4f9fd740 20 client.25229 populate_metadata read hostname 'smithi027' 2024-09-21T00:20:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.726+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/3251061158 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5565a99198e0 0x5565a9d23890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.726+0000 7efd4f9fd740 4 client.25229 mds_command: new command op to 24751 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:20:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.726+0000 7efd4f9fd740 1 -- 172.21.15.27:0/3251061158 --> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565a9c7aca0 con 0x5565a99198e0 2024-09-21T00:20:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.726+0000 7efd42800640 1 --2- 172.21.15.27:0/3251061158 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5565a99198e0 0x5565a9d23890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.727+0000 7efd42800640 1 --2- 172.21.15.27:0/3251061158 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5565a99198e0 0x5565a9d23890 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5942d837022cdab server_cookie=f0f50a028af7f585 in_seq=0 out_seq=0 2024-09-21T00:20:59.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.727+0000 7efd27400640 10 client.25229 ms_handle_connect on v2:172.21.15.27:6840/378707351 2024-09-21T00:20:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.728+0000 7efd27400640 1 -- 172.21.15.27:0/3251061158 <== mds.1 v2:172.21.15.27:6840/378707351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5565a9c7aca0 con 0x5565a99198e0 2024-09-21T00:20:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.728+0000 7efd27400640 10 client.25229 handle_command_reply: tid=0 2024-09-21T00:20:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.728+0000 7efd4f9fd740 1 client.25229 shutdown 2024-09-21T00:20:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.728+0000 7efd4f9fd740 1 -- 172.21.15.27:0/3251061158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 msgr2=0x5565a9d0d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.728+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/3251061158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d0d590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efd2c005ab0 tx=0x7efd2c0042c0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.728+0000 7efd4f9fd740 1 -- 172.21.15.27:0/3251061158 shutdown_connections 2024-09-21T00:20:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.728+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/3251061158 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5565a99198e0 0x5565a9d23890 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.728+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/3251061158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a995af60 0x5565a9d1d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.728+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/3251061158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d0d590 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.728+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/3251061158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 0x5565a9d0d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.728+0000 7efd4f9fd740 1 -- 172.21.15.27:0/3251061158 >> 172.21.15.27:0/3251061158 conn(0x5565a994f350 msgr2=0x5565a9d04620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.728+0000 7efd27400640 0 client.25229 ms_handle_reset on v2:172.21.15.27:6840/378707351 2024-09-21T00:20:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.728+0000 7efd4f9fd740 1 -- 172.21.15.27:0/3251061158 shutdown_connections 2024-09-21T00:20:59.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.729+0000 7efd4f9fd740 1 -- 172.21.15.27:0/3251061158 wait complete. 2024-09-21T00:20:59.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.729+0000 7efd4f9fd740 20 client.25229 trim_cache size 0 max 16384 2024-09-21T00:20:59.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.748+0000 7efd4f9fd740 1 Processor -- start 2024-09-21T00:20:59.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.748+0000 7efd4f9fd740 1 -- start start 2024-09-21T00:20:59.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.748+0000 7efd4f9fd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 0x5565a9d23890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.748+0000 7efd4f9fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d212e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.748+0000 7efd4f9fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a995af60 0x5565a9d07f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.748+0000 7efd4f9fd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5565a9d21bd0 con 0x5565a99320a0 2024-09-21T00:20:59.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.748+0000 7efd4f9fd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5565a9d222a0 con 0x5565a993a280 2024-09-21T00:20:59.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.748+0000 7efd4f9fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5565a9d22970 con 0x5565a995af60 2024-09-21T00:20:59.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.749+0000 7efd42800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 0x5565a9d23890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.749+0000 7efd42800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 0x5565a9d23890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58186/0 (socket says 172.21.15.27:58186) 2024-09-21T00:20:59.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.749+0000 7efd42800640 1 -- 172.21.15.27:0/1501271375 learned_addr learned my addr 172.21.15.27:0/1501271375 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:59.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.749+0000 7efd3be00640 1 --2- 172.21.15.27:0/1501271375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d212e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.749+0000 7efd42800640 1 --2- 172.21.15.27:0/1501271375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a995af60 0x5565a9d07f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.749+0000 7efd3be00640 1 -- 172.21.15.27:0/1501271375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a995af60 msgr2=0x5565a9d07f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:59.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.749+0000 7efd3be00640 1 --2- 172.21.15.27:0/1501271375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a995af60 0x5565a9d07f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.749+0000 7efd3be00640 1 -- 172.21.15.27:0/1501271375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 msgr2=0x5565a9d23890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:59.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.749+0000 7efd3be00640 1 --2- 172.21.15.27:0/1501271375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 0x5565a9d23890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.749+0000 7efd3be00640 1 -- 172.21.15.27:0/1501271375 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a9d0e110 con 0x5565a993a280 2024-09-21T00:20:59.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.749+0000 7efd42800640 1 --2- 172.21.15.27:0/1501271375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 0x5565a9d23890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:20:59.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.750+0000 7efd3be00640 1 --2- 172.21.15.27:0/1501271375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d212e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efd2c003cf0 tx=0x7efd2c001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bce3b3239e80035a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:59.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.750+0000 7efd32600640 1 -- 172.21.15.27:0/1501271375 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c035030 con 0x5565a993a280 2024-09-21T00:20:59.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.750+0000 7efd32600640 1 -- 172.21.15.27:0/1501271375 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd2c040070 con 0x5565a993a280 2024-09-21T00:20:59.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.750+0000 7efd32600640 1 -- 172.21.15.27:0/1501271375 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c004750 con 0x5565a993a280 2024-09-21T00:20:59.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.751+0000 7efd4f9fd740 1 -- 172.21.15.27:0/1501271375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 msgr2=0x5565a9d212e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:59.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.751+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/1501271375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d212e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efd2c003cf0 tx=0x7efd2c001e70 comp rx=0 tx=0).stop 2024-09-21T00:20:59.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.751+0000 7efd4f9fd740 1 -- 172.21.15.27:0/1501271375 shutdown_connections 2024-09-21T00:20:59.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.751+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/1501271375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a995af60 0x5565a9d07f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.751+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/1501271375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d212e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.751+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/1501271375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a99320a0 0x5565a9d23890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.751+0000 7efd4f9fd740 1 -- 172.21.15.27:0/1501271375 >> 172.21.15.27:0/1501271375 conn(0x5565a994f350 msgr2=0x5565a9cfa010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:59.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.751+0000 7efd4f9fd740 1 -- 172.21.15.27:0/1501271375 shutdown_connections 2024-09-21T00:20:59.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.751+0000 7efd4f9fd740 1 -- 172.21.15.27:0/1501271375 wait complete. 2024-09-21T00:20:59.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.752+0000 7efd4f9fd740 1 -- start start 2024-09-21T00:20:59.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.752+0000 7efd4f9fd740 1 Processor -- start 2024-09-21T00:20:59.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.752+0000 7efd4f9fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a99320a0 0x5565a9d25af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.752+0000 7efd4f9fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d26030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.752+0000 7efd4f9fd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a995af60 0x5565a9d26570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.752+0000 7efd4f9fd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5565a9d23710 con 0x5565a995af60 2024-09-21T00:20:59.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.752+0000 7efd4f9fd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5565a9d23040 con 0x5565a993a280 2024-09-21T00:20:59.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.752+0000 7efd4f9fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5565a9d23de0 con 0x5565a99320a0 2024-09-21T00:20:59.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.753+0000 7efd4f9fd740 10 client.0 _ll_register_callbacks cb 0x7efd2c005eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:20:59.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.753+0000 7efd3be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d26030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.753+0000 7efd3be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d26030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58886/0 (socket says 172.21.15.27:58886) 2024-09-21T00:20:59.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.753+0000 7efd3be00640 1 -- 172.21.15.27:0/935112748 learned_addr learned my addr 172.21.15.27:0/935112748 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:20:59.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.753+0000 7efd42800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a995af60 0x5565a9d26570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.753+0000 7efd42800640 1 --2- 172.21.15.27:0/935112748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a99320a0 0x5565a9d25af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.753+0000 7efd3be00640 1 -- 172.21.15.27:0/935112748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a99320a0 msgr2=0x5565a9d25af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:59.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.753+0000 7efd3be00640 1 --2- 172.21.15.27:0/935112748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a99320a0 0x5565a9d25af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.753+0000 7efd3be00640 1 -- 172.21.15.27:0/935112748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a995af60 msgr2=0x5565a9d26570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:20:59.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.754+0000 7efd3be00640 1 --2- 172.21.15.27:0/935112748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a995af60 0x5565a9d26570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.754+0000 7efd3be00640 1 -- 172.21.15.27:0/935112748 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a9d2b280 con 0x5565a993a280 2024-09-21T00:20:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.754+0000 7efd3be00640 1 --2- 172.21.15.27:0/935112748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d26030 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efd2c000c00 tx=0x7efd2c0097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e3b2428809b9b0e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:20:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.754+0000 7efd27400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:20:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.754+0000 7efd27400640 1 -- 172.21.15.27:0/935112748 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c035030 con 0x5565a993a280 2024-09-21T00:20:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.754+0000 7efd4f9fd740 1 -- 172.21.15.27:0/935112748 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd2c040070 con 0x5565a993a280 2024-09-21T00:20:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.754+0000 7efd27400640 1 -- 172.21.15.27:0/935112748 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7efd2c005250 con 0x5565a993a280 2024-09-21T00:20:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.755+0000 7efd27400640 1 -- 172.21.15.27:0/935112748 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efd2c0042c0 con 0x5565a993a280 2024-09-21T00:20:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.755+0000 7efd4f9fd740 10 client.25235 fetch_fsmap learned FSMap version 25 2024-09-21T00:20:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.755+0000 7efd4f9fd740 1 -- 172.21.15.27:0/935112748 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565a9d8efb0 con 0x5565a993a280 2024-09-21T00:20:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.755+0000 7efd27400640 1 -- 172.21.15.27:0/935112748 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd2c040070 con 0x5565a993a280 2024-09-21T00:20:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.755+0000 7efd27400640 1 -- 172.21.15.27:0/935112748 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efd2c0315c0 con 0x5565a993a280 2024-09-21T00:20:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.755+0000 7efd4f9fd740 10 client.25235 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T00:20:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.755+0000 7efd4f9fd740 10 client.25235 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:20:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.755+0000 7efd4f9fd740 20 client.25235 populate_metadata read hostname 'smithi027' 2024-09-21T00:20:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.755+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/935112748 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5565a98eec90 0x5565a9d92480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:20:59.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.756+0000 7efd4f9fd740 4 client.25235 mds_command: new command op to 24751 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:20:59.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.756+0000 7efd4f9fd740 1 -- 172.21.15.27:0/935112748 --> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] -- command(tid 0: {"prefix": "status"}) -- 0x5565a9c76750 con 0x5565a98eec90 2024-09-21T00:20:59.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.756+0000 7efd42800640 1 --2- 172.21.15.27:0/935112748 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5565a98eec90 0x5565a9d92480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:20:59.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.756+0000 7efd42800640 1 --2- 172.21.15.27:0/935112748 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5565a98eec90 0x5565a9d92480 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba9d4874ed8d9d6e server_cookie=843c6c0124be0e79 in_seq=0 out_seq=0 2024-09-21T00:20:59.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.756+0000 7efd27400640 10 client.25235 ms_handle_connect on v2:172.21.15.27:6840/378707351 2024-09-21T00:20:59.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.757+0000 7efd27400640 1 -- 172.21.15.27:0/935112748 <== mds.1 v2:172.21.15.27:6840/378707351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5565a9c76750 con 0x5565a98eec90 2024-09-21T00:20:59.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.757+0000 7efd27400640 10 client.25235 handle_command_reply: tid=0 2024-09-21T00:20:59.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.757+0000 7efd4f9fd740 1 client.25235 shutdown 2024-09-21T00:20:59.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.757+0000 7efd4f9fd740 1 -- 172.21.15.27:0/935112748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 msgr2=0x5565a9d26030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:59.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.757+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/935112748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d26030 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efd2c000c00 tx=0x7efd2c0097f0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.757+0000 7efd4f9fd740 1 -- 172.21.15.27:0/935112748 shutdown_connections 2024-09-21T00:20:59.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.757+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/935112748 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5565a98eec90 0x5565a9d92480 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.757+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/935112748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565a995af60 0x5565a9d26570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.757+0000 7efd27400640 0 client.25235 ms_handle_reset on v2:172.21.15.27:6840/378707351 2024-09-21T00:20:59.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.757+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/935112748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565a993a280 0x5565a9d26030 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.757+0000 7efd4f9fd740 1 --2- 172.21.15.27:0/935112748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565a99320a0 0x5565a9d25af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.757+0000 7efd4f9fd740 1 -- 172.21.15.27:0/935112748 >> 172.21.15.27:0/935112748 conn(0x5565a994f350 msgr2=0x5565a9d07f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:59.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.757+0000 7efd4f9fd740 1 -- 172.21.15.27:0/935112748 shutdown_connections 2024-09-21T00:20:59.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.757+0000 7efd4f9fd740 1 -- 172.21.15.27:0/935112748 wait complete. 2024-09-21T00:20:59.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.758+0000 7efd4f9fd740 20 client.25235 trim_cache size 0 max 16384 2024-09-21T00:20:59.761 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:20:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "9c21de84-77ad-11ef-baf5-efdc52797490", 2024-09-21T00:20:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-09-21T00:20:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24751, 2024-09-21T00:20:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T00:20:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T00:20:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T00:20:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 86.875334699000007, 2024-09-21T00:20:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 25, 2024-09-21T00:20:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T00:20:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T00:20:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 155.13525322800001 2024-09-21T00:20:59.762 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:20:59.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.758+0000 7efd48c00640 1 -- 172.21.15.27:0/983192054 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7efd280b6560 msgr2=0x7efd280b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:59.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.758+0000 7efd48c00640 1 --2- 172.21.15.27:0/983192054 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7efd280b6560 0x7efd280b8a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd340a9ea0 tx=0x7efd34018420 comp rx=0 tx=0).stop 2024-09-21T00:20:59.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.758+0000 7efd48c00640 1 -- 172.21.15.27:0/983192054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd44103b60 msgr2=0x7efd4419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:20:59.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.758+0000 7efd48c00640 1 --2- 172.21.15.27:0/983192054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd44103b60 0x7efd4419a4d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efd3400e9f0 tx=0x7efd3400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.758+0000 7efd48c00640 1 -- 172.21.15.27:0/983192054 shutdown_connections 2024-09-21T00:20:59.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.758+0000 7efd48c00640 1 --2- 172.21.15.27:0/983192054 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7efd280b6560 0x7efd280b8a70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.758+0000 7efd48c00640 1 --2- 172.21.15.27:0/983192054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd44109df0 0x7efd4419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.758+0000 7efd48c00640 1 --2- 172.21.15.27:0/983192054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd441094a0 0x7efd4419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.758+0000 7efd48c00640 1 --2- 172.21.15.27:0/983192054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd44103b60 0x7efd4419a4d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:20:59.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.758+0000 7efd48c00640 1 -- 172.21.15.27:0/983192054 >> 172.21.15.27:0/983192054 conn(0x7efd4407b850 msgr2=0x7efd441054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:20:59.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.759+0000 7efd48c00640 1 -- 172.21.15.27:0/983192054 shutdown_connections 2024-09-21T00:20:59.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:20:59.759+0000 7efd48c00640 1 -- 172.21.15.27:0/983192054 wait complete. 2024-09-21T00:21:00.595 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T00:21:00.984 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:01.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.643+0000 7f1990800640 1 -- 172.21.15.27:0/3528112125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1988105d80 msgr2=0x7f1988106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:01.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.643+0000 7f1990800640 1 --2- 172.21.15.27:0/3528112125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1988105d80 0x7f1988106190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f197c009900 tx=0x7f197c02a410 comp rx=0 tx=0).stop 2024-09-21T00:21:01.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.643+0000 7f1990800640 1 -- 172.21.15.27:0/3528112125 shutdown_connections 2024-09-21T00:21:01.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.643+0000 7f1990800640 1 --2- 172.21.15.27:0/3528112125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198810c010 0x7f1988112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.643+0000 7f1990800640 1 --2- 172.21.15.27:0/3528112125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f198810b6c0 0x7f198810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.643+0000 7f1990800640 1 --2- 172.21.15.27:0/3528112125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1988105d80 0x7f1988106190 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.643+0000 7f1990800640 1 -- 172.21.15.27:0/3528112125 >> 172.21.15.27:0/3528112125 conn(0x7f1988101810 msgr2=0x7f1988103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:01.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.643+0000 7f1990800640 1 -- 172.21.15.27:0/3528112125 shutdown_connections 2024-09-21T00:21:01.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.643+0000 7f1990800640 1 -- 172.21.15.27:0/3528112125 wait complete. 2024-09-21T00:21:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.644+0000 7f1990800640 1 Processor -- start 2024-09-21T00:21:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.645+0000 7f1990800640 1 -- start start 2024-09-21T00:21:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.645+0000 7f1990800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1988105d80 0x7f198819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.645+0000 7f1990800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f198810b6c0 0x7f198819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.645+0000 7f1990800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198810c010 0x7f19881a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.645+0000 7f1990800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1988115cd0 con 0x7f198810b6c0 2024-09-21T00:21:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.645+0000 7f1990800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1988115b50 con 0x7f1988105d80 2024-09-21T00:21:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.645+0000 7f1990800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1988115e50 con 0x7f198810c010 2024-09-21T00:21:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.645+0000 7f198d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f198810b6c0 0x7f198819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.645+0000 7f198de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1988105d80 0x7f198819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.645+0000 7f198de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1988105d80 0x7f198819c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58902/0 (socket says 172.21.15.27:58902) 2024-09-21T00:21:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.645+0000 7f198de00640 1 -- 172.21.15.27:0/3996774570 learned_addr learned my addr 172.21.15.27:0/3996774570 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.645+0000 7f198e800640 1 --2- 172.21.15.27:0/3996774570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198810c010 0x7f19881a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.646+0000 7f198e800640 1 -- 172.21.15.27:0/3996774570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1988105d80 msgr2=0x7f198819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.646+0000 7f198e800640 1 --2- 172.21.15.27:0/3996774570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1988105d80 0x7f198819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.646+0000 7f198e800640 1 -- 172.21.15.27:0/3996774570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f198810b6c0 msgr2=0x7f198819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.646+0000 7f198e800640 1 --2- 172.21.15.27:0/3996774570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f198810b6c0 0x7f198819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.646+0000 7f198e800640 1 -- 172.21.15.27:0/3996774570 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19881a2610 con 0x7f198810c010 2024-09-21T00:21:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.646+0000 7f198d400640 1 --2- 172.21.15.27:0/3996774570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f198810b6c0 0x7f198819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.646+0000 7f198e800640 1 --2- 172.21.15.27:0/3996774570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198810c010 0x7f19881a1ee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1984009fd0 tx=0x7f198400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.647+0000 7f1982a00640 1 -- 172.21.15.27:0/3996774570 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198400ce20 con 0x7f198810c010 2024-09-21T00:21:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.647+0000 7f1982a00640 1 -- 172.21.15.27:0/3996774570 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1984005020 con 0x7f198810c010 2024-09-21T00:21:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.647+0000 7f1990800640 1 -- 172.21.15.27:0/3996774570 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19881a2900 con 0x7f198810c010 2024-09-21T00:21:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.647+0000 7f1982a00640 1 -- 172.21.15.27:0/3996774570 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984010690 con 0x7f198810c010 2024-09-21T00:21:01.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.647+0000 7f1990800640 1 -- 172.21.15.27:0/3996774570 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19881a9060 con 0x7f198810c010 2024-09-21T00:21:01.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.650+0000 7f199744f740 1 Processor -- start 2024-09-21T00:21:01.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.650+0000 7f1982a00640 1 -- 172.21.15.27:0/3996774570 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f19840054d0 con 0x7f198810c010 2024-09-21T00:21:01.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.651+0000 7f199744f740 1 -- start start 2024-09-21T00:21:01.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.651+0000 7f199744f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 0x55f5650cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.651+0000 7f199744f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d000a0 0x55f5650cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.651+0000 7f199744f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 0x55f5650cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.651+0000 7f199744f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f565046d50 con 0x55f564d08280 2024-09-21T00:21:01.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.651+0000 7f199744f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f565036180 con 0x55f564d28f60 2024-09-21T00:21:01.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.651+0000 7f199744f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f565039f90 con 0x55f564d000a0 2024-09-21T00:21:01.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.651+0000 7f198d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 0x55f5650cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.651+0000 7f198de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d000a0 0x55f5650cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.651+0000 7f198d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 0x55f5650cff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58218/0 (socket says 172.21.15.27:58218) 2024-09-21T00:21:01.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.651+0000 7f198d400640 1 -- 172.21.15.27:0/1019838123 learned_addr learned my addr 172.21.15.27:0/1019838123 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:01.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.651+0000 7f198de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d000a0 0x55f5650cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56042/0 (socket says 172.21.15.27:56042) 2024-09-21T00:21:01.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.652+0000 7f198d400640 1 --2- 172.21.15.27:0/1019838123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 0x55f5650cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.652+0000 7f198d400640 1 -- 172.21.15.27:0/1019838123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d000a0 msgr2=0x55f5650cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.652+0000 7f198d400640 1 --2- 172.21.15.27:0/1019838123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d000a0 0x55f5650cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.652+0000 7f198d400640 1 -- 172.21.15.27:0/1019838123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 msgr2=0x55f5650cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.652+0000 7f198d400640 1 --2- 172.21.15.27:0/1019838123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 0x55f5650cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.652+0000 7f198d400640 1 -- 172.21.15.27:0/1019838123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f565000a70 con 0x55f564d08280 2024-09-21T00:21:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.652+0000 7f198d400640 1 --2- 172.21.15.27:0/1019838123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 0x55f5650cff10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1978011b60 tx=0x7f1978005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6104382cb3f8ae1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:01.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.652+0000 7f1977000640 1 -- 172.21.15.27:0/1019838123 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197800fc50 con 0x55f564d08280 2024-09-21T00:21:01.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.653+0000 7f1977000640 1 -- 172.21.15.27:0/1019838123 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f197800fdf0 con 0x55f564d08280 2024-09-21T00:21:01.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.653+0000 7f1977000640 1 -- 172.21.15.27:0/1019838123 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19780038e0 con 0x55f564d08280 2024-09-21T00:21:01.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.653+0000 7f1982a00640 1 --2- 172.21.15.27:0/3996774570 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f19700b6560 0x7f19700b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.653+0000 7f1982a00640 1 -- 172.21.15.27:0/3996774570 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f19840e7f60 con 0x7f198810c010 2024-09-21T00:21:01.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.653+0000 7f198de00640 1 --2- 172.21.15.27:0/3996774570 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f19700b6560 0x7f19700b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.653+0000 7f199744f740 1 -- 172.21.15.27:0/1019838123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 msgr2=0x55f5650cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:01.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.653+0000 7f199744f740 1 --2- 172.21.15.27:0/1019838123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 0x55f5650cff10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1978011b60 tx=0x7f1978005850 comp rx=0 tx=0).stop 2024-09-21T00:21:01.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.654+0000 7f199744f740 1 -- 172.21.15.27:0/1019838123 shutdown_connections 2024-09-21T00:21:01.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.654+0000 7f199744f740 1 --2- 172.21.15.27:0/1019838123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 0x55f5650cff10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.654+0000 7f199744f740 1 --2- 172.21.15.27:0/1019838123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d000a0 0x55f5650cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.654+0000 7f199744f740 1 --2- 172.21.15.27:0/1019838123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 0x55f5650cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.654+0000 7f199744f740 1 -- 172.21.15.27:0/1019838123 >> 172.21.15.27:0/1019838123 conn(0x55f564d1d350 msgr2=0x55f5650cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:01.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.654+0000 7f199744f740 1 -- 172.21.15.27:0/1019838123 shutdown_connections 2024-09-21T00:21:01.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.654+0000 7f199744f740 1 -- 172.21.15.27:0/1019838123 wait complete. 2024-09-21T00:21:01.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.654+0000 7f198de00640 1 --2- 172.21.15.27:0/3996774570 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f19700b6560 0x7f19700b8a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f197c031440 tx=0x7f197c0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.654+0000 7f199744f740 1 -- start start 2024-09-21T00:21:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.655+0000 7f199744f740 1 Processor -- start 2024-09-21T00:21:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.655+0000 7f199744f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d000a0 0x55f5650daf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.655+0000 7f199744f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 0x55f5650db4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.655+0000 7f199744f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d28f60 0x55f5650e6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.655+0000 7f199744f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f564e975f0 con 0x55f564d08280 2024-09-21T00:21:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.655+0000 7f199744f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f564fd1460 con 0x55f564d000a0 2024-09-21T00:21:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.655+0000 7f199744f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f5650ddc70 con 0x55f564d28f60 2024-09-21T00:21:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.655+0000 7f199744f740 10 client.0 _ll_register_callbacks cb 0x55f564fe1210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:21:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.656+0000 7f198d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 0x55f5650db4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.656+0000 7f198de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d000a0 0x55f5650daf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.656+0000 7f198e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d28f60 0x55f5650e6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.656+0000 7f198de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d000a0 0x55f5650daf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58912/0 (socket says 172.21.15.27:58912) 2024-09-21T00:21:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.656+0000 7f198d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 0x55f5650db4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58232/0 (socket says 172.21.15.27:58232) 2024-09-21T00:21:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.656+0000 7f198de00640 1 -- 172.21.15.27:0/1858764788 learned_addr learned my addr 172.21.15.27:0/1858764788 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.656+0000 7f198e800640 1 -- 172.21.15.27:0/1858764788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d000a0 msgr2=0x55f5650daf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.656+0000 7f198e800640 1 --2- 172.21.15.27:0/1858764788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d000a0 0x55f5650daf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.656+0000 7f198e800640 1 -- 172.21.15.27:0/1858764788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 msgr2=0x55f5650db4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.656+0000 7f198e800640 1 --2- 172.21.15.27:0/1858764788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 0x55f5650db4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.656+0000 7f198e800640 1 -- 172.21.15.27:0/1858764788 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f564d94540 con 0x55f564d28f60 2024-09-21T00:21:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.656+0000 7f198e800640 1 --2- 172.21.15.27:0/1858764788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d28f60 0x55f5650e6f60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f19840e5550 tx=0x7f19840a9410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a154ddc81008267d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.656+0000 7f196b400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:21:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.657+0000 7f196b400640 1 -- 172.21.15.27:0/1858764788 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19840ae2e0 con 0x55f564d28f60 2024-09-21T00:21:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.657+0000 7f196b400640 1 -- 172.21.15.27:0/1858764788 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f19840ace40 con 0x55f564d28f60 2024-09-21T00:21:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.657+0000 7f199744f740 1 -- 172.21.15.27:0/1858764788 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f564fd3a70 con 0x55f564d28f60 2024-09-21T00:21:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.657+0000 7f196b400640 1 -- 172.21.15.27:0/1858764788 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f19840b26c0 con 0x55f564d28f60 2024-09-21T00:21:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.657+0000 7f199744f740 10 client.25234 fetch_fsmap learned FSMap version 25 2024-09-21T00:21:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.657+0000 7f199744f740 1 -- 172.21.15.27:0/1858764788 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f564fee6f0 con 0x55f564d28f60 2024-09-21T00:21:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.657+0000 7f196b400640 1 -- 172.21.15.27:0/1858764788 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f564fd3a70 con 0x55f564d28f60 2024-09-21T00:21:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.658+0000 7f196b400640 1 -- 172.21.15.27:0/1858764788 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f19840ac8b0 con 0x55f564d28f60 2024-09-21T00:21:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.658+0000 7f199744f740 10 client.25234 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T00:21:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.658+0000 7f199744f740 10 client.25234 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T00:21:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.658+0000 7f199744f740 20 client.25234 populate_metadata read hostname 'smithi027' 2024-09-21T00:21:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.658+0000 7f199744f740 1 --2- 172.21.15.27:0/1858764788 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f564ce78e0 0x55f5650ed430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.658+0000 7f199744f740 4 client.25234 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:21:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.658+0000 7f199744f740 1 -- 172.21.15.27:0/1858764788 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f565048ca0 con 0x55f564ce78e0 2024-09-21T00:21:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.658+0000 7f198de00640 1 --2- 172.21.15.27:0/1858764788 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f564ce78e0 0x55f5650ed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.659+0000 7f198de00640 1 --2- 172.21.15.27:0/1858764788 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f564ce78e0 0x55f5650ed430 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7930ff0c1aaea206 server_cookie=acb8131d9f005d62 in_seq=0 out_seq=0 2024-09-21T00:21:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.659+0000 7f196b400640 10 client.25234 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:21:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.660+0000 7f196b400640 1 -- 172.21.15.27:0/1858764788 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f565048ca0 con 0x55f564ce78e0 2024-09-21T00:21:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.660+0000 7f196b400640 10 client.25234 handle_command_reply: tid=0 2024-09-21T00:21:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.660+0000 7f199744f740 1 client.25234 shutdown 2024-09-21T00:21:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.660+0000 7f199744f740 1 -- 172.21.15.27:0/1858764788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d28f60 msgr2=0x55f5650e6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.660+0000 7f199744f740 1 --2- 172.21.15.27:0/1858764788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d28f60 0x55f5650e6f60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f19840e5550 tx=0x7f19840a9410 comp rx=0 tx=0).stop 2024-09-21T00:21:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.660+0000 7f199744f740 1 -- 172.21.15.27:0/1858764788 shutdown_connections 2024-09-21T00:21:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.660+0000 7f199744f740 1 --2- 172.21.15.27:0/1858764788 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f564ce78e0 0x55f5650ed430 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.660+0000 7f199744f740 1 --2- 172.21.15.27:0/1858764788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d28f60 0x55f5650e6f60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.660+0000 7f199744f740 1 --2- 172.21.15.27:0/1858764788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 0x55f5650db4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.660+0000 7f199744f740 1 --2- 172.21.15.27:0/1858764788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d000a0 0x55f5650daf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.660+0000 7f196b400640 0 client.25234 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:21:01.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.660+0000 7f199744f740 1 -- 172.21.15.27:0/1858764788 >> 172.21.15.27:0/1858764788 conn(0x55f564d1d350 msgr2=0x55f5650d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:01.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.660+0000 7f199744f740 1 -- 172.21.15.27:0/1858764788 shutdown_connections 2024-09-21T00:21:01.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.660+0000 7f199744f740 1 -- 172.21.15.27:0/1858764788 wait complete. 2024-09-21T00:21:01.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.660+0000 7f199744f740 20 client.25234 trim_cache size 0 max 16384 2024-09-21T00:21:01.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.679+0000 7f199744f740 1 Processor -- start 2024-09-21T00:21:01.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:01 smithi027 ceph-mon[30745]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:21:01.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.680+0000 7f199744f740 1 -- start start 2024-09-21T00:21:01.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.680+0000 7f199744f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d000a0 0x55f5650d55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.680+0000 7f199744f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 0x55f5650d5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.680+0000 7f199744f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 0x55f5650d6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.680+0000 7f199744f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f5650eb770 con 0x55f564d08280 2024-09-21T00:21:01.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.680+0000 7f199744f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f5650ebe40 con 0x55f564d28f60 2024-09-21T00:21:01.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.680+0000 7f199744f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f5650ec510 con 0x55f564d000a0 2024-09-21T00:21:01.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.680+0000 7f198de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d000a0 0x55f5650d55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.680+0000 7f198d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 0x55f5650d5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.680+0000 7f198de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d000a0 0x55f5650d55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56048/0 (socket says 172.21.15.27:56048) 2024-09-21T00:21:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.680+0000 7f198de00640 1 -- 172.21.15.27:0/3528390940 learned_addr learned my addr 172.21.15.27:0/3528390940 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.681+0000 7f198e800640 1 --2- 172.21.15.27:0/3528390940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 0x55f5650d6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.681+0000 7f198de00640 1 -- 172.21.15.27:0/3528390940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 msgr2=0x55f5650d6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.681+0000 7f198de00640 1 --2- 172.21.15.27:0/3528390940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 0x55f5650d6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.681+0000 7f198de00640 1 -- 172.21.15.27:0/3528390940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 msgr2=0x55f5650d5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.681+0000 7f198de00640 1 --2- 172.21.15.27:0/3528390940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 0x55f5650d5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.681+0000 7f198de00640 1 -- 172.21.15.27:0/3528390940 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5650dc040 con 0x55f564d000a0 2024-09-21T00:21:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.681+0000 7f198e800640 1 --2- 172.21.15.27:0/3528390940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 0x55f5650d6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:21:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.681+0000 7f198de00640 1 --2- 172.21.15.27:0/3528390940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d000a0 0x55f5650d55b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f197c005940 tx=0x7f197c03f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10d2d95ec22ec55f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.682+0000 7f1976600640 1 -- 172.21.15.27:0/3528390940 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c035040 con 0x55f564d000a0 2024-09-21T00:21:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.682+0000 7f1976600640 1 -- 172.21.15.27:0/3528390940 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f197c056070 con 0x55f564d000a0 2024-09-21T00:21:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.682+0000 7f1976600640 1 -- 172.21.15.27:0/3528390940 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c05b6d0 con 0x55f564d000a0 2024-09-21T00:21:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.683+0000 7f199744f740 1 -- 172.21.15.27:0/3528390940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d000a0 msgr2=0x55f5650d55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.683+0000 7f199744f740 1 --2- 172.21.15.27:0/3528390940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d000a0 0x55f5650d55b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f197c005940 tx=0x7f197c03f000 comp rx=0 tx=0).stop 2024-09-21T00:21:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.683+0000 7f199744f740 1 -- 172.21.15.27:0/3528390940 shutdown_connections 2024-09-21T00:21:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.683+0000 7f199744f740 1 --2- 172.21.15.27:0/3528390940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 0x55f5650d6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.683+0000 7f199744f740 1 --2- 172.21.15.27:0/3528390940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d08280 0x55f5650d5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.683+0000 7f199744f740 1 --2- 172.21.15.27:0/3528390940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d000a0 0x55f5650d55b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.683+0000 7f199744f740 1 -- 172.21.15.27:0/3528390940 >> 172.21.15.27:0/3528390940 conn(0x55f564d1d350 msgr2=0x55f5650daf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.683+0000 7f199744f740 1 -- 172.21.15.27:0/3528390940 shutdown_connections 2024-09-21T00:21:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.683+0000 7f199744f740 1 -- 172.21.15.27:0/3528390940 wait complete. 2024-09-21T00:21:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.683+0000 7f199744f740 1 -- start start 2024-09-21T00:21:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.684+0000 7f199744f740 1 Processor -- start 2024-09-21T00:21:01.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.684+0000 7f199744f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d000a0 0x55f5650efaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.684+0000 7f199744f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d08280 0x55f5650f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.684+0000 7f199744f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 0x55f5650f5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.684+0000 7f199744f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f5650ed2b0 con 0x55f564d000a0 2024-09-21T00:21:01.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.684+0000 7f199744f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f5650ecbe0 con 0x55f564d28f60 2024-09-21T00:21:01.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.684+0000 7f199744f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f5650ed980 con 0x55f564d08280 2024-09-21T00:21:01.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.684+0000 7f199744f740 10 client.0 _ll_register_callbacks cb 0x7f19840ad9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:21:01.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.684+0000 7f198de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d000a0 0x55f5650efaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.685+0000 7f198de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d000a0 0x55f5650efaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58238/0 (socket says 172.21.15.27:58238) 2024-09-21T00:21:01.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.685+0000 7f198de00640 1 -- 172.21.15.27:0/2203994388 learned_addr learned my addr 172.21.15.27:0/2203994388 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:01.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.685+0000 7f198d400640 1 --2- 172.21.15.27:0/2203994388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d08280 0x55f5650f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.685+0000 7f198e800640 1 --2- 172.21.15.27:0/2203994388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 0x55f5650f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.685+0000 7f198de00640 1 -- 172.21.15.27:0/2203994388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d08280 msgr2=0x55f5650f00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.685+0000 7f198de00640 1 --2- 172.21.15.27:0/2203994388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d08280 0x55f5650f00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.685+0000 7f198de00640 1 -- 172.21.15.27:0/2203994388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 msgr2=0x55f5650f5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.685+0000 7f198de00640 1 --2- 172.21.15.27:0/2203994388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 0x55f5650f5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.685+0000 7f198de00640 1 -- 172.21.15.27:0/2203994388 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5650f57c0 con 0x55f564d000a0 2024-09-21T00:21:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.685+0000 7f198de00640 1 --2- 172.21.15.27:0/2203994388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d000a0 0x55f5650efaf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f197c031a90 tx=0x7f197c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc12bcac7404e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.685+0000 7f196b400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:21:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.686+0000 7f196b400640 1 -- 172.21.15.27:0/2203994388 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c063070 con 0x55f564d000a0 2024-09-21T00:21:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.686+0000 7f196b400640 1 -- 172.21.15.27:0/2203994388 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f197c0035f0 con 0x55f564d000a0 2024-09-21T00:21:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.686+0000 7f199744f740 1 -- 172.21.15.27:0/2203994388 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f197c056070 con 0x55f564d000a0 2024-09-21T00:21:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.686+0000 7f196b400640 1 -- 172.21.15.27:0/2203994388 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f197c06f050 con 0x55f564d000a0 2024-09-21T00:21:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.686+0000 7f199744f740 10 client.15240 fetch_fsmap learned FSMap version 25 2024-09-21T00:21:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.686+0000 7f199744f740 1 -- 172.21.15.27:0/2203994388 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f565158fe0 con 0x55f564d000a0 2024-09-21T00:21:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.687+0000 7f196b400640 1 -- 172.21.15.27:0/2203994388 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f197c056070 con 0x55f564d000a0 2024-09-21T00:21:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.687+0000 7f196b400640 1 -- 172.21.15.27:0/2203994388 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f197c05bae0 con 0x55f564d000a0 2024-09-21T00:21:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.687+0000 7f199744f740 10 client.15240 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T00:21:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.687+0000 7f199744f740 10 client.15240 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T00:21:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.687+0000 7f199744f740 20 client.15240 populate_metadata read hostname 'smithi027' 2024-09-21T00:21:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.687+0000 7f199744f740 1 --2- 172.21.15.27:0/2203994388 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f564cbcc90 0x55f56515c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.687+0000 7f199744f740 4 client.15240 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:21:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.687+0000 7f199744f740 1 -- 172.21.15.27:0/2203994388 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "status"}) -- 0x55f565044750 con 0x55f564cbcc90 2024-09-21T00:21:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.687+0000 7f198d400640 1 --2- 172.21.15.27:0/2203994388 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f564cbcc90 0x55f56515c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.688+0000 7f198d400640 1 --2- 172.21.15.27:0/2203994388 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f564cbcc90 0x55f56515c430 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34f87275f26431be server_cookie=b69509be0b13d365 in_seq=0 out_seq=0 2024-09-21T00:21:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.688+0000 7f196b400640 10 client.15240 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:21:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.688+0000 7f196b400640 1 -- 172.21.15.27:0/2203994388 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x55f565044750 con 0x55f564cbcc90 2024-09-21T00:21:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.688+0000 7f196b400640 10 client.15240 handle_command_reply: tid=0 2024-09-21T00:21:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.688+0000 7f199744f740 1 client.15240 shutdown 2024-09-21T00:21:01.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.689+0000 7f199744f740 1 -- 172.21.15.27:0/2203994388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d000a0 msgr2=0x55f5650efaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:01.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.689+0000 7f199744f740 1 --2- 172.21.15.27:0/2203994388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d000a0 0x55f5650efaf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f197c031a90 tx=0x7f197c02a410 comp rx=0 tx=0).stop 2024-09-21T00:21:01.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.689+0000 7f199744f740 1 -- 172.21.15.27:0/2203994388 shutdown_connections 2024-09-21T00:21:01.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.689+0000 7f199744f740 1 --2- 172.21.15.27:0/2203994388 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f564cbcc90 0x55f56515c430 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.689+0000 7f199744f740 1 --2- 172.21.15.27:0/2203994388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f564d28f60 0x55f5650f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.689+0000 7f199744f740 1 --2- 172.21.15.27:0/2203994388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f564d08280 0x55f5650f00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.689+0000 7f199744f740 1 --2- 172.21.15.27:0/2203994388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f564d000a0 0x55f5650efaf0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.689+0000 7f199744f740 1 -- 172.21.15.27:0/2203994388 >> 172.21.15.27:0/2203994388 conn(0x55f564d1d350 msgr2=0x55f5650daf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:01.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.689+0000 7f196b400640 0 client.15240 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:21:01.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.689+0000 7f199744f740 1 -- 172.21.15.27:0/2203994388 shutdown_connections 2024-09-21T00:21:01.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.689+0000 7f199744f740 1 -- 172.21.15.27:0/2203994388 wait complete. 2024-09-21T00:21:01.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.689+0000 7f199744f740 20 client.15240 trim_cache size 0 max 16384 2024-09-21T00:21:01.692 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:21:01.692 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "9c21de84-77ad-11ef-baf5-efdc52797490", 2024-09-21T00:21:01.693 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-21T00:21:01.693 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24761, 2024-09-21T00:21:01.693 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T00:21:01.693 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T00:21:01.693 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T00:21:01.693 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 141.02633283, 2024-09-21T00:21:01.693 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 25, 2024-09-21T00:21:01.693 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T00:21:01.693 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T00:21:01.693 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 154.49358823 2024-09-21T00:21:01.693 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:21:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.690+0000 7f1990800640 1 -- 172.21.15.27:0/3996774570 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f19700b6560 msgr2=0x7f19700b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.690+0000 7f1990800640 1 --2- 172.21.15.27:0/3996774570 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f19700b6560 0x7f19700b8a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f197c031440 tx=0x7f197c0058d0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.690+0000 7f1990800640 1 -- 172.21.15.27:0/3996774570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198810c010 msgr2=0x7f19881a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.690+0000 7f1990800640 1 --2- 172.21.15.27:0/3996774570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198810c010 0x7f19881a1ee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1984009fd0 tx=0x7f198400ef10 comp rx=0 tx=0).stop 2024-09-21T00:21:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.691+0000 7f1990800640 1 -- 172.21.15.27:0/3996774570 shutdown_connections 2024-09-21T00:21:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.691+0000 7f1990800640 1 --2- 172.21.15.27:0/3996774570 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f19700b6560 0x7f19700b8a70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.691+0000 7f1990800640 1 --2- 172.21.15.27:0/3996774570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198810c010 0x7f19881a1ee0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.691+0000 7f1990800640 1 --2- 172.21.15.27:0/3996774570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f198810b6c0 0x7f198819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.691+0000 7f1990800640 1 --2- 172.21.15.27:0/3996774570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1988105d80 0x7f198819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.691+0000 7f1990800640 1 -- 172.21.15.27:0/3996774570 >> 172.21.15.27:0/3996774570 conn(0x7f1988101810 msgr2=0x7f1988103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.691+0000 7f1990800640 1 -- 172.21.15.27:0/3996774570 shutdown_connections 2024-09-21T00:21:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:01.691+0000 7f1990800640 1 -- 172.21.15.27:0/3996774570 wait complete. 2024-09-21T00:21:01.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:01 smithi089 ceph-mon[34481]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:21:01.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:01 smithi098 ceph-mon[34725]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:21:02.455 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T00:21:02.455 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T00:21:02.467 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T00:21:02.468 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T00:21:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:03.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.495+0000 7f022de00640 1 -- 172.21.15.27:0/3429194799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0228105bb0 msgr2=0x7f0228105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:03.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.495+0000 7f022de00640 1 --2- 172.21.15.27:0/3429194799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0228105bb0 0x7f0228105fc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0218009900 tx=0x7f021802a410 comp rx=0 tx=0).stop 2024-09-21T00:21:03.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.495+0000 7f022de00640 1 -- 172.21.15.27:0/3429194799 shutdown_connections 2024-09-21T00:21:03.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.495+0000 7f022de00640 1 --2- 172.21.15.27:0/3429194799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f022810be40 0x7f0228112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:03.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.495+0000 7f022de00640 1 --2- 172.21.15.27:0/3429194799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022810b4f0 0x7f022810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:03.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.495+0000 7f022de00640 1 --2- 172.21.15.27:0/3429194799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0228105bb0 0x7f0228105fc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:03.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.495+0000 7f022de00640 1 -- 172.21.15.27:0/3429194799 >> 172.21.15.27:0/3429194799 conn(0x7f0228101680 msgr2=0x7f0228103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:03.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.495+0000 7f022de00640 1 -- 172.21.15.27:0/3429194799 shutdown_connections 2024-09-21T00:21:03.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.495+0000 7f022de00640 1 -- 172.21.15.27:0/3429194799 wait complete. 2024-09-21T00:21:03.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.496+0000 7f022de00640 1 Processor -- start 2024-09-21T00:21:03.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.497+0000 7f022de00640 1 -- start start 2024-09-21T00:21:03.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.497+0000 7f022de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0228105bb0 0x7f022819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:03.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.497+0000 7f022de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f022810b4f0 0x7f022819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:03.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.497+0000 7f022de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022810be40 0x7f02281a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:03.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.497+0000 7f022de00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0228115b00 con 0x7f022810be40 2024-09-21T00:21:03.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.497+0000 7f022de00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0228115980 con 0x7f022810b4f0 2024-09-21T00:21:03.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.497+0000 7f022de00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0228115c80 con 0x7f0228105bb0 2024-09-21T00:21:03.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.497+0000 7f022ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0228105bb0 0x7f022819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:03.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.497+0000 7f022ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0228105bb0 0x7f022819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:54210/0 (socket says 172.21.15.27:54210) 2024-09-21T00:21:03.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.497+0000 7f022ca00640 1 -- 172.21.15.27:0/1031522225 learned_addr learned my addr 172.21.15.27:0/1031522225 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:03.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.497+0000 7f021fe00640 1 --2- 172.21.15.27:0/1031522225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f022810b4f0 0x7f022819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:03.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.497+0000 7f022d400640 1 --2- 172.21.15.27:0/1031522225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022810be40 0x7f02281a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:03.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.498+0000 7f022ca00640 1 -- 172.21.15.27:0/1031522225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f022810b4f0 msgr2=0x7f022819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:03.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.498+0000 7f022ca00640 1 --2- 172.21.15.27:0/1031522225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f022810b4f0 0x7f022819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:03.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.498+0000 7f022ca00640 1 -- 172.21.15.27:0/1031522225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022810be40 msgr2=0x7f02281a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:03.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.498+0000 7f022ca00640 1 --2- 172.21.15.27:0/1031522225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022810be40 0x7f02281a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:03.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.498+0000 7f022ca00640 1 -- 172.21.15.27:0/1031522225 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02281a2480 con 0x7f0228105bb0 2024-09-21T00:21:03.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.498+0000 7f022ca00640 1 --2- 172.21.15.27:0/1031522225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0228105bb0 0x7f022819c510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0218005940 tx=0x7f02180040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:03.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.498+0000 7f0226000640 1 -- 172.21.15.27:0/1031522225 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021803f030 con 0x7f0228105bb0 2024-09-21T00:21:03.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.499+0000 7f0226000640 1 -- 172.21.15.27:0/1031522225 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02180044c0 con 0x7f0228105bb0 2024-09-21T00:21:03.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.499+0000 7f022de00640 1 -- 172.21.15.27:0/1031522225 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02281a2710 con 0x7f0228105bb0 2024-09-21T00:21:03.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.499+0000 7f0226000640 1 -- 172.21.15.27:0/1031522225 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218035070 con 0x7f0228105bb0 2024-09-21T00:21:03.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.499+0000 7f022de00640 1 -- 172.21.15.27:0/1031522225 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02281a8ed0 con 0x7f0228105bb0 2024-09-21T00:21:03.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.500+0000 7f022de00640 1 -- 172.21.15.27:0/1031522225 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0228106e30 con 0x7f0228105bb0 2024-09-21T00:21:03.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.502+0000 7f0226000640 1 -- 172.21.15.27:0/1031522225 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f0218042030 con 0x7f0228105bb0 2024-09-21T00:21:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.504+0000 7f0226000640 1 --2- 172.21.15.27:0/1031522225 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f020c0b6560 0x7f020c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.504+0000 7f0226000640 1 -- 172.21.15.27:0/1031522225 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f0218107bc0 con 0x7f0228105bb0 2024-09-21T00:21:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.504+0000 7f021fe00640 1 --2- 172.21.15.27:0/1031522225 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f020c0b6560 0x7f020c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.505+0000 7f021fe00640 1 --2- 172.21.15.27:0/1031522225 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f020c0b6560 0x7f020c0b8a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f022810aef0 tx=0x7f0210005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:03.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.508+0000 7f0226000640 1 -- 172.21.15.27:0/1031522225 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f02180d2320 con 0x7f0228105bb0 2024-09-21T00:21:03.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:03 smithi027 ceph-mon[30745]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:21:03.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.726+0000 7f022de00640 1 -- 172.21.15.27:0/1031522225 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f022810dc70 con 0x7f020c0b6560 2024-09-21T00:21:03.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.727+0000 7f0226000640 1 -- 172.21.15.27:0/1031522225 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f022810dc70 con 0x7f020c0b6560 2024-09-21T00:21:03.728 INFO:teuthology.orchestra.run.smithi027.stdout:Backend: cephadm 2024-09-21T00:21:03.729 INFO:teuthology.orchestra.run.smithi027.stdout:Available: Yes 2024-09-21T00:21:03.729 INFO:teuthology.orchestra.run.smithi027.stdout:Paused: No 2024-09-21T00:21:03.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:03 smithi089 ceph-mon[34481]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:21:03.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.729+0000 7f022de00640 1 -- 172.21.15.27:0/1031522225 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f020c0b6560 msgr2=0x7f020c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:03.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.729+0000 7f022de00640 1 --2- 172.21.15.27:0/1031522225 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f020c0b6560 0x7f020c0b8a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f022810aef0 tx=0x7f0210005f90 comp rx=0 tx=0).stop 2024-09-21T00:21:03.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.729+0000 7f022de00640 1 -- 172.21.15.27:0/1031522225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0228105bb0 msgr2=0x7f022819c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:03.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.729+0000 7f022de00640 1 --2- 172.21.15.27:0/1031522225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0228105bb0 0x7f022819c510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0218005940 tx=0x7f02180040c0 comp rx=0 tx=0).stop 2024-09-21T00:21:03.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.730+0000 7f022de00640 1 -- 172.21.15.27:0/1031522225 shutdown_connections 2024-09-21T00:21:03.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.730+0000 7f022de00640 1 --2- 172.21.15.27:0/1031522225 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f020c0b6560 0x7f020c0b8a70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:03.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.730+0000 7f022de00640 1 --2- 172.21.15.27:0/1031522225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022810be40 0x7f02281a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:03.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.730+0000 7f022de00640 1 --2- 172.21.15.27:0/1031522225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f022810b4f0 0x7f022819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:03.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.730+0000 7f022de00640 1 --2- 172.21.15.27:0/1031522225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0228105bb0 0x7f022819c510 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:03.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.730+0000 7f022de00640 1 -- 172.21.15.27:0/1031522225 >> 172.21.15.27:0/1031522225 conn(0x7f0228101680 msgr2=0x7f0228103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:03.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.730+0000 7f022de00640 1 -- 172.21.15.27:0/1031522225 shutdown_connections 2024-09-21T00:21:03.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:03.730+0000 7f022de00640 1 -- 172.21.15.27:0/1031522225 wait complete. 2024-09-21T00:21:03.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:03 smithi098 ceph-mon[34725]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:21:04.549 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T00:21:04.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:04 smithi089 ceph-mon[34481]: from='client.25246 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:04.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:04 smithi098 ceph-mon[34725]: from='client.25246 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:04.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:04 smithi027 ceph-mon[30745]: from='client.25246 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:04.936 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:05.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.582+0000 7f2fdd400640 1 -- 172.21.15.27:0/3417396204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fd8103b60 msgr2=0x7f2fd8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:05.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.582+0000 7f2fdd400640 1 --2- 172.21.15.27:0/3417396204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fd8103b60 0x7f2fd8103f70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009900 tx=0x7f2fc802a400 comp rx=0 tx=0).stop 2024-09-21T00:21:05.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.582+0000 7f2fdd400640 1 -- 172.21.15.27:0/3417396204 shutdown_connections 2024-09-21T00:21:05.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.582+0000 7f2fdd400640 1 --2- 172.21.15.27:0/3417396204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd8109df0 0x7f2fd8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:05.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.582+0000 7f2fdd400640 1 --2- 172.21.15.27:0/3417396204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd81094a0 0x7f2fd81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:05.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.582+0000 7f2fdd400640 1 --2- 172.21.15.27:0/3417396204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fd8103b60 0x7f2fd8103f70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:05.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.582+0000 7f2fdd400640 1 -- 172.21.15.27:0/3417396204 >> 172.21.15.27:0/3417396204 conn(0x7f2fd807b850 msgr2=0x7f2fd807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:05.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.582+0000 7f2fdd400640 1 -- 172.21.15.27:0/3417396204 shutdown_connections 2024-09-21T00:21:05.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.582+0000 7f2fdd400640 1 -- 172.21.15.27:0/3417396204 wait complete. 2024-09-21T00:21:05.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.583+0000 7f2fdd400640 1 Processor -- start 2024-09-21T00:21:05.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.583+0000 7f2fdd400640 1 -- start start 2024-09-21T00:21:05.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.584+0000 7f2fdd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fd8103b60 0x7f2fd819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:05.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.584+0000 7f2fdd400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd81094a0 0x7f2fd819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:05.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.584+0000 7f2fdd400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd8109df0 0x7f2fd81a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:05.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.584+0000 7f2fdd400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd8113b00 con 0x7f2fd8103b60 2024-09-21T00:21:05.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.584+0000 7f2fdd400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd8113980 con 0x7f2fd81094a0 2024-09-21T00:21:05.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.584+0000 7f2fdd400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd8113c80 con 0x7f2fd8109df0 2024-09-21T00:21:05.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.584+0000 7f2fd6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fd8103b60 0x7f2fd819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:05.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.584+0000 7f2fd6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fd8103b60 0x7f2fd819c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60222/0 (socket says 172.21.15.27:60222) 2024-09-21T00:21:05.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.584+0000 7f2fd6800640 1 -- 172.21.15.27:0/982309855 learned_addr learned my addr 172.21.15.27:0/982309855 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:05.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.584+0000 7f2fcfe00640 1 --2- 172.21.15.27:0/982309855 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd81094a0 0x7f2fd819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:05.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.584+0000 7f2fd7200640 1 --2- 172.21.15.27:0/982309855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd8109df0 0x7f2fd81a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:05.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.585+0000 7f2fd6800640 1 -- 172.21.15.27:0/982309855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd8109df0 msgr2=0x7f2fd81a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:05.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.585+0000 7f2fd6800640 1 --2- 172.21.15.27:0/982309855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd8109df0 0x7f2fd81a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:05.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.585+0000 7f2fd6800640 1 -- 172.21.15.27:0/982309855 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd81094a0 msgr2=0x7f2fd819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:05.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.585+0000 7f2fd6800640 1 --2- 172.21.15.27:0/982309855 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd81094a0 0x7f2fd819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:05.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.585+0000 7f2fd6800640 1 -- 172.21.15.27:0/982309855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd81a2680 con 0x7f2fd8103b60 2024-09-21T00:21:05.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.585+0000 7f2fd6800640 1 --2- 172.21.15.27:0/982309855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fd8103b60 0x7f2fd819c710 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8005ab0 tx=0x7f2fc8003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:05.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.585+0000 7f2fcf400640 1 -- 172.21.15.27:0/982309855 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc803f070 con 0x7f2fd8103b60 2024-09-21T00:21:05.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.586+0000 7f2fcf400640 1 -- 172.21.15.27:0/982309855 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc8031d60 con 0x7f2fd8103b60 2024-09-21T00:21:05.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.586+0000 7f2fdd400640 1 -- 172.21.15.27:0/982309855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd81a2910 con 0x7f2fd8103b60 2024-09-21T00:21:05.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.586+0000 7f2fcf400640 1 -- 172.21.15.27:0/982309855 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8035030 con 0x7f2fd8103b60 2024-09-21T00:21:05.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.586+0000 7f2fdd400640 1 -- 172.21.15.27:0/982309855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd81a90d0 con 0x7f2fd8103b60 2024-09-21T00:21:05.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.587+0000 7f2fdd400640 1 -- 172.21.15.27:0/982309855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd8103f70 con 0x7f2fd8103b60 2024-09-21T00:21:05.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.591+0000 7f2fcf400640 1 -- 172.21.15.27:0/982309855 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f2fc8042070 con 0x7f2fd8103b60 2024-09-21T00:21:05.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.593+0000 7f2fcf400640 1 --2- 172.21.15.27:0/982309855 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2fbc0b6560 0x7f2fbc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:05.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.593+0000 7f2fcf400640 1 -- 172.21.15.27:0/982309855 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f2fc8108650 con 0x7f2fd8103b60 2024-09-21T00:21:05.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.593+0000 7f2fcfe00640 1 --2- 172.21.15.27:0/982309855 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2fbc0b6560 0x7f2fbc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:05.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.594+0000 7f2fcfe00640 1 --2- 172.21.15.27:0/982309855 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2fbc0b6560 0x7f2fbc0b8a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8106370 tx=0x7f2fc000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:05.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.595+0000 7f2fcf400640 1 -- 172.21.15.27:0/982309855 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2fc80d2db0 con 0x7f2fd8103b60 2024-09-21T00:21:05.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:05 smithi089 ceph-mon[34481]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:21:05.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:05 smithi098 ceph-mon[34725]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:21:05.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.807+0000 7f2fdd400640 1 -- 172.21.15.27:0/982309855 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2fd8107a20 con 0x7f2fbc0b6560 2024-09-21T00:21:05.808 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:05 smithi027 ceph-mon[30745]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:21:05.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.820+0000 7f2fcf400640 1 -- 172.21.15.27:0/982309855 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f2fd8107a20 con 0x7f2fbc0b6560 2024-09-21T00:21:05.821 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T00:21:05.822 INFO:teuthology.orchestra.run.smithi027.stdout:mds.a smithi027 running (2m) 22s ago 2m 85.3M - 19.3.0-5120-g18432218 ddce307e1f32 c5af4bbf1b58 2024-09-21T00:21:05.822 INFO:teuthology.orchestra.run.smithi027.stdout:mds.b smithi089 running (2m) 26s ago 2m 95.0M - 19.3.0-5120-g18432218 ddce307e1f32 a4b42d6d8368 2024-09-21T00:21:05.822 INFO:teuthology.orchestra.run.smithi027.stdout:mds.c smithi098 running (2m) 26s ago 2m 87.9M - 19.3.0-5120-g18432218 ddce307e1f32 9bacc637e50c 2024-09-21T00:21:05.822 INFO:teuthology.orchestra.run.smithi027.stdout:mds.d smithi027 running (3m) 22s ago 3m 81.0M - 19.3.0-5120-g18432218 ddce307e1f32 e2ac6703cc5d 2024-09-21T00:21:05.822 INFO:teuthology.orchestra.run.smithi027.stdout:mds.e smithi089 running (3m) 26s ago 2m 82.1M - 19.3.0-5120-g18432218 ddce307e1f32 976b1886d114 2024-09-21T00:21:05.822 INFO:teuthology.orchestra.run.smithi027.stdout:mds.f smithi098 running (2m) 26s ago 2m 88.1M - 19.3.0-5120-g18432218 ddce307e1f32 fac7a7a53ad7 2024-09-21T00:21:05.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.g smithi027 running (2m) 22s ago 2m 80.7M - 19.3.0-5120-g18432218 ddce307e1f32 a57428cef61d 2024-09-21T00:21:05.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.h smithi089 running (2m) 26s ago 2m 87.0M - 19.3.0-5120-g18432218 ddce307e1f32 3483ea512a72 2024-09-21T00:21:05.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.i smithi098 running (2m) 26s ago 2m 86.1M - 19.3.0-5120-g18432218 ddce307e1f32 6d1c23916cc1 2024-09-21T00:21:05.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.j smithi027 running (3m) 22s ago 3m 80.9M - 19.3.0-5120-g18432218 ddce307e1f32 8f3c7975b433 2024-09-21T00:21:05.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.k smithi089 running (3m) 26s ago 3m 82.6M - 19.3.0-5120-g18432218 ddce307e1f32 8038e914072b 2024-09-21T00:21:05.824 INFO:teuthology.orchestra.run.smithi027.stdout:mds.l smithi098 running (3m) 26s ago 3m 82.6M - 19.3.0-5120-g18432218 ddce307e1f32 e5614accb068 2024-09-21T00:21:05.824 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.x smithi027 *:9283,8765 running (11m) 22s ago 11m 586M - 19.3.0-5120-g18432218 ddce307e1f32 c336f5fa731b 2024-09-21T00:21:05.824 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.y smithi089 *:8443,8765 running (9m) 26s ago 9m 524M - 19.3.0-5120-g18432218 ddce307e1f32 69bc3dd20503 2024-09-21T00:21:05.824 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.z smithi098 *:8443,8765 running (9m) 26s ago 9m 527M - 19.3.0-5120-g18432218 ddce307e1f32 422b28bd1a06 2024-09-21T00:21:05.824 INFO:teuthology.orchestra.run.smithi027.stdout:mon.a smithi027 running (11m) 22s ago 11m 121M 2048M 19.3.0-5120-g18432218 ddce307e1f32 3adf3fc84023 2024-09-21T00:21:05.824 INFO:teuthology.orchestra.run.smithi027.stdout:mon.b smithi089 running (9m) 26s ago 9m 111M 2048M 19.3.0-5120-g18432218 ddce307e1f32 834528238ca8 2024-09-21T00:21:05.825 INFO:teuthology.orchestra.run.smithi027.stdout:mon.c smithi098 running (10m) 26s ago 10m 113M 2048M 19.3.0-5120-g18432218 ddce307e1f32 e211261a2fc9 2024-09-21T00:21:05.825 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (8m) 22s ago 8m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 c9317691a4a7 2024-09-21T00:21:05.825 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi089 running (8m) 26s ago 8m 121M 4271M 19.3.0-5120-g18432218 ddce307e1f32 60346f87f015 2024-09-21T00:21:05.825 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi098 running (7m) 26s ago 7m 114M 4272M 19.3.0-5120-g18432218 ddce307e1f32 996f59ee0e0a 2024-09-21T00:21:05.825 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi027 running (7m) 22s ago 7m 113M 4272M 19.3.0-5120-g18432218 ddce307e1f32 a2a08cdf0d89 2024-09-21T00:21:05.825 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi089 running (6m) 26s ago 6m 121M 4271M 19.3.0-5120-g18432218 ddce307e1f32 f274b0470b3b 2024-09-21T00:21:05.826 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi098 running (6m) 26s ago 6m 115M 4272M 19.3.0-5120-g18432218 ddce307e1f32 b2e4837abc90 2024-09-21T00:21:05.826 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 smithi027 running (5m) 22s ago 5m 116M 4272M 19.3.0-5120-g18432218 ddce307e1f32 41b19b872c1f 2024-09-21T00:21:05.826 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 smithi089 running (5m) 26s ago 5m 113M 4271M 19.3.0-5120-g18432218 ddce307e1f32 fc0e65b168e0 2024-09-21T00:21:05.826 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 smithi098 running (4m) 26s ago 4m 120M 4272M 19.3.0-5120-g18432218 ddce307e1f32 5acc01f958b5 2024-09-21T00:21:05.826 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 smithi027 running (4m) 22s ago 4m 116M 4272M 19.3.0-5120-g18432218 ddce307e1f32 1865bd23c2c0 2024-09-21T00:21:05.826 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 smithi089 running (3m) 26s ago 3m 118M 4271M 19.3.0-5120-g18432218 ddce307e1f32 456854bb1edd 2024-09-21T00:21:05.827 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 smithi098 running (3m) 26s ago 3m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 7ce233965c13 2024-09-21T00:21:05.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.822+0000 7f2fdd400640 1 -- 172.21.15.27:0/982309855 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2fbc0b6560 msgr2=0x7f2fbc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:05.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.822+0000 7f2fdd400640 1 --2- 172.21.15.27:0/982309855 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2fbc0b6560 0x7f2fbc0b8a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8106370 tx=0x7f2fc000b390 comp rx=0 tx=0).stop 2024-09-21T00:21:05.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.822+0000 7f2fdd400640 1 -- 172.21.15.27:0/982309855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fd8103b60 msgr2=0x7f2fd819c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:05.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.822+0000 7f2fdd400640 1 --2- 172.21.15.27:0/982309855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fd8103b60 0x7f2fd819c710 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8005ab0 tx=0x7f2fc8003cd0 comp rx=0 tx=0).stop 2024-09-21T00:21:05.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.822+0000 7f2fdd400640 1 -- 172.21.15.27:0/982309855 shutdown_connections 2024-09-21T00:21:05.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.822+0000 7f2fdd400640 1 --2- 172.21.15.27:0/982309855 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2fbc0b6560 0x7f2fbc0b8a70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:05.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.822+0000 7f2fdd400640 1 --2- 172.21.15.27:0/982309855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd8109df0 0x7f2fd81a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:05.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.822+0000 7f2fdd400640 1 --2- 172.21.15.27:0/982309855 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd81094a0 0x7f2fd819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:05.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.822+0000 7f2fdd400640 1 --2- 172.21.15.27:0/982309855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fd8103b60 0x7f2fd819c710 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:05.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.822+0000 7f2fdd400640 1 -- 172.21.15.27:0/982309855 >> 172.21.15.27:0/982309855 conn(0x7f2fd807b850 msgr2=0x7f2fd8107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:05.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.823+0000 7f2fdd400640 1 -- 172.21.15.27:0/982309855 shutdown_connections 2024-09-21T00:21:05.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:05.823+0000 7f2fdd400640 1 -- 172.21.15.27:0/982309855 wait complete. 2024-09-21T00:21:06.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:06 smithi027 ceph-mon[30745]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:06.631 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T00:21:06.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:06 smithi089 ceph-mon[34481]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:06.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:06 smithi098 ceph-mon[34725]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:07.016 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:07.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.664+0000 7fb5fda00640 1 -- 172.21.15.27:0/1462862161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f81092f0 msgr2=0x7fb5f8109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:07.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.664+0000 7fb5fda00640 1 --2- 172.21.15.27:0/1462862161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f81092f0 0x7fb5f8109700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009900 tx=0x7fb5e402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:21:07.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.665+0000 7fb5fda00640 1 -- 172.21.15.27:0/1462862161 shutdown_connections 2024-09-21T00:21:07.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.665+0000 7fb5fda00640 1 --2- 172.21.15.27:0/1462862161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f8109c40 0x7fb5f8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:07.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.665+0000 7fb5fda00640 1 --2- 172.21.15.27:0/1462862161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f81092f0 0x7fb5f8109700 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:07.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.665+0000 7fb5fda00640 1 --2- 172.21.15.27:0/1462862161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f81039b0 0x7fb5f8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:07.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.665+0000 7fb5fda00640 1 -- 172.21.15.27:0/1462862161 >> 172.21.15.27:0/1462862161 conn(0x7fb5f80ff440 msgr2=0x7fb5f8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:07.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.665+0000 7fb5fda00640 1 -- 172.21.15.27:0/1462862161 shutdown_connections 2024-09-21T00:21:07.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.665+0000 7fb5fda00640 1 -- 172.21.15.27:0/1462862161 wait complete. 2024-09-21T00:21:07.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.666+0000 7fb5fda00640 1 Processor -- start 2024-09-21T00:21:07.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.666+0000 7fb5fda00640 1 -- start start 2024-09-21T00:21:07.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.666+0000 7fb5fda00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f81039b0 0x7fb5f819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:07.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.666+0000 7fb5fda00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f81092f0 0x7fb5f819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:07.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.666+0000 7fb5fda00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f8109c40 0x7fb5f819fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:07.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.666+0000 7fb5fda00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f8113900 con 0x7fb5f81039b0 2024-09-21T00:21:07.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.666+0000 7fb5fda00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f8113780 con 0x7fb5f81092f0 2024-09-21T00:21:07.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.666+0000 7fb5fda00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f8113a80 con 0x7fb5f8109c40 2024-09-21T00:21:07.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.667+0000 7fb5f7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f81039b0 0x7fb5f819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:07.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.667+0000 7fb5f7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f81039b0 0x7fb5f819a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60244/0 (socket says 172.21.15.27:60244) 2024-09-21T00:21:07.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.667+0000 7fb5f7e00640 1 -- 172.21.15.27:0/354831819 learned_addr learned my addr 172.21.15.27:0/354831819 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:07.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.667+0000 7fb5f7400640 1 --2- 172.21.15.27:0/354831819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f81092f0 0x7fb5f819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:07.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.667+0000 7fb5fd000640 1 --2- 172.21.15.27:0/354831819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f8109c40 0x7fb5f819fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:07.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.667+0000 7fb5f7e00640 1 -- 172.21.15.27:0/354831819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f8109c40 msgr2=0x7fb5f819fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:07.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.667+0000 7fb5f7e00640 1 --2- 172.21.15.27:0/354831819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f8109c40 0x7fb5f819fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:07.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.667+0000 7fb5f7e00640 1 -- 172.21.15.27:0/354831819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f81092f0 msgr2=0x7fb5f819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:07.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.667+0000 7fb5f7e00640 1 --2- 172.21.15.27:0/354831819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f81092f0 0x7fb5f819a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:07.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.667+0000 7fb5f7e00640 1 -- 172.21.15.27:0/354831819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f81a0280 con 0x7fb5f81039b0 2024-09-21T00:21:07.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.667+0000 7fb5f7400640 1 --2- 172.21.15.27:0/354831819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f81092f0 0x7fb5f819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:07.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.668+0000 7fb5f7e00640 1 --2- 172.21.15.27:0/354831819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f81039b0 0x7fb5f819a310 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e9f0 tx=0x7fb5e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:07.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.668+0000 7fb5f4c00640 1 -- 172.21.15.27:0/354831819 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800cd80 con 0x7fb5f81039b0 2024-09-21T00:21:07.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.668+0000 7fb5f4c00640 1 -- 172.21.15.27:0/354831819 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e80108e0 con 0x7fb5f81039b0 2024-09-21T00:21:07.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.668+0000 7fb5f4c00640 1 -- 172.21.15.27:0/354831819 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8018980 con 0x7fb5f81039b0 2024-09-21T00:21:07.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.668+0000 7fb5fda00640 1 -- 172.21.15.27:0/354831819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f81a0570 con 0x7fb5f81039b0 2024-09-21T00:21:07.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.668+0000 7fb5fda00640 1 -- 172.21.15.27:0/354831819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f81a6cd0 con 0x7fb5f81039b0 2024-09-21T00:21:07.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.669+0000 7fb5fda00640 1 -- 172.21.15.27:0/354831819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f8103dc0 con 0x7fb5f81039b0 2024-09-21T00:21:07.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.672+0000 7fb5f4c00640 1 -- 172.21.15.27:0/354831819 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fb5e8010430 con 0x7fb5f81039b0 2024-09-21T00:21:07.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.673+0000 7fb5f4c00640 1 --2- 172.21.15.27:0/354831819 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb5dc0b6560 0x7fb5dc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:07.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.673+0000 7fb5f4c00640 1 -- 172.21.15.27:0/354831819 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fb5e80e7ff0 con 0x7fb5f81039b0 2024-09-21T00:21:07.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.673+0000 7fb5f7400640 1 --2- 172.21.15.27:0/354831819 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb5dc0b6560 0x7fb5dc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:07.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.674+0000 7fb5f7400640 1 --2- 172.21.15.27:0/354831819 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb5dc0b6560 0x7fb5dc0b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009230 tx=0x7fb5e4031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:07.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.675+0000 7fb5f4c00640 1 -- 172.21.15.27:0/354831819 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb5e80b2790 con 0x7fb5f81039b0 2024-09-21T00:21:07.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:07 smithi089 ceph-mon[34481]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-21T00:21:07.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:07 smithi098 ceph-mon[34725]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-21T00:21:07.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:07 smithi027 ceph-mon[30745]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-21T00:21:07.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.881+0000 7fb5fda00640 1 -- 172.21.15.27:0/354831819 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fb5f8063800 con 0x7fb5dc0b6560 2024-09-21T00:21:07.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.887+0000 7fb5f4c00640 1 -- 172.21.15.27:0/354831819 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fb5f8063800 con 0x7fb5dc0b6560 2024-09-21T00:21:07.888 INFO:teuthology.orchestra.run.smithi027.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T00:21:07.888 INFO:teuthology.orchestra.run.smithi027.stdout:mds.all 12/12 28s ago 3m smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l;count:12 2024-09-21T00:21:07.888 INFO:teuthology.orchestra.run.smithi027.stdout:mgr 3/3 28s ago 9m smithi027=x;smithi089=y;smithi098=z;count:3 2024-09-21T00:21:07.888 INFO:teuthology.orchestra.run.smithi027.stdout:mon 3/3 28s ago 10m smithi027:172.21.15.27=a;smithi089:172.21.15.89=b;smithi098:172.21.15.98=c;count:3 2024-09-21T00:21:07.889 INFO:teuthology.orchestra.run.smithi027.stdout:osd 12 28s ago - 2024-09-21T00:21:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.889+0000 7fb5fda00640 1 -- 172.21.15.27:0/354831819 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb5dc0b6560 msgr2=0x7fb5dc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.889+0000 7fb5fda00640 1 --2- 172.21.15.27:0/354831819 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb5dc0b6560 0x7fb5dc0b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009230 tx=0x7fb5e4031400 comp rx=0 tx=0).stop 2024-09-21T00:21:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.889+0000 7fb5fda00640 1 -- 172.21.15.27:0/354831819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f81039b0 msgr2=0x7fb5f819a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.889+0000 7fb5fda00640 1 --2- 172.21.15.27:0/354831819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f81039b0 0x7fb5f819a310 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e9f0 tx=0x7fb5e800eeb0 comp rx=0 tx=0).stop 2024-09-21T00:21:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.890+0000 7fb5fda00640 1 -- 172.21.15.27:0/354831819 shutdown_connections 2024-09-21T00:21:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.890+0000 7fb5fda00640 1 --2- 172.21.15.27:0/354831819 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb5dc0b6560 0x7fb5dc0b8a70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.890+0000 7fb5fda00640 1 --2- 172.21.15.27:0/354831819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f8109c40 0x7fb5f819fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.890+0000 7fb5fda00640 1 --2- 172.21.15.27:0/354831819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f81092f0 0x7fb5f819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.890+0000 7fb5fda00640 1 --2- 172.21.15.27:0/354831819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f81039b0 0x7fb5f819a310 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.890+0000 7fb5fda00640 1 -- 172.21.15.27:0/354831819 >> 172.21.15.27:0/354831819 conn(0x7fb5f80ff440 msgr2=0x7fb5f8101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.890+0000 7fb5fda00640 1 -- 172.21.15.27:0/354831819 shutdown_connections 2024-09-21T00:21:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:07.890+0000 7fb5fda00640 1 -- 172.21.15.27:0/354831819 wait complete. 2024-09-21T00:21:08.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:08 smithi027 ceph-mon[30745]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:08.642 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T00:21:08.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:08 smithi089 ceph-mon[34481]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:08.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:08 smithi098 ceph-mon[34725]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:09.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.715+0000 7f2983e00640 1 -- 172.21.15.27:0/3420885497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f297c109df0 msgr2=0x7f297c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:09.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.715+0000 7f2983e00640 1 --2- 172.21.15.27:0/3420885497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f297c109df0 0x7f297c110620 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2974009ca0 tx=0x7f297402a8c0 comp rx=0 tx=0).stop 2024-09-21T00:21:09.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.715+0000 7f2983e00640 1 -- 172.21.15.27:0/3420885497 shutdown_connections 2024-09-21T00:21:09.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.715+0000 7f2983e00640 1 --2- 172.21.15.27:0/3420885497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f297c109df0 0x7f297c110620 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:09.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.715+0000 7f2983e00640 1 --2- 172.21.15.27:0/3420885497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f297c1094a0 0x7f297c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:09.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.715+0000 7f2983e00640 1 --2- 172.21.15.27:0/3420885497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f297c103b60 0x7f297c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:09.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.715+0000 7f2983e00640 1 -- 172.21.15.27:0/3420885497 >> 172.21.15.27:0/3420885497 conn(0x7f297c07b850 msgr2=0x7f297c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:09.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.716+0000 7f2983e00640 1 -- 172.21.15.27:0/3420885497 shutdown_connections 2024-09-21T00:21:09.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.716+0000 7f2983e00640 1 -- 172.21.15.27:0/3420885497 wait complete. 2024-09-21T00:21:09.718 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:09 smithi027 ceph-mon[30745]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T00:21:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.717+0000 7f2983e00640 1 Processor -- start 2024-09-21T00:21:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.717+0000 7f2983e00640 1 -- start start 2024-09-21T00:21:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.718+0000 7f2983e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f297c103b60 0x7f297c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.718+0000 7f2983e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f297c1094a0 0x7f297c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.718+0000 7f2983e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f297c109df0 0x7f297c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.718+0000 7f2983e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f297c113ab0 con 0x7f297c1094a0 2024-09-21T00:21:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.718+0000 7f2983e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f297c113930 con 0x7f297c103b60 2024-09-21T00:21:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.718+0000 7f2983e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f297c113c30 con 0x7f297c109df0 2024-09-21T00:21:09.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.718+0000 7f2981400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f297c103b60 0x7f297c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:09.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.718+0000 7f2980a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f297c1094a0 0x7f297c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:09.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.718+0000 7f2981400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f297c103b60 0x7f297c19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:36906/0 (socket says 172.21.15.27:36906) 2024-09-21T00:21:09.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.718+0000 7f2981400640 1 -- 172.21.15.27:0/3381270694 learned_addr learned my addr 172.21.15.27:0/3381270694 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:09.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.718+0000 7f2981e00640 1 --2- 172.21.15.27:0/3381270694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f297c109df0 0x7f297c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:09.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.719+0000 7f2981e00640 1 -- 172.21.15.27:0/3381270694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f297c103b60 msgr2=0x7f297c19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:09.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.719+0000 7f2981e00640 1 --2- 172.21.15.27:0/3381270694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f297c103b60 0x7f297c19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.719+0000 7f2981e00640 1 -- 172.21.15.27:0/3381270694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f297c1094a0 msgr2=0x7f297c19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.719+0000 7f2981e00640 1 --2- 172.21.15.27:0/3381270694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f297c1094a0 0x7f297c19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.719+0000 7f2981e00640 1 -- 172.21.15.27:0/3381270694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297c1a04d0 con 0x7f297c109df0 2024-09-21T00:21:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.719+0000 7f2981e00640 1 --2- 172.21.15.27:0/3381270694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f297c109df0 0x7f297c19fda0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f297402ade0 tx=0x7f2974031660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.719+0000 7f297a000640 1 -- 172.21.15.27:0/3381270694 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974003b70 con 0x7f297c109df0 2024-09-21T00:21:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.719+0000 7f2983e00640 1 -- 172.21.15.27:0/3381270694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297c1a0760 con 0x7f297c109df0 2024-09-21T00:21:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.719+0000 7f297a000640 1 -- 172.21.15.27:0/3381270694 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2974031910 con 0x7f297c109df0 2024-09-21T00:21:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.719+0000 7f297a000640 1 -- 172.21.15.27:0/3381270694 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297402f950 con 0x7f297c109df0 2024-09-21T00:21:09.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.720+0000 7f2983e00640 1 -- 172.21.15.27:0/3381270694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f297c1a6f20 con 0x7f297c109df0 2024-09-21T00:21:09.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.721+0000 7f2983e00640 1 -- 172.21.15.27:0/3381270694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f297c103f70 con 0x7f297c109df0 2024-09-21T00:21:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.723+0000 7f297a000640 1 -- 172.21.15.27:0/3381270694 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f2974004b40 con 0x7f297c109df0 2024-09-21T00:21:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.725+0000 7f297a000640 1 --2- 172.21.15.27:0/3381270694 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f29640b6560 0x7f29640b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.725+0000 7f297a000640 1 -- 172.21.15.27:0/3381270694 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f2974108a60 con 0x7f297c109df0 2024-09-21T00:21:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.725+0000 7f2981400640 1 --2- 172.21.15.27:0/3381270694 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f29640b6560 0x7f29640b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.726+0000 7f2981400640 1 --2- 172.21.15.27:0/3381270694 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f29640b6560 0x7f29640b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2970009c90 tx=0x7f2970005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.727+0000 7f297a000640 1 -- 172.21.15.27:0/3381270694 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f29740cede0 con 0x7f297c109df0 2024-09-21T00:21:09.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:09 smithi089 ceph-mon[34481]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T00:21:09.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:09 smithi098 ceph-mon[34725]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T00:21:09.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.930+0000 7f2983e00640 1 -- 172.21.15.27:0/3381270694 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f297c063800 con 0x7f29640b6560 2024-09-21T00:21:09.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.932+0000 7f297a000640 1 -- 172.21.15.27:0/3381270694 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7f297c063800 con 0x7f29640b6560 2024-09-21T00:21:09.933 INFO:teuthology.orchestra.run.smithi027.stdout:HOST ADDR LABELS STATUS 2024-09-21T00:21:09.933 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 172.21.15.27 2024-09-21T00:21:09.933 INFO:teuthology.orchestra.run.smithi027.stdout:smithi089 172.21.15.89 2024-09-21T00:21:09.933 INFO:teuthology.orchestra.run.smithi027.stdout:smithi098 172.21.15.98 2024-09-21T00:21:09.933 INFO:teuthology.orchestra.run.smithi027.stdout:3 hosts in cluster 2024-09-21T00:21:09.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.934+0000 7f2983e00640 1 -- 172.21.15.27:0/3381270694 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f29640b6560 msgr2=0x7f29640b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:09.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.934+0000 7f2983e00640 1 --2- 172.21.15.27:0/3381270694 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f29640b6560 0x7f29640b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2970009c90 tx=0x7f2970005f90 comp rx=0 tx=0).stop 2024-09-21T00:21:09.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.934+0000 7f2983e00640 1 -- 172.21.15.27:0/3381270694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f297c109df0 msgr2=0x7f297c19fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:09.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.934+0000 7f2983e00640 1 --2- 172.21.15.27:0/3381270694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f297c109df0 0x7f297c19fda0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f297402ade0 tx=0x7f2974031660 comp rx=0 tx=0).stop 2024-09-21T00:21:09.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.934+0000 7f2983e00640 1 -- 172.21.15.27:0/3381270694 shutdown_connections 2024-09-21T00:21:09.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.934+0000 7f2983e00640 1 --2- 172.21.15.27:0/3381270694 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f29640b6560 0x7f29640b8a70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:09.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.934+0000 7f2983e00640 1 --2- 172.21.15.27:0/3381270694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f297c109df0 0x7f297c19fda0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:09.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.934+0000 7f2983e00640 1 --2- 172.21.15.27:0/3381270694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f297c1094a0 0x7f297c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:09.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.934+0000 7f2983e00640 1 --2- 172.21.15.27:0/3381270694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f297c103b60 0x7f297c19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:09.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.934+0000 7f2983e00640 1 -- 172.21.15.27:0/3381270694 >> 172.21.15.27:0/3381270694 conn(0x7f297c07b850 msgr2=0x7f297c1075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:09.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.935+0000 7f2983e00640 1 -- 172.21.15.27:0/3381270694 shutdown_connections 2024-09-21T00:21:09.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:09.935+0000 7f2983e00640 1 -- 172.21.15.27:0/3381270694 wait complete. 2024-09-21T00:21:10.711 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:10 smithi027 ceph-mon[30745]: from='client.25264 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:10.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:10 smithi089 ceph-mon[34481]: from='client.25264 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:10.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:10 smithi098 ceph-mon[34725]: from='client.25264 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:10.755 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T00:21:11.142 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:11.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:11 smithi027 ceph-mon[30745]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:21:11.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:11 smithi089 ceph-mon[34481]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:21:11.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:11 smithi098 ceph-mon[34725]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:21:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.783+0000 7fbeede00640 1 -- 172.21.15.27:0/2761787139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbee8105d80 msgr2=0x7fbee8106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.783+0000 7fbeede00640 1 --2- 172.21.15.27:0/2761787139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbee8105d80 0x7fbee8106190 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009900 tx=0x7fbed802a410 comp rx=0 tx=0).stop 2024-09-21T00:21:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.783+0000 7fbeede00640 1 -- 172.21.15.27:0/2761787139 shutdown_connections 2024-09-21T00:21:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.783+0000 7fbeede00640 1 --2- 172.21.15.27:0/2761787139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbee810c010 0x7fbee8112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.783+0000 7fbeede00640 1 --2- 172.21.15.27:0/2761787139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee810b6c0 0x7fbee810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.783+0000 7fbeede00640 1 --2- 172.21.15.27:0/2761787139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbee8105d80 0x7fbee8106190 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.783+0000 7fbeede00640 1 -- 172.21.15.27:0/2761787139 >> 172.21.15.27:0/2761787139 conn(0x7fbee81017d0 msgr2=0x7fbee8103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.783+0000 7fbeede00640 1 -- 172.21.15.27:0/2761787139 shutdown_connections 2024-09-21T00:21:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.783+0000 7fbeede00640 1 -- 172.21.15.27:0/2761787139 wait complete. 2024-09-21T00:21:11.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.784+0000 7fbeede00640 1 Processor -- start 2024-09-21T00:21:11.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.784+0000 7fbeede00640 1 -- start start 2024-09-21T00:21:11.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.785+0000 7fbeede00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbee8105d80 0x7fbee819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.785+0000 7fbeede00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee810b6c0 0x7fbee819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.785+0000 7fbeede00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbee810c010 0x7fbee81a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.785+0000 7fbeede00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbee8115d20 con 0x7fbee8105d80 2024-09-21T00:21:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.785+0000 7fbeede00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbee8115ba0 con 0x7fbee810b6c0 2024-09-21T00:21:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.785+0000 7fbeede00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbee8115ea0 con 0x7fbee810c010 2024-09-21T00:21:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.785+0000 7fbee6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbee8105d80 0x7fbee819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.785+0000 7fbee6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbee8105d80 0x7fbee819c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60274/0 (socket says 172.21.15.27:60274) 2024-09-21T00:21:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.785+0000 7fbee6a00640 1 -- 172.21.15.27:0/3718106052 learned_addr learned my addr 172.21.15.27:0/3718106052 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.785+0000 7fbee7400640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbee810c010 0x7fbee81a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.785+0000 7fbedfe00640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee810b6c0 0x7fbee819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.786+0000 7fbedfe00640 1 -- 172.21.15.27:0/3718106052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbee810c010 msgr2=0x7fbee81a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.786+0000 7fbedfe00640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbee810c010 0x7fbee81a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.786+0000 7fbedfe00640 1 -- 172.21.15.27:0/3718106052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbee8105d80 msgr2=0x7fbee819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.786+0000 7fbedfe00640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbee8105d80 0x7fbee819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.786+0000 7fbedfe00640 1 -- 172.21.15.27:0/3718106052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee81a2620 con 0x7fbee810b6c0 2024-09-21T00:21:11.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.786+0000 7fbee6a00640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbee8105d80 0x7fbee819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:11.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.786+0000 7fbee7400640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbee810c010 0x7fbee81a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:11.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.786+0000 7fbedfe00640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee810b6c0 0x7fbee819cc50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbee806cac0 tx=0x7fbed000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:11.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.787+0000 7fbeeca00640 1 -- 172.21.15.27:0/3718106052 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed000cc40 con 0x7fbee810b6c0 2024-09-21T00:21:11.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.787+0000 7fbeeca00640 1 -- 172.21.15.27:0/3718106052 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed000cde0 con 0x7fbee810b6c0 2024-09-21T00:21:11.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.787+0000 7fbeede00640 1 -- 172.21.15.27:0/3718106052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee81a2910 con 0x7fbee810b6c0 2024-09-21T00:21:11.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.787+0000 7fbeeca00640 1 -- 172.21.15.27:0/3718106052 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0018780 con 0x7fbee810b6c0 2024-09-21T00:21:11.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.787+0000 7fbeede00640 1 -- 172.21.15.27:0/3718106052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee81a9070 con 0x7fbee810b6c0 2024-09-21T00:21:11.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.788+0000 7fbeede00640 1 -- 172.21.15.27:0/3718106052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee8106190 con 0x7fbee810b6c0 2024-09-21T00:21:11.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.790+0000 7fbeeca00640 1 -- 172.21.15.27:0/3718106052 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fbed0010a90 con 0x7fbee810b6c0 2024-09-21T00:21:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.792+0000 7fbeeca00640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbecc0b6560 0x7fbecc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.792+0000 7fbeeca00640 1 -- 172.21.15.27:0/3718106052 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fbed00e7fd0 con 0x7fbee810b6c0 2024-09-21T00:21:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.792+0000 7fbee6a00640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbecc0b6560 0x7fbecc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.793+0000 7fbee6a00640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbecc0b6560 0x7fbecc0b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbed80096c0 tx=0x7fbed8005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:11.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:11.794+0000 7fbeeca00640 1 -- 172.21.15.27:0/3718106052 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbed00b2730 con 0x7fbee810b6c0 2024-09-21T00:21:12.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:12.004+0000 7fbeede00640 1 -- 172.21.15.27:0/3718106052 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fbee8063800 con 0x7fbecc0b6560 2024-09-21T00:21:12.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:12.006+0000 7fbeeca00640 1 -- 172.21.15.27:0/3718106052 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fbee8063800 con 0x7fbecc0b6560 2024-09-21T00:21:12.007 INFO:teuthology.orchestra.run.smithi027.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T00:21:12.007 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001C400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-21T00:21:12.007 INFO:teuthology.orchestra.run.smithi027.stdout:smithi089 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040126400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T00:21:12.007 INFO:teuthology.orchestra.run.smithi027.stdout:smithi098 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010H400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T00:21:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:12.008+0000 7fbeede00640 1 -- 172.21.15.27:0/3718106052 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbecc0b6560 msgr2=0x7fbecc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:12.008+0000 7fbeede00640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbecc0b6560 0x7fbecc0b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbed80096c0 tx=0x7fbed8005a20 comp rx=0 tx=0).stop 2024-09-21T00:21:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:12.008+0000 7fbeede00640 1 -- 172.21.15.27:0/3718106052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee810b6c0 msgr2=0x7fbee819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:12.008+0000 7fbeede00640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee810b6c0 0x7fbee819cc50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbee806cac0 tx=0x7fbed000ee50 comp rx=0 tx=0).stop 2024-09-21T00:21:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:12.008+0000 7fbeede00640 1 -- 172.21.15.27:0/3718106052 shutdown_connections 2024-09-21T00:21:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:12.008+0000 7fbeede00640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbecc0b6560 0x7fbecc0b8a70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:12.008+0000 7fbeede00640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbee810c010 0x7fbee81a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:12.008+0000 7fbeede00640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee810b6c0 0x7fbee819cc50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:12.008+0000 7fbeede00640 1 --2- 172.21.15.27:0/3718106052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbee8105d80 0x7fbee819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:12.008+0000 7fbeede00640 1 -- 172.21.15.27:0/3718106052 >> 172.21.15.27:0/3718106052 conn(0x7fbee81017d0 msgr2=0x7fbee8103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:12.008+0000 7fbeede00640 1 -- 172.21.15.27:0/3718106052 shutdown_connections 2024-09-21T00:21:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:12.008+0000 7fbeede00640 1 -- 172.21.15.27:0/3718106052 wait complete. 2024-09-21T00:21:12.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:12 smithi089 ceph-mon[34481]: from='client.25283 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:12.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:12 smithi098 ceph-mon[34725]: from='client.25283 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:12.771 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:12 smithi027 ceph-mon[30745]: from='client.25283 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:12.810 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T00:21:12.825 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T00:21:12.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T00:21:13.211 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:13.641 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:13 smithi027 ceph-mon[30745]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:21:13.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:13 smithi089 ceph-mon[34481]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:21:13.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:13 smithi098 ceph-mon[34725]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:21:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.826+0000 7f624d800640 1 -- 172.21.15.27:0/3741427221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6248077940 msgr2=0x7f6248077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.826+0000 7f624d800640 1 --2- 172.21.15.27:0/3741427221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6248077940 0x7f6248077d50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6238009900 tx=0x7f623802a400 comp rx=0 tx=0).stop 2024-09-21T00:21:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.826+0000 7f624d800640 1 -- 172.21.15.27:0/3741427221 shutdown_connections 2024-09-21T00:21:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.826+0000 7f624d800640 1 --2- 172.21.15.27:0/3741427221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6248114740 0x7f6248116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.826+0000 7f624d800640 1 --2- 172.21.15.27:0/3741427221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6248078290 0x7f6248114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:13.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.826+0000 7f624d800640 1 --2- 172.21.15.27:0/3741427221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6248077940 0x7f6248077d50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:13.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.826+0000 7f624d800640 1 -- 172.21.15.27:0/3741427221 >> 172.21.15.27:0/3741427221 conn(0x7f6248101810 msgr2=0x7f6248103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:13.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.826+0000 7f624d800640 1 -- 172.21.15.27:0/3741427221 shutdown_connections 2024-09-21T00:21:13.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.826+0000 7f624d800640 1 -- 172.21.15.27:0/3741427221 wait complete. 2024-09-21T00:21:13.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.827+0000 7f624d800640 1 Processor -- start 2024-09-21T00:21:13.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.827+0000 7f624d800640 1 -- start start 2024-09-21T00:21:13.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f624d800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6248077940 0x7f62481a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:13.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f624d800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6248078290 0x7f62481a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:13.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f624d800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6248114740 0x7f62481a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:13.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f624d800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f624811a0d0 con 0x7f6248114740 2024-09-21T00:21:13.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f624d800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6248119f50 con 0x7f6248078290 2024-09-21T00:21:13.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f624d800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f624811a250 con 0x7f6248077940 2024-09-21T00:21:13.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f6247400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6248114740 0x7f62481a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:13.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f6246000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6248078290 0x7f62481a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f6246a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6248077940 0x7f62481a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f6247400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6248114740 0x7f62481a6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39276/0 (socket says 172.21.15.27:39276) 2024-09-21T00:21:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f6247400640 1 -- 172.21.15.27:0/815594047 learned_addr learned my addr 172.21.15.27:0/815594047 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f6246000640 1 -- 172.21.15.27:0/815594047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6248077940 msgr2=0x7f62481a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f6246000640 1 --2- 172.21.15.27:0/815594047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6248077940 0x7f62481a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f6246000640 1 -- 172.21.15.27:0/815594047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6248114740 msgr2=0x7f62481a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f6246000640 1 --2- 172.21.15.27:0/815594047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6248114740 0x7f62481a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.828+0000 7f6246000640 1 -- 172.21.15.27:0/815594047 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62481a6a50 con 0x7f6248078290 2024-09-21T00:21:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.829+0000 7f6246000640 1 --2- 172.21.15.27:0/815594047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6248078290 0x7f62481a1080 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f624806caa0 tx=0x7f623400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.829+0000 7f623f400640 1 -- 172.21.15.27:0/815594047 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623400cc40 con 0x7f6248078290 2024-09-21T00:21:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.829+0000 7f624d800640 1 -- 172.21.15.27:0/815594047 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62481a6d40 con 0x7f6248078290 2024-09-21T00:21:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.829+0000 7f623f400640 1 -- 172.21.15.27:0/815594047 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f623400cde0 con 0x7f6248078290 2024-09-21T00:21:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.829+0000 7f623f400640 1 -- 172.21.15.27:0/815594047 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234004a00 con 0x7f6248078290 2024-09-21T00:21:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.829+0000 7f624d800640 1 -- 172.21.15.27:0/815594047 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62481ad4a0 con 0x7f6248078290 2024-09-21T00:21:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.830+0000 7f624d800640 1 -- 172.21.15.27:0/815594047 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6248077d50 con 0x7f6248078290 2024-09-21T00:21:13.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.833+0000 7f623f400640 1 -- 172.21.15.27:0/815594047 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f6234004ba0 con 0x7f6248078290 2024-09-21T00:21:13.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.835+0000 7f623f400640 1 --2- 172.21.15.27:0/815594047 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f622c0b6560 0x7f622c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:13.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.835+0000 7f623f400640 1 -- 172.21.15.27:0/815594047 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f62340e7870 con 0x7f6248078290 2024-09-21T00:21:13.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.835+0000 7f6246a00640 1 --2- 172.21.15.27:0/815594047 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f622c0b6560 0x7f622c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:13.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.836+0000 7f6246a00640 1 --2- 172.21.15.27:0/815594047 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f622c0b6560 0x7f622c0b8a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6238031440 tx=0x7f6238009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:13.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:13.836+0000 7f623f400640 1 -- 172.21.15.27:0/815594047 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f62340adc80 con 0x7f6248078290 2024-09-21T00:21:14.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:14.079+0000 7f624d800640 1 -- 172.21.15.27:0/815594047 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f62481a1c10 con 0x7f6248078290 2024-09-21T00:21:14.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:14.080+0000 7f623f400640 1 -- 172.21.15.27:0/815594047 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3076 (secure 0 0 0) 0x7f62481a1c10 con 0x7f6248078290 2024-09-21T00:21:14.081 INFO:teuthology.orchestra.run.smithi027.stdout:e25 2024-09-21T00:21:14.081 INFO:teuthology.orchestra.run.smithi027.stdout:btime 2024-09-21T00:19:38:403694+0000 2024-09-21T00:21:14.081 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T00:21:14.081 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T00:21:14.082 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2024-09-21T00:21:14.082 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:21:14.082 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2024-09-21T00:21:14.082 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2024-09-21T00:21:14.082 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 25 2024-09-21T00:21:14.082 INFO:teuthology.orchestra.run.smithi027.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T00:21:14.082 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-09-21T00:18:39.809850+0000 2024-09-21T00:21:14.082 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-09-21T00:19:37.959611+0000 2024-09-21T00:21:14.082 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2024-09-21T00:21:14.082 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2024-09-21T00:21:14.082 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 300 2024-09-21T00:21:14.082 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2024-09-21T00:21:14.082 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2024-09-21T00:21:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:max_xattr_size 65536 2024-09-21T00:21:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2024-09-21T00:21:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2024-09-21T00:21:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 0 2024-09-21T00:21:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T00:21:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 3 2024-09-21T00:21:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1,2 2024-09-21T00:21:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=24761,1=24751,2=24727} 2024-09-21T00:21:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2024-09-21T00:21:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2024-09-21T00:21:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2024-09-21T00:21:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3,4] 2024-09-21T00:21:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2024-09-21T00:21:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2024-09-21T00:21:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2024-09-21T00:21:14.084 INFO:teuthology.orchestra.run.smithi027.stdout:bal_rank_mask -1 2024-09-21T00:21:14.084 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 0 2024-09-21T00:21:14.084 INFO:teuthology.orchestra.run.smithi027.stdout:qdb_cluster leader: 24761 members: 24761,24751,24727 2024-09-21T00:21:14.084 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b{0:24761} state up:active seq 5 addr [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T00:21:14.084 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.h{0:24722} state up:standby-replay seq 1 addr [v2:172.21.15.89:6836/3044996985,v1:172.21.15.89:6837/3044996985] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T00:21:14.084 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a{1:24751} state up:active seq 19 addr [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T00:21:14.084 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c{1:24715} state up:standby-replay seq 1 addr [v2:172.21.15.98:6836/3858073278,v1:172.21.15.98:6837/3858073278] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T00:21:14.084 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.i{2:24727} state up:active seq 21 addr [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T00:21:14.084 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.f{2:24703} state up:standby-replay seq 1 addr [v2:172.21.15.98:6834/388589971,v1:172.21.15.98:6835/388589971] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T00:21:14.084 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:21:14.084 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:21:14.084 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2024-09-21T00:21:14.084 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:21:14.084 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d{-1:14691} state up:standby seq 1 addr [v2:172.21.15.27:6836/1265394249,v1:172.21.15.27:6837/1265394249] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T00:21:14.084 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.g{-1:14736} state up:standby seq 1 addr [v2:172.21.15.27:6838/3523607333,v1:172.21.15.27:6839/3523607333] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T00:21:14.085 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.l{-1:24658} state up:standby seq 1 addr [v2:172.21.15.98:6832/3448953882,v1:172.21.15.98:6833/3448953882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T00:21:14.085 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.k{-1:24662} state up:standby seq 1 addr [v2:172.21.15.89:6832/191964694,v1:172.21.15.89:6833/191964694] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T00:21:14.085 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.j{-1:24674} state up:standby seq 1 addr [v2:172.21.15.27:6834/4068274538,v1:172.21.15.27:6835/4068274538] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T00:21:14.085 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.e{-1:24692} state up:standby seq 1 addr [v2:172.21.15.89:6834/811483028,v1:172.21.15.89:6835/811483028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T00:21:14.085 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-09-21T00:21:14.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:14.082+0000 7f624d800640 1 -- 172.21.15.27:0/815594047 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f622c0b6560 msgr2=0x7f622c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:14.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:14.082+0000 7f624d800640 1 --2- 172.21.15.27:0/815594047 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f622c0b6560 0x7f622c0b8a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6238031440 tx=0x7f6238009560 comp rx=0 tx=0).stop 2024-09-21T00:21:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:14.082+0000 7f624d800640 1 -- 172.21.15.27:0/815594047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6248078290 msgr2=0x7f62481a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:14.082+0000 7f624d800640 1 --2- 172.21.15.27:0/815594047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6248078290 0x7f62481a1080 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f624806caa0 tx=0x7f623400ee50 comp rx=0 tx=0).stop 2024-09-21T00:21:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:14.082+0000 7f624d800640 1 -- 172.21.15.27:0/815594047 shutdown_connections 2024-09-21T00:21:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:14.082+0000 7f624d800640 1 --2- 172.21.15.27:0/815594047 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f622c0b6560 0x7f622c0b8a70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:14.082+0000 7f624d800640 1 --2- 172.21.15.27:0/815594047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6248114740 0x7f62481a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:14.082+0000 7f624d800640 1 --2- 172.21.15.27:0/815594047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6248078290 0x7f62481a1080 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:14.082+0000 7f624d800640 1 --2- 172.21.15.27:0/815594047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6248077940 0x7f62481a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:14.082+0000 7f624d800640 1 -- 172.21.15.27:0/815594047 >> 172.21.15.27:0/815594047 conn(0x7f6248101810 msgr2=0x7f6248116800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:14.082+0000 7f624d800640 1 -- 172.21.15.27:0/815594047 shutdown_connections 2024-09-21T00:21:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:14.083+0000 7f624d800640 1 -- 172.21.15.27:0/815594047 wait complete. 2024-09-21T00:21:14.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:14 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/815594047' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T00:21:14.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:14 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/815594047' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T00:21:14.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:14 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/815594047' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T00:21:14.881 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T00:21:15.267 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:15.726 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:15 smithi027 ceph-mon[30745]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-21T00:21:15.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:15 smithi089 ceph-mon[34481]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-21T00:21:15.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:15 smithi098 ceph-mon[34725]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-21T00:21:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.913+0000 7f5112200640 1 -- 172.21.15.27:0/1572302343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c102f20 msgr2=0x7f510c10fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.913+0000 7f5112200640 1 --2- 172.21.15.27:0/1572302343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c102f20 0x7f510c10fc40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009900 tx=0x7f50f802a480 comp rx=0 tx=0).stop 2024-09-21T00:21:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.913+0000 7f5112200640 1 -- 172.21.15.27:0/1572302343 shutdown_connections 2024-09-21T00:21:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.913+0000 7f5112200640 1 --2- 172.21.15.27:0/1572302343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f510c110180 0x7f510c1125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.913+0000 7f5112200640 1 --2- 172.21.15.27:0/1572302343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c102f20 0x7f510c10fc40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.913+0000 7f5112200640 1 --2- 172.21.15.27:0/1572302343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f510c1025d0 0x7f510c1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:15.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.913+0000 7f5112200640 1 -- 172.21.15.27:0/1572302343 >> 172.21.15.27:0/1572302343 conn(0x7f510c079760 msgr2=0x7f510c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:15.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.913+0000 7f5112200640 1 -- 172.21.15.27:0/1572302343 shutdown_connections 2024-09-21T00:21:15.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.913+0000 7f5112200640 1 -- 172.21.15.27:0/1572302343 wait complete. 2024-09-21T00:21:15.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.914+0000 7f5112200640 1 Processor -- start 2024-09-21T00:21:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.914+0000 7f5112200640 1 -- start start 2024-09-21T00:21:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.915+0000 7f5112200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f510c1025d0 0x7f510c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.915+0000 7f5112200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f510c102f20 0x7f510c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.915+0000 7f5112200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c110180 0x7f510c1a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.915+0000 7f5112200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f510c115cc0 con 0x7f510c110180 2024-09-21T00:21:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.915+0000 7f5112200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f510c115b40 con 0x7f510c1025d0 2024-09-21T00:21:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.915+0000 7f5112200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f510c115e40 con 0x7f510c102f20 2024-09-21T00:21:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.915+0000 7f510be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c110180 0x7f510c1a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.915+0000 7f510be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c110180 0x7f510c1a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39292/0 (socket says 172.21.15.27:39292) 2024-09-21T00:21:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.915+0000 7f510be00640 1 -- 172.21.15.27:0/4207586191 learned_addr learned my addr 172.21.15.27:0/4207586191 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.915+0000 7f510aa00640 1 --2- 172.21.15.27:0/4207586191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f510c102f20 0x7f510c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.915+0000 7f510b400640 1 --2- 172.21.15.27:0/4207586191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f510c1025d0 0x7f510c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:15.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.915+0000 7f510be00640 1 -- 172.21.15.27:0/4207586191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f510c102f20 msgr2=0x7f510c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:15.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.915+0000 7f510be00640 1 --2- 172.21.15.27:0/4207586191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f510c102f20 0x7f510c19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:15.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.915+0000 7f510be00640 1 -- 172.21.15.27:0/4207586191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f510c1025d0 msgr2=0x7f510c19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:15.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.916+0000 7f510be00640 1 --2- 172.21.15.27:0/4207586191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f510c1025d0 0x7f510c19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:15.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.916+0000 7f510be00640 1 -- 172.21.15.27:0/4207586191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c1a2660 con 0x7f510c110180 2024-09-21T00:21:15.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.916+0000 7f510aa00640 1 --2- 172.21.15.27:0/4207586191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f510c102f20 0x7f510c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:15.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.916+0000 7f510be00640 1 --2- 172.21.15.27:0/4207586191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c110180 0x7f510c1a1f30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f510400eb10 tx=0x7f510400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:15.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.916+0000 7f5103e00640 1 -- 172.21.15.27:0/4207586191 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104004b20 con 0x7f510c110180 2024-09-21T00:21:15.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.916+0000 7f5103e00640 1 -- 172.21.15.27:0/4207586191 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5104004cc0 con 0x7f510c110180 2024-09-21T00:21:15.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.916+0000 7f5103e00640 1 -- 172.21.15.27:0/4207586191 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104010670 con 0x7f510c110180 2024-09-21T00:21:15.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.916+0000 7f5112200640 1 -- 172.21.15.27:0/4207586191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510c1a2950 con 0x7f510c110180 2024-09-21T00:21:15.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.916+0000 7f5112200640 1 -- 172.21.15.27:0/4207586191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510c1a90b0 con 0x7f510c110180 2024-09-21T00:21:15.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.917+0000 7f5112200640 1 -- 172.21.15.27:0/4207586191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510c06c2c0 con 0x7f510c110180 2024-09-21T00:21:15.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.920+0000 7f5103e00640 1 -- 172.21.15.27:0/4207586191 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f5104005000 con 0x7f510c110180 2024-09-21T00:21:15.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.922+0000 7f5103e00640 1 --2- 172.21.15.27:0/4207586191 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f50f00b65e0 0x7f50f00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:15.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.922+0000 7f5103e00640 1 -- 172.21.15.27:0/4207586191 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f51040e7fd0 con 0x7f510c110180 2024-09-21T00:21:15.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.922+0000 7f510b400640 1 --2- 172.21.15.27:0/4207586191 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f50f00b65e0 0x7f50f00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:15.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.923+0000 7f510b400640 1 --2- 172.21.15.27:0/4207586191 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f50f00b65e0 0x7f50f00b8af0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009bd0 tx=0x7f50fc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:15.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:15.924+0000 7f5103e00640 1 -- 172.21.15.27:0/4207586191 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f51040ae3e0 con 0x7f510c110180 2024-09-21T00:21:16.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:16.124+0000 7f5112200640 1 -- 172.21.15.27:0/4207586191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f510c1029e0 con 0x7f510c110180 2024-09-21T00:21:16.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:16.125+0000 7f5103e00640 1 -- 172.21.15.27:0/4207586191 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+6055 (secure 0 0 0) 0x7f510c1029e0 con 0x7f510c110180 2024-09-21T00:21:16.126 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 90 2024-09-21T00:21:16.126 INFO:teuthology.orchestra.run.smithi027.stdout:fsid 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:21:16.126 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-09-21T00:09:20.149938+0000 2024-09-21T00:21:16.126 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-09-21T00:20:26.442227+0000 2024-09-21T00:21:16.126 INFO:teuthology.orchestra.run.smithi027.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T00:21:16.127 INFO:teuthology.orchestra.run.smithi027.stdout:crush_version 27 2024-09-21T00:21:16.127 INFO:teuthology.orchestra.run.smithi027.stdout:full_ratio 0.95 2024-09-21T00:21:16.127 INFO:teuthology.orchestra.run.smithi027.stdout:backfillfull_ratio 0.9 2024-09-21T00:21:16.127 INFO:teuthology.orchestra.run.smithi027.stdout:nearfull_ratio 0.85 2024-09-21T00:21:16.127 INFO:teuthology.orchestra.run.smithi027.stdout:require_min_compat_client luminous 2024-09-21T00:21:16.127 INFO:teuthology.orchestra.run.smithi027.stdout:min_compat_client luminous 2024-09-21T00:21:16.127 INFO:teuthology.orchestra.run.smithi027.stdout:require_osd_release squid 2024-09-21T00:21:16.127 INFO:teuthology.orchestra.run.smithi027.stdout:stretch_mode_enabled false 2024-09-21T00:21:16.127 INFO:teuthology.orchestra.run.smithi027.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T00:21:16.127 INFO:teuthology.orchestra.run.smithi027.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T00:21:16.127 INFO:teuthology.orchestra.run.smithi027.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-21T00:21:16.127 INFO:teuthology.orchestra.run.smithi027.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-21T00:21:16.127 INFO:teuthology.orchestra.run.smithi027.stdout:max_osd 12 2024-09-21T00:21:16.127 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 up in weight 1 up_from 9 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6802/354370090,v1:172.21.15.27:6803/354370090] [v2:172.21.15.27:6804/354370090,v1:172.21.15.27:6805/354370090] exists,up 63ccc094-62d5-446f-844d-1c60d41fda1c 2024-09-21T00:21:16.128 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 up in weight 1 up_from 14 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6800/922528621,v1:172.21.15.89:6801/922528621] [v2:172.21.15.89:6802/922528621,v1:172.21.15.89:6803/922528621] exists,up e59a8ed4-3a37-4477-a60a-4abc33d49b4e 2024-09-21T00:21:16.128 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 up in weight 1 up_from 19 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6800/257636259,v1:172.21.15.98:6801/257636259] [v2:172.21.15.98:6802/257636259,v1:172.21.15.98:6803/257636259] exists,up 6058a875-d5b2-4459-9407-2fd190cb34a9 2024-09-21T00:21:16.128 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 up in weight 1 up_from 26 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6810/821088824,v1:172.21.15.27:6811/821088824] [v2:172.21.15.27:6812/821088824,v1:172.21.15.27:6813/821088824] exists,up dfcdaede-e51c-4985-92af-3e296435a36c 2024-09-21T00:21:16.128 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 up in weight 1 up_from 32 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6808/4001038690,v1:172.21.15.89:6809/4001038690] [v2:172.21.15.89:6810/4001038690,v1:172.21.15.89:6811/4001038690] exists,up 7c4af037-8ab6-47c0-b62b-76bd775b3491 2024-09-21T00:21:16.128 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 up in weight 1 up_from 37 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6808/3029903562,v1:172.21.15.98:6809/3029903562] [v2:172.21.15.98:6810/3029903562,v1:172.21.15.98:6811/3029903562] exists,up e996ed0d-88ad-4aa7-891f-dfade4e09cb0 2024-09-21T00:21:16.128 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 up in weight 1 up_from 43 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6818/1286808857,v1:172.21.15.27:6819/1286808857] [v2:172.21.15.27:6820/1286808857,v1:172.21.15.27:6821/1286808857] exists,up cf080497-57c2-42d0-8c8f-1eed0110702d 2024-09-21T00:21:16.128 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 up in weight 1 up_from 48 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6816/2479349546,v1:172.21.15.89:6817/2479349546] [v2:172.21.15.89:6818/2479349546,v1:172.21.15.89:6819/2479349546] exists,up ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6 2024-09-21T00:21:16.128 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 up in weight 1 up_from 54 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6816/2830381485,v1:172.21.15.98:6817/2830381485] [v2:172.21.15.98:6818/2830381485,v1:172.21.15.98:6819/2830381485] exists,up 0b558458-fc2d-4ed1-b72a-e2ea8cf811b0 2024-09-21T00:21:16.128 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 up in weight 1 up_from 60 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6826/3033977113,v1:172.21.15.27:6827/3033977113] [v2:172.21.15.27:6828/3033977113,v1:172.21.15.27:6829/3033977113] exists,up 1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b 2024-09-21T00:21:16.128 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 up in weight 1 up_from 65 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6824/2370332225,v1:172.21.15.89:6825/2370332225] [v2:172.21.15.89:6826/2370332225,v1:172.21.15.89:6827/2370332225] exists,up 324dd6ce-68fe-4c76-959b-1c9fb0ae92a7 2024-09-21T00:21:16.128 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 up in weight 1 up_from 71 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6824/2795999064,v1:172.21.15.98:6825/2795999064] [v2:172.21.15.98:6826/2795999064,v1:172.21.15.98:6827/2795999064] exists,up 5e34686e-350a-491e-b9d7-59174ea90944 2024-09-21T00:21:16.128 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.a [2,11] 2024-09-21T00:21:16.128 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.c [2,4] 2024-09-21T00:21:16.129 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.f [8,11] 2024-09-21T00:21:16.129 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.14 [2,4] 2024-09-21T00:21:16.129 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.1a [8,1] 2024-09-21T00:21:16.129 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.6 [3,7] 2024-09-21T00:21:16.129 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.14 [4,6] 2024-09-21T00:21:16.129 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.1d [4,8] 2024-09-21T00:21:16.129 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.25 [5,9] 2024-09-21T00:21:16.129 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.37 [1,9] 2024-09-21T00:21:16.129 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.3c [4,9] 2024-09-21T00:21:16.129 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.b [9,8] 2024-09-21T00:21:16.129 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.1b [9,8,5,10] 2024-09-21T00:21:16.129 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.1f [9,10] 2024-09-21T00:21:16.129 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/502883064 expires 2024-09-22T00:11:24.231101+0000 2024-09-21T00:21:16.129 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/831048090 expires 2024-09-22T00:11:24.231101+0000 2024-09-21T00:21:16.129 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/3880378905 expires 2024-09-22T00:09:46.276169+0000 2024-09-21T00:21:16.130 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/793991452 expires 2024-09-22T00:09:46.276169+0000 2024-09-21T00:21:16.130 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/2077956479 expires 2024-09-22T00:09:46.276169+0000 2024-09-21T00:21:16.130 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/3752372220 expires 2024-09-22T00:09:46.276169+0000 2024-09-21T00:21:16.130 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/3752372220 expires 2024-09-22T00:09:46.276169+0000 2024-09-21T00:21:16.130 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/1098432608 expires 2024-09-22T00:10:16.413167+0000 2024-09-21T00:21:16.130 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/2344239267 expires 2024-09-22T00:10:16.413167+0000 2024-09-21T00:21:16.130 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/2344239267 expires 2024-09-22T00:10:16.413167+0000 2024-09-21T00:21:16.130 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/528019907 expires 2024-09-22T00:10:16.413167+0000 2024-09-21T00:21:16.130 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/2173601508 expires 2024-09-22T00:10:16.413167+0000 2024-09-21T00:21:16.130 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/827586431 expires 2024-09-22T00:11:24.231101+0000 2024-09-21T00:21:16.130 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/827586431 expires 2024-09-22T00:11:24.231101+0000 2024-09-21T00:21:16.130 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/764776233 expires 2024-09-22T00:11:24.231101+0000 2024-09-21T00:21:16.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:16.127+0000 7f5112200640 1 -- 172.21.15.27:0/4207586191 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f50f00b65e0 msgr2=0x7f50f00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:16.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:16.127+0000 7f5112200640 1 --2- 172.21.15.27:0/4207586191 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f50f00b65e0 0x7f50f00b8af0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009bd0 tx=0x7f50fc005f90 comp rx=0 tx=0).stop 2024-09-21T00:21:16.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:16.127+0000 7f5112200640 1 -- 172.21.15.27:0/4207586191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c110180 msgr2=0x7f510c1a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:16.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:16.127+0000 7f5112200640 1 --2- 172.21.15.27:0/4207586191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c110180 0x7f510c1a1f30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f510400eb10 tx=0x7f510400c2d0 comp rx=0 tx=0).stop 2024-09-21T00:21:16.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:16.127+0000 7f5112200640 1 -- 172.21.15.27:0/4207586191 shutdown_connections 2024-09-21T00:21:16.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:16.127+0000 7f5112200640 1 --2- 172.21.15.27:0/4207586191 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f50f00b65e0 0x7f50f00b8af0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:16.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:16.127+0000 7f5112200640 1 --2- 172.21.15.27:0/4207586191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c110180 0x7f510c1a1f30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:16.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:16.127+0000 7f5112200640 1 --2- 172.21.15.27:0/4207586191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f510c102f20 0x7f510c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:16.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:16.127+0000 7f5112200640 1 --2- 172.21.15.27:0/4207586191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f510c1025d0 0x7f510c19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:16.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:16.127+0000 7f5112200640 1 -- 172.21.15.27:0/4207586191 >> 172.21.15.27:0/4207586191 conn(0x7f510c079760 msgr2=0x7f510c109d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:16.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:16.127+0000 7f5112200640 1 -- 172.21.15.27:0/4207586191 shutdown_connections 2024-09-21T00:21:16.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:16.127+0000 7f5112200640 1 -- 172.21.15.27:0/4207586191 wait complete. 2024-09-21T00:21:16.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:16 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/4207586191' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T00:21:16.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:16 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/4207586191' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T00:21:16.906 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T00:21:16.917 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:21:16.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:16 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/4207586191' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T00:21:17.302 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:17.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:17 smithi098 ceph-mon[34725]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-21T00:21:17.769 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:17 smithi027 ceph-mon[30745]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-21T00:21:17.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.951+0000 7fe773e00640 1 -- 172.21.15.27:0/2823790180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410be50 msgr2=0x7fe774112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:17.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.951+0000 7fe773e00640 1 --2- 172.21.15.27:0/2823790180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410be50 0x7fe774112680 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009ca0 tx=0x7fe76c02a8c0 comp rx=0 tx=0).stop 2024-09-21T00:21:17.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.951+0000 7fe773e00640 1 -- 172.21.15.27:0/2823790180 shutdown_connections 2024-09-21T00:21:17.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.951+0000 7fe773e00640 1 --2- 172.21.15.27:0/2823790180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410be50 0x7fe774112680 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:17.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.951+0000 7fe773e00640 1 --2- 172.21.15.27:0/2823790180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe77410b500 0x7fe77410b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:17.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.951+0000 7fe773e00640 1 --2- 172.21.15.27:0/2823790180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe774105bc0 0x7fe774105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:17.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.951+0000 7fe773e00640 1 -- 172.21.15.27:0/2823790180 >> 172.21.15.27:0/2823790180 conn(0x7fe774101670 msgr2=0x7fe774103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:17.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.952+0000 7fe773e00640 1 -- 172.21.15.27:0/2823790180 shutdown_connections 2024-09-21T00:21:17.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.952+0000 7fe773e00640 1 -- 172.21.15.27:0/2823790180 wait complete. 2024-09-21T00:21:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.953+0000 7fe773e00640 1 Processor -- start 2024-09-21T00:21:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.953+0000 7fe773e00640 1 -- start start 2024-09-21T00:21:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.953+0000 7fe773e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe774105bc0 0x7fe77419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.953+0000 7fe773e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410b500 0x7fe77419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.953+0000 7fe773e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe77410be50 0x7fe7741a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.953+0000 7fe773e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe774115b60 con 0x7fe77410b500 2024-09-21T00:21:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.953+0000 7fe773e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe7741159e0 con 0x7fe774105bc0 2024-09-21T00:21:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.953+0000 7fe773e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe774115ce0 con 0x7fe77410be50 2024-09-21T00:21:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.954+0000 7fe76a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410b500 0x7fe77419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.954+0000 7fe772a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe774105bc0 0x7fe77419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.954+0000 7fe76a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410b500 0x7fe77419ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39304/0 (socket says 172.21.15.27:39304) 2024-09-21T00:21:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.954+0000 7fe76a000640 1 -- 172.21.15.27:0/1080731693 learned_addr learned my addr 172.21.15.27:0/1080731693 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.954+0000 7fe773400640 1 --2- 172.21.15.27:0/1080731693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe77410be50 0x7fe7741a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.954+0000 7fe772a00640 1 -- 172.21.15.27:0/1080731693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe77410be50 msgr2=0x7fe7741a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.954+0000 7fe772a00640 1 --2- 172.21.15.27:0/1080731693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe77410be50 0x7fe7741a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.954+0000 7fe772a00640 1 -- 172.21.15.27:0/1080731693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410b500 msgr2=0x7fe77419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.954+0000 7fe772a00640 1 --2- 172.21.15.27:0/1080731693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410b500 0x7fe77419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.954+0000 7fe772a00640 1 -- 172.21.15.27:0/1080731693 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7741a2460 con 0x7fe774105bc0 2024-09-21T00:21:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.954+0000 7fe76a000640 1 --2- 172.21.15.27:0/1080731693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410b500 0x7fe77419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.955+0000 7fe772a00640 1 --2- 172.21.15.27:0/1080731693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe774105bc0 0x7fe77419c4f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e9f0 tx=0x7fe76400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.955+0000 7fe778a00640 1 -- 172.21.15.27:0/1080731693 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76400ccc0 con 0x7fe774105bc0 2024-09-21T00:21:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.955+0000 7fe773e00640 1 -- 172.21.15.27:0/1080731693 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7741a2750 con 0x7fe774105bc0 2024-09-21T00:21:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.955+0000 7fe778a00640 1 -- 172.21.15.27:0/1080731693 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7640108e0 con 0x7fe774105bc0 2024-09-21T00:21:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.955+0000 7fe778a00640 1 -- 172.21.15.27:0/1080731693 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764018880 con 0x7fe774105bc0 2024-09-21T00:21:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.955+0000 7fe773e00640 1 -- 172.21.15.27:0/1080731693 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7741a8eb0 con 0x7fe774105bc0 2024-09-21T00:21:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.956+0000 7fe773e00640 1 -- 172.21.15.27:0/1080731693 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe774105fd0 con 0x7fe774105bc0 2024-09-21T00:21:17.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.959+0000 7fe778a00640 1 -- 172.21.15.27:0/1080731693 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fe764010430 con 0x7fe774105bc0 2024-09-21T00:21:17.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.960+0000 7fe778a00640 1 --2- 172.21.15.27:0/1080731693 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7580b6560 0x7fe7580b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:17.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.960+0000 7fe778a00640 1 -- 172.21.15.27:0/1080731693 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fe7640e7fe0 con 0x7fe774105bc0 2024-09-21T00:21:17.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.961+0000 7fe76a000640 1 --2- 172.21.15.27:0/1080731693 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7580b6560 0x7fe7580b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:17.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.961+0000 7fe76a000640 1 --2- 172.21.15.27:0/1080731693 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7580b6560 0x7fe7580b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe774104ec0 tx=0x7fe75c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:17.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:17.962+0000 7fe778a00640 1 -- 172.21.15.27:0/1080731693 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe7640b27a0 con 0x7fe774105bc0 2024-09-21T00:21:17.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:17 smithi089 ceph-mon[34481]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-21T00:21:18.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:18.206+0000 7fe773e00640 1 -- 172.21.15.27:0/1080731693 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe77419d620 con 0x7fe774105bc0 2024-09-21T00:21:18.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:18.207+0000 7fe778a00640 1 -- 172.21.15.27:0/1080731693 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11893 (secure 0 0 0) 0x7fe7640b76e0 con 0x7fe774105bc0 2024-09-21T00:21:18.208 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:21:18.209 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"btime":"2024-09-21T00:19:38:403694+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:19:37.959611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24751,24727]},"id":1}]} 2024-09-21T00:21:18.209 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-09-21T00:21:18.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:18.210+0000 7fe773e00640 1 -- 172.21.15.27:0/1080731693 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7580b6560 msgr2=0x7fe7580b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:18.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:18.210+0000 7fe773e00640 1 --2- 172.21.15.27:0/1080731693 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7580b6560 0x7fe7580b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe774104ec0 tx=0x7fe75c005f90 comp rx=0 tx=0).stop 2024-09-21T00:21:18.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:18.210+0000 7fe773e00640 1 -- 172.21.15.27:0/1080731693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe774105bc0 msgr2=0x7fe77419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:18.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:18.210+0000 7fe773e00640 1 --2- 172.21.15.27:0/1080731693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe774105bc0 0x7fe77419c4f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e9f0 tx=0x7fe76400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:21:18.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:18.210+0000 7fe773e00640 1 -- 172.21.15.27:0/1080731693 shutdown_connections 2024-09-21T00:21:18.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:18.210+0000 7fe773e00640 1 --2- 172.21.15.27:0/1080731693 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe7580b6560 0x7fe7580b8a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:18.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:18.210+0000 7fe773e00640 1 --2- 172.21.15.27:0/1080731693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe77410be50 0x7fe7741a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:18.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:18.210+0000 7fe773e00640 1 --2- 172.21.15.27:0/1080731693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe77410b500 0x7fe77419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:18.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:18.210+0000 7fe773e00640 1 --2- 172.21.15.27:0/1080731693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe774105bc0 0x7fe77419c4f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:18.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:18.210+0000 7fe773e00640 1 -- 172.21.15.27:0/1080731693 >> 172.21.15.27:0/1080731693 conn(0x7fe774101670 msgr2=0x7fe77410ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:18.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:18.210+0000 7fe773e00640 1 -- 172.21.15.27:0/1080731693 shutdown_connections 2024-09-21T00:21:18.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:18.210+0000 7fe773e00640 1 -- 172.21.15.27:0/1080731693 wait complete. 2024-09-21T00:21:18.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:18 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1080731693' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:21:18.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:18 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1080731693' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:21:18.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:18 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1080731693' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:21:19.083 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:21:19.493 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:19.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:19 smithi098 ceph-mon[34725]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-21T00:21:19.804 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:19 smithi027 ceph-mon[30745]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-21T00:21:19.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:19 smithi089 ceph-mon[34481]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-21T00:21:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.154+0000 7fbd1fe00640 1 -- 172.21.15.27:0/3676181566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd201039b0 msgr2=0x7fbd20103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.154+0000 7fbd1fe00640 1 --2- 172.21.15.27:0/3676181566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd201039b0 0x7fbd20103dc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009900 tx=0x7fbd1002a410 comp rx=0 tx=0).stop 2024-09-21T00:21:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.154+0000 7fbd1fe00640 1 -- 172.21.15.27:0/3676181566 shutdown_connections 2024-09-21T00:21:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.154+0000 7fbd1fe00640 1 --2- 172.21.15.27:0/3676181566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd20109c40 0x7fbd20110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.154+0000 7fbd1fe00640 1 --2- 172.21.15.27:0/3676181566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd201092f0 0x7fbd20109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.154+0000 7fbd1fe00640 1 --2- 172.21.15.27:0/3676181566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd201039b0 0x7fbd20103dc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:20.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.154+0000 7fbd1fe00640 1 -- 172.21.15.27:0/3676181566 >> 172.21.15.27:0/3676181566 conn(0x7fbd200ff440 msgr2=0x7fbd20101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:20.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.154+0000 7fbd1fe00640 1 -- 172.21.15.27:0/3676181566 shutdown_connections 2024-09-21T00:21:20.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.154+0000 7fbd1fe00640 1 -- 172.21.15.27:0/3676181566 wait complete. 2024-09-21T00:21:20.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.155+0000 7fbd1fe00640 1 Processor -- start 2024-09-21T00:21:20.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.156+0000 7fbd1fe00640 1 -- start start 2024-09-21T00:21:20.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.156+0000 7fbd1fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd201039b0 0x7fbd20075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:20.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.156+0000 7fbd1fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd201092f0 0x7fbd20075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:20.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.156+0000 7fbd1fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd20109c40 0x7fbd20076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:20.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.156+0000 7fbd1fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd20113900 con 0x7fbd201092f0 2024-09-21T00:21:20.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.156+0000 7fbd1fe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbd20113780 con 0x7fbd201039b0 2024-09-21T00:21:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.156+0000 7fbd1fe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbd20113a80 con 0x7fbd20109c40 2024-09-21T00:21:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.156+0000 7fbd1e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd201092f0 0x7fbd20075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.156+0000 7fbd1ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd201039b0 0x7fbd20075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.157+0000 7fbd1ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd201039b0 0x7fbd20075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45704/0 (socket says 172.21.15.27:45704) 2024-09-21T00:21:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.157+0000 7fbd1e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd201092f0 0x7fbd20075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39318/0 (socket says 172.21.15.27:39318) 2024-09-21T00:21:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.157+0000 7fbd1ea00640 1 -- 172.21.15.27:0/1197453935 learned_addr learned my addr 172.21.15.27:0/1197453935 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.157+0000 7fbd1f400640 1 --2- 172.21.15.27:0/1197453935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd20109c40 0x7fbd20076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.157+0000 7fbd1e000640 1 -- 172.21.15.27:0/1197453935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd20109c40 msgr2=0x7fbd20076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.157+0000 7fbd1e000640 1 --2- 172.21.15.27:0/1197453935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd20109c40 0x7fbd20076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.157+0000 7fbd1e000640 1 -- 172.21.15.27:0/1197453935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd201039b0 msgr2=0x7fbd20075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.157+0000 7fbd1e000640 1 --2- 172.21.15.27:0/1197453935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd201039b0 0x7fbd20075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:20.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.157+0000 7fbd1e000640 1 -- 172.21.15.27:0/1197453935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd201a85b0 con 0x7fbd201092f0 2024-09-21T00:21:20.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.158+0000 7fbd1e000640 1 --2- 172.21.15.27:0/1197453935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd201092f0 0x7fbd20075c50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c005fd0 tx=0x7fbd0c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:20.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.158+0000 7fbd17400640 1 -- 172.21.15.27:0/1197453935 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c009c00 con 0x7fbd201092f0 2024-09-21T00:21:20.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.158+0000 7fbd17400640 1 -- 172.21.15.27:0/1197453935 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd0c009da0 con 0x7fbd201092f0 2024-09-21T00:21:20.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.158+0000 7fbd17400640 1 -- 172.21.15.27:0/1197453935 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c003e10 con 0x7fbd201092f0 2024-09-21T00:21:20.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.158+0000 7fbd1fe00640 1 -- 172.21.15.27:0/1197453935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd201a88a0 con 0x7fbd201092f0 2024-09-21T00:21:20.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.158+0000 7fbd1fe00640 1 -- 172.21.15.27:0/1197453935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd201a8d90 con 0x7fbd201092f0 2024-09-21T00:21:20.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.159+0000 7fbd1fe00640 1 -- 172.21.15.27:0/1197453935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd20103dc0 con 0x7fbd201092f0 2024-09-21T00:21:20.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.162+0000 7fbd17400640 1 -- 172.21.15.27:0/1197453935 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fbd0c004000 con 0x7fbd201092f0 2024-09-21T00:21:20.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.164+0000 7fbd17400640 1 --2- 172.21.15.27:0/1197453935 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbd040b6560 0x7fbd040b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:20.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.164+0000 7fbd17400640 1 -- 172.21.15.27:0/1197453935 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fbd0c0e7120 con 0x7fbd201092f0 2024-09-21T00:21:20.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.164+0000 7fbd1ea00640 1 --2- 172.21.15.27:0/1197453935 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbd040b6560 0x7fbd040b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:20.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.165+0000 7fbd1ea00640 1 --2- 172.21.15.27:0/1197453935 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbd040b6560 0x7fbd040b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009230 tx=0x7fbd100058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:20.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.165+0000 7fbd17400640 1 -- 172.21.15.27:0/1197453935 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbd0c017020 con 0x7fbd201092f0 2024-09-21T00:21:20.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.408+0000 7fbd1fe00640 1 -- 172.21.15.27:0/1197453935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd2002e640 con 0x7fbd201092f0 2024-09-21T00:21:20.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.409+0000 7fbd17400640 1 -- 172.21.15.27:0/1197453935 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11893 (secure 0 0 0) 0x7fbd0c0b1880 con 0x7fbd201092f0 2024-09-21T00:21:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:21:20.412 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"btime":"2024-09-21T00:19:38:403694+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:19:37.959611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24751,24727]},"id":1}]} 2024-09-21T00:21:20.412 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-09-21T00:21:20.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.412+0000 7fbd1fe00640 1 -- 172.21.15.27:0/1197453935 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbd040b6560 msgr2=0x7fbd040b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:20.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.412+0000 7fbd1fe00640 1 --2- 172.21.15.27:0/1197453935 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbd040b6560 0x7fbd040b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009230 tx=0x7fbd100058d0 comp rx=0 tx=0).stop 2024-09-21T00:21:20.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.412+0000 7fbd1fe00640 1 -- 172.21.15.27:0/1197453935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd201092f0 msgr2=0x7fbd20075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:20.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.412+0000 7fbd1fe00640 1 --2- 172.21.15.27:0/1197453935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd201092f0 0x7fbd20075c50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c005fd0 tx=0x7fbd0c00dde0 comp rx=0 tx=0).stop 2024-09-21T00:21:20.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.413+0000 7fbd1fe00640 1 -- 172.21.15.27:0/1197453935 shutdown_connections 2024-09-21T00:21:20.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.413+0000 7fbd1fe00640 1 --2- 172.21.15.27:0/1197453935 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbd040b6560 0x7fbd040b8a70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:20.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.413+0000 7fbd1fe00640 1 --2- 172.21.15.27:0/1197453935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd20109c40 0x7fbd20076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:20.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.413+0000 7fbd1fe00640 1 --2- 172.21.15.27:0/1197453935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd201092f0 0x7fbd20075c50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:20.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.413+0000 7fbd1fe00640 1 --2- 172.21.15.27:0/1197453935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd201039b0 0x7fbd20075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:20.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.413+0000 7fbd1fe00640 1 -- 172.21.15.27:0/1197453935 >> 172.21.15.27:0/1197453935 conn(0x7fbd200ff440 msgr2=0x7fbd20101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:20.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.413+0000 7fbd1fe00640 1 -- 172.21.15.27:0/1197453935 shutdown_connections 2024-09-21T00:21:20.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:20.413+0000 7fbd1fe00640 1 -- 172.21.15.27:0/1197453935 wait complete. 2024-09-21T00:21:20.776 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:20 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1197453935' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:21:20.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:20 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1197453935' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:21:20.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:20 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1197453935' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:21:21.334 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T00:21:21.721 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:22.189 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:22 smithi027 ceph-mon[30745]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-21T00:21:22.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.371+0000 7f1bbdc00640 1 -- 172.21.15.27:0/3873164609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bb81092f0 msgr2=0x7f1bb8109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:22.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.371+0000 7f1bbdc00640 1 --2- 172.21.15.27:0/3873164609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bb81092f0 0x7f1bb8109700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009900 tx=0x7f1ba002a410 comp rx=0 tx=0).stop 2024-09-21T00:21:22.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.371+0000 7f1bbdc00640 1 -- 172.21.15.27:0/3873164609 shutdown_connections 2024-09-21T00:21:22.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.371+0000 7f1bbdc00640 1 --2- 172.21.15.27:0/3873164609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bb8109c40 0x7f1bb8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:22.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.371+0000 7f1bbdc00640 1 --2- 172.21.15.27:0/3873164609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bb81092f0 0x7f1bb8109700 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:22.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.371+0000 7f1bbdc00640 1 --2- 172.21.15.27:0/3873164609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bb81039b0 0x7f1bb8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:22.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.371+0000 7f1bbdc00640 1 -- 172.21.15.27:0/3873164609 >> 172.21.15.27:0/3873164609 conn(0x7f1bb80ff440 msgr2=0x7f1bb8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:22.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.371+0000 7f1bbdc00640 1 -- 172.21.15.27:0/3873164609 shutdown_connections 2024-09-21T00:21:22.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.371+0000 7f1bbdc00640 1 -- 172.21.15.27:0/3873164609 wait complete. 2024-09-21T00:21:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.372+0000 7f1bbdc00640 1 Processor -- start 2024-09-21T00:21:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.372+0000 7f1bbdc00640 1 -- start start 2024-09-21T00:21:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.373+0000 7f1bbdc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bb81039b0 0x7f1bb8077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.373+0000 7f1bbdc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bb81092f0 0x7f1bb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.373+0000 7f1bbdc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bb8109c40 0x7f1bb8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.373+0000 7f1bbdc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb8113900 con 0x7f1bb81092f0 2024-09-21T00:21:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.373+0000 7f1bbdc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb8113780 con 0x7f1bb81039b0 2024-09-21T00:21:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.373+0000 7f1bbdc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb8113a80 con 0x7f1bb8109c40 2024-09-21T00:21:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.373+0000 7f1bb7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bb81039b0 0x7f1bb8077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.373+0000 7f1bafe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bb81092f0 0x7f1bb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:22.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.373+0000 7f1bb7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bb81039b0 0x7f1bb8077120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45716/0 (socket says 172.21.15.27:45716) 2024-09-21T00:21:22.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.373+0000 7f1bb7e00640 1 -- 172.21.15.27:0/2214117758 learned_addr learned my addr 172.21.15.27:0/2214117758 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:22.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.373+0000 7f1bbd200640 1 --2- 172.21.15.27:0/2214117758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bb8109c40 0x7f1bb8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:22.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.373+0000 7f1bb7e00640 1 -- 172.21.15.27:0/2214117758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bb8109c40 msgr2=0x7f1bb8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:22.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.373+0000 7f1bb7e00640 1 --2- 172.21.15.27:0/2214117758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bb8109c40 0x7f1bb8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.374+0000 7f1bb7e00640 1 -- 172.21.15.27:0/2214117758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bb81092f0 msgr2=0x7f1bb8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.374+0000 7f1bb7e00640 1 --2- 172.21.15.27:0/2214117758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bb81092f0 0x7f1bb8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.374+0000 7f1bb7e00640 1 -- 172.21.15.27:0/2214117758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb8076470 con 0x7f1bb81039b0 2024-09-21T00:21:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.374+0000 7f1bb7e00640 1 --2- 172.21.15.27:0/2214117758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bb81039b0 0x7f1bb8077120 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800ea60 tx=0x7f1ba800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.374+0000 7f1bb5600640 1 -- 172.21.15.27:0/2214117758 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800cde0 con 0x7f1bb81039b0 2024-09-21T00:21:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.374+0000 7f1bbdc00640 1 -- 172.21.15.27:0/2214117758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb81acd50 con 0x7f1bb81039b0 2024-09-21T00:21:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.374+0000 7f1bb5600640 1 -- 172.21.15.27:0/2214117758 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ba80035f0 con 0x7f1bb81039b0 2024-09-21T00:21:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.374+0000 7f1bb5600640 1 -- 172.21.15.27:0/2214117758 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8010690 con 0x7f1bb81039b0 2024-09-21T00:21:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.375+0000 7f1bbdc00640 1 -- 172.21.15.27:0/2214117758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb81ad200 con 0x7f1bb81039b0 2024-09-21T00:21:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.376+0000 7f1bbdc00640 1 -- 172.21.15.27:0/2214117758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb8103dc0 con 0x7f1bb81039b0 2024-09-21T00:21:22.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.378+0000 7f1bb5600640 1 -- 172.21.15.27:0/2214117758 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f1ba80037a0 con 0x7f1bb81039b0 2024-09-21T00:21:22.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.380+0000 7f1bb5600640 1 --2- 172.21.15.27:0/2214117758 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1b9c0b6560 0x7f1b9c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:22.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.380+0000 7f1bb5600640 1 -- 172.21.15.27:0/2214117758 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f1ba80e7f50 con 0x7f1bb81039b0 2024-09-21T00:21:22.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.380+0000 7f1bafe00640 1 --2- 172.21.15.27:0/2214117758 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1b9c0b6560 0x7f1b9c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:22.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.381+0000 7f1bafe00640 1 --2- 172.21.15.27:0/2214117758 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1b9c0b6560 0x7f1b9c0b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009230 tx=0x7f1ba0031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:22.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.381+0000 7f1bb5600640 1 -- 172.21.15.27:0/2214117758 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1ba80ae2d0 con 0x7f1bb81039b0 2024-09-21T00:21:22.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:22 smithi089 ceph-mon[34481]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-21T00:21:22.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:22 smithi098 ceph-mon[34725]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-21T00:21:22.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.581+0000 7f1bbdc00640 1 -- 172.21.15.27:0/2214117758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bb802e6d0 con 0x7f1bb81039b0 2024-09-21T00:21:22.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.583+0000 7f1bb5600640 1 -- 172.21.15.27:0/2214117758 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25160 (secure 0 0 0) 0x7f1ba80b26b0 con 0x7f1bb81039b0 2024-09-21T00:21:22.586 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:21:22.586 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","created":"2024-09-21T00:09:20.149938+0000","modified":"2024-09-21T00:20:26.442227+0000","last_up_change":"2024-09-21T00:17:43.978785+0000","last_in_change":"2024-09-21T00:17:21.325177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:13:32.718543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:18:33.992414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:18:36.817691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:18:46.707770+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63ccc094-62d5-446f-844d-1c60d41fda1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6803","nonce":354370090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6805","nonce":354370090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6809","nonce":354370090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6807","nonce":354370090}]},"public_addr":"172.21.15.27:6803/354370090","cluster_addr":"172.21.15.27:6805/354370090","heartbeat_back_addr":"172.21.15.27:6809/354370090","heartbeat_front_addr":"172.21.15.27:6807/354370090","state":["exists","up"]},{"osd":1,"uuid":"e59a8ed4-3a37-4477-a60a-4abc33d49b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6801","nonce":922528621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6803","nonce":922528621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6807","nonce":922528621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6805","nonce":922528621}]},"public_addr":"172.21.15.89:6801/922528621","cluster_addr":"172.21.15.89:6803/922528621","heartbeat_back_addr":"172.21.15.89:6807/922528621","heartbeat_front_addr":"172.21.15.89:6805/922528621","state":["exists","up"]},{"osd":2,"uuid":"6058a875-d5b2-4459-9407-2fd190cb34a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6801","nonce":257636259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6803","nonce":257636259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6807","nonce":257636259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6805","nonce":257636259}]},"public_addr":"172.21.15.98:6801/257636259","cluster_addr":"172.21.15.98:6803/257636259","heartbeat_back_addr":"172.21.15.98:6807/257636259","heartbeat_front_addr":"172.21.15.98:6805/257636259","state":["exists","up"]},{"osd":3,"uuid":"dfcdaede-e51c-4985-92af-3e296435a36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6811","nonce":821088824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6813","nonce":821088824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6817","nonce":821088824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6815","nonce":821088824}]},"public_addr":"172.21.15.27:6811/821088824","cluster_addr":"172.21.15.27:6813/821088824","heartbeat_back_addr":"172.21.15.27:6817/821088824","heartbeat_front_addr":"172.21.15.27:6815/821088824","state":["exists","up"]},{"osd":4,"uuid":"7c4af037-8ab6-47c0-b62b-76bd775b3491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6809","nonce":4001038690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6811","nonce":4001038690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6815","nonce":4001038690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6813","nonce":4001038690}]},"public_addr":"172.21.15.89:6809/4001038690","cluster_addr":"172.21.15.89:6811/4001038690","heartbeat_back_addr":"172.21.15.89:6815/4001038690","heartbeat_front_addr":"172.21.15.89:6813/4001038690","state":["exists","up"]},{"osd":5,"uuid":"e996ed0d-88ad-4aa7-891f-dfade4e09cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6809","nonce":3029903562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6811","nonce":3029903562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6815","nonce":3029903562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6813","nonce":3029903562}]},"public_addr":"172.21.15.98:6809/3029903562","cluster_addr":"172.21.15.98:6811/3029903562","heartbeat_back_addr":"172.21.15.98:6815/3029903562","heartbeat_front_addr":"172.21.15.98:6813/3029903562","state":["exists","up"]},{"osd":6,"uuid":"cf080497-57c2-42d0-8c8f-1eed0110702d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6819","nonce":1286808857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6821","nonce":1286808857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6825","nonce":1286808857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6823","nonce":1286808857}]},"public_addr":"172.21.15.27:6819/1286808857","cluster_addr":"172.21.15.27:6821/1286808857","heartbeat_back_addr":"172.21.15.27:6825/1286808857","heartbeat_front_addr":"172.21.15.27:6823/1286808857","state":["exists","up"]},{"osd":7,"uuid":"ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6817","nonce":2479349546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6819","nonce":2479349546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6823","nonce":2479349546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6821","nonce":2479349546}]},"public_addr":"172.21.15.89:6817/2479349546","cluster_addr":"172.21.15.89:6819/2479349546","heartbeat_back_addr":"172.21.15.89:6823/2479349546","heartbeat_front_addr":"172.21.15.89:6821/2479349546","state":["exists","up"]},{"osd":8,"uuid":"0b558458-fc2d-4ed1-b72a-e2ea8cf811b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6817","nonce":2830381485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6819","nonce":2830381485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6823","nonce":2830381485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6821","nonce":2830381485}]},"public_addr":"172.21.15.98:6817/2830381485","cluster_addr":"172.21.15.98:6819/2830381485","heartbeat_back_addr":"172.21.15.98:6823/2830381485","heartbeat_front_addr":"172.21.15.98:6821/2830381485","state":["exists","up"]},{"osd":9,"uuid":"1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6827","nonce":3033977113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6829","nonce":3033977113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6833","nonce":3033977113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6831","nonce":3033977113}]},"public_addr":"172.21.15.27:6827/3033977113","cluster_addr":"172.21.15.27:6829/3033977113","heartbeat_back_addr":"172.21.15.27:6833/3033977113","heartbeat_front_addr":"172.21.15.27:6831/3033977113","state":["exists","up"]},{"osd":10,"uuid":"324dd6ce-68fe-4c76-959b-1c9fb0ae92a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6825","nonce":2370332225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6827","nonce":2370332225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6831","nonce":2370332225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6829","nonce":2370332225}]},"public_addr":"172.21.15.89:6825/2370332225","cluster_addr":"172.21.15.89:6827/2370332225","heartbeat_back_addr":"172.21.15.89:6831/2370332225","heartbeat_front_addr":"172.21.15.89:6829/2370332225","state":["exists","up"]},{"osd":11,"uuid":"5e34686e-350a-491e-b9d7-59174ea90944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6825","nonce":2795999064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6827","nonce":2795999064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6831","nonce":2795999064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6829","nonce":2795999064}]},"public_addr":"172.21.15.98:6825/2795999064","cluster_addr":"172.21.15.98:6827/2795999064","heartbeat_back_addr":"172.21.15.98:6831/2795999064","heartbeat_front_addr":"172.21.15.98:6829/2795999064","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:12:39.841958+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:06.684417+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:31.041378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:01.612108+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:28.592326+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:54.303714+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:25.523475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:52.523483+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:19.430503+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:50.599120+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:17.520452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:42.725550+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/502883064":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/831048090":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/3880378905":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/793991452":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/2077956479":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6800/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6801/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/1098432608":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6800/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/528019907":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/2173601508":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:6800/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/764776233":"2024-09-22T00:11:24.231101+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:21:22.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.585+0000 7f1bbdc00640 1 -- 172.21.15.27:0/2214117758 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1b9c0b6560 msgr2=0x7f1b9c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:22.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.585+0000 7f1bbdc00640 1 --2- 172.21.15.27:0/2214117758 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1b9c0b6560 0x7f1b9c0b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009230 tx=0x7f1ba0031400 comp rx=0 tx=0).stop 2024-09-21T00:21:22.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.585+0000 7f1bbdc00640 1 -- 172.21.15.27:0/2214117758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bb81039b0 msgr2=0x7f1bb8077120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:22.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.585+0000 7f1bbdc00640 1 --2- 172.21.15.27:0/2214117758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bb81039b0 0x7f1bb8077120 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800ea60 tx=0x7f1ba800ef20 comp rx=0 tx=0).stop 2024-09-21T00:21:22.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.585+0000 7f1bbdc00640 1 -- 172.21.15.27:0/2214117758 shutdown_connections 2024-09-21T00:21:22.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.585+0000 7f1bbdc00640 1 --2- 172.21.15.27:0/2214117758 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1b9c0b6560 0x7f1b9c0b8a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:22.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.586+0000 7f1bbdc00640 1 --2- 172.21.15.27:0/2214117758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bb8109c40 0x7f1bb8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:22.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.586+0000 7f1bbdc00640 1 --2- 172.21.15.27:0/2214117758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bb81092f0 0x7f1bb8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:22.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.586+0000 7f1bbdc00640 1 --2- 172.21.15.27:0/2214117758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bb81039b0 0x7f1bb8077120 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:22.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.586+0000 7f1bbdc00640 1 -- 172.21.15.27:0/2214117758 >> 172.21.15.27:0/2214117758 conn(0x7f1bb80ff440 msgr2=0x7f1bb8101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:22.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.586+0000 7f1bbdc00640 1 -- 172.21.15.27:0/2214117758 shutdown_connections 2024-09-21T00:21:22.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:22.586+0000 7f1bbdc00640 1 -- 172.21.15.27:0/2214117758 wait complete. 2024-09-21T00:21:23.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:23 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2214117758' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:21:23.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T00:18:39.809850+0000', 'modified': '2024-09-21T00:19:37.959611+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24761, 'mds_1': 24751, 'mds_2': 24727}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24703': {'gid': 24703, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6835/388589971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 388589971}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 388589971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24715': {'gid': 24715, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6837/3858073278', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 3858073278}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 3858073278}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24722': {'gid': 24722, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6837/3044996985', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6836', 'nonce': 3044996985}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 3044996985}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24727': {'gid': 24727, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.98:6839/3074699335', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 3074699335}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 3074699335}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24751': {'gid': 24751, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.27:6841/378707351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6840', 'nonce': 378707351}, {'type': 'v1', 'addr': '172.21.15.27:6841', 'nonce': 378707351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24761': {'gid': 24761, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.89:6839/2432693964', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 2432693964}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 2432693964}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24761, 'qdb_cluster': [24761, 24751, 24727]} 2024-09-21T00:21:23.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T00:21:23.461 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T00:21:23.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:23 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2214117758' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:21:23.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:23 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2214117758' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:21:23.847 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:24.290 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:24 smithi027 ceph-mon[30745]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-09-21T00:21:24.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.473+0000 7f679de00640 1 -- 172.21.15.27:0/673906979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6798105d80 msgr2=0x7f6798106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:24.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.473+0000 7f679de00640 1 --2- 172.21.15.27:0/673906979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6798105d80 0x7f6798106190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6788009860 tx=0x7f678802a340 comp rx=0 tx=0).stop 2024-09-21T00:21:24.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.473+0000 7f679de00640 1 -- 172.21.15.27:0/673906979 shutdown_connections 2024-09-21T00:21:24.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.473+0000 7f679de00640 1 --2- 172.21.15.27:0/673906979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f679810c010 0x7f6798112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.473+0000 7f679de00640 1 --2- 172.21.15.27:0/673906979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f679810b6c0 0x7f679810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.473+0000 7f679de00640 1 --2- 172.21.15.27:0/673906979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6798105d80 0x7f6798106190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.473+0000 7f679de00640 1 -- 172.21.15.27:0/673906979 >> 172.21.15.27:0/673906979 conn(0x7f6798101810 msgr2=0x7f6798103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:24.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.473+0000 7f679de00640 1 -- 172.21.15.27:0/673906979 shutdown_connections 2024-09-21T00:21:24.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.473+0000 7f679de00640 1 -- 172.21.15.27:0/673906979 wait complete. 2024-09-21T00:21:24.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.474+0000 7f679de00640 1 Processor -- start 2024-09-21T00:21:24.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.475+0000 7f679de00640 1 -- start start 2024-09-21T00:21:24.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.475+0000 7f679de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6798105d80 0x7f679819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.475+0000 7f679de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f679810b6c0 0x7f679819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.475+0000 7f679de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f679810c010 0x7f67981a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.475+0000 7f679de00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6798115cd0 con 0x7f6798105d80 2024-09-21T00:21:24.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.475+0000 7f679de00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6798115b50 con 0x7f679810b6c0 2024-09-21T00:21:24.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.475+0000 7f679de00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6798115e50 con 0x7f679810c010 2024-09-21T00:21:24.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.475+0000 7f6796a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6798105d80 0x7f679819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.475+0000 7f6796a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6798105d80 0x7f679819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38646/0 (socket says 172.21.15.27:38646) 2024-09-21T00:21:24.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.476+0000 7f6796a00640 1 -- 172.21.15.27:0/517911155 learned_addr learned my addr 172.21.15.27:0/517911155 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:24.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.476+0000 7f6796000640 1 --2- 172.21.15.27:0/517911155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f679810b6c0 0x7f679819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.476+0000 7f6797400640 1 --2- 172.21.15.27:0/517911155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f679810c010 0x7f67981a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:24 smithi089 ceph-mon[34481]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-09-21T00:21:24.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.476+0000 7f6796a00640 1 -- 172.21.15.27:0/517911155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f679810c010 msgr2=0x7f67981a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:24.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.476+0000 7f6796a00640 1 --2- 172.21.15.27:0/517911155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f679810c010 0x7f67981a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.476+0000 7f6796a00640 1 -- 172.21.15.27:0/517911155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f679810b6c0 msgr2=0x7f679819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:24.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.476+0000 7f6796a00640 1 --2- 172.21.15.27:0/517911155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f679810b6c0 0x7f679819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.476+0000 7f6796a00640 1 -- 172.21.15.27:0/517911155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67981a25f0 con 0x7f6798105d80 2024-09-21T00:21:24.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.476+0000 7f6797400640 1 --2- 172.21.15.27:0/517911155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f679810c010 0x7f67981a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:24.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.476+0000 7f6796a00640 1 --2- 172.21.15.27:0/517911155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6798105d80 0x7f679819c680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f67880303f0 tx=0x7f6788032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:24.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.477+0000 7f679ca00640 1 -- 172.21.15.27:0/517911155 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788040020 con 0x7f6798105d80 2024-09-21T00:21:24.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.477+0000 7f679de00640 1 -- 172.21.15.27:0/517911155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67981a2880 con 0x7f6798105d80 2024-09-21T00:21:24.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.477+0000 7f679ca00640 1 -- 172.21.15.27:0/517911155 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6788030dd0 con 0x7f6798105d80 2024-09-21T00:21:24.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.477+0000 7f679ca00640 1 -- 172.21.15.27:0/517911155 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788037040 con 0x7f6798105d80 2024-09-21T00:21:24.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.477+0000 7f679de00640 1 -- 172.21.15.27:0/517911155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67981a9040 con 0x7f6798105d80 2024-09-21T00:21:24.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.480+0000 7f67a4a53740 1 Processor -- start 2024-09-21T00:21:24.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.480+0000 7f67a4a53740 1 -- start start 2024-09-21T00:21:24.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.481+0000 7f679ca00640 1 -- 172.21.15.27:0/517911155 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f6788043020 con 0x7f6798105d80 2024-09-21T00:21:24.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.481+0000 7f67a4a53740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1e5f60 0x5600cc58c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.481+0000 7f67a4a53740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1bd0a0 0x5600cc58c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.481+0000 7f67a4a53740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1c5280 0x5600cc58cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.481+0000 7f67a4a53740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5600cc503d50 con 0x5600cc1e5f60 2024-09-21T00:21:24.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.481+0000 7f67a4a53740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5600cc4f3180 con 0x5600cc1bd0a0 2024-09-21T00:21:24.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.481+0000 7f67a4a53740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5600cc4f6f90 con 0x5600cc1c5280 2024-09-21T00:21:24.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.481+0000 7f6797400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1e5f60 0x5600cc58c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.481+0000 7f6796a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1bd0a0 0x5600cc58c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.481+0000 7f6797400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1e5f60 0x5600cc58c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38662/0 (socket says 172.21.15.27:38662) 2024-09-21T00:21:24.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.481+0000 7f6796000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1c5280 0x5600cc58cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.481+0000 7f6797400640 1 -- 172.21.15.27:0/1938481586 learned_addr learned my addr 172.21.15.27:0/1938481586 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:24.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.482+0000 7f6796000640 1 -- 172.21.15.27:0/1938481586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1bd0a0 msgr2=0x5600cc58c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:24.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.482+0000 7f6796000640 1 --2- 172.21.15.27:0/1938481586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1bd0a0 0x5600cc58c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.482+0000 7f6796000640 1 -- 172.21.15.27:0/1938481586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1e5f60 msgr2=0x5600cc58c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:24.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.482+0000 7f6796000640 1 --2- 172.21.15.27:0/1938481586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1e5f60 0x5600cc58c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.482+0000 7f6796000640 1 -- 172.21.15.27:0/1938481586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600cc4bda70 con 0x5600cc1c5280 2024-09-21T00:21:24.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.482+0000 7f6796000640 1 --2- 172.21.15.27:0/1938481586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1c5280 0x5600cc58cf10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6784009530 tx=0x7f6784011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7ae9e28d36c30fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:24.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.482+0000 7f6783e00640 1 -- 172.21.15.27:0/1938481586 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784004020 con 0x5600cc1c5280 2024-09-21T00:21:24.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.482+0000 7f6783e00640 1 -- 172.21.15.27:0/1938481586 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67840041c0 con 0x5600cc1c5280 2024-09-21T00:21:24.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.482+0000 7f6783e00640 1 -- 172.21.15.27:0/1938481586 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67840197f0 con 0x5600cc1c5280 2024-09-21T00:21:24.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.483+0000 7f679ca00640 1 --2- 172.21.15.27:0/517911155 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f677c0b6560 0x7f677c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.483+0000 7f67a4a53740 1 -- 172.21.15.27:0/1938481586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1c5280 msgr2=0x5600cc58cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:24.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.483+0000 7f67a4a53740 1 --2- 172.21.15.27:0/1938481586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1c5280 0x5600cc58cf10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6784009530 tx=0x7f6784011810 comp rx=0 tx=0).stop 2024-09-21T00:21:24.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.483+0000 7f679ca00640 1 -- 172.21.15.27:0/517911155 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f6788107440 con 0x7f6798105d80 2024-09-21T00:21:24.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.483+0000 7f67a4a53740 1 -- 172.21.15.27:0/1938481586 shutdown_connections 2024-09-21T00:21:24.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.483+0000 7f67a4a53740 1 --2- 172.21.15.27:0/1938481586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1c5280 0x5600cc58cf10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.483+0000 7f67a4a53740 1 --2- 172.21.15.27:0/1938481586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1bd0a0 0x5600cc58c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.483+0000 7f67a4a53740 1 --2- 172.21.15.27:0/1938481586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1e5f60 0x5600cc58c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.483+0000 7f67a4a53740 1 -- 172.21.15.27:0/1938481586 >> 172.21.15.27:0/1938481586 conn(0x5600cc1da350 msgr2=0x5600cc58a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:24.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.483+0000 7f6797400640 1 --2- 172.21.15.27:0/517911155 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f677c0b6560 0x7f677c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.484+0000 7f67a4a53740 1 -- 172.21.15.27:0/1938481586 shutdown_connections 2024-09-21T00:21:24.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.484+0000 7f67a4a53740 1 -- 172.21.15.27:0/1938481586 wait complete. 2024-09-21T00:21:24.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.484+0000 7f6797400640 1 --2- 172.21.15.27:0/517911155 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f677c0b6560 0x7f677c0b8a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6790009fd0 tx=0x7f6790009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:24.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.484+0000 7f67a4a53740 1 -- start start 2024-09-21T00:21:24.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.485+0000 7f67a4a53740 1 Processor -- start 2024-09-21T00:21:24.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.485+0000 7f67a4a53740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1bd0a0 0x5600cc598060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.485+0000 7f67a4a53740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1c5280 0x5600cc5985a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.485+0000 7f67a4a53740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1e5f60 0x5600cc5a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.485+0000 7f67a4a53740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5600cc3545f0 con 0x5600cc1c5280 2024-09-21T00:21:24.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.485+0000 7f67a4a53740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5600cc48e460 con 0x5600cc1e5f60 2024-09-21T00:21:24.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.485+0000 7f67a4a53740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5600cc59ac70 con 0x5600cc1bd0a0 2024-09-21T00:21:24.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.485+0000 7f67a4a53740 10 client.0 _ll_register_callbacks cb 0x5600cc49e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:21:24.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.486+0000 7f6797400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1e5f60 0x5600cc5a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.486+0000 7f6796000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1c5280 0x5600cc5985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.486+0000 7f6797400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1e5f60 0x5600cc5a3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:33450/0 (socket says 172.21.15.27:33450) 2024-09-21T00:21:24.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.486+0000 7f6797400640 1 -- 172.21.15.27:0/3871087934 learned_addr learned my addr 172.21.15.27:0/3871087934 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:24.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.486+0000 7f6796a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1bd0a0 0x5600cc598060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.486+0000 7f6796a00640 1 -- 172.21.15.27:0/3871087934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1e5f60 msgr2=0x5600cc5a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:24.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.486+0000 7f6796a00640 1 --2- 172.21.15.27:0/3871087934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1e5f60 0x5600cc5a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.486+0000 7f6796a00640 1 -- 172.21.15.27:0/3871087934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1c5280 msgr2=0x5600cc5985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:24.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.486+0000 7f6796a00640 1 --2- 172.21.15.27:0/3871087934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1c5280 0x5600cc5985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.486+0000 7f6796a00640 1 -- 172.21.15.27:0/3871087934 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600cc251540 con 0x5600cc1bd0a0 2024-09-21T00:21:24.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.486+0000 7f6796a00640 1 --2- 172.21.15.27:0/3871087934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1bd0a0 0x5600cc598060 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5600cc58b110 tx=0x7f678802ab40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a534aaa6b48f3e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:24.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.486+0000 7f6780c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:21:24.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.486+0000 7f6780c00640 1 -- 172.21.15.27:0/3871087934 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67880c9710 con 0x5600cc1bd0a0 2024-09-21T00:21:24.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.487+0000 7f6780c00640 1 -- 172.21.15.27:0/3871087934 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f67880c8a10 con 0x5600cc1bd0a0 2024-09-21T00:21:24.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.487+0000 7f67a4a53740 1 -- 172.21.15.27:0/3871087934 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5600cc490a70 con 0x5600cc1bd0a0 2024-09-21T00:21:24.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.487+0000 7f6780c00640 1 -- 172.21.15.27:0/3871087934 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f67880cc830 con 0x5600cc1bd0a0 2024-09-21T00:21:24.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.487+0000 7f67a4a53740 10 client.25312 fetch_fsmap learned FSMap version 25 2024-09-21T00:21:24.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.487+0000 7f67a4a53740 1 -- 172.21.15.27:0/3871087934 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5600cc4ab6f0 con 0x5600cc1bd0a0 2024-09-21T00:21:24.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.487+0000 7f6780c00640 1 -- 172.21.15.27:0/3871087934 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5600cc490a70 con 0x5600cc1bd0a0 2024-09-21T00:21:24.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.487+0000 7f6780c00640 1 -- 172.21.15.27:0/3871087934 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f67880c8c10 con 0x5600cc1bd0a0 2024-09-21T00:21:24.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.488+0000 7f67a4a53740 10 client.25312 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T00:21:24.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.488+0000 7f67a4a53740 10 client.25312 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T00:21:24.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.488+0000 7f67a4a53740 20 client.25312 populate_metadata read hostname 'smithi027' 2024-09-21T00:21:24.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.488+0000 7f67a4a53740 1 --2- 172.21.15.27:0/3871087934 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x5600cc1a48e0 0x5600cc5aa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.488+0000 7f67a4a53740 4 client.25312 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:21:24.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.488+0000 7f67a4a53740 1 -- 172.21.15.27:0/3871087934 --> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600cc505ca0 con 0x5600cc1a48e0 2024-09-21T00:21:24.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.488+0000 7f6796000640 1 --2- 172.21.15.27:0/3871087934 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x5600cc1a48e0 0x5600cc5aa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.489+0000 7f6796000640 1 --2- 172.21.15.27:0/3871087934 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x5600cc1a48e0 0x5600cc5aa580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=533e72668498e11d server_cookie=2f7ba09dfcedfbad in_seq=0 out_seq=0 2024-09-21T00:21:24.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.489+0000 7f6780c00640 10 client.25312 ms_handle_connect on v2:172.21.15.98:6838/3074699335 2024-09-21T00:21:24.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.490+0000 7f6780c00640 1 -- 172.21.15.27:0/3871087934 <== mds.2 v2:172.21.15.98:6838/3074699335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5600cc505ca0 con 0x5600cc1a48e0 2024-09-21T00:21:24.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.490+0000 7f6780c00640 10 client.25312 handle_command_reply: tid=0 2024-09-21T00:21:24.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.490+0000 7f67a4a53740 1 client.25312 shutdown 2024-09-21T00:21:24.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.490+0000 7f67a4a53740 1 -- 172.21.15.27:0/3871087934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1bd0a0 msgr2=0x5600cc598060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:24.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.490+0000 7f67a4a53740 1 --2- 172.21.15.27:0/3871087934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1bd0a0 0x5600cc598060 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5600cc58b110 tx=0x7f678802ab40 comp rx=0 tx=0).stop 2024-09-21T00:21:24.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.490+0000 7f67a4a53740 1 -- 172.21.15.27:0/3871087934 shutdown_connections 2024-09-21T00:21:24.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.490+0000 7f67a4a53740 1 --2- 172.21.15.27:0/3871087934 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x5600cc1a48e0 0x5600cc5aa580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.490+0000 7f67a4a53740 1 --2- 172.21.15.27:0/3871087934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1e5f60 0x5600cc5a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.490+0000 7f67a4a53740 1 --2- 172.21.15.27:0/3871087934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1c5280 0x5600cc5985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.490+0000 7f6780c00640 0 client.25312 ms_handle_reset on v2:172.21.15.98:6838/3074699335 2024-09-21T00:21:24.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.490+0000 7f67a4a53740 1 --2- 172.21.15.27:0/3871087934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1bd0a0 0x5600cc598060 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.490+0000 7f67a4a53740 1 -- 172.21.15.27:0/3871087934 >> 172.21.15.27:0/3871087934 conn(0x5600cc1da350 msgr2=0x5600cc58f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:24.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.490+0000 7f67a4a53740 1 -- 172.21.15.27:0/3871087934 shutdown_connections 2024-09-21T00:21:24.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.490+0000 7f67a4a53740 1 -- 172.21.15.27:0/3871087934 wait complete. 2024-09-21T00:21:24.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.490+0000 7f67a4a53740 20 client.25312 trim_cache size 0 max 16384 2024-09-21T00:21:24.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:24 smithi098 ceph-mon[34725]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-09-21T00:21:24.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.510+0000 7f67a4a53740 1 Processor -- start 2024-09-21T00:21:24.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.510+0000 7f67a4a53740 1 -- start start 2024-09-21T00:21:24.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.510+0000 7f67a4a53740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1bd0a0 0x5600cc598040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.510+0000 7f67a4a53740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1c5280 0x5600cc598580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.510+0000 7f67a4a53740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1e5f60 0x5600cc5966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.510+0000 7f67a4a53740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5600cc5a88c0 con 0x5600cc1e5f60 2024-09-21T00:21:24.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.510+0000 7f67a4a53740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5600cc5a8f90 con 0x5600cc1bd0a0 2024-09-21T00:21:24.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.510+0000 7f67a4a53740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5600cc5a9660 con 0x5600cc1c5280 2024-09-21T00:21:24.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.510+0000 7f6797400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1e5f60 0x5600cc5966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.511+0000 7f6797400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1e5f60 0x5600cc5966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38680/0 (socket says 172.21.15.27:38680) 2024-09-21T00:21:24.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.511+0000 7f6797400640 1 -- 172.21.15.27:0/527621079 learned_addr learned my addr 172.21.15.27:0/527621079 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:24.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.511+0000 7f6796a00640 1 --2- 172.21.15.27:0/527621079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1bd0a0 0x5600cc598040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.511+0000 7f6796000640 1 --2- 172.21.15.27:0/527621079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1c5280 0x5600cc598580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.511+0000 7f6797400640 1 -- 172.21.15.27:0/527621079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1c5280 msgr2=0x5600cc598580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:24.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.511+0000 7f6797400640 1 --2- 172.21.15.27:0/527621079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1c5280 0x5600cc598580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.511+0000 7f6797400640 1 -- 172.21.15.27:0/527621079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1bd0a0 msgr2=0x5600cc598040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:24.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.511+0000 7f6797400640 1 --2- 172.21.15.27:0/527621079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1bd0a0 0x5600cc598040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.511+0000 7f6797400640 1 -- 172.21.15.27:0/527621079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600cc58e780 con 0x5600cc1e5f60 2024-09-21T00:21:24.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.511+0000 7f6796000640 1 --2- 172.21.15.27:0/527621079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1c5280 0x5600cc598580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:24.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.511+0000 7f6796a00640 1 --2- 172.21.15.27:0/527621079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1bd0a0 0x5600cc598040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:24.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.511+0000 7f6797400640 1 --2- 172.21.15.27:0/527621079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1e5f60 0x5600cc5966e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6790011710 tx=0x7f679001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d030fb4b3fe9d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:24.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.512+0000 7f6783400640 1 -- 172.21.15.27:0/527621079 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679001e6b0 con 0x5600cc1e5f60 2024-09-21T00:21:24.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.512+0000 7f6783400640 1 -- 172.21.15.27:0/527621079 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f679001ed00 con 0x5600cc1e5f60 2024-09-21T00:21:24.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.512+0000 7f6783400640 1 -- 172.21.15.27:0/527621079 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790029dc0 con 0x5600cc1e5f60 2024-09-21T00:21:24.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.513+0000 7f67a4a53740 1 -- 172.21.15.27:0/527621079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1e5f60 msgr2=0x5600cc5966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.513+0000 7f67a4a53740 1 --2- 172.21.15.27:0/527621079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1e5f60 0x5600cc5966e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6790011710 tx=0x7f679001a000 comp rx=0 tx=0).stop 2024-09-21T00:21:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.513+0000 7f67a4a53740 1 -- 172.21.15.27:0/527621079 shutdown_connections 2024-09-21T00:21:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.513+0000 7f67a4a53740 1 --2- 172.21.15.27:0/527621079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1e5f60 0x5600cc5966e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.513+0000 7f67a4a53740 1 --2- 172.21.15.27:0/527621079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1c5280 0x5600cc598580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.513+0000 7f67a4a53740 1 --2- 172.21.15.27:0/527621079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1bd0a0 0x5600cc598040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.513+0000 7f67a4a53740 1 -- 172.21.15.27:0/527621079 >> 172.21.15.27:0/527621079 conn(0x5600cc1da350 msgr2=0x5600cc593010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.513+0000 7f67a4a53740 1 -- 172.21.15.27:0/527621079 shutdown_connections 2024-09-21T00:21:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.513+0000 7f67a4a53740 1 -- 172.21.15.27:0/527621079 wait complete. 2024-09-21T00:21:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.513+0000 7f67a4a53740 1 -- start start 2024-09-21T00:21:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.513+0000 7f67a4a53740 1 Processor -- start 2024-09-21T00:21:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.514+0000 7f67a4a53740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1bd0a0 0x5600cc5aca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.514+0000 7f67a4a53740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1c5280 0x5600cc5acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.514+0000 7f67a4a53740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1e5f60 0x5600cc5b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.514+0000 7f67a4a53740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5600cc5aa400 con 0x5600cc1c5280 2024-09-21T00:21:24.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.514+0000 7f67a4a53740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5600cc5a9d30 con 0x5600cc1e5f60 2024-09-21T00:21:24.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.514+0000 7f67a4a53740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5600cc5aaad0 con 0x5600cc1bd0a0 2024-09-21T00:21:24.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.514+0000 7f67a4a53740 10 client.0 _ll_register_callbacks cb 0x7f6788005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:21:24.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.514+0000 7f6796a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1bd0a0 0x5600cc5aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.514+0000 7f6796000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1c5280 0x5600cc5acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.514+0000 7f6796a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1bd0a0 0x5600cc5aca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:57110/0 (socket says 172.21.15.27:57110) 2024-09-21T00:21:24.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.514+0000 7f6796a00640 1 -- 172.21.15.27:0/3375451072 learned_addr learned my addr 172.21.15.27:0/3375451072 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:24.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.514+0000 7f6797400640 1 --2- 172.21.15.27:0/3375451072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1e5f60 0x5600cc5b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.515+0000 7f6796a00640 1 -- 172.21.15.27:0/3375451072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1e5f60 msgr2=0x5600cc5b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:24.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.515+0000 7f6796a00640 1 --2- 172.21.15.27:0/3375451072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1e5f60 0x5600cc5b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.515+0000 7f6796a00640 1 -- 172.21.15.27:0/3375451072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1c5280 msgr2=0x5600cc5acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:24.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.515+0000 7f6796a00640 1 --2- 172.21.15.27:0/3375451072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1c5280 0x5600cc5acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.515+0000 7f6796a00640 1 -- 172.21.15.27:0/3375451072 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600cc5ad790 con 0x5600cc1bd0a0 2024-09-21T00:21:24.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.515+0000 7f6796a00640 1 --2- 172.21.15.27:0/3375451072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1bd0a0 0x5600cc5aca60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f67880cd050 tx=0x7f67881049e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f2eb9c1ec9266e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:24.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.515+0000 7f6780c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:21:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.515+0000 7f6780c00640 1 -- 172.21.15.27:0/3375451072 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67880cca10 con 0x5600cc1bd0a0 2024-09-21T00:21:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.515+0000 7f6780c00640 1 -- 172.21.15.27:0/3375451072 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f67880f45a0 con 0x5600cc1bd0a0 2024-09-21T00:21:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.515+0000 7f67a4a53740 1 -- 172.21.15.27:0/3375451072 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f679001ed00 con 0x5600cc1bd0a0 2024-09-21T00:21:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.516+0000 7f6780c00640 1 -- 172.21.15.27:0/3375451072 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f67880cce90 con 0x5600cc1bd0a0 2024-09-21T00:21:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.516+0000 7f67a4a53740 10 client.25318 fetch_fsmap learned FSMap version 25 2024-09-21T00:21:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.516+0000 7f67a4a53740 1 -- 172.21.15.27:0/3375451072 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5600cc615f30 con 0x5600cc1bd0a0 2024-09-21T00:21:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.516+0000 7f6780c00640 1 -- 172.21.15.27:0/3375451072 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f679001ed00 con 0x5600cc1bd0a0 2024-09-21T00:21:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.516+0000 7f6780c00640 1 -- 172.21.15.27:0/3375451072 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f67880e5070 con 0x5600cc1bd0a0 2024-09-21T00:21:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.516+0000 7f67a4a53740 10 client.25318 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T00:21:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.516+0000 7f67a4a53740 10 client.25318 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T00:21:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.516+0000 7f67a4a53740 20 client.25318 populate_metadata read hostname 'smithi027' 2024-09-21T00:21:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.516+0000 7f67a4a53740 1 --2- 172.21.15.27:0/3375451072 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x5600cc179c90 0x5600cc619660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.516+0000 7f67a4a53740 4 client.25318 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:21:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.516+0000 7f67a4a53740 1 -- 172.21.15.27:0/3375451072 --> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] -- command(tid 0: {"prefix": "status"}) -- 0x5600cc501750 con 0x5600cc179c90 2024-09-21T00:21:24.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.517+0000 7f6796000640 1 --2- 172.21.15.27:0/3375451072 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x5600cc179c90 0x5600cc619660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:24.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.517+0000 7f6796000640 1 --2- 172.21.15.27:0/3375451072 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x5600cc179c90 0x5600cc619660 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f66993c94128a3a4 server_cookie=e53fd6156f04a001 in_seq=0 out_seq=0 2024-09-21T00:21:24.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.517+0000 7f6780c00640 10 client.25318 ms_handle_connect on v2:172.21.15.98:6838/3074699335 2024-09-21T00:21:24.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.518+0000 7f6780c00640 1 -- 172.21.15.27:0/3375451072 <== mds.2 v2:172.21.15.98:6838/3074699335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5600cc501750 con 0x5600cc179c90 2024-09-21T00:21:24.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.518+0000 7f6780c00640 10 client.25318 handle_command_reply: tid=0 2024-09-21T00:21:24.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.518+0000 7f67a4a53740 1 client.25318 shutdown 2024-09-21T00:21:24.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.518+0000 7f67a4a53740 1 -- 172.21.15.27:0/3375451072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1bd0a0 msgr2=0x5600cc5aca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:24.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.518+0000 7f67a4a53740 1 --2- 172.21.15.27:0/3375451072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1bd0a0 0x5600cc5aca60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f67880cd050 tx=0x7f67881049e0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.518+0000 7f67a4a53740 1 -- 172.21.15.27:0/3375451072 shutdown_connections 2024-09-21T00:21:24.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.518+0000 7f67a4a53740 1 --2- 172.21.15.27:0/3375451072 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x5600cc179c90 0x5600cc619660 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.518+0000 7f67a4a53740 1 --2- 172.21.15.27:0/3375451072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600cc1e5f60 0x5600cc5b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.518+0000 7f67a4a53740 1 --2- 172.21.15.27:0/3375451072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600cc1c5280 0x5600cc5acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.518+0000 7f67a4a53740 1 --2- 172.21.15.27:0/3375451072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600cc1bd0a0 0x5600cc5aca60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.518+0000 7f67a4a53740 1 -- 172.21.15.27:0/3375451072 >> 172.21.15.27:0/3375451072 conn(0x5600cc1da350 msgr2=0x5600cc598040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:24.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.518+0000 7f6780c00640 0 client.25318 ms_handle_reset on v2:172.21.15.98:6838/3074699335 2024-09-21T00:21:24.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.518+0000 7f67a4a53740 1 -- 172.21.15.27:0/3375451072 shutdown_connections 2024-09-21T00:21:24.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.518+0000 7f67a4a53740 1 -- 172.21.15.27:0/3375451072 wait complete. 2024-09-21T00:21:24.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.518+0000 7f67a4a53740 20 client.25318 trim_cache size 0 max 16384 2024-09-21T00:21:24.524 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:21:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "9c21de84-77ad-11ef-baf5-efdc52797490", 2024-09-21T00:21:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-09-21T00:21:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24727, 2024-09-21T00:21:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T00:21:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T00:21:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T00:21:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 110.626702505, 2024-09-21T00:21:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 25, 2024-09-21T00:21:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T00:21:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T00:21:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 187.89048796 2024-09-21T00:21:24.524 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:21:24.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.519+0000 7f679de00640 1 -- 172.21.15.27:0/517911155 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f677c0b6560 msgr2=0x7f677c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:24.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.519+0000 7f679de00640 1 --2- 172.21.15.27:0/517911155 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f677c0b6560 0x7f677c0b8a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6790009fd0 tx=0x7f6790009500 comp rx=0 tx=0).stop 2024-09-21T00:21:24.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.519+0000 7f679de00640 1 -- 172.21.15.27:0/517911155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6798105d80 msgr2=0x7f679819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:24.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.519+0000 7f679de00640 1 --2- 172.21.15.27:0/517911155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6798105d80 0x7f679819c680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f67880303f0 tx=0x7f6788032da0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.519+0000 7f679de00640 1 -- 172.21.15.27:0/517911155 shutdown_connections 2024-09-21T00:21:24.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.519+0000 7f679de00640 1 --2- 172.21.15.27:0/517911155 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f677c0b6560 0x7f677c0b8a70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.519+0000 7f679de00640 1 --2- 172.21.15.27:0/517911155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f679810c010 0x7f67981a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.519+0000 7f679de00640 1 --2- 172.21.15.27:0/517911155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f679810b6c0 0x7f679819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.519+0000 7f679de00640 1 --2- 172.21.15.27:0/517911155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6798105d80 0x7f679819c680 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:24.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.519+0000 7f679de00640 1 -- 172.21.15.27:0/517911155 >> 172.21.15.27:0/517911155 conn(0x7f6798101810 msgr2=0x7f6798103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:24.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.519+0000 7f679de00640 1 -- 172.21.15.27:0/517911155 shutdown_connections 2024-09-21T00:21:24.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:24.519+0000 7f679de00640 1 -- 172.21.15.27:0/517911155 wait complete. 2024-09-21T00:21:25.352 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell mds.a status 2024-09-21T00:21:25.736 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:26.217 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:26 smithi027 ceph-mon[30745]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-21T00:21:26.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.399+0000 7fb413000640 1 -- 172.21.15.27:0/322084933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c10a1b0 msgr2=0x7fb40c10a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:26.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.399+0000 7fb413000640 1 --2- 172.21.15.27:0/322084933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c10a1b0 0x7fb40c10a5c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009900 tx=0x7fb3fc02a3f0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.399+0000 7fb413000640 1 -- 172.21.15.27:0/322084933 shutdown_connections 2024-09-21T00:21:26.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.399+0000 7fb413000640 1 --2- 172.21.15.27:0/322084933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40c110440 0x7fb40c116c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.399+0000 7fb413000640 1 --2- 172.21.15.27:0/322084933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb40c10faf0 0x7fb40c10ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.399+0000 7fb413000640 1 --2- 172.21.15.27:0/322084933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c10a1b0 0x7fb40c10a5c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.399+0000 7fb413000640 1 -- 172.21.15.27:0/322084933 >> 172.21.15.27:0/322084933 conn(0x7fb40c0780e0 msgr2=0x7fb40c1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:26.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.399+0000 7fb413000640 1 -- 172.21.15.27:0/322084933 shutdown_connections 2024-09-21T00:21:26.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.399+0000 7fb413000640 1 -- 172.21.15.27:0/322084933 wait complete. 2024-09-21T00:21:26.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.400+0000 7fb413000640 1 Processor -- start 2024-09-21T00:21:26.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.401+0000 7fb413000640 1 -- start start 2024-09-21T00:21:26.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.401+0000 7fb413000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c10a1b0 0x7fb40c1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.401+0000 7fb413000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40c10faf0 0x7fb40c1a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.401+0000 7fb413000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb40c110440 0x7fb40c1a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.401+0000 7fb413000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c11a0f0 con 0x7fb40c10a1b0 2024-09-21T00:21:26.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.401+0000 7fb413000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c119f70 con 0x7fb40c110440 2024-09-21T00:21:26.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.401+0000 7fb413000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c11a270 con 0x7fb40c10faf0 2024-09-21T00:21:26.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.401+0000 7fb40be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c10a1b0 0x7fb40c1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.402+0000 7fb40be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c10a1b0 0x7fb40c1a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38706/0 (socket says 172.21.15.27:38706) 2024-09-21T00:21:26.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.402+0000 7fb40be00640 1 -- 172.21.15.27:0/1708546763 learned_addr learned my addr 172.21.15.27:0/1708546763 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:26.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.402+0000 7fb411000640 1 --2- 172.21.15.27:0/1708546763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb40c110440 0x7fb40c1a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.402+0000 7fb403400640 1 --2- 172.21.15.27:0/1708546763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40c10faf0 0x7fb40c1a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.402+0000 7fb40be00640 1 -- 172.21.15.27:0/1708546763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40c10faf0 msgr2=0x7fb40c1a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:26.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.402+0000 7fb40be00640 1 --2- 172.21.15.27:0/1708546763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40c10faf0 0x7fb40c1a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.402+0000 7fb40be00640 1 -- 172.21.15.27:0/1708546763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb40c110440 msgr2=0x7fb40c1a6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:26.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.402+0000 7fb40be00640 1 --2- 172.21.15.27:0/1708546763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb40c110440 0x7fb40c1a6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.402+0000 7fb40be00640 1 -- 172.21.15.27:0/1708546763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c1a6ab0 con 0x7fb40c10a1b0 2024-09-21T00:21:26.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.402+0000 7fb411000640 1 --2- 172.21.15.27:0/1708546763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb40c110440 0x7fb40c1a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:26.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.402+0000 7fb40be00640 1 --2- 172.21.15.27:0/1708546763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c10a1b0 0x7fb40c1a0b40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc005940 tx=0x7fb3fc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:26.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.403+0000 7fb409600640 1 -- 172.21.15.27:0/1708546763 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc03f030 con 0x7fb40c10a1b0 2024-09-21T00:21:26.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.403+0000 7fb409600640 1 -- 172.21.15.27:0/1708546763 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3fc0044c0 con 0x7fb40c10a1b0 2024-09-21T00:21:26.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.403+0000 7fb409600640 1 -- 172.21.15.27:0/1708546763 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc035070 con 0x7fb40c10a1b0 2024-09-21T00:21:26.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.403+0000 7fb413000640 1 -- 172.21.15.27:0/1708546763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c1a6d40 con 0x7fb40c10a1b0 2024-09-21T00:21:26.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.403+0000 7fb413000640 1 -- 172.21.15.27:0/1708546763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c1ad500 con 0x7fb40c10a1b0 2024-09-21T00:21:26.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.406+0000 7fb419c3e740 1 Processor -- start 2024-09-21T00:21:26.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.406+0000 7fb419c3e740 1 -- start start 2024-09-21T00:21:26.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.407+0000 7fb409600640 1 -- 172.21.15.27:0/1708546763 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fb3fc042030 con 0x7fb40c10a1b0 2024-09-21T00:21:26.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.407+0000 7fb419c3e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73cdf60 0x55dde7774490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.407+0000 7fb419c3e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73a50a0 0x55dde77749d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.407+0000 7fb419c3e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73ad280 0x55dde7774f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.407+0000 7fb419c3e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55dde76ebd50 con 0x55dde73cdf60 2024-09-21T00:21:26.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.407+0000 7fb419c3e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dde76db180 con 0x55dde73a50a0 2024-09-21T00:21:26.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.407+0000 7fb419c3e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55dde76def90 con 0x55dde73ad280 2024-09-21T00:21:26.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.407+0000 7fb411000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73cdf60 0x55dde7774490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.407+0000 7fb411000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73cdf60 0x55dde7774490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38720/0 (socket says 172.21.15.27:38720) 2024-09-21T00:21:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.407+0000 7fb411000640 1 -- 172.21.15.27:0/2612522314 learned_addr learned my addr 172.21.15.27:0/2612522314 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.407+0000 7fb40be00640 1 --2- 172.21.15.27:0/2612522314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73a50a0 0x55dde77749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.407+0000 7fb403400640 1 --2- 172.21.15.27:0/2612522314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73ad280 0x55dde7774f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.408+0000 7fb403400640 1 -- 172.21.15.27:0/2612522314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73a50a0 msgr2=0x55dde77749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.408+0000 7fb403400640 1 --2- 172.21.15.27:0/2612522314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73a50a0 0x55dde77749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.408+0000 7fb403400640 1 -- 172.21.15.27:0/2612522314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73cdf60 msgr2=0x55dde7774490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.408+0000 7fb403400640 1 --2- 172.21.15.27:0/2612522314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73cdf60 0x55dde7774490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.408+0000 7fb403400640 1 -- 172.21.15.27:0/2612522314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dde76a5a70 con 0x55dde73ad280 2024-09-21T00:21:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.408+0000 7fb411000640 1 --2- 172.21.15.27:0/2612522314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73cdf60 0x55dde7774490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.408+0000 7fb403400640 1 --2- 172.21.15.27:0/2612522314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73ad280 0x55dde7774f10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb40c06c790 tx=0x7fb3f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=991ecdd0863b9860 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.408+0000 7fb401600640 1 -- 172.21.15.27:0/2612522314 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4004050 con 0x55dde73ad280 2024-09-21T00:21:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.408+0000 7fb401600640 1 -- 172.21.15.27:0/2612522314 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f40054d0 con 0x55dde73ad280 2024-09-21T00:21:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.408+0000 7fb401600640 1 -- 172.21.15.27:0/2612522314 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4004c80 con 0x55dde73ad280 2024-09-21T00:21:26.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.409+0000 7fb409600640 1 --2- 172.21.15.27:0/1708546763 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb3f00b6560 0x7fb3f00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.409+0000 7fb409600640 1 -- 172.21.15.27:0/1708546763 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fb3fc107bb0 con 0x7fb40c10a1b0 2024-09-21T00:21:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.409+0000 7fb419c3e740 1 -- 172.21.15.27:0/2612522314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73ad280 msgr2=0x55dde7774f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.409+0000 7fb419c3e740 1 --2- 172.21.15.27:0/2612522314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73ad280 0x55dde7774f10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb40c06c790 tx=0x7fb3f4011810 comp rx=0 tx=0).stop 2024-09-21T00:21:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.409+0000 7fb411000640 1 --2- 172.21.15.27:0/1708546763 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb3f00b6560 0x7fb3f00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.409+0000 7fb419c3e740 1 -- 172.21.15.27:0/2612522314 shutdown_connections 2024-09-21T00:21:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.409+0000 7fb419c3e740 1 --2- 172.21.15.27:0/2612522314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73ad280 0x55dde7774f10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.409+0000 7fb419c3e740 1 --2- 172.21.15.27:0/2612522314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73a50a0 0x55dde77749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.409+0000 7fb419c3e740 1 --2- 172.21.15.27:0/2612522314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73cdf60 0x55dde7774490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.409+0000 7fb419c3e740 1 -- 172.21.15.27:0/2612522314 >> 172.21.15.27:0/2612522314 conn(0x55dde73c2350 msgr2=0x55dde7772380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.410+0000 7fb419c3e740 1 -- 172.21.15.27:0/2612522314 shutdown_connections 2024-09-21T00:21:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.410+0000 7fb419c3e740 1 -- 172.21.15.27:0/2612522314 wait complete. 2024-09-21T00:21:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.410+0000 7fb411000640 1 --2- 172.21.15.27:0/1708546763 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb3f00b6560 0x7fb3f00b8a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb4040115a0 tx=0x7fb404009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:26.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.410+0000 7fb419c3e740 1 -- start start 2024-09-21T00:21:26.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.411+0000 7fb419c3e740 1 Processor -- start 2024-09-21T00:21:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.411+0000 7fb419c3e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73a50a0 0x55dde7780080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.411+0000 7fb419c3e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73ad280 0x55dde77805c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.411+0000 7fb419c3e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73cdf60 0x55dde778bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.411+0000 7fb419c3e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55dde753c5f0 con 0x55dde73ad280 2024-09-21T00:21:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.411+0000 7fb419c3e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dde7676460 con 0x55dde73a50a0 2024-09-21T00:21:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.411+0000 7fb419c3e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55dde7782c70 con 0x55dde73cdf60 2024-09-21T00:21:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.411+0000 7fb419c3e740 10 client.0 _ll_register_callbacks cb 0x55dde7686210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:21:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.411+0000 7fb403400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73ad280 0x55dde77805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.412+0000 7fb403400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73ad280 0x55dde77805c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38728/0 (socket says 172.21.15.27:38728) 2024-09-21T00:21:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.412+0000 7fb40be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73a50a0 0x55dde7780080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.412+0000 7fb411000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73cdf60 0x55dde778bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.412+0000 7fb403400640 1 -- 172.21.15.27:0/1025646642 learned_addr learned my addr 172.21.15.27:0/1025646642 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.412+0000 7fb40be00640 1 -- 172.21.15.27:0/1025646642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73cdf60 msgr2=0x55dde778bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.412+0000 7fb40be00640 1 --2- 172.21.15.27:0/1025646642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73cdf60 0x55dde778bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.412+0000 7fb40be00640 1 -- 172.21.15.27:0/1025646642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73ad280 msgr2=0x55dde77805c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.412+0000 7fb40be00640 1 --2- 172.21.15.27:0/1025646642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73ad280 0x55dde77805c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.412+0000 7fb40be00640 1 -- 172.21.15.27:0/1025646642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dde7439540 con 0x55dde73a50a0 2024-09-21T00:21:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.412+0000 7fb411000640 1 --2- 172.21.15.27:0/1025646642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73cdf60 0x55dde778bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.412+0000 7fb40be00640 1 --2- 172.21.15.27:0/1025646642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73a50a0 0x55dde7780080 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55dde7773110 tx=0x7fb3fc0c9ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5238ebace50f01f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.412+0000 7fb3fa000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:21:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.413+0000 7fb3fa000640 1 -- 172.21.15.27:0/1025646642 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc049030 con 0x55dde73a50a0 2024-09-21T00:21:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.413+0000 7fb3fa000640 1 -- 172.21.15.27:0/1025646642 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fb3fc0ccd40 con 0x55dde73a50a0 2024-09-21T00:21:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.413+0000 7fb419c3e740 1 -- 172.21.15.27:0/1025646642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dde7678a70 con 0x55dde73a50a0 2024-09-21T00:21:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.413+0000 7fb3fa000640 1 -- 172.21.15.27:0/1025646642 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb3fc0cd6b0 con 0x55dde73a50a0 2024-09-21T00:21:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.413+0000 7fb419c3e740 10 client.25343 fetch_fsmap learned FSMap version 25 2024-09-21T00:21:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.413+0000 7fb419c3e740 1 -- 172.21.15.27:0/1025646642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dde76936f0 con 0x55dde73a50a0 2024-09-21T00:21:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.413+0000 7fb3fa000640 1 -- 172.21.15.27:0/1025646642 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dde7678a70 con 0x55dde73a50a0 2024-09-21T00:21:26.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.413+0000 7fb3fa000640 1 -- 172.21.15.27:0/1025646642 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb3fc031b30 con 0x55dde73a50a0 2024-09-21T00:21:26.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.414+0000 7fb419c3e740 10 client.25343 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T00:21:26.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.414+0000 7fb419c3e740 10 client.25343 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:21:26.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.414+0000 7fb419c3e740 20 client.25343 populate_metadata read hostname 'smithi027' 2024-09-21T00:21:26.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.414+0000 7fb419c3e740 1 --2- 172.21.15.27:0/1025646642 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x55dde738c8e0 0x55dde7792570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.414+0000 7fb419c3e740 4 client.25343 mds_command: new command op to 24751 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:21:26.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.414+0000 7fb419c3e740 1 -- 172.21.15.27:0/1025646642 --> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dde76edca0 con 0x55dde738c8e0 2024-09-21T00:21:26.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.414+0000 7fb403400640 1 --2- 172.21.15.27:0/1025646642 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x55dde738c8e0 0x55dde7792570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.415+0000 7fb403400640 1 --2- 172.21.15.27:0/1025646642 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x55dde738c8e0 0x55dde7792570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7fd645cc19a99dfd server_cookie=7aed27c4e3e91448 in_seq=0 out_seq=0 2024-09-21T00:21:26.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.415+0000 7fb3fa000640 10 client.25343 ms_handle_connect on v2:172.21.15.27:6840/378707351 2024-09-21T00:21:26.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.415+0000 7fb3fa000640 1 -- 172.21.15.27:0/1025646642 <== mds.1 v2:172.21.15.27:6840/378707351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dde76edca0 con 0x55dde738c8e0 2024-09-21T00:21:26.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.415+0000 7fb3fa000640 10 client.25343 handle_command_reply: tid=0 2024-09-21T00:21:26.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.415+0000 7fb419c3e740 1 client.25343 shutdown 2024-09-21T00:21:26.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.416+0000 7fb419c3e740 1 -- 172.21.15.27:0/1025646642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73a50a0 msgr2=0x55dde7780080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:26.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.416+0000 7fb419c3e740 1 --2- 172.21.15.27:0/1025646642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73a50a0 0x55dde7780080 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55dde7773110 tx=0x7fb3fc0c9ff0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.416+0000 7fb419c3e740 1 -- 172.21.15.27:0/1025646642 shutdown_connections 2024-09-21T00:21:26.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.416+0000 7fb419c3e740 1 --2- 172.21.15.27:0/1025646642 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x55dde738c8e0 0x55dde7792570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.416+0000 7fb419c3e740 1 --2- 172.21.15.27:0/1025646642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73cdf60 0x55dde778bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.416+0000 7fb419c3e740 1 --2- 172.21.15.27:0/1025646642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73ad280 0x55dde77805c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.416+0000 7fb419c3e740 1 --2- 172.21.15.27:0/1025646642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73a50a0 0x55dde7780080 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.416+0000 7fb419c3e740 1 -- 172.21.15.27:0/1025646642 >> 172.21.15.27:0/1025646642 conn(0x55dde73c2350 msgr2=0x55dde7777670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:26.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.416+0000 7fb3fa000640 0 client.25343 ms_handle_reset on v2:172.21.15.27:6840/378707351 2024-09-21T00:21:26.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.416+0000 7fb419c3e740 1 -- 172.21.15.27:0/1025646642 shutdown_connections 2024-09-21T00:21:26.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.416+0000 7fb419c3e740 1 -- 172.21.15.27:0/1025646642 wait complete. 2024-09-21T00:21:26.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.416+0000 7fb419c3e740 20 client.25343 trim_cache size 0 max 16384 2024-09-21T00:21:26.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.435+0000 7fb419c3e740 1 Processor -- start 2024-09-21T00:21:26.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.435+0000 7fb419c3e740 1 -- start start 2024-09-21T00:21:26.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb419c3e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73a50a0 0x55dde7792570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb419c3e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73ad280 0x55dde7779710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb419c3e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73cdf60 0x55dde7779c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb419c3e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55dde77908b0 con 0x55dde73a50a0 2024-09-21T00:21:26.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb419c3e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dde7790f80 con 0x55dde73cdf60 2024-09-21T00:21:26.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb419c3e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55dde7791650 con 0x55dde73ad280 2024-09-21T00:21:26.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb40be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73a50a0 0x55dde7792570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb40be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73a50a0 0x55dde7792570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38742/0 (socket says 172.21.15.27:38742) 2024-09-21T00:21:26.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb40be00640 1 -- 172.21.15.27:0/1831872202 learned_addr learned my addr 172.21.15.27:0/1831872202 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb403400640 1 --2- 172.21.15.27:0/1831872202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73ad280 0x55dde7779710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb411000640 1 --2- 172.21.15.27:0/1831872202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73cdf60 0x55dde7779c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb403400640 1 -- 172.21.15.27:0/1831872202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73cdf60 msgr2=0x55dde7779c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb403400640 1 --2- 172.21.15.27:0/1831872202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73cdf60 0x55dde7779c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb403400640 1 -- 172.21.15.27:0/1831872202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73a50a0 msgr2=0x55dde7792570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb403400640 1 --2- 172.21.15.27:0/1831872202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73a50a0 0x55dde7792570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.436+0000 7fb403400640 1 -- 172.21.15.27:0/1831872202 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dde7776780 con 0x55dde73ad280 2024-09-21T00:21:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.437+0000 7fb40be00640 1 --2- 172.21.15.27:0/1831872202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73a50a0 0x55dde7792570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.437+0000 7fb403400640 1 --2- 172.21.15.27:0/1831872202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73ad280 0x55dde7779710 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4005e00 tx=0x7fb3f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5a33ef80a6ce9c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.437+0000 7fb400c00640 1 -- 172.21.15.27:0/1831872202 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f401d040 con 0x55dde73ad280 2024-09-21T00:21:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.437+0000 7fb400c00640 1 -- 172.21.15.27:0/1831872202 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f4004050 con 0x55dde73ad280 2024-09-21T00:21:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.437+0000 7fb400c00640 1 -- 172.21.15.27:0/1831872202 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4024690 con 0x55dde73ad280 2024-09-21T00:21:26.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.438+0000 7fb419c3e740 1 -- 172.21.15.27:0/1831872202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73ad280 msgr2=0x55dde7779710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:26.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.438+0000 7fb419c3e740 1 --2- 172.21.15.27:0/1831872202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73ad280 0x55dde7779710 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4005e00 tx=0x7fb3f4011810 comp rx=0 tx=0).stop 2024-09-21T00:21:26.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.438+0000 7fb419c3e740 1 -- 172.21.15.27:0/1831872202 shutdown_connections 2024-09-21T00:21:26.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.438+0000 7fb419c3e740 1 --2- 172.21.15.27:0/1831872202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73cdf60 0x55dde7779c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.438+0000 7fb419c3e740 1 --2- 172.21.15.27:0/1831872202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73ad280 0x55dde7779710 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.438+0000 7fb419c3e740 1 --2- 172.21.15.27:0/1831872202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73a50a0 0x55dde7792570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.438+0000 7fb419c3e740 1 -- 172.21.15.27:0/1831872202 >> 172.21.15.27:0/1831872202 conn(0x55dde73c2350 msgr2=0x55dde777b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.438+0000 7fb419c3e740 1 -- 172.21.15.27:0/1831872202 shutdown_connections 2024-09-21T00:21:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.438+0000 7fb419c3e740 1 -- 172.21.15.27:0/1831872202 wait complete. 2024-09-21T00:21:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.439+0000 7fb419c3e740 1 -- start start 2024-09-21T00:21:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.439+0000 7fb419c3e740 1 Processor -- start 2024-09-21T00:21:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.439+0000 7fb419c3e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73a50a0 0x55dde779ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.439+0000 7fb419c3e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73ad280 0x55dde7771700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.439+0000 7fb419c3e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73cdf60 0x55dde7771c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.439+0000 7fb419c3e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55dde77923f0 con 0x55dde73cdf60 2024-09-21T00:21:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.439+0000 7fb419c3e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dde7791d20 con 0x55dde73ad280 2024-09-21T00:21:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.439+0000 7fb419c3e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55dde7792ac0 con 0x55dde73a50a0 2024-09-21T00:21:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.439+0000 7fb419c3e740 10 client.0 _ll_register_callbacks cb 0x7fb3fc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:21:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.440+0000 7fb40be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73a50a0 0x55dde779ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.440+0000 7fb411000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73cdf60 0x55dde7771c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.440+0000 7fb40be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73a50a0 0x55dde779ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:57170/0 (socket says 172.21.15.27:57170) 2024-09-21T00:21:26.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.440+0000 7fb40be00640 1 -- 172.21.15.27:0/4034484098 learned_addr learned my addr 172.21.15.27:0/4034484098 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:26.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.440+0000 7fb411000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73cdf60 0x55dde7771c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38744/0 (socket says 172.21.15.27:38744) 2024-09-21T00:21:26.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.440+0000 7fb403400640 1 --2- 172.21.15.27:0/4034484098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73ad280 0x55dde7771700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.440+0000 7fb40be00640 1 -- 172.21.15.27:0/4034484098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73ad280 msgr2=0x55dde7771700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:26.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.440+0000 7fb40be00640 1 --2- 172.21.15.27:0/4034484098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73ad280 0x55dde7771700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.440+0000 7fb40be00640 1 -- 172.21.15.27:0/4034484098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73cdf60 msgr2=0x55dde7771c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:26.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.440+0000 7fb40be00640 1 --2- 172.21.15.27:0/4034484098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73cdf60 0x55dde7771c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.440+0000 7fb40be00640 1 -- 172.21.15.27:0/4034484098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dde7772200 con 0x55dde73a50a0 2024-09-21T00:21:26.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.440+0000 7fb40be00640 1 --2- 172.21.15.27:0/4034484098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73a50a0 0x55dde779ae10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0096c0 tx=0x7fb3fc0cfda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d4e25ed36004156 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:26.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.440+0000 7fb3fa000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:21:26.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.441+0000 7fb3fa000640 1 -- 172.21.15.27:0/4034484098 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc049070 con 0x55dde73a50a0 2024-09-21T00:21:26.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.441+0000 7fb419c3e740 1 -- 172.21.15.27:0/4034484098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3f4004050 con 0x55dde73a50a0 2024-09-21T00:21:26.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.441+0000 7fb3fa000640 1 -- 172.21.15.27:0/4034484098 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fb3fc0ccd40 con 0x55dde73a50a0 2024-09-21T00:21:26.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.441+0000 7fb3fa000640 1 -- 172.21.15.27:0/4034484098 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb3fc07b030 con 0x55dde73a50a0 2024-09-21T00:21:26.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.441+0000 7fb419c3e740 10 client.25336 fetch_fsmap learned FSMap version 25 2024-09-21T00:21:26.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.441+0000 7fb419c3e740 1 -- 172.21.15.27:0/4034484098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dde77fda40 con 0x55dde73a50a0 2024-09-21T00:21:26.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.441+0000 7fb3fa000640 1 -- 172.21.15.27:0/4034484098 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f4004050 con 0x55dde73a50a0 2024-09-21T00:21:26.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.442+0000 7fb3fa000640 1 -- 172.21.15.27:0/4034484098 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb3fc0d34a0 con 0x55dde73a50a0 2024-09-21T00:21:26.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.442+0000 7fb419c3e740 10 client.25336 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T00:21:26.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.442+0000 7fb419c3e740 10 client.25336 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:21:26.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.442+0000 7fb419c3e740 20 client.25336 populate_metadata read hostname 'smithi027' 2024-09-21T00:21:26.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.442+0000 7fb419c3e740 1 --2- 172.21.15.27:0/4034484098 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x55dde7361c90 0x55dde7801030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:26.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.442+0000 7fb419c3e740 4 client.25336 mds_command: new command op to 24751 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:21:26.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.442+0000 7fb419c3e740 1 -- 172.21.15.27:0/4034484098 --> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] -- command(tid 0: {"prefix": "status"}) -- 0x55dde76e9750 con 0x55dde7361c90 2024-09-21T00:21:26.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.442+0000 7fb403400640 1 --2- 172.21.15.27:0/4034484098 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x55dde7361c90 0x55dde7801030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:26.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.443+0000 7fb3fa000640 10 client.25336 ms_handle_connect on v2:172.21.15.27:6840/378707351 2024-09-21T00:21:26.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.443+0000 7fb403400640 1 --2- 172.21.15.27:0/4034484098 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x55dde7361c90 0x55dde7801030 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=54e6f251b8c35f64 server_cookie=508b0bf3ef50a9bf in_seq=0 out_seq=0 2024-09-21T00:21:26.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.443+0000 7fb3fa000640 1 -- 172.21.15.27:0/4034484098 <== mds.1 v2:172.21.15.27:6840/378707351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55dde76e9750 con 0x55dde7361c90 2024-09-21T00:21:26.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.443+0000 7fb3fa000640 10 client.25336 handle_command_reply: tid=0 2024-09-21T00:21:26.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.443+0000 7fb419c3e740 1 client.25336 shutdown 2024-09-21T00:21:26.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.443+0000 7fb419c3e740 1 -- 172.21.15.27:0/4034484098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73a50a0 msgr2=0x55dde779ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:26.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.443+0000 7fb419c3e740 1 --2- 172.21.15.27:0/4034484098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73a50a0 0x55dde779ae10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0096c0 tx=0x7fb3fc0cfda0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.443+0000 7fb419c3e740 1 -- 172.21.15.27:0/4034484098 shutdown_connections 2024-09-21T00:21:26.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.443+0000 7fb419c3e740 1 --2- 172.21.15.27:0/4034484098 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x55dde7361c90 0x55dde7801030 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.443+0000 7fb419c3e740 1 --2- 172.21.15.27:0/4034484098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55dde73cdf60 0x55dde7771c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.443+0000 7fb419c3e740 1 --2- 172.21.15.27:0/4034484098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde73ad280 0x55dde7771700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.443+0000 7fb3fa000640 0 client.25336 ms_handle_reset on v2:172.21.15.27:6840/378707351 2024-09-21T00:21:26.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.443+0000 7fb419c3e740 1 --2- 172.21.15.27:0/4034484098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dde73a50a0 0x55dde779ae10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.443+0000 7fb419c3e740 1 -- 172.21.15.27:0/4034484098 >> 172.21.15.27:0/4034484098 conn(0x55dde73c2350 msgr2=0x55dde7780060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:26.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.444+0000 7fb419c3e740 1 -- 172.21.15.27:0/4034484098 shutdown_connections 2024-09-21T00:21:26.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.444+0000 7fb419c3e740 1 -- 172.21.15.27:0/4034484098 wait complete. 2024-09-21T00:21:26.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.444+0000 7fb419c3e740 20 client.25336 trim_cache size 0 max 16384 2024-09-21T00:21:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:21:26.452 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "9c21de84-77ad-11ef-baf5-efdc52797490", 2024-09-21T00:21:26.452 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-09-21T00:21:26.452 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24751, 2024-09-21T00:21:26.452 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T00:21:26.452 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T00:21:26.452 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T00:21:26.452 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 113.56265058699999, 2024-09-21T00:21:26.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 25, 2024-09-21T00:21:26.452 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T00:21:26.453 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T00:21:26.453 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 181.822569848 2024-09-21T00:21:26.453 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:21:26.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.444+0000 7fb413000640 1 -- 172.21.15.27:0/1708546763 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb3f00b6560 msgr2=0x7fb3f00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:26.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.444+0000 7fb413000640 1 --2- 172.21.15.27:0/1708546763 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb3f00b6560 0x7fb3f00b8a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb4040115a0 tx=0x7fb404009500 comp rx=0 tx=0).stop 2024-09-21T00:21:26.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.444+0000 7fb413000640 1 -- 172.21.15.27:0/1708546763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c10a1b0 msgr2=0x7fb40c1a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:26.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.444+0000 7fb413000640 1 --2- 172.21.15.27:0/1708546763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c10a1b0 0x7fb40c1a0b40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc005940 tx=0x7fb3fc0040c0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.445+0000 7fb413000640 1 -- 172.21.15.27:0/1708546763 shutdown_connections 2024-09-21T00:21:26.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.445+0000 7fb413000640 1 --2- 172.21.15.27:0/1708546763 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb3f00b6560 0x7fb3f00b8a70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.445+0000 7fb413000640 1 --2- 172.21.15.27:0/1708546763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb40c110440 0x7fb40c1a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.445+0000 7fb413000640 1 --2- 172.21.15.27:0/1708546763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40c10faf0 0x7fb40c1a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.445+0000 7fb413000640 1 --2- 172.21.15.27:0/1708546763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c10a1b0 0x7fb40c1a0b40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:26.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.445+0000 7fb413000640 1 -- 172.21.15.27:0/1708546763 >> 172.21.15.27:0/1708546763 conn(0x7fb40c0780e0 msgr2=0x7fb40c1085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:26.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.445+0000 7fb413000640 1 -- 172.21.15.27:0/1708546763 shutdown_connections 2024-09-21T00:21:26.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:26.445+0000 7fb413000640 1 -- 172.21.15.27:0/1708546763 wait complete. 2024-09-21T00:21:26.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:26 smithi089 ceph-mon[34481]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-21T00:21:26.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:26 smithi098 ceph-mon[34725]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-21T00:21:27.311 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T00:21:27.697 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:28.252 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:28 smithi027 ceph-mon[30745]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:21:28.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.436+0000 7f2a53800640 1 -- 172.21.15.27:0/1224186624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a4c1100c0 msgr2=0x7f2a4c112500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:28.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.436+0000 7f2a53800640 1 --2- 172.21.15.27:0/1224186624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a4c1100c0 0x7f2a4c112500 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009ca0 tx=0x7f2a4402a8c0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.436+0000 7f2a53800640 1 -- 172.21.15.27:0/1224186624 shutdown_connections 2024-09-21T00:21:28.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.436+0000 7f2a53800640 1 --2- 172.21.15.27:0/1224186624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a4c1100c0 0x7f2a4c112500 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.436+0000 7f2a53800640 1 --2- 172.21.15.27:0/1224186624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a4c076100 0x7f2a4c10fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.436+0000 7f2a53800640 1 --2- 172.21.15.27:0/1224186624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a4c0757b0 0x7f2a4c075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.436+0000 7f2a53800640 1 -- 172.21.15.27:0/1224186624 >> 172.21.15.27:0/1224186624 conn(0x7f2a4c07b850 msgr2=0x7f2a4c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:28.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.436+0000 7f2a53800640 1 -- 172.21.15.27:0/1224186624 shutdown_connections 2024-09-21T00:21:28.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.436+0000 7f2a53800640 1 -- 172.21.15.27:0/1224186624 wait complete. 2024-09-21T00:21:28.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.437+0000 7f2a53800640 1 Processor -- start 2024-09-21T00:21:28.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.437+0000 7f2a53800640 1 -- start start 2024-09-21T00:21:28.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.438+0000 7f2a53800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a4c0757b0 0x7f2a4c19c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.438+0000 7f2a53800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a4c076100 0x7f2a4c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.438+0000 7f2a53800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a4c1100c0 0x7f2a4c1a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.438+0000 7f2a53800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c115c40 con 0x7f2a4c1100c0 2024-09-21T00:21:28.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.438+0000 7f2a53800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c115ac0 con 0x7f2a4c076100 2024-09-21T00:21:28.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.438+0000 7f2a53800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c115dc0 con 0x7f2a4c0757b0 2024-09-21T00:21:28.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.438+0000 7f2a50e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a4c0757b0 0x7f2a4c19c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.438+0000 7f2a50e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a4c0757b0 0x7f2a4c19c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:57198/0 (socket says 172.21.15.27:57198) 2024-09-21T00:21:28.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.438+0000 7f2a50e00640 1 -- 172.21.15.27:0/1452436370 learned_addr learned my addr 172.21.15.27:0/1452436370 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:28.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.438+0000 7f2a51800640 1 --2- 172.21.15.27:0/1452436370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a4c1100c0 0x7f2a4c1a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.438+0000 7f2a43e00640 1 --2- 172.21.15.27:0/1452436370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a4c076100 0x7f2a4c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.439+0000 7f2a50e00640 1 -- 172.21.15.27:0/1452436370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a4c076100 msgr2=0x7f2a4c19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:28.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.439+0000 7f2a50e00640 1 --2- 172.21.15.27:0/1452436370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a4c076100 0x7f2a4c19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.439+0000 7f2a50e00640 1 -- 172.21.15.27:0/1452436370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a4c1100c0 msgr2=0x7f2a4c1a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:28.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.439+0000 7f2a50e00640 1 --2- 172.21.15.27:0/1452436370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a4c1100c0 0x7f2a4c1a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.439+0000 7f2a50e00640 1 -- 172.21.15.27:0/1452436370 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4c1a2540 con 0x7f2a4c0757b0 2024-09-21T00:21:28.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.439+0000 7f2a50e00640 1 --2- 172.21.15.27:0/1452436370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a4c0757b0 0x7f2a4c19c5d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00e9f0 tx=0x7f2a3c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:28.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.439+0000 7f2a4a000640 1 -- 172.21.15.27:0/1452436370 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c00cd80 con 0x7f2a4c0757b0 2024-09-21T00:21:28.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.439+0000 7f2a53800640 1 -- 172.21.15.27:0/1452436370 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4c1a2830 con 0x7f2a4c0757b0 2024-09-21T00:21:28.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.440+0000 7f2a4a000640 1 -- 172.21.15.27:0/1452436370 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a3c0108e0 con 0x7f2a4c0757b0 2024-09-21T00:21:28.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.440+0000 7f2a53800640 1 -- 172.21.15.27:0/1452436370 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a4c1a8f90 con 0x7f2a4c0757b0 2024-09-21T00:21:28.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.440+0000 7f2a4a000640 1 -- 172.21.15.27:0/1452436370 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c0188e0 con 0x7f2a4c0757b0 2024-09-21T00:21:28.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.443+0000 7f2a5a5c4740 1 Processor -- start 2024-09-21T00:21:28.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.443+0000 7f2a5a5c4740 1 -- start start 2024-09-21T00:21:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.443+0000 7f2a5a5c4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4f2f60 0x56155b899490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.443+0000 7f2a5a5c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 0x56155b8999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.443+0000 7f2a5a5c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4d2280 0x56155b899f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.443+0000 7f2a5a5c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56155b810d50 con 0x56155b4d2280 2024-09-21T00:21:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.443+0000 7f2a5a5c4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56155b800180 con 0x56155b4f2f60 2024-09-21T00:21:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.443+0000 7f2a5a5c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56155b803f90 con 0x56155b4ca0a0 2024-09-21T00:21:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.443+0000 7f2a4a000640 1 -- 172.21.15.27:0/1452436370 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f2a3c010430 con 0x7f2a4c0757b0 2024-09-21T00:21:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.444+0000 7f2a51800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4f2f60 0x56155b899490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.444+0000 7f2a43e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4d2280 0x56155b899f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.444+0000 7f2a51800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4f2f60 0x56155b899490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:33536/0 (socket says 172.21.15.27:33536) 2024-09-21T00:21:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.444+0000 7f2a51800640 1 -- 172.21.15.27:0/1733824463 learned_addr learned my addr 172.21.15.27:0/1733824463 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:28.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.444+0000 7f2a50e00640 1 --2- 172.21.15.27:0/1733824463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 0x56155b8999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.444+0000 7f2a51800640 1 -- 172.21.15.27:0/1733824463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 msgr2=0x56155b8999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.444+0000 7f2a51800640 1 --2- 172.21.15.27:0/1733824463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 0x56155b8999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.444+0000 7f2a51800640 1 -- 172.21.15.27:0/1733824463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4d2280 msgr2=0x56155b899f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.444+0000 7f2a51800640 1 --2- 172.21.15.27:0/1733824463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4d2280 0x56155b899f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.444+0000 7f2a51800640 1 -- 172.21.15.27:0/1733824463 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56155b7caa70 con 0x56155b4f2f60 2024-09-21T00:21:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.444+0000 7f2a43e00640 1 --2- 172.21.15.27:0/1733824463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4d2280 0x56155b899f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.445+0000 7f2a51800640 1 --2- 172.21.15.27:0/1733824463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4f2f60 0x56155b899490 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2a4402f410 tx=0x7f2a4402f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f835495683f2473d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.445+0000 7f2a42000640 1 -- 172.21.15.27:0/1733824463 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44031cf0 con 0x56155b4f2f60 2024-09-21T00:21:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.445+0000 7f2a42000640 1 -- 172.21.15.27:0/1733824463 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a44036030 con 0x56155b4f2f60 2024-09-21T00:21:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.445+0000 7f2a42000640 1 -- 172.21.15.27:0/1733824463 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a440042a0 con 0x56155b4f2f60 2024-09-21T00:21:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.445+0000 7f2a4a000640 1 --2- 172.21.15.27:0/1452436370 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2a300b65e0 0x7f2a300b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.446+0000 7f2a4a000640 1 -- 172.21.15.27:0/1452436370 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f2a3c0e7b30 con 0x7f2a4c0757b0 2024-09-21T00:21:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.446+0000 7f2a50e00640 1 --2- 172.21.15.27:0/1452436370 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2a300b65e0 0x7f2a300b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.446+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1733824463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4f2f60 msgr2=0x56155b899490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:28.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.446+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1733824463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4f2f60 0x56155b899490 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2a4402f410 tx=0x7f2a4402f440 comp rx=0 tx=0).stop 2024-09-21T00:21:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.446+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1733824463 shutdown_connections 2024-09-21T00:21:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.446+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1733824463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4d2280 0x56155b899f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.446+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1733824463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 0x56155b8999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.446+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1733824463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4f2f60 0x56155b899490 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.446+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1733824463 >> 172.21.15.27:0/1733824463 conn(0x56155b4e7350 msgr2=0x56155b897380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.446+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1733824463 shutdown_connections 2024-09-21T00:21:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.446+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1733824463 wait complete. 2024-09-21T00:21:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.447+0000 7f2a50e00640 1 --2- 172.21.15.27:0/1452436370 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2a300b65e0 0x7f2a300b8af0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0e7dd0 tx=0x7f2a3c000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.447+0000 7f2a5a5c4740 1 -- start start 2024-09-21T00:21:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.447+0000 7f2a5a5c4740 1 Processor -- start 2024-09-21T00:21:28.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.448+0000 7f2a5a5c4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4ca0a0 0x56155b8a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.448+0000 7f2a5a5c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4d2280 0x56155b8a5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.448+0000 7f2a5a5c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4f2f60 0x56155b8b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.448+0000 7f2a5a5c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56155b6615f0 con 0x56155b4f2f60 2024-09-21T00:21:28.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.448+0000 7f2a5a5c4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56155b79b460 con 0x56155b4ca0a0 2024-09-21T00:21:28.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.448+0000 7f2a5a5c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56155b8a7a90 con 0x56155b4d2280 2024-09-21T00:21:28.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.448+0000 7f2a5a5c4740 10 client.0 _ll_register_callbacks cb 0x56155b7ab210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:21:28.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.448+0000 7f2a51800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4f2f60 0x56155b8b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.448+0000 7f2a51800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4f2f60 0x56155b8b0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38784/0 (socket says 172.21.15.27:38784) 2024-09-21T00:21:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.448+0000 7f2a51800640 1 -- 172.21.15.27:0/1466918005 learned_addr learned my addr 172.21.15.27:0/1466918005 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.448+0000 7f2a43e00640 1 --2- 172.21.15.27:0/1466918005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4d2280 0x56155b8a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.448+0000 7f2a51800640 1 --2- 172.21.15.27:0/1466918005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4ca0a0 0x56155b8a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.449+0000 7f2a51800640 1 -- 172.21.15.27:0/1466918005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4d2280 msgr2=0x56155b8a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.449+0000 7f2a51800640 1 --2- 172.21.15.27:0/1466918005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4d2280 0x56155b8a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.449+0000 7f2a51800640 1 -- 172.21.15.27:0/1466918005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4ca0a0 msgr2=0x56155b8a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.449+0000 7f2a51800640 1 --2- 172.21.15.27:0/1466918005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4ca0a0 0x56155b8a4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.449+0000 7f2a51800640 1 -- 172.21.15.27:0/1466918005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56155b55e540 con 0x56155b4f2f60 2024-09-21T00:21:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.449+0000 7f2a51800640 1 --2- 172.21.15.27:0/1466918005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4f2f60 0x56155b8b0de0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009a60 tx=0x7f2a44003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7159752a8ad25a4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.449+0000 7f2a3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:21:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.449+0000 7f2a3aa00640 1 -- 172.21.15.27:0/1466918005 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4404a040 con 0x56155b4f2f60 2024-09-21T00:21:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.449+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1466918005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56155b79da70 con 0x56155b4f2f60 2024-09-21T00:21:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.449+0000 7f2a3aa00640 1 -- 172.21.15.27:0/1466918005 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f2a44004b40 con 0x56155b4f2f60 2024-09-21T00:21:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.450+0000 7f2a3aa00640 1 -- 172.21.15.27:0/1466918005 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2a44004d40 con 0x56155b4f2f60 2024-09-21T00:21:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.450+0000 7f2a5a5c4740 10 client.15348 fetch_fsmap learned FSMap version 25 2024-09-21T00:21:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.450+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1466918005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56155b7b86f0 con 0x56155b4f2f60 2024-09-21T00:21:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.450+0000 7f2a3aa00640 1 -- 172.21.15.27:0/1466918005 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56155b79da70 con 0x56155b4f2f60 2024-09-21T00:21:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.450+0000 7f2a3aa00640 1 -- 172.21.15.27:0/1466918005 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2a44050030 con 0x56155b4f2f60 2024-09-21T00:21:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.450+0000 7f2a5a5c4740 10 client.15348 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T00:21:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.450+0000 7f2a5a5c4740 10 client.15348 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T00:21:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.450+0000 7f2a5a5c4740 20 client.15348 populate_metadata read hostname 'smithi027' 2024-09-21T00:21:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.451+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1466918005 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56155b4b18e0 0x56155b8b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.451+0000 7f2a5a5c4740 4 client.15348 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:21:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.451+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1466918005 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56155b812ca0 con 0x56155b4b18e0 2024-09-21T00:21:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.451+0000 7f2a43e00640 1 --2- 172.21.15.27:0/1466918005 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56155b4b18e0 0x56155b8b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.451+0000 7f2a43e00640 1 --2- 172.21.15.27:0/1466918005 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56155b4b18e0 0x56155b8b70b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eeeac623334adc4 server_cookie=2a8764651383631e in_seq=0 out_seq=0 2024-09-21T00:21:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.451+0000 7f2a3aa00640 10 client.15348 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:21:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.452+0000 7f2a3aa00640 1 -- 172.21.15.27:0/1466918005 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56155b812ca0 con 0x56155b4b18e0 2024-09-21T00:21:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.452+0000 7f2a3aa00640 10 client.15348 handle_command_reply: tid=0 2024-09-21T00:21:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.452+0000 7f2a5a5c4740 1 client.15348 shutdown 2024-09-21T00:21:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.453+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1466918005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4f2f60 msgr2=0x56155b8b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.453+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1466918005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4f2f60 0x56155b8b0de0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009a60 tx=0x7f2a44003d70 comp rx=0 tx=0).stop 2024-09-21T00:21:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.453+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1466918005 shutdown_connections 2024-09-21T00:21:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.453+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1466918005 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56155b4b18e0 0x56155b8b70b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.453+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1466918005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4f2f60 0x56155b8b0de0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.453+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1466918005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4d2280 0x56155b8a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.453+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1466918005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4ca0a0 0x56155b8a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.453+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1466918005 >> 172.21.15.27:0/1466918005 conn(0x56155b4e7350 msgr2=0x56155b89c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:28.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.453+0000 7f2a3aa00640 0 client.15348 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:21:28.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.453+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1466918005 shutdown_connections 2024-09-21T00:21:28.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.453+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1466918005 wait complete. 2024-09-21T00:21:28.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.453+0000 7f2a5a5c4740 20 client.15348 trim_cache size 0 max 16384 2024-09-21T00:21:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.473+0000 7f2a5a5c4740 1 Processor -- start 2024-09-21T00:21:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.473+0000 7f2a5a5c4740 1 -- start start 2024-09-21T00:21:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.473+0000 7f2a5a5c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 0x56155b89ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.473+0000 7f2a5a5c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4d2280 0x56155b8a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.473+0000 7f2a5a5c4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4f2f60 0x56155b8b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.473+0000 7f2a5a5c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56155b8b53f0 con 0x56155b4d2280 2024-09-21T00:21:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.473+0000 7f2a5a5c4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56155b8b5ac0 con 0x56155b4f2f60 2024-09-21T00:21:28.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.473+0000 7f2a5a5c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56155b8b6190 con 0x56155b4ca0a0 2024-09-21T00:21:28.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.473+0000 7f2a43e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4d2280 0x56155b8a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.473+0000 7f2a51800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 0x56155b89ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.473+0000 7f2a51800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 0x56155b89ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:57222/0 (socket says 172.21.15.27:57222) 2024-09-21T00:21:28.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.473+0000 7f2a51800640 1 -- 172.21.15.27:0/527566181 learned_addr learned my addr 172.21.15.27:0/527566181 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:28.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.474+0000 7f2a51800640 1 --2- 172.21.15.27:0/527566181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4f2f60 0x56155b8b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.474+0000 7f2a51800640 1 -- 172.21.15.27:0/527566181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4f2f60 msgr2=0x56155b8b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:28.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.474+0000 7f2a51800640 1 --2- 172.21.15.27:0/527566181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4f2f60 0x56155b8b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.474+0000 7f2a51800640 1 -- 172.21.15.27:0/527566181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4d2280 msgr2=0x56155b8a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:28.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.474+0000 7f2a51800640 1 --2- 172.21.15.27:0/527566181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4d2280 0x56155b8a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.474+0000 7f2a51800640 1 -- 172.21.15.27:0/527566181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56155b8a77a0 con 0x56155b4ca0a0 2024-09-21T00:21:28.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.474+0000 7f2a51800640 1 --2- 172.21.15.27:0/527566181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 0x56155b89ffa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2a44031910 tx=0x7f2a44033040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf8b80b34131e00e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:28.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.474+0000 7f2a41600640 1 -- 172.21.15.27:0/527566181 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4402f3a0 con 0x56155b4ca0a0 2024-09-21T00:21:28.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.474+0000 7f2a41600640 1 -- 172.21.15.27:0/527566181 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a44036070 con 0x56155b4ca0a0 2024-09-21T00:21:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.474+0000 7f2a41600640 1 -- 172.21.15.27:0/527566181 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4402f730 con 0x56155b4ca0a0 2024-09-21T00:21:28.481 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:28 smithi089 ceph-mon[34481]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:21:28.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.475+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/527566181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 msgr2=0x56155b89ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:28.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.475+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/527566181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 0x56155b89ffa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2a44031910 tx=0x7f2a44033040 comp rx=0 tx=0).stop 2024-09-21T00:21:28.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.475+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/527566181 shutdown_connections 2024-09-21T00:21:28.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.475+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/527566181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4f2f60 0x56155b8b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.475+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/527566181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4d2280 0x56155b8a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.475+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/527566181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 0x56155b89ffa0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.475+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/527566181 >> 172.21.15.27:0/527566181 conn(0x56155b4e7350 msgr2=0x56155b892010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.475+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/527566181 shutdown_connections 2024-09-21T00:21:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.475+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/527566181 wait complete. 2024-09-21T00:21:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.476+0000 7f2a5a5c4740 1 -- start start 2024-09-21T00:21:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.476+0000 7f2a5a5c4740 1 Processor -- start 2024-09-21T00:21:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a5a5c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 0x56155b8b9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a5a5c4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4d2280 0x56155b8ba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a5a5c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4f2f60 0x56155b8bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a5a5c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56155b8b6f30 con 0x56155b4f2f60 2024-09-21T00:21:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a5a5c4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56155b8b6860 con 0x56155b4d2280 2024-09-21T00:21:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a5a5c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56155b8b7600 con 0x56155b4ca0a0 2024-09-21T00:21:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a5a5c4740 10 client.0 _ll_register_callbacks cb 0x7f2a44003b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:21:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a43e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4d2280 0x56155b8ba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a43e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4d2280 0x56155b8ba050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:33568/0 (socket says 172.21.15.27:33568) 2024-09-21T00:21:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a51800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4f2f60 0x56155b8bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a43e00640 1 -- 172.21.15.27:0/1278254653 learned_addr learned my addr 172.21.15.27:0/1278254653 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a51800640 1 --2- 172.21.15.27:0/1278254653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 0x56155b8b9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a43e00640 1 -- 172.21.15.27:0/1278254653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 msgr2=0x56155b8b9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a43e00640 1 --2- 172.21.15.27:0/1278254653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 0x56155b8b9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a43e00640 1 -- 172.21.15.27:0/1278254653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4f2f60 msgr2=0x56155b8bdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a43e00640 1 --2- 172.21.15.27:0/1278254653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4f2f60 0x56155b8bdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.477+0000 7f2a43e00640 1 -- 172.21.15.27:0/1278254653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56155b8be170 con 0x56155b4d2280 2024-09-21T00:21:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.478+0000 7f2a43e00640 1 --2- 172.21.15.27:0/1278254653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4d2280 0x56155b8ba050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c06c760 tx=0x7f2a34012a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1df7d12a79ab962d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.478+0000 7f2a3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:21:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.478+0000 7f2a3aa00640 1 -- 172.21.15.27:0/1278254653 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34004030 con 0x56155b4d2280 2024-09-21T00:21:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.478+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1278254653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a44036070 con 0x56155b4d2280 2024-09-21T00:21:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.478+0000 7f2a3aa00640 1 -- 172.21.15.27:0/1278254653 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f2a340041d0 con 0x56155b4d2280 2024-09-21T00:21:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.479+0000 7f2a3aa00640 1 -- 172.21.15.27:0/1278254653 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2a34031050 con 0x56155b4d2280 2024-09-21T00:21:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.479+0000 7f2a5a5c4740 10 client.25367 fetch_fsmap learned FSMap version 25 2024-09-21T00:21:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.479+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1278254653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56155b8bf5e0 con 0x56155b4d2280 2024-09-21T00:21:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.479+0000 7f2a3aa00640 1 -- 172.21.15.27:0/1278254653 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a44036070 con 0x56155b4d2280 2024-09-21T00:21:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.479+0000 7f2a3aa00640 1 -- 172.21.15.27:0/1278254653 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2a34004a70 con 0x56155b4d2280 2024-09-21T00:21:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.479+0000 7f2a5a5c4740 10 client.25367 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T00:21:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.479+0000 7f2a5a5c4740 10 client.25367 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T00:21:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.479+0000 7f2a5a5c4740 20 client.25367 populate_metadata read hostname 'smithi027' 2024-09-21T00:21:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.479+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1278254653 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56155b486c90 0x56155b9251d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.479+0000 7f2a5a5c4740 4 client.25367 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:21:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.479+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1278254653 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "status"}) -- 0x56155b80e750 con 0x56155b486c90 2024-09-21T00:21:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.480+0000 7f2a51800640 1 --2- 172.21.15.27:0/1278254653 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56155b486c90 0x56155b9251d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.480+0000 7f2a51800640 1 --2- 172.21.15.27:0/1278254653 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56155b486c90 0x56155b9251d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30d69871ee0b1c59 server_cookie=294534c15800b06b in_seq=0 out_seq=0 2024-09-21T00:21:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.480+0000 7f2a3aa00640 10 client.25367 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:21:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.480+0000 7f2a3aa00640 1 -- 172.21.15.27:0/1278254653 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56155b80e750 con 0x56155b486c90 2024-09-21T00:21:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.480+0000 7f2a3aa00640 10 client.25367 handle_command_reply: tid=0 2024-09-21T00:21:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.481+0000 7f2a5a5c4740 1 client.25367 shutdown 2024-09-21T00:21:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.481+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1278254653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4d2280 msgr2=0x56155b8ba050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.481+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1278254653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4d2280 0x56155b8ba050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c06c760 tx=0x7f2a34012a90 comp rx=0 tx=0).stop 2024-09-21T00:21:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.481+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1278254653 shutdown_connections 2024-09-21T00:21:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.481+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1278254653 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56155b486c90 0x56155b9251d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.481+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1278254653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56155b4f2f60 0x56155b8bdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.481+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1278254653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56155b4d2280 0x56155b8ba050 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.481+0000 7f2a5a5c4740 1 --2- 172.21.15.27:0/1278254653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56155b4ca0a0 0x56155b8b9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.481+0000 7f2a3aa00640 0 client.25367 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:21:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.481+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1278254653 >> 172.21.15.27:0/1278254653 conn(0x56155b4e7350 msgr2=0x56155b89ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.481+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1278254653 shutdown_connections 2024-09-21T00:21:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.481+0000 7f2a5a5c4740 1 -- 172.21.15.27:0/1278254653 wait complete. 2024-09-21T00:21:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.481+0000 7f2a5a5c4740 20 client.25367 trim_cache size 0 max 16384 2024-09-21T00:21:28.489 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:21:28.489 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "9c21de84-77ad-11ef-baf5-efdc52797490", 2024-09-21T00:21:28.489 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-21T00:21:28.489 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24761, 2024-09-21T00:21:28.489 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T00:21:28.489 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T00:21:28.489 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T00:21:28.489 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 167.819538292, 2024-09-21T00:21:28.489 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 25, 2024-09-21T00:21:28.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T00:21:28.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T00:21:28.490 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 181.28679161299999 2024-09-21T00:21:28.490 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:21:28.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.482+0000 7f2a53800640 1 -- 172.21.15.27:0/1452436370 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2a300b65e0 msgr2=0x7f2a300b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:28.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.482+0000 7f2a53800640 1 --2- 172.21.15.27:0/1452436370 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2a300b65e0 0x7f2a300b8af0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0e7dd0 tx=0x7f2a3c000bc0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.482+0000 7f2a53800640 1 -- 172.21.15.27:0/1452436370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a4c0757b0 msgr2=0x7f2a4c19c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:28.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.482+0000 7f2a53800640 1 --2- 172.21.15.27:0/1452436370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a4c0757b0 0x7f2a4c19c5d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00e9f0 tx=0x7f2a3c00eeb0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.482+0000 7f2a53800640 1 -- 172.21.15.27:0/1452436370 shutdown_connections 2024-09-21T00:21:28.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.482+0000 7f2a53800640 1 --2- 172.21.15.27:0/1452436370 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2a300b65e0 0x7f2a300b8af0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.482+0000 7f2a53800640 1 --2- 172.21.15.27:0/1452436370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a4c1100c0 0x7f2a4c1a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.482+0000 7f2a53800640 1 --2- 172.21.15.27:0/1452436370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a4c076100 0x7f2a4c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.482+0000 7f2a53800640 1 --2- 172.21.15.27:0/1452436370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a4c0757b0 0x7f2a4c19c5d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:28.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.482+0000 7f2a53800640 1 -- 172.21.15.27:0/1452436370 >> 172.21.15.27:0/1452436370 conn(0x7f2a4c07b850 msgr2=0x7f2a4c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:28.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.482+0000 7f2a53800640 1 -- 172.21.15.27:0/1452436370 shutdown_connections 2024-09-21T00:21:28.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:28.482+0000 7f2a53800640 1 -- 172.21.15.27:0/1452436370 wait complete. 2024-09-21T00:21:28.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:28 smithi098 ceph-mon[34725]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:21:29.386 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T00:21:29.404 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T00:21:29.404 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T00:21:29.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:21:29.404 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-21T00:21:29.441 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-21T00:21:29.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:21:29.442 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T00:21:29.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:21:29.541 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-09-21T00:21:29.847 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-21T00:21:29.848 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T00:21:29.848 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T00:21:29.848 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T00:21:30.236 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:30.375 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:30 smithi027 ceph-mon[30745]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:21:30.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:30 smithi089 ceph-mon[34481]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:21:30.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:30 smithi098 ceph-mon[34725]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:21:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.888+0000 7f8fb5000640 1 -- 172.21.15.27:0/1631878897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb0105d80 msgr2=0x7f8fb0106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.888+0000 7f8fb5000640 1 --2- 172.21.15.27:0/1631878897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb0105d80 0x7f8fb0106190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009900 tx=0x7f8fa002a400 comp rx=0 tx=0).stop 2024-09-21T00:21:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.888+0000 7f8fb5000640 1 -- 172.21.15.27:0/1631878897 shutdown_connections 2024-09-21T00:21:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.888+0000 7f8fb5000640 1 --2- 172.21.15.27:0/1631878897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fb010c010 0x7f8fb0112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.888+0000 7f8fb5000640 1 --2- 172.21.15.27:0/1631878897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fb010b6c0 0x7f8fb010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.888+0000 7f8fb5000640 1 --2- 172.21.15.27:0/1631878897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb0105d80 0x7f8fb0106190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.888+0000 7f8fb5000640 1 -- 172.21.15.27:0/1631878897 >> 172.21.15.27:0/1631878897 conn(0x7f8fb0101810 msgr2=0x7f8fb0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.888+0000 7f8fb5000640 1 -- 172.21.15.27:0/1631878897 shutdown_connections 2024-09-21T00:21:30.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.888+0000 7f8fb5000640 1 -- 172.21.15.27:0/1631878897 wait complete. 2024-09-21T00:21:30.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.889+0000 7f8fb5000640 1 Processor -- start 2024-09-21T00:21:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.890+0000 7f8fb5000640 1 -- start start 2024-09-21T00:21:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.890+0000 7f8fb5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fb0105d80 0x7f8fb019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.890+0000 7f8fb5000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb010b6c0 0x7f8fb019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.890+0000 7f8fb5000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fb010c010 0x7f8fb01a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.890+0000 7f8fb5000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0115d20 con 0x7f8fb0105d80 2024-09-21T00:21:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.890+0000 7f8fb5000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0115ba0 con 0x7f8fb010b6c0 2024-09-21T00:21:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.890+0000 7f8fb5000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0115ea0 con 0x7f8fb010c010 2024-09-21T00:21:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.890+0000 7f8fade00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fb0105d80 0x7f8fb019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.890+0000 7f8fade00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fb0105d80 0x7f8fb019c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38824/0 (socket says 172.21.15.27:38824) 2024-09-21T00:21:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.890+0000 7f8fade00640 1 -- 172.21.15.27:0/2614193072 learned_addr learned my addr 172.21.15.27:0/2614193072 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.890+0000 7f8fa7e00640 1 --2- 172.21.15.27:0/2614193072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb010b6c0 0x7f8fb019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.890+0000 7f8fae800640 1 --2- 172.21.15.27:0/2614193072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fb010c010 0x7f8fb01a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.891+0000 7f8fade00640 1 -- 172.21.15.27:0/2614193072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fb010c010 msgr2=0x7f8fb01a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.891+0000 7f8fade00640 1 --2- 172.21.15.27:0/2614193072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fb010c010 0x7f8fb01a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.891+0000 7f8fade00640 1 -- 172.21.15.27:0/2614193072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb010b6c0 msgr2=0x7f8fb019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.891+0000 7f8fade00640 1 --2- 172.21.15.27:0/2614193072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb010b6c0 0x7f8fb019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.891+0000 7f8fade00640 1 -- 172.21.15.27:0/2614193072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb01a2620 con 0x7f8fb0105d80 2024-09-21T00:21:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.891+0000 7f8fade00640 1 --2- 172.21.15.27:0/2614193072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fb0105d80 0x7f8fb019c6b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0005ab0 tx=0x7f8fa0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.891+0000 7f8fa6a00640 1 -- 172.21.15.27:0/2614193072 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa003f070 con 0x7f8fb0105d80 2024-09-21T00:21:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.892+0000 7f8fa6a00640 1 -- 172.21.15.27:0/2614193072 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fa0031d60 con 0x7f8fb0105d80 2024-09-21T00:21:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.892+0000 7f8fb5000640 1 -- 172.21.15.27:0/2614193072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb01a28b0 con 0x7f8fb0105d80 2024-09-21T00:21:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.892+0000 7f8fa6a00640 1 -- 172.21.15.27:0/2614193072 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0035030 con 0x7f8fb0105d80 2024-09-21T00:21:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.892+0000 7f8fb5000640 1 -- 172.21.15.27:0/2614193072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb01a9070 con 0x7f8fb0105d80 2024-09-21T00:21:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.893+0000 7f8fb5000640 1 -- 172.21.15.27:0/2614193072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb0106190 con 0x7f8fb0105d80 2024-09-21T00:21:30.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.895+0000 7f8fa6a00640 1 -- 172.21.15.27:0/2614193072 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f8fa0042070 con 0x7f8fb0105d80 2024-09-21T00:21:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.897+0000 7f8fa6a00640 1 --2- 172.21.15.27:0/2614193072 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8f940b6560 0x7f8f940b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.897+0000 7f8fa6a00640 1 -- 172.21.15.27:0/2614193072 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f8fa0107fe0 con 0x7f8fb0105d80 2024-09-21T00:21:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.897+0000 7f8fa7e00640 1 --2- 172.21.15.27:0/2614193072 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8f940b6560 0x7f8f940b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:30.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.898+0000 7f8fa7e00640 1 --2- 172.21.15.27:0/2614193072 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8f940b6560 0x7f8f940b8a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8fb010b190 tx=0x7f8f9800a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:30.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:30.901+0000 7f8fa6a00640 1 -- 172.21.15.27:0/2614193072 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8fa00d27a0 con 0x7f8fb0105d80 2024-09-21T00:21:31.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:31.139+0000 7f8fb5000640 1 -- 172.21.15.27:0/2614193072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8fb010de10 con 0x7f8fb0105d80 2024-09-21T00:21:31.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:31.140+0000 7f8fa6a00640 1 -- 172.21.15.27:0/2614193072 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+88 (secure 0 0 0) 0x7f8fa00d76e0 con 0x7f8fb0105d80 2024-09-21T00:21:31.141 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T00:21:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:31.142+0000 7f8fb5000640 1 -- 172.21.15.27:0/2614193072 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8f940b6560 msgr2=0x7f8f940b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:31.142+0000 7f8fb5000640 1 --2- 172.21.15.27:0/2614193072 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8f940b6560 0x7f8f940b8a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8fb010b190 tx=0x7f8f9800a2d0 comp rx=0 tx=0).stop 2024-09-21T00:21:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:31.142+0000 7f8fb5000640 1 -- 172.21.15.27:0/2614193072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fb0105d80 msgr2=0x7f8fb019c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:31.142+0000 7f8fb5000640 1 --2- 172.21.15.27:0/2614193072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fb0105d80 0x7f8fb019c6b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0005ab0 tx=0x7f8fa0003c80 comp rx=0 tx=0).stop 2024-09-21T00:21:31.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:31.143+0000 7f8fb5000640 1 -- 172.21.15.27:0/2614193072 shutdown_connections 2024-09-21T00:21:31.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:31.143+0000 7f8fb5000640 1 --2- 172.21.15.27:0/2614193072 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8f940b6560 0x7f8f940b8a70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:31.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:31.143+0000 7f8fb5000640 1 --2- 172.21.15.27:0/2614193072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fb010c010 0x7f8fb01a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:31.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:31.143+0000 7f8fb5000640 1 --2- 172.21.15.27:0/2614193072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb010b6c0 0x7f8fb019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:31.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:31.143+0000 7f8fb5000640 1 --2- 172.21.15.27:0/2614193072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fb0105d80 0x7f8fb019c6b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:31.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:31.143+0000 7f8fb5000640 1 -- 172.21.15.27:0/2614193072 >> 172.21.15.27:0/2614193072 conn(0x7f8fb0101810 msgr2=0x7f8fb0103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:31.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:31.143+0000 7f8fb5000640 1 -- 172.21.15.27:0/2614193072 shutdown_connections 2024-09-21T00:21:31.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:31.143+0000 7f8fb5000640 1 -- 172.21.15.27:0/2614193072 wait complete. 2024-09-21T00:21:32.015 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T00:21:32.016 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:21:32.016 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-21T00:21:32.016 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T00:21:32.016 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T00:21:32.016 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T00:21:32.016 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T00:21:32.016 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:21:32.016 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:21:32.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:21:32.017 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-21T00:21:32.054 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:21:32.054 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:21:32.054 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:21:32.054 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:21:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-21T00:21:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:21:32.055 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:21:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:21:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-21T00:21:32.055 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:21:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:21:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-21T00:21:32.055 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:21:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-21T00:21:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-21T00:21:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T00:21:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41990sec preferred_lft 41990sec 2024-09-21T00:21:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-21T00:21:32.056 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:21:32.056 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:21:32.056 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-21T00:21:32.056 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-21T00:21:32.056 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T00:21:32.056 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:21:32.056 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T00:21:32.056 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2024-09-21T00:21:32.056 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2024-09-21T00:21:32.056 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2024-09-21T00:21:32.056 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T00:21:32.057 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T00:21:32.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:21:32.228 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:32 smithi027 ceph-mon[30745]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:21:32.228 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:32 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2614193072' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T00:21:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:21:32.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:21:32.287 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:21:32.343 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-21T00:21:32.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:21:32.347 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-21T00:21:32.384 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-21T00:21:32.385 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:21:32.385 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T00:21:32.385 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T00:21:32.385 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:21:32.385 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:21:32.385 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T00:21:32.385 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T00:21:32.385 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T00:21:32.385 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T00:21:32.386 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T00:21:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:32 smithi089 ceph-mon[34481]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:21:32.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:32 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2614193072' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T00:21:32.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:32 smithi098 ceph-mon[34725]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:21:32.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:32 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2614193072' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T00:21:32.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:21:32.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:21:32.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:21:32.588 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-21T00:21:32.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:21:32.625 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-09-21T00:21:32.701 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:21:32.701 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T00:21:32.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:21:32.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T00:21:32.701 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T00:21:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:21:32.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:21:32.883 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T00:21:32.883 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:21:32.883 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T00:21:32.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T00:21:32.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T00:21:32.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T00:21:32.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T00:21:32.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T00:21:32.883 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T00:21:32.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:21:33.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:21:33.108 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:21:33.108 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T00:21:33.108 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-21T00:21:33.108 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T00:21:33.109 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T00:21:33.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:21:33.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:21:33.280 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:21:33.280 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T00:21:33.316 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:21:33.317 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:21:33.317 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:21:33.391 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T00:21:33.391 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T00:21:33.391 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/494a9dee-140a-4837-be70-199fec6a493d 2024-09-21T00:21:33.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:21:33.391 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/494a9dee-140a-4837-be70-199fec6a493d /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-21T00:21:33.596 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-21T00:21:33.596 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-21T00:21:34.203 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:34 smithi027 ceph-mon[30745]: pgmap v382: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:21:34.224 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-21T00:21:34.225 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-21T00:21:34.225 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:6789,172.21.15.89:6789,172.21.15.98:6789" 2024-09-21T00:21:34.225 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@9c21de84-77ad-11ef-baf5-efdc52797490.cephfs=/volumes/qa/sv_1/494a9dee-140a-4837-be70-199fec6a493d 2024-09-21T00:21:34.225 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.27:6789/172.21.15.89:6789/172.21.15.98:6789" will pass to kernel 2024-09-21T00:21:34.227 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T00:21:34.227 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:21:34.227 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:21:34.227 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:21:34.227 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:21:34.230 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:21:34.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:21:34.230 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:21:34.291 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:21:34.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:21:34.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:21:34.386 INFO:teuthology.orchestra.run.smithi027.stdout:client25376 2024-09-21T00:21:34.387 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:21:34.387 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:21:34.424 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:21:34.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:34 smithi089 ceph-mon[34481]: pgmap v382: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:21:34.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:34 smithi098 ceph-mon[34725]: pgmap v382: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:21:34.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:21:34.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:21:34.558 INFO:teuthology.orchestra.run.smithi027.stdout:client25376 2024-09-21T00:21:34.558 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:21:34.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:21:34.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:21:34.650 INFO:teuthology.orchestra.run.smithi027.stdout:client25376 2024-09-21T00:21:34.650 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9c21de84-77ad-11ef-baf5-efdc52797490.client25376/status') 2024-09-21T00:21:34.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:21:34.755 INFO:teuthology.orchestra.run.smithi027.stdout:instance: client.25376 (3)172.21.15.27:0/597024777 2024-09-21T00:21:34.755 INFO:teuthology.orchestra.run.smithi027.stdout:blocklisted: false 2024-09-21T00:21:34.755 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records in 2024-09-21T00:21:34.756 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records out 2024-09-21T00:21:34.756 INFO:teuthology.orchestra.run.smithi027.stderr:70 bytes copied, 5.9137e-05 s, 1.2 MB/s 2024-09-21T00:21:34.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:21:34.767 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T00:21:34.782 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T00:21:34.782 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T00:21:34.782 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-21T00:21:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.012+0000 7f5550c00640 1 -- 172.21.15.27:0/3392592309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f554c10bd90 msgr2=0x7f554c1125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.012+0000 7f5550c00640 1 --2- 172.21.15.27:0/3392592309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f554c10bd90 0x7f554c1125c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5544009ca0 tx=0x7f554402a8c0 comp rx=0 tx=0).stop 2024-09-21T00:21:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.013+0000 7f5550c00640 1 -- 172.21.15.27:0/3392592309 shutdown_connections 2024-09-21T00:21:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.013+0000 7f5550c00640 1 --2- 172.21.15.27:0/3392592309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f554c10bd90 0x7f554c1125c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.013+0000 7f5550c00640 1 --2- 172.21.15.27:0/3392592309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f554c10b440 0x7f554c10b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.013+0000 7f5550c00640 1 --2- 172.21.15.27:0/3392592309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f554c105b10 0x7f554c105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.013+0000 7f5550c00640 1 -- 172.21.15.27:0/3392592309 >> 172.21.15.27:0/3392592309 conn(0x7f554c101510 msgr2=0x7f554c103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.013+0000 7f5550c00640 1 -- 172.21.15.27:0/3392592309 shutdown_connections 2024-09-21T00:21:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.013+0000 7f5550c00640 1 -- 172.21.15.27:0/3392592309 wait complete. 2024-09-21T00:21:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.014+0000 7f5550c00640 1 Processor -- start 2024-09-21T00:21:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.014+0000 7f5550c00640 1 -- start start 2024-09-21T00:21:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.014+0000 7f5550c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f554c105b10 0x7f554c19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.014+0000 7f5550c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f554c10b440 0x7f554c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.014+0000 7f5550c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f554c10bd90 0x7f554c1a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.014+0000 7f5550c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f554c115a50 con 0x7f554c105b10 2024-09-21T00:21:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.014+0000 7f5550c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f554c1158d0 con 0x7f554c10b440 2024-09-21T00:21:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.014+0000 7f5550c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f554c115bd0 con 0x7f554c10bd90 2024-09-21T00:21:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.014+0000 7f554b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f554c105b10 0x7f554c19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.015+0000 7f554b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f554c105b10 0x7f554c19c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50360/0 (socket says 172.21.15.27:50360) 2024-09-21T00:21:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.015+0000 7f554b400640 1 -- 172.21.15.27:0/2447835470 learned_addr learned my addr 172.21.15.27:0/2447835470 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:35.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.015+0000 7f5543e00640 1 --2- 172.21.15.27:0/2447835470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f554c10b440 0x7f554c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:35.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.015+0000 7f554be00640 1 --2- 172.21.15.27:0/2447835470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f554c10bd90 0x7f554c1a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:35.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.015+0000 7f554b400640 1 -- 172.21.15.27:0/2447835470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f554c10bd90 msgr2=0x7f554c1a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:35.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.015+0000 7f554b400640 1 --2- 172.21.15.27:0/2447835470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f554c10bd90 0x7f554c1a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:35.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.015+0000 7f554b400640 1 -- 172.21.15.27:0/2447835470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f554c10b440 msgr2=0x7f554c19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:35.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.015+0000 7f554b400640 1 --2- 172.21.15.27:0/2447835470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f554c10b440 0x7f554c19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:35.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.015+0000 7f554b400640 1 -- 172.21.15.27:0/2447835470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554c1a2320 con 0x7f554c105b10 2024-09-21T00:21:35.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.015+0000 7f554b400640 1 --2- 172.21.15.27:0/2447835470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f554c105b10 0x7f554c19c3f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f553c00e9f0 tx=0x7f553c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:35.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.016+0000 7f5548c00640 1 -- 172.21.15.27:0/2447835470 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c00cd80 con 0x7f554c105b10 2024-09-21T00:21:35.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.016+0000 7f5548c00640 1 -- 172.21.15.27:0/2447835470 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f553c0108e0 con 0x7f554c105b10 2024-09-21T00:21:35.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.016+0000 7f5548c00640 1 -- 172.21.15.27:0/2447835470 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c018980 con 0x7f554c105b10 2024-09-21T00:21:35.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.016+0000 7f5550c00640 1 -- 172.21.15.27:0/2447835470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554c1a2610 con 0x7f554c105b10 2024-09-21T00:21:35.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.016+0000 7f5550c00640 1 -- 172.21.15.27:0/2447835470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f554c1a8da0 con 0x7f554c105b10 2024-09-21T00:21:35.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.017+0000 7f5550c00640 1 -- 172.21.15.27:0/2447835470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f554c105f20 con 0x7f554c105b10 2024-09-21T00:21:35.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.019+0000 7f5548c00640 1 -- 172.21.15.27:0/2447835470 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f553c010430 con 0x7f554c105b10 2024-09-21T00:21:35.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.022+0000 7f5548c00640 1 --2- 172.21.15.27:0/2447835470 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f55300b6550 0x7f55300b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:35.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.022+0000 7f5548c00640 1 -- 172.21.15.27:0/2447835470 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f553c0e7ff0 con 0x7f554c105b10 2024-09-21T00:21:35.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.022+0000 7f5543e00640 1 --2- 172.21.15.27:0/2447835470 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f55300b6550 0x7f55300b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:35.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.023+0000 7f5543e00640 1 --2- 172.21.15.27:0/2447835470 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f55300b6550 0x7f55300b8a60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f554c104cf0 tx=0x7f5534005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:35.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.024+0000 7f5548c00640 1 -- 172.21.15.27:0/2447835470 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f553c0b27b0 con 0x7f554c105b10 2024-09-21T00:21:35.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:35.263+0000 7f5550c00640 1 -- 172.21.15.27:0/2447835470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f554c19d510 con 0x7f554c105b10 2024-09-21T00:21:36.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.086+0000 7f5548c00640 1 -- 172.21.15.27:0/2447835470 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7f553c0b76f0 con 0x7f554c105b10 2024-09-21T00:21:36.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.090+0000 7f5550c00640 1 -- 172.21.15.27:0/2447835470 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f55300b6550 msgr2=0x7f55300b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:36.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.090+0000 7f5550c00640 1 --2- 172.21.15.27:0/2447835470 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f55300b6550 0x7f55300b8a60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f554c104cf0 tx=0x7f5534005f90 comp rx=0 tx=0).stop 2024-09-21T00:21:36.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.090+0000 7f5550c00640 1 -- 172.21.15.27:0/2447835470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f554c105b10 msgr2=0x7f554c19c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:36.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.090+0000 7f5550c00640 1 --2- 172.21.15.27:0/2447835470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f554c105b10 0x7f554c19c3f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f553c00e9f0 tx=0x7f553c00eeb0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.090+0000 7f5550c00640 1 -- 172.21.15.27:0/2447835470 shutdown_connections 2024-09-21T00:21:36.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.090+0000 7f5550c00640 1 --2- 172.21.15.27:0/2447835470 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f55300b6550 0x7f55300b8a60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.090+0000 7f5550c00640 1 --2- 172.21.15.27:0/2447835470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f554c10bd90 0x7f554c1a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.090+0000 7f5550c00640 1 --2- 172.21.15.27:0/2447835470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f554c10b440 0x7f554c19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.090+0000 7f5550c00640 1 --2- 172.21.15.27:0/2447835470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f554c105b10 0x7f554c19c3f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.090+0000 7f5550c00640 1 -- 172.21.15.27:0/2447835470 >> 172.21.15.27:0/2447835470 conn(0x7f554c101510 msgr2=0x7f554c10a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:36.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.090+0000 7f5550c00640 1 -- 172.21.15.27:0/2447835470 shutdown_connections 2024-09-21T00:21:36.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.090+0000 7f5550c00640 1 -- 172.21.15.27:0/2447835470 wait complete. 2024-09-21T00:21:36.120 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T00:21:36.137 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T00:21:36.166 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T00:21:36.167 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T00:21:36.167 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-21T00:21:36.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.402+0000 7f028e800640 1 -- 172.21.15.27:0/927395132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0288110020 msgr2=0x7f0288112480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:36.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.402+0000 7f028e800640 1 --2- 172.21.15.27:0/927395132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0288110020 0x7f0288112480 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0280009ca0 tx=0x7f028002a8c0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.402+0000 7f028e800640 1 -- 172.21.15.27:0/927395132 shutdown_connections 2024-09-21T00:21:36.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.402+0000 7f028e800640 1 --2- 172.21.15.27:0/927395132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0288110020 0x7f0288112480 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.402+0000 7f028e800640 1 --2- 172.21.15.27:0/927395132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0288076100 0x7f028810fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.402+0000 7f028e800640 1 --2- 172.21.15.27:0/927395132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02880757b0 0x7f0288075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.402+0000 7f028e800640 1 -- 172.21.15.27:0/927395132 >> 172.21.15.27:0/927395132 conn(0x7f02880ff4d0 msgr2=0x7f0288101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:36.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.402+0000 7f028e800640 1 -- 172.21.15.27:0/927395132 shutdown_connections 2024-09-21T00:21:36.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.402+0000 7f028e800640 1 -- 172.21.15.27:0/927395132 wait complete. 2024-09-21T00:21:36.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.403+0000 7f028e800640 1 Processor -- start 2024-09-21T00:21:36.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.403+0000 7f028e800640 1 -- start start 2024-09-21T00:21:36.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.403+0000 7f028e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02880757b0 0x7f028819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:36.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.403+0000 7f028e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0288076100 0x7f028819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:36.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.404+0000 7f028e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0288110020 0x7f02881a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:36.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.404+0000 7f028e800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0288115c40 con 0x7f0288076100 2024-09-21T00:21:36.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.404+0000 7f028e800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0288115ac0 con 0x7f02880757b0 2024-09-21T00:21:36.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.404+0000 7f028e800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0288115dc0 con 0x7f0288110020 2024-09-21T00:21:36.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.404+0000 7f0286a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0288076100 0x7f028819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:36.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.404+0000 7f0287400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02880757b0 0x7f028819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:36.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.404+0000 7f0286a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0288076100 0x7f028819cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50370/0 (socket says 172.21.15.27:50370) 2024-09-21T00:21:36.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.404+0000 7f0286a00640 1 -- 172.21.15.27:0/2111692484 learned_addr learned my addr 172.21.15.27:0/2111692484 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:36.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.404+0000 7f0287e00640 1 --2- 172.21.15.27:0/2111692484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0288110020 0x7f02881a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:36.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.404+0000 7f0287400640 1 -- 172.21.15.27:0/2111692484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0288110020 msgr2=0x7f02881a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:36.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.404+0000 7f0287400640 1 --2- 172.21.15.27:0/2111692484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0288110020 0x7f02881a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.404+0000 7f0287400640 1 -- 172.21.15.27:0/2111692484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0288076100 msgr2=0x7f028819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:36.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.404+0000 7f0287400640 1 --2- 172.21.15.27:0/2111692484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0288076100 0x7f028819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.404+0000 7f0287400640 1 -- 172.21.15.27:0/2111692484 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02881a24f0 con 0x7f02880757b0 2024-09-21T00:21:36.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.404+0000 7f0286a00640 1 --2- 172.21.15.27:0/2111692484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0288076100 0x7f028819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:36.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.405+0000 7f0287400640 1 --2- 172.21.15.27:0/2111692484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02880757b0 0x7f028819c5c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f027800e9f0 tx=0x7f027800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:36.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.405+0000 7f027fe00640 1 -- 172.21.15.27:0/2111692484 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027800ccc0 con 0x7f02880757b0 2024-09-21T00:21:36.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.405+0000 7f028e800640 1 -- 172.21.15.27:0/2111692484 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02881a27e0 con 0x7f02880757b0 2024-09-21T00:21:36.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.405+0000 7f027fe00640 1 -- 172.21.15.27:0/2111692484 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02780108e0 con 0x7f02880757b0 2024-09-21T00:21:36.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.405+0000 7f027fe00640 1 -- 172.21.15.27:0/2111692484 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278018880 con 0x7f02880757b0 2024-09-21T00:21:36.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.405+0000 7f028e800640 1 -- 172.21.15.27:0/2111692484 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02881a8f70 con 0x7f02880757b0 2024-09-21T00:21:36.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.406+0000 7f028e800640 1 -- 172.21.15.27:0/2111692484 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0288076cc0 con 0x7f02880757b0 2024-09-21T00:21:36.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.409+0000 7f027fe00640 1 -- 172.21.15.27:0/2111692484 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f0278010430 con 0x7f02880757b0 2024-09-21T00:21:36.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.410+0000 7f027fe00640 1 --2- 172.21.15.27:0/2111692484 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f026c0b6560 0x7f026c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:36.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.410+0000 7f027fe00640 1 -- 172.21.15.27:0/2111692484 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f02780e7fe0 con 0x7f02880757b0 2024-09-21T00:21:36.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.411+0000 7f0286a00640 1 --2- 172.21.15.27:0/2111692484 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f026c0b6560 0x7f026c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:36.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.411+0000 7f0286a00640 1 --2- 172.21.15.27:0/2111692484 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f026c0b6560 0x7f026c0b8a70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0288112230 tx=0x7f0274005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:36.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.412+0000 7f027fe00640 1 -- 172.21.15.27:0/2111692484 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f02780b27a0 con 0x7f02880757b0 2024-09-21T00:21:36.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:36 smithi089 ceph-mon[34481]: pgmap v383: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:21:36.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:36 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2447835470' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T00:21:36.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:36 smithi098 ceph-mon[34725]: pgmap v383: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:21:36.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:36 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2447835470' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T00:21:36.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:36 smithi027 ceph-mon[30745]: pgmap v383: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:21:36.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:36 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2447835470' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T00:21:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.635+0000 7f028e800640 1 -- 172.21.15.27:0/2111692484 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f0288102680 con 0x7f026c0b6560 2024-09-21T00:21:36.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.819+0000 7f027fe00640 1 -- 172.21.15.27:0/2111692484 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0288102680 con 0x7f026c0b6560 2024-09-21T00:21:36.821 INFO:teuthology.orchestra.run.smithi027.stdout:Schedule set for path / 2024-09-21T00:21:36.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.822+0000 7f028e800640 1 -- 172.21.15.27:0/2111692484 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f026c0b6560 msgr2=0x7f026c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:36.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.822+0000 7f028e800640 1 --2- 172.21.15.27:0/2111692484 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f026c0b6560 0x7f026c0b8a70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0288112230 tx=0x7f0274005f90 comp rx=0 tx=0).stop 2024-09-21T00:21:36.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.822+0000 7f028e800640 1 -- 172.21.15.27:0/2111692484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02880757b0 msgr2=0x7f028819c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:36.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.822+0000 7f028e800640 1 --2- 172.21.15.27:0/2111692484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02880757b0 0x7f028819c5c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f027800e9f0 tx=0x7f027800eeb0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.822+0000 7f028e800640 1 -- 172.21.15.27:0/2111692484 shutdown_connections 2024-09-21T00:21:36.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.822+0000 7f028e800640 1 --2- 172.21.15.27:0/2111692484 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f026c0b6560 0x7f026c0b8a70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.822+0000 7f028e800640 1 --2- 172.21.15.27:0/2111692484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0288110020 0x7f02881a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.822+0000 7f028e800640 1 --2- 172.21.15.27:0/2111692484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0288076100 0x7f028819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.822+0000 7f028e800640 1 --2- 172.21.15.27:0/2111692484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02880757b0 0x7f028819c5c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:36.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.822+0000 7f028e800640 1 -- 172.21.15.27:0/2111692484 >> 172.21.15.27:0/2111692484 conn(0x7f02880ff4d0 msgr2=0x7f0288100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:36.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.822+0000 7f028e800640 1 -- 172.21.15.27:0/2111692484 shutdown_connections 2024-09-21T00:21:36.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:36.822+0000 7f028e800640 1 -- 172.21.15.27:0/2111692484 wait complete. 2024-09-21T00:21:36.847 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-21T00:21:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.075+0000 7f7ec2400640 1 -- 172.21.15.27:0/1138467138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ebc077390 msgr2=0x7f7ebc0757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.075+0000 7f7ec2400640 1 --2- 172.21.15.27:0/1138467138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ebc077390 0x7f7ebc0757b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009900 tx=0x7f7eb002a410 comp rx=0 tx=0).stop 2024-09-21T00:21:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.075+0000 7f7ec2400640 1 -- 172.21.15.27:0/1138467138 shutdown_connections 2024-09-21T00:21:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.075+0000 7f7ec2400640 1 --2- 172.21.15.27:0/1138467138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ebc1141d0 0x7f7ebc1166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.075+0000 7f7ec2400640 1 --2- 172.21.15.27:0/1138467138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ebc075dd0 0x7f7ebc0761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.075+0000 7f7ec2400640 1 --2- 172.21.15.27:0/1138467138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ebc077390 0x7f7ebc0757b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.075+0000 7f7ec2400640 1 -- 172.21.15.27:0/1138467138 >> 172.21.15.27:0/1138467138 conn(0x7f7ebc0ff390 msgr2=0x7f7ebc1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.075+0000 7f7ec2400640 1 -- 172.21.15.27:0/1138467138 shutdown_connections 2024-09-21T00:21:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.075+0000 7f7ec2400640 1 -- 172.21.15.27:0/1138467138 wait complete. 2024-09-21T00:21:37.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.076+0000 7f7ec2400640 1 Processor -- start 2024-09-21T00:21:37.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.077+0000 7f7ec2400640 1 -- start start 2024-09-21T00:21:37.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.077+0000 7f7ec2400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ebc075dd0 0x7f7ebc1a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:37.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.077+0000 7f7ec2400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ebc077390 0x7f7ebc1a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:37.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.077+0000 7f7ec2400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ebc1141d0 0x7f7ebc1a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:37.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.077+0000 7f7ec2400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc119e60 con 0x7f7ebc1141d0 2024-09-21T00:21:37.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.077+0000 7f7ec2400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc119ce0 con 0x7f7ebc075dd0 2024-09-21T00:21:37.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.077+0000 7f7ec2400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc119fe0 con 0x7f7ebc077390 2024-09-21T00:21:37.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.077+0000 7f7ec1a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ebc1141d0 0x7f7ebc1a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:37.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.077+0000 7f7ec1000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ebc075dd0 0x7f7ebc1a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:37.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.077+0000 7f7ec1a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ebc1141d0 0x7f7ebc1a6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50402/0 (socket says 172.21.15.27:50402) 2024-09-21T00:21:37.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.077+0000 7f7ec1a00640 1 -- 172.21.15.27:0/433563392 learned_addr learned my addr 172.21.15.27:0/433563392 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:37.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.077+0000 7f7ebbe00640 1 --2- 172.21.15.27:0/433563392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ebc077390 0x7f7ebc1a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:37.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.078+0000 7f7ec1a00640 1 -- 172.21.15.27:0/433563392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ebc077390 msgr2=0x7f7ebc1a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:37.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.078+0000 7f7ec1a00640 1 --2- 172.21.15.27:0/433563392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ebc077390 0x7f7ebc1a0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.078+0000 7f7ec1a00640 1 -- 172.21.15.27:0/433563392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ebc075dd0 msgr2=0x7f7ebc1a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:37.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.078+0000 7f7ec1a00640 1 --2- 172.21.15.27:0/433563392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ebc075dd0 0x7f7ebc1a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.078+0000 7f7ec1a00640 1 -- 172.21.15.27:0/433563392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc1a6810 con 0x7f7ebc1141d0 2024-09-21T00:21:37.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.078+0000 7f7ebbe00640 1 --2- 172.21.15.27:0/433563392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ebc077390 0x7f7ebc1a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:37.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.078+0000 7f7ec1a00640 1 --2- 172.21.15.27:0/433563392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ebc1141d0 0x7f7ebc1a6110 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400eb10 tx=0x7f7eb400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:37.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.078+0000 7f7eb9600640 1 -- 172.21.15.27:0/433563392 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4004b20 con 0x7f7ebc1141d0 2024-09-21T00:21:37.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.078+0000 7f7eb9600640 1 -- 172.21.15.27:0/433563392 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eb4004cc0 con 0x7f7ebc1141d0 2024-09-21T00:21:37.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.078+0000 7f7eb9600640 1 -- 172.21.15.27:0/433563392 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4010670 con 0x7f7ebc1141d0 2024-09-21T00:21:37.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.078+0000 7f7ec2400640 1 -- 172.21.15.27:0/433563392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc1a6b00 con 0x7f7ebc1141d0 2024-09-21T00:21:37.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.079+0000 7f7ec2400640 1 -- 172.21.15.27:0/433563392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc1ad290 con 0x7f7ebc1141d0 2024-09-21T00:21:37.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.080+0000 7f7ec2400640 1 -- 172.21.15.27:0/433563392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ebc0761e0 con 0x7f7ebc1141d0 2024-09-21T00:21:37.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.082+0000 7f7eb9600640 1 -- 172.21.15.27:0/433563392 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f7eb4005000 con 0x7f7ebc1141d0 2024-09-21T00:21:37.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.084+0000 7f7eb9600640 1 --2- 172.21.15.27:0/433563392 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7ea40b6550 0x7f7ea40b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:37.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.084+0000 7f7eb9600640 1 -- 172.21.15.27:0/433563392 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f7eb40e7fe0 con 0x7f7ebc1141d0 2024-09-21T00:21:37.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.084+0000 7f7ec1000640 1 --2- 172.21.15.27:0/433563392 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7ea40b6550 0x7f7ea40b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:37.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.085+0000 7f7ec1000640 1 --2- 172.21.15.27:0/433563392 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7ea40b6550 0x7f7ea40b8a60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0005ab0 tx=0x7f7eb0005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:37.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.086+0000 7f7eb9600640 1 -- 172.21.15.27:0/433563392 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7eb4056710 con 0x7f7ebc1141d0 2024-09-21T00:21:37.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.308+0000 7f7ec2400640 1 -- 172.21.15.27:0/433563392 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f7ebc063800 con 0x7f7ea40b6550 2024-09-21T00:21:37.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.382+0000 7f7eb9600640 1 -- 172.21.15.27:0/433563392 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f7ebc063800 con 0x7f7ea40b6550 2024-09-21T00:21:37.384 INFO:teuthology.orchestra.run.smithi027.stdout:Retention added to path / 2024-09-21T00:21:37.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.385+0000 7f7ec2400640 1 -- 172.21.15.27:0/433563392 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7ea40b6550 msgr2=0x7f7ea40b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:37.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.385+0000 7f7ec2400640 1 --2- 172.21.15.27:0/433563392 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7ea40b6550 0x7f7ea40b8a60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0005ab0 tx=0x7f7eb0005a20 comp rx=0 tx=0).stop 2024-09-21T00:21:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.385+0000 7f7ec2400640 1 -- 172.21.15.27:0/433563392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ebc1141d0 msgr2=0x7f7ebc1a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.385+0000 7f7ec2400640 1 --2- 172.21.15.27:0/433563392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ebc1141d0 0x7f7ebc1a6110 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400eb10 tx=0x7f7eb400c2d0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.385+0000 7f7ec2400640 1 -- 172.21.15.27:0/433563392 shutdown_connections 2024-09-21T00:21:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.385+0000 7f7ec2400640 1 --2- 172.21.15.27:0/433563392 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7ea40b6550 0x7f7ea40b8a60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.385+0000 7f7ec2400640 1 --2- 172.21.15.27:0/433563392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ebc1141d0 0x7f7ebc1a6110 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.385+0000 7f7ec2400640 1 --2- 172.21.15.27:0/433563392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ebc077390 0x7f7ebc1a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.385+0000 7f7ec2400640 1 --2- 172.21.15.27:0/433563392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ebc075dd0 0x7f7ebc1a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.385+0000 7f7ec2400640 1 -- 172.21.15.27:0/433563392 >> 172.21.15.27:0/433563392 conn(0x7f7ebc0ff390 msgr2=0x7f7ebc1100d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.385+0000 7f7ec2400640 1 -- 172.21.15.27:0/433563392 shutdown_connections 2024-09-21T00:21:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.385+0000 7f7ec2400640 1 -- 172.21.15.27:0/433563392 wait complete. 2024-09-21T00:21:37.409 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:37 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2447835470' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T00:21:37.409 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:37 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:21:37.411 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T00:21:37.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:37 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2447835470' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T00:21:37.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:37 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:21:37.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:37 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2447835470' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T00:21:37.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:37 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:21:37.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.636+0000 7f3c6fe00640 1 -- 172.21.15.27:0/552914794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c7010b440 msgr2=0x7f3c7010b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:37.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.636+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/552914794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c7010b440 0x7f3c7010b850 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009900 tx=0x7f3c5802a410 comp rx=0 tx=0).stop 2024-09-21T00:21:37.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.636+0000 7f3c6fe00640 1 -- 172.21.15.27:0/552914794 shutdown_connections 2024-09-21T00:21:37.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.636+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/552914794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c7010bd90 0x7f3c701125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.636+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/552914794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c7010b440 0x7f3c7010b850 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.636+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/552914794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c70105b10 0x7f3c70105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.636+0000 7f3c6fe00640 1 -- 172.21.15.27:0/552914794 >> 172.21.15.27:0/552914794 conn(0x7f3c70101510 msgr2=0x7f3c70103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:37.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.637+0000 7f3c6fe00640 1 -- 172.21.15.27:0/552914794 shutdown_connections 2024-09-21T00:21:37.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.637+0000 7f3c6fe00640 1 -- 172.21.15.27:0/552914794 wait complete. 2024-09-21T00:21:37.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.638+0000 7f3c6fe00640 1 Processor -- start 2024-09-21T00:21:37.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.638+0000 7f3c6fe00640 1 -- start start 2024-09-21T00:21:37.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.638+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c70105b10 0x7f3c7019c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:37.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.638+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c7010b440 0x7f3c7019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:37.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.638+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c7010bd90 0x7f3c701a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:37.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.638+0000 7f3c6fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70115a50 con 0x7f3c7010bd90 2024-09-21T00:21:37.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.638+0000 7f3c6fe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3c701158d0 con 0x7f3c7010b440 2024-09-21T00:21:37.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.638+0000 7f3c6fe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70115bd0 con 0x7f3c70105b10 2024-09-21T00:21:37.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.639+0000 7f3c6ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c70105b10 0x7f3c7019c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:37.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.639+0000 7f3c66000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c7010b440 0x7f3c7019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:37.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.639+0000 7f3c6f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c7010bd90 0x7f3c701a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:37.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.639+0000 7f3c6ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c70105b10 0x7f3c7019c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39632/0 (socket says 172.21.15.27:39632) 2024-09-21T00:21:37.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.639+0000 7f3c6ea00640 1 -- 172.21.15.27:0/3340539204 learned_addr learned my addr 172.21.15.27:0/3340539204 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.639+0000 7f3c6ea00640 1 -- 172.21.15.27:0/3340539204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c7010b440 msgr2=0x7f3c7019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.639+0000 7f3c6ea00640 1 --2- 172.21.15.27:0/3340539204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c7010b440 0x7f3c7019cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.639+0000 7f3c6ea00640 1 -- 172.21.15.27:0/3340539204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c7010bd90 msgr2=0x7f3c701a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.639+0000 7f3c6ea00640 1 --2- 172.21.15.27:0/3340539204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c7010bd90 0x7f3c701a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.639+0000 7f3c6ea00640 1 -- 172.21.15.27:0/3340539204 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c701a2460 con 0x7f3c70105b10 2024-09-21T00:21:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.639+0000 7f3c66000640 1 --2- 172.21.15.27:0/3340539204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c7010b440 0x7f3c7019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.639+0000 7f3c6f400640 1 --2- 172.21.15.27:0/3340539204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c7010bd90 0x7f3c701a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.639+0000 7f3c6ea00640 1 --2- 172.21.15.27:0/3340539204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c70105b10 0x7f3c7019c530 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000e9f0 tx=0x7f3c6000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.640+0000 7f3c67e00640 1 -- 172.21.15.27:0/3340539204 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6000cd80 con 0x7f3c70105b10 2024-09-21T00:21:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.640+0000 7f3c67e00640 1 -- 172.21.15.27:0/3340539204 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c600108e0 con 0x7f3c70105b10 2024-09-21T00:21:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.640+0000 7f3c6fe00640 1 -- 172.21.15.27:0/3340539204 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c701a2750 con 0x7f3c70105b10 2024-09-21T00:21:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.640+0000 7f3c67e00640 1 -- 172.21.15.27:0/3340539204 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c60018980 con 0x7f3c70105b10 2024-09-21T00:21:37.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.640+0000 7f3c6fe00640 1 -- 172.21.15.27:0/3340539204 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c701a8ee0 con 0x7f3c70105b10 2024-09-21T00:21:37.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.641+0000 7f3c6fe00640 1 -- 172.21.15.27:0/3340539204 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c70105f20 con 0x7f3c70105b10 2024-09-21T00:21:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.644+0000 7f3c67e00640 1 -- 172.21.15.27:0/3340539204 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f3c60010430 con 0x7f3c70105b10 2024-09-21T00:21:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.645+0000 7f3c67e00640 1 --2- 172.21.15.27:0/3340539204 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3c540b6550 0x7f3c540b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.645+0000 7f3c67e00640 1 -- 172.21.15.27:0/3340539204 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f3c600e7ff0 con 0x7f3c70105b10 2024-09-21T00:21:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.645+0000 7f3c66000640 1 --2- 172.21.15.27:0/3340539204 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3c540b6550 0x7f3c540b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:37.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.646+0000 7f3c66000640 1 --2- 172.21.15.27:0/3340539204 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3c540b6550 0x7f3c540b8a60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009230 tx=0x7f3c58031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:37.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.646+0000 7f3c67e00640 1 -- 172.21.15.27:0/3340539204 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3c600b27b0 con 0x7f3c70105b10 2024-09-21T00:21:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.872+0000 7f3c6fe00640 1 -- 172.21.15.27:0/3340539204 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f3c70063800 con 0x7f3c540b6550 2024-09-21T00:21:37.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.880+0000 7f3c67e00640 1 -- 172.21.15.27:0/3340539204 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f3c70063800 con 0x7f3c540b6550 2024-09-21T00:21:37.881 INFO:teuthology.orchestra.run.smithi027.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T00:21:36", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-21T00:21:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.883+0000 7f3c6fe00640 1 -- 172.21.15.27:0/3340539204 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3c540b6550 msgr2=0x7f3c540b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.883+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/3340539204 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3c540b6550 0x7f3c540b8a60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009230 tx=0x7f3c58031400 comp rx=0 tx=0).stop 2024-09-21T00:21:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.883+0000 7f3c6fe00640 1 -- 172.21.15.27:0/3340539204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c70105b10 msgr2=0x7f3c7019c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.883+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/3340539204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c70105b10 0x7f3c7019c530 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000e9f0 tx=0x7f3c6000eeb0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.883+0000 7f3c6fe00640 1 -- 172.21.15.27:0/3340539204 shutdown_connections 2024-09-21T00:21:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.883+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/3340539204 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3c540b6550 0x7f3c540b8a60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.883+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/3340539204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c7010bd90 0x7f3c701a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.883+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/3340539204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c7010b440 0x7f3c7019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.883+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/3340539204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c70105b10 0x7f3c7019c530 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.883+0000 7f3c6fe00640 1 -- 172.21.15.27:0/3340539204 >> 172.21.15.27:0/3340539204 conn(0x7f3c70101510 msgr2=0x7f3c70108a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:37.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.883+0000 7f3c6fe00640 1 -- 172.21.15.27:0/3340539204 shutdown_connections 2024-09-21T00:21:37.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:37.883+0000 7f3c6fe00640 1 -- 172.21.15.27:0/3340539204 wait complete. 2024-09-21T00:21:37.909 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T00:21:38.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.137+0000 7fee3fe00640 1 -- 172.21.15.27:0/3866013455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4010bd90 msgr2=0x7fee401125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:38.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.137+0000 7fee3fe00640 1 --2- 172.21.15.27:0/3866013455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4010bd90 0x7fee401125c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fee38009ca0 tx=0x7fee3802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:21:38.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.137+0000 7fee3fe00640 1 -- 172.21.15.27:0/3866013455 shutdown_connections 2024-09-21T00:21:38.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.137+0000 7fee3fe00640 1 --2- 172.21.15.27:0/3866013455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4010bd90 0x7fee401125c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:38.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.137+0000 7fee3fe00640 1 --2- 172.21.15.27:0/3866013455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee4010b440 0x7fee4010b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:38.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.137+0000 7fee3fe00640 1 --2- 172.21.15.27:0/3866013455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee40105b10 0x7fee40105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:38.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.137+0000 7fee3fe00640 1 -- 172.21.15.27:0/3866013455 >> 172.21.15.27:0/3866013455 conn(0x7fee40101510 msgr2=0x7fee40103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:38.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.137+0000 7fee3fe00640 1 -- 172.21.15.27:0/3866013455 shutdown_connections 2024-09-21T00:21:38.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.137+0000 7fee3fe00640 1 -- 172.21.15.27:0/3866013455 wait complete. 2024-09-21T00:21:38.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.138+0000 7fee3fe00640 1 Processor -- start 2024-09-21T00:21:38.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.139+0000 7fee3fe00640 1 -- start start 2024-09-21T00:21:38.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.139+0000 7fee3fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee40105b10 0x7fee4019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:38.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.139+0000 7fee3fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4010b440 0x7fee4019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:38.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.139+0000 7fee3fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee4010bd90 0x7fee401a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:38.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.139+0000 7fee3fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fee40115a50 con 0x7fee4010b440 2024-09-21T00:21:38.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.139+0000 7fee3fe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fee401158d0 con 0x7fee4010bd90 2024-09-21T00:21:38.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.139+0000 7fee3fe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fee40115bd0 con 0x7fee40105b10 2024-09-21T00:21:38.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.139+0000 7fee36000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4010b440 0x7fee4019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:38.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.139+0000 7fee3f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee4010bd90 0x7fee401a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:38.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.139+0000 7fee36000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4010b440 0x7fee4019c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50436/0 (socket says 172.21.15.27:50436) 2024-09-21T00:21:38.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.139+0000 7fee36000640 1 -- 172.21.15.27:0/3789015752 learned_addr learned my addr 172.21.15.27:0/3789015752 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:38.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.139+0000 7fee3ea00640 1 --2- 172.21.15.27:0/3789015752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee40105b10 0x7fee4019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:38.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.140+0000 7fee36000640 1 -- 172.21.15.27:0/3789015752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee40105b10 msgr2=0x7fee4019c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.140+0000 7fee36000640 1 --2- 172.21.15.27:0/3789015752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee40105b10 0x7fee4019c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.140+0000 7fee36000640 1 -- 172.21.15.27:0/3789015752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee4010bd90 msgr2=0x7fee401a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.140+0000 7fee36000640 1 --2- 172.21.15.27:0/3789015752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee4010bd90 0x7fee401a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.140+0000 7fee36000640 1 -- 172.21.15.27:0/3789015752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee401a2320 con 0x7fee4010b440 2024-09-21T00:21:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.140+0000 7fee3f400640 1 --2- 172.21.15.27:0/3789015752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee4010bd90 0x7fee401a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.140+0000 7fee36000640 1 --2- 172.21.15.27:0/3789015752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4010b440 0x7fee4019c990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fee2800e9f0 tx=0x7fee2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.140+0000 7fee37e00640 1 -- 172.21.15.27:0/3789015752 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2800ccc0 con 0x7fee4010b440 2024-09-21T00:21:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.141+0000 7fee3fe00640 1 -- 172.21.15.27:0/3789015752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee401a2610 con 0x7fee4010b440 2024-09-21T00:21:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.141+0000 7fee37e00640 1 -- 172.21.15.27:0/3789015752 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee280108e0 con 0x7fee4010b440 2024-09-21T00:21:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.141+0000 7fee37e00640 1 -- 172.21.15.27:0/3789015752 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28018860 con 0x7fee4010b440 2024-09-21T00:21:38.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.141+0000 7fee3fe00640 1 -- 172.21.15.27:0/3789015752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee401a8da0 con 0x7fee4010b440 2024-09-21T00:21:38.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.142+0000 7fee3fe00640 1 -- 172.21.15.27:0/3789015752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee40105f20 con 0x7fee4010b440 2024-09-21T00:21:38.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.146+0000 7fee37e00640 1 -- 172.21.15.27:0/3789015752 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fee28010430 con 0x7fee4010b440 2024-09-21T00:21:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.147+0000 7fee37e00640 1 --2- 172.21.15.27:0/3789015752 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee240b6550 0x7fee240b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.147+0000 7fee37e00640 1 -- 172.21.15.27:0/3789015752 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fee280e7fe0 con 0x7fee4010b440 2024-09-21T00:21:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.148+0000 7fee3ea00640 1 --2- 172.21.15.27:0/3789015752 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee240b6550 0x7fee240b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:38.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.148+0000 7fee3ea00640 1 --2- 172.21.15.27:0/3789015752 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee240b6550 0x7fee240b8a60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fee30009bd0 tx=0x7fee30005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.150+0000 7fee37e00640 1 -- 172.21.15.27:0/3789015752 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fee280b2740 con 0x7fee4010b440 2024-09-21T00:21:38.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:38 smithi027 ceph-mon[30745]: pgmap v384: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:21:38.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:38 smithi027 ceph-mon[30745]: from='client.25388 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:38.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.371+0000 7fee3fe00640 1 -- 172.21.15.27:0/3789015752 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fee40063800 con 0x7fee240b6550 2024-09-21T00:21:38.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.379+0000 7fee37e00640 1 -- 172.21.15.27:0/3789015752 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fee40063800 con 0x7fee240b6550 2024-09-21T00:21:38.380 INFO:teuthology.orchestra.run.smithi027.stdout:/ 1m 6m3h 2024-09-21T00:21:38.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.382+0000 7fee3fe00640 1 -- 172.21.15.27:0/3789015752 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee240b6550 msgr2=0x7fee240b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.382+0000 7fee3fe00640 1 --2- 172.21.15.27:0/3789015752 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee240b6550 0x7fee240b8a60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fee30009bd0 tx=0x7fee30005f90 comp rx=0 tx=0).stop 2024-09-21T00:21:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.382+0000 7fee3fe00640 1 -- 172.21.15.27:0/3789015752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4010b440 msgr2=0x7fee4019c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.382+0000 7fee3fe00640 1 --2- 172.21.15.27:0/3789015752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4010b440 0x7fee4019c990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fee2800e9f0 tx=0x7fee2800eeb0 comp rx=0 tx=0).stop 2024-09-21T00:21:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.382+0000 7fee3fe00640 1 -- 172.21.15.27:0/3789015752 shutdown_connections 2024-09-21T00:21:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.382+0000 7fee3fe00640 1 --2- 172.21.15.27:0/3789015752 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee240b6550 0x7fee240b8a60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.382+0000 7fee3fe00640 1 --2- 172.21.15.27:0/3789015752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee4010bd90 0x7fee401a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:38.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.382+0000 7fee3fe00640 1 --2- 172.21.15.27:0/3789015752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4010b440 0x7fee4019c990 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:38.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.382+0000 7fee3fe00640 1 --2- 172.21.15.27:0/3789015752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee40105b10 0x7fee4019c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:38.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.382+0000 7fee3fe00640 1 -- 172.21.15.27:0/3789015752 >> 172.21.15.27:0/3789015752 conn(0x7fee40101510 msgr2=0x7fee40108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:38.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.382+0000 7fee3fe00640 1 -- 172.21.15.27:0/3789015752 shutdown_connections 2024-09-21T00:21:38.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:38.382+0000 7fee3fe00640 1 -- 172.21.15.27:0/3789015752 wait complete. 2024-09-21T00:21:38.405 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-21T00:21:38.465 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-21T00:21:40.396 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-21T00:21:40.437 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T00:21:40.437 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-21T00:21:40.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:39 smithi027 ceph-mon[30745]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:40.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:39 smithi027 ceph-mon[30745]: from='client.25387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:40.440 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:38 smithi098 ceph-mon[34725]: pgmap v384: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:21:40.441 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:38 smithi098 ceph-mon[34725]: from='client.25388 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:40.441 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:39 smithi098 ceph-mon[34725]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:40.441 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:39 smithi098 ceph-mon[34725]: from='client.25387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:40.441 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:38 smithi089 ceph-mon[34481]: pgmap v384: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:21:40.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:38 smithi089 ceph-mon[34481]: from='client.25388 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:40.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:39 smithi089 ceph-mon[34481]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:40.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:39 smithi089 ceph-mon[34481]: from='client.25387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:40.444 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-21T00:21:40.474 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T00:21:40.475 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T00:21:40.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:40 smithi089 ceph-mon[34481]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:40.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:40 smithi089 ceph-mon[34481]: pgmap v385: 129 pgs: 129 active+clean; 1.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 102 KiB/s wr, 44 op/s 2024-09-21T00:21:40.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:40 smithi089 ceph-mon[34481]: mgrmap e23: x(active, since 10m), standbys: z, y 2024-09-21T00:21:40.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:40 smithi098 ceph-mon[34725]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:40.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:40 smithi098 ceph-mon[34725]: pgmap v385: 129 pgs: 129 active+clean; 1.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 102 KiB/s wr, 44 op/s 2024-09-21T00:21:40.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:40 smithi098 ceph-mon[34725]: mgrmap e23: x(active, since 10m), standbys: z, y 2024-09-21T00:21:40.497 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:40 smithi027 ceph-mon[30745]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:21:40.497 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:40 smithi027 ceph-mon[30745]: pgmap v385: 129 pgs: 129 active+clean; 1.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 102 KiB/s wr, 44 op/s 2024-09-21T00:21:40.497 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:40 smithi027 ceph-mon[30745]: mgrmap e23: x(active, since 10m), standbys: z, y 2024-09-21T00:21:40.869 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:41.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.510+0000 7f0b8b200640 1 -- 172.21.15.27:0/2905594212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8c10ff90 msgr2=0x7f0b8c1123b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:41.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.510+0000 7f0b8b200640 1 --2- 172.21.15.27:0/2905594212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8c10ff90 0x7f0b8c1123b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009ca0 tx=0x7f0b8402a940 comp rx=0 tx=0).stop 2024-09-21T00:21:41.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.511+0000 7f0b8b200640 1 -- 172.21.15.27:0/2905594212 shutdown_connections 2024-09-21T00:21:41.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.511+0000 7f0b8b200640 1 --2- 172.21.15.27:0/2905594212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8c10ff90 0x7f0b8c1123b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:41.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.511+0000 7f0b8b200640 1 --2- 172.21.15.27:0/2905594212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b8c102d30 0x7f0b8c10fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:41.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.511+0000 7f0b8b200640 1 --2- 172.21.15.27:0/2905594212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c1023e0 0x7f0b8c1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:41.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.511+0000 7f0b8b200640 1 -- 172.21.15.27:0/2905594212 >> 172.21.15.27:0/2905594212 conn(0x7f0b8c0fe2c0 msgr2=0x7f0b8c100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:41.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.511+0000 7f0b8b200640 1 -- 172.21.15.27:0/2905594212 shutdown_connections 2024-09-21T00:21:41.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.511+0000 7f0b8b200640 1 -- 172.21.15.27:0/2905594212 wait complete. 2024-09-21T00:21:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.512+0000 7f0b8b200640 1 Processor -- start 2024-09-21T00:21:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.512+0000 7f0b8b200640 1 -- start start 2024-09-21T00:21:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.512+0000 7f0b8b200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c1023e0 0x7f0b8c19c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.512+0000 7f0b8b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8c102d30 0x7f0b8c19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.512+0000 7f0b8b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b8c10ff90 0x7f0b8c1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:41.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.512+0000 7f0b8b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c115a80 con 0x7f0b8c102d30 2024-09-21T00:21:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.512+0000 7f0b8b200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c115900 con 0x7f0b8c1023e0 2024-09-21T00:21:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.512+0000 7f0b8b200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c115c00 con 0x7f0b8c10ff90 2024-09-21T00:21:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.513+0000 7f0b83e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8c102d30 0x7f0b8c19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.513+0000 7f0b83e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8c102d30 0x7f0b8c19c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50440/0 (socket says 172.21.15.27:50440) 2024-09-21T00:21:41.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.513+0000 7f0b83e00640 1 -- 172.21.15.27:0/1952679884 learned_addr learned my addr 172.21.15.27:0/1952679884 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.513+0000 7f0b89e00640 1 --2- 172.21.15.27:0/1952679884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c1023e0 0x7f0b8c19c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.513+0000 7f0b8a800640 1 --2- 172.21.15.27:0/1952679884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b8c10ff90 0x7f0b8c1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.513+0000 7f0b83e00640 1 -- 172.21.15.27:0/1952679884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b8c10ff90 msgr2=0x7f0b8c1a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.513+0000 7f0b83e00640 1 --2- 172.21.15.27:0/1952679884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b8c10ff90 0x7f0b8c1a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.513+0000 7f0b83e00640 1 -- 172.21.15.27:0/1952679884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c1023e0 msgr2=0x7f0b8c19c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:41.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.513+0000 7f0b83e00640 1 --2- 172.21.15.27:0/1952679884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c1023e0 0x7f0b8c19c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:41.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.513+0000 7f0b83e00640 1 -- 172.21.15.27:0/1952679884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c1a2380 con 0x7f0b8c102d30 2024-09-21T00:21:41.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.514+0000 7f0b83e00640 1 --2- 172.21.15.27:0/1952679884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8c102d30 0x7f0b8c19c9b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400d950 tx=0x7f0b7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:41.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.514+0000 7f0b82a00640 1 -- 172.21.15.27:0/1952679884 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74017070 con 0x7f0b8c102d30 2024-09-21T00:21:41.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.514+0000 7f0b82a00640 1 -- 172.21.15.27:0/1952679884 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b74009c30 con 0x7f0b8c102d30 2024-09-21T00:21:41.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.514+0000 7f0b8b200640 1 -- 172.21.15.27:0/1952679884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8c1a2670 con 0x7f0b8c102d30 2024-09-21T00:21:41.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.514+0000 7f0b82a00640 1 -- 172.21.15.27:0/1952679884 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7400f620 con 0x7f0b8c102d30 2024-09-21T00:21:41.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.514+0000 7f0b8b200640 1 -- 172.21.15.27:0/1952679884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b8c1a8dd0 con 0x7f0b8c102d30 2024-09-21T00:21:41.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.515+0000 7f0b8b200640 1 -- 172.21.15.27:0/1952679884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b8c1027f0 con 0x7f0b8c102d30 2024-09-21T00:21:41.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.517+0000 7f0b82a00640 1 -- 172.21.15.27:0/1952679884 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f0b74010460 con 0x7f0b8c102d30 2024-09-21T00:21:41.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.520+0000 7f0b82a00640 1 --2- 172.21.15.27:0/1952679884 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0b700b6670 0x7f0b700b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:41.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.520+0000 7f0b82a00640 1 -- 172.21.15.27:0/1952679884 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f0b740e7540 con 0x7f0b8c102d30 2024-09-21T00:21:41.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.520+0000 7f0b89e00640 1 --2- 172.21.15.27:0/1952679884 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0b700b6670 0x7f0b700b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:41.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.521+0000 7f0b89e00640 1 --2- 172.21.15.27:0/1952679884 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0b700b6670 0x7f0b700b8b80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00bbe0 tx=0x7f0b7c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:41.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.523+0000 7f0b82a00640 1 -- 172.21.15.27:0/1952679884 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0b74009de0 con 0x7f0b8c102d30 2024-09-21T00:21:41.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.717+0000 7f0b8b200640 1 -- 172.21.15.27:0/1952679884 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0b8c063800 con 0x7f0b700b6670 2024-09-21T00:21:41.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.725+0000 7f0b82a00640 1 -- 172.21.15.27:0/1952679884 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340528 (secure 0 0 0) 0x7f0b8c063800 con 0x7f0b700b6670 2024-09-21T00:21:41.730 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:21:41.731 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T00:21:41.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.731+0000 7f0b8b200640 1 -- 172.21.15.27:0/1952679884 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0b700b6670 msgr2=0x7f0b700b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:41.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.731+0000 7f0b8b200640 1 --2- 172.21.15.27:0/1952679884 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0b700b6670 0x7f0b700b8b80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00bbe0 tx=0x7f0b7c00b390 comp rx=0 tx=0).stop 2024-09-21T00:21:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.731+0000 7f0b8b200640 1 -- 172.21.15.27:0/1952679884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8c102d30 msgr2=0x7f0b8c19c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.731+0000 7f0b8b200640 1 --2- 172.21.15.27:0/1952679884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8c102d30 0x7f0b8c19c9b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400d950 tx=0x7f0b7400de10 comp rx=0 tx=0).stop 2024-09-21T00:21:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.731+0000 7f0b8b200640 1 -- 172.21.15.27:0/1952679884 shutdown_connections 2024-09-21T00:21:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.731+0000 7f0b8b200640 1 --2- 172.21.15.27:0/1952679884 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0b700b6670 0x7f0b700b8b80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.731+0000 7f0b8b200640 1 --2- 172.21.15.27:0/1952679884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b8c10ff90 0x7f0b8c1a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.731+0000 7f0b8b200640 1 --2- 172.21.15.27:0/1952679884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8c102d30 0x7f0b8c19c9b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.731+0000 7f0b8b200640 1 --2- 172.21.15.27:0/1952679884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c1023e0 0x7f0b8c19c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.731+0000 7f0b8b200640 1 -- 172.21.15.27:0/1952679884 >> 172.21.15.27:0/1952679884 conn(0x7f0b8c0fe2c0 msgr2=0x7f0b8c112100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.731+0000 7f0b8b200640 1 -- 172.21.15.27:0/1952679884 shutdown_connections 2024-09-21T00:21:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:41.731+0000 7f0b8b200640 1 -- 172.21.15.27:0/1952679884 wait complete. 2024-09-21T00:21:42.321 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:42 smithi027 ceph-mon[30745]: pgmap v386: 129 pgs: 129 active+clean; 1.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 102 KiB/s wr, 45 op/s 2024-09-21T00:21:42.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:42 smithi089 ceph-mon[34481]: pgmap v386: 129 pgs: 129 active+clean; 1.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 102 KiB/s wr, 45 op/s 2024-09-21T00:21:42.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:42 smithi098 ceph-mon[34725]: pgmap v386: 129 pgs: 129 active+clean; 1.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 102 KiB/s wr, 45 op/s 2024-09-21T00:21:42.682 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-21T00:21:40.379203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1060134,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1148,"num_read_kb":800,"num_write":463,"num_write_kb":3502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2955704,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":831,"ondisk_log_size":831,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381860,"kb_used_data":15852,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124559964,"statfs":{"total":1151940427776,"available":1151549403136,"internally_reserved":0,"allocated":16232448,"data_stored":9968525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":399482,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":374,"num_read_kb":310,"num_write":168,"num_write_kb":1227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003391"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453054+0000","last_change":"2024-09-21T00:18:35.627618+0000","last_active":"2024-09-21T00:20:26.453054+0000","last_peered":"2024-09-21T00:20:26.453054+0000","last_clean":"2024-09-21T00:20:26.453054+0000","last_became_active":"2024-09-21T00:18:35.626475+0000","last_became_peered":"2024-09-21T00:18:35.626475+0000","last_unstale":"2024-09-21T00:20:26.453054+0000","last_undegraded":"2024-09-21T00:20:26.453054+0000","last_fullsized":"2024-09-21T00:20:26.453054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:50.379225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754687+0000","last_change":"2024-09-21T00:18:38.665891+0000","last_active":"2024-09-21T00:20:54.754687+0000","last_peered":"2024-09-21T00:20:54.754687+0000","last_clean":"2024-09-21T00:20:54.754687+0000","last_became_active":"2024-09-21T00:18:38.665472+0000","last_became_peered":"2024-09-21T00:18:38.665472+0000","last_unstale":"2024-09-21T00:20:54.754687+0000","last_undegraded":"2024-09-21T00:20:54.754687+0000","last_fullsized":"2024-09-21T00:20:54.754687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:34:32.490828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451446+0000","last_change":"2024-09-21T00:18:48.730206+0000","last_active":"2024-09-21T00:20:26.451446+0000","last_peered":"2024-09-21T00:20:26.451446+0000","last_clean":"2024-09-21T00:20:26.451446+0000","last_became_active":"2024-09-21T00:18:48.729729+0000","last_became_peered":"2024-09-21T00:18:48.729729+0000","last_unstale":"2024-09-21T00:20:26.451446+0000","last_undegraded":"2024-09-21T00:20:26.451446+0000","last_fullsized":"2024-09-21T00:20:26.451446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:08:17.019877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.575433+0000","last_change":"2024-09-21T00:18:35.768554+0000","last_active":"2024-09-21T00:20:26.575433+0000","last_peered":"2024-09-21T00:20:26.575433+0000","last_clean":"2024-09-21T00:20:26.575433+0000","last_became_active":"2024-09-21T00:18:35.767642+0000","last_became_peered":"2024-09-21T00:18:35.767642+0000","last_unstale":"2024-09-21T00:20:26.575433+0000","last_undegraded":"2024-09-21T00:20:26.575433+0000","last_fullsized":"2024-09-21T00:20:26.575433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:10.442974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768099+0000","last_change":"2024-09-21T00:18:38.677997+0000","last_active":"2024-09-21T00:20:54.768099+0000","last_peered":"2024-09-21T00:20:54.768099+0000","last_clean":"2024-09-21T00:20:54.768099+0000","last_became_active":"2024-09-21T00:18:38.676698+0000","last_became_peered":"2024-09-21T00:18:38.676698+0000","last_unstale":"2024-09-21T00:20:54.768099+0000","last_undegraded":"2024-09-21T00:20:54.768099+0000","last_fullsized":"2024-09-21T00:20:54.768099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:11.662765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451676+0000","last_change":"2024-09-21T00:18:48.739563+0000","last_active":"2024-09-21T00:20:26.451676+0000","last_peered":"2024-09-21T00:20:26.451676+0000","last_clean":"2024-09-21T00:20:26.451676+0000","last_became_active":"2024-09-21T00:18:48.739092+0000","last_became_peered":"2024-09-21T00:18:48.739092+0000","last_unstale":"2024-09-21T00:20:26.451676+0000","last_undegraded":"2024-09-21T00:20:26.451676+0000","last_fullsized":"2024-09-21T00:20:26.451676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:44.185556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510989+0000","last_change":"2024-09-21T00:18:35.626742+0000","last_active":"2024-09-21T00:20:26.510989+0000","last_peered":"2024-09-21T00:20:26.510989+0000","last_clean":"2024-09-21T00:20:26.510989+0000","last_became_active":"2024-09-21T00:18:35.626359+0000","last_became_peered":"2024-09-21T00:18:35.626359+0000","last_unstale":"2024-09-21T00:20:26.510989+0000","last_undegraded":"2024-09-21T00:20:26.510989+0000","last_fullsized":"2024-09-21T00:20:26.510989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:38.860184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.761254+0000","last_change":"2024-09-21T00:18:38.666003+0000","last_active":"2024-09-21T00:20:54.761254+0000","last_peered":"2024-09-21T00:20:54.761254+0000","last_clean":"2024-09-21T00:20:54.761254+0000","last_became_active":"2024-09-21T00:18:38.665425+0000","last_became_peered":"2024-09-21T00:18:38.665425+0000","last_unstale":"2024-09-21T00:20:54.761254+0000","last_undegraded":"2024-09-21T00:20:54.761254+0000","last_fullsized":"2024-09-21T00:20:54.761254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:14.121864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.577303+0000","last_change":"2024-09-21T00:20:26.576748+0000","last_active":"2024-09-21T00:20:26.577303+0000","last_peered":"2024-09-21T00:20:26.577303+0000","last_clean":"2024-09-21T00:20:26.577303+0000","last_became_active":"2024-09-21T00:20:26.576186+0000","last_became_peered":"2024-09-21T00:20:26.576186+0000","last_unstale":"2024-09-21T00:20:26.577303+0000","last_undegraded":"2024-09-21T00:20:26.577303+0000","last_fullsized":"2024-09-21T00:20:26.577303+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:20.283717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510680+0000","last_change":"2024-09-21T00:18:35.628091+0000","last_active":"2024-09-21T00:20:26.510680+0000","last_peered":"2024-09-21T00:20:26.510680+0000","last_clean":"2024-09-21T00:20:26.510680+0000","last_became_active":"2024-09-21T00:18:35.627654+0000","last_became_peered":"2024-09-21T00:18:35.627654+0000","last_unstale":"2024-09-21T00:20:26.510680+0000","last_undegraded":"2024-09-21T00:20:26.510680+0000","last_fullsized":"2024-09-21T00:20:26.510680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:05.229036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.765336+0000","last_change":"2024-09-21T00:19:25.844116+0000","last_active":"2024-09-21T00:20:54.765336+0000","last_peered":"2024-09-21T00:20:54.765336+0000","last_clean":"2024-09-21T00:20:54.765336+0000","last_became_active":"2024-09-21T00:19:25.842745+0000","last_became_peered":"2024-09-21T00:19:25.842745+0000","last_unstale":"2024-09-21T00:20:54.765336+0000","last_undegraded":"2024-09-21T00:20:54.765336+0000","last_fullsized":"2024-09-21T00:20:54.765336+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:42.651251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451830+0000","last_change":"2024-09-21T00:18:48.740106+0000","last_active":"2024-09-21T00:20:26.451830+0000","last_peered":"2024-09-21T00:20:26.451830+0000","last_clean":"2024-09-21T00:20:26.451830+0000","last_became_active":"2024-09-21T00:18:48.739299+0000","last_became_peered":"2024-09-21T00:18:48.739299+0000","last_unstale":"2024-09-21T00:20:26.451830+0000","last_undegraded":"2024-09-21T00:20:26.451830+0000","last_fullsized":"2024-09-21T00:20:26.451830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:59.058333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.766986+0000","last_change":"2024-09-21T00:18:38.667331+0000","last_active":"2024-09-21T00:20:54.766986+0000","last_peered":"2024-09-21T00:20:54.766986+0000","last_clean":"2024-09-21T00:20:54.766986+0000","last_became_active":"2024-09-21T00:18:38.666960+0000","last_became_peered":"2024-09-21T00:18:38.666960+0000","last_unstale":"2024-09-21T00:20:54.766986+0000","last_undegraded":"2024-09-21T00:20:54.766986+0000","last_fullsized":"2024-09-21T00:20:54.766986+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:57.569460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510354+0000","last_change":"2024-09-21T00:18:35.625570+0000","last_active":"2024-09-21T00:20:26.510354+0000","last_peered":"2024-09-21T00:20:26.510354+0000","last_clean":"2024-09-21T00:20:26.510354+0000","last_became_active":"2024-09-21T00:18:35.625170+0000","last_became_peered":"2024-09-21T00:18:35.625170+0000","last_unstale":"2024-09-21T00:20:26.510354+0000","last_undegraded":"2024-09-21T00:20:26.510354+0000","last_fullsized":"2024-09-21T00:20:26.510354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:58.614258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452085+0000","last_change":"2024-09-21T00:18:48.727390+0000","last_active":"2024-09-21T00:20:26.452085+0000","last_peered":"2024-09-21T00:20:26.452085+0000","last_clean":"2024-09-21T00:20:26.452085+0000","last_became_active":"2024-09-21T00:18:48.726864+0000","last_became_peered":"2024-09-21T00:18:48.726864+0000","last_unstale":"2024-09-21T00:20:26.452085+0000","last_undegraded":"2024-09-21T00:20:26.452085+0000","last_fullsized":"2024-09-21T00:20:26.452085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:10.917069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767288+0000","last_change":"2024-09-21T00:18:38.664072+0000","last_active":"2024-09-21T00:20:54.767288+0000","last_peered":"2024-09-21T00:20:54.767288+0000","last_clean":"2024-09-21T00:20:54.767288+0000","last_became_active":"2024-09-21T00:18:38.662562+0000","last_became_peered":"2024-09-21T00:18:38.662562+0000","last_unstale":"2024-09-21T00:20:54.767288+0000","last_undegraded":"2024-09-21T00:20:54.767288+0000","last_fullsized":"2024-09-21T00:20:54.767288+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.021174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453878+0000","last_change":"2024-09-21T00:19:25.843599+0000","last_active":"2024-09-21T00:20:26.453878+0000","last_peered":"2024-09-21T00:20:26.453878+0000","last_clean":"2024-09-21T00:20:26.453878+0000","last_became_active":"2024-09-21T00:19:25.843178+0000","last_became_peered":"2024-09-21T00:19:25.843178+0000","last_unstale":"2024-09-21T00:20:26.453878+0000","last_undegraded":"2024-09-21T00:20:26.453878+0000","last_fullsized":"2024-09-21T00:20:26.453878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:18.809857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452141+0000","last_change":"2024-09-21T00:18:48.729940+0000","last_active":"2024-09-21T00:20:26.452141+0000","last_peered":"2024-09-21T00:20:26.452141+0000","last_clean":"2024-09-21T00:20:26.452141+0000","last_became_active":"2024-09-21T00:18:48.729465+0000","last_became_peered":"2024-09-21T00:18:48.729465+0000","last_unstale":"2024-09-21T00:20:26.452141+0000","last_undegraded":"2024-09-21T00:20:26.452141+0000","last_fullsized":"2024-09-21T00:20:26.452141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:44.566386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768271+0000","last_change":"2024-09-21T00:18:38.662510+0000","last_active":"2024-09-21T00:20:54.768271+0000","last_peered":"2024-09-21T00:20:54.768271+0000","last_clean":"2024-09-21T00:20:54.768271+0000","last_became_active":"2024-09-21T00:18:38.659788+0000","last_became_peered":"2024-09-21T00:18:38.659788+0000","last_unstale":"2024-09-21T00:20:54.768271+0000","last_undegraded":"2024-09-21T00:20:54.768271+0000","last_fullsized":"2024-09-21T00:20:54.768271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:14.532084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.575037+0000","last_change":"2024-09-21T00:18:35.622949+0000","last_active":"2024-09-21T00:20:26.575037+0000","last_peered":"2024-09-21T00:20:26.575037+0000","last_clean":"2024-09-21T00:20:26.575037+0000","last_became_active":"2024-09-21T00:18:35.621787+0000","last_became_peered":"2024-09-21T00:18:35.621787+0000","last_unstale":"2024-09-21T00:20:26.575037+0000","last_undegraded":"2024-09-21T00:20:26.575037+0000","last_fullsized":"2024-09-21T00:20:26.575037+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:39:15.060123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452739+0000","last_change":"2024-09-21T00:18:48.730711+0000","last_active":"2024-09-21T00:20:26.452739+0000","last_peered":"2024-09-21T00:20:26.452739+0000","last_clean":"2024-09-21T00:20:26.452739+0000","last_became_active":"2024-09-21T00:18:48.730287+0000","last_became_peered":"2024-09-21T00:18:48.730287+0000","last_unstale":"2024-09-21T00:20:26.452739+0000","last_undegraded":"2024-09-21T00:20:26.452739+0000","last_fullsized":"2024-09-21T00:20:26.452739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:34.444747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.734412+0000","last_change":"2024-09-21T00:18:38.669381+0000","last_active":"2024-09-21T00:20:54.734412+0000","last_peered":"2024-09-21T00:20:54.734412+0000","last_clean":"2024-09-21T00:20:54.734412+0000","last_became_active":"2024-09-21T00:18:38.668757+0000","last_became_peered":"2024-09-21T00:18:38.668757+0000","last_unstale":"2024-09-21T00:20:54.734412+0000","last_undegraded":"2024-09-21T00:20:54.734412+0000","last_fullsized":"2024-09-21T00:20:54.734412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:52.475872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508594+0000","last_change":"2024-09-21T00:18:35.767700+0000","last_active":"2024-09-21T00:20:26.508594+0000","last_peered":"2024-09-21T00:20:26.508594+0000","last_clean":"2024-09-21T00:20:26.508594+0000","last_became_active":"2024-09-21T00:18:35.767369+0000","last_became_peered":"2024-09-21T00:18:35.767369+0000","last_unstale":"2024-09-21T00:20:26.508594+0000","last_undegraded":"2024-09-21T00:20:26.508594+0000","last_fullsized":"2024-09-21T00:20:26.508594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:44.955659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452286+0000","last_change":"2024-09-21T00:18:48.737411+0000","last_active":"2024-09-21T00:20:26.452286+0000","last_peered":"2024-09-21T00:20:26.452286+0000","last_clean":"2024-09-21T00:20:26.452286+0000","last_became_active":"2024-09-21T00:18:48.736715+0000","last_became_peered":"2024-09-21T00:18:48.736715+0000","last_unstale":"2024-09-21T00:20:26.452286+0000","last_undegraded":"2024-09-21T00:20:26.452286+0000","last_fullsized":"2024-09-21T00:20:26.452286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:01.045318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768197+0000","last_change":"2024-09-21T00:19:25.842355+0000","last_active":"2024-09-21T00:20:54.768197+0000","last_peered":"2024-09-21T00:20:54.768197+0000","last_clean":"2024-09-21T00:20:54.768197+0000","last_became_active":"2024-09-21T00:19:25.841961+0000","last_became_peered":"2024-09-21T00:19:25.841961+0000","last_unstale":"2024-09-21T00:20:54.768197+0000","last_undegraded":"2024-09-21T00:20:54.768197+0000","last_fullsized":"2024-09-21T00:20:54.768197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:30.490992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452475+0000","last_change":"2024-09-21T00:18:35.919003+0000","last_active":"2024-09-21T00:20:26.452475+0000","last_peered":"2024-09-21T00:20:26.452475+0000","last_clean":"2024-09-21T00:20:26.452475+0000","last_became_active":"2024-09-21T00:18:35.918593+0000","last_became_peered":"2024-09-21T00:18:35.918593+0000","last_unstale":"2024-09-21T00:20:26.452475+0000","last_undegraded":"2024-09-21T00:20:26.452475+0000","last_fullsized":"2024-09-21T00:20:26.452475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:35.982726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451542+0000","last_change":"2024-09-21T00:18:48.725906+0000","last_active":"2024-09-21T00:20:26.451542+0000","last_peered":"2024-09-21T00:20:26.451542+0000","last_clean":"2024-09-21T00:20:26.451542+0000","last_became_active":"2024-09-21T00:18:48.725450+0000","last_became_peered":"2024-09-21T00:18:48.725450+0000","last_unstale":"2024-09-21T00:20:26.451542+0000","last_undegraded":"2024-09-21T00:20:26.451542+0000","last_fullsized":"2024-09-21T00:20:26.451542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:35.163208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764430+0000","last_change":"2024-09-21T00:18:38.666442+0000","last_active":"2024-09-21T00:20:54.764430+0000","last_peered":"2024-09-21T00:20:54.764430+0000","last_clean":"2024-09-21T00:20:54.764430+0000","last_became_active":"2024-09-21T00:18:38.666083+0000","last_became_peered":"2024-09-21T00:18:38.666083+0000","last_unstale":"2024-09-21T00:20:54.764430+0000","last_undegraded":"2024-09-21T00:20:54.764430+0000","last_fullsized":"2024-09-21T00:20:54.764430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:39.986440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'94","reported_seq":225,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:37.878132+0000","last_change":"2024-09-21T00:18:35.624883+0000","last_active":"2024-09-21T00:21:37.878132+0000","last_peered":"2024-09-21T00:21:37.878132+0000","last_clean":"2024-09-21T00:21:37.878132+0000","last_became_active":"2024-09-21T00:18:35.624382+0000","last_became_peered":"2024-09-21T00:18:35.624382+0000","last_unstale":"2024-09-21T00:21:37.878132+0000","last_undegraded":"2024-09-21T00:21:37.878132+0000","last_fullsized":"2024-09-21T00:21:37.878132+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:23.475521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":259,"num_write":172,"num_write_kb":1220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574454+0000","last_change":"2024-09-21T00:18:48.719850+0000","last_active":"2024-09-21T00:20:26.574454+0000","last_peered":"2024-09-21T00:20:26.574454+0000","last_clean":"2024-09-21T00:20:26.574454+0000","last_became_active":"2024-09-21T00:18:48.719414+0000","last_became_peered":"2024-09-21T00:18:48.719414+0000","last_unstale":"2024-09-21T00:20:26.574454+0000","last_undegraded":"2024-09-21T00:20:26.574454+0000","last_fullsized":"2024-09-21T00:20:26.574454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:05.978977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.755833+0000","last_change":"2024-09-21T00:18:38.671920+0000","last_active":"2024-09-21T00:20:54.755833+0000","last_peered":"2024-09-21T00:20:54.755833+0000","last_clean":"2024-09-21T00:20:54.755833+0000","last_became_active":"2024-09-21T00:18:38.671592+0000","last_became_peered":"2024-09-21T00:18:38.671592+0000","last_unstale":"2024-09-21T00:20:54.755833+0000","last_undegraded":"2024-09-21T00:20:54.755833+0000","last_fullsized":"2024-09-21T00:20:54.755833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:40:48.257086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.511127+0000","last_change":"2024-09-21T00:18:35.769197+0000","last_active":"2024-09-21T00:20:26.511127+0000","last_peered":"2024-09-21T00:20:26.511127+0000","last_clean":"2024-09-21T00:20:26.511127+0000","last_became_active":"2024-09-21T00:18:35.768740+0000","last_became_peered":"2024-09-21T00:18:35.768740+0000","last_unstale":"2024-09-21T00:20:26.511127+0000","last_undegraded":"2024-09-21T00:20:26.511127+0000","last_fullsized":"2024-09-21T00:20:26.511127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:32.253006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454104+0000","last_change":"2024-09-21T00:18:48.740189+0000","last_active":"2024-09-21T00:20:26.454104+0000","last_peered":"2024-09-21T00:20:26.454104+0000","last_clean":"2024-09-21T00:20:26.454104+0000","last_became_active":"2024-09-21T00:18:48.739700+0000","last_became_peered":"2024-09-21T00:18:48.739700+0000","last_unstale":"2024-09-21T00:20:26.454104+0000","last_undegraded":"2024-09-21T00:20:26.454104+0000","last_fullsized":"2024-09-21T00:20:26.454104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:06.312160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764470+0000","last_change":"2024-09-21T00:18:38.665635+0000","last_active":"2024-09-21T00:20:54.764470+0000","last_peered":"2024-09-21T00:20:54.764470+0000","last_clean":"2024-09-21T00:20:54.764470+0000","last_became_active":"2024-09-21T00:18:38.665236+0000","last_became_peered":"2024-09-21T00:18:38.665236+0000","last_unstale":"2024-09-21T00:20:54.764470+0000","last_undegraded":"2024-09-21T00:20:54.764470+0000","last_fullsized":"2024-09-21T00:20:54.764470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:52.186480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452058+0000","last_change":"2024-09-21T00:18:35.771465+0000","last_active":"2024-09-21T00:20:26.452058+0000","last_peered":"2024-09-21T00:20:26.452058+0000","last_clean":"2024-09-21T00:20:26.452058+0000","last_became_active":"2024-09-21T00:18:35.770550+0000","last_became_peered":"2024-09-21T00:18:35.770550+0000","last_unstale":"2024-09-21T00:20:26.452058+0000","last_undegraded":"2024-09-21T00:20:26.452058+0000","last_fullsized":"2024-09-21T00:20:26.452058+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:43.042563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510077+0000","last_change":"2024-09-21T00:18:48.736334+0000","last_active":"2024-09-21T00:20:26.510077+0000","last_peered":"2024-09-21T00:20:26.510077+0000","last_clean":"2024-09-21T00:20:26.510077+0000","last_became_active":"2024-09-21T00:18:48.735784+0000","last_became_peered":"2024-09-21T00:18:48.735784+0000","last_unstale":"2024-09-21T00:20:26.510077+0000","last_undegraded":"2024-09-21T00:20:26.510077+0000","last_fullsized":"2024-09-21T00:20:26.510077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:29.637335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767464+0000","last_change":"2024-09-21T00:18:38.674181+0000","last_active":"2024-09-21T00:20:54.767464+0000","last_peered":"2024-09-21T00:20:54.767464+0000","last_clean":"2024-09-21T00:20:54.767464+0000","last_became_active":"2024-09-21T00:18:38.673784+0000","last_became_peered":"2024-09-21T00:18:38.673784+0000","last_unstale":"2024-09-21T00:20:54.767464+0000","last_undegraded":"2024-09-21T00:20:54.767464+0000","last_fullsized":"2024-09-21T00:20:54.767464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:33.471328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510165+0000","last_change":"2024-09-21T00:18:35.628149+0000","last_active":"2024-09-21T00:20:26.510165+0000","last_peered":"2024-09-21T00:20:26.510165+0000","last_clean":"2024-09-21T00:20:26.510165+0000","last_became_active":"2024-09-21T00:18:35.627674+0000","last_became_peered":"2024-09-21T00:18:35.627674+0000","last_unstale":"2024-09-21T00:20:26.510165+0000","last_undegraded":"2024-09-21T00:20:26.510165+0000","last_fullsized":"2024-09-21T00:20:26.510165+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:28:47.244288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508884+0000","last_change":"2024-09-21T00:18:48.740283+0000","last_active":"2024-09-21T00:20:26.508884+0000","last_peered":"2024-09-21T00:20:26.508884+0000","last_clean":"2024-09-21T00:20:26.508884+0000","last_became_active":"2024-09-21T00:18:48.739026+0000","last_became_peered":"2024-09-21T00:18:48.739026+0000","last_unstale":"2024-09-21T00:20:26.508884+0000","last_undegraded":"2024-09-21T00:20:26.508884+0000","last_fullsized":"2024-09-21T00:20:26.508884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:50.898116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.759757+0000","last_change":"2024-09-21T00:18:38.667206+0000","last_active":"2024-09-21T00:20:54.759757+0000","last_peered":"2024-09-21T00:20:54.759757+0000","last_clean":"2024-09-21T00:20:54.759757+0000","last_became_active":"2024-09-21T00:18:38.666770+0000","last_became_peered":"2024-09-21T00:18:38.666770+0000","last_unstale":"2024-09-21T00:20:54.759757+0000","last_undegraded":"2024-09-21T00:20:54.759757+0000","last_fullsized":"2024-09-21T00:20:54.759757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:56.332071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":171,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:36.110950+0000","last_change":"2024-09-21T00:18:35.768012+0000","last_active":"2024-09-21T00:21:36.110950+0000","last_peered":"2024-09-21T00:21:36.110950+0000","last_clean":"2024-09-21T00:21:36.110950+0000","last_became_active":"2024-09-21T00:18:35.767438+0000","last_became_peered":"2024-09-21T00:18:35.767438+0000","last_unstale":"2024-09-21T00:21:36.110950+0000","last_undegraded":"2024-09-21T00:21:36.110950+0000","last_fullsized":"2024-09-21T00:21:36.110950+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:35:40.606871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508643+0000","last_change":"2024-09-21T00:18:48.738228+0000","last_active":"2024-09-21T00:20:26.508643+0000","last_peered":"2024-09-21T00:20:26.508643+0000","last_clean":"2024-09-21T00:20:26.508643+0000","last_became_active":"2024-09-21T00:18:48.737786+0000","last_became_peered":"2024-09-21T00:18:48.737786+0000","last_unstale":"2024-09-21T00:20:26.508643+0000","last_undegraded":"2024-09-21T00:20:26.508643+0000","last_fullsized":"2024-09-21T00:20:26.508643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:52.329677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754320+0000","last_change":"2024-09-21T00:18:38.678126+0000","last_active":"2024-09-21T00:20:54.754320+0000","last_peered":"2024-09-21T00:20:54.754320+0000","last_clean":"2024-09-21T00:20:54.754320+0000","last_became_active":"2024-09-21T00:18:38.677757+0000","last_became_peered":"2024-09-21T00:18:38.677757+0000","last_unstale":"2024-09-21T00:20:54.754320+0000","last_undegraded":"2024-09-21T00:20:54.754320+0000","last_fullsized":"2024-09-21T00:20:54.754320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:26.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508836+0000","last_change":"2024-09-21T00:18:35.625955+0000","last_active":"2024-09-21T00:20:26.508836+0000","last_peered":"2024-09-21T00:20:26.508836+0000","last_clean":"2024-09-21T00:20:26.508836+0000","last_became_active":"2024-09-21T00:18:35.625548+0000","last_became_peered":"2024-09-21T00:18:35.625548+0000","last_unstale":"2024-09-21T00:20:26.508836+0000","last_undegraded":"2024-09-21T00:20:26.508836+0000","last_fullsized":"2024-09-21T00:20:26.508836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:04.163421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509821+0000","last_change":"2024-09-21T00:18:48.740765+0000","last_active":"2024-09-21T00:20:26.509821+0000","last_peered":"2024-09-21T00:20:26.509821+0000","last_clean":"2024-09-21T00:20:26.509821+0000","last_became_active":"2024-09-21T00:18:48.740081+0000","last_became_peered":"2024-09-21T00:18:48.740081+0000","last_unstale":"2024-09-21T00:20:26.509821+0000","last_undegraded":"2024-09-21T00:20:26.509821+0000","last_fullsized":"2024-09-21T00:20:26.509821+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:38.251475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.755958+0000","last_change":"2024-09-21T00:18:38.667581+0000","last_active":"2024-09-21T00:20:54.755958+0000","last_peered":"2024-09-21T00:20:54.755958+0000","last_clean":"2024-09-21T00:20:54.755958+0000","last_became_active":"2024-09-21T00:18:38.665959+0000","last_became_peered":"2024-09-21T00:18:38.665959+0000","last_unstale":"2024-09-21T00:20:54.755958+0000","last_undegraded":"2024-09-21T00:20:54.755958+0000","last_fullsized":"2024-09-21T00:20:54.755958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:52.321654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452596+0000","last_change":"2024-09-21T00:18:35.624291+0000","last_active":"2024-09-21T00:20:26.452596+0000","last_peered":"2024-09-21T00:20:26.452596+0000","last_clean":"2024-09-21T00:20:26.452596+0000","last_became_active":"2024-09-21T00:18:35.623297+0000","last_became_peered":"2024-09-21T00:18:35.623297+0000","last_unstale":"2024-09-21T00:20:26.452596+0000","last_undegraded":"2024-09-21T00:20:26.452596+0000","last_fullsized":"2024-09-21T00:20:26.452596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:44.824409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509191+0000","last_change":"2024-09-21T00:17:21.233627+0000","last_active":"2024-09-21T00:20:26.509191+0000","last_peered":"2024-09-21T00:20:26.509191+0000","last_clean":"2024-09-21T00:20:26.509191+0000","last_became_active":"2024-09-21T00:17:21.227897+0000","last_became_peered":"2024-09-21T00:17:21.227897+0000","last_unstale":"2024-09-21T00:20:26.509191+0000","last_undegraded":"2024-09-21T00:20:26.509191+0000","last_fullsized":"2024-09-21T00:20:26.509191+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:13:33.120691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:13:33.120691+0000","last_clean_scrub_stamp":"2024-09-21T00:13:33.120691+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:20.198660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509421+0000","last_change":"2024-09-21T00:18:48.730107+0000","last_active":"2024-09-21T00:20:26.509421+0000","last_peered":"2024-09-21T00:20:26.509421+0000","last_clean":"2024-09-21T00:20:26.509421+0000","last_became_active":"2024-09-21T00:18:48.729463+0000","last_became_peered":"2024-09-21T00:18:48.729463+0000","last_unstale":"2024-09-21T00:20:26.509421+0000","last_undegraded":"2024-09-21T00:20:26.509421+0000","last_fullsized":"2024-09-21T00:20:26.509421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:55.352870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764244+0000","last_change":"2024-09-21T00:18:38.671244+0000","last_active":"2024-09-21T00:20:54.764244+0000","last_peered":"2024-09-21T00:20:54.764244+0000","last_clean":"2024-09-21T00:20:54.764244+0000","last_became_active":"2024-09-21T00:18:38.670630+0000","last_became_peered":"2024-09-21T00:18:38.670630+0000","last_unstale":"2024-09-21T00:20:54.764244+0000","last_undegraded":"2024-09-21T00:20:54.764244+0000","last_fullsized":"2024-09-21T00:20:54.764244+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:23.695660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451789+0000","last_change":"2024-09-21T00:19:25.843900+0000","last_active":"2024-09-21T00:20:26.451789+0000","last_peered":"2024-09-21T00:20:26.451789+0000","last_clean":"2024-09-21T00:20:26.451789+0000","last_became_active":"2024-09-21T00:19:25.843438+0000","last_became_peered":"2024-09-21T00:19:25.843438+0000","last_unstale":"2024-09-21T00:20:26.451789+0000","last_undegraded":"2024-09-21T00:20:26.451789+0000","last_fullsized":"2024-09-21T00:20:26.451789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:17.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509404+0000","last_change":"2024-09-21T00:18:48.739640+0000","last_active":"2024-09-21T00:20:26.509404+0000","last_peered":"2024-09-21T00:20:26.509404+0000","last_clean":"2024-09-21T00:20:26.509404+0000","last_became_active":"2024-09-21T00:18:48.739118+0000","last_became_peered":"2024-09-21T00:18:48.739118+0000","last_unstale":"2024-09-21T00:20:26.509404+0000","last_undegraded":"2024-09-21T00:20:26.509404+0000","last_fullsized":"2024-09-21T00:20:26.509404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:03.093561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.766430+0000","last_change":"2024-09-21T00:18:38.673020+0000","last_active":"2024-09-21T00:20:54.766430+0000","last_peered":"2024-09-21T00:20:54.766430+0000","last_clean":"2024-09-21T00:20:54.766430+0000","last_became_active":"2024-09-21T00:18:38.672391+0000","last_became_peered":"2024-09-21T00:18:38.672391+0000","last_unstale":"2024-09-21T00:20:54.766430+0000","last_undegraded":"2024-09-21T00:20:54.766430+0000","last_fullsized":"2024-09-21T00:20:54.766430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:13.866576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'6","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:36.677909+0000","last_change":"2024-09-21T00:18:35.615844+0000","last_active":"2024-09-21T00:21:36.677909+0000","last_peered":"2024-09-21T00:21:36.677909+0000","last_clean":"2024-09-21T00:21:36.677909+0000","last_became_active":"2024-09-21T00:18:35.615385+0000","last_became_peered":"2024-09-21T00:18:35.615385+0000","last_unstale":"2024-09-21T00:21:36.677909+0000","last_undegraded":"2024-09-21T00:21:36.677909+0000","last_fullsized":"2024-09-21T00:21:36.677909+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:03.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.512987+0000","last_change":"2024-09-21T00:20:26.511995+0000","last_active":"2024-09-21T00:20:26.512987+0000","last_peered":"2024-09-21T00:20:26.512987+0000","last_clean":"2024-09-21T00:20:26.512987+0000","last_became_active":"2024-09-21T00:20:26.511607+0000","last_became_peered":"2024-09-21T00:20:26.511607+0000","last_unstale":"2024-09-21T00:20:26.512987+0000","last_undegraded":"2024-09-21T00:20:26.512987+0000","last_fullsized":"2024-09-21T00:20:26.512987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:38.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754513+0000","last_change":"2024-09-21T00:18:38.671280+0000","last_active":"2024-09-21T00:20:54.754513+0000","last_peered":"2024-09-21T00:20:54.754513+0000","last_clean":"2024-09-21T00:20:54.754513+0000","last_became_active":"2024-09-21T00:18:38.670770+0000","last_became_peered":"2024-09-21T00:18:38.670770+0000","last_unstale":"2024-09-21T00:20:54.754513+0000","last_undegraded":"2024-09-21T00:20:54.754513+0000","last_fullsized":"2024-09-21T00:20:54.754513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:06.809450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509590+0000","last_change":"2024-09-21T00:18:35.626071+0000","last_active":"2024-09-21T00:20:26.509590+0000","last_peered":"2024-09-21T00:20:26.509590+0000","last_clean":"2024-09-21T00:20:26.509590+0000","last_became_active":"2024-09-21T00:18:35.625649+0000","last_became_peered":"2024-09-21T00:18:35.625649+0000","last_unstale":"2024-09-21T00:20:26.509590+0000","last_undegraded":"2024-09-21T00:20:26.509590+0000","last_fullsized":"2024-09-21T00:20:26.509590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:16:27.516303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508631+0000","last_change":"2024-09-21T00:18:48.728451+0000","last_active":"2024-09-21T00:20:26.508631+0000","last_peered":"2024-09-21T00:20:26.508631+0000","last_clean":"2024-09-21T00:20:26.508631+0000","last_became_active":"2024-09-21T00:18:48.727840+0000","last_became_peered":"2024-09-21T00:18:48.727840+0000","last_unstale":"2024-09-21T00:20:26.508631+0000","last_undegraded":"2024-09-21T00:20:26.508631+0000","last_fullsized":"2024-09-21T00:20:26.508631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:13.457138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754460+0000","last_change":"2024-09-21T00:18:38.664090+0000","last_active":"2024-09-21T00:20:54.754460+0000","last_peered":"2024-09-21T00:20:54.754460+0000","last_clean":"2024-09-21T00:20:54.754460+0000","last_became_active":"2024-09-21T00:18:38.663728+0000","last_became_peered":"2024-09-21T00:18:38.663728+0000","last_unstale":"2024-09-21T00:20:54.754460+0000","last_undegraded":"2024-09-21T00:20:54.754460+0000","last_fullsized":"2024-09-21T00:20:54.754460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:46.367766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454914+0000","last_change":"2024-09-21T00:19:25.847280+0000","last_active":"2024-09-21T00:20:26.454914+0000","last_peered":"2024-09-21T00:20:26.454914+0000","last_clean":"2024-09-21T00:20:26.454914+0000","last_became_active":"2024-09-21T00:19:25.840919+0000","last_became_peered":"2024-09-21T00:19:25.840919+0000","last_unstale":"2024-09-21T00:20:26.454914+0000","last_undegraded":"2024-09-21T00:20:26.454914+0000","last_fullsized":"2024-09-21T00:20:26.454914+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:15.411674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508811+0000","last_change":"2024-09-21T00:18:48.732427+0000","last_active":"2024-09-21T00:20:26.508811+0000","last_peered":"2024-09-21T00:20:26.508811+0000","last_clean":"2024-09-21T00:20:26.508811+0000","last_became_active":"2024-09-21T00:18:48.732078+0000","last_became_peered":"2024-09-21T00:18:48.732078+0000","last_unstale":"2024-09-21T00:20:26.508811+0000","last_undegraded":"2024-09-21T00:20:26.508811+0000","last_fullsized":"2024-09-21T00:20:26.508811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:35.055680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764481+0000","last_change":"2024-09-21T00:18:38.675022+0000","last_active":"2024-09-21T00:20:54.764481+0000","last_peered":"2024-09-21T00:20:54.764481+0000","last_clean":"2024-09-21T00:20:54.764481+0000","last_became_active":"2024-09-21T00:18:38.674650+0000","last_became_peered":"2024-09-21T00:18:38.674650+0000","last_unstale":"2024-09-21T00:20:54.764481+0000","last_undegraded":"2024-09-21T00:20:54.764481+0000","last_fullsized":"2024-09-21T00:20:54.764481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:05.191404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510139+0000","last_change":"2024-09-21T00:18:35.769555+0000","last_active":"2024-09-21T00:20:26.510139+0000","last_peered":"2024-09-21T00:20:26.510139+0000","last_clean":"2024-09-21T00:20:26.510139+0000","last_became_active":"2024-09-21T00:18:35.769080+0000","last_became_peered":"2024-09-21T00:18:35.769080+0000","last_unstale":"2024-09-21T00:20:26.510139+0000","last_undegraded":"2024-09-21T00:20:26.510139+0000","last_fullsized":"2024-09-21T00:20:26.510139+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:19.306830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509529+0000","last_change":"2024-09-21T00:18:48.742957+0000","last_active":"2024-09-21T00:20:26.509529+0000","last_peered":"2024-09-21T00:20:26.509529+0000","last_clean":"2024-09-21T00:20:26.509529+0000","last_became_active":"2024-09-21T00:18:48.742511+0000","last_became_peered":"2024-09-21T00:18:48.742511+0000","last_unstale":"2024-09-21T00:20:26.509529+0000","last_undegraded":"2024-09-21T00:20:26.509529+0000","last_fullsized":"2024-09-21T00:20:26.509529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:15.231189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.761135+0000","last_change":"2024-09-21T00:18:38.666576+0000","last_active":"2024-09-21T00:20:54.761135+0000","last_peered":"2024-09-21T00:20:54.761135+0000","last_clean":"2024-09-21T00:20:54.761135+0000","last_became_active":"2024-09-21T00:18:38.665643+0000","last_became_peered":"2024-09-21T00:18:38.665643+0000","last_unstale":"2024-09-21T00:20:54.761135+0000","last_undegraded":"2024-09-21T00:20:54.761135+0000","last_fullsized":"2024-09-21T00:20:54.761135+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:35.397847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":172,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:35.109955+0000","last_change":"2024-09-21T00:18:35.623143+0000","last_active":"2024-09-21T00:21:35.109955+0000","last_peered":"2024-09-21T00:21:35.109955+0000","last_clean":"2024-09-21T00:21:35.109955+0000","last_became_active":"2024-09-21T00:18:35.622693+0000","last_became_peered":"2024-09-21T00:18:35.622693+0000","last_unstale":"2024-09-21T00:21:35.109955+0000","last_undegraded":"2024-09-21T00:21:35.109955+0000","last_fullsized":"2024-09-21T00:21:35.109955+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:01.331665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452872+0000","last_change":"2024-09-21T00:18:48.723877+0000","last_active":"2024-09-21T00:20:26.452872+0000","last_peered":"2024-09-21T00:20:26.452872+0000","last_clean":"2024-09-21T00:20:26.452872+0000","last_became_active":"2024-09-21T00:18:48.723360+0000","last_became_peered":"2024-09-21T00:18:48.723360+0000","last_unstale":"2024-09-21T00:20:26.452872+0000","last_undegraded":"2024-09-21T00:20:26.452872+0000","last_fullsized":"2024-09-21T00:20:26.452872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:07:20.442867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767171+0000","last_change":"2024-09-21T00:18:38.669643+0000","last_active":"2024-09-21T00:20:54.767171+0000","last_peered":"2024-09-21T00:20:54.767171+0000","last_clean":"2024-09-21T00:20:54.767171+0000","last_became_active":"2024-09-21T00:18:38.669245+0000","last_became_peered":"2024-09-21T00:18:38.669245+0000","last_unstale":"2024-09-21T00:20:54.767171+0000","last_undegraded":"2024-09-21T00:20:54.767171+0000","last_fullsized":"2024-09-21T00:20:54.767171+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:17.789291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509577+0000","last_change":"2024-09-21T00:18:35.623189+0000","last_active":"2024-09-21T00:20:26.509577+0000","last_peered":"2024-09-21T00:20:26.509577+0000","last_clean":"2024-09-21T00:20:26.509577+0000","last_became_active":"2024-09-21T00:18:35.622748+0000","last_became_peered":"2024-09-21T00:18:35.622748+0000","last_unstale":"2024-09-21T00:20:26.509577+0000","last_undegraded":"2024-09-21T00:20:26.509577+0000","last_fullsized":"2024-09-21T00:20:26.509577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:15.570789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453838+0000","last_change":"2024-09-21T00:18:48.739612+0000","last_active":"2024-09-21T00:20:26.453838+0000","last_peered":"2024-09-21T00:20:26.453838+0000","last_clean":"2024-09-21T00:20:26.453838+0000","last_became_active":"2024-09-21T00:18:48.739173+0000","last_became_peered":"2024-09-21T00:18:48.739173+0000","last_unstale":"2024-09-21T00:20:26.453838+0000","last_undegraded":"2024-09-21T00:20:26.453838+0000","last_fullsized":"2024-09-21T00:20:26.453838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:26.327178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.766649+0000","last_change":"2024-09-21T00:18:38.670876+0000","last_active":"2024-09-21T00:20:54.766649+0000","last_peered":"2024-09-21T00:20:54.766649+0000","last_clean":"2024-09-21T00:20:54.766649+0000","last_became_active":"2024-09-21T00:18:38.670371+0000","last_became_peered":"2024-09-21T00:18:38.670371+0000","last_unstale":"2024-09-21T00:20:54.766649+0000","last_undegraded":"2024-09-21T00:20:54.766649+0000","last_fullsized":"2024-09-21T00:20:54.766649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:48.641071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452030+0000","last_change":"2024-09-21T00:18:35.622919+0000","last_active":"2024-09-21T00:20:26.452030+0000","last_peered":"2024-09-21T00:20:26.452030+0000","last_clean":"2024-09-21T00:20:26.452030+0000","last_became_active":"2024-09-21T00:18:35.622435+0000","last_became_peered":"2024-09-21T00:18:35.622435+0000","last_unstale":"2024-09-21T00:20:26.452030+0000","last_undegraded":"2024-09-21T00:20:26.452030+0000","last_fullsized":"2024-09-21T00:20:26.452030+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:22.113871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509031+0000","last_change":"2024-09-21T00:18:48.741736+0000","last_active":"2024-09-21T00:20:26.509031+0000","last_peered":"2024-09-21T00:20:26.509031+0000","last_clean":"2024-09-21T00:20:26.509031+0000","last_became_active":"2024-09-21T00:18:48.741135+0000","last_became_peered":"2024-09-21T00:18:48.741135+0000","last_unstale":"2024-09-21T00:20:26.509031+0000","last_undegraded":"2024-09-21T00:20:26.509031+0000","last_fullsized":"2024-09-21T00:20:26.509031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:50.492351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.748256+0000","last_change":"2024-09-21T00:18:38.667377+0000","last_active":"2024-09-21T00:20:54.748256+0000","last_peered":"2024-09-21T00:20:54.748256+0000","last_clean":"2024-09-21T00:20:54.748256+0000","last_became_active":"2024-09-21T00:18:38.667010+0000","last_became_peered":"2024-09-21T00:18:38.667010+0000","last_unstale":"2024-09-21T00:20:54.748256+0000","last_undegraded":"2024-09-21T00:20:54.748256+0000","last_fullsized":"2024-09-21T00:20:54.748256+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:06.634767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":361,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:40.117626+0000","last_change":"2024-09-21T00:19:25.853603+0000","last_active":"2024-09-21T00:21:40.117626+0000","last_peered":"2024-09-21T00:21:40.117626+0000","last_clean":"2024-09-21T00:21:40.117626+0000","last_became_active":"2024-09-21T00:19:25.841126+0000","last_became_peered":"2024-09-21T00:19:25.841126+0000","last_unstale":"2024-09-21T00:21:40.117626+0000","last_undegraded":"2024-09-21T00:21:40.117626+0000","last_fullsized":"2024-09-21T00:21:40.117626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:27:47.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71662,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":261,"num_read_kb":201,"num_write":38,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451696+0000","last_change":"2024-09-21T00:18:48.731004+0000","last_active":"2024-09-21T00:20:26.451696+0000","last_peered":"2024-09-21T00:20:26.451696+0000","last_clean":"2024-09-21T00:20:26.451696+0000","last_became_active":"2024-09-21T00:18:48.730549+0000","last_became_peered":"2024-09-21T00:18:48.730549+0000","last_unstale":"2024-09-21T00:20:26.451696+0000","last_undegraded":"2024-09-21T00:20:26.451696+0000","last_fullsized":"2024-09-21T00:20:26.451696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:30.988300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764366+0000","last_change":"2024-09-21T00:19:25.843532+0000","last_active":"2024-09-21T00:20:54.764366+0000","last_peered":"2024-09-21T00:20:54.764366+0000","last_clean":"2024-09-21T00:20:54.764366+0000","last_became_active":"2024-09-21T00:19:25.843065+0000","last_became_peered":"2024-09-21T00:19:25.843065+0000","last_unstale":"2024-09-21T00:20:54.764366+0000","last_undegraded":"2024-09-21T00:20:54.764366+0000","last_fullsized":"2024-09-21T00:20:54.764366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:36:26.488692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509317+0000","last_change":"2024-09-21T00:18:35.770494+0000","last_active":"2024-09-21T00:20:26.509317+0000","last_peered":"2024-09-21T00:20:26.509317+0000","last_clean":"2024-09-21T00:20:26.509317+0000","last_became_active":"2024-09-21T00:18:35.770122+0000","last_became_peered":"2024-09-21T00:18:35.770122+0000","last_unstale":"2024-09-21T00:20:26.509317+0000","last_undegraded":"2024-09-21T00:20:26.509317+0000","last_fullsized":"2024-09-21T00:20:26.509317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:41.415904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451852+0000","last_change":"2024-09-21T00:18:48.736802+0000","last_active":"2024-09-21T00:20:26.451852+0000","last_peered":"2024-09-21T00:20:26.451852+0000","last_clean":"2024-09-21T00:20:26.451852+0000","last_became_active":"2024-09-21T00:18:48.736008+0000","last_became_peered":"2024-09-21T00:18:48.736008+0000","last_unstale":"2024-09-21T00:20:26.451852+0000","last_undegraded":"2024-09-21T00:20:26.451852+0000","last_fullsized":"2024-09-21T00:20:26.451852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:07.995614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767189+0000","last_change":"2024-09-21T00:18:38.678385+0000","last_active":"2024-09-21T00:20:54.767189+0000","last_peered":"2024-09-21T00:20:54.767189+0000","last_clean":"2024-09-21T00:20:54.767189+0000","last_became_active":"2024-09-21T00:18:38.677174+0000","last_became_peered":"2024-09-21T00:18:38.677174+0000","last_unstale":"2024-09-21T00:20:54.767189+0000","last_undegraded":"2024-09-21T00:20:54.767189+0000","last_fullsized":"2024-09-21T00:20:54.767189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:23.486222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509498+0000","last_change":"2024-09-21T00:18:35.627703+0000","last_active":"2024-09-21T00:20:26.509498+0000","last_peered":"2024-09-21T00:20:26.509498+0000","last_clean":"2024-09-21T00:20:26.509498+0000","last_became_active":"2024-09-21T00:18:35.627274+0000","last_became_peered":"2024-09-21T00:18:35.627274+0000","last_unstale":"2024-09-21T00:20:26.509498+0000","last_undegraded":"2024-09-21T00:20:26.509498+0000","last_fullsized":"2024-09-21T00:20:26.509498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:14.326810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451122+0000","last_change":"2024-09-21T00:18:48.729926+0000","last_active":"2024-09-21T00:20:26.451122+0000","last_peered":"2024-09-21T00:20:26.451122+0000","last_clean":"2024-09-21T00:20:26.451122+0000","last_became_active":"2024-09-21T00:18:48.728366+0000","last_became_peered":"2024-09-21T00:18:48.728366+0000","last_unstale":"2024-09-21T00:20:26.451122+0000","last_undegraded":"2024-09-21T00:20:26.451122+0000","last_fullsized":"2024-09-21T00:20:26.451122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:48:50.005748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.763272+0000","last_change":"2024-09-21T00:18:38.664173+0000","last_active":"2024-09-21T00:20:54.763272+0000","last_peered":"2024-09-21T00:20:54.763272+0000","last_clean":"2024-09-21T00:20:54.763272+0000","last_became_active":"2024-09-21T00:18:38.663358+0000","last_became_peered":"2024-09-21T00:18:38.663358+0000","last_unstale":"2024-09-21T00:20:54.763272+0000","last_undegraded":"2024-09-21T00:20:54.763272+0000","last_fullsized":"2024-09-21T00:20:54.763272+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:34:31.203621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":175,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:39.115736+0000","last_change":"2024-09-21T00:18:35.622133+0000","last_active":"2024-09-21T00:21:39.115736+0000","last_peered":"2024-09-21T00:21:39.115736+0000","last_clean":"2024-09-21T00:21:39.115736+0000","last_became_active":"2024-09-21T00:18:35.621032+0000","last_became_peered":"2024-09-21T00:18:35.621032+0000","last_unstale":"2024-09-21T00:21:39.115736+0000","last_undegraded":"2024-09-21T00:21:39.115736+0000","last_fullsized":"2024-09-21T00:21:39.115736+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:24.121665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574990+0000","last_change":"2024-09-21T00:18:48.733339+0000","last_active":"2024-09-21T00:20:26.574990+0000","last_peered":"2024-09-21T00:20:26.574990+0000","last_clean":"2024-09-21T00:20:26.574990+0000","last_became_active":"2024-09-21T00:18:48.731843+0000","last_became_peered":"2024-09-21T00:18:48.731843+0000","last_unstale":"2024-09-21T00:20:26.574990+0000","last_undegraded":"2024-09-21T00:20:26.574990+0000","last_fullsized":"2024-09-21T00:20:26.574990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:42:55.474197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455859+0000","last_change":"2024-09-21T00:18:48.740108+0000","last_active":"2024-09-21T00:20:26.455859+0000","last_peered":"2024-09-21T00:20:26.455859+0000","last_clean":"2024-09-21T00:20:26.455859+0000","last_became_active":"2024-09-21T00:18:48.739330+0000","last_became_peered":"2024-09-21T00:18:48.739330+0000","last_unstale":"2024-09-21T00:20:26.455859+0000","last_undegraded":"2024-09-21T00:20:26.455859+0000","last_fullsized":"2024-09-21T00:20:26.455859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:00.103502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.763622+0000","last_change":"2024-09-21T00:18:38.660469+0000","last_active":"2024-09-21T00:20:54.763622+0000","last_peered":"2024-09-21T00:20:54.763622+0000","last_clean":"2024-09-21T00:20:54.763622+0000","last_became_active":"2024-09-21T00:18:38.660138+0000","last_became_peered":"2024-09-21T00:18:38.660138+0000","last_unstale":"2024-09-21T00:20:54.763622+0000","last_undegraded":"2024-09-21T00:20:54.763622+0000","last_fullsized":"2024-09-21T00:20:54.763622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:01.079112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509206+0000","last_change":"2024-09-21T00:18:35.624528+0000","last_active":"2024-09-21T00:20:26.509206+0000","last_peered":"2024-09-21T00:20:26.509206+0000","last_clean":"2024-09-21T00:20:26.509206+0000","last_became_active":"2024-09-21T00:18:35.624170+0000","last_became_peered":"2024-09-21T00:18:35.624170+0000","last_unstale":"2024-09-21T00:20:26.509206+0000","last_undegraded":"2024-09-21T00:20:26.509206+0000","last_fullsized":"2024-09-21T00:20:26.509206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:07.914521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.512325+0000","last_change":"2024-09-21T00:20:26.511824+0000","last_active":"2024-09-21T00:20:26.512325+0000","last_peered":"2024-09-21T00:20:26.512325+0000","last_clean":"2024-09-21T00:20:26.512325+0000","last_became_active":"2024-09-21T00:20:26.511254+0000","last_became_peered":"2024-09-21T00:20:26.511254+0000","last_unstale":"2024-09-21T00:20:26.512325+0000","last_undegraded":"2024-09-21T00:20:26.512325+0000","last_fullsized":"2024-09-21T00:20:26.512325+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:12.884034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.760218+0000","last_change":"2024-09-21T00:18:38.666928+0000","last_active":"2024-09-21T00:20:54.760218+0000","last_peered":"2024-09-21T00:20:54.760218+0000","last_clean":"2024-09-21T00:20:54.760218+0000","last_became_active":"2024-09-21T00:18:38.666236+0000","last_became_peered":"2024-09-21T00:18:38.666236+0000","last_unstale":"2024-09-21T00:20:54.760218+0000","last_undegraded":"2024-09-21T00:20:54.760218+0000","last_fullsized":"2024-09-21T00:20:54.760218+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:21.043778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508330+0000","last_change":"2024-09-21T00:18:35.624514+0000","last_active":"2024-09-21T00:20:26.508330+0000","last_peered":"2024-09-21T00:20:26.508330+0000","last_clean":"2024-09-21T00:20:26.508330+0000","last_became_active":"2024-09-21T00:18:35.624098+0000","last_became_peered":"2024-09-21T00:18:35.624098+0000","last_unstale":"2024-09-21T00:20:26.508330+0000","last_undegraded":"2024-09-21T00:20:26.508330+0000","last_fullsized":"2024-09-21T00:20:26.508330+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:21.777568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509891+0000","last_change":"2024-09-21T00:19:25.841232+0000","last_active":"2024-09-21T00:20:26.509891+0000","last_peered":"2024-09-21T00:20:26.509891+0000","last_clean":"2024-09-21T00:20:26.509891+0000","last_became_active":"2024-09-21T00:19:25.840655+0000","last_became_peered":"2024-09-21T00:19:25.840655+0000","last_unstale":"2024-09-21T00:20:26.509891+0000","last_undegraded":"2024-09-21T00:20:26.509891+0000","last_fullsized":"2024-09-21T00:20:26.509891+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:05.846222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.762522+0000","last_change":"2024-09-21T00:18:38.667466+0000","last_active":"2024-09-21T00:20:54.762522+0000","last_peered":"2024-09-21T00:20:54.762522+0000","last_clean":"2024-09-21T00:20:54.762522+0000","last_became_active":"2024-09-21T00:18:38.666960+0000","last_became_peered":"2024-09-21T00:18:38.666960+0000","last_unstale":"2024-09-21T00:20:54.762522+0000","last_undegraded":"2024-09-21T00:20:54.762522+0000","last_fullsized":"2024-09-21T00:20:54.762522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:29:56.614766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451707+0000","last_change":"2024-09-21T00:18:48.739656+0000","last_active":"2024-09-21T00:20:26.451707+0000","last_peered":"2024-09-21T00:20:26.451707+0000","last_clean":"2024-09-21T00:20:26.451707+0000","last_became_active":"2024-09-21T00:18:48.739228+0000","last_became_peered":"2024-09-21T00:18:48.739228+0000","last_unstale":"2024-09-21T00:20:26.451707+0000","last_undegraded":"2024-09-21T00:20:26.451707+0000","last_fullsized":"2024-09-21T00:20:26.451707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:11.427595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":178,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:35.109887+0000","last_change":"2024-09-21T00:18:35.624108+0000","last_active":"2024-09-21T00:21:35.109887+0000","last_peered":"2024-09-21T00:21:35.109887+0000","last_clean":"2024-09-21T00:21:35.109887+0000","last_became_active":"2024-09-21T00:18:35.623708+0000","last_became_peered":"2024-09-21T00:18:35.623708+0000","last_unstale":"2024-09-21T00:21:35.109887+0000","last_undegraded":"2024-09-21T00:21:35.109887+0000","last_fullsized":"2024-09-21T00:21:35.109887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:50.020662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.759755+0000","last_change":"2024-09-21T00:18:38.665402+0000","last_active":"2024-09-21T00:20:54.759755+0000","last_peered":"2024-09-21T00:20:54.759755+0000","last_clean":"2024-09-21T00:20:54.759755+0000","last_became_active":"2024-09-21T00:18:38.665034+0000","last_became_peered":"2024-09-21T00:18:38.665034+0000","last_unstale":"2024-09-21T00:20:54.759755+0000","last_undegraded":"2024-09-21T00:20:54.759755+0000","last_fullsized":"2024-09-21T00:20:54.759755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:33:10.824705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451804+0000","last_change":"2024-09-21T00:18:48.729928+0000","last_active":"2024-09-21T00:20:26.451804+0000","last_peered":"2024-09-21T00:20:26.451804+0000","last_clean":"2024-09-21T00:20:26.451804+0000","last_became_active":"2024-09-21T00:18:48.729498+0000","last_became_peered":"2024-09-21T00:18:48.729498+0000","last_unstale":"2024-09-21T00:20:26.451804+0000","last_undegraded":"2024-09-21T00:20:26.451804+0000","last_fullsized":"2024-09-21T00:20:26.451804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:10:27.140012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754670+0000","last_change":"2024-09-21T00:18:38.662221+0000","last_active":"2024-09-21T00:20:54.754670+0000","last_peered":"2024-09-21T00:20:54.754670+0000","last_clean":"2024-09-21T00:20:54.754670+0000","last_became_active":"2024-09-21T00:18:38.660543+0000","last_became_peered":"2024-09-21T00:18:38.660543+0000","last_unstale":"2024-09-21T00:20:54.754670+0000","last_undegraded":"2024-09-21T00:20:54.754670+0000","last_fullsized":"2024-09-21T00:20:54.754670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:43.159699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.759670+0000","last_change":"2024-09-21T00:18:38.672696+0000","last_active":"2024-09-21T00:20:54.759670+0000","last_peered":"2024-09-21T00:20:54.759670+0000","last_clean":"2024-09-21T00:20:54.759670+0000","last_became_active":"2024-09-21T00:18:38.672356+0000","last_became_peered":"2024-09-21T00:18:38.672356+0000","last_unstale":"2024-09-21T00:20:54.759670+0000","last_undegraded":"2024-09-21T00:20:54.759670+0000","last_fullsized":"2024-09-21T00:20:54.759670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.549939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768251+0000","last_change":"2024-09-21T00:18:38.667017+0000","last_active":"2024-09-21T00:20:54.768251+0000","last_peered":"2024-09-21T00:20:54.768251+0000","last_clean":"2024-09-21T00:20:54.768251+0000","last_became_active":"2024-09-21T00:18:38.666643+0000","last_became_peered":"2024-09-21T00:18:38.666643+0000","last_unstale":"2024-09-21T00:20:54.768251+0000","last_undegraded":"2024-09-21T00:20:54.768251+0000","last_fullsized":"2024-09-21T00:20:54.768251+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:30.496339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767616+0000","last_change":"2024-09-21T00:18:38.667705+0000","last_active":"2024-09-21T00:20:54.767616+0000","last_peered":"2024-09-21T00:20:54.767616+0000","last_clean":"2024-09-21T00:20:54.767616+0000","last_became_active":"2024-09-21T00:18:38.665141+0000","last_became_peered":"2024-09-21T00:18:38.665141+0000","last_unstale":"2024-09-21T00:20:54.767616+0000","last_undegraded":"2024-09-21T00:20:54.767616+0000","last_fullsized":"2024-09-21T00:20:54.767616+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:17.624035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764929+0000","last_change":"2024-09-21T00:18:38.681364+0000","last_active":"2024-09-21T00:20:54.764929+0000","last_peered":"2024-09-21T00:20:54.764929+0000","last_clean":"2024-09-21T00:20:54.764929+0000","last_became_active":"2024-09-21T00:18:38.680937+0000","last_became_peered":"2024-09-21T00:18:38.680937+0000","last_unstale":"2024-09-21T00:20:54.764929+0000","last_undegraded":"2024-09-21T00:20:54.764929+0000","last_fullsized":"2024-09-21T00:20:54.764929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:34.206201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754004+0000","last_change":"2024-09-21T00:19:25.843091+0000","last_active":"2024-09-21T00:20:54.754004+0000","last_peered":"2024-09-21T00:20:54.754004+0000","last_clean":"2024-09-21T00:20:54.754004+0000","last_became_active":"2024-09-21T00:19:25.842600+0000","last_became_peered":"2024-09-21T00:19:25.842600+0000","last_unstale":"2024-09-21T00:20:54.754004+0000","last_undegraded":"2024-09-21T00:20:54.754004+0000","last_fullsized":"2024-09-21T00:20:54.754004+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:10.256474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.753901+0000","last_change":"2024-09-21T00:18:38.668656+0000","last_active":"2024-09-21T00:20:54.753901+0000","last_peered":"2024-09-21T00:20:54.753901+0000","last_clean":"2024-09-21T00:20:54.753901+0000","last_became_active":"2024-09-21T00:18:38.668153+0000","last_became_peered":"2024-09-21T00:18:38.668153+0000","last_unstale":"2024-09-21T00:20:54.753901+0000","last_undegraded":"2024-09-21T00:20:54.753901+0000","last_fullsized":"2024-09-21T00:20:54.753901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:03.763609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.743954+0000","last_change":"2024-09-21T00:18:38.675945+0000","last_active":"2024-09-21T00:20:54.743954+0000","last_peered":"2024-09-21T00:20:54.743954+0000","last_clean":"2024-09-21T00:20:54.743954+0000","last_became_active":"2024-09-21T00:18:38.675397+0000","last_became_peered":"2024-09-21T00:18:38.675397+0000","last_unstale":"2024-09-21T00:20:54.743954+0000","last_undegraded":"2024-09-21T00:20:54.743954+0000","last_fullsized":"2024-09-21T00:20:54.743954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:56.082314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.762260+0000","last_change":"2024-09-21T00:18:38.662854+0000","last_active":"2024-09-21T00:20:54.762260+0000","last_peered":"2024-09-21T00:20:54.762260+0000","last_clean":"2024-09-21T00:20:54.762260+0000","last_became_active":"2024-09-21T00:18:38.662094+0000","last_became_peered":"2024-09-21T00:18:38.662094+0000","last_unstale":"2024-09-21T00:20:54.762260+0000","last_undegraded":"2024-09-21T00:20:54.762260+0000","last_fullsized":"2024-09-21T00:20:54.762260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:45.756641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.744870+0000","last_change":"2024-09-21T00:18:38.663282+0000","last_active":"2024-09-21T00:20:54.744870+0000","last_peered":"2024-09-21T00:20:54.744870+0000","last_clean":"2024-09-21T00:20:54.744870+0000","last_became_active":"2024-09-21T00:18:38.662148+0000","last_became_peered":"2024-09-21T00:18:38.662148+0000","last_unstale":"2024-09-21T00:20:54.744870+0000","last_undegraded":"2024-09-21T00:20:54.744870+0000","last_fullsized":"2024-09-21T00:20:54.744870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:51.090591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.763683+0000","last_change":"2024-09-21T00:18:38.669184+0000","last_active":"2024-09-21T00:20:54.763683+0000","last_peered":"2024-09-21T00:20:54.763683+0000","last_clean":"2024-09-21T00:20:54.763683+0000","last_became_active":"2024-09-21T00:18:38.668799+0000","last_became_peered":"2024-09-21T00:18:38.668799+0000","last_unstale":"2024-09-21T00:20:54.763683+0000","last_undegraded":"2024-09-21T00:20:54.763683+0000","last_fullsized":"2024-09-21T00:20:54.763683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:49.642568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767674+0000","last_change":"2024-09-21T00:18:38.679501+0000","last_active":"2024-09-21T00:20:54.767674+0000","last_peered":"2024-09-21T00:20:54.767674+0000","last_clean":"2024-09-21T00:20:54.767674+0000","last_became_active":"2024-09-21T00:18:38.678582+0000","last_became_peered":"2024-09-21T00:18:38.678582+0000","last_unstale":"2024-09-21T00:20:54.767674+0000","last_undegraded":"2024-09-21T00:20:54.767674+0000","last_fullsized":"2024-09-21T00:20:54.767674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:30.277979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.751399+0000","last_change":"2024-09-21T00:18:38.659657+0000","last_active":"2024-09-21T00:20:54.751399+0000","last_peered":"2024-09-21T00:20:54.751399+0000","last_clean":"2024-09-21T00:20:54.751399+0000","last_became_active":"2024-09-21T00:18:38.659257+0000","last_became_peered":"2024-09-21T00:18:38.659257+0000","last_unstale":"2024-09-21T00:20:54.751399+0000","last_undegraded":"2024-09-21T00:20:54.751399+0000","last_fullsized":"2024-09-21T00:20:54.751399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:08.653729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.758299+0000","last_change":"2024-09-21T00:18:38.671817+0000","last_active":"2024-09-21T00:20:54.758299+0000","last_peered":"2024-09-21T00:20:54.758299+0000","last_clean":"2024-09-21T00:20:54.758299+0000","last_became_active":"2024-09-21T00:18:38.671469+0000","last_became_peered":"2024-09-21T00:18:38.671469+0000","last_unstale":"2024-09-21T00:20:54.758299+0000","last_undegraded":"2024-09-21T00:20:54.758299+0000","last_fullsized":"2024-09-21T00:20:54.758299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:58:15.036623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.761325+0000","last_change":"2024-09-21T00:18:38.664481+0000","last_active":"2024-09-21T00:20:54.761325+0000","last_peered":"2024-09-21T00:20:54.761325+0000","last_clean":"2024-09-21T00:20:54.761325+0000","last_became_active":"2024-09-21T00:18:38.664098+0000","last_became_peered":"2024-09-21T00:18:38.664098+0000","last_unstale":"2024-09-21T00:20:54.761325+0000","last_undegraded":"2024-09-21T00:20:54.761325+0000","last_fullsized":"2024-09-21T00:20:54.761325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:09.271251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768665+0000","last_change":"2024-09-21T00:18:38.662416+0000","last_active":"2024-09-21T00:20:54.768665+0000","last_peered":"2024-09-21T00:20:54.768665+0000","last_clean":"2024-09-21T00:20:54.768665+0000","last_became_active":"2024-09-21T00:18:38.662041+0000","last_became_peered":"2024-09-21T00:18:38.662041+0000","last_unstale":"2024-09-21T00:20:54.768665+0000","last_undegraded":"2024-09-21T00:20:54.768665+0000","last_fullsized":"2024-09-21T00:20:54.768665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:03.150179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768448+0000","last_change":"2024-09-21T00:18:38.670797+0000","last_active":"2024-09-21T00:20:54.768448+0000","last_peered":"2024-09-21T00:20:54.768448+0000","last_clean":"2024-09-21T00:20:54.768448+0000","last_became_active":"2024-09-21T00:18:38.670258+0000","last_became_peered":"2024-09-21T00:18:38.670258+0000","last_unstale":"2024-09-21T00:20:54.768448+0000","last_undegraded":"2024-09-21T00:20:54.768448+0000","last_fullsized":"2024-09-21T00:20:54.768448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:55.669836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.765633+0000","last_change":"2024-09-21T00:18:38.665594+0000","last_active":"2024-09-21T00:20:54.765633+0000","last_peered":"2024-09-21T00:20:54.765633+0000","last_clean":"2024-09-21T00:20:54.765633+0000","last_became_active":"2024-09-21T00:18:38.665260+0000","last_became_peered":"2024-09-21T00:18:38.665260+0000","last_unstale":"2024-09-21T00:20:54.765633+0000","last_undegraded":"2024-09-21T00:20:54.765633+0000","last_fullsized":"2024-09-21T00:20:54.765633+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:52:09.014327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768070+0000","last_change":"2024-09-21T00:18:38.679294+0000","last_active":"2024-09-21T00:20:54.768070+0000","last_peered":"2024-09-21T00:20:54.768070+0000","last_clean":"2024-09-21T00:20:54.768070+0000","last_became_active":"2024-09-21T00:18:38.673765+0000","last_became_peered":"2024-09-21T00:18:38.673765+0000","last_unstale":"2024-09-21T00:20:54.768070+0000","last_undegraded":"2024-09-21T00:20:54.768070+0000","last_fullsized":"2024-09-21T00:20:54.768070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:32.076299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767090+0000","last_change":"2024-09-21T00:18:38.667869+0000","last_active":"2024-09-21T00:20:54.767090+0000","last_peered":"2024-09-21T00:20:54.767090+0000","last_clean":"2024-09-21T00:20:54.767090+0000","last_became_active":"2024-09-21T00:18:38.667499+0000","last_became_peered":"2024-09-21T00:18:38.667499+0000","last_unstale":"2024-09-21T00:20:54.767090+0000","last_undegraded":"2024-09-21T00:20:54.767090+0000","last_fullsized":"2024-09-21T00:20:54.767090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:13.501043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.761252+0000","last_change":"2024-09-21T00:18:38.663107+0000","last_active":"2024-09-21T00:20:54.761252+0000","last_peered":"2024-09-21T00:20:54.761252+0000","last_clean":"2024-09-21T00:20:54.761252+0000","last_became_active":"2024-09-21T00:18:38.662680+0000","last_became_peered":"2024-09-21T00:18:38.662680+0000","last_unstale":"2024-09-21T00:20:54.761252+0000","last_undegraded":"2024-09-21T00:20:54.761252+0000","last_fullsized":"2024-09-21T00:20:54.761252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:48.154705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764194+0000","last_change":"2024-09-21T00:18:38.668167+0000","last_active":"2024-09-21T00:20:54.764194+0000","last_peered":"2024-09-21T00:20:54.764194+0000","last_clean":"2024-09-21T00:20:54.764194+0000","last_became_active":"2024-09-21T00:18:38.667842+0000","last_became_peered":"2024-09-21T00:18:38.667842+0000","last_unstale":"2024-09-21T00:20:54.764194+0000","last_undegraded":"2024-09-21T00:20:54.764194+0000","last_fullsized":"2024-09-21T00:20:54.764194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:55.358351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.761794+0000","last_change":"2024-09-21T00:18:38.666580+0000","last_active":"2024-09-21T00:20:54.761794+0000","last_peered":"2024-09-21T00:20:54.761794+0000","last_clean":"2024-09-21T00:20:54.761794+0000","last_became_active":"2024-09-21T00:18:38.666231+0000","last_became_peered":"2024-09-21T00:18:38.666231+0000","last_unstale":"2024-09-21T00:20:54.761794+0000","last_undegraded":"2024-09-21T00:20:54.761794+0000","last_fullsized":"2024-09-21T00:20:54.761794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:07.362516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.751359+0000","last_change":"2024-09-21T00:20:26.511248+0000","last_active":"2024-09-21T00:20:54.751359+0000","last_peered":"2024-09-21T00:20:54.751359+0000","last_clean":"2024-09-21T00:20:54.751359+0000","last_became_active":"2024-09-21T00:20:26.510743+0000","last_became_peered":"2024-09-21T00:20:26.510743+0000","last_unstale":"2024-09-21T00:20:54.751359+0000","last_undegraded":"2024-09-21T00:20:54.751359+0000","last_fullsized":"2024-09-21T00:20:54.751359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:41.577950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.760033+0000","last_change":"2024-09-21T00:18:38.666580+0000","last_active":"2024-09-21T00:20:54.760033+0000","last_peered":"2024-09-21T00:20:54.760033+0000","last_clean":"2024-09-21T00:20:54.760033+0000","last_became_active":"2024-09-21T00:18:38.665698+0000","last_became_peered":"2024-09-21T00:18:38.665698+0000","last_unstale":"2024-09-21T00:20:54.760033+0000","last_undegraded":"2024-09-21T00:20:54.760033+0000","last_fullsized":"2024-09-21T00:20:54.760033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:45.627667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.765480+0000","last_change":"2024-09-21T00:18:38.670729+0000","last_active":"2024-09-21T00:20:54.765480+0000","last_peered":"2024-09-21T00:20:54.765480+0000","last_clean":"2024-09-21T00:20:54.765480+0000","last_became_active":"2024-09-21T00:18:38.670337+0000","last_became_peered":"2024-09-21T00:18:38.670337+0000","last_unstale":"2024-09-21T00:20:54.765480+0000","last_undegraded":"2024-09-21T00:20:54.765480+0000","last_fullsized":"2024-09-21T00:20:54.765480+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:49.880689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.763380+0000","last_change":"2024-09-21T00:18:38.664644+0000","last_active":"2024-09-21T00:20:54.763380+0000","last_peered":"2024-09-21T00:20:54.763380+0000","last_clean":"2024-09-21T00:20:54.763380+0000","last_became_active":"2024-09-21T00:18:38.664220+0000","last_became_peered":"2024-09-21T00:18:38.664220+0000","last_unstale":"2024-09-21T00:20:54.763380+0000","last_undegraded":"2024-09-21T00:20:54.763380+0000","last_fullsized":"2024-09-21T00:20:54.763380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:09.610962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.759277+0000","last_change":"2024-09-21T00:18:38.671592+0000","last_active":"2024-09-21T00:20:54.759277+0000","last_peered":"2024-09-21T00:20:54.759277+0000","last_clean":"2024-09-21T00:20:54.759277+0000","last_became_active":"2024-09-21T00:18:38.671228+0000","last_became_peered":"2024-09-21T00:18:38.671228+0000","last_unstale":"2024-09-21T00:20:54.759277+0000","last_undegraded":"2024-09-21T00:20:54.759277+0000","last_fullsized":"2024-09-21T00:20:54.759277+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:51.906058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.762137+0000","last_change":"2024-09-21T00:19:25.843696+0000","last_active":"2024-09-21T00:20:54.762137+0000","last_peered":"2024-09-21T00:20:54.762137+0000","last_clean":"2024-09-21T00:20:54.762137+0000","last_became_active":"2024-09-21T00:19:25.843287+0000","last_became_peered":"2024-09-21T00:19:25.843287+0000","last_unstale":"2024-09-21T00:20:54.762137+0000","last_undegraded":"2024-09-21T00:20:54.762137+0000","last_fullsized":"2024-09-21T00:20:54.762137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:21.024555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767946+0000","last_change":"2024-09-21T00:18:38.671820+0000","last_active":"2024-09-21T00:20:54.767946+0000","last_peered":"2024-09-21T00:20:54.767946+0000","last_clean":"2024-09-21T00:20:54.767946+0000","last_became_active":"2024-09-21T00:18:38.667107+0000","last_became_peered":"2024-09-21T00:18:38.667107+0000","last_unstale":"2024-09-21T00:20:54.767946+0000","last_undegraded":"2024-09-21T00:20:54.767946+0000","last_fullsized":"2024-09-21T00:20:54.767946+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:58.099750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.759553+0000","last_change":"2024-09-21T00:18:38.671271+0000","last_active":"2024-09-21T00:20:54.759553+0000","last_peered":"2024-09-21T00:20:54.759553+0000","last_clean":"2024-09-21T00:20:54.759553+0000","last_became_active":"2024-09-21T00:18:38.670826+0000","last_became_peered":"2024-09-21T00:18:38.670826+0000","last_unstale":"2024-09-21T00:20:54.759553+0000","last_undegraded":"2024-09-21T00:20:54.759553+0000","last_fullsized":"2024-09-21T00:20:54.759553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:57:01.723120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768399+0000","last_change":"2024-09-21T00:18:38.662851+0000","last_active":"2024-09-21T00:20:54.768399+0000","last_peered":"2024-09-21T00:20:54.768399+0000","last_clean":"2024-09-21T00:20:54.768399+0000","last_became_active":"2024-09-21T00:18:38.661259+0000","last_became_peered":"2024-09-21T00:18:38.661259+0000","last_unstale":"2024-09-21T00:20:54.768399+0000","last_undegraded":"2024-09-21T00:20:54.768399+0000","last_fullsized":"2024-09-21T00:20:54.768399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:10.508980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":469532,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1067,"num_read_kb":731,"num_write":338,"num_write_kb":1350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3864,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2101248,"data_stored":1862009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678067,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":535425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874296,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37492,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707660,"statfs":{"total":95995035648,"available":95956643840,"internally_reserved":0,"allocated":2138112,"data_stored":1588109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037820,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712548,"statfs":{"total":95995035648,"available":95961649152,"internally_reserved":0,"allocated":1454080,"data_stored":932951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234050,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":535537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430279,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1131609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593805,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":530815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790036,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1127197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953561,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1130496,"data_stored":602266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149790,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28880,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716272,"statfs":{"total":95995035648,"available":95965462528,"internally_reserved":0,"allocated":1839104,"data_stored":1318001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378725,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":532025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542250,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":602387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705775,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":532203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":806912,"data_stored":787276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":512000,"data_stored":467016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:21:42.688 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T00:21:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:43.443 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:43 smithi027 ceph-mon[30745]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:21:43.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:43 smithi089 ceph-mon[34481]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:21:43.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:43 smithi098 ceph-mon[34725]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:21:43.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.778+0000 7f9c14a00640 1 -- 172.21.15.27:0/214999244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c10114580 msgr2=0x7f9c101169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:43.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.778+0000 7f9c14a00640 1 --2- 172.21.15.27:0/214999244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c10114580 0x7f9c101169a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009ca0 tx=0x7f9c0802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:21:43.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.778+0000 7f9c14a00640 1 -- 172.21.15.27:0/214999244 shutdown_connections 2024-09-21T00:21:43.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.778+0000 7f9c14a00640 1 --2- 172.21.15.27:0/214999244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c10114580 0x7f9c101169a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:43.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.778+0000 7f9c14a00640 1 --2- 172.21.15.27:0/214999244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c10078290 0x7f9c10114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:43.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.778+0000 7f9c14a00640 1 --2- 172.21.15.27:0/214999244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c10077940 0x7f9c10077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:43.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.778+0000 7f9c14a00640 1 -- 172.21.15.27:0/214999244 >> 172.21.15.27:0/214999244 conn(0x7f9c10101670 msgr2=0x7f9c10103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:43.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.778+0000 7f9c14a00640 1 -- 172.21.15.27:0/214999244 shutdown_connections 2024-09-21T00:21:43.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.778+0000 7f9c14a00640 1 -- 172.21.15.27:0/214999244 wait complete. 2024-09-21T00:21:43.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.779+0000 7f9c14a00640 1 Processor -- start 2024-09-21T00:21:43.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.779+0000 7f9c14a00640 1 -- start start 2024-09-21T00:21:43.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.779+0000 7f9c14a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c10077940 0x7f9c101a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:43.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.779+0000 7f9c14a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c10078290 0x7f9c101a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:43.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.779+0000 7f9c14a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c10114580 0x7f9c101a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:43.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.779+0000 7f9c14a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10119ee0 con 0x7f9c10114580 2024-09-21T00:21:43.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.779+0000 7f9c14a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10119d60 con 0x7f9c10077940 2024-09-21T00:21:43.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.779+0000 7f9c14a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1011a060 con 0x7f9c10078290 2024-09-21T00:21:43.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.780+0000 7f9c0fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c10114580 0x7f9c101a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:43.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.780+0000 7f9c0f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c10077940 0x7f9c101a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:43.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.780+0000 7f9c0fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c10114580 0x7f9c101a60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60700/0 (socket says 172.21.15.27:60700) 2024-09-21T00:21:43.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.780+0000 7f9c0f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c10077940 0x7f9c101a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:52566/0 (socket says 172.21.15.27:52566) 2024-09-21T00:21:43.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.780+0000 7f9c0fe00640 1 -- 172.21.15.27:0/2745994308 learned_addr learned my addr 172.21.15.27:0/2745994308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:43.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.780+0000 7f9c0ea00640 1 --2- 172.21.15.27:0/2745994308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c10078290 0x7f9c101a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:43.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.780+0000 7f9c0f400640 1 -- 172.21.15.27:0/2745994308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c10078290 msgr2=0x7f9c101a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:43.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.780+0000 7f9c0f400640 1 --2- 172.21.15.27:0/2745994308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c10078290 0x7f9c101a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:43.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.780+0000 7f9c0f400640 1 -- 172.21.15.27:0/2745994308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c10114580 msgr2=0x7f9c101a60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:43.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.780+0000 7f9c0f400640 1 --2- 172.21.15.27:0/2745994308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c10114580 0x7f9c101a60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:43.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.780+0000 7f9c0f400640 1 -- 172.21.15.27:0/2745994308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c101a67e0 con 0x7f9c10077940 2024-09-21T00:21:43.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.781+0000 7f9c0f400640 1 --2- 172.21.15.27:0/2745994308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c10077940 0x7f9c101a0870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000e9f0 tx=0x7f9c0000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:43.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.781+0000 7f9c07e00640 1 -- 172.21.15.27:0/2745994308 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000ccc0 con 0x7f9c10077940 2024-09-21T00:21:43.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.781+0000 7f9c14a00640 1 -- 172.21.15.27:0/2745994308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c101a6ad0 con 0x7f9c10077940 2024-09-21T00:21:43.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.781+0000 7f9c07e00640 1 -- 172.21.15.27:0/2745994308 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c000108e0 con 0x7f9c10077940 2024-09-21T00:21:43.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.781+0000 7f9c07e00640 1 -- 172.21.15.27:0/2745994308 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00018880 con 0x7f9c10077940 2024-09-21T00:21:43.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.781+0000 7f9c14a00640 1 -- 172.21.15.27:0/2745994308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c101ad230 con 0x7f9c10077940 2024-09-21T00:21:43.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.782+0000 7f9c14a00640 1 -- 172.21.15.27:0/2745994308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c10077d50 con 0x7f9c10077940 2024-09-21T00:21:43.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.785+0000 7f9c07e00640 1 -- 172.21.15.27:0/2745994308 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f9c00010430 con 0x7f9c10077940 2024-09-21T00:21:43.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.786+0000 7f9c07e00640 1 --2- 172.21.15.27:0/2745994308 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9bf40b65f0 0x7f9bf40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:43.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.786+0000 7f9c07e00640 1 -- 172.21.15.27:0/2745994308 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f9c000e8220 con 0x7f9c10077940 2024-09-21T00:21:43.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.786+0000 7f9c0ea00640 1 --2- 172.21.15.27:0/2745994308 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9bf40b65f0 0x7f9bf40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.787+0000 7f9c0ea00640 1 --2- 172.21.15.27:0/2745994308 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9bf40b65f0 0x7f9bf40b8b00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0095f0 tx=0x7f9bfc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.788+0000 7f9c07e00640 1 -- 172.21.15.27:0/2745994308 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9c000b2820 con 0x7f9c10077940 2024-09-21T00:21:43.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.983+0000 7f9c14a00640 1 -- 172.21.15.27:0/2745994308 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c10063800 con 0x7f9bf40b65f0 2024-09-21T00:21:43.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.989+0000 7f9c07e00640 1 -- 172.21.15.27:0/2745994308 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340546 (secure 0 0 0) 0x7f9c10063800 con 0x7f9bf40b65f0 2024-09-21T00:21:43.995 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:21:43.998 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T00:21:44.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.995+0000 7f9c14a00640 1 -- 172.21.15.27:0/2745994308 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9bf40b65f0 msgr2=0x7f9bf40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:44.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.995+0000 7f9c14a00640 1 --2- 172.21.15.27:0/2745994308 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9bf40b65f0 0x7f9bf40b8b00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0095f0 tx=0x7f9bfc005f90 comp rx=0 tx=0).stop 2024-09-21T00:21:44.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.995+0000 7f9c14a00640 1 -- 172.21.15.27:0/2745994308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c10077940 msgr2=0x7f9c101a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:44.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.995+0000 7f9c14a00640 1 --2- 172.21.15.27:0/2745994308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c10077940 0x7f9c101a0870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000e9f0 tx=0x7f9c0000eeb0 comp rx=0 tx=0).stop 2024-09-21T00:21:44.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.995+0000 7f9c14a00640 1 -- 172.21.15.27:0/2745994308 shutdown_connections 2024-09-21T00:21:44.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.995+0000 7f9c14a00640 1 --2- 172.21.15.27:0/2745994308 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f9bf40b65f0 0x7f9bf40b8b00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:44.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.995+0000 7f9c14a00640 1 --2- 172.21.15.27:0/2745994308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c10114580 0x7f9c101a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:44.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.995+0000 7f9c14a00640 1 --2- 172.21.15.27:0/2745994308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c10078290 0x7f9c101a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:44.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.995+0000 7f9c14a00640 1 --2- 172.21.15.27:0/2745994308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c10077940 0x7f9c101a0870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:44.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.995+0000 7f9c14a00640 1 -- 172.21.15.27:0/2745994308 >> 172.21.15.27:0/2745994308 conn(0x7f9c10101670 msgr2=0x7f9c101165f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:44.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.995+0000 7f9c14a00640 1 -- 172.21.15.27:0/2745994308 shutdown_connections 2024-09-21T00:21:44.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:43.995+0000 7f9c14a00640 1 -- 172.21.15.27:0/2745994308 wait complete. 2024-09-21T00:21:44.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:44 smithi027 ceph-mon[30745]: pgmap v387: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 51 op/s 2024-09-21T00:21:44.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:44 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:21:44.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:44 smithi089 ceph-mon[34481]: pgmap v387: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 51 op/s 2024-09-21T00:21:44.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:44 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:21:44.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:44 smithi098 ceph-mon[34725]: pgmap v387: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 51 op/s 2024-09-21T00:21:44.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:44 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:21:44.924 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-21T00:21:42.379878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1453398,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1184,"num_read_kb":827,"num_write":520,"num_write_kb":4372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2955704,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":885,"ondisk_log_size":885,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382696,"kb_used_data":16628,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124559128,"statfs":{"total":1151940427776,"available":1151548547072,"internally_reserved":0,"allocated":17027072,"data_stored":10760960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":792746,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":395,"num_read_kb":327,"num_write":225,"num_write_kb":2097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003513"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453054+0000","last_change":"2024-09-21T00:18:35.627618+0000","last_active":"2024-09-21T00:20:26.453054+0000","last_peered":"2024-09-21T00:20:26.453054+0000","last_clean":"2024-09-21T00:20:26.453054+0000","last_became_active":"2024-09-21T00:18:35.626475+0000","last_became_peered":"2024-09-21T00:18:35.626475+0000","last_unstale":"2024-09-21T00:20:26.453054+0000","last_undegraded":"2024-09-21T00:20:26.453054+0000","last_fullsized":"2024-09-21T00:20:26.453054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:50.379225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754687+0000","last_change":"2024-09-21T00:18:38.665891+0000","last_active":"2024-09-21T00:20:54.754687+0000","last_peered":"2024-09-21T00:20:54.754687+0000","last_clean":"2024-09-21T00:20:54.754687+0000","last_became_active":"2024-09-21T00:18:38.665472+0000","last_became_peered":"2024-09-21T00:18:38.665472+0000","last_unstale":"2024-09-21T00:20:54.754687+0000","last_undegraded":"2024-09-21T00:20:54.754687+0000","last_fullsized":"2024-09-21T00:20:54.754687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:34:32.490828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451446+0000","last_change":"2024-09-21T00:18:48.730206+0000","last_active":"2024-09-21T00:20:26.451446+0000","last_peered":"2024-09-21T00:20:26.451446+0000","last_clean":"2024-09-21T00:20:26.451446+0000","last_became_active":"2024-09-21T00:18:48.729729+0000","last_became_peered":"2024-09-21T00:18:48.729729+0000","last_unstale":"2024-09-21T00:20:26.451446+0000","last_undegraded":"2024-09-21T00:20:26.451446+0000","last_fullsized":"2024-09-21T00:20:26.451446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:08:17.019877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.575433+0000","last_change":"2024-09-21T00:18:35.768554+0000","last_active":"2024-09-21T00:20:26.575433+0000","last_peered":"2024-09-21T00:20:26.575433+0000","last_clean":"2024-09-21T00:20:26.575433+0000","last_became_active":"2024-09-21T00:18:35.767642+0000","last_became_peered":"2024-09-21T00:18:35.767642+0000","last_unstale":"2024-09-21T00:20:26.575433+0000","last_undegraded":"2024-09-21T00:20:26.575433+0000","last_fullsized":"2024-09-21T00:20:26.575433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:10.442974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768099+0000","last_change":"2024-09-21T00:18:38.677997+0000","last_active":"2024-09-21T00:20:54.768099+0000","last_peered":"2024-09-21T00:20:54.768099+0000","last_clean":"2024-09-21T00:20:54.768099+0000","last_became_active":"2024-09-21T00:18:38.676698+0000","last_became_peered":"2024-09-21T00:18:38.676698+0000","last_unstale":"2024-09-21T00:20:54.768099+0000","last_undegraded":"2024-09-21T00:20:54.768099+0000","last_fullsized":"2024-09-21T00:20:54.768099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:11.662765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451676+0000","last_change":"2024-09-21T00:18:48.739563+0000","last_active":"2024-09-21T00:20:26.451676+0000","last_peered":"2024-09-21T00:20:26.451676+0000","last_clean":"2024-09-21T00:20:26.451676+0000","last_became_active":"2024-09-21T00:18:48.739092+0000","last_became_peered":"2024-09-21T00:18:48.739092+0000","last_unstale":"2024-09-21T00:20:26.451676+0000","last_undegraded":"2024-09-21T00:20:26.451676+0000","last_fullsized":"2024-09-21T00:20:26.451676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:44.185556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510989+0000","last_change":"2024-09-21T00:18:35.626742+0000","last_active":"2024-09-21T00:20:26.510989+0000","last_peered":"2024-09-21T00:20:26.510989+0000","last_clean":"2024-09-21T00:20:26.510989+0000","last_became_active":"2024-09-21T00:18:35.626359+0000","last_became_peered":"2024-09-21T00:18:35.626359+0000","last_unstale":"2024-09-21T00:20:26.510989+0000","last_undegraded":"2024-09-21T00:20:26.510989+0000","last_fullsized":"2024-09-21T00:20:26.510989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:38.860184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.761254+0000","last_change":"2024-09-21T00:18:38.666003+0000","last_active":"2024-09-21T00:20:54.761254+0000","last_peered":"2024-09-21T00:20:54.761254+0000","last_clean":"2024-09-21T00:20:54.761254+0000","last_became_active":"2024-09-21T00:18:38.665425+0000","last_became_peered":"2024-09-21T00:18:38.665425+0000","last_unstale":"2024-09-21T00:20:54.761254+0000","last_undegraded":"2024-09-21T00:20:54.761254+0000","last_fullsized":"2024-09-21T00:20:54.761254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:14.121864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.577303+0000","last_change":"2024-09-21T00:20:26.576748+0000","last_active":"2024-09-21T00:20:26.577303+0000","last_peered":"2024-09-21T00:20:26.577303+0000","last_clean":"2024-09-21T00:20:26.577303+0000","last_became_active":"2024-09-21T00:20:26.576186+0000","last_became_peered":"2024-09-21T00:20:26.576186+0000","last_unstale":"2024-09-21T00:20:26.577303+0000","last_undegraded":"2024-09-21T00:20:26.577303+0000","last_fullsized":"2024-09-21T00:20:26.577303+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:20.283717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510680+0000","last_change":"2024-09-21T00:18:35.628091+0000","last_active":"2024-09-21T00:20:26.510680+0000","last_peered":"2024-09-21T00:20:26.510680+0000","last_clean":"2024-09-21T00:20:26.510680+0000","last_became_active":"2024-09-21T00:18:35.627654+0000","last_became_peered":"2024-09-21T00:18:35.627654+0000","last_unstale":"2024-09-21T00:20:26.510680+0000","last_undegraded":"2024-09-21T00:20:26.510680+0000","last_fullsized":"2024-09-21T00:20:26.510680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:05.229036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.765336+0000","last_change":"2024-09-21T00:19:25.844116+0000","last_active":"2024-09-21T00:20:54.765336+0000","last_peered":"2024-09-21T00:20:54.765336+0000","last_clean":"2024-09-21T00:20:54.765336+0000","last_became_active":"2024-09-21T00:19:25.842745+0000","last_became_peered":"2024-09-21T00:19:25.842745+0000","last_unstale":"2024-09-21T00:20:54.765336+0000","last_undegraded":"2024-09-21T00:20:54.765336+0000","last_fullsized":"2024-09-21T00:20:54.765336+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:42.651251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451830+0000","last_change":"2024-09-21T00:18:48.740106+0000","last_active":"2024-09-21T00:20:26.451830+0000","last_peered":"2024-09-21T00:20:26.451830+0000","last_clean":"2024-09-21T00:20:26.451830+0000","last_became_active":"2024-09-21T00:18:48.739299+0000","last_became_peered":"2024-09-21T00:18:48.739299+0000","last_unstale":"2024-09-21T00:20:26.451830+0000","last_undegraded":"2024-09-21T00:20:26.451830+0000","last_fullsized":"2024-09-21T00:20:26.451830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:59.058333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.766986+0000","last_change":"2024-09-21T00:18:38.667331+0000","last_active":"2024-09-21T00:20:54.766986+0000","last_peered":"2024-09-21T00:20:54.766986+0000","last_clean":"2024-09-21T00:20:54.766986+0000","last_became_active":"2024-09-21T00:18:38.666960+0000","last_became_peered":"2024-09-21T00:18:38.666960+0000","last_unstale":"2024-09-21T00:20:54.766986+0000","last_undegraded":"2024-09-21T00:20:54.766986+0000","last_fullsized":"2024-09-21T00:20:54.766986+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:57.569460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510354+0000","last_change":"2024-09-21T00:18:35.625570+0000","last_active":"2024-09-21T00:20:26.510354+0000","last_peered":"2024-09-21T00:20:26.510354+0000","last_clean":"2024-09-21T00:20:26.510354+0000","last_became_active":"2024-09-21T00:18:35.625170+0000","last_became_peered":"2024-09-21T00:18:35.625170+0000","last_unstale":"2024-09-21T00:20:26.510354+0000","last_undegraded":"2024-09-21T00:20:26.510354+0000","last_fullsized":"2024-09-21T00:20:26.510354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:58.614258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452085+0000","last_change":"2024-09-21T00:18:48.727390+0000","last_active":"2024-09-21T00:20:26.452085+0000","last_peered":"2024-09-21T00:20:26.452085+0000","last_clean":"2024-09-21T00:20:26.452085+0000","last_became_active":"2024-09-21T00:18:48.726864+0000","last_became_peered":"2024-09-21T00:18:48.726864+0000","last_unstale":"2024-09-21T00:20:26.452085+0000","last_undegraded":"2024-09-21T00:20:26.452085+0000","last_fullsized":"2024-09-21T00:20:26.452085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:10.917069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767288+0000","last_change":"2024-09-21T00:18:38.664072+0000","last_active":"2024-09-21T00:20:54.767288+0000","last_peered":"2024-09-21T00:20:54.767288+0000","last_clean":"2024-09-21T00:20:54.767288+0000","last_became_active":"2024-09-21T00:18:38.662562+0000","last_became_peered":"2024-09-21T00:18:38.662562+0000","last_unstale":"2024-09-21T00:20:54.767288+0000","last_undegraded":"2024-09-21T00:20:54.767288+0000","last_fullsized":"2024-09-21T00:20:54.767288+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.021174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453878+0000","last_change":"2024-09-21T00:19:25.843599+0000","last_active":"2024-09-21T00:20:26.453878+0000","last_peered":"2024-09-21T00:20:26.453878+0000","last_clean":"2024-09-21T00:20:26.453878+0000","last_became_active":"2024-09-21T00:19:25.843178+0000","last_became_peered":"2024-09-21T00:19:25.843178+0000","last_unstale":"2024-09-21T00:20:26.453878+0000","last_undegraded":"2024-09-21T00:20:26.453878+0000","last_fullsized":"2024-09-21T00:20:26.453878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:18.809857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452141+0000","last_change":"2024-09-21T00:18:48.729940+0000","last_active":"2024-09-21T00:20:26.452141+0000","last_peered":"2024-09-21T00:20:26.452141+0000","last_clean":"2024-09-21T00:20:26.452141+0000","last_became_active":"2024-09-21T00:18:48.729465+0000","last_became_peered":"2024-09-21T00:18:48.729465+0000","last_unstale":"2024-09-21T00:20:26.452141+0000","last_undegraded":"2024-09-21T00:20:26.452141+0000","last_fullsized":"2024-09-21T00:20:26.452141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:44.566386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768271+0000","last_change":"2024-09-21T00:18:38.662510+0000","last_active":"2024-09-21T00:20:54.768271+0000","last_peered":"2024-09-21T00:20:54.768271+0000","last_clean":"2024-09-21T00:20:54.768271+0000","last_became_active":"2024-09-21T00:18:38.659788+0000","last_became_peered":"2024-09-21T00:18:38.659788+0000","last_unstale":"2024-09-21T00:20:54.768271+0000","last_undegraded":"2024-09-21T00:20:54.768271+0000","last_fullsized":"2024-09-21T00:20:54.768271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:14.532084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.575037+0000","last_change":"2024-09-21T00:18:35.622949+0000","last_active":"2024-09-21T00:20:26.575037+0000","last_peered":"2024-09-21T00:20:26.575037+0000","last_clean":"2024-09-21T00:20:26.575037+0000","last_became_active":"2024-09-21T00:18:35.621787+0000","last_became_peered":"2024-09-21T00:18:35.621787+0000","last_unstale":"2024-09-21T00:20:26.575037+0000","last_undegraded":"2024-09-21T00:20:26.575037+0000","last_fullsized":"2024-09-21T00:20:26.575037+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:39:15.060123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452739+0000","last_change":"2024-09-21T00:18:48.730711+0000","last_active":"2024-09-21T00:20:26.452739+0000","last_peered":"2024-09-21T00:20:26.452739+0000","last_clean":"2024-09-21T00:20:26.452739+0000","last_became_active":"2024-09-21T00:18:48.730287+0000","last_became_peered":"2024-09-21T00:18:48.730287+0000","last_unstale":"2024-09-21T00:20:26.452739+0000","last_undegraded":"2024-09-21T00:20:26.452739+0000","last_fullsized":"2024-09-21T00:20:26.452739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:34.444747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.734412+0000","last_change":"2024-09-21T00:18:38.669381+0000","last_active":"2024-09-21T00:20:54.734412+0000","last_peered":"2024-09-21T00:20:54.734412+0000","last_clean":"2024-09-21T00:20:54.734412+0000","last_became_active":"2024-09-21T00:18:38.668757+0000","last_became_peered":"2024-09-21T00:18:38.668757+0000","last_unstale":"2024-09-21T00:20:54.734412+0000","last_undegraded":"2024-09-21T00:20:54.734412+0000","last_fullsized":"2024-09-21T00:20:54.734412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:52.475872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508594+0000","last_change":"2024-09-21T00:18:35.767700+0000","last_active":"2024-09-21T00:20:26.508594+0000","last_peered":"2024-09-21T00:20:26.508594+0000","last_clean":"2024-09-21T00:20:26.508594+0000","last_became_active":"2024-09-21T00:18:35.767369+0000","last_became_peered":"2024-09-21T00:18:35.767369+0000","last_unstale":"2024-09-21T00:20:26.508594+0000","last_undegraded":"2024-09-21T00:20:26.508594+0000","last_fullsized":"2024-09-21T00:20:26.508594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:44.955659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452286+0000","last_change":"2024-09-21T00:18:48.737411+0000","last_active":"2024-09-21T00:20:26.452286+0000","last_peered":"2024-09-21T00:20:26.452286+0000","last_clean":"2024-09-21T00:20:26.452286+0000","last_became_active":"2024-09-21T00:18:48.736715+0000","last_became_peered":"2024-09-21T00:18:48.736715+0000","last_unstale":"2024-09-21T00:20:26.452286+0000","last_undegraded":"2024-09-21T00:20:26.452286+0000","last_fullsized":"2024-09-21T00:20:26.452286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:01.045318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768197+0000","last_change":"2024-09-21T00:19:25.842355+0000","last_active":"2024-09-21T00:20:54.768197+0000","last_peered":"2024-09-21T00:20:54.768197+0000","last_clean":"2024-09-21T00:20:54.768197+0000","last_became_active":"2024-09-21T00:19:25.841961+0000","last_became_peered":"2024-09-21T00:19:25.841961+0000","last_unstale":"2024-09-21T00:20:54.768197+0000","last_undegraded":"2024-09-21T00:20:54.768197+0000","last_fullsized":"2024-09-21T00:20:54.768197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:30.490992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452475+0000","last_change":"2024-09-21T00:18:35.919003+0000","last_active":"2024-09-21T00:20:26.452475+0000","last_peered":"2024-09-21T00:20:26.452475+0000","last_clean":"2024-09-21T00:20:26.452475+0000","last_became_active":"2024-09-21T00:18:35.918593+0000","last_became_peered":"2024-09-21T00:18:35.918593+0000","last_unstale":"2024-09-21T00:20:26.452475+0000","last_undegraded":"2024-09-21T00:20:26.452475+0000","last_fullsized":"2024-09-21T00:20:26.452475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:35.982726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451542+0000","last_change":"2024-09-21T00:18:48.725906+0000","last_active":"2024-09-21T00:20:26.451542+0000","last_peered":"2024-09-21T00:20:26.451542+0000","last_clean":"2024-09-21T00:20:26.451542+0000","last_became_active":"2024-09-21T00:18:48.725450+0000","last_became_peered":"2024-09-21T00:18:48.725450+0000","last_unstale":"2024-09-21T00:20:26.451542+0000","last_undegraded":"2024-09-21T00:20:26.451542+0000","last_fullsized":"2024-09-21T00:20:26.451542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:35.163208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764430+0000","last_change":"2024-09-21T00:18:38.666442+0000","last_active":"2024-09-21T00:20:54.764430+0000","last_peered":"2024-09-21T00:20:54.764430+0000","last_clean":"2024-09-21T00:20:54.764430+0000","last_became_active":"2024-09-21T00:18:38.666083+0000","last_became_peered":"2024-09-21T00:18:38.666083+0000","last_unstale":"2024-09-21T00:20:54.764430+0000","last_undegraded":"2024-09-21T00:20:54.764430+0000","last_fullsized":"2024-09-21T00:20:54.764430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:39.986440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'94","reported_seq":225,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:37.878132+0000","last_change":"2024-09-21T00:18:35.624883+0000","last_active":"2024-09-21T00:21:37.878132+0000","last_peered":"2024-09-21T00:21:37.878132+0000","last_clean":"2024-09-21T00:21:37.878132+0000","last_became_active":"2024-09-21T00:18:35.624382+0000","last_became_peered":"2024-09-21T00:18:35.624382+0000","last_unstale":"2024-09-21T00:21:37.878132+0000","last_undegraded":"2024-09-21T00:21:37.878132+0000","last_fullsized":"2024-09-21T00:21:37.878132+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:23.475521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":259,"num_write":172,"num_write_kb":1220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574454+0000","last_change":"2024-09-21T00:18:48.719850+0000","last_active":"2024-09-21T00:20:26.574454+0000","last_peered":"2024-09-21T00:20:26.574454+0000","last_clean":"2024-09-21T00:20:26.574454+0000","last_became_active":"2024-09-21T00:18:48.719414+0000","last_became_peered":"2024-09-21T00:18:48.719414+0000","last_unstale":"2024-09-21T00:20:26.574454+0000","last_undegraded":"2024-09-21T00:20:26.574454+0000","last_fullsized":"2024-09-21T00:20:26.574454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:05.978977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.755833+0000","last_change":"2024-09-21T00:18:38.671920+0000","last_active":"2024-09-21T00:20:54.755833+0000","last_peered":"2024-09-21T00:20:54.755833+0000","last_clean":"2024-09-21T00:20:54.755833+0000","last_became_active":"2024-09-21T00:18:38.671592+0000","last_became_peered":"2024-09-21T00:18:38.671592+0000","last_unstale":"2024-09-21T00:20:54.755833+0000","last_undegraded":"2024-09-21T00:20:54.755833+0000","last_fullsized":"2024-09-21T00:20:54.755833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:40:48.257086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.511127+0000","last_change":"2024-09-21T00:18:35.769197+0000","last_active":"2024-09-21T00:20:26.511127+0000","last_peered":"2024-09-21T00:20:26.511127+0000","last_clean":"2024-09-21T00:20:26.511127+0000","last_became_active":"2024-09-21T00:18:35.768740+0000","last_became_peered":"2024-09-21T00:18:35.768740+0000","last_unstale":"2024-09-21T00:20:26.511127+0000","last_undegraded":"2024-09-21T00:20:26.511127+0000","last_fullsized":"2024-09-21T00:20:26.511127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:32.253006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454104+0000","last_change":"2024-09-21T00:18:48.740189+0000","last_active":"2024-09-21T00:20:26.454104+0000","last_peered":"2024-09-21T00:20:26.454104+0000","last_clean":"2024-09-21T00:20:26.454104+0000","last_became_active":"2024-09-21T00:18:48.739700+0000","last_became_peered":"2024-09-21T00:18:48.739700+0000","last_unstale":"2024-09-21T00:20:26.454104+0000","last_undegraded":"2024-09-21T00:20:26.454104+0000","last_fullsized":"2024-09-21T00:20:26.454104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:06.312160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764470+0000","last_change":"2024-09-21T00:18:38.665635+0000","last_active":"2024-09-21T00:20:54.764470+0000","last_peered":"2024-09-21T00:20:54.764470+0000","last_clean":"2024-09-21T00:20:54.764470+0000","last_became_active":"2024-09-21T00:18:38.665236+0000","last_became_peered":"2024-09-21T00:18:38.665236+0000","last_unstale":"2024-09-21T00:20:54.764470+0000","last_undegraded":"2024-09-21T00:20:54.764470+0000","last_fullsized":"2024-09-21T00:20:54.764470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:52.186480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452058+0000","last_change":"2024-09-21T00:18:35.771465+0000","last_active":"2024-09-21T00:20:26.452058+0000","last_peered":"2024-09-21T00:20:26.452058+0000","last_clean":"2024-09-21T00:20:26.452058+0000","last_became_active":"2024-09-21T00:18:35.770550+0000","last_became_peered":"2024-09-21T00:18:35.770550+0000","last_unstale":"2024-09-21T00:20:26.452058+0000","last_undegraded":"2024-09-21T00:20:26.452058+0000","last_fullsized":"2024-09-21T00:20:26.452058+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:43.042563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510077+0000","last_change":"2024-09-21T00:18:48.736334+0000","last_active":"2024-09-21T00:20:26.510077+0000","last_peered":"2024-09-21T00:20:26.510077+0000","last_clean":"2024-09-21T00:20:26.510077+0000","last_became_active":"2024-09-21T00:18:48.735784+0000","last_became_peered":"2024-09-21T00:18:48.735784+0000","last_unstale":"2024-09-21T00:20:26.510077+0000","last_undegraded":"2024-09-21T00:20:26.510077+0000","last_fullsized":"2024-09-21T00:20:26.510077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:29.637335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767464+0000","last_change":"2024-09-21T00:18:38.674181+0000","last_active":"2024-09-21T00:20:54.767464+0000","last_peered":"2024-09-21T00:20:54.767464+0000","last_clean":"2024-09-21T00:20:54.767464+0000","last_became_active":"2024-09-21T00:18:38.673784+0000","last_became_peered":"2024-09-21T00:18:38.673784+0000","last_unstale":"2024-09-21T00:20:54.767464+0000","last_undegraded":"2024-09-21T00:20:54.767464+0000","last_fullsized":"2024-09-21T00:20:54.767464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:33.471328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510165+0000","last_change":"2024-09-21T00:18:35.628149+0000","last_active":"2024-09-21T00:20:26.510165+0000","last_peered":"2024-09-21T00:20:26.510165+0000","last_clean":"2024-09-21T00:20:26.510165+0000","last_became_active":"2024-09-21T00:18:35.627674+0000","last_became_peered":"2024-09-21T00:18:35.627674+0000","last_unstale":"2024-09-21T00:20:26.510165+0000","last_undegraded":"2024-09-21T00:20:26.510165+0000","last_fullsized":"2024-09-21T00:20:26.510165+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:28:47.244288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508884+0000","last_change":"2024-09-21T00:18:48.740283+0000","last_active":"2024-09-21T00:20:26.508884+0000","last_peered":"2024-09-21T00:20:26.508884+0000","last_clean":"2024-09-21T00:20:26.508884+0000","last_became_active":"2024-09-21T00:18:48.739026+0000","last_became_peered":"2024-09-21T00:18:48.739026+0000","last_unstale":"2024-09-21T00:20:26.508884+0000","last_undegraded":"2024-09-21T00:20:26.508884+0000","last_fullsized":"2024-09-21T00:20:26.508884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:50.898116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.759757+0000","last_change":"2024-09-21T00:18:38.667206+0000","last_active":"2024-09-21T00:20:54.759757+0000","last_peered":"2024-09-21T00:20:54.759757+0000","last_clean":"2024-09-21T00:20:54.759757+0000","last_became_active":"2024-09-21T00:18:38.666770+0000","last_became_peered":"2024-09-21T00:18:38.666770+0000","last_unstale":"2024-09-21T00:20:54.759757+0000","last_undegraded":"2024-09-21T00:20:54.759757+0000","last_fullsized":"2024-09-21T00:20:54.759757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:56.332071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":171,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:36.110950+0000","last_change":"2024-09-21T00:18:35.768012+0000","last_active":"2024-09-21T00:21:36.110950+0000","last_peered":"2024-09-21T00:21:36.110950+0000","last_clean":"2024-09-21T00:21:36.110950+0000","last_became_active":"2024-09-21T00:18:35.767438+0000","last_became_peered":"2024-09-21T00:18:35.767438+0000","last_unstale":"2024-09-21T00:21:36.110950+0000","last_undegraded":"2024-09-21T00:21:36.110950+0000","last_fullsized":"2024-09-21T00:21:36.110950+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:35:40.606871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508643+0000","last_change":"2024-09-21T00:18:48.738228+0000","last_active":"2024-09-21T00:20:26.508643+0000","last_peered":"2024-09-21T00:20:26.508643+0000","last_clean":"2024-09-21T00:20:26.508643+0000","last_became_active":"2024-09-21T00:18:48.737786+0000","last_became_peered":"2024-09-21T00:18:48.737786+0000","last_unstale":"2024-09-21T00:20:26.508643+0000","last_undegraded":"2024-09-21T00:20:26.508643+0000","last_fullsized":"2024-09-21T00:20:26.508643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:52.329677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754320+0000","last_change":"2024-09-21T00:18:38.678126+0000","last_active":"2024-09-21T00:20:54.754320+0000","last_peered":"2024-09-21T00:20:54.754320+0000","last_clean":"2024-09-21T00:20:54.754320+0000","last_became_active":"2024-09-21T00:18:38.677757+0000","last_became_peered":"2024-09-21T00:18:38.677757+0000","last_unstale":"2024-09-21T00:20:54.754320+0000","last_undegraded":"2024-09-21T00:20:54.754320+0000","last_fullsized":"2024-09-21T00:20:54.754320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:26.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508836+0000","last_change":"2024-09-21T00:18:35.625955+0000","last_active":"2024-09-21T00:20:26.508836+0000","last_peered":"2024-09-21T00:20:26.508836+0000","last_clean":"2024-09-21T00:20:26.508836+0000","last_became_active":"2024-09-21T00:18:35.625548+0000","last_became_peered":"2024-09-21T00:18:35.625548+0000","last_unstale":"2024-09-21T00:20:26.508836+0000","last_undegraded":"2024-09-21T00:20:26.508836+0000","last_fullsized":"2024-09-21T00:20:26.508836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:04.163421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509821+0000","last_change":"2024-09-21T00:18:48.740765+0000","last_active":"2024-09-21T00:20:26.509821+0000","last_peered":"2024-09-21T00:20:26.509821+0000","last_clean":"2024-09-21T00:20:26.509821+0000","last_became_active":"2024-09-21T00:18:48.740081+0000","last_became_peered":"2024-09-21T00:18:48.740081+0000","last_unstale":"2024-09-21T00:20:26.509821+0000","last_undegraded":"2024-09-21T00:20:26.509821+0000","last_fullsized":"2024-09-21T00:20:26.509821+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:38.251475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.755958+0000","last_change":"2024-09-21T00:18:38.667581+0000","last_active":"2024-09-21T00:20:54.755958+0000","last_peered":"2024-09-21T00:20:54.755958+0000","last_clean":"2024-09-21T00:20:54.755958+0000","last_became_active":"2024-09-21T00:18:38.665959+0000","last_became_peered":"2024-09-21T00:18:38.665959+0000","last_unstale":"2024-09-21T00:20:54.755958+0000","last_undegraded":"2024-09-21T00:20:54.755958+0000","last_fullsized":"2024-09-21T00:20:54.755958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:52.321654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452596+0000","last_change":"2024-09-21T00:18:35.624291+0000","last_active":"2024-09-21T00:20:26.452596+0000","last_peered":"2024-09-21T00:20:26.452596+0000","last_clean":"2024-09-21T00:20:26.452596+0000","last_became_active":"2024-09-21T00:18:35.623297+0000","last_became_peered":"2024-09-21T00:18:35.623297+0000","last_unstale":"2024-09-21T00:20:26.452596+0000","last_undegraded":"2024-09-21T00:20:26.452596+0000","last_fullsized":"2024-09-21T00:20:26.452596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:44.824409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509191+0000","last_change":"2024-09-21T00:17:21.233627+0000","last_active":"2024-09-21T00:20:26.509191+0000","last_peered":"2024-09-21T00:20:26.509191+0000","last_clean":"2024-09-21T00:20:26.509191+0000","last_became_active":"2024-09-21T00:17:21.227897+0000","last_became_peered":"2024-09-21T00:17:21.227897+0000","last_unstale":"2024-09-21T00:20:26.509191+0000","last_undegraded":"2024-09-21T00:20:26.509191+0000","last_fullsized":"2024-09-21T00:20:26.509191+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:13:33.120691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:13:33.120691+0000","last_clean_scrub_stamp":"2024-09-21T00:13:33.120691+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:20.198660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509421+0000","last_change":"2024-09-21T00:18:48.730107+0000","last_active":"2024-09-21T00:20:26.509421+0000","last_peered":"2024-09-21T00:20:26.509421+0000","last_clean":"2024-09-21T00:20:26.509421+0000","last_became_active":"2024-09-21T00:18:48.729463+0000","last_became_peered":"2024-09-21T00:18:48.729463+0000","last_unstale":"2024-09-21T00:20:26.509421+0000","last_undegraded":"2024-09-21T00:20:26.509421+0000","last_fullsized":"2024-09-21T00:20:26.509421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:55.352870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764244+0000","last_change":"2024-09-21T00:18:38.671244+0000","last_active":"2024-09-21T00:20:54.764244+0000","last_peered":"2024-09-21T00:20:54.764244+0000","last_clean":"2024-09-21T00:20:54.764244+0000","last_became_active":"2024-09-21T00:18:38.670630+0000","last_became_peered":"2024-09-21T00:18:38.670630+0000","last_unstale":"2024-09-21T00:20:54.764244+0000","last_undegraded":"2024-09-21T00:20:54.764244+0000","last_fullsized":"2024-09-21T00:20:54.764244+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:23.695660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451865+0000","last_change":"2024-09-21T00:19:25.843900+0000","last_active":"2024-09-21T00:20:26.451865+0000","last_peered":"2024-09-21T00:20:26.451865+0000","last_clean":"2024-09-21T00:20:26.451865+0000","last_became_active":"2024-09-21T00:19:25.843438+0000","last_became_peered":"2024-09-21T00:19:25.843438+0000","last_unstale":"2024-09-21T00:20:26.451865+0000","last_undegraded":"2024-09-21T00:20:26.451865+0000","last_fullsized":"2024-09-21T00:20:26.451865+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:17.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509404+0000","last_change":"2024-09-21T00:18:48.739640+0000","last_active":"2024-09-21T00:20:26.509404+0000","last_peered":"2024-09-21T00:20:26.509404+0000","last_clean":"2024-09-21T00:20:26.509404+0000","last_became_active":"2024-09-21T00:18:48.739118+0000","last_became_peered":"2024-09-21T00:18:48.739118+0000","last_unstale":"2024-09-21T00:20:26.509404+0000","last_undegraded":"2024-09-21T00:20:26.509404+0000","last_fullsized":"2024-09-21T00:20:26.509404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:03.093561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.766430+0000","last_change":"2024-09-21T00:18:38.673020+0000","last_active":"2024-09-21T00:20:54.766430+0000","last_peered":"2024-09-21T00:20:54.766430+0000","last_clean":"2024-09-21T00:20:54.766430+0000","last_became_active":"2024-09-21T00:18:38.672391+0000","last_became_peered":"2024-09-21T00:18:38.672391+0000","last_unstale":"2024-09-21T00:20:54.766430+0000","last_undegraded":"2024-09-21T00:20:54.766430+0000","last_fullsized":"2024-09-21T00:20:54.766430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:13.866576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'60","reported_seq":136,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:37.877664+0000","last_change":"2024-09-21T00:18:35.615844+0000","last_active":"2024-09-21T00:21:37.877664+0000","last_peered":"2024-09-21T00:21:37.877664+0000","last_clean":"2024-09-21T00:21:37.877664+0000","last_became_active":"2024-09-21T00:18:35.615385+0000","last_became_peered":"2024-09-21T00:18:35.615385+0000","last_unstale":"2024-09-21T00:21:37.877664+0000","last_undegraded":"2024-09-21T00:21:37.877664+0000","last_fullsized":"2024-09-21T00:21:37.877664+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:03.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.512987+0000","last_change":"2024-09-21T00:20:26.511995+0000","last_active":"2024-09-21T00:20:26.512987+0000","last_peered":"2024-09-21T00:20:26.512987+0000","last_clean":"2024-09-21T00:20:26.512987+0000","last_became_active":"2024-09-21T00:20:26.511607+0000","last_became_peered":"2024-09-21T00:20:26.511607+0000","last_unstale":"2024-09-21T00:20:26.512987+0000","last_undegraded":"2024-09-21T00:20:26.512987+0000","last_fullsized":"2024-09-21T00:20:26.512987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:38.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754513+0000","last_change":"2024-09-21T00:18:38.671280+0000","last_active":"2024-09-21T00:20:54.754513+0000","last_peered":"2024-09-21T00:20:54.754513+0000","last_clean":"2024-09-21T00:20:54.754513+0000","last_became_active":"2024-09-21T00:18:38.670770+0000","last_became_peered":"2024-09-21T00:18:38.670770+0000","last_unstale":"2024-09-21T00:20:54.754513+0000","last_undegraded":"2024-09-21T00:20:54.754513+0000","last_fullsized":"2024-09-21T00:20:54.754513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:06.809450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509590+0000","last_change":"2024-09-21T00:18:35.626071+0000","last_active":"2024-09-21T00:20:26.509590+0000","last_peered":"2024-09-21T00:20:26.509590+0000","last_clean":"2024-09-21T00:20:26.509590+0000","last_became_active":"2024-09-21T00:18:35.625649+0000","last_became_peered":"2024-09-21T00:18:35.625649+0000","last_unstale":"2024-09-21T00:20:26.509590+0000","last_undegraded":"2024-09-21T00:20:26.509590+0000","last_fullsized":"2024-09-21T00:20:26.509590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:16:27.516303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508631+0000","last_change":"2024-09-21T00:18:48.728451+0000","last_active":"2024-09-21T00:20:26.508631+0000","last_peered":"2024-09-21T00:20:26.508631+0000","last_clean":"2024-09-21T00:20:26.508631+0000","last_became_active":"2024-09-21T00:18:48.727840+0000","last_became_peered":"2024-09-21T00:18:48.727840+0000","last_unstale":"2024-09-21T00:20:26.508631+0000","last_undegraded":"2024-09-21T00:20:26.508631+0000","last_fullsized":"2024-09-21T00:20:26.508631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:13.457138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754460+0000","last_change":"2024-09-21T00:18:38.664090+0000","last_active":"2024-09-21T00:20:54.754460+0000","last_peered":"2024-09-21T00:20:54.754460+0000","last_clean":"2024-09-21T00:20:54.754460+0000","last_became_active":"2024-09-21T00:18:38.663728+0000","last_became_peered":"2024-09-21T00:18:38.663728+0000","last_unstale":"2024-09-21T00:20:54.754460+0000","last_undegraded":"2024-09-21T00:20:54.754460+0000","last_fullsized":"2024-09-21T00:20:54.754460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:46.367766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454914+0000","last_change":"2024-09-21T00:19:25.847280+0000","last_active":"2024-09-21T00:20:26.454914+0000","last_peered":"2024-09-21T00:20:26.454914+0000","last_clean":"2024-09-21T00:20:26.454914+0000","last_became_active":"2024-09-21T00:19:25.840919+0000","last_became_peered":"2024-09-21T00:19:25.840919+0000","last_unstale":"2024-09-21T00:20:26.454914+0000","last_undegraded":"2024-09-21T00:20:26.454914+0000","last_fullsized":"2024-09-21T00:20:26.454914+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:15.411674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508811+0000","last_change":"2024-09-21T00:18:48.732427+0000","last_active":"2024-09-21T00:20:26.508811+0000","last_peered":"2024-09-21T00:20:26.508811+0000","last_clean":"2024-09-21T00:20:26.508811+0000","last_became_active":"2024-09-21T00:18:48.732078+0000","last_became_peered":"2024-09-21T00:18:48.732078+0000","last_unstale":"2024-09-21T00:20:26.508811+0000","last_undegraded":"2024-09-21T00:20:26.508811+0000","last_fullsized":"2024-09-21T00:20:26.508811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:35.055680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764481+0000","last_change":"2024-09-21T00:18:38.675022+0000","last_active":"2024-09-21T00:20:54.764481+0000","last_peered":"2024-09-21T00:20:54.764481+0000","last_clean":"2024-09-21T00:20:54.764481+0000","last_became_active":"2024-09-21T00:18:38.674650+0000","last_became_peered":"2024-09-21T00:18:38.674650+0000","last_unstale":"2024-09-21T00:20:54.764481+0000","last_undegraded":"2024-09-21T00:20:54.764481+0000","last_fullsized":"2024-09-21T00:20:54.764481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:05.191404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510139+0000","last_change":"2024-09-21T00:18:35.769555+0000","last_active":"2024-09-21T00:20:26.510139+0000","last_peered":"2024-09-21T00:20:26.510139+0000","last_clean":"2024-09-21T00:20:26.510139+0000","last_became_active":"2024-09-21T00:18:35.769080+0000","last_became_peered":"2024-09-21T00:18:35.769080+0000","last_unstale":"2024-09-21T00:20:26.510139+0000","last_undegraded":"2024-09-21T00:20:26.510139+0000","last_fullsized":"2024-09-21T00:20:26.510139+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:19.306830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509529+0000","last_change":"2024-09-21T00:18:48.742957+0000","last_active":"2024-09-21T00:20:26.509529+0000","last_peered":"2024-09-21T00:20:26.509529+0000","last_clean":"2024-09-21T00:20:26.509529+0000","last_became_active":"2024-09-21T00:18:48.742511+0000","last_became_peered":"2024-09-21T00:18:48.742511+0000","last_unstale":"2024-09-21T00:20:26.509529+0000","last_undegraded":"2024-09-21T00:20:26.509529+0000","last_fullsized":"2024-09-21T00:20:26.509529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:15.231189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.761135+0000","last_change":"2024-09-21T00:18:38.666576+0000","last_active":"2024-09-21T00:20:54.761135+0000","last_peered":"2024-09-21T00:20:54.761135+0000","last_clean":"2024-09-21T00:20:54.761135+0000","last_became_active":"2024-09-21T00:18:38.665643+0000","last_became_peered":"2024-09-21T00:18:38.665643+0000","last_unstale":"2024-09-21T00:20:54.761135+0000","last_undegraded":"2024-09-21T00:20:54.761135+0000","last_fullsized":"2024-09-21T00:20:54.761135+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:35.397847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":176,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:39.116713+0000","last_change":"2024-09-21T00:18:35.623143+0000","last_active":"2024-09-21T00:21:39.116713+0000","last_peered":"2024-09-21T00:21:39.116713+0000","last_clean":"2024-09-21T00:21:39.116713+0000","last_became_active":"2024-09-21T00:18:35.622693+0000","last_became_peered":"2024-09-21T00:18:35.622693+0000","last_unstale":"2024-09-21T00:21:39.116713+0000","last_undegraded":"2024-09-21T00:21:39.116713+0000","last_fullsized":"2024-09-21T00:21:39.116713+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:01.331665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452872+0000","last_change":"2024-09-21T00:18:48.723877+0000","last_active":"2024-09-21T00:20:26.452872+0000","last_peered":"2024-09-21T00:20:26.452872+0000","last_clean":"2024-09-21T00:20:26.452872+0000","last_became_active":"2024-09-21T00:18:48.723360+0000","last_became_peered":"2024-09-21T00:18:48.723360+0000","last_unstale":"2024-09-21T00:20:26.452872+0000","last_undegraded":"2024-09-21T00:20:26.452872+0000","last_fullsized":"2024-09-21T00:20:26.452872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:07:20.442867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767171+0000","last_change":"2024-09-21T00:18:38.669643+0000","last_active":"2024-09-21T00:20:54.767171+0000","last_peered":"2024-09-21T00:20:54.767171+0000","last_clean":"2024-09-21T00:20:54.767171+0000","last_became_active":"2024-09-21T00:18:38.669245+0000","last_became_peered":"2024-09-21T00:18:38.669245+0000","last_unstale":"2024-09-21T00:20:54.767171+0000","last_undegraded":"2024-09-21T00:20:54.767171+0000","last_fullsized":"2024-09-21T00:20:54.767171+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:17.789291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509577+0000","last_change":"2024-09-21T00:18:35.623189+0000","last_active":"2024-09-21T00:20:26.509577+0000","last_peered":"2024-09-21T00:20:26.509577+0000","last_clean":"2024-09-21T00:20:26.509577+0000","last_became_active":"2024-09-21T00:18:35.622748+0000","last_became_peered":"2024-09-21T00:18:35.622748+0000","last_unstale":"2024-09-21T00:20:26.509577+0000","last_undegraded":"2024-09-21T00:20:26.509577+0000","last_fullsized":"2024-09-21T00:20:26.509577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:15.570789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453838+0000","last_change":"2024-09-21T00:18:48.739612+0000","last_active":"2024-09-21T00:20:26.453838+0000","last_peered":"2024-09-21T00:20:26.453838+0000","last_clean":"2024-09-21T00:20:26.453838+0000","last_became_active":"2024-09-21T00:18:48.739173+0000","last_became_peered":"2024-09-21T00:18:48.739173+0000","last_unstale":"2024-09-21T00:20:26.453838+0000","last_undegraded":"2024-09-21T00:20:26.453838+0000","last_fullsized":"2024-09-21T00:20:26.453838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:26.327178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.766649+0000","last_change":"2024-09-21T00:18:38.670876+0000","last_active":"2024-09-21T00:20:54.766649+0000","last_peered":"2024-09-21T00:20:54.766649+0000","last_clean":"2024-09-21T00:20:54.766649+0000","last_became_active":"2024-09-21T00:18:38.670371+0000","last_became_peered":"2024-09-21T00:18:38.670371+0000","last_unstale":"2024-09-21T00:20:54.766649+0000","last_undegraded":"2024-09-21T00:20:54.766649+0000","last_fullsized":"2024-09-21T00:20:54.766649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:48.641071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452030+0000","last_change":"2024-09-21T00:18:35.622919+0000","last_active":"2024-09-21T00:20:26.452030+0000","last_peered":"2024-09-21T00:20:26.452030+0000","last_clean":"2024-09-21T00:20:26.452030+0000","last_became_active":"2024-09-21T00:18:35.622435+0000","last_became_peered":"2024-09-21T00:18:35.622435+0000","last_unstale":"2024-09-21T00:20:26.452030+0000","last_undegraded":"2024-09-21T00:20:26.452030+0000","last_fullsized":"2024-09-21T00:20:26.452030+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:22.113871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509031+0000","last_change":"2024-09-21T00:18:48.741736+0000","last_active":"2024-09-21T00:20:26.509031+0000","last_peered":"2024-09-21T00:20:26.509031+0000","last_clean":"2024-09-21T00:20:26.509031+0000","last_became_active":"2024-09-21T00:18:48.741135+0000","last_became_peered":"2024-09-21T00:18:48.741135+0000","last_unstale":"2024-09-21T00:20:26.509031+0000","last_undegraded":"2024-09-21T00:20:26.509031+0000","last_fullsized":"2024-09-21T00:20:26.509031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:50.492351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.748256+0000","last_change":"2024-09-21T00:18:38.667377+0000","last_active":"2024-09-21T00:20:54.748256+0000","last_peered":"2024-09-21T00:20:54.748256+0000","last_clean":"2024-09-21T00:20:54.748256+0000","last_became_active":"2024-09-21T00:18:38.667010+0000","last_became_peered":"2024-09-21T00:18:38.667010+0000","last_unstale":"2024-09-21T00:20:54.748256+0000","last_undegraded":"2024-09-21T00:20:54.748256+0000","last_fullsized":"2024-09-21T00:20:54.748256+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:06.634767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":361,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:40.117626+0000","last_change":"2024-09-21T00:19:25.853603+0000","last_active":"2024-09-21T00:21:40.117626+0000","last_peered":"2024-09-21T00:21:40.117626+0000","last_clean":"2024-09-21T00:21:40.117626+0000","last_became_active":"2024-09-21T00:19:25.841126+0000","last_became_peered":"2024-09-21T00:19:25.841126+0000","last_unstale":"2024-09-21T00:21:40.117626+0000","last_undegraded":"2024-09-21T00:21:40.117626+0000","last_fullsized":"2024-09-21T00:21:40.117626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:27:47.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71662,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":261,"num_read_kb":201,"num_write":38,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451696+0000","last_change":"2024-09-21T00:18:48.731004+0000","last_active":"2024-09-21T00:20:26.451696+0000","last_peered":"2024-09-21T00:20:26.451696+0000","last_clean":"2024-09-21T00:20:26.451696+0000","last_became_active":"2024-09-21T00:18:48.730549+0000","last_became_peered":"2024-09-21T00:18:48.730549+0000","last_unstale":"2024-09-21T00:20:26.451696+0000","last_undegraded":"2024-09-21T00:20:26.451696+0000","last_fullsized":"2024-09-21T00:20:26.451696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:30.988300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764366+0000","last_change":"2024-09-21T00:19:25.843532+0000","last_active":"2024-09-21T00:20:54.764366+0000","last_peered":"2024-09-21T00:20:54.764366+0000","last_clean":"2024-09-21T00:20:54.764366+0000","last_became_active":"2024-09-21T00:19:25.843065+0000","last_became_peered":"2024-09-21T00:19:25.843065+0000","last_unstale":"2024-09-21T00:20:54.764366+0000","last_undegraded":"2024-09-21T00:20:54.764366+0000","last_fullsized":"2024-09-21T00:20:54.764366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:36:26.488692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509317+0000","last_change":"2024-09-21T00:18:35.770494+0000","last_active":"2024-09-21T00:20:26.509317+0000","last_peered":"2024-09-21T00:20:26.509317+0000","last_clean":"2024-09-21T00:20:26.509317+0000","last_became_active":"2024-09-21T00:18:35.770122+0000","last_became_peered":"2024-09-21T00:18:35.770122+0000","last_unstale":"2024-09-21T00:20:26.509317+0000","last_undegraded":"2024-09-21T00:20:26.509317+0000","last_fullsized":"2024-09-21T00:20:26.509317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:41.415904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451852+0000","last_change":"2024-09-21T00:18:48.736802+0000","last_active":"2024-09-21T00:20:26.451852+0000","last_peered":"2024-09-21T00:20:26.451852+0000","last_clean":"2024-09-21T00:20:26.451852+0000","last_became_active":"2024-09-21T00:18:48.736008+0000","last_became_peered":"2024-09-21T00:18:48.736008+0000","last_unstale":"2024-09-21T00:20:26.451852+0000","last_undegraded":"2024-09-21T00:20:26.451852+0000","last_fullsized":"2024-09-21T00:20:26.451852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:07.995614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767189+0000","last_change":"2024-09-21T00:18:38.678385+0000","last_active":"2024-09-21T00:20:54.767189+0000","last_peered":"2024-09-21T00:20:54.767189+0000","last_clean":"2024-09-21T00:20:54.767189+0000","last_became_active":"2024-09-21T00:18:38.677174+0000","last_became_peered":"2024-09-21T00:18:38.677174+0000","last_unstale":"2024-09-21T00:20:54.767189+0000","last_undegraded":"2024-09-21T00:20:54.767189+0000","last_fullsized":"2024-09-21T00:20:54.767189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:23.486222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509498+0000","last_change":"2024-09-21T00:18:35.627703+0000","last_active":"2024-09-21T00:20:26.509498+0000","last_peered":"2024-09-21T00:20:26.509498+0000","last_clean":"2024-09-21T00:20:26.509498+0000","last_became_active":"2024-09-21T00:18:35.627274+0000","last_became_peered":"2024-09-21T00:18:35.627274+0000","last_unstale":"2024-09-21T00:20:26.509498+0000","last_undegraded":"2024-09-21T00:20:26.509498+0000","last_fullsized":"2024-09-21T00:20:26.509498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:14.326810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451122+0000","last_change":"2024-09-21T00:18:48.729926+0000","last_active":"2024-09-21T00:20:26.451122+0000","last_peered":"2024-09-21T00:20:26.451122+0000","last_clean":"2024-09-21T00:20:26.451122+0000","last_became_active":"2024-09-21T00:18:48.728366+0000","last_became_peered":"2024-09-21T00:18:48.728366+0000","last_unstale":"2024-09-21T00:20:26.451122+0000","last_undegraded":"2024-09-21T00:20:26.451122+0000","last_fullsized":"2024-09-21T00:20:26.451122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:48:50.005748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.763272+0000","last_change":"2024-09-21T00:18:38.664173+0000","last_active":"2024-09-21T00:20:54.763272+0000","last_peered":"2024-09-21T00:20:54.763272+0000","last_clean":"2024-09-21T00:20:54.763272+0000","last_became_active":"2024-09-21T00:18:38.663358+0000","last_became_peered":"2024-09-21T00:18:38.663358+0000","last_unstale":"2024-09-21T00:20:54.763272+0000","last_undegraded":"2024-09-21T00:20:54.763272+0000","last_fullsized":"2024-09-21T00:20:54.763272+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:34:31.203621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":175,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:39.115736+0000","last_change":"2024-09-21T00:18:35.622133+0000","last_active":"2024-09-21T00:21:39.115736+0000","last_peered":"2024-09-21T00:21:39.115736+0000","last_clean":"2024-09-21T00:21:39.115736+0000","last_became_active":"2024-09-21T00:18:35.621032+0000","last_became_peered":"2024-09-21T00:18:35.621032+0000","last_unstale":"2024-09-21T00:21:39.115736+0000","last_undegraded":"2024-09-21T00:21:39.115736+0000","last_fullsized":"2024-09-21T00:21:39.115736+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:24.121665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574990+0000","last_change":"2024-09-21T00:18:48.733339+0000","last_active":"2024-09-21T00:20:26.574990+0000","last_peered":"2024-09-21T00:20:26.574990+0000","last_clean":"2024-09-21T00:20:26.574990+0000","last_became_active":"2024-09-21T00:18:48.731843+0000","last_became_peered":"2024-09-21T00:18:48.731843+0000","last_unstale":"2024-09-21T00:20:26.574990+0000","last_undegraded":"2024-09-21T00:20:26.574990+0000","last_fullsized":"2024-09-21T00:20:26.574990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:42:55.474197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455859+0000","last_change":"2024-09-21T00:18:48.740108+0000","last_active":"2024-09-21T00:20:26.455859+0000","last_peered":"2024-09-21T00:20:26.455859+0000","last_clean":"2024-09-21T00:20:26.455859+0000","last_became_active":"2024-09-21T00:18:48.739330+0000","last_became_peered":"2024-09-21T00:18:48.739330+0000","last_unstale":"2024-09-21T00:20:26.455859+0000","last_undegraded":"2024-09-21T00:20:26.455859+0000","last_fullsized":"2024-09-21T00:20:26.455859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:00.103502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.763622+0000","last_change":"2024-09-21T00:18:38.660469+0000","last_active":"2024-09-21T00:20:54.763622+0000","last_peered":"2024-09-21T00:20:54.763622+0000","last_clean":"2024-09-21T00:20:54.763622+0000","last_became_active":"2024-09-21T00:18:38.660138+0000","last_became_peered":"2024-09-21T00:18:38.660138+0000","last_unstale":"2024-09-21T00:20:54.763622+0000","last_undegraded":"2024-09-21T00:20:54.763622+0000","last_fullsized":"2024-09-21T00:20:54.763622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:01.079112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509206+0000","last_change":"2024-09-21T00:18:35.624528+0000","last_active":"2024-09-21T00:20:26.509206+0000","last_peered":"2024-09-21T00:20:26.509206+0000","last_clean":"2024-09-21T00:20:26.509206+0000","last_became_active":"2024-09-21T00:18:35.624170+0000","last_became_peered":"2024-09-21T00:18:35.624170+0000","last_unstale":"2024-09-21T00:20:26.509206+0000","last_undegraded":"2024-09-21T00:20:26.509206+0000","last_fullsized":"2024-09-21T00:20:26.509206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:07.914521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.512325+0000","last_change":"2024-09-21T00:20:26.511824+0000","last_active":"2024-09-21T00:20:26.512325+0000","last_peered":"2024-09-21T00:20:26.512325+0000","last_clean":"2024-09-21T00:20:26.512325+0000","last_became_active":"2024-09-21T00:20:26.511254+0000","last_became_peered":"2024-09-21T00:20:26.511254+0000","last_unstale":"2024-09-21T00:20:26.512325+0000","last_undegraded":"2024-09-21T00:20:26.512325+0000","last_fullsized":"2024-09-21T00:20:26.512325+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:12.884034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.760218+0000","last_change":"2024-09-21T00:18:38.666928+0000","last_active":"2024-09-21T00:20:54.760218+0000","last_peered":"2024-09-21T00:20:54.760218+0000","last_clean":"2024-09-21T00:20:54.760218+0000","last_became_active":"2024-09-21T00:18:38.666236+0000","last_became_peered":"2024-09-21T00:18:38.666236+0000","last_unstale":"2024-09-21T00:20:54.760218+0000","last_undegraded":"2024-09-21T00:20:54.760218+0000","last_fullsized":"2024-09-21T00:20:54.760218+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:21.043778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508330+0000","last_change":"2024-09-21T00:18:35.624514+0000","last_active":"2024-09-21T00:20:26.508330+0000","last_peered":"2024-09-21T00:20:26.508330+0000","last_clean":"2024-09-21T00:20:26.508330+0000","last_became_active":"2024-09-21T00:18:35.624098+0000","last_became_peered":"2024-09-21T00:18:35.624098+0000","last_unstale":"2024-09-21T00:20:26.508330+0000","last_undegraded":"2024-09-21T00:20:26.508330+0000","last_fullsized":"2024-09-21T00:20:26.508330+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:21.777568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509891+0000","last_change":"2024-09-21T00:19:25.841232+0000","last_active":"2024-09-21T00:20:26.509891+0000","last_peered":"2024-09-21T00:20:26.509891+0000","last_clean":"2024-09-21T00:20:26.509891+0000","last_became_active":"2024-09-21T00:19:25.840655+0000","last_became_peered":"2024-09-21T00:19:25.840655+0000","last_unstale":"2024-09-21T00:20:26.509891+0000","last_undegraded":"2024-09-21T00:20:26.509891+0000","last_fullsized":"2024-09-21T00:20:26.509891+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:05.846222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.762522+0000","last_change":"2024-09-21T00:18:38.667466+0000","last_active":"2024-09-21T00:20:54.762522+0000","last_peered":"2024-09-21T00:20:54.762522+0000","last_clean":"2024-09-21T00:20:54.762522+0000","last_became_active":"2024-09-21T00:18:38.666960+0000","last_became_peered":"2024-09-21T00:18:38.666960+0000","last_unstale":"2024-09-21T00:20:54.762522+0000","last_undegraded":"2024-09-21T00:20:54.762522+0000","last_fullsized":"2024-09-21T00:20:54.762522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:29:56.614766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451707+0000","last_change":"2024-09-21T00:18:48.739656+0000","last_active":"2024-09-21T00:20:26.451707+0000","last_peered":"2024-09-21T00:20:26.451707+0000","last_clean":"2024-09-21T00:20:26.451707+0000","last_became_active":"2024-09-21T00:18:48.739228+0000","last_became_peered":"2024-09-21T00:18:48.739228+0000","last_unstale":"2024-09-21T00:20:26.451707+0000","last_undegraded":"2024-09-21T00:20:26.451707+0000","last_fullsized":"2024-09-21T00:20:26.451707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:11.427595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":183,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:40.118153+0000","last_change":"2024-09-21T00:18:35.624108+0000","last_active":"2024-09-21T00:21:40.118153+0000","last_peered":"2024-09-21T00:21:40.118153+0000","last_clean":"2024-09-21T00:21:40.118153+0000","last_became_active":"2024-09-21T00:18:35.623708+0000","last_became_peered":"2024-09-21T00:18:35.623708+0000","last_unstale":"2024-09-21T00:21:40.118153+0000","last_undegraded":"2024-09-21T00:21:40.118153+0000","last_fullsized":"2024-09-21T00:21:40.118153+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:50.020662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.759755+0000","last_change":"2024-09-21T00:18:38.665402+0000","last_active":"2024-09-21T00:20:54.759755+0000","last_peered":"2024-09-21T00:20:54.759755+0000","last_clean":"2024-09-21T00:20:54.759755+0000","last_became_active":"2024-09-21T00:18:38.665034+0000","last_became_peered":"2024-09-21T00:18:38.665034+0000","last_unstale":"2024-09-21T00:20:54.759755+0000","last_undegraded":"2024-09-21T00:20:54.759755+0000","last_fullsized":"2024-09-21T00:20:54.759755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:33:10.824705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451804+0000","last_change":"2024-09-21T00:18:48.729928+0000","last_active":"2024-09-21T00:20:26.451804+0000","last_peered":"2024-09-21T00:20:26.451804+0000","last_clean":"2024-09-21T00:20:26.451804+0000","last_became_active":"2024-09-21T00:18:48.729498+0000","last_became_peered":"2024-09-21T00:18:48.729498+0000","last_unstale":"2024-09-21T00:20:26.451804+0000","last_undegraded":"2024-09-21T00:20:26.451804+0000","last_fullsized":"2024-09-21T00:20:26.451804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:10:27.140012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754670+0000","last_change":"2024-09-21T00:18:38.662221+0000","last_active":"2024-09-21T00:20:54.754670+0000","last_peered":"2024-09-21T00:20:54.754670+0000","last_clean":"2024-09-21T00:20:54.754670+0000","last_became_active":"2024-09-21T00:18:38.660543+0000","last_became_peered":"2024-09-21T00:18:38.660543+0000","last_unstale":"2024-09-21T00:20:54.754670+0000","last_undegraded":"2024-09-21T00:20:54.754670+0000","last_fullsized":"2024-09-21T00:20:54.754670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:43.159699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.759670+0000","last_change":"2024-09-21T00:18:38.672696+0000","last_active":"2024-09-21T00:20:54.759670+0000","last_peered":"2024-09-21T00:20:54.759670+0000","last_clean":"2024-09-21T00:20:54.759670+0000","last_became_active":"2024-09-21T00:18:38.672356+0000","last_became_peered":"2024-09-21T00:18:38.672356+0000","last_unstale":"2024-09-21T00:20:54.759670+0000","last_undegraded":"2024-09-21T00:20:54.759670+0000","last_fullsized":"2024-09-21T00:20:54.759670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.549939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768251+0000","last_change":"2024-09-21T00:18:38.667017+0000","last_active":"2024-09-21T00:20:54.768251+0000","last_peered":"2024-09-21T00:20:54.768251+0000","last_clean":"2024-09-21T00:20:54.768251+0000","last_became_active":"2024-09-21T00:18:38.666643+0000","last_became_peered":"2024-09-21T00:18:38.666643+0000","last_unstale":"2024-09-21T00:20:54.768251+0000","last_undegraded":"2024-09-21T00:20:54.768251+0000","last_fullsized":"2024-09-21T00:20:54.768251+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:30.496339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767616+0000","last_change":"2024-09-21T00:18:38.667705+0000","last_active":"2024-09-21T00:20:54.767616+0000","last_peered":"2024-09-21T00:20:54.767616+0000","last_clean":"2024-09-21T00:20:54.767616+0000","last_became_active":"2024-09-21T00:18:38.665141+0000","last_became_peered":"2024-09-21T00:18:38.665141+0000","last_unstale":"2024-09-21T00:20:54.767616+0000","last_undegraded":"2024-09-21T00:20:54.767616+0000","last_fullsized":"2024-09-21T00:20:54.767616+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:17.624035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764929+0000","last_change":"2024-09-21T00:18:38.681364+0000","last_active":"2024-09-21T00:20:54.764929+0000","last_peered":"2024-09-21T00:20:54.764929+0000","last_clean":"2024-09-21T00:20:54.764929+0000","last_became_active":"2024-09-21T00:18:38.680937+0000","last_became_peered":"2024-09-21T00:18:38.680937+0000","last_unstale":"2024-09-21T00:20:54.764929+0000","last_undegraded":"2024-09-21T00:20:54.764929+0000","last_fullsized":"2024-09-21T00:20:54.764929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:34.206201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754004+0000","last_change":"2024-09-21T00:19:25.843091+0000","last_active":"2024-09-21T00:20:54.754004+0000","last_peered":"2024-09-21T00:20:54.754004+0000","last_clean":"2024-09-21T00:20:54.754004+0000","last_became_active":"2024-09-21T00:19:25.842600+0000","last_became_peered":"2024-09-21T00:19:25.842600+0000","last_unstale":"2024-09-21T00:20:54.754004+0000","last_undegraded":"2024-09-21T00:20:54.754004+0000","last_fullsized":"2024-09-21T00:20:54.754004+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:10.256474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.753901+0000","last_change":"2024-09-21T00:18:38.668656+0000","last_active":"2024-09-21T00:20:54.753901+0000","last_peered":"2024-09-21T00:20:54.753901+0000","last_clean":"2024-09-21T00:20:54.753901+0000","last_became_active":"2024-09-21T00:18:38.668153+0000","last_became_peered":"2024-09-21T00:18:38.668153+0000","last_unstale":"2024-09-21T00:20:54.753901+0000","last_undegraded":"2024-09-21T00:20:54.753901+0000","last_fullsized":"2024-09-21T00:20:54.753901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:03.763609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.743954+0000","last_change":"2024-09-21T00:18:38.675945+0000","last_active":"2024-09-21T00:20:54.743954+0000","last_peered":"2024-09-21T00:20:54.743954+0000","last_clean":"2024-09-21T00:20:54.743954+0000","last_became_active":"2024-09-21T00:18:38.675397+0000","last_became_peered":"2024-09-21T00:18:38.675397+0000","last_unstale":"2024-09-21T00:20:54.743954+0000","last_undegraded":"2024-09-21T00:20:54.743954+0000","last_fullsized":"2024-09-21T00:20:54.743954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:56.082314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.762260+0000","last_change":"2024-09-21T00:18:38.662854+0000","last_active":"2024-09-21T00:20:54.762260+0000","last_peered":"2024-09-21T00:20:54.762260+0000","last_clean":"2024-09-21T00:20:54.762260+0000","last_became_active":"2024-09-21T00:18:38.662094+0000","last_became_peered":"2024-09-21T00:18:38.662094+0000","last_unstale":"2024-09-21T00:20:54.762260+0000","last_undegraded":"2024-09-21T00:20:54.762260+0000","last_fullsized":"2024-09-21T00:20:54.762260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:45.756641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.744870+0000","last_change":"2024-09-21T00:18:38.663282+0000","last_active":"2024-09-21T00:20:54.744870+0000","last_peered":"2024-09-21T00:20:54.744870+0000","last_clean":"2024-09-21T00:20:54.744870+0000","last_became_active":"2024-09-21T00:18:38.662148+0000","last_became_peered":"2024-09-21T00:18:38.662148+0000","last_unstale":"2024-09-21T00:20:54.744870+0000","last_undegraded":"2024-09-21T00:20:54.744870+0000","last_fullsized":"2024-09-21T00:20:54.744870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:51.090591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.763683+0000","last_change":"2024-09-21T00:18:38.669184+0000","last_active":"2024-09-21T00:20:54.763683+0000","last_peered":"2024-09-21T00:20:54.763683+0000","last_clean":"2024-09-21T00:20:54.763683+0000","last_became_active":"2024-09-21T00:18:38.668799+0000","last_became_peered":"2024-09-21T00:18:38.668799+0000","last_unstale":"2024-09-21T00:20:54.763683+0000","last_undegraded":"2024-09-21T00:20:54.763683+0000","last_fullsized":"2024-09-21T00:20:54.763683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:49.642568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767674+0000","last_change":"2024-09-21T00:18:38.679501+0000","last_active":"2024-09-21T00:20:54.767674+0000","last_peered":"2024-09-21T00:20:54.767674+0000","last_clean":"2024-09-21T00:20:54.767674+0000","last_became_active":"2024-09-21T00:18:38.678582+0000","last_became_peered":"2024-09-21T00:18:38.678582+0000","last_unstale":"2024-09-21T00:20:54.767674+0000","last_undegraded":"2024-09-21T00:20:54.767674+0000","last_fullsized":"2024-09-21T00:20:54.767674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:30.277979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.751399+0000","last_change":"2024-09-21T00:18:38.659657+0000","last_active":"2024-09-21T00:20:54.751399+0000","last_peered":"2024-09-21T00:20:54.751399+0000","last_clean":"2024-09-21T00:20:54.751399+0000","last_became_active":"2024-09-21T00:18:38.659257+0000","last_became_peered":"2024-09-21T00:18:38.659257+0000","last_unstale":"2024-09-21T00:20:54.751399+0000","last_undegraded":"2024-09-21T00:20:54.751399+0000","last_fullsized":"2024-09-21T00:20:54.751399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:08.653729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.758299+0000","last_change":"2024-09-21T00:18:38.671817+0000","last_active":"2024-09-21T00:20:54.758299+0000","last_peered":"2024-09-21T00:20:54.758299+0000","last_clean":"2024-09-21T00:20:54.758299+0000","last_became_active":"2024-09-21T00:18:38.671469+0000","last_became_peered":"2024-09-21T00:18:38.671469+0000","last_unstale":"2024-09-21T00:20:54.758299+0000","last_undegraded":"2024-09-21T00:20:54.758299+0000","last_fullsized":"2024-09-21T00:20:54.758299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:58:15.036623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.761325+0000","last_change":"2024-09-21T00:18:38.664481+0000","last_active":"2024-09-21T00:20:54.761325+0000","last_peered":"2024-09-21T00:20:54.761325+0000","last_clean":"2024-09-21T00:20:54.761325+0000","last_became_active":"2024-09-21T00:18:38.664098+0000","last_became_peered":"2024-09-21T00:18:38.664098+0000","last_unstale":"2024-09-21T00:20:54.761325+0000","last_undegraded":"2024-09-21T00:20:54.761325+0000","last_fullsized":"2024-09-21T00:20:54.761325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:09.271251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768665+0000","last_change":"2024-09-21T00:18:38.662416+0000","last_active":"2024-09-21T00:20:54.768665+0000","last_peered":"2024-09-21T00:20:54.768665+0000","last_clean":"2024-09-21T00:20:54.768665+0000","last_became_active":"2024-09-21T00:18:38.662041+0000","last_became_peered":"2024-09-21T00:18:38.662041+0000","last_unstale":"2024-09-21T00:20:54.768665+0000","last_undegraded":"2024-09-21T00:20:54.768665+0000","last_fullsized":"2024-09-21T00:20:54.768665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:03.150179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768448+0000","last_change":"2024-09-21T00:18:38.670797+0000","last_active":"2024-09-21T00:20:54.768448+0000","last_peered":"2024-09-21T00:20:54.768448+0000","last_clean":"2024-09-21T00:20:54.768448+0000","last_became_active":"2024-09-21T00:18:38.670258+0000","last_became_peered":"2024-09-21T00:18:38.670258+0000","last_unstale":"2024-09-21T00:20:54.768448+0000","last_undegraded":"2024-09-21T00:20:54.768448+0000","last_fullsized":"2024-09-21T00:20:54.768448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:55.669836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.765633+0000","last_change":"2024-09-21T00:18:38.665594+0000","last_active":"2024-09-21T00:20:54.765633+0000","last_peered":"2024-09-21T00:20:54.765633+0000","last_clean":"2024-09-21T00:20:54.765633+0000","last_became_active":"2024-09-21T00:18:38.665260+0000","last_became_peered":"2024-09-21T00:18:38.665260+0000","last_unstale":"2024-09-21T00:20:54.765633+0000","last_undegraded":"2024-09-21T00:20:54.765633+0000","last_fullsized":"2024-09-21T00:20:54.765633+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:52:09.014327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768070+0000","last_change":"2024-09-21T00:18:38.679294+0000","last_active":"2024-09-21T00:20:54.768070+0000","last_peered":"2024-09-21T00:20:54.768070+0000","last_clean":"2024-09-21T00:20:54.768070+0000","last_became_active":"2024-09-21T00:18:38.673765+0000","last_became_peered":"2024-09-21T00:18:38.673765+0000","last_unstale":"2024-09-21T00:20:54.768070+0000","last_undegraded":"2024-09-21T00:20:54.768070+0000","last_fullsized":"2024-09-21T00:20:54.768070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:32.076299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767090+0000","last_change":"2024-09-21T00:18:38.667869+0000","last_active":"2024-09-21T00:20:54.767090+0000","last_peered":"2024-09-21T00:20:54.767090+0000","last_clean":"2024-09-21T00:20:54.767090+0000","last_became_active":"2024-09-21T00:18:38.667499+0000","last_became_peered":"2024-09-21T00:18:38.667499+0000","last_unstale":"2024-09-21T00:20:54.767090+0000","last_undegraded":"2024-09-21T00:20:54.767090+0000","last_fullsized":"2024-09-21T00:20:54.767090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:13.501043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.761252+0000","last_change":"2024-09-21T00:18:38.663107+0000","last_active":"2024-09-21T00:20:54.761252+0000","last_peered":"2024-09-21T00:20:54.761252+0000","last_clean":"2024-09-21T00:20:54.761252+0000","last_became_active":"2024-09-21T00:18:38.662680+0000","last_became_peered":"2024-09-21T00:18:38.662680+0000","last_unstale":"2024-09-21T00:20:54.761252+0000","last_undegraded":"2024-09-21T00:20:54.761252+0000","last_fullsized":"2024-09-21T00:20:54.761252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:48.154705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764194+0000","last_change":"2024-09-21T00:18:38.668167+0000","last_active":"2024-09-21T00:20:54.764194+0000","last_peered":"2024-09-21T00:20:54.764194+0000","last_clean":"2024-09-21T00:20:54.764194+0000","last_became_active":"2024-09-21T00:18:38.667842+0000","last_became_peered":"2024-09-21T00:18:38.667842+0000","last_unstale":"2024-09-21T00:20:54.764194+0000","last_undegraded":"2024-09-21T00:20:54.764194+0000","last_fullsized":"2024-09-21T00:20:54.764194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:55.358351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.761794+0000","last_change":"2024-09-21T00:18:38.666580+0000","last_active":"2024-09-21T00:20:54.761794+0000","last_peered":"2024-09-21T00:20:54.761794+0000","last_clean":"2024-09-21T00:20:54.761794+0000","last_became_active":"2024-09-21T00:18:38.666231+0000","last_became_peered":"2024-09-21T00:18:38.666231+0000","last_unstale":"2024-09-21T00:20:54.761794+0000","last_undegraded":"2024-09-21T00:20:54.761794+0000","last_fullsized":"2024-09-21T00:20:54.761794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:07.362516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.751359+0000","last_change":"2024-09-21T00:20:26.511248+0000","last_active":"2024-09-21T00:20:54.751359+0000","last_peered":"2024-09-21T00:20:54.751359+0000","last_clean":"2024-09-21T00:20:54.751359+0000","last_became_active":"2024-09-21T00:20:26.510743+0000","last_became_peered":"2024-09-21T00:20:26.510743+0000","last_unstale":"2024-09-21T00:20:54.751359+0000","last_undegraded":"2024-09-21T00:20:54.751359+0000","last_fullsized":"2024-09-21T00:20:54.751359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:41.577950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.760033+0000","last_change":"2024-09-21T00:18:38.666580+0000","last_active":"2024-09-21T00:20:54.760033+0000","last_peered":"2024-09-21T00:20:54.760033+0000","last_clean":"2024-09-21T00:20:54.760033+0000","last_became_active":"2024-09-21T00:18:38.665698+0000","last_became_peered":"2024-09-21T00:18:38.665698+0000","last_unstale":"2024-09-21T00:20:54.760033+0000","last_undegraded":"2024-09-21T00:20:54.760033+0000","last_fullsized":"2024-09-21T00:20:54.760033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:45.627667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.765480+0000","last_change":"2024-09-21T00:18:38.670729+0000","last_active":"2024-09-21T00:20:54.765480+0000","last_peered":"2024-09-21T00:20:54.765480+0000","last_clean":"2024-09-21T00:20:54.765480+0000","last_became_active":"2024-09-21T00:18:38.670337+0000","last_became_peered":"2024-09-21T00:18:38.670337+0000","last_unstale":"2024-09-21T00:20:54.765480+0000","last_undegraded":"2024-09-21T00:20:54.765480+0000","last_fullsized":"2024-09-21T00:20:54.765480+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:49.880689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.763380+0000","last_change":"2024-09-21T00:18:38.664644+0000","last_active":"2024-09-21T00:20:54.763380+0000","last_peered":"2024-09-21T00:20:54.763380+0000","last_clean":"2024-09-21T00:20:54.763380+0000","last_became_active":"2024-09-21T00:18:38.664220+0000","last_became_peered":"2024-09-21T00:18:38.664220+0000","last_unstale":"2024-09-21T00:20:54.763380+0000","last_undegraded":"2024-09-21T00:20:54.763380+0000","last_fullsized":"2024-09-21T00:20:54.763380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:09.610962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.759277+0000","last_change":"2024-09-21T00:18:38.671592+0000","last_active":"2024-09-21T00:20:54.759277+0000","last_peered":"2024-09-21T00:20:54.759277+0000","last_clean":"2024-09-21T00:20:54.759277+0000","last_became_active":"2024-09-21T00:18:38.671228+0000","last_became_peered":"2024-09-21T00:18:38.671228+0000","last_unstale":"2024-09-21T00:20:54.759277+0000","last_undegraded":"2024-09-21T00:20:54.759277+0000","last_fullsized":"2024-09-21T00:20:54.759277+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:51.906058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.762137+0000","last_change":"2024-09-21T00:19:25.843696+0000","last_active":"2024-09-21T00:20:54.762137+0000","last_peered":"2024-09-21T00:20:54.762137+0000","last_clean":"2024-09-21T00:20:54.762137+0000","last_became_active":"2024-09-21T00:19:25.843287+0000","last_became_peered":"2024-09-21T00:19:25.843287+0000","last_unstale":"2024-09-21T00:20:54.762137+0000","last_undegraded":"2024-09-21T00:20:54.762137+0000","last_fullsized":"2024-09-21T00:20:54.762137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:21.024555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767946+0000","last_change":"2024-09-21T00:18:38.671820+0000","last_active":"2024-09-21T00:20:54.767946+0000","last_peered":"2024-09-21T00:20:54.767946+0000","last_clean":"2024-09-21T00:20:54.767946+0000","last_became_active":"2024-09-21T00:18:38.667107+0000","last_became_peered":"2024-09-21T00:18:38.667107+0000","last_unstale":"2024-09-21T00:20:54.767946+0000","last_undegraded":"2024-09-21T00:20:54.767946+0000","last_fullsized":"2024-09-21T00:20:54.767946+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:58.099750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.759553+0000","last_change":"2024-09-21T00:18:38.671271+0000","last_active":"2024-09-21T00:20:54.759553+0000","last_peered":"2024-09-21T00:20:54.759553+0000","last_clean":"2024-09-21T00:20:54.759553+0000","last_became_active":"2024-09-21T00:18:38.670826+0000","last_became_peered":"2024-09-21T00:18:38.670826+0000","last_unstale":"2024-09-21T00:20:54.759553+0000","last_undegraded":"2024-09-21T00:20:54.759553+0000","last_fullsized":"2024-09-21T00:20:54.759553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:57:01.723120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768399+0000","last_change":"2024-09-21T00:18:38.662851+0000","last_active":"2024-09-21T00:20:54.768399+0000","last_peered":"2024-09-21T00:20:54.768399+0000","last_clean":"2024-09-21T00:20:54.768399+0000","last_became_active":"2024-09-21T00:18:38.661259+0000","last_became_peered":"2024-09-21T00:18:38.661259+0000","last_unstale":"2024-09-21T00:20:54.768399+0000","last_undegraded":"2024-09-21T00:20:54.768399+0000","last_fullsized":"2024-09-21T00:20:54.768399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:10.508980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":862796,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1103,"num_read_kb":758,"num_write":395,"num_write_kb":2220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3864,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2826240,"data_stored":2588388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":291,"ondisk_log_size":291,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678067,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":535425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874296,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37492,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707660,"statfs":{"total":95995035648,"available":95956643840,"internally_reserved":0,"allocated":2138112,"data_stored":1588109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037820,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712548,"statfs":{"total":95995035648,"available":95961649152,"internally_reserved":0,"allocated":1454080,"data_stored":932951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234051,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36756,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708396,"statfs":{"total":95995035648,"available":95957397504,"internally_reserved":0,"allocated":1445888,"data_stored":928753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430279,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1131609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593805,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":530815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790037,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29092,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716060,"statfs":{"total":95995035648,"available":95965245440,"internally_reserved":0,"allocated":2056192,"data_stored":1520973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953562,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1134592,"data_stored":607709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149790,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28880,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716272,"statfs":{"total":95995035648,"available":95965462528,"internally_reserved":0,"allocated":1839104,"data_stored":1318001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378725,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":532025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542250,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":602387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705775,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":532203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":806912,"data_stored":787276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":395784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":512000,"data_stored":467016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:21:44.929 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T00:21:44.930 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T00:21:45.314 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:45.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:45 smithi027 ceph-mon[30745]: from='client.25424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:21:45.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:45 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:21:45.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:45 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:21:45.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:45 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:21:45.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:45 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:21:45.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:45 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:21:45.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:45 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:21:45.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:45 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:21:45.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:45 smithi089 ceph-mon[34481]: from='client.25424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:21:45.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:45 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:21:45.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:45 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:21:45.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:45 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:21:45.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:45 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:21:45.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:45 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:21:45.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:45 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:21:45.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:45 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:21:45.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:45 smithi098 ceph-mon[34725]: from='client.25424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:21:45.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:45 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:21:45.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:45 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:21:45.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:45 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:21:45.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:45 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:21:45.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:45 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:21:45.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:45 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:21:45.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:45 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:21:45.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.963+0000 7f67f6800640 1 -- 172.21.15.27:0/1742586415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67f010b4f0 msgr2=0x7f67f010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:45.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.963+0000 7f67f6800640 1 --2- 172.21.15.27:0/1742586415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67f010b4f0 0x7f67f010b900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009900 tx=0x7f67dc02a3f0 comp rx=0 tx=0).stop 2024-09-21T00:21:45.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.963+0000 7f67f6800640 1 -- 172.21.15.27:0/1742586415 shutdown_connections 2024-09-21T00:21:45.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.963+0000 7f67f6800640 1 --2- 172.21.15.27:0/1742586415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67f010be40 0x7f67f0112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:45.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.963+0000 7f67f6800640 1 --2- 172.21.15.27:0/1742586415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67f010b4f0 0x7f67f010b900 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:45.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.963+0000 7f67f6800640 1 --2- 172.21.15.27:0/1742586415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f0105bb0 0x7f67f0105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:45.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.963+0000 7f67f6800640 1 -- 172.21.15.27:0/1742586415 >> 172.21.15.27:0/1742586415 conn(0x7f67f0101680 msgr2=0x7f67f0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:45.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.963+0000 7f67f6800640 1 -- 172.21.15.27:0/1742586415 shutdown_connections 2024-09-21T00:21:45.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.963+0000 7f67f6800640 1 -- 172.21.15.27:0/1742586415 wait complete. 2024-09-21T00:21:45.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.964+0000 7f67f6800640 1 Processor -- start 2024-09-21T00:21:45.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.964+0000 7f67f6800640 1 -- start start 2024-09-21T00:21:45.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.965+0000 7f67f6800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67f0105bb0 0x7f67f019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:45.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.965+0000 7f67f6800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f010b4f0 0x7f67f019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:45.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.965+0000 7f67f6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67f010be40 0x7f67f01a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:45.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.965+0000 7f67f6800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f67f0115b50 con 0x7f67f010be40 2024-09-21T00:21:45.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.965+0000 7f67f6800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01159d0 con 0x7f67f010b4f0 2024-09-21T00:21:45.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.965+0000 7f67f6800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f67f0115cd0 con 0x7f67f0105bb0 2024-09-21T00:21:45.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.965+0000 7f67f5400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67f0105bb0 0x7f67f019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:45.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.965+0000 7f67f4a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f010b4f0 0x7f67f019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:45.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.965+0000 7f67f5400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67f0105bb0 0x7f67f019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:45332/0 (socket says 172.21.15.27:45332) 2024-09-21T00:21:45.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.965+0000 7f67f5400640 1 -- 172.21.15.27:0/1724767728 learned_addr learned my addr 172.21.15.27:0/1724767728 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:45.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.965+0000 7f67f5e00640 1 --2- 172.21.15.27:0/1724767728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67f010be40 0x7f67f01a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:45.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.966+0000 7f67f5400640 1 -- 172.21.15.27:0/1724767728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f010b4f0 msgr2=0x7f67f019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:45.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.966+0000 7f67f5400640 1 --2- 172.21.15.27:0/1724767728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f010b4f0 0x7f67f019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:45.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.966+0000 7f67f5400640 1 -- 172.21.15.27:0/1724767728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67f010be40 msgr2=0x7f67f01a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:45.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.966+0000 7f67f5400640 1 --2- 172.21.15.27:0/1724767728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67f010be40 0x7f67f01a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:45.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.966+0000 7f67f5400640 1 -- 172.21.15.27:0/1724767728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f01a2470 con 0x7f67f0105bb0 2024-09-21T00:21:45.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.966+0000 7f67f4a00640 1 --2- 172.21.15.27:0/1724767728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f010b4f0 0x7f67f019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:45.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.966+0000 7f67f5400640 1 --2- 172.21.15.27:0/1724767728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67f0105bb0 0x7f67f019c500 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f67e400ea60 tx=0x7f67e400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:45.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.966+0000 7f67ee000640 1 -- 172.21.15.27:0/1724767728 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e400cde0 con 0x7f67f0105bb0 2024-09-21T00:21:45.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.966+0000 7f67ee000640 1 -- 172.21.15.27:0/1724767728 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e40035f0 con 0x7f67f0105bb0 2024-09-21T00:21:45.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.966+0000 7f67f6800640 1 -- 172.21.15.27:0/1724767728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f01a2760 con 0x7f67f0105bb0 2024-09-21T00:21:45.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.966+0000 7f67ee000640 1 -- 172.21.15.27:0/1724767728 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4010690 con 0x7f67f0105bb0 2024-09-21T00:21:45.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.966+0000 7f67f6800640 1 -- 172.21.15.27:0/1724767728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f01a8ec0 con 0x7f67f0105bb0 2024-09-21T00:21:45.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.967+0000 7f67f6800640 1 -- 172.21.15.27:0/1724767728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f0105fc0 con 0x7f67f0105bb0 2024-09-21T00:21:45.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.970+0000 7f67ee000640 1 -- 172.21.15.27:0/1724767728 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f67e40037a0 con 0x7f67f0105bb0 2024-09-21T00:21:45.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.972+0000 7f67ee000640 1 --2- 172.21.15.27:0/1724767728 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f67d80b65f0 0x7f67d80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:45.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.972+0000 7f67ee000640 1 -- 172.21.15.27:0/1724767728 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f67e40e7eb0 con 0x7f67f0105bb0 2024-09-21T00:21:45.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.972+0000 7f67f4a00640 1 --2- 172.21.15.27:0/1724767728 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f67d80b65f0 0x7f67d80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:45.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.973+0000 7f67f4a00640 1 --2- 172.21.15.27:0/1724767728 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f67d80b65f0 0x7f67d80b8b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f67dc02a9e0 tx=0x7f67dc005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:45.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:45.973+0000 7f67ee000640 1 -- 172.21.15.27:0/1724767728 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f67e4056710 con 0x7f67f0105bb0 2024-09-21T00:21:46.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:46.169+0000 7f67f6800640 1 -- 172.21.15.27:0/1724767728 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67f0063800 con 0x7f67d80b65f0 2024-09-21T00:21:46.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:46.176+0000 7f67ee000640 1 -- 172.21.15.27:0/1724767728 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340546 (secure 0 0 0) 0x7f67f0063800 con 0x7f67d80b65f0 2024-09-21T00:21:46.181 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:21:46.182 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T00:21:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:46.181+0000 7f67f6800640 1 -- 172.21.15.27:0/1724767728 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f67d80b65f0 msgr2=0x7f67d80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:46.181+0000 7f67f6800640 1 --2- 172.21.15.27:0/1724767728 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f67d80b65f0 0x7f67d80b8b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f67dc02a9e0 tx=0x7f67dc005a20 comp rx=0 tx=0).stop 2024-09-21T00:21:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:46.181+0000 7f67f6800640 1 -- 172.21.15.27:0/1724767728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67f0105bb0 msgr2=0x7f67f019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:46.181+0000 7f67f6800640 1 --2- 172.21.15.27:0/1724767728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67f0105bb0 0x7f67f019c500 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f67e400ea60 tx=0x7f67e400ef20 comp rx=0 tx=0).stop 2024-09-21T00:21:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:46.181+0000 7f67f6800640 1 -- 172.21.15.27:0/1724767728 shutdown_connections 2024-09-21T00:21:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:46.181+0000 7f67f6800640 1 --2- 172.21.15.27:0/1724767728 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f67d80b65f0 0x7f67d80b8b00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:46.181+0000 7f67f6800640 1 --2- 172.21.15.27:0/1724767728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67f010be40 0x7f67f01a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:46.181+0000 7f67f6800640 1 --2- 172.21.15.27:0/1724767728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f010b4f0 0x7f67f019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:46.181+0000 7f67f6800640 1 --2- 172.21.15.27:0/1724767728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67f0105bb0 0x7f67f019c500 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:46.181+0000 7f67f6800640 1 -- 172.21.15.27:0/1724767728 >> 172.21.15.27:0/1724767728 conn(0x7f67f0101680 msgr2=0x7f67f010ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:46.181+0000 7f67f6800640 1 -- 172.21.15.27:0/1724767728 shutdown_connections 2024-09-21T00:21:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:46.181+0000 7f67f6800640 1 -- 172.21.15.27:0/1724767728 wait complete. 2024-09-21T00:21:46.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:46 smithi089 ceph-mon[34481]: pgmap v388: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 52 op/s 2024-09-21T00:21:46.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:46 smithi098 ceph-mon[34725]: pgmap v388: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 52 op/s 2024-09-21T00:21:46.534 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:46 smithi027 ceph-mon[30745]: pgmap v388: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 52 op/s 2024-09-21T00:21:46.965 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-21T00:21:44.380372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1453398,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1199,"num_read_kb":841,"num_write":524,"num_write_kb":4372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2955704,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":887,"ondisk_log_size":887,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382696,"kb_used_data":16628,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124559128,"statfs":{"total":1151940427776,"available":1151548547072,"internally_reserved":0,"allocated":17027072,"data_stored":10760960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":792746,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":400,"num_read_kb":341,"num_write":229,"num_write_kb":2097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003486"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453054+0000","last_change":"2024-09-21T00:18:35.627618+0000","last_active":"2024-09-21T00:20:26.453054+0000","last_peered":"2024-09-21T00:20:26.453054+0000","last_clean":"2024-09-21T00:20:26.453054+0000","last_became_active":"2024-09-21T00:18:35.626475+0000","last_became_peered":"2024-09-21T00:18:35.626475+0000","last_unstale":"2024-09-21T00:20:26.453054+0000","last_undegraded":"2024-09-21T00:20:26.453054+0000","last_fullsized":"2024-09-21T00:20:26.453054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:50.379225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754687+0000","last_change":"2024-09-21T00:18:38.665891+0000","last_active":"2024-09-21T00:20:54.754687+0000","last_peered":"2024-09-21T00:20:54.754687+0000","last_clean":"2024-09-21T00:20:54.754687+0000","last_became_active":"2024-09-21T00:18:38.665472+0000","last_became_peered":"2024-09-21T00:18:38.665472+0000","last_unstale":"2024-09-21T00:20:54.754687+0000","last_undegraded":"2024-09-21T00:20:54.754687+0000","last_fullsized":"2024-09-21T00:20:54.754687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:34:32.490828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451446+0000","last_change":"2024-09-21T00:18:48.730206+0000","last_active":"2024-09-21T00:20:26.451446+0000","last_peered":"2024-09-21T00:20:26.451446+0000","last_clean":"2024-09-21T00:20:26.451446+0000","last_became_active":"2024-09-21T00:18:48.729729+0000","last_became_peered":"2024-09-21T00:18:48.729729+0000","last_unstale":"2024-09-21T00:20:26.451446+0000","last_undegraded":"2024-09-21T00:20:26.451446+0000","last_fullsized":"2024-09-21T00:20:26.451446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:08:17.019877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.575433+0000","last_change":"2024-09-21T00:18:35.768554+0000","last_active":"2024-09-21T00:20:26.575433+0000","last_peered":"2024-09-21T00:20:26.575433+0000","last_clean":"2024-09-21T00:20:26.575433+0000","last_became_active":"2024-09-21T00:18:35.767642+0000","last_became_peered":"2024-09-21T00:18:35.767642+0000","last_unstale":"2024-09-21T00:20:26.575433+0000","last_undegraded":"2024-09-21T00:20:26.575433+0000","last_fullsized":"2024-09-21T00:20:26.575433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:10.442974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768099+0000","last_change":"2024-09-21T00:18:38.677997+0000","last_active":"2024-09-21T00:20:54.768099+0000","last_peered":"2024-09-21T00:20:54.768099+0000","last_clean":"2024-09-21T00:20:54.768099+0000","last_became_active":"2024-09-21T00:18:38.676698+0000","last_became_peered":"2024-09-21T00:18:38.676698+0000","last_unstale":"2024-09-21T00:20:54.768099+0000","last_undegraded":"2024-09-21T00:20:54.768099+0000","last_fullsized":"2024-09-21T00:20:54.768099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:11.662765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451676+0000","last_change":"2024-09-21T00:18:48.739563+0000","last_active":"2024-09-21T00:20:26.451676+0000","last_peered":"2024-09-21T00:20:26.451676+0000","last_clean":"2024-09-21T00:20:26.451676+0000","last_became_active":"2024-09-21T00:18:48.739092+0000","last_became_peered":"2024-09-21T00:18:48.739092+0000","last_unstale":"2024-09-21T00:20:26.451676+0000","last_undegraded":"2024-09-21T00:20:26.451676+0000","last_fullsized":"2024-09-21T00:20:26.451676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:44.185556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510989+0000","last_change":"2024-09-21T00:18:35.626742+0000","last_active":"2024-09-21T00:20:26.510989+0000","last_peered":"2024-09-21T00:20:26.510989+0000","last_clean":"2024-09-21T00:20:26.510989+0000","last_became_active":"2024-09-21T00:18:35.626359+0000","last_became_peered":"2024-09-21T00:18:35.626359+0000","last_unstale":"2024-09-21T00:20:26.510989+0000","last_undegraded":"2024-09-21T00:20:26.510989+0000","last_fullsized":"2024-09-21T00:20:26.510989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:38.860184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.761254+0000","last_change":"2024-09-21T00:18:38.666003+0000","last_active":"2024-09-21T00:20:54.761254+0000","last_peered":"2024-09-21T00:20:54.761254+0000","last_clean":"2024-09-21T00:20:54.761254+0000","last_became_active":"2024-09-21T00:18:38.665425+0000","last_became_peered":"2024-09-21T00:18:38.665425+0000","last_unstale":"2024-09-21T00:20:54.761254+0000","last_undegraded":"2024-09-21T00:20:54.761254+0000","last_fullsized":"2024-09-21T00:20:54.761254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:14.121864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.577303+0000","last_change":"2024-09-21T00:20:26.576748+0000","last_active":"2024-09-21T00:20:26.577303+0000","last_peered":"2024-09-21T00:20:26.577303+0000","last_clean":"2024-09-21T00:20:26.577303+0000","last_became_active":"2024-09-21T00:20:26.576186+0000","last_became_peered":"2024-09-21T00:20:26.576186+0000","last_unstale":"2024-09-21T00:20:26.577303+0000","last_undegraded":"2024-09-21T00:20:26.577303+0000","last_fullsized":"2024-09-21T00:20:26.577303+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:20.283717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510680+0000","last_change":"2024-09-21T00:18:35.628091+0000","last_active":"2024-09-21T00:20:26.510680+0000","last_peered":"2024-09-21T00:20:26.510680+0000","last_clean":"2024-09-21T00:20:26.510680+0000","last_became_active":"2024-09-21T00:18:35.627654+0000","last_became_peered":"2024-09-21T00:18:35.627654+0000","last_unstale":"2024-09-21T00:20:26.510680+0000","last_undegraded":"2024-09-21T00:20:26.510680+0000","last_fullsized":"2024-09-21T00:20:26.510680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:05.229036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.765336+0000","last_change":"2024-09-21T00:19:25.844116+0000","last_active":"2024-09-21T00:20:54.765336+0000","last_peered":"2024-09-21T00:20:54.765336+0000","last_clean":"2024-09-21T00:20:54.765336+0000","last_became_active":"2024-09-21T00:19:25.842745+0000","last_became_peered":"2024-09-21T00:19:25.842745+0000","last_unstale":"2024-09-21T00:20:54.765336+0000","last_undegraded":"2024-09-21T00:20:54.765336+0000","last_fullsized":"2024-09-21T00:20:54.765336+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:42.651251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451830+0000","last_change":"2024-09-21T00:18:48.740106+0000","last_active":"2024-09-21T00:20:26.451830+0000","last_peered":"2024-09-21T00:20:26.451830+0000","last_clean":"2024-09-21T00:20:26.451830+0000","last_became_active":"2024-09-21T00:18:48.739299+0000","last_became_peered":"2024-09-21T00:18:48.739299+0000","last_unstale":"2024-09-21T00:20:26.451830+0000","last_undegraded":"2024-09-21T00:20:26.451830+0000","last_fullsized":"2024-09-21T00:20:26.451830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:59.058333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.766986+0000","last_change":"2024-09-21T00:18:38.667331+0000","last_active":"2024-09-21T00:20:54.766986+0000","last_peered":"2024-09-21T00:20:54.766986+0000","last_clean":"2024-09-21T00:20:54.766986+0000","last_became_active":"2024-09-21T00:18:38.666960+0000","last_became_peered":"2024-09-21T00:18:38.666960+0000","last_unstale":"2024-09-21T00:20:54.766986+0000","last_undegraded":"2024-09-21T00:20:54.766986+0000","last_fullsized":"2024-09-21T00:20:54.766986+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:57.569460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510354+0000","last_change":"2024-09-21T00:18:35.625570+0000","last_active":"2024-09-21T00:20:26.510354+0000","last_peered":"2024-09-21T00:20:26.510354+0000","last_clean":"2024-09-21T00:20:26.510354+0000","last_became_active":"2024-09-21T00:18:35.625170+0000","last_became_peered":"2024-09-21T00:18:35.625170+0000","last_unstale":"2024-09-21T00:20:26.510354+0000","last_undegraded":"2024-09-21T00:20:26.510354+0000","last_fullsized":"2024-09-21T00:20:26.510354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:58.614258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452085+0000","last_change":"2024-09-21T00:18:48.727390+0000","last_active":"2024-09-21T00:20:26.452085+0000","last_peered":"2024-09-21T00:20:26.452085+0000","last_clean":"2024-09-21T00:20:26.452085+0000","last_became_active":"2024-09-21T00:18:48.726864+0000","last_became_peered":"2024-09-21T00:18:48.726864+0000","last_unstale":"2024-09-21T00:20:26.452085+0000","last_undegraded":"2024-09-21T00:20:26.452085+0000","last_fullsized":"2024-09-21T00:20:26.452085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:10.917069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767288+0000","last_change":"2024-09-21T00:18:38.664072+0000","last_active":"2024-09-21T00:20:54.767288+0000","last_peered":"2024-09-21T00:20:54.767288+0000","last_clean":"2024-09-21T00:20:54.767288+0000","last_became_active":"2024-09-21T00:18:38.662562+0000","last_became_peered":"2024-09-21T00:18:38.662562+0000","last_unstale":"2024-09-21T00:20:54.767288+0000","last_undegraded":"2024-09-21T00:20:54.767288+0000","last_fullsized":"2024-09-21T00:20:54.767288+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.021174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453878+0000","last_change":"2024-09-21T00:19:25.843599+0000","last_active":"2024-09-21T00:20:26.453878+0000","last_peered":"2024-09-21T00:20:26.453878+0000","last_clean":"2024-09-21T00:20:26.453878+0000","last_became_active":"2024-09-21T00:19:25.843178+0000","last_became_peered":"2024-09-21T00:19:25.843178+0000","last_unstale":"2024-09-21T00:20:26.453878+0000","last_undegraded":"2024-09-21T00:20:26.453878+0000","last_fullsized":"2024-09-21T00:20:26.453878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:18.809857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452141+0000","last_change":"2024-09-21T00:18:48.729940+0000","last_active":"2024-09-21T00:20:26.452141+0000","last_peered":"2024-09-21T00:20:26.452141+0000","last_clean":"2024-09-21T00:20:26.452141+0000","last_became_active":"2024-09-21T00:18:48.729465+0000","last_became_peered":"2024-09-21T00:18:48.729465+0000","last_unstale":"2024-09-21T00:20:26.452141+0000","last_undegraded":"2024-09-21T00:20:26.452141+0000","last_fullsized":"2024-09-21T00:20:26.452141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:44.566386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768271+0000","last_change":"2024-09-21T00:18:38.662510+0000","last_active":"2024-09-21T00:20:54.768271+0000","last_peered":"2024-09-21T00:20:54.768271+0000","last_clean":"2024-09-21T00:20:54.768271+0000","last_became_active":"2024-09-21T00:18:38.659788+0000","last_became_peered":"2024-09-21T00:18:38.659788+0000","last_unstale":"2024-09-21T00:20:54.768271+0000","last_undegraded":"2024-09-21T00:20:54.768271+0000","last_fullsized":"2024-09-21T00:20:54.768271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:14.532084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.575037+0000","last_change":"2024-09-21T00:18:35.622949+0000","last_active":"2024-09-21T00:20:26.575037+0000","last_peered":"2024-09-21T00:20:26.575037+0000","last_clean":"2024-09-21T00:20:26.575037+0000","last_became_active":"2024-09-21T00:18:35.621787+0000","last_became_peered":"2024-09-21T00:18:35.621787+0000","last_unstale":"2024-09-21T00:20:26.575037+0000","last_undegraded":"2024-09-21T00:20:26.575037+0000","last_fullsized":"2024-09-21T00:20:26.575037+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:39:15.060123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452739+0000","last_change":"2024-09-21T00:18:48.730711+0000","last_active":"2024-09-21T00:20:26.452739+0000","last_peered":"2024-09-21T00:20:26.452739+0000","last_clean":"2024-09-21T00:20:26.452739+0000","last_became_active":"2024-09-21T00:18:48.730287+0000","last_became_peered":"2024-09-21T00:18:48.730287+0000","last_unstale":"2024-09-21T00:20:26.452739+0000","last_undegraded":"2024-09-21T00:20:26.452739+0000","last_fullsized":"2024-09-21T00:20:26.452739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:34.444747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.734412+0000","last_change":"2024-09-21T00:18:38.669381+0000","last_active":"2024-09-21T00:20:54.734412+0000","last_peered":"2024-09-21T00:20:54.734412+0000","last_clean":"2024-09-21T00:20:54.734412+0000","last_became_active":"2024-09-21T00:18:38.668757+0000","last_became_peered":"2024-09-21T00:18:38.668757+0000","last_unstale":"2024-09-21T00:20:54.734412+0000","last_undegraded":"2024-09-21T00:20:54.734412+0000","last_fullsized":"2024-09-21T00:20:54.734412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:52.475872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508594+0000","last_change":"2024-09-21T00:18:35.767700+0000","last_active":"2024-09-21T00:20:26.508594+0000","last_peered":"2024-09-21T00:20:26.508594+0000","last_clean":"2024-09-21T00:20:26.508594+0000","last_became_active":"2024-09-21T00:18:35.767369+0000","last_became_peered":"2024-09-21T00:18:35.767369+0000","last_unstale":"2024-09-21T00:20:26.508594+0000","last_undegraded":"2024-09-21T00:20:26.508594+0000","last_fullsized":"2024-09-21T00:20:26.508594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:44.955659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452286+0000","last_change":"2024-09-21T00:18:48.737411+0000","last_active":"2024-09-21T00:20:26.452286+0000","last_peered":"2024-09-21T00:20:26.452286+0000","last_clean":"2024-09-21T00:20:26.452286+0000","last_became_active":"2024-09-21T00:18:48.736715+0000","last_became_peered":"2024-09-21T00:18:48.736715+0000","last_unstale":"2024-09-21T00:20:26.452286+0000","last_undegraded":"2024-09-21T00:20:26.452286+0000","last_fullsized":"2024-09-21T00:20:26.452286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:01.045318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768197+0000","last_change":"2024-09-21T00:19:25.842355+0000","last_active":"2024-09-21T00:20:54.768197+0000","last_peered":"2024-09-21T00:20:54.768197+0000","last_clean":"2024-09-21T00:20:54.768197+0000","last_became_active":"2024-09-21T00:19:25.841961+0000","last_became_peered":"2024-09-21T00:19:25.841961+0000","last_unstale":"2024-09-21T00:20:54.768197+0000","last_undegraded":"2024-09-21T00:20:54.768197+0000","last_fullsized":"2024-09-21T00:20:54.768197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:30.490992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452475+0000","last_change":"2024-09-21T00:18:35.919003+0000","last_active":"2024-09-21T00:20:26.452475+0000","last_peered":"2024-09-21T00:20:26.452475+0000","last_clean":"2024-09-21T00:20:26.452475+0000","last_became_active":"2024-09-21T00:18:35.918593+0000","last_became_peered":"2024-09-21T00:18:35.918593+0000","last_unstale":"2024-09-21T00:20:26.452475+0000","last_undegraded":"2024-09-21T00:20:26.452475+0000","last_fullsized":"2024-09-21T00:20:26.452475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:35.982726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451542+0000","last_change":"2024-09-21T00:18:48.725906+0000","last_active":"2024-09-21T00:20:26.451542+0000","last_peered":"2024-09-21T00:20:26.451542+0000","last_clean":"2024-09-21T00:20:26.451542+0000","last_became_active":"2024-09-21T00:18:48.725450+0000","last_became_peered":"2024-09-21T00:18:48.725450+0000","last_unstale":"2024-09-21T00:20:26.451542+0000","last_undegraded":"2024-09-21T00:20:26.451542+0000","last_fullsized":"2024-09-21T00:20:26.451542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:35.163208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764430+0000","last_change":"2024-09-21T00:18:38.666442+0000","last_active":"2024-09-21T00:20:54.764430+0000","last_peered":"2024-09-21T00:20:54.764430+0000","last_clean":"2024-09-21T00:20:54.764430+0000","last_became_active":"2024-09-21T00:18:38.666083+0000","last_became_peered":"2024-09-21T00:18:38.666083+0000","last_unstale":"2024-09-21T00:20:54.764430+0000","last_undegraded":"2024-09-21T00:20:54.764430+0000","last_fullsized":"2024-09-21T00:20:54.764430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:39.986440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'96","reported_seq":230,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:38.377337+0000","last_change":"2024-09-21T00:18:35.624883+0000","last_active":"2024-09-21T00:21:38.377337+0000","last_peered":"2024-09-21T00:21:38.377337+0000","last_clean":"2024-09-21T00:21:38.377337+0000","last_became_active":"2024-09-21T00:18:35.624382+0000","last_became_peered":"2024-09-21T00:18:35.624382+0000","last_unstale":"2024-09-21T00:21:38.377337+0000","last_undegraded":"2024-09-21T00:21:38.377337+0000","last_fullsized":"2024-09-21T00:21:38.377337+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:23.475521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":176,"num_write_kb":1220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574454+0000","last_change":"2024-09-21T00:18:48.719850+0000","last_active":"2024-09-21T00:20:26.574454+0000","last_peered":"2024-09-21T00:20:26.574454+0000","last_clean":"2024-09-21T00:20:26.574454+0000","last_became_active":"2024-09-21T00:18:48.719414+0000","last_became_peered":"2024-09-21T00:18:48.719414+0000","last_unstale":"2024-09-21T00:20:26.574454+0000","last_undegraded":"2024-09-21T00:20:26.574454+0000","last_fullsized":"2024-09-21T00:20:26.574454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:05.978977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.755833+0000","last_change":"2024-09-21T00:18:38.671920+0000","last_active":"2024-09-21T00:20:54.755833+0000","last_peered":"2024-09-21T00:20:54.755833+0000","last_clean":"2024-09-21T00:20:54.755833+0000","last_became_active":"2024-09-21T00:18:38.671592+0000","last_became_peered":"2024-09-21T00:18:38.671592+0000","last_unstale":"2024-09-21T00:20:54.755833+0000","last_undegraded":"2024-09-21T00:20:54.755833+0000","last_fullsized":"2024-09-21T00:20:54.755833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:40:48.257086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.511127+0000","last_change":"2024-09-21T00:18:35.769197+0000","last_active":"2024-09-21T00:20:26.511127+0000","last_peered":"2024-09-21T00:20:26.511127+0000","last_clean":"2024-09-21T00:20:26.511127+0000","last_became_active":"2024-09-21T00:18:35.768740+0000","last_became_peered":"2024-09-21T00:18:35.768740+0000","last_unstale":"2024-09-21T00:20:26.511127+0000","last_undegraded":"2024-09-21T00:20:26.511127+0000","last_fullsized":"2024-09-21T00:20:26.511127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:32.253006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454104+0000","last_change":"2024-09-21T00:18:48.740189+0000","last_active":"2024-09-21T00:20:26.454104+0000","last_peered":"2024-09-21T00:20:26.454104+0000","last_clean":"2024-09-21T00:20:26.454104+0000","last_became_active":"2024-09-21T00:18:48.739700+0000","last_became_peered":"2024-09-21T00:18:48.739700+0000","last_unstale":"2024-09-21T00:20:26.454104+0000","last_undegraded":"2024-09-21T00:20:26.454104+0000","last_fullsized":"2024-09-21T00:20:26.454104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:06.312160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764470+0000","last_change":"2024-09-21T00:18:38.665635+0000","last_active":"2024-09-21T00:20:54.764470+0000","last_peered":"2024-09-21T00:20:54.764470+0000","last_clean":"2024-09-21T00:20:54.764470+0000","last_became_active":"2024-09-21T00:18:38.665236+0000","last_became_peered":"2024-09-21T00:18:38.665236+0000","last_unstale":"2024-09-21T00:20:54.764470+0000","last_undegraded":"2024-09-21T00:20:54.764470+0000","last_fullsized":"2024-09-21T00:20:54.764470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:52.186480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452058+0000","last_change":"2024-09-21T00:18:35.771465+0000","last_active":"2024-09-21T00:20:26.452058+0000","last_peered":"2024-09-21T00:20:26.452058+0000","last_clean":"2024-09-21T00:20:26.452058+0000","last_became_active":"2024-09-21T00:18:35.770550+0000","last_became_peered":"2024-09-21T00:18:35.770550+0000","last_unstale":"2024-09-21T00:20:26.452058+0000","last_undegraded":"2024-09-21T00:20:26.452058+0000","last_fullsized":"2024-09-21T00:20:26.452058+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:43.042563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510077+0000","last_change":"2024-09-21T00:18:48.736334+0000","last_active":"2024-09-21T00:20:26.510077+0000","last_peered":"2024-09-21T00:20:26.510077+0000","last_clean":"2024-09-21T00:20:26.510077+0000","last_became_active":"2024-09-21T00:18:48.735784+0000","last_became_peered":"2024-09-21T00:18:48.735784+0000","last_unstale":"2024-09-21T00:20:26.510077+0000","last_undegraded":"2024-09-21T00:20:26.510077+0000","last_fullsized":"2024-09-21T00:20:26.510077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:29.637335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767464+0000","last_change":"2024-09-21T00:18:38.674181+0000","last_active":"2024-09-21T00:20:54.767464+0000","last_peered":"2024-09-21T00:20:54.767464+0000","last_clean":"2024-09-21T00:20:54.767464+0000","last_became_active":"2024-09-21T00:18:38.673784+0000","last_became_peered":"2024-09-21T00:18:38.673784+0000","last_unstale":"2024-09-21T00:20:54.767464+0000","last_undegraded":"2024-09-21T00:20:54.767464+0000","last_fullsized":"2024-09-21T00:20:54.767464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:33.471328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510165+0000","last_change":"2024-09-21T00:18:35.628149+0000","last_active":"2024-09-21T00:20:26.510165+0000","last_peered":"2024-09-21T00:20:26.510165+0000","last_clean":"2024-09-21T00:20:26.510165+0000","last_became_active":"2024-09-21T00:18:35.627674+0000","last_became_peered":"2024-09-21T00:18:35.627674+0000","last_unstale":"2024-09-21T00:20:26.510165+0000","last_undegraded":"2024-09-21T00:20:26.510165+0000","last_fullsized":"2024-09-21T00:20:26.510165+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:28:47.244288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508884+0000","last_change":"2024-09-21T00:18:48.740283+0000","last_active":"2024-09-21T00:20:26.508884+0000","last_peered":"2024-09-21T00:20:26.508884+0000","last_clean":"2024-09-21T00:20:26.508884+0000","last_became_active":"2024-09-21T00:18:48.739026+0000","last_became_peered":"2024-09-21T00:18:48.739026+0000","last_unstale":"2024-09-21T00:20:26.508884+0000","last_undegraded":"2024-09-21T00:20:26.508884+0000","last_fullsized":"2024-09-21T00:20:26.508884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:50.898116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.759757+0000","last_change":"2024-09-21T00:18:38.667206+0000","last_active":"2024-09-21T00:20:54.759757+0000","last_peered":"2024-09-21T00:20:54.759757+0000","last_clean":"2024-09-21T00:20:54.759757+0000","last_became_active":"2024-09-21T00:18:38.666770+0000","last_became_peered":"2024-09-21T00:18:38.666770+0000","last_unstale":"2024-09-21T00:20:54.759757+0000","last_undegraded":"2024-09-21T00:20:54.759757+0000","last_fullsized":"2024-09-21T00:20:54.759757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:56.332071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":176,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:41.119141+0000","last_change":"2024-09-21T00:18:35.768012+0000","last_active":"2024-09-21T00:21:41.119141+0000","last_peered":"2024-09-21T00:21:41.119141+0000","last_clean":"2024-09-21T00:21:41.119141+0000","last_became_active":"2024-09-21T00:18:35.767438+0000","last_became_peered":"2024-09-21T00:18:35.767438+0000","last_unstale":"2024-09-21T00:21:41.119141+0000","last_undegraded":"2024-09-21T00:21:41.119141+0000","last_fullsized":"2024-09-21T00:21:41.119141+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:35:40.606871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508643+0000","last_change":"2024-09-21T00:18:48.738228+0000","last_active":"2024-09-21T00:20:26.508643+0000","last_peered":"2024-09-21T00:20:26.508643+0000","last_clean":"2024-09-21T00:20:26.508643+0000","last_became_active":"2024-09-21T00:18:48.737786+0000","last_became_peered":"2024-09-21T00:18:48.737786+0000","last_unstale":"2024-09-21T00:20:26.508643+0000","last_undegraded":"2024-09-21T00:20:26.508643+0000","last_fullsized":"2024-09-21T00:20:26.508643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:52.329677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754320+0000","last_change":"2024-09-21T00:18:38.678126+0000","last_active":"2024-09-21T00:20:54.754320+0000","last_peered":"2024-09-21T00:20:54.754320+0000","last_clean":"2024-09-21T00:20:54.754320+0000","last_became_active":"2024-09-21T00:18:38.677757+0000","last_became_peered":"2024-09-21T00:18:38.677757+0000","last_unstale":"2024-09-21T00:20:54.754320+0000","last_undegraded":"2024-09-21T00:20:54.754320+0000","last_fullsized":"2024-09-21T00:20:54.754320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:26.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508836+0000","last_change":"2024-09-21T00:18:35.625955+0000","last_active":"2024-09-21T00:20:26.508836+0000","last_peered":"2024-09-21T00:20:26.508836+0000","last_clean":"2024-09-21T00:20:26.508836+0000","last_became_active":"2024-09-21T00:18:35.625548+0000","last_became_peered":"2024-09-21T00:18:35.625548+0000","last_unstale":"2024-09-21T00:20:26.508836+0000","last_undegraded":"2024-09-21T00:20:26.508836+0000","last_fullsized":"2024-09-21T00:20:26.508836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:04.163421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509821+0000","last_change":"2024-09-21T00:18:48.740765+0000","last_active":"2024-09-21T00:20:26.509821+0000","last_peered":"2024-09-21T00:20:26.509821+0000","last_clean":"2024-09-21T00:20:26.509821+0000","last_became_active":"2024-09-21T00:18:48.740081+0000","last_became_peered":"2024-09-21T00:18:48.740081+0000","last_unstale":"2024-09-21T00:20:26.509821+0000","last_undegraded":"2024-09-21T00:20:26.509821+0000","last_fullsized":"2024-09-21T00:20:26.509821+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:38.251475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.755958+0000","last_change":"2024-09-21T00:18:38.667581+0000","last_active":"2024-09-21T00:20:54.755958+0000","last_peered":"2024-09-21T00:20:54.755958+0000","last_clean":"2024-09-21T00:20:54.755958+0000","last_became_active":"2024-09-21T00:18:38.665959+0000","last_became_peered":"2024-09-21T00:18:38.665959+0000","last_unstale":"2024-09-21T00:20:54.755958+0000","last_undegraded":"2024-09-21T00:20:54.755958+0000","last_fullsized":"2024-09-21T00:20:54.755958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:52.321654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452596+0000","last_change":"2024-09-21T00:18:35.624291+0000","last_active":"2024-09-21T00:20:26.452596+0000","last_peered":"2024-09-21T00:20:26.452596+0000","last_clean":"2024-09-21T00:20:26.452596+0000","last_became_active":"2024-09-21T00:18:35.623297+0000","last_became_peered":"2024-09-21T00:18:35.623297+0000","last_unstale":"2024-09-21T00:20:26.452596+0000","last_undegraded":"2024-09-21T00:20:26.452596+0000","last_fullsized":"2024-09-21T00:20:26.452596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:44.824409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509191+0000","last_change":"2024-09-21T00:17:21.233627+0000","last_active":"2024-09-21T00:20:26.509191+0000","last_peered":"2024-09-21T00:20:26.509191+0000","last_clean":"2024-09-21T00:20:26.509191+0000","last_became_active":"2024-09-21T00:17:21.227897+0000","last_became_peered":"2024-09-21T00:17:21.227897+0000","last_unstale":"2024-09-21T00:20:26.509191+0000","last_undegraded":"2024-09-21T00:20:26.509191+0000","last_fullsized":"2024-09-21T00:20:26.509191+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:13:33.120691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:13:33.120691+0000","last_clean_scrub_stamp":"2024-09-21T00:13:33.120691+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:20.198660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509421+0000","last_change":"2024-09-21T00:18:48.730107+0000","last_active":"2024-09-21T00:20:26.509421+0000","last_peered":"2024-09-21T00:20:26.509421+0000","last_clean":"2024-09-21T00:20:26.509421+0000","last_became_active":"2024-09-21T00:18:48.729463+0000","last_became_peered":"2024-09-21T00:18:48.729463+0000","last_unstale":"2024-09-21T00:20:26.509421+0000","last_undegraded":"2024-09-21T00:20:26.509421+0000","last_fullsized":"2024-09-21T00:20:26.509421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:55.352870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764244+0000","last_change":"2024-09-21T00:18:38.671244+0000","last_active":"2024-09-21T00:20:54.764244+0000","last_peered":"2024-09-21T00:20:54.764244+0000","last_clean":"2024-09-21T00:20:54.764244+0000","last_became_active":"2024-09-21T00:18:38.670630+0000","last_became_peered":"2024-09-21T00:18:38.670630+0000","last_unstale":"2024-09-21T00:20:54.764244+0000","last_undegraded":"2024-09-21T00:20:54.764244+0000","last_fullsized":"2024-09-21T00:20:54.764244+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:23.695660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451865+0000","last_change":"2024-09-21T00:19:25.843900+0000","last_active":"2024-09-21T00:20:26.451865+0000","last_peered":"2024-09-21T00:20:26.451865+0000","last_clean":"2024-09-21T00:20:26.451865+0000","last_became_active":"2024-09-21T00:19:25.843438+0000","last_became_peered":"2024-09-21T00:19:25.843438+0000","last_unstale":"2024-09-21T00:20:26.451865+0000","last_undegraded":"2024-09-21T00:20:26.451865+0000","last_fullsized":"2024-09-21T00:20:26.451865+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:17.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509404+0000","last_change":"2024-09-21T00:18:48.739640+0000","last_active":"2024-09-21T00:20:26.509404+0000","last_peered":"2024-09-21T00:20:26.509404+0000","last_clean":"2024-09-21T00:20:26.509404+0000","last_became_active":"2024-09-21T00:18:48.739118+0000","last_became_peered":"2024-09-21T00:18:48.739118+0000","last_unstale":"2024-09-21T00:20:26.509404+0000","last_undegraded":"2024-09-21T00:20:26.509404+0000","last_fullsized":"2024-09-21T00:20:26.509404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:03.093561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.766430+0000","last_change":"2024-09-21T00:18:38.673020+0000","last_active":"2024-09-21T00:20:54.766430+0000","last_peered":"2024-09-21T00:20:54.766430+0000","last_clean":"2024-09-21T00:20:54.766430+0000","last_became_active":"2024-09-21T00:18:38.672391+0000","last_became_peered":"2024-09-21T00:18:38.672391+0000","last_unstale":"2024-09-21T00:20:54.766430+0000","last_undegraded":"2024-09-21T00:20:54.766430+0000","last_fullsized":"2024-09-21T00:20:54.766430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:13.866576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'60","reported_seq":136,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:37.877664+0000","last_change":"2024-09-21T00:18:35.615844+0000","last_active":"2024-09-21T00:21:37.877664+0000","last_peered":"2024-09-21T00:21:37.877664+0000","last_clean":"2024-09-21T00:21:37.877664+0000","last_became_active":"2024-09-21T00:18:35.615385+0000","last_became_peered":"2024-09-21T00:18:35.615385+0000","last_unstale":"2024-09-21T00:21:37.877664+0000","last_undegraded":"2024-09-21T00:21:37.877664+0000","last_fullsized":"2024-09-21T00:21:37.877664+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:03.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.512987+0000","last_change":"2024-09-21T00:20:26.511995+0000","last_active":"2024-09-21T00:20:26.512987+0000","last_peered":"2024-09-21T00:20:26.512987+0000","last_clean":"2024-09-21T00:20:26.512987+0000","last_became_active":"2024-09-21T00:20:26.511607+0000","last_became_peered":"2024-09-21T00:20:26.511607+0000","last_unstale":"2024-09-21T00:20:26.512987+0000","last_undegraded":"2024-09-21T00:20:26.512987+0000","last_fullsized":"2024-09-21T00:20:26.512987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:38.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754513+0000","last_change":"2024-09-21T00:18:38.671280+0000","last_active":"2024-09-21T00:20:54.754513+0000","last_peered":"2024-09-21T00:20:54.754513+0000","last_clean":"2024-09-21T00:20:54.754513+0000","last_became_active":"2024-09-21T00:18:38.670770+0000","last_became_peered":"2024-09-21T00:18:38.670770+0000","last_unstale":"2024-09-21T00:20:54.754513+0000","last_undegraded":"2024-09-21T00:20:54.754513+0000","last_fullsized":"2024-09-21T00:20:54.754513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:06.809450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509590+0000","last_change":"2024-09-21T00:18:35.626071+0000","last_active":"2024-09-21T00:20:26.509590+0000","last_peered":"2024-09-21T00:20:26.509590+0000","last_clean":"2024-09-21T00:20:26.509590+0000","last_became_active":"2024-09-21T00:18:35.625649+0000","last_became_peered":"2024-09-21T00:18:35.625649+0000","last_unstale":"2024-09-21T00:20:26.509590+0000","last_undegraded":"2024-09-21T00:20:26.509590+0000","last_fullsized":"2024-09-21T00:20:26.509590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:16:27.516303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508631+0000","last_change":"2024-09-21T00:18:48.728451+0000","last_active":"2024-09-21T00:20:26.508631+0000","last_peered":"2024-09-21T00:20:26.508631+0000","last_clean":"2024-09-21T00:20:26.508631+0000","last_became_active":"2024-09-21T00:18:48.727840+0000","last_became_peered":"2024-09-21T00:18:48.727840+0000","last_unstale":"2024-09-21T00:20:26.508631+0000","last_undegraded":"2024-09-21T00:20:26.508631+0000","last_fullsized":"2024-09-21T00:20:26.508631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:13.457138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754460+0000","last_change":"2024-09-21T00:18:38.664090+0000","last_active":"2024-09-21T00:20:54.754460+0000","last_peered":"2024-09-21T00:20:54.754460+0000","last_clean":"2024-09-21T00:20:54.754460+0000","last_became_active":"2024-09-21T00:18:38.663728+0000","last_became_peered":"2024-09-21T00:18:38.663728+0000","last_unstale":"2024-09-21T00:20:54.754460+0000","last_undegraded":"2024-09-21T00:20:54.754460+0000","last_fullsized":"2024-09-21T00:20:54.754460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:46.367766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.454914+0000","last_change":"2024-09-21T00:19:25.847280+0000","last_active":"2024-09-21T00:20:26.454914+0000","last_peered":"2024-09-21T00:20:26.454914+0000","last_clean":"2024-09-21T00:20:26.454914+0000","last_became_active":"2024-09-21T00:19:25.840919+0000","last_became_peered":"2024-09-21T00:19:25.840919+0000","last_unstale":"2024-09-21T00:20:26.454914+0000","last_undegraded":"2024-09-21T00:20:26.454914+0000","last_fullsized":"2024-09-21T00:20:26.454914+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:15.411674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508811+0000","last_change":"2024-09-21T00:18:48.732427+0000","last_active":"2024-09-21T00:20:26.508811+0000","last_peered":"2024-09-21T00:20:26.508811+0000","last_clean":"2024-09-21T00:20:26.508811+0000","last_became_active":"2024-09-21T00:18:48.732078+0000","last_became_peered":"2024-09-21T00:18:48.732078+0000","last_unstale":"2024-09-21T00:20:26.508811+0000","last_undegraded":"2024-09-21T00:20:26.508811+0000","last_fullsized":"2024-09-21T00:20:26.508811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:35.055680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764481+0000","last_change":"2024-09-21T00:18:38.675022+0000","last_active":"2024-09-21T00:20:54.764481+0000","last_peered":"2024-09-21T00:20:54.764481+0000","last_clean":"2024-09-21T00:20:54.764481+0000","last_became_active":"2024-09-21T00:18:38.674650+0000","last_became_peered":"2024-09-21T00:18:38.674650+0000","last_unstale":"2024-09-21T00:20:54.764481+0000","last_undegraded":"2024-09-21T00:20:54.764481+0000","last_fullsized":"2024-09-21T00:20:54.764481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:05.191404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.510139+0000","last_change":"2024-09-21T00:18:35.769555+0000","last_active":"2024-09-21T00:20:26.510139+0000","last_peered":"2024-09-21T00:20:26.510139+0000","last_clean":"2024-09-21T00:20:26.510139+0000","last_became_active":"2024-09-21T00:18:35.769080+0000","last_became_peered":"2024-09-21T00:18:35.769080+0000","last_unstale":"2024-09-21T00:20:26.510139+0000","last_undegraded":"2024-09-21T00:20:26.510139+0000","last_fullsized":"2024-09-21T00:20:26.510139+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:19.306830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509529+0000","last_change":"2024-09-21T00:18:48.742957+0000","last_active":"2024-09-21T00:20:26.509529+0000","last_peered":"2024-09-21T00:20:26.509529+0000","last_clean":"2024-09-21T00:20:26.509529+0000","last_became_active":"2024-09-21T00:18:48.742511+0000","last_became_peered":"2024-09-21T00:18:48.742511+0000","last_unstale":"2024-09-21T00:20:26.509529+0000","last_undegraded":"2024-09-21T00:20:26.509529+0000","last_fullsized":"2024-09-21T00:20:26.509529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:15.231189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.761135+0000","last_change":"2024-09-21T00:18:38.666576+0000","last_active":"2024-09-21T00:20:54.761135+0000","last_peered":"2024-09-21T00:20:54.761135+0000","last_clean":"2024-09-21T00:20:54.761135+0000","last_became_active":"2024-09-21T00:18:38.665643+0000","last_became_peered":"2024-09-21T00:18:38.665643+0000","last_unstale":"2024-09-21T00:20:54.761135+0000","last_undegraded":"2024-09-21T00:20:54.761135+0000","last_fullsized":"2024-09-21T00:20:54.761135+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:35.397847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":176,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:39.116713+0000","last_change":"2024-09-21T00:18:35.623143+0000","last_active":"2024-09-21T00:21:39.116713+0000","last_peered":"2024-09-21T00:21:39.116713+0000","last_clean":"2024-09-21T00:21:39.116713+0000","last_became_active":"2024-09-21T00:18:35.622693+0000","last_became_peered":"2024-09-21T00:18:35.622693+0000","last_unstale":"2024-09-21T00:21:39.116713+0000","last_undegraded":"2024-09-21T00:21:39.116713+0000","last_fullsized":"2024-09-21T00:21:39.116713+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:01.331665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452872+0000","last_change":"2024-09-21T00:18:48.723877+0000","last_active":"2024-09-21T00:20:26.452872+0000","last_peered":"2024-09-21T00:20:26.452872+0000","last_clean":"2024-09-21T00:20:26.452872+0000","last_became_active":"2024-09-21T00:18:48.723360+0000","last_became_peered":"2024-09-21T00:18:48.723360+0000","last_unstale":"2024-09-21T00:20:26.452872+0000","last_undegraded":"2024-09-21T00:20:26.452872+0000","last_fullsized":"2024-09-21T00:20:26.452872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:07:20.442867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767171+0000","last_change":"2024-09-21T00:18:38.669643+0000","last_active":"2024-09-21T00:20:54.767171+0000","last_peered":"2024-09-21T00:20:54.767171+0000","last_clean":"2024-09-21T00:20:54.767171+0000","last_became_active":"2024-09-21T00:18:38.669245+0000","last_became_peered":"2024-09-21T00:18:38.669245+0000","last_unstale":"2024-09-21T00:20:54.767171+0000","last_undegraded":"2024-09-21T00:20:54.767171+0000","last_fullsized":"2024-09-21T00:20:54.767171+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:17.789291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509577+0000","last_change":"2024-09-21T00:18:35.623189+0000","last_active":"2024-09-21T00:20:26.509577+0000","last_peered":"2024-09-21T00:20:26.509577+0000","last_clean":"2024-09-21T00:20:26.509577+0000","last_became_active":"2024-09-21T00:18:35.622748+0000","last_became_peered":"2024-09-21T00:18:35.622748+0000","last_unstale":"2024-09-21T00:20:26.509577+0000","last_undegraded":"2024-09-21T00:20:26.509577+0000","last_fullsized":"2024-09-21T00:20:26.509577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:15.570789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.453838+0000","last_change":"2024-09-21T00:18:48.739612+0000","last_active":"2024-09-21T00:20:26.453838+0000","last_peered":"2024-09-21T00:20:26.453838+0000","last_clean":"2024-09-21T00:20:26.453838+0000","last_became_active":"2024-09-21T00:18:48.739173+0000","last_became_peered":"2024-09-21T00:18:48.739173+0000","last_unstale":"2024-09-21T00:20:26.453838+0000","last_undegraded":"2024-09-21T00:20:26.453838+0000","last_fullsized":"2024-09-21T00:20:26.453838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:26.327178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.766649+0000","last_change":"2024-09-21T00:18:38.670876+0000","last_active":"2024-09-21T00:20:54.766649+0000","last_peered":"2024-09-21T00:20:54.766649+0000","last_clean":"2024-09-21T00:20:54.766649+0000","last_became_active":"2024-09-21T00:18:38.670371+0000","last_became_peered":"2024-09-21T00:18:38.670371+0000","last_unstale":"2024-09-21T00:20:54.766649+0000","last_undegraded":"2024-09-21T00:20:54.766649+0000","last_fullsized":"2024-09-21T00:20:54.766649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:48.641071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.452030+0000","last_change":"2024-09-21T00:18:35.622919+0000","last_active":"2024-09-21T00:20:26.452030+0000","last_peered":"2024-09-21T00:20:26.452030+0000","last_clean":"2024-09-21T00:20:26.452030+0000","last_became_active":"2024-09-21T00:18:35.622435+0000","last_became_peered":"2024-09-21T00:18:35.622435+0000","last_unstale":"2024-09-21T00:20:26.452030+0000","last_undegraded":"2024-09-21T00:20:26.452030+0000","last_fullsized":"2024-09-21T00:20:26.452030+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:22.113871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509031+0000","last_change":"2024-09-21T00:18:48.741736+0000","last_active":"2024-09-21T00:20:26.509031+0000","last_peered":"2024-09-21T00:20:26.509031+0000","last_clean":"2024-09-21T00:20:26.509031+0000","last_became_active":"2024-09-21T00:18:48.741135+0000","last_became_peered":"2024-09-21T00:18:48.741135+0000","last_unstale":"2024-09-21T00:20:26.509031+0000","last_undegraded":"2024-09-21T00:20:26.509031+0000","last_fullsized":"2024-09-21T00:20:26.509031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:50.492351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.748256+0000","last_change":"2024-09-21T00:18:38.667377+0000","last_active":"2024-09-21T00:20:54.748256+0000","last_peered":"2024-09-21T00:20:54.748256+0000","last_clean":"2024-09-21T00:20:54.748256+0000","last_became_active":"2024-09-21T00:18:38.667010+0000","last_became_peered":"2024-09-21T00:18:38.667010+0000","last_unstale":"2024-09-21T00:20:54.748256+0000","last_undegraded":"2024-09-21T00:20:54.748256+0000","last_fullsized":"2024-09-21T00:20:54.748256+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:06.634767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":361,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:40.117626+0000","last_change":"2024-09-21T00:19:25.853603+0000","last_active":"2024-09-21T00:21:40.117626+0000","last_peered":"2024-09-21T00:21:40.117626+0000","last_clean":"2024-09-21T00:21:40.117626+0000","last_became_active":"2024-09-21T00:19:25.841126+0000","last_became_peered":"2024-09-21T00:19:25.841126+0000","last_unstale":"2024-09-21T00:21:40.117626+0000","last_undegraded":"2024-09-21T00:21:40.117626+0000","last_fullsized":"2024-09-21T00:21:40.117626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:27:47.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71662,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":261,"num_read_kb":201,"num_write":38,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451696+0000","last_change":"2024-09-21T00:18:48.731004+0000","last_active":"2024-09-21T00:20:26.451696+0000","last_peered":"2024-09-21T00:20:26.451696+0000","last_clean":"2024-09-21T00:20:26.451696+0000","last_became_active":"2024-09-21T00:18:48.730549+0000","last_became_peered":"2024-09-21T00:18:48.730549+0000","last_unstale":"2024-09-21T00:20:26.451696+0000","last_undegraded":"2024-09-21T00:20:26.451696+0000","last_fullsized":"2024-09-21T00:20:26.451696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:30.988300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764366+0000","last_change":"2024-09-21T00:19:25.843532+0000","last_active":"2024-09-21T00:20:54.764366+0000","last_peered":"2024-09-21T00:20:54.764366+0000","last_clean":"2024-09-21T00:20:54.764366+0000","last_became_active":"2024-09-21T00:19:25.843065+0000","last_became_peered":"2024-09-21T00:19:25.843065+0000","last_unstale":"2024-09-21T00:20:54.764366+0000","last_undegraded":"2024-09-21T00:20:54.764366+0000","last_fullsized":"2024-09-21T00:20:54.764366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:36:26.488692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509317+0000","last_change":"2024-09-21T00:18:35.770494+0000","last_active":"2024-09-21T00:20:26.509317+0000","last_peered":"2024-09-21T00:20:26.509317+0000","last_clean":"2024-09-21T00:20:26.509317+0000","last_became_active":"2024-09-21T00:18:35.770122+0000","last_became_peered":"2024-09-21T00:18:35.770122+0000","last_unstale":"2024-09-21T00:20:26.509317+0000","last_undegraded":"2024-09-21T00:20:26.509317+0000","last_fullsized":"2024-09-21T00:20:26.509317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:41.415904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451852+0000","last_change":"2024-09-21T00:18:48.736802+0000","last_active":"2024-09-21T00:20:26.451852+0000","last_peered":"2024-09-21T00:20:26.451852+0000","last_clean":"2024-09-21T00:20:26.451852+0000","last_became_active":"2024-09-21T00:18:48.736008+0000","last_became_peered":"2024-09-21T00:18:48.736008+0000","last_unstale":"2024-09-21T00:20:26.451852+0000","last_undegraded":"2024-09-21T00:20:26.451852+0000","last_fullsized":"2024-09-21T00:20:26.451852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:07.995614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767189+0000","last_change":"2024-09-21T00:18:38.678385+0000","last_active":"2024-09-21T00:20:54.767189+0000","last_peered":"2024-09-21T00:20:54.767189+0000","last_clean":"2024-09-21T00:20:54.767189+0000","last_became_active":"2024-09-21T00:18:38.677174+0000","last_became_peered":"2024-09-21T00:18:38.677174+0000","last_unstale":"2024-09-21T00:20:54.767189+0000","last_undegraded":"2024-09-21T00:20:54.767189+0000","last_fullsized":"2024-09-21T00:20:54.767189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:23.486222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509498+0000","last_change":"2024-09-21T00:18:35.627703+0000","last_active":"2024-09-21T00:20:26.509498+0000","last_peered":"2024-09-21T00:20:26.509498+0000","last_clean":"2024-09-21T00:20:26.509498+0000","last_became_active":"2024-09-21T00:18:35.627274+0000","last_became_peered":"2024-09-21T00:18:35.627274+0000","last_unstale":"2024-09-21T00:20:26.509498+0000","last_undegraded":"2024-09-21T00:20:26.509498+0000","last_fullsized":"2024-09-21T00:20:26.509498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:14.326810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451122+0000","last_change":"2024-09-21T00:18:48.729926+0000","last_active":"2024-09-21T00:20:26.451122+0000","last_peered":"2024-09-21T00:20:26.451122+0000","last_clean":"2024-09-21T00:20:26.451122+0000","last_became_active":"2024-09-21T00:18:48.728366+0000","last_became_peered":"2024-09-21T00:18:48.728366+0000","last_unstale":"2024-09-21T00:20:26.451122+0000","last_undegraded":"2024-09-21T00:20:26.451122+0000","last_fullsized":"2024-09-21T00:20:26.451122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:48:50.005748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.763272+0000","last_change":"2024-09-21T00:18:38.664173+0000","last_active":"2024-09-21T00:20:54.763272+0000","last_peered":"2024-09-21T00:20:54.763272+0000","last_clean":"2024-09-21T00:20:54.763272+0000","last_became_active":"2024-09-21T00:18:38.663358+0000","last_became_peered":"2024-09-21T00:18:38.663358+0000","last_unstale":"2024-09-21T00:20:54.763272+0000","last_undegraded":"2024-09-21T00:20:54.763272+0000","last_fullsized":"2024-09-21T00:20:54.763272+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:34:31.203621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":175,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:39.115736+0000","last_change":"2024-09-21T00:18:35.622133+0000","last_active":"2024-09-21T00:21:39.115736+0000","last_peered":"2024-09-21T00:21:39.115736+0000","last_clean":"2024-09-21T00:21:39.115736+0000","last_became_active":"2024-09-21T00:18:35.621032+0000","last_became_peered":"2024-09-21T00:18:35.621032+0000","last_unstale":"2024-09-21T00:21:39.115736+0000","last_undegraded":"2024-09-21T00:21:39.115736+0000","last_fullsized":"2024-09-21T00:21:39.115736+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:24.121665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.574990+0000","last_change":"2024-09-21T00:18:48.733339+0000","last_active":"2024-09-21T00:20:26.574990+0000","last_peered":"2024-09-21T00:20:26.574990+0000","last_clean":"2024-09-21T00:20:26.574990+0000","last_became_active":"2024-09-21T00:18:48.731843+0000","last_became_peered":"2024-09-21T00:18:48.731843+0000","last_unstale":"2024-09-21T00:20:26.574990+0000","last_undegraded":"2024-09-21T00:20:26.574990+0000","last_fullsized":"2024-09-21T00:20:26.574990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:42:55.474197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.455859+0000","last_change":"2024-09-21T00:18:48.740108+0000","last_active":"2024-09-21T00:20:26.455859+0000","last_peered":"2024-09-21T00:20:26.455859+0000","last_clean":"2024-09-21T00:20:26.455859+0000","last_became_active":"2024-09-21T00:18:48.739330+0000","last_became_peered":"2024-09-21T00:18:48.739330+0000","last_unstale":"2024-09-21T00:20:26.455859+0000","last_undegraded":"2024-09-21T00:20:26.455859+0000","last_fullsized":"2024-09-21T00:20:26.455859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:00.103502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.763622+0000","last_change":"2024-09-21T00:18:38.660469+0000","last_active":"2024-09-21T00:20:54.763622+0000","last_peered":"2024-09-21T00:20:54.763622+0000","last_clean":"2024-09-21T00:20:54.763622+0000","last_became_active":"2024-09-21T00:18:38.660138+0000","last_became_peered":"2024-09-21T00:18:38.660138+0000","last_unstale":"2024-09-21T00:20:54.763622+0000","last_undegraded":"2024-09-21T00:20:54.763622+0000","last_fullsized":"2024-09-21T00:20:54.763622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:01.079112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509206+0000","last_change":"2024-09-21T00:18:35.624528+0000","last_active":"2024-09-21T00:20:26.509206+0000","last_peered":"2024-09-21T00:20:26.509206+0000","last_clean":"2024-09-21T00:20:26.509206+0000","last_became_active":"2024-09-21T00:18:35.624170+0000","last_became_peered":"2024-09-21T00:18:35.624170+0000","last_unstale":"2024-09-21T00:20:26.509206+0000","last_undegraded":"2024-09-21T00:20:26.509206+0000","last_fullsized":"2024-09-21T00:20:26.509206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:07.914521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.512325+0000","last_change":"2024-09-21T00:20:26.511824+0000","last_active":"2024-09-21T00:20:26.512325+0000","last_peered":"2024-09-21T00:20:26.512325+0000","last_clean":"2024-09-21T00:20:26.512325+0000","last_became_active":"2024-09-21T00:20:26.511254+0000","last_became_peered":"2024-09-21T00:20:26.511254+0000","last_unstale":"2024-09-21T00:20:26.512325+0000","last_undegraded":"2024-09-21T00:20:26.512325+0000","last_fullsized":"2024-09-21T00:20:26.512325+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:12.884034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.760218+0000","last_change":"2024-09-21T00:18:38.666928+0000","last_active":"2024-09-21T00:20:54.760218+0000","last_peered":"2024-09-21T00:20:54.760218+0000","last_clean":"2024-09-21T00:20:54.760218+0000","last_became_active":"2024-09-21T00:18:38.666236+0000","last_became_peered":"2024-09-21T00:18:38.666236+0000","last_unstale":"2024-09-21T00:20:54.760218+0000","last_undegraded":"2024-09-21T00:20:54.760218+0000","last_fullsized":"2024-09-21T00:20:54.760218+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:21.043778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.508330+0000","last_change":"2024-09-21T00:18:35.624514+0000","last_active":"2024-09-21T00:20:26.508330+0000","last_peered":"2024-09-21T00:20:26.508330+0000","last_clean":"2024-09-21T00:20:26.508330+0000","last_became_active":"2024-09-21T00:18:35.624098+0000","last_became_peered":"2024-09-21T00:18:35.624098+0000","last_unstale":"2024-09-21T00:20:26.508330+0000","last_undegraded":"2024-09-21T00:20:26.508330+0000","last_fullsized":"2024-09-21T00:20:26.508330+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:21.777568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.509891+0000","last_change":"2024-09-21T00:19:25.841232+0000","last_active":"2024-09-21T00:20:26.509891+0000","last_peered":"2024-09-21T00:20:26.509891+0000","last_clean":"2024-09-21T00:20:26.509891+0000","last_became_active":"2024-09-21T00:19:25.840655+0000","last_became_peered":"2024-09-21T00:19:25.840655+0000","last_unstale":"2024-09-21T00:20:26.509891+0000","last_undegraded":"2024-09-21T00:20:26.509891+0000","last_fullsized":"2024-09-21T00:20:26.509891+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:05.846222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.762522+0000","last_change":"2024-09-21T00:18:38.667466+0000","last_active":"2024-09-21T00:20:54.762522+0000","last_peered":"2024-09-21T00:20:54.762522+0000","last_clean":"2024-09-21T00:20:54.762522+0000","last_became_active":"2024-09-21T00:18:38.666960+0000","last_became_peered":"2024-09-21T00:18:38.666960+0000","last_unstale":"2024-09-21T00:20:54.762522+0000","last_undegraded":"2024-09-21T00:20:54.762522+0000","last_fullsized":"2024-09-21T00:20:54.762522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:29:56.614766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451707+0000","last_change":"2024-09-21T00:18:48.739656+0000","last_active":"2024-09-21T00:20:26.451707+0000","last_peered":"2024-09-21T00:20:26.451707+0000","last_clean":"2024-09-21T00:20:26.451707+0000","last_became_active":"2024-09-21T00:18:48.739228+0000","last_became_peered":"2024-09-21T00:18:48.739228+0000","last_unstale":"2024-09-21T00:20:26.451707+0000","last_undegraded":"2024-09-21T00:20:26.451707+0000","last_fullsized":"2024-09-21T00:20:26.451707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:11.427595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":183,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:21:40.118153+0000","last_change":"2024-09-21T00:18:35.624108+0000","last_active":"2024-09-21T00:21:40.118153+0000","last_peered":"2024-09-21T00:21:40.118153+0000","last_clean":"2024-09-21T00:21:40.118153+0000","last_became_active":"2024-09-21T00:18:35.623708+0000","last_became_peered":"2024-09-21T00:18:35.623708+0000","last_unstale":"2024-09-21T00:21:40.118153+0000","last_undegraded":"2024-09-21T00:21:40.118153+0000","last_fullsized":"2024-09-21T00:21:40.118153+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:34.599029+0000","last_clean_scrub_stamp":"2024-09-21T00:18:34.599029+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:50.020662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.759755+0000","last_change":"2024-09-21T00:18:38.665402+0000","last_active":"2024-09-21T00:20:54.759755+0000","last_peered":"2024-09-21T00:20:54.759755+0000","last_clean":"2024-09-21T00:20:54.759755+0000","last_became_active":"2024-09-21T00:18:38.665034+0000","last_became_peered":"2024-09-21T00:18:38.665034+0000","last_unstale":"2024-09-21T00:20:54.759755+0000","last_undegraded":"2024-09-21T00:20:54.759755+0000","last_fullsized":"2024-09-21T00:20:54.759755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:33:10.824705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:26.451804+0000","last_change":"2024-09-21T00:18:48.729928+0000","last_active":"2024-09-21T00:20:26.451804+0000","last_peered":"2024-09-21T00:20:26.451804+0000","last_clean":"2024-09-21T00:20:26.451804+0000","last_became_active":"2024-09-21T00:18:48.729498+0000","last_became_peered":"2024-09-21T00:18:48.729498+0000","last_unstale":"2024-09-21T00:20:26.451804+0000","last_undegraded":"2024-09-21T00:20:26.451804+0000","last_fullsized":"2024-09-21T00:20:26.451804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:47.691073+0000","last_clean_scrub_stamp":"2024-09-21T00:18:47.691073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:10:27.140012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754670+0000","last_change":"2024-09-21T00:18:38.662221+0000","last_active":"2024-09-21T00:20:54.754670+0000","last_peered":"2024-09-21T00:20:54.754670+0000","last_clean":"2024-09-21T00:20:54.754670+0000","last_became_active":"2024-09-21T00:18:38.660543+0000","last_became_peered":"2024-09-21T00:18:38.660543+0000","last_unstale":"2024-09-21T00:20:54.754670+0000","last_undegraded":"2024-09-21T00:20:54.754670+0000","last_fullsized":"2024-09-21T00:20:54.754670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:43.159699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.759670+0000","last_change":"2024-09-21T00:18:38.672696+0000","last_active":"2024-09-21T00:20:54.759670+0000","last_peered":"2024-09-21T00:20:54.759670+0000","last_clean":"2024-09-21T00:20:54.759670+0000","last_became_active":"2024-09-21T00:18:38.672356+0000","last_became_peered":"2024-09-21T00:18:38.672356+0000","last_unstale":"2024-09-21T00:20:54.759670+0000","last_undegraded":"2024-09-21T00:20:54.759670+0000","last_fullsized":"2024-09-21T00:20:54.759670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.549939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768251+0000","last_change":"2024-09-21T00:18:38.667017+0000","last_active":"2024-09-21T00:20:54.768251+0000","last_peered":"2024-09-21T00:20:54.768251+0000","last_clean":"2024-09-21T00:20:54.768251+0000","last_became_active":"2024-09-21T00:18:38.666643+0000","last_became_peered":"2024-09-21T00:18:38.666643+0000","last_unstale":"2024-09-21T00:20:54.768251+0000","last_undegraded":"2024-09-21T00:20:54.768251+0000","last_fullsized":"2024-09-21T00:20:54.768251+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:30.496339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767616+0000","last_change":"2024-09-21T00:18:38.667705+0000","last_active":"2024-09-21T00:20:54.767616+0000","last_peered":"2024-09-21T00:20:54.767616+0000","last_clean":"2024-09-21T00:20:54.767616+0000","last_became_active":"2024-09-21T00:18:38.665141+0000","last_became_peered":"2024-09-21T00:18:38.665141+0000","last_unstale":"2024-09-21T00:20:54.767616+0000","last_undegraded":"2024-09-21T00:20:54.767616+0000","last_fullsized":"2024-09-21T00:20:54.767616+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:17.624035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764929+0000","last_change":"2024-09-21T00:18:38.681364+0000","last_active":"2024-09-21T00:20:54.764929+0000","last_peered":"2024-09-21T00:20:54.764929+0000","last_clean":"2024-09-21T00:20:54.764929+0000","last_became_active":"2024-09-21T00:18:38.680937+0000","last_became_peered":"2024-09-21T00:18:38.680937+0000","last_unstale":"2024-09-21T00:20:54.764929+0000","last_undegraded":"2024-09-21T00:20:54.764929+0000","last_fullsized":"2024-09-21T00:20:54.764929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:34.206201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.754004+0000","last_change":"2024-09-21T00:19:25.843091+0000","last_active":"2024-09-21T00:20:54.754004+0000","last_peered":"2024-09-21T00:20:54.754004+0000","last_clean":"2024-09-21T00:20:54.754004+0000","last_became_active":"2024-09-21T00:19:25.842600+0000","last_became_peered":"2024-09-21T00:19:25.842600+0000","last_unstale":"2024-09-21T00:20:54.754004+0000","last_undegraded":"2024-09-21T00:20:54.754004+0000","last_fullsized":"2024-09-21T00:20:54.754004+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:10.256474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.753901+0000","last_change":"2024-09-21T00:18:38.668656+0000","last_active":"2024-09-21T00:20:54.753901+0000","last_peered":"2024-09-21T00:20:54.753901+0000","last_clean":"2024-09-21T00:20:54.753901+0000","last_became_active":"2024-09-21T00:18:38.668153+0000","last_became_peered":"2024-09-21T00:18:38.668153+0000","last_unstale":"2024-09-21T00:20:54.753901+0000","last_undegraded":"2024-09-21T00:20:54.753901+0000","last_fullsized":"2024-09-21T00:20:54.753901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:03.763609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.743954+0000","last_change":"2024-09-21T00:18:38.675945+0000","last_active":"2024-09-21T00:20:54.743954+0000","last_peered":"2024-09-21T00:20:54.743954+0000","last_clean":"2024-09-21T00:20:54.743954+0000","last_became_active":"2024-09-21T00:18:38.675397+0000","last_became_peered":"2024-09-21T00:18:38.675397+0000","last_unstale":"2024-09-21T00:20:54.743954+0000","last_undegraded":"2024-09-21T00:20:54.743954+0000","last_fullsized":"2024-09-21T00:20:54.743954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:56.082314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.762260+0000","last_change":"2024-09-21T00:18:38.662854+0000","last_active":"2024-09-21T00:20:54.762260+0000","last_peered":"2024-09-21T00:20:54.762260+0000","last_clean":"2024-09-21T00:20:54.762260+0000","last_became_active":"2024-09-21T00:18:38.662094+0000","last_became_peered":"2024-09-21T00:18:38.662094+0000","last_unstale":"2024-09-21T00:20:54.762260+0000","last_undegraded":"2024-09-21T00:20:54.762260+0000","last_fullsized":"2024-09-21T00:20:54.762260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:45.756641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.744870+0000","last_change":"2024-09-21T00:18:38.663282+0000","last_active":"2024-09-21T00:20:54.744870+0000","last_peered":"2024-09-21T00:20:54.744870+0000","last_clean":"2024-09-21T00:20:54.744870+0000","last_became_active":"2024-09-21T00:18:38.662148+0000","last_became_peered":"2024-09-21T00:18:38.662148+0000","last_unstale":"2024-09-21T00:20:54.744870+0000","last_undegraded":"2024-09-21T00:20:54.744870+0000","last_fullsized":"2024-09-21T00:20:54.744870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:51.090591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.763683+0000","last_change":"2024-09-21T00:18:38.669184+0000","last_active":"2024-09-21T00:20:54.763683+0000","last_peered":"2024-09-21T00:20:54.763683+0000","last_clean":"2024-09-21T00:20:54.763683+0000","last_became_active":"2024-09-21T00:18:38.668799+0000","last_became_peered":"2024-09-21T00:18:38.668799+0000","last_unstale":"2024-09-21T00:20:54.763683+0000","last_undegraded":"2024-09-21T00:20:54.763683+0000","last_fullsized":"2024-09-21T00:20:54.763683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:49.642568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767674+0000","last_change":"2024-09-21T00:18:38.679501+0000","last_active":"2024-09-21T00:20:54.767674+0000","last_peered":"2024-09-21T00:20:54.767674+0000","last_clean":"2024-09-21T00:20:54.767674+0000","last_became_active":"2024-09-21T00:18:38.678582+0000","last_became_peered":"2024-09-21T00:18:38.678582+0000","last_unstale":"2024-09-21T00:20:54.767674+0000","last_undegraded":"2024-09-21T00:20:54.767674+0000","last_fullsized":"2024-09-21T00:20:54.767674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:30.277979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.751399+0000","last_change":"2024-09-21T00:18:38.659657+0000","last_active":"2024-09-21T00:20:54.751399+0000","last_peered":"2024-09-21T00:20:54.751399+0000","last_clean":"2024-09-21T00:20:54.751399+0000","last_became_active":"2024-09-21T00:18:38.659257+0000","last_became_peered":"2024-09-21T00:18:38.659257+0000","last_unstale":"2024-09-21T00:20:54.751399+0000","last_undegraded":"2024-09-21T00:20:54.751399+0000","last_fullsized":"2024-09-21T00:20:54.751399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:08.653729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.758299+0000","last_change":"2024-09-21T00:18:38.671817+0000","last_active":"2024-09-21T00:20:54.758299+0000","last_peered":"2024-09-21T00:20:54.758299+0000","last_clean":"2024-09-21T00:20:54.758299+0000","last_became_active":"2024-09-21T00:18:38.671469+0000","last_became_peered":"2024-09-21T00:18:38.671469+0000","last_unstale":"2024-09-21T00:20:54.758299+0000","last_undegraded":"2024-09-21T00:20:54.758299+0000","last_fullsized":"2024-09-21T00:20:54.758299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:58:15.036623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.761325+0000","last_change":"2024-09-21T00:18:38.664481+0000","last_active":"2024-09-21T00:20:54.761325+0000","last_peered":"2024-09-21T00:20:54.761325+0000","last_clean":"2024-09-21T00:20:54.761325+0000","last_became_active":"2024-09-21T00:18:38.664098+0000","last_became_peered":"2024-09-21T00:18:38.664098+0000","last_unstale":"2024-09-21T00:20:54.761325+0000","last_undegraded":"2024-09-21T00:20:54.761325+0000","last_fullsized":"2024-09-21T00:20:54.761325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:09.271251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768665+0000","last_change":"2024-09-21T00:18:38.662416+0000","last_active":"2024-09-21T00:20:54.768665+0000","last_peered":"2024-09-21T00:20:54.768665+0000","last_clean":"2024-09-21T00:20:54.768665+0000","last_became_active":"2024-09-21T00:18:38.662041+0000","last_became_peered":"2024-09-21T00:18:38.662041+0000","last_unstale":"2024-09-21T00:20:54.768665+0000","last_undegraded":"2024-09-21T00:20:54.768665+0000","last_fullsized":"2024-09-21T00:20:54.768665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:03.150179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768448+0000","last_change":"2024-09-21T00:18:38.670797+0000","last_active":"2024-09-21T00:20:54.768448+0000","last_peered":"2024-09-21T00:20:54.768448+0000","last_clean":"2024-09-21T00:20:54.768448+0000","last_became_active":"2024-09-21T00:18:38.670258+0000","last_became_peered":"2024-09-21T00:18:38.670258+0000","last_unstale":"2024-09-21T00:20:54.768448+0000","last_undegraded":"2024-09-21T00:20:54.768448+0000","last_fullsized":"2024-09-21T00:20:54.768448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:55.669836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.765633+0000","last_change":"2024-09-21T00:18:38.665594+0000","last_active":"2024-09-21T00:20:54.765633+0000","last_peered":"2024-09-21T00:20:54.765633+0000","last_clean":"2024-09-21T00:20:54.765633+0000","last_became_active":"2024-09-21T00:18:38.665260+0000","last_became_peered":"2024-09-21T00:18:38.665260+0000","last_unstale":"2024-09-21T00:20:54.765633+0000","last_undegraded":"2024-09-21T00:20:54.765633+0000","last_fullsized":"2024-09-21T00:20:54.765633+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:52:09.014327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768070+0000","last_change":"2024-09-21T00:18:38.679294+0000","last_active":"2024-09-21T00:20:54.768070+0000","last_peered":"2024-09-21T00:20:54.768070+0000","last_clean":"2024-09-21T00:20:54.768070+0000","last_became_active":"2024-09-21T00:18:38.673765+0000","last_became_peered":"2024-09-21T00:18:38.673765+0000","last_unstale":"2024-09-21T00:20:54.768070+0000","last_undegraded":"2024-09-21T00:20:54.768070+0000","last_fullsized":"2024-09-21T00:20:54.768070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:32.076299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767090+0000","last_change":"2024-09-21T00:18:38.667869+0000","last_active":"2024-09-21T00:20:54.767090+0000","last_peered":"2024-09-21T00:20:54.767090+0000","last_clean":"2024-09-21T00:20:54.767090+0000","last_became_active":"2024-09-21T00:18:38.667499+0000","last_became_peered":"2024-09-21T00:18:38.667499+0000","last_unstale":"2024-09-21T00:20:54.767090+0000","last_undegraded":"2024-09-21T00:20:54.767090+0000","last_fullsized":"2024-09-21T00:20:54.767090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:13.501043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.761252+0000","last_change":"2024-09-21T00:18:38.663107+0000","last_active":"2024-09-21T00:20:54.761252+0000","last_peered":"2024-09-21T00:20:54.761252+0000","last_clean":"2024-09-21T00:20:54.761252+0000","last_became_active":"2024-09-21T00:18:38.662680+0000","last_became_peered":"2024-09-21T00:18:38.662680+0000","last_unstale":"2024-09-21T00:20:54.761252+0000","last_undegraded":"2024-09-21T00:20:54.761252+0000","last_fullsized":"2024-09-21T00:20:54.761252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:48.154705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.764194+0000","last_change":"2024-09-21T00:18:38.668167+0000","last_active":"2024-09-21T00:20:54.764194+0000","last_peered":"2024-09-21T00:20:54.764194+0000","last_clean":"2024-09-21T00:20:54.764194+0000","last_became_active":"2024-09-21T00:18:38.667842+0000","last_became_peered":"2024-09-21T00:18:38.667842+0000","last_unstale":"2024-09-21T00:20:54.764194+0000","last_undegraded":"2024-09-21T00:20:54.764194+0000","last_fullsized":"2024-09-21T00:20:54.764194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:55.358351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.761794+0000","last_change":"2024-09-21T00:18:38.666580+0000","last_active":"2024-09-21T00:20:54.761794+0000","last_peered":"2024-09-21T00:20:54.761794+0000","last_clean":"2024-09-21T00:20:54.761794+0000","last_became_active":"2024-09-21T00:18:38.666231+0000","last_became_peered":"2024-09-21T00:18:38.666231+0000","last_unstale":"2024-09-21T00:20:54.761794+0000","last_undegraded":"2024-09-21T00:20:54.761794+0000","last_fullsized":"2024-09-21T00:20:54.761794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:07.362516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.751359+0000","last_change":"2024-09-21T00:20:26.511248+0000","last_active":"2024-09-21T00:20:54.751359+0000","last_peered":"2024-09-21T00:20:54.751359+0000","last_clean":"2024-09-21T00:20:54.751359+0000","last_became_active":"2024-09-21T00:20:26.510743+0000","last_became_peered":"2024-09-21T00:20:26.510743+0000","last_unstale":"2024-09-21T00:20:54.751359+0000","last_undegraded":"2024-09-21T00:20:54.751359+0000","last_fullsized":"2024-09-21T00:20:54.751359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:41.577950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.760033+0000","last_change":"2024-09-21T00:18:38.666580+0000","last_active":"2024-09-21T00:20:54.760033+0000","last_peered":"2024-09-21T00:20:54.760033+0000","last_clean":"2024-09-21T00:20:54.760033+0000","last_became_active":"2024-09-21T00:18:38.665698+0000","last_became_peered":"2024-09-21T00:18:38.665698+0000","last_unstale":"2024-09-21T00:20:54.760033+0000","last_undegraded":"2024-09-21T00:20:54.760033+0000","last_fullsized":"2024-09-21T00:20:54.760033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:45.627667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.765480+0000","last_change":"2024-09-21T00:18:38.670729+0000","last_active":"2024-09-21T00:20:54.765480+0000","last_peered":"2024-09-21T00:20:54.765480+0000","last_clean":"2024-09-21T00:20:54.765480+0000","last_became_active":"2024-09-21T00:18:38.670337+0000","last_became_peered":"2024-09-21T00:18:38.670337+0000","last_unstale":"2024-09-21T00:20:54.765480+0000","last_undegraded":"2024-09-21T00:20:54.765480+0000","last_fullsized":"2024-09-21T00:20:54.765480+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:49.880689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.763380+0000","last_change":"2024-09-21T00:18:38.664644+0000","last_active":"2024-09-21T00:20:54.763380+0000","last_peered":"2024-09-21T00:20:54.763380+0000","last_clean":"2024-09-21T00:20:54.763380+0000","last_became_active":"2024-09-21T00:18:38.664220+0000","last_became_peered":"2024-09-21T00:18:38.664220+0000","last_unstale":"2024-09-21T00:20:54.763380+0000","last_undegraded":"2024-09-21T00:20:54.763380+0000","last_fullsized":"2024-09-21T00:20:54.763380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:09.610962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.759277+0000","last_change":"2024-09-21T00:18:38.671592+0000","last_active":"2024-09-21T00:20:54.759277+0000","last_peered":"2024-09-21T00:20:54.759277+0000","last_clean":"2024-09-21T00:20:54.759277+0000","last_became_active":"2024-09-21T00:18:38.671228+0000","last_became_peered":"2024-09-21T00:18:38.671228+0000","last_unstale":"2024-09-21T00:20:54.759277+0000","last_undegraded":"2024-09-21T00:20:54.759277+0000","last_fullsized":"2024-09-21T00:20:54.759277+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:51.906058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.762137+0000","last_change":"2024-09-21T00:19:25.843696+0000","last_active":"2024-09-21T00:20:54.762137+0000","last_peered":"2024-09-21T00:20:54.762137+0000","last_clean":"2024-09-21T00:20:54.762137+0000","last_became_active":"2024-09-21T00:19:25.843287+0000","last_became_peered":"2024-09-21T00:19:25.843287+0000","last_unstale":"2024-09-21T00:20:54.762137+0000","last_undegraded":"2024-09-21T00:20:54.762137+0000","last_fullsized":"2024-09-21T00:20:54.762137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:21.024555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.767946+0000","last_change":"2024-09-21T00:18:38.671820+0000","last_active":"2024-09-21T00:20:54.767946+0000","last_peered":"2024-09-21T00:20:54.767946+0000","last_clean":"2024-09-21T00:20:54.767946+0000","last_became_active":"2024-09-21T00:18:38.667107+0000","last_became_peered":"2024-09-21T00:18:38.667107+0000","last_unstale":"2024-09-21T00:20:54.767946+0000","last_undegraded":"2024-09-21T00:20:54.767946+0000","last_fullsized":"2024-09-21T00:20:54.767946+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:58.099750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.759553+0000","last_change":"2024-09-21T00:18:38.671271+0000","last_active":"2024-09-21T00:20:54.759553+0000","last_peered":"2024-09-21T00:20:54.759553+0000","last_clean":"2024-09-21T00:20:54.759553+0000","last_became_active":"2024-09-21T00:18:38.670826+0000","last_became_peered":"2024-09-21T00:18:38.670826+0000","last_unstale":"2024-09-21T00:20:54.759553+0000","last_undegraded":"2024-09-21T00:20:54.759553+0000","last_fullsized":"2024-09-21T00:20:54.759553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:57:01.723120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T00:20:54.768399+0000","last_change":"2024-09-21T00:18:38.662851+0000","last_active":"2024-09-21T00:20:54.768399+0000","last_peered":"2024-09-21T00:20:54.768399+0000","last_clean":"2024-09-21T00:20:54.768399+0000","last_became_active":"2024-09-21T00:18:38.661259+0000","last_became_peered":"2024-09-21T00:18:38.661259+0000","last_unstale":"2024-09-21T00:20:54.768399+0000","last_undegraded":"2024-09-21T00:20:54.768399+0000","last_fullsized":"2024-09-21T00:20:54.768399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:18:37.624343+0000","last_clean_scrub_stamp":"2024-09-21T00:18:37.624343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:10.508980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":862796,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1118,"num_read_kb":772,"num_write":399,"num_write_kb":2220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3864,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2826240,"data_stored":2588388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678067,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":535425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874296,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37492,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707660,"statfs":{"total":95995035648,"available":95956643840,"internally_reserved":0,"allocated":2138112,"data_stored":1588109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037821,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712548,"statfs":{"total":95995035648,"available":95961649152,"internally_reserved":0,"allocated":1454080,"data_stored":932951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234051,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36756,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708396,"statfs":{"total":95995035648,"available":95957397504,"internally_reserved":0,"allocated":1445888,"data_stored":928753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430280,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1131609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593806,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":530815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790037,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29092,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716060,"statfs":{"total":95995035648,"available":95965245440,"internally_reserved":0,"allocated":2056192,"data_stored":1520973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953562,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1134592,"data_stored":607709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149790,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28880,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716272,"statfs":{"total":95995035648,"available":95965462528,"internally_reserved":0,"allocated":1839104,"data_stored":1318001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378726,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":532025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542251,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":602387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705776,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":532203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":806912,"data_stored":787276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":395784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":512000,"data_stored":467016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:21:46.971 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:21:47.393 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:47.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:47 smithi089 ceph-mon[34481]: from='client.25411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:21:47.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:47 smithi098 ceph-mon[34725]: from='client.25411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:21:47.533 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:47 smithi027 ceph-mon[30745]: from='client.25411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T00:21:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.074+0000 7fdd71a00640 1 -- 172.21.15.27:0/3678819164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd6c10b500 msgr2=0x7fdd6c10b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.074+0000 7fdd71a00640 1 --2- 172.21.15.27:0/3678819164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd6c10b500 0x7fdd6c10b910 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009900 tx=0x7fdd5802a410 comp rx=0 tx=0).stop 2024-09-21T00:21:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.074+0000 7fdd71a00640 1 -- 172.21.15.27:0/3678819164 shutdown_connections 2024-09-21T00:21:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.074+0000 7fdd71a00640 1 --2- 172.21.15.27:0/3678819164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd6c10be50 0x7fdd6c112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.074+0000 7fdd71a00640 1 --2- 172.21.15.27:0/3678819164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd6c10b500 0x7fdd6c10b910 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.074+0000 7fdd71a00640 1 --2- 172.21.15.27:0/3678819164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd6c105bc0 0x7fdd6c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.074+0000 7fdd71a00640 1 -- 172.21.15.27:0/3678819164 >> 172.21.15.27:0/3678819164 conn(0x7fdd6c101670 msgr2=0x7fdd6c103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.074+0000 7fdd71a00640 1 -- 172.21.15.27:0/3678819164 shutdown_connections 2024-09-21T00:21:48.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.074+0000 7fdd71a00640 1 -- 172.21.15.27:0/3678819164 wait complete. 2024-09-21T00:21:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.075+0000 7fdd71a00640 1 Processor -- start 2024-09-21T00:21:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.076+0000 7fdd71a00640 1 -- start start 2024-09-21T00:21:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.076+0000 7fdd71a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd6c105bc0 0x7fdd6c19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:48.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.076+0000 7fdd71a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd6c10b500 0x7fdd6c19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:48.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.076+0000 7fdd71a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd6c10be50 0x7fdd6c1a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:48.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.076+0000 7fdd71a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c115b60 con 0x7fdd6c10be50 2024-09-21T00:21:48.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.076+0000 7fdd71a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1159e0 con 0x7fdd6c105bc0 2024-09-21T00:21:48.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.076+0000 7fdd71a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c115ce0 con 0x7fdd6c10b500 2024-09-21T00:21:48.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.076+0000 7fdd71000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd6c10be50 0x7fdd6c1a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:48.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.077+0000 7fdd6b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd6c10b500 0x7fdd6c19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:48.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.077+0000 7fdd6be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd6c105bc0 0x7fdd6c19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:48.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.077+0000 7fdd6b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd6c10b500 0x7fdd6c19cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:45342/0 (socket says 172.21.15.27:45342) 2024-09-21T00:21:48.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.077+0000 7fdd71000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd6c10be50 0x7fdd6c1a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60738/0 (socket says 172.21.15.27:60738) 2024-09-21T00:21:48.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.077+0000 7fdd6b400640 1 -- 172.21.15.27:0/3610108991 learned_addr learned my addr 172.21.15.27:0/3610108991 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:48.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.077+0000 7fdd71000640 1 -- 172.21.15.27:0/3610108991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd6c10b500 msgr2=0x7fdd6c19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:48.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.077+0000 7fdd71000640 1 --2- 172.21.15.27:0/3610108991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd6c10b500 0x7fdd6c19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:48.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.077+0000 7fdd71000640 1 -- 172.21.15.27:0/3610108991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd6c105bc0 msgr2=0x7fdd6c19c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:48.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.077+0000 7fdd71000640 1 --2- 172.21.15.27:0/3610108991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd6c105bc0 0x7fdd6c19c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:48.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.077+0000 7fdd71000640 1 -- 172.21.15.27:0/3610108991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c1a2480 con 0x7fdd6c10be50 2024-09-21T00:21:48.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.077+0000 7fdd6be00640 1 --2- 172.21.15.27:0/3610108991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd6c105bc0 0x7fdd6c19c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:48.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.077+0000 7fdd71000640 1 --2- 172.21.15.27:0/3610108991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd6c10be50 0x7fdd6c1a1d50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009fd0 tx=0x7fdd6400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:48.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.078+0000 7fdd68c00640 1 -- 172.21.15.27:0/3610108991 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6400ce20 con 0x7fdd6c10be50 2024-09-21T00:21:48.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.078+0000 7fdd68c00640 1 -- 172.21.15.27:0/3610108991 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd64005020 con 0x7fdd6c10be50 2024-09-21T00:21:48.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.078+0000 7fdd71a00640 1 -- 172.21.15.27:0/3610108991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6c1a2770 con 0x7fdd6c10be50 2024-09-21T00:21:48.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.078+0000 7fdd68c00640 1 -- 172.21.15.27:0/3610108991 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64010690 con 0x7fdd6c10be50 2024-09-21T00:21:48.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.078+0000 7fdd71a00640 1 -- 172.21.15.27:0/3610108991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd6c1a8ed0 con 0x7fdd6c10be50 2024-09-21T00:21:48.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.079+0000 7fdd71a00640 1 -- 172.21.15.27:0/3610108991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd6c105fd0 con 0x7fdd6c10be50 2024-09-21T00:21:48.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.081+0000 7fdd68c00640 1 -- 172.21.15.27:0/3610108991 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7fdd640054d0 con 0x7fdd6c10be50 2024-09-21T00:21:48.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.084+0000 7fdd68c00640 1 --2- 172.21.15.27:0/3610108991 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdd500b6670 0x7fdd500b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:48.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.084+0000 7fdd68c00640 1 -- 172.21.15.27:0/3610108991 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fdd640e7ff0 con 0x7fdd6c10be50 2024-09-21T00:21:48.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.084+0000 7fdd6be00640 1 --2- 172.21.15.27:0/3610108991 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdd500b6670 0x7fdd500b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:48.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.085+0000 7fdd6be00640 1 --2- 172.21.15.27:0/3610108991 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdd500b6670 0x7fdd500b8b80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009fc0 tx=0x7fdd5c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:48.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.086+0000 7fdd68c00640 1 -- 172.21.15.27:0/3610108991 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdd640ae270 con 0x7fdd6c10be50 2024-09-21T00:21:48.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:48 smithi027 ceph-mon[30745]: pgmap v389: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-09-21T00:21:48.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.323+0000 7fdd71a00640 1 -- 172.21.15.27:0/3610108991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd6c19d640 con 0x7fdd6c10be50 2024-09-21T00:21:48.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.324+0000 7fdd68c00640 1 -- 172.21.15.27:0/3610108991 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11893 (secure 0 0 0) 0x7fdd6c19d640 con 0x7fdd6c10be50 2024-09-21T00:21:48.326 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:21:48.326 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":26,"btime":"2024-09-21T00:21:36:085073+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:21:35.264444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:21:48.326 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-09-21T00:21:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.327+0000 7fdd71a00640 1 -- 172.21.15.27:0/3610108991 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdd500b6670 msgr2=0x7fdd500b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.327+0000 7fdd71a00640 1 --2- 172.21.15.27:0/3610108991 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdd500b6670 0x7fdd500b8b80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009fc0 tx=0x7fdd5c005f90 comp rx=0 tx=0).stop 2024-09-21T00:21:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.327+0000 7fdd71a00640 1 -- 172.21.15.27:0/3610108991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd6c10be50 msgr2=0x7fdd6c1a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.327+0000 7fdd71a00640 1 --2- 172.21.15.27:0/3610108991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd6c10be50 0x7fdd6c1a1d50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009fd0 tx=0x7fdd6400ef10 comp rx=0 tx=0).stop 2024-09-21T00:21:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.327+0000 7fdd71a00640 1 -- 172.21.15.27:0/3610108991 shutdown_connections 2024-09-21T00:21:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.327+0000 7fdd71a00640 1 --2- 172.21.15.27:0/3610108991 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdd500b6670 0x7fdd500b8b80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.327+0000 7fdd71a00640 1 --2- 172.21.15.27:0/3610108991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd6c10be50 0x7fdd6c1a1d50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.327+0000 7fdd71a00640 1 --2- 172.21.15.27:0/3610108991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd6c10b500 0x7fdd6c19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.327+0000 7fdd71a00640 1 --2- 172.21.15.27:0/3610108991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd6c105bc0 0x7fdd6c19c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.327+0000 7fdd71a00640 1 -- 172.21.15.27:0/3610108991 >> 172.21.15.27:0/3610108991 conn(0x7fdd6c101670 msgr2=0x7fdd6c108b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:48.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.327+0000 7fdd71a00640 1 -- 172.21.15.27:0/3610108991 shutdown_connections 2024-09-21T00:21:48.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:48.327+0000 7fdd71a00640 1 -- 172.21.15.27:0/3610108991 wait complete. 2024-09-21T00:21:48.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:48 smithi089 ceph-mon[34481]: pgmap v389: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-09-21T00:21:48.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:48 smithi098 ceph-mon[34725]: pgmap v389: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-09-21T00:21:49.290 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:49 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3610108991' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:21:49.293 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:21:49.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:49 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3610108991' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:21:49.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:49 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3610108991' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:21:49.688 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.327+0000 7f5a52800640 1 -- 172.21.15.27:0/479580998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a4c10c010 msgr2=0x7f5a4c112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.327+0000 7f5a52800640 1 --2- 172.21.15.27:0/479580998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a4c10c010 0x7f5a4c112840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009ca0 tx=0x7f5a4402a8c0 comp rx=0 tx=0).stop 2024-09-21T00:21:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.327+0000 7f5a52800640 1 -- 172.21.15.27:0/479580998 shutdown_connections 2024-09-21T00:21:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.327+0000 7f5a52800640 1 --2- 172.21.15.27:0/479580998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a4c10c010 0x7f5a4c112840 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.327+0000 7f5a52800640 1 --2- 172.21.15.27:0/479580998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c10b6c0 0x7f5a4c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.327+0000 7f5a52800640 1 --2- 172.21.15.27:0/479580998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a4c105d80 0x7f5a4c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.327+0000 7f5a52800640 1 -- 172.21.15.27:0/479580998 >> 172.21.15.27:0/479580998 conn(0x7f5a4c1017f0 msgr2=0x7f5a4c103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.327+0000 7f5a52800640 1 -- 172.21.15.27:0/479580998 shutdown_connections 2024-09-21T00:21:50.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.327+0000 7f5a52800640 1 -- 172.21.15.27:0/479580998 wait complete. 2024-09-21T00:21:50.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.329+0000 7f5a52800640 1 Processor -- start 2024-09-21T00:21:50.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.329+0000 7f5a52800640 1 -- start start 2024-09-21T00:21:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a52800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a4c105d80 0x7f5a4c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a52800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a4c10b6c0 0x7f5a4c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a52800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c10c010 0x7f5a4c1a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a52800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c115d20 con 0x7f5a4c10b6c0 2024-09-21T00:21:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a52800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c115ba0 con 0x7f5a4c105d80 2024-09-21T00:21:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a52800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c115ea0 con 0x7f5a4c10c010 2024-09-21T00:21:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a4b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a4c105d80 0x7f5a4c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a42a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a4c10b6c0 0x7f5a4c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a4b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a4c105d80 0x7f5a4c19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:52618/0 (socket says 172.21.15.27:52618) 2024-09-21T00:21:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a4b400640 1 -- 172.21.15.27:0/4211408134 learned_addr learned my addr 172.21.15.27:0/4211408134 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a4be00640 1 --2- 172.21.15.27:0/4211408134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c10c010 0x7f5a4c1a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a4b400640 1 -- 172.21.15.27:0/4211408134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c10c010 msgr2=0x7f5a4c1a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a4b400640 1 --2- 172.21.15.27:0/4211408134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c10c010 0x7f5a4c1a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a4b400640 1 -- 172.21.15.27:0/4211408134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a4c10b6c0 msgr2=0x7f5a4c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a4b400640 1 --2- 172.21.15.27:0/4211408134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a4c10b6c0 0x7f5a4c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a4b400640 1 -- 172.21.15.27:0/4211408134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c1a25c0 con 0x7f5a4c105d80 2024-09-21T00:21:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.330+0000 7f5a4be00640 1 --2- 172.21.15.27:0/4211408134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c10c010 0x7f5a4c1a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:21:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.331+0000 7f5a4b400640 1 --2- 172.21.15.27:0/4211408134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a4c105d80 0x7f5a4c19c650 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00e9f0 tx=0x7f5a3c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.331+0000 7f5a48c00640 1 -- 172.21.15.27:0/4211408134 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c00cd80 con 0x7f5a4c105d80 2024-09-21T00:21:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.331+0000 7f5a48c00640 1 -- 172.21.15.27:0/4211408134 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a3c0108e0 con 0x7f5a4c105d80 2024-09-21T00:21:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.331+0000 7f5a48c00640 1 -- 172.21.15.27:0/4211408134 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c018980 con 0x7f5a4c105d80 2024-09-21T00:21:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.331+0000 7f5a52800640 1 -- 172.21.15.27:0/4211408134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4c1a28b0 con 0x7f5a4c105d80 2024-09-21T00:21:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.331+0000 7f5a52800640 1 -- 172.21.15.27:0/4211408134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4c1a9010 con 0x7f5a4c105d80 2024-09-21T00:21:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.332+0000 7f5a52800640 1 -- 172.21.15.27:0/4211408134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a4c106190 con 0x7f5a4c105d80 2024-09-21T00:21:50.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.335+0000 7f5a48c00640 1 -- 172.21.15.27:0/4211408134 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f5a3c010430 con 0x7f5a4c105d80 2024-09-21T00:21:50.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.336+0000 7f5a48c00640 1 --2- 172.21.15.27:0/4211408134 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5a300b6670 0x7f5a300b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:50.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.336+0000 7f5a48c00640 1 -- 172.21.15.27:0/4211408134 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f5a3c0e8190 con 0x7f5a4c105d80 2024-09-21T00:21:50.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.337+0000 7f5a42a00640 1 --2- 172.21.15.27:0/4211408134 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5a300b6670 0x7f5a300b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:50.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.337+0000 7f5a42a00640 1 --2- 172.21.15.27:0/4211408134 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5a300b6670 0x7f5a300b8b80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c105040 tx=0x7f5a34005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:50.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.338+0000 7f5a48c00640 1 -- 172.21.15.27:0/4211408134 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5a3c0ae410 con 0x7f5a4c105d80 2024-09-21T00:21:50.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:50 smithi089 ceph-mon[34481]: pgmap v390: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 52 op/s 2024-09-21T00:21:50.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:50 smithi098 ceph-mon[34725]: pgmap v390: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 52 op/s 2024-09-21T00:21:50.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:50 smithi027 ceph-mon[30745]: pgmap v390: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 52 op/s 2024-09-21T00:21:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.580+0000 7f5a52800640 1 -- 172.21.15.27:0/4211408134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a4c02e640 con 0x7f5a4c105d80 2024-09-21T00:21:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.581+0000 7f5a48c00640 1 -- 172.21.15.27:0/4211408134 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11893 (secure 0 0 0) 0x7f5a3c0b27f0 con 0x7f5a4c105d80 2024-09-21T00:21:50.582 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:21:50.582 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":26,"btime":"2024-09-21T00:21:36:085073+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:21:35.264444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:21:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-09-21T00:21:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.583+0000 7f5a52800640 1 -- 172.21.15.27:0/4211408134 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5a300b6670 msgr2=0x7f5a300b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.583+0000 7f5a52800640 1 --2- 172.21.15.27:0/4211408134 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5a300b6670 0x7f5a300b8b80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c105040 tx=0x7f5a34005f90 comp rx=0 tx=0).stop 2024-09-21T00:21:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.584+0000 7f5a52800640 1 -- 172.21.15.27:0/4211408134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a4c105d80 msgr2=0x7f5a4c19c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.584+0000 7f5a52800640 1 --2- 172.21.15.27:0/4211408134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a4c105d80 0x7f5a4c19c650 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00e9f0 tx=0x7f5a3c00eeb0 comp rx=0 tx=0).stop 2024-09-21T00:21:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.584+0000 7f5a52800640 1 -- 172.21.15.27:0/4211408134 shutdown_connections 2024-09-21T00:21:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.584+0000 7f5a52800640 1 --2- 172.21.15.27:0/4211408134 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5a300b6670 0x7f5a300b8b80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.584+0000 7f5a52800640 1 --2- 172.21.15.27:0/4211408134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c10c010 0x7f5a4c1a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.584+0000 7f5a52800640 1 --2- 172.21.15.27:0/4211408134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a4c10b6c0 0x7f5a4c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.584+0000 7f5a52800640 1 --2- 172.21.15.27:0/4211408134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a4c105d80 0x7f5a4c19c650 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.584+0000 7f5a52800640 1 -- 172.21.15.27:0/4211408134 >> 172.21.15.27:0/4211408134 conn(0x7f5a4c1017f0 msgr2=0x7f5a4c10acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:50.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.584+0000 7f5a52800640 1 -- 172.21.15.27:0/4211408134 shutdown_connections 2024-09-21T00:21:50.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:50.584+0000 7f5a52800640 1 -- 172.21.15.27:0/4211408134 wait complete. 2024-09-21T00:21:51.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:51 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/4211408134' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:21:51.464 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T00:21:51.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:51 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/4211408134' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:21:51.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:51 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/4211408134' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:21:51.852 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:52.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:52 smithi027 ceph-mon[30745]: pgmap v391: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 73 KiB/s wr, 14 op/s 2024-09-21T00:21:52.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:52 smithi089 ceph-mon[34481]: pgmap v391: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 73 KiB/s wr, 14 op/s 2024-09-21T00:21:52.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:52 smithi098 ceph-mon[34725]: pgmap v391: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 73 KiB/s wr, 14 op/s 2024-09-21T00:21:52.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.503+0000 7f854c800640 1 -- 172.21.15.27:0/2059382420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544078290 msgr2=0x7f8544114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:52.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.503+0000 7f854c800640 1 --2- 172.21.15.27:0/2059382420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544078290 0x7f8544114200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8534009900 tx=0x7f853402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:21:52.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.503+0000 7f854c800640 1 -- 172.21.15.27:0/2059382420 shutdown_connections 2024-09-21T00:21:52.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.503+0000 7f854c800640 1 --2- 172.21.15.27:0/2059382420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8544114740 0x7f8544116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:52.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.503+0000 7f854c800640 1 --2- 172.21.15.27:0/2059382420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544078290 0x7f8544114200 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:52.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.503+0000 7f854c800640 1 --2- 172.21.15.27:0/2059382420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8544077940 0x7f8544077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:52.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.503+0000 7f854c800640 1 -- 172.21.15.27:0/2059382420 >> 172.21.15.27:0/2059382420 conn(0x7f8544101810 msgr2=0x7f8544103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:52.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.503+0000 7f854c800640 1 -- 172.21.15.27:0/2059382420 shutdown_connections 2024-09-21T00:21:52.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.503+0000 7f854c800640 1 -- 172.21.15.27:0/2059382420 wait complete. 2024-09-21T00:21:52.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.504+0000 7f854c800640 1 Processor -- start 2024-09-21T00:21:52.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.504+0000 7f854c800640 1 -- start start 2024-09-21T00:21:52.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.505+0000 7f854c800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8544077940 0x7f85441a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:52.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.505+0000 7f854c800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8544078290 0x7f85441a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:52.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.505+0000 7f854c800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544114740 0x7f85441a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:52.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.505+0000 7f854c800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f854411a080 con 0x7f8544114740 2024-09-21T00:21:52.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.505+0000 7f854c800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8544119f00 con 0x7f8544078290 2024-09-21T00:21:52.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.505+0000 7f854c800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f854411a200 con 0x7f8544077940 2024-09-21T00:21:52.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.505+0000 7f8549e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8544077940 0x7f85441a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:52.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.505+0000 7f854a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544114740 0x7f85441a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:52.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.505+0000 7f8549400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8544078290 0x7f85441a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:52.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.505+0000 7f8549e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8544077940 0x7f85441a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:45388/0 (socket says 172.21.15.27:45388) 2024-09-21T00:21:52.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.505+0000 7f8549e00640 1 -- 172.21.15.27:0/3120842017 learned_addr learned my addr 172.21.15.27:0/3120842017 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:52.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.506+0000 7f8549e00640 1 -- 172.21.15.27:0/3120842017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8544078290 msgr2=0x7f85441a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:52.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.506+0000 7f8549e00640 1 --2- 172.21.15.27:0/3120842017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8544078290 0x7f85441a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:52.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.506+0000 7f8549e00640 1 -- 172.21.15.27:0/3120842017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544114740 msgr2=0x7f85441a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:52.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.506+0000 7f8549e00640 1 --2- 172.21.15.27:0/3120842017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544114740 0x7f85441a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:52.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.506+0000 7f8549e00640 1 -- 172.21.15.27:0/3120842017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85441a6aa0 con 0x7f8544077940 2024-09-21T00:21:52.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.506+0000 7f8549e00640 1 --2- 172.21.15.27:0/3120842017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8544077940 0x7f85441a0b30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f853800e9f0 tx=0x7f853800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:52.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.506+0000 7f853ea00640 1 -- 172.21.15.27:0/3120842017 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853800cd80 con 0x7f8544077940 2024-09-21T00:21:52.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.506+0000 7f853ea00640 1 -- 172.21.15.27:0/3120842017 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85380054d0 con 0x7f8544077940 2024-09-21T00:21:52.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.506+0000 7f854c800640 1 -- 172.21.15.27:0/3120842017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85441a6d90 con 0x7f8544077940 2024-09-21T00:21:52.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.506+0000 7f853ea00640 1 -- 172.21.15.27:0/3120842017 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538010690 con 0x7f8544077940 2024-09-21T00:21:52.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.507+0000 7f854c800640 1 -- 172.21.15.27:0/3120842017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85441ad4f0 con 0x7f8544077940 2024-09-21T00:21:52.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.508+0000 7f854c800640 1 -- 172.21.15.27:0/3120842017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8544077d50 con 0x7f8544077940 2024-09-21T00:21:52.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.510+0000 7f853ea00640 1 -- 172.21.15.27:0/3120842017 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f8538004e30 con 0x7f8544077940 2024-09-21T00:21:52.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.512+0000 7f853ea00640 1 --2- 172.21.15.27:0/3120842017 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f852c0b65f0 0x7f852c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:52.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.512+0000 7f853ea00640 1 -- 172.21.15.27:0/3120842017 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f85380e7fc0 con 0x7f8544077940 2024-09-21T00:21:52.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.512+0000 7f8549400640 1 --2- 172.21.15.27:0/3120842017 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f852c0b65f0 0x7f852c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:52.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.513+0000 7f8549400640 1 --2- 172.21.15.27:0/3120842017 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f852c0b65f0 0x7f852c0b8b00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8534009230 tx=0x7f8534031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:52.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.513+0000 7f853ea00640 1 -- 172.21.15.27:0/3120842017 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f85380ae1e0 con 0x7f8544077940 2024-09-21T00:21:52.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.714+0000 7f854c800640 1 -- 172.21.15.27:0/3120842017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85441a1c60 con 0x7f8544077940 2024-09-21T00:21:52.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.716+0000 7f853ea00640 1 -- 172.21.15.27:0/3120842017 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25160 (secure 0 0 0) 0x7f85441a1c60 con 0x7f8544077940 2024-09-21T00:21:52.717 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:21:52.717 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","created":"2024-09-21T00:09:20.149938+0000","modified":"2024-09-21T00:20:26.442227+0000","last_up_change":"2024-09-21T00:17:43.978785+0000","last_in_change":"2024-09-21T00:17:21.325177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:13:32.718543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:18:33.992414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:18:36.817691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:18:46.707770+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63ccc094-62d5-446f-844d-1c60d41fda1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6803","nonce":354370090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6805","nonce":354370090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6809","nonce":354370090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6807","nonce":354370090}]},"public_addr":"172.21.15.27:6803/354370090","cluster_addr":"172.21.15.27:6805/354370090","heartbeat_back_addr":"172.21.15.27:6809/354370090","heartbeat_front_addr":"172.21.15.27:6807/354370090","state":["exists","up"]},{"osd":1,"uuid":"e59a8ed4-3a37-4477-a60a-4abc33d49b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6801","nonce":922528621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6803","nonce":922528621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6807","nonce":922528621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6805","nonce":922528621}]},"public_addr":"172.21.15.89:6801/922528621","cluster_addr":"172.21.15.89:6803/922528621","heartbeat_back_addr":"172.21.15.89:6807/922528621","heartbeat_front_addr":"172.21.15.89:6805/922528621","state":["exists","up"]},{"osd":2,"uuid":"6058a875-d5b2-4459-9407-2fd190cb34a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6801","nonce":257636259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6803","nonce":257636259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6807","nonce":257636259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6805","nonce":257636259}]},"public_addr":"172.21.15.98:6801/257636259","cluster_addr":"172.21.15.98:6803/257636259","heartbeat_back_addr":"172.21.15.98:6807/257636259","heartbeat_front_addr":"172.21.15.98:6805/257636259","state":["exists","up"]},{"osd":3,"uuid":"dfcdaede-e51c-4985-92af-3e296435a36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6811","nonce":821088824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6813","nonce":821088824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6817","nonce":821088824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6815","nonce":821088824}]},"public_addr":"172.21.15.27:6811/821088824","cluster_addr":"172.21.15.27:6813/821088824","heartbeat_back_addr":"172.21.15.27:6817/821088824","heartbeat_front_addr":"172.21.15.27:6815/821088824","state":["exists","up"]},{"osd":4,"uuid":"7c4af037-8ab6-47c0-b62b-76bd775b3491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6809","nonce":4001038690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6811","nonce":4001038690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6815","nonce":4001038690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6813","nonce":4001038690}]},"public_addr":"172.21.15.89:6809/4001038690","cluster_addr":"172.21.15.89:6811/4001038690","heartbeat_back_addr":"172.21.15.89:6815/4001038690","heartbeat_front_addr":"172.21.15.89:6813/4001038690","state":["exists","up"]},{"osd":5,"uuid":"e996ed0d-88ad-4aa7-891f-dfade4e09cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6809","nonce":3029903562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6811","nonce":3029903562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6815","nonce":3029903562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6813","nonce":3029903562}]},"public_addr":"172.21.15.98:6809/3029903562","cluster_addr":"172.21.15.98:6811/3029903562","heartbeat_back_addr":"172.21.15.98:6815/3029903562","heartbeat_front_addr":"172.21.15.98:6813/3029903562","state":["exists","up"]},{"osd":6,"uuid":"cf080497-57c2-42d0-8c8f-1eed0110702d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6819","nonce":1286808857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6821","nonce":1286808857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6825","nonce":1286808857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6823","nonce":1286808857}]},"public_addr":"172.21.15.27:6819/1286808857","cluster_addr":"172.21.15.27:6821/1286808857","heartbeat_back_addr":"172.21.15.27:6825/1286808857","heartbeat_front_addr":"172.21.15.27:6823/1286808857","state":["exists","up"]},{"osd":7,"uuid":"ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6817","nonce":2479349546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6819","nonce":2479349546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6823","nonce":2479349546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6821","nonce":2479349546}]},"public_addr":"172.21.15.89:6817/2479349546","cluster_addr":"172.21.15.89:6819/2479349546","heartbeat_back_addr":"172.21.15.89:6823/2479349546","heartbeat_front_addr":"172.21.15.89:6821/2479349546","state":["exists","up"]},{"osd":8,"uuid":"0b558458-fc2d-4ed1-b72a-e2ea8cf811b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6817","nonce":2830381485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6819","nonce":2830381485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6823","nonce":2830381485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6821","nonce":2830381485}]},"public_addr":"172.21.15.98:6817/2830381485","cluster_addr":"172.21.15.98:6819/2830381485","heartbeat_back_addr":"172.21.15.98:6823/2830381485","heartbeat_front_addr":"172.21.15.98:6821/2830381485","state":["exists","up"]},{"osd":9,"uuid":"1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6827","nonce":3033977113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6829","nonce":3033977113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6833","nonce":3033977113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6831","nonce":3033977113}]},"public_addr":"172.21.15.27:6827/3033977113","cluster_addr":"172.21.15.27:6829/3033977113","heartbeat_back_addr":"172.21.15.27:6833/3033977113","heartbeat_front_addr":"172.21.15.27:6831/3033977113","state":["exists","up"]},{"osd":10,"uuid":"324dd6ce-68fe-4c76-959b-1c9fb0ae92a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6825","nonce":2370332225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6827","nonce":2370332225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6831","nonce":2370332225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6829","nonce":2370332225}]},"public_addr":"172.21.15.89:6825/2370332225","cluster_addr":"172.21.15.89:6827/2370332225","heartbeat_back_addr":"172.21.15.89:6831/2370332225","heartbeat_front_addr":"172.21.15.89:6829/2370332225","state":["exists","up"]},{"osd":11,"uuid":"5e34686e-350a-491e-b9d7-59174ea90944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6825","nonce":2795999064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6827","nonce":2795999064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6831","nonce":2795999064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6829","nonce":2795999064}]},"public_addr":"172.21.15.98:6825/2795999064","cluster_addr":"172.21.15.98:6827/2795999064","heartbeat_back_addr":"172.21.15.98:6831/2795999064","heartbeat_front_addr":"172.21.15.98:6829/2795999064","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:12:39.841958+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:06.684417+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:31.041378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:01.612108+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:28.592326+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:54.303714+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:25.523475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:52.523483+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:19.430503+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:50.599120+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:17.520452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:42.725550+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/502883064":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/831048090":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/3880378905":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/793991452":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/2077956479":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6800/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6801/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/1098432608":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6800/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/528019907":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/2173601508":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:6800/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/764776233":"2024-09-22T00:11:24.231101+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:21:52.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.718+0000 7f854c800640 1 -- 172.21.15.27:0/3120842017 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f852c0b65f0 msgr2=0x7f852c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:52.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.718+0000 7f854c800640 1 --2- 172.21.15.27:0/3120842017 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f852c0b65f0 0x7f852c0b8b00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8534009230 tx=0x7f8534031400 comp rx=0 tx=0).stop 2024-09-21T00:21:52.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.718+0000 7f854c800640 1 -- 172.21.15.27:0/3120842017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8544077940 msgr2=0x7f85441a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:52.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.718+0000 7f854c800640 1 --2- 172.21.15.27:0/3120842017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8544077940 0x7f85441a0b30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f853800e9f0 tx=0x7f853800edf0 comp rx=0 tx=0).stop 2024-09-21T00:21:52.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.719+0000 7f854c800640 1 -- 172.21.15.27:0/3120842017 shutdown_connections 2024-09-21T00:21:52.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.719+0000 7f854c800640 1 --2- 172.21.15.27:0/3120842017 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f852c0b65f0 0x7f852c0b8b00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:52.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.719+0000 7f854c800640 1 --2- 172.21.15.27:0/3120842017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544114740 0x7f85441a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:52.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.719+0000 7f854c800640 1 --2- 172.21.15.27:0/3120842017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8544078290 0x7f85441a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:52.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.719+0000 7f854c800640 1 --2- 172.21.15.27:0/3120842017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8544077940 0x7f85441a0b30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:52.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.719+0000 7f854c800640 1 -- 172.21.15.27:0/3120842017 >> 172.21.15.27:0/3120842017 conn(0x7f8544101810 msgr2=0x7f85441167f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:52.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.719+0000 7f854c800640 1 -- 172.21.15.27:0/3120842017 shutdown_connections 2024-09-21T00:21:52.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:52.719+0000 7f854c800640 1 -- 172.21.15.27:0/3120842017 wait complete. 2024-09-21T00:21:53.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:53 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3120842017' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:21:53.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:53 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3120842017' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:21:53.550 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:53 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3120842017' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:21:53.593 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph daemon mon.a --format=json config get run_dir 2024-09-21T00:21:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.725+0000 7fd4f6400640 1 -- 172.21.15.27:0/3812577459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4f0109240 msgr2=0x7fd4f0109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.725+0000 7fd4f6400640 1 --2- 172.21.15.27:0/3812577459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4f0109240 0x7fd4f0109650 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009860 tx=0x7fd4dc02a340 comp rx=0 tx=0).stop 2024-09-21T00:21:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.725+0000 7fd4f6400640 1 -- 172.21.15.27:0/3812577459 shutdown_connections 2024-09-21T00:21:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.725+0000 7fd4f6400640 1 --2- 172.21.15.27:0/3812577459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4f0109b90 0x7fd4f01103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.725+0000 7fd4f6400640 1 --2- 172.21.15.27:0/3812577459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4f0109240 0x7fd4f0109650 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.725+0000 7fd4f6400640 1 --2- 172.21.15.27:0/3812577459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4f0103910 0x7fd4f0103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.725+0000 7fd4f6400640 1 -- 172.21.15.27:0/3812577459 >> 172.21.15.27:0/3812577459 conn(0x7fd4f00ff390 msgr2=0x7fd4f01017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.725+0000 7fd4f6400640 1 -- 172.21.15.27:0/3812577459 shutdown_connections 2024-09-21T00:21:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.725+0000 7fd4f6400640 1 -- 172.21.15.27:0/3812577459 wait complete. 2024-09-21T00:21:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.726+0000 7fd4f6400640 1 Processor -- start 2024-09-21T00:21:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.726+0000 7fd4f6400640 1 -- start start 2024-09-21T00:21:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.726+0000 7fd4f6400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4f0103910 0x7fd4f019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.726+0000 7fd4f6400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4f0109240 0x7fd4f019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.727+0000 7fd4f6400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4f0109b90 0x7fd4f01a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.727+0000 7fd4f6400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f0113850 con 0x7fd4f0103910 2024-09-21T00:21:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.727+0000 7fd4f6400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01136d0 con 0x7fd4f0109b90 2024-09-21T00:21:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.727+0000 7fd4f6400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01139d0 con 0x7fd4f0109240 2024-09-21T00:21:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.727+0000 7fd4f5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4f0103910 0x7fd4f019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.727+0000 7fd4f5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4f0103910 0x7fd4f019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49044/0 (socket says 172.21.15.27:49044) 2024-09-21T00:21:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.727+0000 7fd4f5000640 1 -- 172.21.15.27:0/2173207988 learned_addr learned my addr 172.21.15.27:0/2173207988 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.727+0000 7fd4efe00640 1 --2- 172.21.15.27:0/2173207988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4f0109240 0x7fd4f019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.727+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/2173207988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4f0109b90 0x7fd4f01a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.727+0000 7fd4f5000640 1 -- 172.21.15.27:0/2173207988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4f0109240 msgr2=0x7fd4f019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.727+0000 7fd4f5000640 1 --2- 172.21.15.27:0/2173207988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4f0109240 0x7fd4f019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.727+0000 7fd4f5000640 1 -- 172.21.15.27:0/2173207988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4f0109b90 msgr2=0x7fd4f01a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.727+0000 7fd4f5000640 1 --2- 172.21.15.27:0/2173207988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4f0109b90 0x7fd4f01a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.727+0000 7fd4f5000640 1 -- 172.21.15.27:0/2173207988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f01a23e0 con 0x7fd4f0103910 2024-09-21T00:21:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.728+0000 7fd4f5000640 1 --2- 172.21.15.27:0/2173207988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4f0103910 0x7fd4f019c4b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400ea60 tx=0x7fd4e400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.728+0000 7fd4ed600640 1 -- 172.21.15.27:0/2173207988 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400cde0 con 0x7fd4f0103910 2024-09-21T00:21:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.728+0000 7fd4f6400640 1 -- 172.21.15.27:0/2173207988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f01a26d0 con 0x7fd4f0103910 2024-09-21T00:21:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.728+0000 7fd4ed600640 1 -- 172.21.15.27:0/2173207988 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e40035f0 con 0x7fd4f0103910 2024-09-21T00:21:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.728+0000 7fd4ed600640 1 -- 172.21.15.27:0/2173207988 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4005220 con 0x7fd4f0103910 2024-09-21T00:21:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.728+0000 7fd4f6400640 1 -- 172.21.15.27:0/2173207988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f01a8e60 con 0x7fd4f0103910 2024-09-21T00:21:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.731+0000 7fd4feabd740 1 Processor -- start 2024-09-21T00:21:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.732+0000 7fd4feabd740 1 -- start start 2024-09-21T00:21:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.732+0000 7fd4ed600640 1 -- 172.21.15.27:0/2173207988 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7fd4e40037a0 con 0x7fd4f0103910 2024-09-21T00:21:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.732+0000 7fd4feabd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240621b0 0x55d624406e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.732+0000 7fd4feabd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240392f0 0x55d624407350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.732+0000 7fd4feabd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240414d0 0x55d624407890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.732+0000 7fd4feabd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d62437c6b0 con 0x55d6240621b0 2024-09-21T00:21:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.732+0000 7fd4feabd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d62436bae0 con 0x55d6240392f0 2024-09-21T00:21:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.732+0000 7fd4feabd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d62436f8f0 con 0x55d6240414d0 2024-09-21T00:21:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.732+0000 7fd4f5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240621b0 0x55d624406e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.732+0000 7fd4f5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240621b0 0x55d624406e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49058/0 (socket says 172.21.15.27:49058) 2024-09-21T00:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.733+0000 7fd4f5a00640 1 -- 172.21.15.27:0/2464770782 learned_addr learned my addr 172.21.15.27:0/2464770782 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.733+0000 7fd4f5000640 1 --2- 172.21.15.27:0/2464770782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240392f0 0x55d624407350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.733+0000 7fd4efe00640 1 --2- 172.21.15.27:0/2464770782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240414d0 0x55d624407890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.733+0000 7fd4f5a00640 1 -- 172.21.15.27:0/2464770782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240414d0 msgr2=0x55d624407890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.733+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/2464770782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240414d0 0x55d624407890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.733+0000 7fd4f5a00640 1 -- 172.21.15.27:0/2464770782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240392f0 msgr2=0x55d624407350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.733+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/2464770782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240392f0 0x55d624407350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.733+0000 7fd4f5a00640 1 -- 172.21.15.27:0/2464770782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d624338780 con 0x55d6240621b0 2024-09-21T00:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.733+0000 7fd4f5000640 1 --2- 172.21.15.27:0/2464770782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240392f0 0x55d624407350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.733+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/2464770782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240621b0 0x55d624406e10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800e580 tx=0x7fd4e80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677bcd2743c5f4aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.734+0000 7fd4e2000640 1 -- 172.21.15.27:0/2464770782 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8003970 con 0x55d6240621b0 2024-09-21T00:21:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.734+0000 7fd4e2000640 1 -- 172.21.15.27:0/2464770782 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e800d830 con 0x55d6240621b0 2024-09-21T00:21:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.734+0000 7fd4e2000640 1 -- 172.21.15.27:0/2464770782 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80044b0 con 0x55d6240621b0 2024-09-21T00:21:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.734+0000 7fd4ed600640 1 --2- 172.21.15.27:0/2173207988 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd4d80b65e0 0x7fd4d80b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.734+0000 7fd4ed600640 1 -- 172.21.15.27:0/2173207988 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fd4e40e7cb0 con 0x7fd4f0103910 2024-09-21T00:21:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.734+0000 7fd4f5000640 1 --2- 172.21.15.27:0/2173207988 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd4d80b65e0 0x7fd4d80b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.735+0000 7fd4feabd740 1 -- 172.21.15.27:0/2464770782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240621b0 msgr2=0x55d624406e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.735+0000 7fd4feabd740 1 --2- 172.21.15.27:0/2464770782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240621b0 0x55d624406e10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800e580 tx=0x7fd4e80095c0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.735+0000 7fd4feabd740 1 -- 172.21.15.27:0/2464770782 shutdown_connections 2024-09-21T00:21:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.735+0000 7fd4feabd740 1 --2- 172.21.15.27:0/2464770782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240414d0 0x55d624407890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.735+0000 7fd4feabd740 1 --2- 172.21.15.27:0/2464770782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240392f0 0x55d624407350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.735+0000 7fd4feabd740 1 --2- 172.21.15.27:0/2464770782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240621b0 0x55d624406e10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.735+0000 7fd4feabd740 1 -- 172.21.15.27:0/2464770782 >> 172.21.15.27:0/2464770782 conn(0x55d624056300 msgr2=0x55d624404d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.735+0000 7fd4f5000640 1 --2- 172.21.15.27:0/2173207988 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd4d80b65e0 0x7fd4d80b8af0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400c760 tx=0x7fd4e40eb760 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.735+0000 7fd4feabd740 1 -- 172.21.15.27:0/2464770782 shutdown_connections 2024-09-21T00:21:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.735+0000 7fd4feabd740 1 -- 172.21.15.27:0/2464770782 wait complete. 2024-09-21T00:21:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.736+0000 7fd4feabd740 1 -- start start 2024-09-21T00:21:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.736+0000 7fd4feabd740 1 Processor -- start 2024-09-21T00:21:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.736+0000 7fd4feabd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240392f0 0x55d624412440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.736+0000 7fd4feabd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240414d0 0x55d624412980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.736+0000 7fd4feabd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240621b0 0x55d624412f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.736+0000 7fd4feabd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d6242a3ab0 con 0x55d6240392f0 2024-09-21T00:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.736+0000 7fd4feabd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d624312570 con 0x55d6240621b0 2024-09-21T00:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.736+0000 7fd4feabd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d624415590 con 0x55d6240414d0 2024-09-21T00:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.737+0000 7fd4feabd740 10 client.0 _ll_register_callbacks cb 0x55d624321100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.737+0000 7fd4f5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240392f0 0x55d624412440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.737+0000 7fd4f5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240392f0 0x55d624412440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49062/0 (socket says 172.21.15.27:49062) 2024-09-21T00:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.737+0000 7fd4f5a00640 1 -- 172.21.15.27:0/2773559944 learned_addr learned my addr 172.21.15.27:0/2773559944 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.737+0000 7fd4efe00640 1 --2- 172.21.15.27:0/2773559944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240414d0 0x55d624412980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.737+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/2773559944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240621b0 0x55d624412f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.737+0000 7fd4efe00640 1 -- 172.21.15.27:0/2773559944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240621b0 msgr2=0x55d624412f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.737+0000 7fd4efe00640 1 --2- 172.21.15.27:0/2773559944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240621b0 0x55d624412f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.737+0000 7fd4efe00640 1 -- 172.21.15.27:0/2773559944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240392f0 msgr2=0x55d624412440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.737+0000 7fd4efe00640 1 --2- 172.21.15.27:0/2773559944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240392f0 0x55d624412440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.737+0000 7fd4efe00640 1 -- 172.21.15.27:0/2773559944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6240cd790 con 0x55d6240414d0 2024-09-21T00:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.737+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/2773559944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240392f0 0x55d624412440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.738+0000 7fd4efe00640 1 --2- 172.21.15.27:0/2773559944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240414d0 0x55d624412980 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc02a860 tx=0x7fd4dc005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72e3d856cd92cc3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.738+0000 7fd4d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.738+0000 7fd4d6a00640 1 -- 172.21.15.27:0/2773559944 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc037040 con 0x55d6240414d0 2024-09-21T00:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.738+0000 7fd4feabd740 1 -- 172.21.15.27:0/2773559944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e800d830 con 0x55d6240414d0 2024-09-21T00:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.738+0000 7fd4d6a00640 1 -- 172.21.15.27:0/2773559944 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fd4dc030450 con 0x55d6240414d0 2024-09-21T00:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.738+0000 7fd4d6a00640 1 -- 172.21.15.27:0/2773559944 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd4dc0044d0 con 0x55d6240414d0 2024-09-21T00:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.738+0000 7fd4feabd740 10 client.25441 fetch_fsmap learned FSMap version 26 2024-09-21T00:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.738+0000 7fd4feabd740 1 -- 172.21.15.27:0/2773559944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d624146620 con 0x55d6240414d0 2024-09-21T00:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.739+0000 7fd4d6a00640 1 -- 172.21.15.27:0/2773559944 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e800d830 con 0x55d6240414d0 2024-09-21T00:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.739+0000 7fd4d6a00640 1 -- 172.21.15.27:0/2773559944 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd4dc03d450 con 0x55d6240414d0 2024-09-21T00:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.739+0000 7fd4feabd740 10 client.25441 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.739+0000 7fd4feabd740 10 client.25441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:21:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.739+0000 7fd4feabd740 20 client.25441 populate_metadata read hostname 'smithi027' 2024-09-21T00:21:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.739+0000 7fd4feabd740 1 --2- 172.21.15.27:0/2773559944 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d624020d80 0x55d6244245a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.739+0000 7fd4feabd740 4 client.25441 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:21:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.739+0000 7fd4feabd740 1 -- 172.21.15.27:0/2773559944 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d62437e600 con 0x55d624020d80 2024-09-21T00:21:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.739+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/2773559944 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d624020d80 0x55d6244245a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.740+0000 7fd4d6a00640 10 client.25441 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:21:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.740+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/2773559944 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d624020d80 0x55d6244245a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c19971bb7bb6dd server_cookie=1fbcb3183c31b0ca in_seq=0 out_seq=0 2024-09-21T00:21:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.741+0000 7fd4d6a00640 1 -- 172.21.15.27:0/2773559944 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d62437e600 con 0x55d624020d80 2024-09-21T00:21:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.741+0000 7fd4d6a00640 10 client.25441 handle_command_reply: tid=0 2024-09-21T00:21:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.741+0000 7fd4feabd740 1 client.25441 shutdown 2024-09-21T00:21:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.741+0000 7fd4feabd740 1 -- 172.21.15.27:0/2773559944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240414d0 msgr2=0x55d624412980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.741+0000 7fd4feabd740 1 --2- 172.21.15.27:0/2773559944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240414d0 0x55d624412980 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc02a860 tx=0x7fd4dc005d30 comp rx=0 tx=0).stop 2024-09-21T00:21:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.741+0000 7fd4feabd740 1 -- 172.21.15.27:0/2773559944 shutdown_connections 2024-09-21T00:21:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.741+0000 7fd4feabd740 1 --2- 172.21.15.27:0/2773559944 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d624020d80 0x55d6244245a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.741+0000 7fd4feabd740 1 --2- 172.21.15.27:0/2773559944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240621b0 0x55d624412f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.741+0000 7fd4feabd740 1 --2- 172.21.15.27:0/2773559944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240414d0 0x55d624412980 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.741+0000 7fd4d6a00640 0 client.25441 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:21:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.741+0000 7fd4feabd740 1 --2- 172.21.15.27:0/2773559944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240392f0 0x55d624412440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.741+0000 7fd4feabd740 1 -- 172.21.15.27:0/2773559944 >> 172.21.15.27:0/2773559944 conn(0x55d624056300 msgr2=0x55d624407e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.741+0000 7fd4feabd740 1 -- 172.21.15.27:0/2773559944 shutdown_connections 2024-09-21T00:21:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.741+0000 7fd4feabd740 1 -- 172.21.15.27:0/2773559944 wait complete. 2024-09-21T00:21:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.741+0000 7fd4feabd740 20 client.25441 trim_cache size 0 max 16384 2024-09-21T00:21:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.757+0000 7fd4feabd740 1 Processor -- start 2024-09-21T00:21:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.757+0000 7fd4feabd740 1 -- start start 2024-09-21T00:21:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.757+0000 7fd4feabd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240392f0 0x55d624408550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.757+0000 7fd4feabd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240414d0 0x55d62440f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.757+0000 7fd4feabd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240621b0 0x55d62440fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.757+0000 7fd4feabd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01b5820 con 0x55d6240414d0 2024-09-21T00:21:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.757+0000 7fd4feabd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01b30f0 con 0x55d6240392f0 2024-09-21T00:21:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.757+0000 7fd4feabd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01b0f40 con 0x55d6240621b0 2024-09-21T00:21:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.757+0000 7fd4efe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240414d0 0x55d62440f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.757+0000 7fd4f5a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240392f0 0x55d624408550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.757+0000 7fd4efe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240414d0 0x55d62440f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49068/0 (socket says 172.21.15.27:49068) 2024-09-21T00:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.757+0000 7fd4efe00640 1 -- 172.21.15.27:0/3965419503 learned_addr learned my addr 172.21.15.27:0/3965419503 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.757+0000 7fd4f5a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240392f0 0x55d624408550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58458/0 (socket says 172.21.15.27:58458) 2024-09-21T00:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.758+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/3965419503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240621b0 0x55d62440fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.758+0000 7fd4f5a00640 1 -- 172.21.15.27:0/3965419503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240621b0 msgr2=0x55d62440fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.758+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/3965419503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240621b0 0x55d62440fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.758+0000 7fd4f5a00640 1 -- 172.21.15.27:0/3965419503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240414d0 msgr2=0x55d62440f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.758+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/3965419503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240414d0 0x55d62440f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.758+0000 7fd4f5a00640 1 -- 172.21.15.27:0/3965419503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d62441e780 con 0x55d6240392f0 2024-09-21T00:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.758+0000 7fd4efe00640 1 --2- 172.21.15.27:0/3965419503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240414d0 0x55d62440f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.758+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/3965419503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240392f0 0x55d624408550 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800e010 tx=0x7fd4e8026040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=620fec8d7ea8763c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.758+0000 7fd4e1600640 1 -- 172.21.15.27:0/3965419503 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e802e040 con 0x55d6240392f0 2024-09-21T00:21:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.758+0000 7fd4e1600640 1 -- 172.21.15.27:0/3965419503 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e8005580 con 0x55d6240392f0 2024-09-21T00:21:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.758+0000 7fd4e1600640 1 -- 172.21.15.27:0/3965419503 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8022a50 con 0x55d6240392f0 2024-09-21T00:21:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.759+0000 7fd4feabd740 1 -- 172.21.15.27:0/3965419503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240392f0 msgr2=0x55d624408550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.759+0000 7fd4feabd740 1 --2- 172.21.15.27:0/3965419503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240392f0 0x55d624408550 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800e010 tx=0x7fd4e8026040 comp rx=0 tx=0).stop 2024-09-21T00:21:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.759+0000 7fd4feabd740 1 -- 172.21.15.27:0/3965419503 shutdown_connections 2024-09-21T00:21:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.759+0000 7fd4feabd740 1 --2- 172.21.15.27:0/3965419503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240621b0 0x55d62440fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.759+0000 7fd4feabd740 1 --2- 172.21.15.27:0/3965419503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240414d0 0x55d62440f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.759+0000 7fd4feabd740 1 --2- 172.21.15.27:0/3965419503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240392f0 0x55d624408550 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.759+0000 7fd4feabd740 1 -- 172.21.15.27:0/3965419503 >> 172.21.15.27:0/3965419503 conn(0x55d624056300 msgr2=0x55d624407e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.759+0000 7fd4feabd740 1 -- 172.21.15.27:0/3965419503 shutdown_connections 2024-09-21T00:21:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.760+0000 7fd4feabd740 1 -- 172.21.15.27:0/3965419503 wait complete. 2024-09-21T00:21:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.760+0000 7fd4feabd740 1 -- start start 2024-09-21T00:21:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.760+0000 7fd4feabd740 1 Processor -- start 2024-09-21T00:21:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4feabd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240392f0 0x55d62441b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4feabd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240414d0 0x55d624418610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4feabd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240621b0 0x55d624418d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4feabd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01b15b0 con 0x55d6240621b0 2024-09-21T00:21:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4feabd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01b2a20 con 0x55d6240414d0 2024-09-21T00:21:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4feabd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01b1c80 con 0x55d6240392f0 2024-09-21T00:21:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4feabd740 10 client.0 _ll_register_callbacks cb 0x7fd4e8009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:21:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4efe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240414d0 0x55d624418610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4efe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240414d0 0x55d624418610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58460/0 (socket says 172.21.15.27:58460) 2024-09-21T00:21:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4efe00640 1 -- 172.21.15.27:0/1961825689 learned_addr learned my addr 172.21.15.27:0/1961825689 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/1961825689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240392f0 0x55d62441b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/1961825689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240621b0 0x55d624418d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4efe00640 1 -- 172.21.15.27:0/1961825689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240392f0 msgr2=0x55d62441b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4efe00640 1 --2- 172.21.15.27:0/1961825689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240392f0 0x55d62441b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4efe00640 1 -- 172.21.15.27:0/1961825689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240621b0 msgr2=0x55d624418d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4efe00640 1 --2- 172.21.15.27:0/1961825689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240621b0 0x55d624418d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4efe00640 1 -- 172.21.15.27:0/1961825689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d62448df90 con 0x55d6240414d0 2024-09-21T00:21:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.761+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/1961825689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240392f0 0x55d62441b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:21:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.762+0000 7fd4efe00640 1 --2- 172.21.15.27:0/1961825689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240414d0 0x55d624418610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc03d8c0 tx=0x7fd4dc03dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e259ea3af0dd5eb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.762+0000 7fd4d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:21:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.762+0000 7fd4d6a00640 1 -- 172.21.15.27:0/1961825689 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc0044d0 con 0x55d6240414d0 2024-09-21T00:21:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.762+0000 7fd4feabd740 1 -- 172.21.15.27:0/1961825689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e8005580 con 0x55d6240414d0 2024-09-21T00:21:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.762+0000 7fd4d6a00640 1 -- 172.21.15.27:0/1961825689 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fd4dc030450 con 0x55d6240414d0 2024-09-21T00:21:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.762+0000 7fd4d6a00640 1 -- 172.21.15.27:0/1961825689 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd4dc03d660 con 0x55d6240414d0 2024-09-21T00:21:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.762+0000 7fd4feabd740 10 client.25466 fetch_fsmap learned FSMap version 26 2024-09-21T00:21:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.762+0000 7fd4feabd740 1 -- 172.21.15.27:0/1961825689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d62448f070 con 0x55d6240414d0 2024-09-21T00:21:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.762+0000 7fd4d6a00640 1 -- 172.21.15.27:0/1961825689 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e8005580 con 0x55d6240414d0 2024-09-21T00:21:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.763+0000 7fd4d6a00640 1 -- 172.21.15.27:0/1961825689 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd4dc004730 con 0x55d6240414d0 2024-09-21T00:21:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.763+0000 7fd4feabd740 10 client.25466 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:21:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.763+0000 7fd4feabd740 10 client.25466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:21:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.763+0000 7fd4feabd740 20 client.25466 populate_metadata read hostname 'smithi027' 2024-09-21T00:21:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.763+0000 7fd4feabd740 1 --2- 172.21.15.27:0/1961825689 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d623ff61c0 0x55d6244918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.763+0000 7fd4feabd740 4 client.25466 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:21:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.763+0000 7fd4feabd740 1 -- 172.21.15.27:0/1961825689 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d62437a0b0 con 0x55d623ff61c0 2024-09-21T00:21:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.763+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/1961825689 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d623ff61c0 0x55d6244918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.764+0000 7fd4f5a00640 1 --2- 172.21.15.27:0/1961825689 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d623ff61c0 0x55d6244918b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9ff9d767179e352 server_cookie=ee22ad2014a131a7 in_seq=0 out_seq=0 2024-09-21T00:21:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.764+0000 7fd4d6a00640 10 client.25466 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.777+0000 7fd4d6a00640 1 -- 172.21.15.27:0/1961825689 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d62437a0b0 con 0x55d623ff61c0 2024-09-21T00:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.777+0000 7fd4d6a00640 10 client.25466 handle_command_reply: tid=0 2024-09-21T00:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.778+0000 7fd4feabd740 1 client.25466 shutdown 2024-09-21T00:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.778+0000 7fd4feabd740 1 -- 172.21.15.27:0/1961825689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240414d0 msgr2=0x55d624418610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.778+0000 7fd4feabd740 1 --2- 172.21.15.27:0/1961825689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240414d0 0x55d624418610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc03d8c0 tx=0x7fd4dc03dcc0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.778+0000 7fd4feabd740 1 -- 172.21.15.27:0/1961825689 shutdown_connections 2024-09-21T00:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.778+0000 7fd4feabd740 1 --2- 172.21.15.27:0/1961825689 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d623ff61c0 0x55d6244918b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.778+0000 7fd4feabd740 1 --2- 172.21.15.27:0/1961825689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6240621b0 0x55d624418d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.778+0000 7fd4feabd740 1 --2- 172.21.15.27:0/1961825689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6240414d0 0x55d624418610 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.778+0000 7fd4feabd740 1 --2- 172.21.15.27:0/1961825689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6240392f0 0x55d62441b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.778+0000 7fd4d6a00640 0 client.25466 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:21:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.778+0000 7fd4feabd740 1 -- 172.21.15.27:0/1961825689 >> 172.21.15.27:0/1961825689 conn(0x55d624056300 msgr2=0x55d624407e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.778+0000 7fd4feabd740 1 -- 172.21.15.27:0/1961825689 shutdown_connections 2024-09-21T00:21:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.778+0000 7fd4feabd740 1 -- 172.21.15.27:0/1961825689 wait complete. 2024-09-21T00:21:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.778+0000 7fd4feabd740 20 client.25466 trim_cache size 0 max 16384 2024-09-21T00:21:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:21:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:21:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:21:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:21:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.779+0000 7fd4f6400640 1 -- 172.21.15.27:0/2173207988 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd4d80b65e0 msgr2=0x7fd4d80b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.779+0000 7fd4f6400640 1 --2- 172.21.15.27:0/2173207988 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd4d80b65e0 0x7fd4d80b8af0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400c760 tx=0x7fd4e40eb760 comp rx=0 tx=0).stop 2024-09-21T00:21:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.779+0000 7fd4f6400640 1 -- 172.21.15.27:0/2173207988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4f0103910 msgr2=0x7fd4f019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.779+0000 7fd4f6400640 1 --2- 172.21.15.27:0/2173207988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4f0103910 0x7fd4f019c4b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400ea60 tx=0x7fd4e400ef20 comp rx=0 tx=0).stop 2024-09-21T00:21:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.779+0000 7fd4f6400640 1 -- 172.21.15.27:0/2173207988 shutdown_connections 2024-09-21T00:21:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.779+0000 7fd4f6400640 1 --2- 172.21.15.27:0/2173207988 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd4d80b65e0 0x7fd4d80b8af0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.779+0000 7fd4f6400640 1 --2- 172.21.15.27:0/2173207988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4f0109b90 0x7fd4f01a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.779+0000 7fd4f6400640 1 --2- 172.21.15.27:0/2173207988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4f0109240 0x7fd4f019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.779+0000 7fd4f6400640 1 --2- 172.21.15.27:0/2173207988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4f0103910 0x7fd4f019c4b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.779+0000 7fd4f6400640 1 -- 172.21.15.27:0/2173207988 >> 172.21.15.27:0/2173207988 conn(0x7fd4f00ff390 msgr2=0x7fd4f0108900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.779+0000 7fd4f6400640 1 -- 172.21.15.27:0/2173207988 shutdown_connections 2024-09-21T00:21:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:21:53.779+0000 7fd4f6400640 1 -- 172.21.15.27:0/2173207988 wait complete. 2024-09-21T00:21:53.987 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:54.454 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:54 smithi027 ceph-mon[30745]: pgmap v392: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 72 KiB/s wr, 13 op/s 2024-09-21T00:21:54.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:54 smithi089 ceph-mon[34481]: pgmap v392: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 72 KiB/s wr, 13 op/s 2024-09-21T00:21:54.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:54 smithi098 ceph-mon[34725]: pgmap v392: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 72 KiB/s wr, 13 op/s 2024-09-21T00:21:55.519 INFO:teuthology.orchestra.run.smithi027.stdout:{"run_dir":"/var/run/ceph"} 2024-09-21T00:21:55.520 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-21T00:21:55.520 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 7864 2024-09-21T00:21:55.520 DEBUG:tasks.quiescer:Yielding 2024-09-21T00:21:55.521 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T00:21:55.537 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:21:55.537 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T00:21:55.538 INFO:tasks.workunit:timeout=3h 2024-09-21T00:21:55.538 INFO:tasks.workunit:cleanup=True 2024-09-21T00:21:55.538 DEBUG:teuthology.orchestra.run.smithi027:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T00:21:55.538 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:21:55.578 INFO:teuthology.orchestra.run.smithi027.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T00:21:55.578 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-21T00:21:55.578 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 54h/84d Inode: 1099511628283 Links: 2 2024-09-21T00:21:55.579 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:21:55.579 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-21T00:21:55.579 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-21 00:19:46.868970628 +0000 2024-09-21T00:21:55.579 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-21 00:19:46.868970628 +0000 2024-09-21T00:21:55.579 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-21 00:21:34.284627833 +0000 2024-09-21T00:21:55.579 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-21 00:19:46.868970628 +0000 2024-09-21T00:21:55.580 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T00:21:55.580 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T00:21:55.693 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:21:55.732 INFO:tasks.workunit.client.0.smithi027.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T00:21:55.946 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:56.455 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:56 smithi027 ceph-mon[30745]: pgmap v393: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T00:21:56.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:56 smithi089 ceph-mon[34481]: pgmap v393: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T00:21:56.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:56 smithi098 ceph-mon[34725]: pgmap v393: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T00:21:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.638+0000 7f0dbbe00640 1 -- 172.21.15.27:0/3739587573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1092f0 msgr2=0x7f0dbc109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.638+0000 7f0dbbe00640 1 --2- 172.21.15.27:0/3739587573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1092f0 0x7f0dbc109700 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009900 tx=0x7f0da802a410 comp rx=0 tx=0).stop 2024-09-21T00:21:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.638+0000 7f0dbbe00640 1 -- 172.21.15.27:0/3739587573 shutdown_connections 2024-09-21T00:21:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.638+0000 7f0dbbe00640 1 --2- 172.21.15.27:0/3739587573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dbc109c40 0x7f0dbc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.638+0000 7f0dbbe00640 1 --2- 172.21.15.27:0/3739587573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1092f0 0x7f0dbc109700 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.638+0000 7f0dbbe00640 1 --2- 172.21.15.27:0/3739587573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0dbc1039b0 0x7f0dbc103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.638+0000 7f0dbbe00640 1 -- 172.21.15.27:0/3739587573 >> 172.21.15.27:0/3739587573 conn(0x7f0dbc0ff400 msgr2=0x7f0dbc101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.639+0000 7f0dbbe00640 1 -- 172.21.15.27:0/3739587573 shutdown_connections 2024-09-21T00:21:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.639+0000 7f0dbbe00640 1 -- 172.21.15.27:0/3739587573 wait complete. 2024-09-21T00:21:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.640+0000 7f0dbbe00640 1 Processor -- start 2024-09-21T00:21:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.640+0000 7f0dbbe00640 1 -- start start 2024-09-21T00:21:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.640+0000 7f0dbbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0dbc1039b0 0x7f0dbc19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.640+0000 7f0dbbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1092f0 0x7f0dbc19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.640+0000 7f0dbbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dbc109c40 0x7f0dbc19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.640+0000 7f0dbbe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc113900 con 0x7f0dbc1092f0 2024-09-21T00:21:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.640+0000 7f0dbbe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc113780 con 0x7f0dbc109c40 2024-09-21T00:21:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.640+0000 7f0dbbe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc113a80 con 0x7f0dbc1039b0 2024-09-21T00:21:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.641+0000 7f0dba000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1092f0 0x7f0dbc19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.641+0000 7f0dba000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1092f0 0x7f0dbc19a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49090/0 (socket says 172.21.15.27:49090) 2024-09-21T00:21:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.641+0000 7f0dba000640 1 -- 172.21.15.27:0/1516963930 learned_addr learned my addr 172.21.15.27:0/1516963930 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.641+0000 7f0dbaa00640 1 --2- 172.21.15.27:0/1516963930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0dbc1039b0 0x7f0dbc19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.641+0000 7f0dbb400640 1 --2- 172.21.15.27:0/1516963930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dbc109c40 0x7f0dbc19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.641+0000 7f0dba000640 1 -- 172.21.15.27:0/1516963930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0dbc1039b0 msgr2=0x7f0dbc19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.641+0000 7f0dba000640 1 --2- 172.21.15.27:0/1516963930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0dbc1039b0 0x7f0dbc19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.641+0000 7f0dba000640 1 -- 172.21.15.27:0/1516963930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dbc109c40 msgr2=0x7f0dbc19fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.641+0000 7f0dba000640 1 --2- 172.21.15.27:0/1516963930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dbc109c40 0x7f0dbc19fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.641+0000 7f0dba000640 1 -- 172.21.15.27:0/1516963930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dbc1a0260 con 0x7f0dbc1092f0 2024-09-21T00:21:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.641+0000 7f0dbb400640 1 --2- 172.21.15.27:0/1516963930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dbc109c40 0x7f0dbc19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:21:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.642+0000 7f0dba000640 1 --2- 172.21.15.27:0/1516963930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1092f0 0x7f0dbc19a890 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0da8006010 tx=0x7f0da8004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.642+0000 7f0db3400640 1 -- 172.21.15.27:0/1516963930 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da803f030 con 0x7f0dbc1092f0 2024-09-21T00:21:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.642+0000 7f0dbbe00640 1 -- 172.21.15.27:0/1516963930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dbc1a04f0 con 0x7f0dbc1092f0 2024-09-21T00:21:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.642+0000 7f0db3400640 1 -- 172.21.15.27:0/1516963930 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0da8003aa0 con 0x7f0dbc1092f0 2024-09-21T00:21:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.642+0000 7f0db3400640 1 -- 172.21.15.27:0/1516963930 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8035070 con 0x7f0dbc1092f0 2024-09-21T00:21:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.642+0000 7f0dbbe00640 1 -- 172.21.15.27:0/1516963930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dbc1a6cb0 con 0x7f0dbc1092f0 2024-09-21T00:21:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.643+0000 7f0dbbe00640 1 -- 172.21.15.27:0/1516963930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dbc104c30 con 0x7f0dbc1092f0 2024-09-21T00:21:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.645+0000 7f0db3400640 1 -- 172.21.15.27:0/1516963930 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f0da8042030 con 0x7f0dbc1092f0 2024-09-21T00:21:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.647+0000 7f0db3400640 1 --2- 172.21.15.27:0/1516963930 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0da00b65f0 0x7f0da00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.647+0000 7f0db3400640 1 -- 172.21.15.27:0/1516963930 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f0da8107f40 con 0x7f0dbc1092f0 2024-09-21T00:21:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.647+0000 7f0dbaa00640 1 --2- 172.21.15.27:0/1516963930 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0da00b65f0 0x7f0da00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.648+0000 7f0dbaa00640 1 --2- 172.21.15.27:0/1516963930 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0da00b65f0 0x7f0da00b8b00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00bbe0 tx=0x7f0dac00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.652+0000 7f0db3400640 1 -- 172.21.15.27:0/1516963930 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0da80ce160 con 0x7f0dbc1092f0 2024-09-21T00:21:56.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.892+0000 7f0dbbe00640 1 -- 172.21.15.27:0/1516963930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dbc10ba30 con 0x7f0dbc1092f0 2024-09-21T00:21:56.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.892+0000 7f0db3400640 1 -- 172.21.15.27:0/1516963930 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11893 (secure 0 0 0) 0x7f0da80d2540 con 0x7f0dbc1092f0 2024-09-21T00:21:56.894 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:21:56.894 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":26,"btime":"2024-09-21T00:21:36:085073+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:21:35.264444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:21:56.894 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-09-21T00:21:56.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.895+0000 7f0dbbe00640 1 -- 172.21.15.27:0/1516963930 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0da00b65f0 msgr2=0x7f0da00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:56.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.895+0000 7f0dbbe00640 1 --2- 172.21.15.27:0/1516963930 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0da00b65f0 0x7f0da00b8b00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00bbe0 tx=0x7f0dac00b390 comp rx=0 tx=0).stop 2024-09-21T00:21:56.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.895+0000 7f0dbbe00640 1 -- 172.21.15.27:0/1516963930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1092f0 msgr2=0x7f0dbc19a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.895+0000 7f0dbbe00640 1 --2- 172.21.15.27:0/1516963930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1092f0 0x7f0dbc19a890 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0da8006010 tx=0x7f0da8004060 comp rx=0 tx=0).stop 2024-09-21T00:21:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.895+0000 7f0dbbe00640 1 -- 172.21.15.27:0/1516963930 shutdown_connections 2024-09-21T00:21:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.895+0000 7f0dbbe00640 1 --2- 172.21.15.27:0/1516963930 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0da00b65f0 0x7f0da00b8b00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.895+0000 7f0dbbe00640 1 --2- 172.21.15.27:0/1516963930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dbc109c40 0x7f0dbc19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.895+0000 7f0dbbe00640 1 --2- 172.21.15.27:0/1516963930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1092f0 0x7f0dbc19a890 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.895+0000 7f0dbbe00640 1 --2- 172.21.15.27:0/1516963930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0dbc1039b0 0x7f0dbc19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.895+0000 7f0dbbe00640 1 -- 172.21.15.27:0/1516963930 >> 172.21.15.27:0/1516963930 conn(0x7f0dbc0ff400 msgr2=0x7f0dbc101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:56.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.896+0000 7f0dbbe00640 1 -- 172.21.15.27:0/1516963930 shutdown_connections 2024-09-21T00:21:56.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:56.896+0000 7f0dbbe00640 1 -- 172.21.15.27:0/1516963930 wait complete. 2024-09-21T00:21:57.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:57 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1516963930' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:21:57.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:57 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1516963930' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:21:57.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:57 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1516963930' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:21:57.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T00:18:39.809850+0000', 'modified': '2024-09-21T00:21:35.264444+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24761, 'mds_1': 24751, 'mds_2': 24727}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24703': {'gid': 24703, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6835/388589971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 388589971}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 388589971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24715': {'gid': 24715, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6837/3858073278', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 3858073278}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 3858073278}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24722': {'gid': 24722, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6837/3044996985', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6836', 'nonce': 3044996985}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 3044996985}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24727': {'gid': 24727, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.98:6839/3074699335', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 3074699335}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 3074699335}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24751': {'gid': 24751, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.27:6841/378707351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6840', 'nonce': 378707351}, {'type': 'v1', 'addr': '172.21.15.27:6841', 'nonce': 378707351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24761': {'gid': 24761, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.89:6839/2432693964', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 2432693964}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 2432693964}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24761, 'qdb_cluster': [24727, 24751, 24761]} 2024-09-21T00:21:57.747 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T00:21:57.747 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T00:21:58.133 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:21:58.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:21:58 smithi027 ceph-mon[30745]: pgmap v394: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-21T00:21:58.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:21:58 smithi089 ceph-mon[34481]: pgmap v394: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-21T00:21:58.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:21:58 smithi098 ceph-mon[34725]: pgmap v394: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-21T00:21:58.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.812+0000 7f5c33e00640 1 -- 172.21.15.27:0/3446354190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c34109c40 msgr2=0x7f5c34110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:58.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.812+0000 7f5c33e00640 1 --2- 172.21.15.27:0/3446354190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c34109c40 0x7f5c34110470 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009ca0 tx=0x7f5c2c02a8c0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.813+0000 7f5c33e00640 1 -- 172.21.15.27:0/3446354190 shutdown_connections 2024-09-21T00:21:58.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.813+0000 7f5c33e00640 1 --2- 172.21.15.27:0/3446354190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c34109c40 0x7f5c34110470 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.813+0000 7f5c33e00640 1 --2- 172.21.15.27:0/3446354190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c341092f0 0x7f5c34109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.813+0000 7f5c33e00640 1 --2- 172.21.15.27:0/3446354190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c341039b0 0x7f5c34103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.813+0000 7f5c33e00640 1 -- 172.21.15.27:0/3446354190 >> 172.21.15.27:0/3446354190 conn(0x7f5c340ff440 msgr2=0x7f5c34101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:58.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.813+0000 7f5c33e00640 1 -- 172.21.15.27:0/3446354190 shutdown_connections 2024-09-21T00:21:58.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.813+0000 7f5c33e00640 1 -- 172.21.15.27:0/3446354190 wait complete. 2024-09-21T00:21:58.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.814+0000 7f5c33e00640 1 Processor -- start 2024-09-21T00:21:58.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.814+0000 7f5c33e00640 1 -- start start 2024-09-21T00:21:58.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.814+0000 7f5c33e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c341039b0 0x7f5c3419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.814+0000 7f5c33e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c341092f0 0x7f5c3419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.814+0000 7f5c33e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c34109c40 0x7f5c341a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.814+0000 7f5c33e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34113900 con 0x7f5c341039b0 2024-09-21T00:21:58.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.814+0000 7f5c33e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34113780 con 0x7f5c341092f0 2024-09-21T00:21:58.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.814+0000 7f5c33e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34113a80 con 0x7f5c34109c40 2024-09-21T00:21:58.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.815+0000 7f5c32a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c341039b0 0x7f5c3419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.815+0000 7f5c32a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c341039b0 0x7f5c3419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49122/0 (socket says 172.21.15.27:49122) 2024-09-21T00:21:58.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.815+0000 7f5c32a00640 1 -- 172.21.15.27:0/192660510 learned_addr learned my addr 172.21.15.27:0/192660510 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:58.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.815+0000 7f5c2be00640 1 --2- 172.21.15.27:0/192660510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c341092f0 0x7f5c3419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.815+0000 7f5c33400640 1 --2- 172.21.15.27:0/192660510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c34109c40 0x7f5c341a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.815+0000 7f5c32a00640 1 -- 172.21.15.27:0/192660510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c34109c40 msgr2=0x7f5c341a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:58.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.815+0000 7f5c32a00640 1 --2- 172.21.15.27:0/192660510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c34109c40 0x7f5c341a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.815+0000 7f5c32a00640 1 -- 172.21.15.27:0/192660510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c341092f0 msgr2=0x7f5c3419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:58.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.815+0000 7f5c32a00640 1 --2- 172.21.15.27:0/192660510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c341092f0 0x7f5c3419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.815+0000 7f5c32a00640 1 -- 172.21.15.27:0/192660510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c341a24c0 con 0x7f5c341039b0 2024-09-21T00:21:58.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.815+0000 7f5c32a00640 1 --2- 172.21.15.27:0/192660510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c341039b0 0x7f5c3419c550 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400e9f0 tx=0x7f5c2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:58.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.816+0000 7f5c2b400640 1 -- 172.21.15.27:0/192660510 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2400cd80 con 0x7f5c341039b0 2024-09-21T00:21:58.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.816+0000 7f5c33e00640 1 -- 172.21.15.27:0/192660510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c341a27b0 con 0x7f5c341039b0 2024-09-21T00:21:58.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.816+0000 7f5c2b400640 1 -- 172.21.15.27:0/192660510 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c240108e0 con 0x7f5c341039b0 2024-09-21T00:21:58.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.816+0000 7f5c2b400640 1 -- 172.21.15.27:0/192660510 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24018980 con 0x7f5c341039b0 2024-09-21T00:21:58.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.816+0000 7f5c33e00640 1 -- 172.21.15.27:0/192660510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c341a8f10 con 0x7f5c341039b0 2024-09-21T00:21:58.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.819+0000 7f5c4062c740 1 Processor -- start 2024-09-21T00:21:58.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.819+0000 7f5c4062c740 1 -- start start 2024-09-21T00:21:58.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c4062c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021dbf60 0x55a102582670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c4062c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021b30a0 0x55a102582bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c4062c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021bb280 0x55a1025830f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c2b400640 1 -- 172.21.15.27:0/192660510 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f5c24010430 con 0x7f5c341039b0 2024-09-21T00:21:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c4062c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a1024f9d50 con 0x55a1021b30a0 2024-09-21T00:21:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c4062c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a1024e9180 con 0x55a1021dbf60 2024-09-21T00:21:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c4062c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a1024ecf90 con 0x55a1021bb280 2024-09-21T00:21:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c33400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021dbf60 0x55a102582670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c33400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021dbf60 0x55a102582670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58494/0 (socket says 172.21.15.27:58494) 2024-09-21T00:21:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c33400640 1 -- 172.21.15.27:0/868743618 learned_addr learned my addr 172.21.15.27:0/868743618 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c32a00640 1 --2- 172.21.15.27:0/868743618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021b30a0 0x55a102582bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c2be00640 1 --2- 172.21.15.27:0/868743618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021bb280 0x55a1025830f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c33400640 1 -- 172.21.15.27:0/868743618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021bb280 msgr2=0x55a1025830f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c33400640 1 --2- 172.21.15.27:0/868743618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021bb280 0x55a1025830f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c33400640 1 -- 172.21.15.27:0/868743618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021b30a0 msgr2=0x55a102582bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:58.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c33400640 1 --2- 172.21.15.27:0/868743618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021b30a0 0x55a102582bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c33400640 1 -- 172.21.15.27:0/868743618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1024b3a70 con 0x55a1021dbf60 2024-09-21T00:21:58.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.820+0000 7f5c32a00640 1 --2- 172.21.15.27:0/868743618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021b30a0 0x55a102582bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:21:58.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.821+0000 7f5c33400640 1 --2- 172.21.15.27:0/868743618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021dbf60 0x55a102582670 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c02f410 tx=0x7f5c2c02f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b5fee6840bc2a81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:58.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.821+0000 7f5c23e00640 1 -- 172.21.15.27:0/868743618 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c031cf0 con 0x55a1021dbf60 2024-09-21T00:21:58.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.821+0000 7f5c23e00640 1 -- 172.21.15.27:0/868743618 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c2c036030 con 0x55a1021dbf60 2024-09-21T00:21:58.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.821+0000 7f5c23e00640 1 -- 172.21.15.27:0/868743618 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c0042a0 con 0x55a1021dbf60 2024-09-21T00:21:58.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.822+0000 7f5c2b400640 1 --2- 172.21.15.27:0/192660510 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5c180b6670 0x7f5c180b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.822+0000 7f5c2b400640 1 -- 172.21.15.27:0/192660510 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f5c240e7ca0 con 0x7f5c341039b0 2024-09-21T00:21:58.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.822+0000 7f5c4062c740 1 -- 172.21.15.27:0/868743618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021dbf60 msgr2=0x55a102582670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:58.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.822+0000 7f5c4062c740 1 --2- 172.21.15.27:0/868743618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021dbf60 0x55a102582670 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c02f410 tx=0x7f5c2c02f440 comp rx=0 tx=0).stop 2024-09-21T00:21:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.822+0000 7f5c32a00640 1 --2- 172.21.15.27:0/192660510 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5c180b6670 0x7f5c180b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.822+0000 7f5c4062c740 1 -- 172.21.15.27:0/868743618 shutdown_connections 2024-09-21T00:21:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.822+0000 7f5c4062c740 1 --2- 172.21.15.27:0/868743618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021bb280 0x55a1025830f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.822+0000 7f5c4062c740 1 --2- 172.21.15.27:0/868743618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021b30a0 0x55a102582bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.822+0000 7f5c4062c740 1 --2- 172.21.15.27:0/868743618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021dbf60 0x55a102582670 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.822+0000 7f5c4062c740 1 -- 172.21.15.27:0/868743618 >> 172.21.15.27:0/868743618 conn(0x55a1021d0350 msgr2=0x55a102580560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.822+0000 7f5c4062c740 1 -- 172.21.15.27:0/868743618 shutdown_connections 2024-09-21T00:21:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.823+0000 7f5c4062c740 1 -- 172.21.15.27:0/868743618 wait complete. 2024-09-21T00:21:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.823+0000 7f5c32a00640 1 --2- 172.21.15.27:0/192660510 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5c180b6670 0x7f5c180b8b80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5c240ad570 tx=0x7f5c24018420 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.823+0000 7f5c4062c740 1 -- start start 2024-09-21T00:21:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.823+0000 7f5c4062c740 1 Processor -- start 2024-09-21T00:21:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.824+0000 7f5c4062c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021b30a0 0x55a10258e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.824+0000 7f5c4062c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021bb280 0x55a10258e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.824+0000 7f5c4062c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021dbf60 0x55a102599f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.824+0000 7f5c4062c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a10234a5f0 con 0x55a1021b30a0 2024-09-21T00:21:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.824+0000 7f5c4062c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a102484460 con 0x55a1021bb280 2024-09-21T00:21:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.824+0000 7f5c4062c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a102590c70 con 0x55a1021dbf60 2024-09-21T00:21:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.824+0000 7f5c4062c740 10 client.0 _ll_register_callbacks cb 0x55a102494210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:21:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.824+0000 7f5c2be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021bb280 0x55a10258e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.824+0000 7f5c33400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021b30a0 0x55a10258e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.824+0000 7f5c2be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021bb280 0x55a10258e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58500/0 (socket says 172.21.15.27:58500) 2024-09-21T00:21:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.824+0000 7f5c2be00640 1 -- 172.21.15.27:0/3973138814 learned_addr learned my addr 172.21.15.27:0/3973138814 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.825+0000 7f5c33400640 1 --2- 172.21.15.27:0/3973138814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021dbf60 0x55a102599f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.825+0000 7f5c33400640 1 -- 172.21.15.27:0/3973138814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021dbf60 msgr2=0x55a102599f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.825+0000 7f5c33400640 1 --2- 172.21.15.27:0/3973138814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021dbf60 0x55a102599f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.825+0000 7f5c33400640 1 -- 172.21.15.27:0/3973138814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021bb280 msgr2=0x55a10258e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.825+0000 7f5c33400640 1 --2- 172.21.15.27:0/3973138814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021bb280 0x55a10258e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.825+0000 7f5c33400640 1 -- 172.21.15.27:0/3973138814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a102247540 con 0x55a1021b30a0 2024-09-21T00:21:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.825+0000 7f5c2be00640 1 --2- 172.21.15.27:0/3973138814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021bb280 0x55a10258e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.825+0000 7f5c33400640 1 --2- 172.21.15.27:0/3973138814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021b30a0 0x55a10258e2b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c005e30 tx=0x7f5c2c003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397ef91d982ed94d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.825+0000 7f5c20c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:21:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.826+0000 7f5c20c00640 1 -- 172.21.15.27:0/3973138814 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c057060 con 0x55a1021b30a0 2024-09-21T00:21:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.826+0000 7f5c4062c740 1 -- 172.21.15.27:0/3973138814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a102486a70 con 0x55a1021b30a0 2024-09-21T00:21:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.826+0000 7f5c20c00640 1 -- 172.21.15.27:0/3973138814 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f5c2c031790 con 0x55a1021b30a0 2024-09-21T00:21:58.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.826+0000 7f5c20c00640 1 -- 172.21.15.27:0/3973138814 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5c2c05e050 con 0x55a1021b30a0 2024-09-21T00:21:58.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.826+0000 7f5c4062c740 10 client.15468 fetch_fsmap learned FSMap version 26 2024-09-21T00:21:58.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.826+0000 7f5c4062c740 1 -- 172.21.15.27:0/3973138814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a1024a16f0 con 0x55a1021b30a0 2024-09-21T00:21:58.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.826+0000 7f5c20c00640 1 -- 172.21.15.27:0/3973138814 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a102486a70 con 0x55a1021b30a0 2024-09-21T00:21:58.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.827+0000 7f5c20c00640 1 -- 172.21.15.27:0/3973138814 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5c2c043070 con 0x55a1021b30a0 2024-09-21T00:21:58.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.827+0000 7f5c4062c740 10 client.15468 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:21:58.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.827+0000 7f5c4062c740 10 client.15468 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T00:21:58.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.827+0000 7f5c4062c740 20 client.15468 populate_metadata read hostname 'smithi027' 2024-09-21T00:21:58.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.827+0000 7f5c4062c740 1 --2- 172.21.15.27:0/3973138814 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x55a10219a8e0 0x55a1025a0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.827+0000 7f5c4062c740 4 client.15468 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:21:58.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.827+0000 7f5c4062c740 1 -- 172.21.15.27:0/3973138814 --> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1024fbca0 con 0x55a10219a8e0 2024-09-21T00:21:58.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.827+0000 7f5c2be00640 1 --2- 172.21.15.27:0/3973138814 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x55a10219a8e0 0x55a1025a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.828+0000 7f5c2be00640 1 --2- 172.21.15.27:0/3973138814 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x55a10219a8e0 0x55a1025a0160 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d53690b68dfcda server_cookie=5a345e6adf76f90 in_seq=0 out_seq=0 2024-09-21T00:21:58.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.828+0000 7f5c20c00640 10 client.15468 ms_handle_connect on v2:172.21.15.98:6838/3074699335 2024-09-21T00:21:58.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.828+0000 7f5c20c00640 1 -- 172.21.15.27:0/3973138814 <== mds.2 v2:172.21.15.98:6838/3074699335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a1024fbca0 con 0x55a10219a8e0 2024-09-21T00:21:58.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.828+0000 7f5c20c00640 10 client.15468 handle_command_reply: tid=0 2024-09-21T00:21:58.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.828+0000 7f5c4062c740 1 client.15468 shutdown 2024-09-21T00:21:58.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.829+0000 7f5c4062c740 1 -- 172.21.15.27:0/3973138814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021b30a0 msgr2=0x55a10258e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:58.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.829+0000 7f5c4062c740 1 --2- 172.21.15.27:0/3973138814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021b30a0 0x55a10258e2b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c005e30 tx=0x7f5c2c003d70 comp rx=0 tx=0).stop 2024-09-21T00:21:58.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.829+0000 7f5c4062c740 1 -- 172.21.15.27:0/3973138814 shutdown_connections 2024-09-21T00:21:58.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.829+0000 7f5c4062c740 1 --2- 172.21.15.27:0/3973138814 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x55a10219a8e0 0x55a1025a0160 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.829+0000 7f5c4062c740 1 --2- 172.21.15.27:0/3973138814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021dbf60 0x55a102599f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.829+0000 7f5c4062c740 1 --2- 172.21.15.27:0/3973138814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021bb280 0x55a10258e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.829+0000 7f5c4062c740 1 --2- 172.21.15.27:0/3973138814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021b30a0 0x55a10258e2b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.829+0000 7f5c20c00640 0 client.15468 ms_handle_reset on v2:172.21.15.98:6838/3074699335 2024-09-21T00:21:58.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.829+0000 7f5c4062c740 1 -- 172.21.15.27:0/3973138814 >> 172.21.15.27:0/3973138814 conn(0x55a1021d0350 msgr2=0x55a102585800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:58.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.829+0000 7f5c4062c740 1 -- 172.21.15.27:0/3973138814 shutdown_connections 2024-09-21T00:21:58.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.829+0000 7f5c4062c740 1 -- 172.21.15.27:0/3973138814 wait complete. 2024-09-21T00:21:58.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.829+0000 7f5c4062c740 20 client.15468 trim_cache size 0 max 16384 2024-09-21T00:21:58.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.849+0000 7f5c4062c740 1 Processor -- start 2024-09-21T00:21:58.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.849+0000 7f5c4062c740 1 -- start start 2024-09-21T00:21:58.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.849+0000 7f5c4062c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021b30a0 0x55a10259fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.849+0000 7f5c4062c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021bb280 0x55a102599f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.849+0000 7f5c4062c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021dbf60 0x55a1025891f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.849+0000 7f5c4062c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a10259e4a0 con 0x55a1021bb280 2024-09-21T00:21:58.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.849+0000 7f5c4062c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a10259eb70 con 0x55a1021dbf60 2024-09-21T00:21:58.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.849+0000 7f5c4062c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a10259f240 con 0x55a1021b30a0 2024-09-21T00:21:58.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.849+0000 7f5c2be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021bb280 0x55a102599f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.849+0000 7f5c2be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021bb280 0x55a102599f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49164/0 (socket says 172.21.15.27:49164) 2024-09-21T00:21:58.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.849+0000 7f5c2be00640 1 -- 172.21.15.27:0/411330604 learned_addr learned my addr 172.21.15.27:0/411330604 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:58.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.849+0000 7f5c33400640 1 --2- 172.21.15.27:0/411330604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021b30a0 0x55a10259fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.849+0000 7f5c33400640 1 --2- 172.21.15.27:0/411330604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021dbf60 0x55a1025891f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.850+0000 7f5c2be00640 1 -- 172.21.15.27:0/411330604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021b30a0 msgr2=0x55a10259fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:58.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.850+0000 7f5c2be00640 1 --2- 172.21.15.27:0/411330604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021b30a0 0x55a10259fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.850+0000 7f5c2be00640 1 -- 172.21.15.27:0/411330604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021dbf60 msgr2=0x55a1025891f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:58.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.850+0000 7f5c2be00640 1 --2- 172.21.15.27:0/411330604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021dbf60 0x55a1025891f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.850+0000 7f5c2be00640 1 -- 172.21.15.27:0/411330604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a10258f370 con 0x55a1021bb280 2024-09-21T00:21:58.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.850+0000 7f5c33400640 1 --2- 172.21.15.27:0/411330604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021b30a0 0x55a10259fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:21:58.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.850+0000 7f5c2be00640 1 --2- 172.21.15.27:0/411330604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021bb280 0x55a102599f00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55a1025a0830 tx=0x7f5c1c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7346f3c0213f73fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:58.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.850+0000 7f5c23400640 1 -- 172.21.15.27:0/411330604 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c005020 con 0x55a1021bb280 2024-09-21T00:21:58.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.850+0000 7f5c23400640 1 -- 172.21.15.27:0/411330604 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c1c00f7f0 con 0x55a1021bb280 2024-09-21T00:21:58.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.851+0000 7f5c23400640 1 -- 172.21.15.27:0/411330604 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c004a00 con 0x55a1021bb280 2024-09-21T00:21:58.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.851+0000 7f5c4062c740 1 -- 172.21.15.27:0/411330604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021bb280 msgr2=0x55a102599f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:58.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.851+0000 7f5c4062c740 1 --2- 172.21.15.27:0/411330604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021bb280 0x55a102599f00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55a1025a0830 tx=0x7f5c1c005f90 comp rx=0 tx=0).stop 2024-09-21T00:21:58.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.851+0000 7f5c4062c740 1 -- 172.21.15.27:0/411330604 shutdown_connections 2024-09-21T00:21:58.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.851+0000 7f5c4062c740 1 --2- 172.21.15.27:0/411330604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021dbf60 0x55a1025891f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.851+0000 7f5c4062c740 1 --2- 172.21.15.27:0/411330604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021bb280 0x55a102599f00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.851+0000 7f5c4062c740 1 --2- 172.21.15.27:0/411330604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021b30a0 0x55a10259fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.851+0000 7f5c4062c740 1 -- 172.21.15.27:0/411330604 >> 172.21.15.27:0/411330604 conn(0x55a1021d0350 msgr2=0x55a10258e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:58.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.851+0000 7f5c4062c740 1 -- 172.21.15.27:0/411330604 shutdown_connections 2024-09-21T00:21:58.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.851+0000 7f5c4062c740 1 -- 172.21.15.27:0/411330604 wait complete. 2024-09-21T00:21:58.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.852+0000 7f5c4062c740 1 -- start start 2024-09-21T00:21:58.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.852+0000 7f5c4062c740 1 Processor -- start 2024-09-21T00:21:58.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.853+0000 7f5c4062c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021b30a0 0x55a1025a87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.853+0000 7f5c4062c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021bb280 0x55a1025a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.853+0000 7f5c4062c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021dbf60 0x55a1025a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.853+0000 7f5c4062c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a10259ffe0 con 0x55a1021bb280 2024-09-21T00:21:58.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.853+0000 7f5c4062c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a10259f910 con 0x55a1021b30a0 2024-09-21T00:21:58.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.853+0000 7f5c4062c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a1025a06b0 con 0x55a1021dbf60 2024-09-21T00:21:58.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.853+0000 7f5c4062c740 10 client.0 _ll_register_callbacks cb 0x7f5c2c031b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:21:58.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.853+0000 7f5c2be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021bb280 0x55a1025a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.853+0000 7f5c33400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021b30a0 0x55a1025a87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.853+0000 7f5c33400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021b30a0 0x55a1025a87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58504/0 (socket says 172.21.15.27:58504) 2024-09-21T00:21:58.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.853+0000 7f5c2be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021bb280 0x55a1025a8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49172/0 (socket says 172.21.15.27:49172) 2024-09-21T00:21:58.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.853+0000 7f5c33400640 1 -- 172.21.15.27:0/4148952611 learned_addr learned my addr 172.21.15.27:0/4148952611 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:21:58.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.853+0000 7f5c33400640 1 --2- 172.21.15.27:0/4148952611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021dbf60 0x55a1025a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.854+0000 7f5c2be00640 1 -- 172.21.15.27:0/4148952611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021dbf60 msgr2=0x55a1025a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:58.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.854+0000 7f5c2be00640 1 --2- 172.21.15.27:0/4148952611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021dbf60 0x55a1025a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.854+0000 7f5c2be00640 1 -- 172.21.15.27:0/4148952611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021b30a0 msgr2=0x55a1025a87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:21:58.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.854+0000 7f5c2be00640 1 --2- 172.21.15.27:0/4148952611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021b30a0 0x55a1025a87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.854+0000 7f5c2be00640 1 -- 172.21.15.27:0/4148952611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1025a2500 con 0x55a1021bb280 2024-09-21T00:21:58.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.854+0000 7f5c2be00640 1 --2- 172.21.15.27:0/4148952611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021bb280 0x55a1025a8d90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c014010 tx=0x7f5c1c005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50cec2a0a95dbd61 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:21:58.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.854+0000 7f5c20c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:21:58.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.854+0000 7f5c20c00640 1 -- 172.21.15.27:0/4148952611 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c0121d0 con 0x55a1021bb280 2024-09-21T00:21:58.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.854+0000 7f5c4062c740 1 -- 172.21.15.27:0/4148952611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c1c00f7f0 con 0x55a1021bb280 2024-09-21T00:21:58.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.854+0000 7f5c20c00640 1 -- 172.21.15.27:0/4148952611 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f5c1c004240 con 0x55a1021bb280 2024-09-21T00:21:58.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.855+0000 7f5c20c00640 1 -- 172.21.15.27:0/4148952611 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5c1c004440 con 0x55a1021bb280 2024-09-21T00:21:58.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.855+0000 7f5c4062c740 10 client.15474 fetch_fsmap learned FSMap version 26 2024-09-21T00:21:58.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.855+0000 7f5c4062c740 1 -- 172.21.15.27:0/4148952611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a1025a7f50 con 0x55a1021bb280 2024-09-21T00:21:58.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.855+0000 7f5c20c00640 1 -- 172.21.15.27:0/4148952611 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c1c00f7f0 con 0x55a1021bb280 2024-09-21T00:21:58.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.855+0000 7f5c20c00640 1 -- 172.21.15.27:0/4148952611 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5c1c027470 con 0x55a1021bb280 2024-09-21T00:21:58.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.855+0000 7f5c4062c740 10 client.15474 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:21:58.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.855+0000 7f5c4062c740 10 client.15474 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T00:21:58.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.855+0000 7f5c4062c740 20 client.15474 populate_metadata read hostname 'smithi027' 2024-09-21T00:21:58.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.855+0000 7f5c4062c740 1 --2- 172.21.15.27:0/4148952611 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x55a10216fc90 0x55a10260e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:21:58.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.855+0000 7f5c4062c740 4 client.15474 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:21:58.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.855+0000 7f5c4062c740 1 -- 172.21.15.27:0/4148952611 --> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] -- command(tid 0: {"prefix": "status"}) -- 0x55a1024f7750 con 0x55a10216fc90 2024-09-21T00:21:58.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.856+0000 7f5c33400640 1 --2- 172.21.15.27:0/4148952611 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x55a10216fc90 0x55a10260e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:21:58.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.856+0000 7f5c33400640 1 --2- 172.21.15.27:0/4148952611 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x55a10216fc90 0x55a10260e870 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a9acd70a7e1fbac server_cookie=d6c1004056ee80eb in_seq=0 out_seq=0 2024-09-21T00:21:58.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.856+0000 7f5c20c00640 10 client.15474 ms_handle_connect on v2:172.21.15.98:6838/3074699335 2024-09-21T00:21:58.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.856+0000 7f5c20c00640 1 -- 172.21.15.27:0/4148952611 <== mds.2 v2:172.21.15.98:6838/3074699335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a1024f7750 con 0x55a10216fc90 2024-09-21T00:21:58.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.856+0000 7f5c20c00640 10 client.15474 handle_command_reply: tid=0 2024-09-21T00:21:58.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.856+0000 7f5c4062c740 1 client.15474 shutdown 2024-09-21T00:21:58.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.857+0000 7f5c4062c740 1 -- 172.21.15.27:0/4148952611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021bb280 msgr2=0x55a1025a8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:58.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.857+0000 7f5c4062c740 1 --2- 172.21.15.27:0/4148952611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021bb280 0x55a1025a8d90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c014010 tx=0x7f5c1c005b40 comp rx=0 tx=0).stop 2024-09-21T00:21:58.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.857+0000 7f5c4062c740 1 -- 172.21.15.27:0/4148952611 shutdown_connections 2024-09-21T00:21:58.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.857+0000 7f5c4062c740 1 --2- 172.21.15.27:0/4148952611 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x55a10216fc90 0x55a10260e870 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.857+0000 7f5c4062c740 1 --2- 172.21.15.27:0/4148952611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1021dbf60 0x55a1025a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.857+0000 7f5c4062c740 1 --2- 172.21.15.27:0/4148952611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1021bb280 0x55a1025a8d90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.857+0000 7f5c4062c740 1 --2- 172.21.15.27:0/4148952611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1021b30a0 0x55a1025a87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.857+0000 7f5c4062c740 1 -- 172.21.15.27:0/4148952611 >> 172.21.15.27:0/4148952611 conn(0x55a1021d0350 msgr2=0x55a10257fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:58.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.857+0000 7f5c20c00640 0 client.15474 ms_handle_reset on v2:172.21.15.98:6838/3074699335 2024-09-21T00:21:58.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.857+0000 7f5c4062c740 1 -- 172.21.15.27:0/4148952611 shutdown_connections 2024-09-21T00:21:58.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.857+0000 7f5c4062c740 1 -- 172.21.15.27:0/4148952611 wait complete. 2024-09-21T00:21:58.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.857+0000 7f5c4062c740 20 client.15474 trim_cache size 0 max 16384 2024-09-21T00:21:58.861 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:21:58.861 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "9c21de84-77ad-11ef-baf5-efdc52797490", 2024-09-21T00:21:58.861 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-09-21T00:21:58.862 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24727, 2024-09-21T00:21:58.862 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T00:21:58.862 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T00:21:58.862 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T00:21:58.862 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 144.964572721, 2024-09-21T00:21:58.862 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 26, 2024-09-21T00:21:58.862 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T00:21:58.862 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 90, 2024-09-21T00:21:58.862 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 222.22836025000001 2024-09-21T00:21:58.862 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:21:58.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.858+0000 7f5c33e00640 1 -- 172.21.15.27:0/192660510 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5c180b6670 msgr2=0x7f5c180b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:58.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.858+0000 7f5c33e00640 1 --2- 172.21.15.27:0/192660510 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5c180b6670 0x7f5c180b8b80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5c240ad570 tx=0x7f5c24018420 comp rx=0 tx=0).stop 2024-09-21T00:21:58.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.858+0000 7f5c33e00640 1 -- 172.21.15.27:0/192660510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c341039b0 msgr2=0x7f5c3419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:21:58.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.858+0000 7f5c33e00640 1 --2- 172.21.15.27:0/192660510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c341039b0 0x7f5c3419c550 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400e9f0 tx=0x7f5c2400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.858+0000 7f5c33e00640 1 -- 172.21.15.27:0/192660510 shutdown_connections 2024-09-21T00:21:58.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.858+0000 7f5c33e00640 1 --2- 172.21.15.27:0/192660510 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5c180b6670 0x7f5c180b8b80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.858+0000 7f5c33e00640 1 --2- 172.21.15.27:0/192660510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c34109c40 0x7f5c341a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.858+0000 7f5c33e00640 1 --2- 172.21.15.27:0/192660510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c341092f0 0x7f5c3419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.858+0000 7f5c33e00640 1 --2- 172.21.15.27:0/192660510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c341039b0 0x7f5c3419c550 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:21:58.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.858+0000 7f5c33e00640 1 -- 172.21.15.27:0/192660510 >> 172.21.15.27:0/192660510 conn(0x7f5c340ff440 msgr2=0x7f5c34108930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:21:58.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.858+0000 7f5c33e00640 1 -- 172.21.15.27:0/192660510 shutdown_connections 2024-09-21T00:21:58.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:21:58.858+0000 7f5c33e00640 1 -- 172.21.15.27:0/192660510 wait complete. 2024-09-21T00:21:59.787 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell mds.a status 2024-09-21T00:22:00.172 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:22:00.465 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:00 smithi027 ceph-mon[30745]: pgmap v395: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-21T00:22:00.465 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:22:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:22:00.395+0000 7fbee5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T00:22:00.465 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:22:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:22:00.395+0000 7fbee5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T00:22:00.465 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:22:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:22:00.395+0000 7fbee5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T00:22:00.465 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:22:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:22:00.395+0000 7fbee5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T00:22:00.465 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:22:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:22:00.395+0000 7fbee5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T00:22:00.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:00 smithi089 ceph-mon[34481]: pgmap v395: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-21T00:22:00.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:00 smithi098 ceph-mon[34725]: pgmap v395: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-21T00:22:00.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.814+0000 7f9658200640 1 -- 172.21.15.27:0/3019574282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9650109df0 msgr2=0x7f9650110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:00.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.814+0000 7f9658200640 1 --2- 172.21.15.27:0/3019574282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9650109df0 0x7f9650110620 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9648009ca0 tx=0x7f964802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.814+0000 7f9658200640 1 -- 172.21.15.27:0/3019574282 shutdown_connections 2024-09-21T00:22:00.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.814+0000 7f9658200640 1 --2- 172.21.15.27:0/3019574282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9650109df0 0x7f9650110620 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.814+0000 7f9658200640 1 --2- 172.21.15.27:0/3019574282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96501094a0 0x7f96501098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.814+0000 7f9658200640 1 --2- 172.21.15.27:0/3019574282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9650103b60 0x7f9650103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.814+0000 7f9658200640 1 -- 172.21.15.27:0/3019574282 >> 172.21.15.27:0/3019574282 conn(0x7f965007b850 msgr2=0x7f965007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:00.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.814+0000 7f9658200640 1 -- 172.21.15.27:0/3019574282 shutdown_connections 2024-09-21T00:22:00.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.814+0000 7f9658200640 1 -- 172.21.15.27:0/3019574282 wait complete. 2024-09-21T00:22:00.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.816+0000 7f9658200640 1 Processor -- start 2024-09-21T00:22:00.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.816+0000 7f9658200640 1 -- start start 2024-09-21T00:22:00.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.816+0000 7f9658200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9650103b60 0x7f965019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.816+0000 7f9658200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96501094a0 0x7f965019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.816+0000 7f9658200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9650109df0 0x7f96501a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.816+0000 7f9658200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9650113ab0 con 0x7f9650109df0 2024-09-21T00:22:00.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.816+0000 7f9658200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9650113930 con 0x7f9650103b60 2024-09-21T00:22:00.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.816+0000 7f9658200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9650113c30 con 0x7f96501094a0 2024-09-21T00:22:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.816+0000 7f9654e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96501094a0 0x7f965019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.817+0000 7f9656200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9650109df0 0x7f96501a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.817+0000 7f9654e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96501094a0 0x7f965019cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:51672/0 (socket says 172.21.15.27:51672) 2024-09-21T00:22:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.817+0000 7f9654e00640 1 -- 172.21.15.27:0/1950701752 learned_addr learned my addr 172.21.15.27:0/1950701752 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.817+0000 7f9655800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9650103b60 0x7f965019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.817+0000 7f9654e00640 1 -- 172.21.15.27:0/1950701752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9650103b60 msgr2=0x7f965019c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.817+0000 7f9654e00640 1 --2- 172.21.15.27:0/1950701752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9650103b60 0x7f965019c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.817+0000 7f9654e00640 1 -- 172.21.15.27:0/1950701752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9650109df0 msgr2=0x7f96501a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.817+0000 7f9654e00640 1 --2- 172.21.15.27:0/1950701752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9650109df0 0x7f96501a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.817+0000 7f9654e00640 1 -- 172.21.15.27:0/1950701752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96501a25b0 con 0x7f96501094a0 2024-09-21T00:22:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.817+0000 7f9655800640 1 --2- 172.21.15.27:0/1950701752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9650103b60 0x7f965019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:22:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.817+0000 7f9654e00640 1 --2- 172.21.15.27:0/1950701752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96501094a0 0x7f965019cbe0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f963c00e9c0 tx=0x7f963c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.818+0000 7f964e000640 1 -- 172.21.15.27:0/1950701752 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c00ccc0 con 0x7f96501094a0 2024-09-21T00:22:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.818+0000 7f9658200640 1 -- 172.21.15.27:0/1950701752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96501a28a0 con 0x7f96501094a0 2024-09-21T00:22:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.818+0000 7f964e000640 1 -- 172.21.15.27:0/1950701752 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f963c0054d0 con 0x7f96501094a0 2024-09-21T00:22:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.818+0000 7f964e000640 1 -- 172.21.15.27:0/1950701752 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c010430 con 0x7f96501094a0 2024-09-21T00:22:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.818+0000 7f9658200640 1 -- 172.21.15.27:0/1950701752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96501a9000 con 0x7f96501094a0 2024-09-21T00:22:00.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.821+0000 7f965ee0a740 1 Processor -- start 2024-09-21T00:22:00.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.821+0000 7f965ee0a740 1 -- start start 2024-09-21T00:22:00.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f965ee0a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 0x562b604ea490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f965ee0a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 0x562b604ea9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f965ee0a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b604eaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f965ee0a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562b60461d50 con 0x562b60123280 2024-09-21T00:22:00.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f965ee0a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562b60451180 con 0x562b60143f60 2024-09-21T00:22:00.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f965ee0a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562b60454f90 con 0x562b6011b0a0 2024-09-21T00:22:00.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f9655800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 0x562b604ea9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f9656200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 0x562b604ea490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f9655800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 0x562b604ea9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:51686/0 (socket says 172.21.15.27:51686) 2024-09-21T00:22:00.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f9656200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 0x562b604ea490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58566/0 (socket says 172.21.15.27:58566) 2024-09-21T00:22:00.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f9655800640 1 -- 172.21.15.27:0/723132228 learned_addr learned my addr 172.21.15.27:0/723132228 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:00.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f9656200640 1 -- 172.21.15.27:0/723132228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 msgr2=0x562b604ea9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:00.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f9656200640 1 --2- 172.21.15.27:0/723132228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 0x562b604ea9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f9656200640 1 -- 172.21.15.27:0/723132228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 msgr2=0x562b604eaf10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:22:00.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f9656200640 1 --2- 172.21.15.27:0/723132228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b604eaf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f9656200640 1 -- 172.21.15.27:0/723132228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b6041ba70 con 0x562b60143f60 2024-09-21T00:22:00.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.822+0000 7f964e000640 1 -- 172.21.15.27:0/1950701752 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f963c005020 con 0x7f96501094a0 2024-09-21T00:22:00.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.823+0000 7f9656200640 1 --2- 172.21.15.27:0/723132228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 0x562b604ea490 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9648005e30 tx=0x7f964802f410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac2b82088820adc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:00.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.823+0000 7f9642a00640 1 -- 172.21.15.27:0/723132228 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648031cf0 con 0x562b60143f60 2024-09-21T00:22:00.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.823+0000 7f9642a00640 1 -- 172.21.15.27:0/723132228 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9648036030 con 0x562b60143f60 2024-09-21T00:22:00.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.823+0000 7f9642a00640 1 -- 172.21.15.27:0/723132228 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648004040 con 0x562b60143f60 2024-09-21T00:22:00.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.824+0000 7f964e000640 1 --2- 172.21.15.27:0/1950701752 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96380b65f0 0x7f96380b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.824+0000 7f965ee0a740 1 -- 172.21.15.27:0/723132228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 msgr2=0x562b604ea490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:00.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.824+0000 7f965ee0a740 1 --2- 172.21.15.27:0/723132228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 0x562b604ea490 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9648005e30 tx=0x7f964802f410 comp rx=0 tx=0).stop 2024-09-21T00:22:00.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.824+0000 7f965ee0a740 1 -- 172.21.15.27:0/723132228 shutdown_connections 2024-09-21T00:22:00.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.824+0000 7f965ee0a740 1 --2- 172.21.15.27:0/723132228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b604eaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.824+0000 7f965ee0a740 1 --2- 172.21.15.27:0/723132228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 0x562b604ea9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.824+0000 7f965ee0a740 1 --2- 172.21.15.27:0/723132228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 0x562b604ea490 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.824+0000 7f965ee0a740 1 -- 172.21.15.27:0/723132228 >> 172.21.15.27:0/723132228 conn(0x562b60138350 msgr2=0x562b604e8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:00.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.824+0000 7f964e000640 1 -- 172.21.15.27:0/1950701752 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f963c0e7d30 con 0x7f96501094a0 2024-09-21T00:22:00.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.825+0000 7f9655800640 1 --2- 172.21.15.27:0/1950701752 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96380b65f0 0x7f96380b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.825+0000 7f965ee0a740 1 -- 172.21.15.27:0/723132228 shutdown_connections 2024-09-21T00:22:00.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.825+0000 7f965ee0a740 1 -- 172.21.15.27:0/723132228 wait complete. 2024-09-21T00:22:00.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.825+0000 7f9655800640 1 --2- 172.21.15.27:0/1950701752 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96380b65f0 0x7f96380b8b00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x562b604e9110 tx=0x7f9644005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:00.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.825+0000 7f965ee0a740 1 -- start start 2024-09-21T00:22:00.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.826+0000 7f965ee0a740 1 Processor -- start 2024-09-21T00:22:00.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.826+0000 7f965ee0a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 0x562b604f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.826+0000 7f965ee0a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b604f6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.826+0000 7f965ee0a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 0x562b60501d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.826+0000 7f965ee0a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562b602b25f0 con 0x562b60123280 2024-09-21T00:22:00.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.826+0000 7f965ee0a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562b603ec460 con 0x562b60143f60 2024-09-21T00:22:00.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.826+0000 7f965ee0a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562b604f8a90 con 0x562b6011b0a0 2024-09-21T00:22:00.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.826+0000 7f965ee0a740 10 client.0 _ll_register_callbacks cb 0x562b603fc210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:00.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.827+0000 7f9654e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b604f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.827+0000 7f9655800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 0x562b604f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.827+0000 7f9654e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b604f6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49220/0 (socket says 172.21.15.27:49220) 2024-09-21T00:22:00.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.827+0000 7f9654e00640 1 -- 172.21.15.27:0/702451199 learned_addr learned my addr 172.21.15.27:0/702451199 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:00.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.827+0000 7f9656200640 1 --2- 172.21.15.27:0/702451199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 0x562b60501d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.827+0000 7f9654e00640 1 -- 172.21.15.27:0/702451199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 msgr2=0x562b604f5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:00.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.827+0000 7f9654e00640 1 --2- 172.21.15.27:0/702451199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 0x562b604f5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.827+0000 7f9654e00640 1 -- 172.21.15.27:0/702451199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 msgr2=0x562b60501d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:00.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.827+0000 7f9654e00640 1 --2- 172.21.15.27:0/702451199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 0x562b60501d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.827+0000 7f9654e00640 1 -- 172.21.15.27:0/702451199 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b601af540 con 0x562b60123280 2024-09-21T00:22:00.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.827+0000 7f9655800640 1 --2- 172.21.15.27:0/702451199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 0x562b604f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:22:00.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.828+0000 7f9654e00640 1 --2- 172.21.15.27:0/702451199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b604f6520 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f963c00c6a0 tx=0x7f963c0e56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e780396f087826af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:00.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.828+0000 7f9637400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:22:00.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.828+0000 7f9637400640 1 -- 172.21.15.27:0/702451199 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c0b2020 con 0x562b60123280 2024-09-21T00:22:00.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.828+0000 7f9637400640 1 -- 172.21.15.27:0/702451199 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f963c0aebe0 con 0x562b60123280 2024-09-21T00:22:00.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.828+0000 7f965ee0a740 1 -- 172.21.15.27:0/702451199 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562b603eea70 con 0x562b60123280 2024-09-21T00:22:00.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.828+0000 7f9637400640 1 -- 172.21.15.27:0/702451199 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f963c005680 con 0x562b60123280 2024-09-21T00:22:00.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.828+0000 7f965ee0a740 10 client.15489 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:00.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.828+0000 7f965ee0a740 1 -- 172.21.15.27:0/702451199 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b604096f0 con 0x562b60123280 2024-09-21T00:22:00.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.828+0000 7f9637400640 1 -- 172.21.15.27:0/702451199 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562b603eea70 con 0x562b60123280 2024-09-21T00:22:00.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.829+0000 7f9637400640 1 -- 172.21.15.27:0/702451199 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f963c0ac580 con 0x562b60123280 2024-09-21T00:22:00.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.829+0000 7f965ee0a740 10 client.15489 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:00.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.829+0000 7f965ee0a740 10 client.15489 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:22:00.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.829+0000 7f965ee0a740 20 client.15489 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:00.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.829+0000 7f965ee0a740 1 --2- 172.21.15.27:0/702451199 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x562b601028e0 0x562b605082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.829+0000 7f965ee0a740 4 client.15489 mds_command: new command op to 24751 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:22:00.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.829+0000 7f965ee0a740 1 -- 172.21.15.27:0/702451199 --> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b60463ca0 con 0x562b601028e0 2024-09-21T00:22:00.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.829+0000 7f9655800640 1 --2- 172.21.15.27:0/702451199 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x562b601028e0 0x562b605082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.830+0000 7f9655800640 1 --2- 172.21.15.27:0/702451199 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x562b601028e0 0x562b605082a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf8944f09bf81253 server_cookie=5b1d0f27b5293d9f in_seq=0 out_seq=0 2024-09-21T00:22:00.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.830+0000 7f9637400640 10 client.15489 ms_handle_connect on v2:172.21.15.27:6840/378707351 2024-09-21T00:22:00.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.831+0000 7f9637400640 1 -- 172.21.15.27:0/702451199 <== mds.1 v2:172.21.15.27:6840/378707351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562b60463ca0 con 0x562b601028e0 2024-09-21T00:22:00.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.831+0000 7f9637400640 10 client.15489 handle_command_reply: tid=0 2024-09-21T00:22:00.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.831+0000 7f965ee0a740 1 client.15489 shutdown 2024-09-21T00:22:00.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.831+0000 7f965ee0a740 1 -- 172.21.15.27:0/702451199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 msgr2=0x562b604f6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:00.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.831+0000 7f965ee0a740 1 --2- 172.21.15.27:0/702451199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b604f6520 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f963c00c6a0 tx=0x7f963c0e56b0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.831+0000 7f965ee0a740 1 -- 172.21.15.27:0/702451199 shutdown_connections 2024-09-21T00:22:00.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.831+0000 7f965ee0a740 1 --2- 172.21.15.27:0/702451199 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x562b601028e0 0x562b605082a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.831+0000 7f965ee0a740 1 --2- 172.21.15.27:0/702451199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 0x562b60501d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.831+0000 7f965ee0a740 1 --2- 172.21.15.27:0/702451199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b604f6520 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.831+0000 7f965ee0a740 1 --2- 172.21.15.27:0/702451199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 0x562b604f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.831+0000 7f965ee0a740 1 -- 172.21.15.27:0/702451199 >> 172.21.15.27:0/702451199 conn(0x562b60138350 msgr2=0x562b604ed670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:00.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.831+0000 7f9637400640 0 client.15489 ms_handle_reset on v2:172.21.15.27:6840/378707351 2024-09-21T00:22:00.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.831+0000 7f965ee0a740 1 -- 172.21.15.27:0/702451199 shutdown_connections 2024-09-21T00:22:00.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.831+0000 7f965ee0a740 1 -- 172.21.15.27:0/702451199 wait complete. 2024-09-21T00:22:00.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.831+0000 7f965ee0a740 20 client.15489 trim_cache size 0 max 16384 2024-09-21T00:22:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.850+0000 7f965ee0a740 1 Processor -- start 2024-09-21T00:22:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.850+0000 7f965ee0a740 1 -- start start 2024-09-21T00:22:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.851+0000 7f965ee0a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 0x562b60506e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.851+0000 7f965ee0a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b604f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.851+0000 7f965ee0a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 0x562b604f0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.851+0000 7f965ee0a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562b605065e0 con 0x562b60123280 2024-09-21T00:22:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.851+0000 7f965ee0a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562b60506cb0 con 0x562b60143f60 2024-09-21T00:22:00.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.851+0000 7f965ee0a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562b60507380 con 0x562b6011b0a0 2024-09-21T00:22:00.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.851+0000 7f9654e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b604f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.851+0000 7f9654e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b604f05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49226/0 (socket says 172.21.15.27:49226) 2024-09-21T00:22:00.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.851+0000 7f9654e00640 1 -- 172.21.15.27:0/235781292 learned_addr learned my addr 172.21.15.27:0/235781292 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:00.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.851+0000 7f9655800640 1 --2- 172.21.15.27:0/235781292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 0x562b60506e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.851+0000 7f9656200640 1 --2- 172.21.15.27:0/235781292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 0x562b604f0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.852+0000 7f9654e00640 1 -- 172.21.15.27:0/235781292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 msgr2=0x562b60506e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:00.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.852+0000 7f9654e00640 1 --2- 172.21.15.27:0/235781292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 0x562b60506e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.852+0000 7f9654e00640 1 -- 172.21.15.27:0/235781292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 msgr2=0x562b604f0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:00.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.852+0000 7f9654e00640 1 --2- 172.21.15.27:0/235781292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 0x562b604f0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.852+0000 7f9654e00640 1 -- 172.21.15.27:0/235781292 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b604f70a0 con 0x562b60123280 2024-09-21T00:22:00.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.852+0000 7f9656200640 1 --2- 172.21.15.27:0/235781292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 0x562b604f0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:22:00.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.852+0000 7f9654e00640 1 --2- 172.21.15.27:0/235781292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b604f05b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f963c005820 tx=0x7f963c009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96858a43d85d2999 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:00.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.852+0000 7f9642000640 1 -- 172.21.15.27:0/235781292 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c0b2070 con 0x562b60123280 2024-09-21T00:22:00.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.852+0000 7f9642000640 1 -- 172.21.15.27:0/235781292 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f963c0ad3c0 con 0x562b60123280 2024-09-21T00:22:00.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.852+0000 7f9642000640 1 -- 172.21.15.27:0/235781292 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c0adfb0 con 0x562b60123280 2024-09-21T00:22:00.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.853+0000 7f965ee0a740 1 -- 172.21.15.27:0/235781292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 msgr2=0x562b604f05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:00.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.853+0000 7f965ee0a740 1 --2- 172.21.15.27:0/235781292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b604f05b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f963c005820 tx=0x7f963c009730 comp rx=0 tx=0).stop 2024-09-21T00:22:00.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.853+0000 7f965ee0a740 1 -- 172.21.15.27:0/235781292 shutdown_connections 2024-09-21T00:22:00.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.853+0000 7f965ee0a740 1 --2- 172.21.15.27:0/235781292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b60143f60 0x562b604f0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.853+0000 7f965ee0a740 1 --2- 172.21.15.27:0/235781292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b604f05b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.853+0000 7f965ee0a740 1 --2- 172.21.15.27:0/235781292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b6011b0a0 0x562b60506e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.853+0000 7f965ee0a740 1 -- 172.21.15.27:0/235781292 >> 172.21.15.27:0/235781292 conn(0x562b60138350 msgr2=0x562b604e3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:00.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.854+0000 7f965ee0a740 1 -- 172.21.15.27:0/235781292 shutdown_connections 2024-09-21T00:22:00.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.854+0000 7f965ee0a740 1 -- 172.21.15.27:0/235781292 wait complete. 2024-09-21T00:22:00.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.854+0000 7f965ee0a740 1 -- start start 2024-09-21T00:22:00.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.854+0000 7f965ee0a740 1 Processor -- start 2024-09-21T00:22:00.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.855+0000 7f965ee0a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b6011b0a0 0x562b605098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.855+0000 7f965ee0a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b60509e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.855+0000 7f965ee0a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b60143f60 0x562b60571760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.855+0000 7f965ee0a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562b60508120 con 0x562b60123280 2024-09-21T00:22:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.855+0000 7f965ee0a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562b60507a50 con 0x562b6011b0a0 2024-09-21T00:22:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.855+0000 7f965ee0a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562b605087f0 con 0x562b60143f60 2024-09-21T00:22:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.855+0000 7f965ee0a740 10 client.0 _ll_register_callbacks cb 0x7f963c0ad200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.855+0000 7f9654e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b60509e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.855+0000 7f9656200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b60143f60 0x562b60571760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.855+0000 7f9655800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b6011b0a0 0x562b605098d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.855+0000 7f9654e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b60509e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49240/0 (socket says 172.21.15.27:49240) 2024-09-21T00:22:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.855+0000 7f9654e00640 1 -- 172.21.15.27:0/3535483741 learned_addr learned my addr 172.21.15.27:0/3535483741 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.855+0000 7f9656200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b60143f60 0x562b60571760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:51702/0 (socket says 172.21.15.27:51702) 2024-09-21T00:22:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.856+0000 7f9655800640 1 -- 172.21.15.27:0/3535483741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b60143f60 msgr2=0x562b60571760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.856+0000 7f9655800640 1 --2- 172.21.15.27:0/3535483741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b60143f60 0x562b60571760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.856+0000 7f9655800640 1 -- 172.21.15.27:0/3535483741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 msgr2=0x562b60509e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.856+0000 7f9655800640 1 --2- 172.21.15.27:0/3535483741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b60509e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.856+0000 7f9655800640 1 -- 172.21.15.27:0/3535483741 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b6050a600 con 0x562b6011b0a0 2024-09-21T00:22:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.856+0000 7f9654e00640 1 --2- 172.21.15.27:0/3535483741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b60509e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:22:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.856+0000 7f9655800640 1 --2- 172.21.15.27:0/3535483741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b6011b0a0 0x562b605098d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x562b605089c0 tx=0x7f96440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d4465067578eda5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.856+0000 7f9637400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:22:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.856+0000 7f9637400640 1 -- 172.21.15.27:0/3535483741 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644036060 con 0x562b6011b0a0 2024-09-21T00:22:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.856+0000 7f9637400640 1 -- 172.21.15.27:0/3535483741 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f964400f950 con 0x562b6011b0a0 2024-09-21T00:22:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.856+0000 7f965ee0a740 1 -- 172.21.15.27:0/3535483741 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f963c0ad3c0 con 0x562b6011b0a0 2024-09-21T00:22:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.857+0000 7f9637400640 1 -- 172.21.15.27:0/3535483741 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f964403d050 con 0x562b6011b0a0 2024-09-21T00:22:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.857+0000 7f965ee0a740 10 client.25514 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.857+0000 7f965ee0a740 1 -- 172.21.15.27:0/3535483741 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b60572cb0 con 0x562b6011b0a0 2024-09-21T00:22:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.857+0000 7f9637400640 1 -- 172.21.15.27:0/3535483741 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f963c0ad3c0 con 0x562b6011b0a0 2024-09-21T00:22:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.857+0000 7f9637400640 1 -- 172.21.15.27:0/3535483741 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f964402bb80 con 0x562b6011b0a0 2024-09-21T00:22:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.857+0000 7f965ee0a740 10 client.25514 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.857+0000 7f965ee0a740 10 client.25514 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:22:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.857+0000 7f965ee0a740 20 client.25514 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.857+0000 7f965ee0a740 1 --2- 172.21.15.27:0/3535483741 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x562b600d7c90 0x562b60576510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.857+0000 7f965ee0a740 4 client.25514 mds_command: new command op to 24751 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:22:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.857+0000 7f965ee0a740 1 -- 172.21.15.27:0/3535483741 --> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] -- command(tid 0: {"prefix": "status"}) -- 0x562b6045f750 con 0x562b600d7c90 2024-09-21T00:22:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.858+0000 7f9654e00640 1 --2- 172.21.15.27:0/3535483741 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x562b600d7c90 0x562b60576510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.858+0000 7f9654e00640 1 --2- 172.21.15.27:0/3535483741 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x562b600d7c90 0x562b60576510 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0b7309ef2d87f7b server_cookie=6d264efaa77ebc1d in_seq=0 out_seq=0 2024-09-21T00:22:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.858+0000 7f9637400640 10 client.25514 ms_handle_connect on v2:172.21.15.27:6840/378707351 2024-09-21T00:22:00.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.859+0000 7f9637400640 1 -- 172.21.15.27:0/3535483741 <== mds.1 v2:172.21.15.27:6840/378707351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562b6045f750 con 0x562b600d7c90 2024-09-21T00:22:00.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.859+0000 7f9637400640 10 client.25514 handle_command_reply: tid=0 2024-09-21T00:22:00.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.859+0000 7f965ee0a740 1 client.25514 shutdown 2024-09-21T00:22:00.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.859+0000 7f965ee0a740 1 -- 172.21.15.27:0/3535483741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b6011b0a0 msgr2=0x562b605098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:00.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.859+0000 7f965ee0a740 1 --2- 172.21.15.27:0/3535483741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b6011b0a0 0x562b605098d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x562b605089c0 tx=0x7f96440046c0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.859+0000 7f965ee0a740 1 -- 172.21.15.27:0/3535483741 shutdown_connections 2024-09-21T00:22:00.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.859+0000 7f965ee0a740 1 --2- 172.21.15.27:0/3535483741 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x562b600d7c90 0x562b60576510 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.859+0000 7f965ee0a740 1 --2- 172.21.15.27:0/3535483741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562b60143f60 0x562b60571760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.859+0000 7f965ee0a740 1 --2- 172.21.15.27:0/3535483741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b60123280 0x562b60509e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.859+0000 7f965ee0a740 1 --2- 172.21.15.27:0/3535483741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b6011b0a0 0x562b605098d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.859+0000 7f965ee0a740 1 -- 172.21.15.27:0/3535483741 >> 172.21.15.27:0/3535483741 conn(0x562b60138350 msgr2=0x562b604f0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:00.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.859+0000 7f9637400640 0 client.25514 ms_handle_reset on v2:172.21.15.27:6840/378707351 2024-09-21T00:22:00.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.859+0000 7f965ee0a740 1 -- 172.21.15.27:0/3535483741 shutdown_connections 2024-09-21T00:22:00.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.859+0000 7f965ee0a740 1 -- 172.21.15.27:0/3535483741 wait complete. 2024-09-21T00:22:00.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.859+0000 7f965ee0a740 20 client.25514 trim_cache size 0 max 16384 2024-09-21T00:22:00.868 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:22:00.869 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "9c21de84-77ad-11ef-baf5-efdc52797490", 2024-09-21T00:22:00.869 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-09-21T00:22:00.869 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24751, 2024-09-21T00:22:00.869 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T00:22:00.869 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T00:22:00.869 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T00:22:00.869 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 147.978172876, 2024-09-21T00:22:00.869 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 26, 2024-09-21T00:22:00.870 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T00:22:00.870 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 90, 2024-09-21T00:22:00.870 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 216.238090962 2024-09-21T00:22:00.870 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:22:00.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.860+0000 7f9658200640 1 -- 172.21.15.27:0/1950701752 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96380b65f0 msgr2=0x7f96380b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:00.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.860+0000 7f9658200640 1 --2- 172.21.15.27:0/1950701752 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96380b65f0 0x7f96380b8b00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x562b604e9110 tx=0x7f9644005f90 comp rx=0 tx=0).stop 2024-09-21T00:22:00.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.860+0000 7f9658200640 1 -- 172.21.15.27:0/1950701752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96501094a0 msgr2=0x7f965019cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:00.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.860+0000 7f9658200640 1 --2- 172.21.15.27:0/1950701752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96501094a0 0x7f965019cbe0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f963c00e9c0 tx=0x7f963c00ee80 comp rx=0 tx=0).stop 2024-09-21T00:22:00.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.860+0000 7f9658200640 1 -- 172.21.15.27:0/1950701752 shutdown_connections 2024-09-21T00:22:00.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.860+0000 7f9658200640 1 --2- 172.21.15.27:0/1950701752 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96380b65f0 0x7f96380b8b00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.860+0000 7f9658200640 1 --2- 172.21.15.27:0/1950701752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9650109df0 0x7f96501a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.860+0000 7f9658200640 1 --2- 172.21.15.27:0/1950701752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96501094a0 0x7f965019cbe0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.860+0000 7f9658200640 1 --2- 172.21.15.27:0/1950701752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9650103b60 0x7f965019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:00.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.860+0000 7f9658200640 1 -- 172.21.15.27:0/1950701752 >> 172.21.15.27:0/1950701752 conn(0x7f965007b850 msgr2=0x7f96501072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:00.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.860+0000 7f9658200640 1 -- 172.21.15.27:0/1950701752 shutdown_connections 2024-09-21T00:22:00.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:00.860+0000 7f9658200640 1 -- 172.21.15.27:0/1950701752 wait complete. 2024-09-21T00:22:01.849 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T00:22:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:22:02.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:02 smithi089 ceph-mon[34481]: pgmap v396: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-21T00:22:02.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:02 smithi089 ceph-mon[34481]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-09-21T00:22:02.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:02 smithi098 ceph-mon[34725]: pgmap v396: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-21T00:22:02.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:02 smithi098 ceph-mon[34725]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-09-21T00:22:02.536 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:02 smithi027 ceph-mon[30745]: pgmap v396: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-21T00:22:02.536 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:02 smithi027 ceph-mon[30745]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-09-21T00:22:03.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.032+0000 7fe545200640 1 -- 172.21.15.27:0/3923876560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe540102a60 msgr2=0x7fe540102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:03.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.032+0000 7fe545200640 1 --2- 172.21.15.27:0/3923876560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe540102a60 0x7fe540102e70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe530009ca0 tx=0x7fe53002a8c0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.032+0000 7fe545200640 1 -- 172.21.15.27:0/3923876560 shutdown_connections 2024-09-21T00:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.032+0000 7fe545200640 1 --2- 172.21.15.27:0/3923876560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe540108cf0 0x7fe54010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.032+0000 7fe545200640 1 --2- 172.21.15.27:0/3923876560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5401083a0 0x7fe5401087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.032+0000 7fe545200640 1 --2- 172.21.15.27:0/3923876560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe540102a60 0x7fe540102e70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.032+0000 7fe545200640 1 -- 172.21.15.27:0/3923876560 >> 172.21.15.27:0/3923876560 conn(0x7fe540079760 msgr2=0x7fe540079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.032+0000 7fe545200640 1 -- 172.21.15.27:0/3923876560 shutdown_connections 2024-09-21T00:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.032+0000 7fe545200640 1 -- 172.21.15.27:0/3923876560 wait complete. 2024-09-21T00:22:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.033+0000 7fe545200640 1 Processor -- start 2024-09-21T00:22:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.033+0000 7fe545200640 1 -- start start 2024-09-21T00:22:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.033+0000 7fe545200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe540102a60 0x7fe5400ff650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.033+0000 7fe545200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5401083a0 0x7fe5400ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.033+0000 7fe545200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe540108cf0 0x7fe5401000d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.034+0000 7fe545200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401129d0 con 0x7fe5401083a0 2024-09-21T00:22:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.034+0000 7fe545200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe540112850 con 0x7fe540102a60 2024-09-21T00:22:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.034+0000 7fe545200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe540112b50 con 0x7fe540108cf0 2024-09-21T00:22:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.034+0000 7fe53de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe540102a60 0x7fe5400ff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.034+0000 7fe535400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5401083a0 0x7fe5400ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.034+0000 7fe53de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe540102a60 0x7fe5400ff650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:46510/0 (socket says 172.21.15.27:46510) 2024-09-21T00:22:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.034+0000 7fe53de00640 1 -- 172.21.15.27:0/2229643796 learned_addr learned my addr 172.21.15.27:0/2229643796 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.034+0000 7fe53e800640 1 --2- 172.21.15.27:0/2229643796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe540108cf0 0x7fe5401000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.034+0000 7fe535400640 1 -- 172.21.15.27:0/2229643796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe540108cf0 msgr2=0x7fe5401000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.034+0000 7fe535400640 1 --2- 172.21.15.27:0/2229643796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe540108cf0 0x7fe5401000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.034+0000 7fe535400640 1 -- 172.21.15.27:0/2229643796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe540102a60 msgr2=0x7fe5400ff650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.034+0000 7fe535400640 1 --2- 172.21.15.27:0/2229643796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe540102a60 0x7fe5400ff650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.034+0000 7fe535400640 1 -- 172.21.15.27:0/2229643796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5401a8970 con 0x7fe5401083a0 2024-09-21T00:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.035+0000 7fe535400640 1 --2- 172.21.15.27:0/2229643796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5401083a0 0x7fe5400ffb90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe54006b950 tx=0x7fe52800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.035+0000 7fe537400640 1 -- 172.21.15.27:0/2229643796 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52800cc40 con 0x7fe5401083a0 2024-09-21T00:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.035+0000 7fe537400640 1 -- 172.21.15.27:0/2229643796 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe52800cde0 con 0x7fe5401083a0 2024-09-21T00:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.035+0000 7fe545200640 1 -- 172.21.15.27:0/2229643796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5401a8c60 con 0x7fe5401083a0 2024-09-21T00:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.035+0000 7fe537400640 1 -- 172.21.15.27:0/2229643796 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528004a00 con 0x7fe5401083a0 2024-09-21T00:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.035+0000 7fe545200640 1 -- 172.21.15.27:0/2229643796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5401a91a0 con 0x7fe5401083a0 2024-09-21T00:22:03.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.038+0000 7fe54bfba740 1 Processor -- start 2024-09-21T00:22:03.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.038+0000 7fe54bfba740 1 -- start start 2024-09-21T00:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.039+0000 7fe54bfba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e8ff60 0x55eb29236490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.039+0000 7fe54bfba740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e670a0 0x55eb292369d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.039+0000 7fe54bfba740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e6f280 0x55eb29236f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.039+0000 7fe54bfba740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55eb291add50 con 0x55eb28e6f280 2024-09-21T00:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.039+0000 7fe54bfba740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55eb2919d180 con 0x55eb28e670a0 2024-09-21T00:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.039+0000 7fe54bfba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55eb291a0f90 con 0x55eb28e8ff60 2024-09-21T00:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.039+0000 7fe537400640 1 -- 172.21.15.27:0/2229643796 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe528004ba0 con 0x7fe5401083a0 2024-09-21T00:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.039+0000 7fe53e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e8ff60 0x55eb29236490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.039+0000 7fe53de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e670a0 0x55eb292369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.039+0000 7fe53e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e8ff60 0x55eb29236490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:54422/0 (socket says 172.21.15.27:54422) 2024-09-21T00:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.039+0000 7fe53e800640 1 -- 172.21.15.27:0/2650777702 learned_addr learned my addr 172.21.15.27:0/2650777702 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.040+0000 7fe535400640 1 --2- 172.21.15.27:0/2650777702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e6f280 0x55eb29236f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.040+0000 7fe53de00640 1 -- 172.21.15.27:0/2650777702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e8ff60 msgr2=0x55eb29236490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.040+0000 7fe53de00640 1 --2- 172.21.15.27:0/2650777702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e8ff60 0x55eb29236490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.040+0000 7fe53de00640 1 -- 172.21.15.27:0/2650777702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e6f280 msgr2=0x55eb29236f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.040+0000 7fe53de00640 1 --2- 172.21.15.27:0/2650777702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e6f280 0x55eb29236f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.040+0000 7fe53de00640 1 -- 172.21.15.27:0/2650777702 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb29167a70 con 0x55eb28e670a0 2024-09-21T00:22:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.040+0000 7fe535400640 1 --2- 172.21.15.27:0/2650777702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e6f280 0x55eb29236f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:22:03.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.040+0000 7fe53de00640 1 --2- 172.21.15.27:0/2650777702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e670a0 0x55eb292369d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe530000f80 tx=0x7fe530009470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d669905b7f65f95f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:03.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.041+0000 7fe52f400640 1 -- 172.21.15.27:0/2650777702 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530031cf0 con 0x55eb28e670a0 2024-09-21T00:22:03.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.041+0000 7fe52f400640 1 -- 172.21.15.27:0/2650777702 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe530036030 con 0x55eb28e670a0 2024-09-21T00:22:03.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.041+0000 7fe52f400640 1 -- 172.21.15.27:0/2650777702 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530004040 con 0x55eb28e670a0 2024-09-21T00:22:03.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.041+0000 7fe537400640 1 --2- 172.21.15.27:0/2229643796 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe5240b6700 0x7fe5240b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.041+0000 7fe537400640 1 -- 172.21.15.27:0/2229643796 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fe5280e7ab0 con 0x7fe5401083a0 2024-09-21T00:22:03.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.041+0000 7fe53e800640 1 --2- 172.21.15.27:0/2229643796 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe5240b6700 0x7fe5240b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.042+0000 7fe54bfba740 1 -- 172.21.15.27:0/2650777702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e670a0 msgr2=0x55eb292369d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:03.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.042+0000 7fe54bfba740 1 --2- 172.21.15.27:0/2650777702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e670a0 0x55eb292369d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe530000f80 tx=0x7fe530009470 comp rx=0 tx=0).stop 2024-09-21T00:22:03.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.042+0000 7fe54bfba740 1 -- 172.21.15.27:0/2650777702 shutdown_connections 2024-09-21T00:22:03.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.042+0000 7fe54bfba740 1 --2- 172.21.15.27:0/2650777702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e6f280 0x55eb29236f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.042+0000 7fe54bfba740 1 --2- 172.21.15.27:0/2650777702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e670a0 0x55eb292369d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.042+0000 7fe54bfba740 1 --2- 172.21.15.27:0/2650777702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e8ff60 0x55eb29236490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.042+0000 7fe54bfba740 1 -- 172.21.15.27:0/2650777702 >> 172.21.15.27:0/2650777702 conn(0x55eb28e84350 msgr2=0x55eb29234380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:03.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.042+0000 7fe54bfba740 1 -- 172.21.15.27:0/2650777702 shutdown_connections 2024-09-21T00:22:03.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.042+0000 7fe53e800640 1 --2- 172.21.15.27:0/2229643796 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe5240b6700 0x7fe5240b8c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe538009d90 tx=0x7fe538005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:03.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.042+0000 7fe54bfba740 1 -- 172.21.15.27:0/2650777702 wait complete. 2024-09-21T00:22:03.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.043+0000 7fe54bfba740 1 -- start start 2024-09-21T00:22:03.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.043+0000 7fe54bfba740 1 Processor -- start 2024-09-21T00:22:03.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe54bfba740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e670a0 0x55eb29241fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe54bfba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e6f280 0x55eb29242500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe54bfba740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e8ff60 0x55eb2924ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe54bfba740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55eb28ffe5f0 con 0x55eb28e8ff60 2024-09-21T00:22:03.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe54bfba740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55eb29138460 con 0x55eb28e670a0 2024-09-21T00:22:03.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe54bfba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55eb29244a90 con 0x55eb28e6f280 2024-09-21T00:22:03.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe54bfba740 10 client.0 _ll_register_callbacks cb 0x55eb29148210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:03.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe53e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e8ff60 0x55eb2924ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe53de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e670a0 0x55eb29241fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe53e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e8ff60 0x55eb2924ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39360/0 (socket says 172.21.15.27:39360) 2024-09-21T00:22:03.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe53e800640 1 -- 172.21.15.27:0/2214198574 learned_addr learned my addr 172.21.15.27:0/2214198574 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:03.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe535400640 1 --2- 172.21.15.27:0/2214198574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e6f280 0x55eb29242500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe53de00640 1 -- 172.21.15.27:0/2214198574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e6f280 msgr2=0x55eb29242500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:03.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe53de00640 1 --2- 172.21.15.27:0/2214198574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e6f280 0x55eb29242500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe53de00640 1 -- 172.21.15.27:0/2214198574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e8ff60 msgr2=0x55eb2924ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:03.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe53de00640 1 --2- 172.21.15.27:0/2214198574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e8ff60 0x55eb2924ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.044+0000 7fe53de00640 1 -- 172.21.15.27:0/2214198574 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb28efb540 con 0x55eb28e670a0 2024-09-21T00:22:03.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.045+0000 7fe53de00640 1 --2- 172.21.15.27:0/2214198574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e670a0 0x55eb29241fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe5300094e0 tx=0x7fe530031da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46a08b6491be0798 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:03.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.045+0000 7fe523e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:22:03.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.045+0000 7fe523e00640 1 -- 172.21.15.27:0/2214198574 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530031660 con 0x55eb28e670a0 2024-09-21T00:22:03.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.045+0000 7fe523e00640 1 -- 172.21.15.27:0/2214198574 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fe530031800 con 0x55eb28e670a0 2024-09-21T00:22:03.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.045+0000 7fe54bfba740 1 -- 172.21.15.27:0/2214198574 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb2913aa70 con 0x55eb28e670a0 2024-09-21T00:22:03.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.046+0000 7fe523e00640 1 -- 172.21.15.27:0/2214198574 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe53002f660 con 0x55eb28e670a0 2024-09-21T00:22:03.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.046+0000 7fe54bfba740 10 client.25526 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:03.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.046+0000 7fe54bfba740 1 -- 172.21.15.27:0/2214198574 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb291556f0 con 0x55eb28e670a0 2024-09-21T00:22:03.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.046+0000 7fe523e00640 1 -- 172.21.15.27:0/2214198574 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55eb2913aa70 con 0x55eb28e670a0 2024-09-21T00:22:03.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.046+0000 7fe523e00640 1 -- 172.21.15.27:0/2214198574 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe53004e070 con 0x55eb28e670a0 2024-09-21T00:22:03.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.046+0000 7fe54bfba740 10 client.25526 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:03.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.046+0000 7fe54bfba740 10 client.25526 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T00:22:03.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.046+0000 7fe54bfba740 20 client.25526 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:03.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.046+0000 7fe54bfba740 1 --2- 172.21.15.27:0/2214198574 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55eb28e4e8e0 0x55eb29253ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.047+0000 7fe54bfba740 4 client.25526 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:22:03.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.047+0000 7fe54bfba740 1 -- 172.21.15.27:0/2214198574 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb291afca0 con 0x55eb28e4e8e0 2024-09-21T00:22:03.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.047+0000 7fe535400640 1 --2- 172.21.15.27:0/2214198574 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55eb28e4e8e0 0x55eb29253ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.047+0000 7fe535400640 1 --2- 172.21.15.27:0/2214198574 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55eb28e4e8e0 0x55eb29253ed0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb9eac16ee2a878 server_cookie=10f930af6bf2344b in_seq=0 out_seq=0 2024-09-21T00:22:03.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.047+0000 7fe523e00640 10 client.25526 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:03.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.048+0000 7fe523e00640 1 -- 172.21.15.27:0/2214198574 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eb291afca0 con 0x55eb28e4e8e0 2024-09-21T00:22:03.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.048+0000 7fe523e00640 10 client.25526 handle_command_reply: tid=0 2024-09-21T00:22:03.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.048+0000 7fe54bfba740 1 client.25526 shutdown 2024-09-21T00:22:03.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.049+0000 7fe54bfba740 1 -- 172.21.15.27:0/2214198574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e670a0 msgr2=0x55eb29241fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:03.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.049+0000 7fe54bfba740 1 --2- 172.21.15.27:0/2214198574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e670a0 0x55eb29241fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe5300094e0 tx=0x7fe530031da0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.049+0000 7fe54bfba740 1 -- 172.21.15.27:0/2214198574 shutdown_connections 2024-09-21T00:22:03.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.049+0000 7fe54bfba740 1 --2- 172.21.15.27:0/2214198574 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55eb28e4e8e0 0x55eb29253ed0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.049+0000 7fe54bfba740 1 --2- 172.21.15.27:0/2214198574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e8ff60 0x55eb2924ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.049+0000 7fe54bfba740 1 --2- 172.21.15.27:0/2214198574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e6f280 0x55eb29242500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.049+0000 7fe523e00640 0 client.25526 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:03.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.049+0000 7fe54bfba740 1 --2- 172.21.15.27:0/2214198574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e670a0 0x55eb29241fc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.049+0000 7fe54bfba740 1 -- 172.21.15.27:0/2214198574 >> 172.21.15.27:0/2214198574 conn(0x55eb28e84350 msgr2=0x55eb29239620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:03.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.049+0000 7fe54bfba740 1 -- 172.21.15.27:0/2214198574 shutdown_connections 2024-09-21T00:22:03.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.049+0000 7fe54bfba740 1 -- 172.21.15.27:0/2214198574 wait complete. 2024-09-21T00:22:03.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.049+0000 7fe54bfba740 20 client.25526 trim_cache size 0 max 16384 2024-09-21T00:22:03.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.068+0000 7fe54bfba740 1 Processor -- start 2024-09-21T00:22:03.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.068+0000 7fe54bfba740 1 -- start start 2024-09-21T00:22:03.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.069+0000 7fe54bfba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e670a0 0x55eb2923cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.069+0000 7fe54bfba740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e6f280 0x55eb2923c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.069+0000 7fe54bfba740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e8ff60 0x55eb29237bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.069+0000 7fe54bfba740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55eb29252210 con 0x55eb28e6f280 2024-09-21T00:22:03.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.069+0000 7fe54bfba740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55eb292528e0 con 0x55eb28e8ff60 2024-09-21T00:22:03.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.069+0000 7fe54bfba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55eb29252fb0 con 0x55eb28e670a0 2024-09-21T00:22:03.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.069+0000 7fe535400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e6f280 0x55eb2923c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.069+0000 7fe535400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e6f280 0x55eb2923c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39372/0 (socket says 172.21.15.27:39372) 2024-09-21T00:22:03.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.069+0000 7fe535400640 1 -- 172.21.15.27:0/3386524361 learned_addr learned my addr 172.21.15.27:0/3386524361 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:03.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.069+0000 7fe53e800640 1 --2- 172.21.15.27:0/3386524361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e8ff60 0x55eb29237bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.069+0000 7fe53de00640 1 --2- 172.21.15.27:0/3386524361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e670a0 0x55eb2923cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.070+0000 7fe535400640 1 -- 172.21.15.27:0/3386524361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e670a0 msgr2=0x55eb2923cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:03.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.070+0000 7fe535400640 1 --2- 172.21.15.27:0/3386524361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e670a0 0x55eb2923cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.070+0000 7fe535400640 1 -- 172.21.15.27:0/3386524361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e8ff60 msgr2=0x55eb29237bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:03.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.070+0000 7fe535400640 1 --2- 172.21.15.27:0/3386524361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e8ff60 0x55eb29237bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.070+0000 7fe535400640 1 -- 172.21.15.27:0/3386524361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb292392f0 con 0x55eb28e6f280 2024-09-21T00:22:03.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.070+0000 7fe53de00640 1 --2- 172.21.15.27:0/3386524361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e670a0 0x55eb2923cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:22:03.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.070+0000 7fe53e800640 1 --2- 172.21.15.27:0/3386524361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e8ff60 0x55eb29237bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:22:03.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.070+0000 7fe535400640 1 --2- 172.21.15.27:0/3386524361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e6f280 0x55eb2923c500 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe5280ad210 tx=0x7fe52800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45463438f33a64c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:03.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.070+0000 7fe52ea00640 1 -- 172.21.15.27:0/3386524361 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52801c040 con 0x55eb28e6f280 2024-09-21T00:22:03.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.070+0000 7fe52ea00640 1 -- 172.21.15.27:0/3386524361 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5280adc10 con 0x55eb28e6f280 2024-09-21T00:22:03.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.070+0000 7fe52ea00640 1 -- 172.21.15.27:0/3386524361 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5280a8e60 con 0x55eb28e6f280 2024-09-21T00:22:03.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.071+0000 7fe54bfba740 1 -- 172.21.15.27:0/3386524361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e6f280 msgr2=0x55eb2923c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:03.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.071+0000 7fe54bfba740 1 --2- 172.21.15.27:0/3386524361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e6f280 0x55eb2923c500 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe5280ad210 tx=0x7fe52800cf90 comp rx=0 tx=0).stop 2024-09-21T00:22:03.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.071+0000 7fe54bfba740 1 -- 172.21.15.27:0/3386524361 shutdown_connections 2024-09-21T00:22:03.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.071+0000 7fe54bfba740 1 --2- 172.21.15.27:0/3386524361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e8ff60 0x55eb29237bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.071+0000 7fe54bfba740 1 --2- 172.21.15.27:0/3386524361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e6f280 0x55eb2923c500 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.071+0000 7fe54bfba740 1 --2- 172.21.15.27:0/3386524361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e670a0 0x55eb2923cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.071+0000 7fe54bfba740 1 -- 172.21.15.27:0/3386524361 >> 172.21.15.27:0/3386524361 conn(0x55eb28e84350 msgr2=0x55eb292374c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:03.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.071+0000 7fe54bfba740 1 -- 172.21.15.27:0/3386524361 shutdown_connections 2024-09-21T00:22:03.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.071+0000 7fe54bfba740 1 -- 172.21.15.27:0/3386524361 wait complete. 2024-09-21T00:22:03.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.072+0000 7fe54bfba740 1 -- start start 2024-09-21T00:22:03.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.072+0000 7fe54bfba740 1 Processor -- start 2024-09-21T00:22:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.073+0000 7fe54bfba740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e670a0 0x55eb29256960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.073+0000 7fe54bfba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e6f280 0x55eb29256ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.073+0000 7fe54bfba740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e8ff60 0x55eb2925bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.073+0000 7fe54bfba740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55eb29253d50 con 0x55eb28e670a0 2024-09-21T00:22:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.073+0000 7fe54bfba740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55eb29253680 con 0x55eb28e8ff60 2024-09-21T00:22:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.073+0000 7fe54bfba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55eb29254420 con 0x55eb28e6f280 2024-09-21T00:22:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.073+0000 7fe54bfba740 10 client.0 _ll_register_callbacks cb 0x7fe53002b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.073+0000 7fe535400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e6f280 0x55eb29256ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.073+0000 7fe53de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e670a0 0x55eb29256960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.073+0000 7fe535400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e6f280 0x55eb29256ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:54456/0 (socket says 172.21.15.27:54456) 2024-09-21T00:22:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.073+0000 7fe535400640 1 -- 172.21.15.27:0/3963771233 learned_addr learned my addr 172.21.15.27:0/3963771233 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.073+0000 7fe53e800640 1 --2- 172.21.15.27:0/3963771233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e8ff60 0x55eb2925bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.074+0000 7fe535400640 1 -- 172.21.15.27:0/3963771233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e8ff60 msgr2=0x55eb2925bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.074+0000 7fe535400640 1 --2- 172.21.15.27:0/3963771233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e8ff60 0x55eb2925bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.074+0000 7fe535400640 1 -- 172.21.15.27:0/3963771233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e670a0 msgr2=0x55eb29256960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.074+0000 7fe535400640 1 --2- 172.21.15.27:0/3963771233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e670a0 0x55eb29256960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.074+0000 7fe535400640 1 -- 172.21.15.27:0/3963771233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb29238aa0 con 0x55eb28e6f280 2024-09-21T00:22:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.074+0000 7fe535400640 1 --2- 172.21.15.27:0/3963771233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e6f280 0x55eb29256ee0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe528004770 tx=0x7fe5280ad150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeaa22370cb2c940 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.074+0000 7fe523e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:22:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.074+0000 7fe523e00640 1 -- 172.21.15.27:0/3963771233 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52801c070 con 0x55eb28e6f280 2024-09-21T00:22:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.074+0000 7fe523e00640 1 -- 172.21.15.27:0/3963771233 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fe5280c8650 con 0x55eb28e6f280 2024-09-21T00:22:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.074+0000 7fe54bfba740 1 -- 172.21.15.27:0/3963771233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5280adc10 con 0x55eb28e6f280 2024-09-21T00:22:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.075+0000 7fe523e00640 1 -- 172.21.15.27:0/3963771233 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe5280e5290 con 0x55eb28e6f280 2024-09-21T00:22:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.075+0000 7fe54bfba740 10 client.25513 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.075+0000 7fe54bfba740 1 -- 172.21.15.27:0/3963771233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb292bfa60 con 0x55eb28e6f280 2024-09-21T00:22:03.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.075+0000 7fe523e00640 1 -- 172.21.15.27:0/3963771233 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5280adc10 con 0x55eb28e6f280 2024-09-21T00:22:03.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.075+0000 7fe523e00640 1 -- 172.21.15.27:0/3963771233 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe5280c8b60 con 0x55eb28e6f280 2024-09-21T00:22:03.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.075+0000 7fe54bfba740 10 client.25513 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:03.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.075+0000 7fe54bfba740 10 client.25513 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T00:22:03.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.075+0000 7fe54bfba740 20 client.25513 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.075+0000 7fe54bfba740 1 --2- 172.21.15.27:0/3963771233 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55eb28e23c90 0x55eb292c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.075+0000 7fe54bfba740 4 client.25513 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:22:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.075+0000 7fe54bfba740 1 -- 172.21.15.27:0/3963771233 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "status"}) -- 0x55eb291ab750 con 0x55eb28e23c90 2024-09-21T00:22:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.076+0000 7fe53de00640 1 --2- 172.21.15.27:0/3963771233 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55eb28e23c90 0x55eb292c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.076+0000 7fe53de00640 1 --2- 172.21.15.27:0/3963771233 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55eb28e23c90 0x55eb292c2e20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e1d9d84fc32b9e server_cookie=aa93c4ddec773e55 in_seq=0 out_seq=0 2024-09-21T00:22:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.076+0000 7fe523e00640 10 client.25513 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.076+0000 7fe523e00640 1 -- 172.21.15.27:0/3963771233 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55eb291ab750 con 0x55eb28e23c90 2024-09-21T00:22:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.077+0000 7fe523e00640 10 client.25513 handle_command_reply: tid=0 2024-09-21T00:22:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.077+0000 7fe54bfba740 1 client.25513 shutdown 2024-09-21T00:22:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.077+0000 7fe54bfba740 1 -- 172.21.15.27:0/3963771233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e6f280 msgr2=0x55eb29256ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.077+0000 7fe54bfba740 1 --2- 172.21.15.27:0/3963771233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e6f280 0x55eb29256ee0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe528004770 tx=0x7fe5280ad150 comp rx=0 tx=0).stop 2024-09-21T00:22:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.077+0000 7fe54bfba740 1 -- 172.21.15.27:0/3963771233 shutdown_connections 2024-09-21T00:22:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.077+0000 7fe54bfba740 1 --2- 172.21.15.27:0/3963771233 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55eb28e23c90 0x55eb292c2e20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.077+0000 7fe54bfba740 1 --2- 172.21.15.27:0/3963771233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb28e8ff60 0x55eb2925bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.077+0000 7fe54bfba740 1 --2- 172.21.15.27:0/3963771233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb28e6f280 0x55eb29256ee0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.077+0000 7fe54bfba740 1 --2- 172.21.15.27:0/3963771233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eb28e670a0 0x55eb29256960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.077+0000 7fe54bfba740 1 -- 172.21.15.27:0/3963771233 >> 172.21.15.27:0/3963771233 conn(0x55eb28e84350 msgr2=0x55eb29240640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:03.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.077+0000 7fe523e00640 0 client.25513 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:03.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.077+0000 7fe54bfba740 1 -- 172.21.15.27:0/3963771233 shutdown_connections 2024-09-21T00:22:03.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.077+0000 7fe54bfba740 1 -- 172.21.15.27:0/3963771233 wait complete. 2024-09-21T00:22:03.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.077+0000 7fe54bfba740 20 client.25513 trim_cache size 0 max 16384 2024-09-21T00:22:03.082 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:22:03.082 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "9c21de84-77ad-11ef-baf5-efdc52797490", 2024-09-21T00:22:03.082 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-21T00:22:03.083 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24761, 2024-09-21T00:22:03.083 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T00:22:03.083 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T00:22:03.083 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T00:22:03.083 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 202.41556711800001, 2024-09-21T00:22:03.083 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 26, 2024-09-21T00:22:03.083 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T00:22:03.083 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 90, 2024-09-21T00:22:03.083 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 215.88282247000001 2024-09-21T00:22:03.083 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:22:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.078+0000 7fe545200640 1 -- 172.21.15.27:0/2229643796 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe5240b6700 msgr2=0x7fe5240b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.078+0000 7fe545200640 1 --2- 172.21.15.27:0/2229643796 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe5240b6700 0x7fe5240b8c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe538009d90 tx=0x7fe538005f90 comp rx=0 tx=0).stop 2024-09-21T00:22:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.078+0000 7fe545200640 1 -- 172.21.15.27:0/2229643796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5401083a0 msgr2=0x7fe5400ffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.078+0000 7fe545200640 1 --2- 172.21.15.27:0/2229643796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5401083a0 0x7fe5400ffb90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe54006b950 tx=0x7fe52800ee50 comp rx=0 tx=0).stop 2024-09-21T00:22:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.078+0000 7fe545200640 1 -- 172.21.15.27:0/2229643796 shutdown_connections 2024-09-21T00:22:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.078+0000 7fe545200640 1 --2- 172.21.15.27:0/2229643796 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe5240b6700 0x7fe5240b8c10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.078+0000 7fe545200640 1 --2- 172.21.15.27:0/2229643796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe540108cf0 0x7fe5401000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.078+0000 7fe545200640 1 --2- 172.21.15.27:0/2229643796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5401083a0 0x7fe5400ffb90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.078+0000 7fe545200640 1 --2- 172.21.15.27:0/2229643796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe540102a60 0x7fe5400ff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.078+0000 7fe545200640 1 -- 172.21.15.27:0/2229643796 >> 172.21.15.27:0/2229643796 conn(0x7fe540079760 msgr2=0x7fe54006c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.078+0000 7fe545200640 1 -- 172.21.15.27:0/2229643796 shutdown_connections 2024-09-21T00:22:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:03.078+0000 7fe545200640 1 -- 172.21.15.27:0/2229643796 wait complete. 2024-09-21T00:22:04.086 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-21T00:22:04.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:04 smithi027 ceph-mon[30745]: pgmap v397: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-21T00:22:04.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:04 smithi089 ceph-mon[34481]: pgmap v397: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-21T00:22:04.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:04 smithi098 ceph-mon[34725]: pgmap v397: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-21T00:22:06.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:06 smithi089 ceph-mon[34481]: pgmap v398: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-21T00:22:06.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:06 smithi098 ceph-mon[34725]: pgmap v398: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-21T00:22:06.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:06 smithi027 ceph-mon[30745]: pgmap v398: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-21T00:22:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.984+0000 7f239ae00640 1 -- 172.21.15.27:0/2870574074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2394109bc0 msgr2=0x7f23941103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.984+0000 7f239ae00640 1 --2- 172.21.15.27:0/2870574074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2394109bc0 0x7f23941103f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f238c009ca0 tx=0x7f238c02a8c0 comp rx=0 tx=0).stop 2024-09-21T00:22:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.984+0000 7f239ae00640 1 -- 172.21.15.27:0/2870574074 shutdown_connections 2024-09-21T00:22:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.984+0000 7f239ae00640 1 --2- 172.21.15.27:0/2870574074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2394109bc0 0x7f23941103f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.984+0000 7f239ae00640 1 --2- 172.21.15.27:0/2870574074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2394109270 0x7f2394109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.984+0000 7f239ae00640 1 --2- 172.21.15.27:0/2870574074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2394103940 0x7f2394103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.984+0000 7f239ae00640 1 -- 172.21.15.27:0/2870574074 >> 172.21.15.27:0/2870574074 conn(0x7f23940ff3a0 msgr2=0x7f23941017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.984+0000 7f239ae00640 1 -- 172.21.15.27:0/2870574074 shutdown_connections 2024-09-21T00:22:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.984+0000 7f239ae00640 1 -- 172.21.15.27:0/2870574074 wait complete. 2024-09-21T00:22:06.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.985+0000 7f239ae00640 1 Processor -- start 2024-09-21T00:22:06.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.986+0000 7f239ae00640 1 -- start start 2024-09-21T00:22:06.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.986+0000 7f239ae00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2394103940 0x7f2394075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:06.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.986+0000 7f239ae00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2394109270 0x7f2394075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:06.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.986+0000 7f239ae00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2394109bc0 0x7f2394076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:06.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.986+0000 7f239ae00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23941138d0 con 0x7f2394109bc0 2024-09-21T00:22:06.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.986+0000 7f239ae00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2394113750 con 0x7f2394109270 2024-09-21T00:22:06.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.986+0000 7f239ae00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2394113a50 con 0x7f2394103940 2024-09-21T00:22:06.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.986+0000 7f239a400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2394109bc0 0x7f2394076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:06.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.986+0000 7f2399a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2394103940 0x7f2394075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:06.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.986+0000 7f2399a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2394103940 0x7f2394075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:54486/0 (socket says 172.21.15.27:54486) 2024-09-21T00:22:06.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.986+0000 7f239a400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2394109bc0 0x7f2394076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39410/0 (socket says 172.21.15.27:39410) 2024-09-21T00:22:06.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.986+0000 7f2399a00640 1 -- 172.21.15.27:0/3140920656 learned_addr learned my addr 172.21.15.27:0/3140920656 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:06.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.987+0000 7f2399000640 1 --2- 172.21.15.27:0/3140920656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2394109270 0x7f2394075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.987+0000 7f239a400640 1 -- 172.21.15.27:0/3140920656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2394103940 msgr2=0x7f2394075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.987+0000 7f239a400640 1 --2- 172.21.15.27:0/3140920656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2394103940 0x7f2394075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.987+0000 7f239a400640 1 -- 172.21.15.27:0/3140920656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2394109270 msgr2=0x7f2394075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.987+0000 7f239a400640 1 --2- 172.21.15.27:0/3140920656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2394109270 0x7f2394075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.987+0000 7f239a400640 1 -- 172.21.15.27:0/3140920656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23941a8540 con 0x7f2394109bc0 2024-09-21T00:22:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.987+0000 7f239a400640 1 --2- 172.21.15.27:0/3140920656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2394109bc0 0x7f2394076190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f238c005e30 tx=0x7f238c0317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.988+0000 7f2392000640 1 -- 172.21.15.27:0/3140920656 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c003b70 con 0x7f2394109bc0 2024-09-21T00:22:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.988+0000 7f2392000640 1 -- 172.21.15.27:0/3140920656 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f238c003d10 con 0x7f2394109bc0 2024-09-21T00:22:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.988+0000 7f239ae00640 1 -- 172.21.15.27:0/3140920656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23941a87d0 con 0x7f2394109bc0 2024-09-21T00:22:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.988+0000 7f2392000640 1 -- 172.21.15.27:0/3140920656 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c02f970 con 0x7f2394109bc0 2024-09-21T00:22:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.988+0000 7f239ae00640 1 -- 172.21.15.27:0/3140920656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23941a8d20 con 0x7f2394109bc0 2024-09-21T00:22:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.992+0000 7f23a34d4740 1 Processor -- start 2024-09-21T00:22:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.993+0000 7f2392000640 1 -- 172.21.15.27:0/3140920656 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f238c02fb10 con 0x7f2394109bc0 2024-09-21T00:22:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.993+0000 7f23a34d4740 1 -- start start 2024-09-21T00:22:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.993+0000 7f23a34d4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8df51b0 0x561fe9199e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.993+0000 7f23a34d4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8dcc2f0 0x561fe919a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.993+0000 7f23a34d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dd44d0 0x561fe919a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.993+0000 7f23a34d4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561fe910f6b0 con 0x561fe8df51b0 2024-09-21T00:22:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.993+0000 7f23a34d4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561fe90feae0 con 0x561fe8dcc2f0 2024-09-21T00:22:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.993+0000 7f23a34d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561fe91028f0 con 0x561fe8dd44d0 2024-09-21T00:22:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.993+0000 7f2399a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8dcc2f0 0x561fe919a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.993+0000 7f2399000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8df51b0 0x561fe9199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.993+0000 7f2399a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8dcc2f0 0x561fe919a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:46570/0 (socket says 172.21.15.27:46570) 2024-09-21T00:22:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.994+0000 7f2399a00640 1 -- 172.21.15.27:0/601461471 learned_addr learned my addr 172.21.15.27:0/601461471 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.994+0000 7f2399a00640 1 -- 172.21.15.27:0/601461471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dd44d0 msgr2=0x561fe919a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.994+0000 7f2399a00640 1 --2- 172.21.15.27:0/601461471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dd44d0 0x561fe919a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.994+0000 7f2399a00640 1 -- 172.21.15.27:0/601461471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8df51b0 msgr2=0x561fe9199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.994+0000 7f2399a00640 1 --2- 172.21.15.27:0/601461471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8df51b0 0x561fe9199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.994+0000 7f2399a00640 1 -- 172.21.15.27:0/601461471 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe90cb780 con 0x561fe8dcc2f0 2024-09-21T00:22:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.994+0000 7f2399a00640 1 --2- 172.21.15.27:0/601461471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8dcc2f0 0x561fe919a350 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2388009dd0 tx=0x7f2388011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3641aa55495cdaa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.994+0000 7f2386a00640 1 -- 172.21.15.27:0/601461471 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238801e070 con 0x561fe8dcc2f0 2024-09-21T00:22:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.994+0000 7f2392000640 1 --2- 172.21.15.27:0/3140920656 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f237c0b66a0 0x7f237c0b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.994+0000 7f2386a00640 1 -- 172.21.15.27:0/601461471 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2388019ae0 con 0x561fe8dcc2f0 2024-09-21T00:22:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.994+0000 7f2386a00640 1 -- 172.21.15.27:0/601461471 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238800fac0 con 0x561fe8dcc2f0 2024-09-21T00:22:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.995+0000 7f2392000640 1 -- 172.21.15.27:0/3140920656 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f238c108a40 con 0x7f2394109bc0 2024-09-21T00:22:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.995+0000 7f2399000640 1 --2- 172.21.15.27:0/3140920656 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f237c0b66a0 0x7f237c0b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.995+0000 7f23a34d4740 1 -- 172.21.15.27:0/601461471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8dcc2f0 msgr2=0x561fe919a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.995+0000 7f23a34d4740 1 --2- 172.21.15.27:0/601461471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8dcc2f0 0x561fe919a350 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2388009dd0 tx=0x7f2388011810 comp rx=0 tx=0).stop 2024-09-21T00:22:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.995+0000 7f23a34d4740 1 -- 172.21.15.27:0/601461471 shutdown_connections 2024-09-21T00:22:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.995+0000 7f23a34d4740 1 --2- 172.21.15.27:0/601461471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dd44d0 0x561fe919a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.995+0000 7f23a34d4740 1 --2- 172.21.15.27:0/601461471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8dcc2f0 0x561fe919a350 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.995+0000 7f23a34d4740 1 --2- 172.21.15.27:0/601461471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8df51b0 0x561fe9199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.995+0000 7f23a34d4740 1 -- 172.21.15.27:0/601461471 >> 172.21.15.27:0/601461471 conn(0x561fe8de9300 msgr2=0x561fe9197d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.996+0000 7f23a34d4740 1 -- 172.21.15.27:0/601461471 shutdown_connections 2024-09-21T00:22:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.996+0000 7f2399000640 1 --2- 172.21.15.27:0/3140920656 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f237c0b66a0 0x7f237c0b8bb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2380005f60 tx=0x7f23800096a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.996+0000 7f23a34d4740 1 -- 172.21.15.27:0/601461471 wait complete. 2024-09-21T00:22:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.996+0000 7f23a34d4740 1 -- start start 2024-09-21T00:22:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.996+0000 7f23a34d4740 1 Processor -- start 2024-09-21T00:22:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.997+0000 7f23a34d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dcc2f0 0x561fe91a5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.997+0000 7f23a34d4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8dd44d0 0x561fe91a5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.997+0000 7f23a34d4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8df51b0 0x561fe91a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.997+0000 7f23a34d4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561fe9036ab0 con 0x561fe8df51b0 2024-09-21T00:22:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.997+0000 7f23a34d4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561fe90a5570 con 0x561fe8dd44d0 2024-09-21T00:22:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.997+0000 7f23a34d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561fe91a8590 con 0x561fe8dcc2f0 2024-09-21T00:22:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.997+0000 7f23a34d4740 10 client.0 _ll_register_callbacks cb 0x561fe90b4100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.997+0000 7f239a400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8df51b0 0x561fe91a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.997+0000 7f239a400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8df51b0 0x561fe91a5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39434/0 (socket says 172.21.15.27:39434) 2024-09-21T00:22:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.997+0000 7f239a400640 1 -- 172.21.15.27:0/2778020032 learned_addr learned my addr 172.21.15.27:0/2778020032 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.997+0000 7f2399a00640 1 --2- 172.21.15.27:0/2778020032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dcc2f0 0x561fe91a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.997+0000 7f2399000640 1 --2- 172.21.15.27:0/2778020032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8dd44d0 0x561fe91a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.998+0000 7f239a400640 1 -- 172.21.15.27:0/2778020032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dcc2f0 msgr2=0x561fe91a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.998+0000 7f239a400640 1 --2- 172.21.15.27:0/2778020032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dcc2f0 0x561fe91a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.998+0000 7f239a400640 1 -- 172.21.15.27:0/2778020032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8dd44d0 msgr2=0x561fe91a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.998+0000 7f239a400640 1 --2- 172.21.15.27:0/2778020032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8dd44d0 0x561fe91a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.998+0000 7f239a400640 1 -- 172.21.15.27:0/2778020032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe8e60790 con 0x561fe8df51b0 2024-09-21T00:22:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.998+0000 7f239a400640 1 --2- 172.21.15.27:0/2778020032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8df51b0 0x561fe91a5ec0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f238c0ce490 tx=0x7f238c0ce390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f9424988fc2ea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.998+0000 7f237b400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:22:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.998+0000 7f237b400640 1 -- 172.21.15.27:0/2778020032 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c0d7c90 con 0x561fe8df51b0 2024-09-21T00:22:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.998+0000 7f237b400640 1 -- 172.21.15.27:0/2778020032 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f238c004b40 con 0x561fe8df51b0 2024-09-21T00:22:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.998+0000 7f23a34d4740 1 -- 172.21.15.27:0/2778020032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2388019ae0 con 0x561fe8df51b0 2024-09-21T00:22:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.999+0000 7f237b400640 1 -- 172.21.15.27:0/2778020032 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f238c0ce010 con 0x561fe8df51b0 2024-09-21T00:22:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.999+0000 7f23a34d4740 10 client.15522 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.999+0000 7f23a34d4740 1 -- 172.21.15.27:0/2778020032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561fe8ed9620 con 0x561fe8df51b0 2024-09-21T00:22:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.999+0000 7f237b400640 1 -- 172.21.15.27:0/2778020032 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2388019ae0 con 0x561fe8df51b0 2024-09-21T00:22:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.999+0000 7f237b400640 1 -- 172.21.15.27:0/2778020032 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f238c0cd760 con 0x561fe8df51b0 2024-09-21T00:22:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.999+0000 7f23a34d4740 10 client.15522 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.999+0000 7f23a34d4740 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:22:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.999+0000 7f23a34d4740 20 client.15522 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.999+0000 7f23a34d4740 1 --2- 172.21.15.27:0/2778020032 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fe8db3d80 0x561fe91b7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.999+0000 7f23a34d4740 4 client.15522 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:22:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.999+0000 7f23a34d4740 1 -- 172.21.15.27:0/2778020032 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fe9111600 con 0x561fe8db3d80 2024-09-21T00:22:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:06.999+0000 7f2399a00640 1 --2- 172.21.15.27:0/2778020032 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fe8db3d80 0x561fe91b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.000+0000 7f2399a00640 1 --2- 172.21.15.27:0/2778020032 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fe8db3d80 0x561fe91b7650 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d69f6cdba0eb96 server_cookie=9dfb54d2c710eb96 in_seq=0 out_seq=0 2024-09-21T00:22:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.000+0000 7f237b400640 10 client.15522 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.001+0000 7f237b400640 1 -- 172.21.15.27:0/2778020032 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561fe9111600 con 0x561fe8db3d80 2024-09-21T00:22:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.001+0000 7f237b400640 10 client.15522 handle_command_reply: tid=0 2024-09-21T00:22:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.001+0000 7f23a34d4740 1 client.15522 shutdown 2024-09-21T00:22:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.001+0000 7f23a34d4740 1 -- 172.21.15.27:0/2778020032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8df51b0 msgr2=0x561fe91a5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.001+0000 7f23a34d4740 1 --2- 172.21.15.27:0/2778020032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8df51b0 0x561fe91a5ec0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f238c0ce490 tx=0x7f238c0ce390 comp rx=0 tx=0).stop 2024-09-21T00:22:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.001+0000 7f23a34d4740 1 -- 172.21.15.27:0/2778020032 shutdown_connections 2024-09-21T00:22:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.001+0000 7f23a34d4740 1 --2- 172.21.15.27:0/2778020032 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fe8db3d80 0x561fe91b7650 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.001+0000 7f23a34d4740 1 --2- 172.21.15.27:0/2778020032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8df51b0 0x561fe91a5ec0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.001+0000 7f23a34d4740 1 --2- 172.21.15.27:0/2778020032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8dd44d0 0x561fe91a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.001+0000 7f23a34d4740 1 --2- 172.21.15.27:0/2778020032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dcc2f0 0x561fe91a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.001+0000 7f237b400640 0 client.15522 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.001+0000 7f23a34d4740 1 -- 172.21.15.27:0/2778020032 >> 172.21.15.27:0/2778020032 conn(0x561fe8de9300 msgr2=0x561fe919ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.001+0000 7f23a34d4740 1 -- 172.21.15.27:0/2778020032 shutdown_connections 2024-09-21T00:22:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.001+0000 7f23a34d4740 1 -- 172.21.15.27:0/2778020032 wait complete. 2024-09-21T00:22:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.001+0000 7f23a34d4740 20 client.15522 trim_cache size 0 max 16384 2024-09-21T00:22:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.017+0000 7f23a34d4740 1 Processor -- start 2024-09-21T00:22:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.017+0000 7f23a34d4740 1 -- start start 2024-09-21T00:22:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.017+0000 7f23a34d4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8dcc2f0 0x561fe919b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.017+0000 7f23a34d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dd44d0 0x561fe91a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.017+0000 7f23a34d4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8df51b0 0x561fe91a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.017+0000 7f23a34d4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23941b5700 con 0x561fe8dcc2f0 2024-09-21T00:22:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.017+0000 7f23a34d4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f23941b2fd0 con 0x561fe8df51b0 2024-09-21T00:22:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.017+0000 7f23a34d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f23941b0e20 con 0x561fe8dd44d0 2024-09-21T00:22:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.017+0000 7f2399a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8dcc2f0 0x561fe919b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.018+0000 7f2399a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8dcc2f0 0x561fe919b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39438/0 (socket says 172.21.15.27:39438) 2024-09-21T00:22:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.018+0000 7f2399a00640 1 -- 172.21.15.27:0/1643129959 learned_addr learned my addr 172.21.15.27:0/1643129959 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.018+0000 7f2399000640 1 --2- 172.21.15.27:0/1643129959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dd44d0 0x561fe91a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.018+0000 7f239a400640 1 --2- 172.21.15.27:0/1643129959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8df51b0 0x561fe91a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.018+0000 7f2399a00640 1 -- 172.21.15.27:0/1643129959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dd44d0 msgr2=0x561fe91a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.018+0000 7f2399a00640 1 --2- 172.21.15.27:0/1643129959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dd44d0 0x561fe91a25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.018+0000 7f2399a00640 1 -- 172.21.15.27:0/1643129959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8df51b0 msgr2=0x561fe91a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.018+0000 7f2399a00640 1 --2- 172.21.15.27:0/1643129959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8df51b0 0x561fe91a2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.018+0000 7f2399a00640 1 -- 172.21.15.27:0/1643129959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe919c110 con 0x561fe8dcc2f0 2024-09-21T00:22:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.018+0000 7f239a400640 1 --2- 172.21.15.27:0/1643129959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8df51b0 0x561fe91a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:22:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.018+0000 7f2399a00640 1 --2- 172.21.15.27:0/1643129959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8dcc2f0 0x561fe919b550 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2388014040 tx=0x7f23880096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e1a8568c61b442 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.018+0000 7f2386000640 1 -- 172.21.15.27:0/1643129959 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238801e040 con 0x561fe8dcc2f0 2024-09-21T00:22:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.019+0000 7f2386000640 1 -- 172.21.15.27:0/1643129959 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23880054f0 con 0x561fe8dcc2f0 2024-09-21T00:22:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.019+0000 7f2386000640 1 -- 172.21.15.27:0/1643129959 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238800f760 con 0x561fe8dcc2f0 2024-09-21T00:22:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.019+0000 7f23a34d4740 1 -- 172.21.15.27:0/1643129959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8dcc2f0 msgr2=0x561fe919b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.019+0000 7f23a34d4740 1 --2- 172.21.15.27:0/1643129959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8dcc2f0 0x561fe919b550 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2388014040 tx=0x7f23880096c0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.019+0000 7f23a34d4740 1 -- 172.21.15.27:0/1643129959 shutdown_connections 2024-09-21T00:22:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.019+0000 7f23a34d4740 1 --2- 172.21.15.27:0/1643129959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8df51b0 0x561fe91a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.019+0000 7f23a34d4740 1 --2- 172.21.15.27:0/1643129959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dd44d0 0x561fe91a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.019+0000 7f23a34d4740 1 --2- 172.21.15.27:0/1643129959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8dcc2f0 0x561fe919b550 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.019+0000 7f23a34d4740 1 -- 172.21.15.27:0/1643129959 >> 172.21.15.27:0/1643129959 conn(0x561fe8de9300 msgr2=0x561fe919ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.019+0000 7f23a34d4740 1 -- 172.21.15.27:0/1643129959 shutdown_connections 2024-09-21T00:22:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.019+0000 7f23a34d4740 1 -- 172.21.15.27:0/1643129959 wait complete. 2024-09-21T00:22:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.020+0000 7f23a34d4740 1 -- start start 2024-09-21T00:22:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.020+0000 7f23a34d4740 1 Processor -- start 2024-09-21T00:22:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.020+0000 7f23a34d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dcc2f0 0x561fe91adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.020+0000 7f23a34d4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8dd44d0 0x561fe91ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.020+0000 7f23a34d4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8df51b0 0x561fe91ab920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.020+0000 7f23a34d4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23941b1490 con 0x561fe8dd44d0 2024-09-21T00:22:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.021+0000 7f23a34d4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f23941b2900 con 0x561fe8df51b0 2024-09-21T00:22:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.021+0000 7f23a34d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f23941b1b60 con 0x561fe8dcc2f0 2024-09-21T00:22:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.021+0000 7f23a34d4740 10 client.0 _ll_register_callbacks cb 0x7f238c009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.021+0000 7f2399000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8dd44d0 0x561fe91ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.021+0000 7f2399000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8dd44d0 0x561fe91ab3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39448/0 (socket says 172.21.15.27:39448) 2024-09-21T00:22:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.021+0000 7f2399000640 1 -- 172.21.15.27:0/1598644604 learned_addr learned my addr 172.21.15.27:0/1598644604 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.021+0000 7f239a400640 1 --2- 172.21.15.27:0/1598644604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8df51b0 0x561fe91ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.021+0000 7f2399a00640 1 --2- 172.21.15.27:0/1598644604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dcc2f0 0x561fe91adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.021+0000 7f2399000640 1 -- 172.21.15.27:0/1598644604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dcc2f0 msgr2=0x561fe91adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.021+0000 7f2399000640 1 --2- 172.21.15.27:0/1598644604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dcc2f0 0x561fe91adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.021+0000 7f2399000640 1 -- 172.21.15.27:0/1598644604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8df51b0 msgr2=0x561fe91ab920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.021+0000 7f2399000640 1 --2- 172.21.15.27:0/1598644604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8df51b0 0x561fe91ab920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.021+0000 7f2399000640 1 -- 172.21.15.27:0/1598644604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe91ac010 con 0x561fe8dd44d0 2024-09-21T00:22:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.021+0000 7f2399a00640 1 --2- 172.21.15.27:0/1598644604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dcc2f0 0x561fe91adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:22:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.021+0000 7f239a400640 1 --2- 172.21.15.27:0/1598644604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8df51b0 0x561fe91ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:22:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.022+0000 7f2399000640 1 --2- 172.21.15.27:0/1598644604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8dd44d0 0x561fe91ab3e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f239406a3b0 tx=0x7f23800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a78f0242a11734 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.022+0000 7f237b400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:22:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.022+0000 7f237b400640 1 -- 172.21.15.27:0/1598644604 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380005020 con 0x561fe8dd44d0 2024-09-21T00:22:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.022+0000 7f237b400640 1 -- 172.21.15.27:0/1598644604 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f23800051c0 con 0x561fe8dd44d0 2024-09-21T00:22:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.022+0000 7f23a34d4740 1 -- 172.21.15.27:0/1598644604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23880054f0 con 0x561fe8dd44d0 2024-09-21T00:22:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.022+0000 7f237b400640 1 -- 172.21.15.27:0/1598644604 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2380035050 con 0x561fe8dd44d0 2024-09-21T00:22:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.022+0000 7f23a34d4740 10 client.15528 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.022+0000 7f23a34d4740 1 -- 172.21.15.27:0/1598644604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561fe9221f20 con 0x561fe8dd44d0 2024-09-21T00:22:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.023+0000 7f237b400640 1 -- 172.21.15.27:0/1598644604 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23880054f0 con 0x561fe8dd44d0 2024-09-21T00:22:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.023+0000 7f237b400640 1 -- 172.21.15.27:0/1598644604 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f23800048c0 con 0x561fe8dd44d0 2024-09-21T00:22:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.023+0000 7f23a34d4740 10 client.15528 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.023+0000 7f23a34d4740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:22:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.023+0000 7f23a34d4740 20 client.15528 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.023+0000 7f23a34d4740 1 --2- 172.21.15.27:0/1598644604 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fe8d891c0 0x561fe9224760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.023+0000 7f23a34d4740 4 client.15528 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:22:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.023+0000 7f23a34d4740 1 -- 172.21.15.27:0/1598644604 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561fe910d0b0 con 0x561fe8d891c0 2024-09-21T00:22:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.023+0000 7f2399a00640 1 --2- 172.21.15.27:0/1598644604 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fe8d891c0 0x561fe9224760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.024+0000 7f2399a00640 1 --2- 172.21.15.27:0/1598644604 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fe8d891c0 0x561fe9224760 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=419ab1fa1fcbaa28 server_cookie=53bfba1a51a8b7aa in_seq=0 out_seq=0 2024-09-21T00:22:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.024+0000 7f237b400640 10 client.15528 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.032+0000 7f237b400640 1 -- 172.21.15.27:0/1598644604 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561fe910d0b0 con 0x561fe8d891c0 2024-09-21T00:22:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.032+0000 7f237b400640 10 client.15528 handle_command_reply: tid=0 2024-09-21T00:22:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.032+0000 7f23a34d4740 1 client.15528 shutdown 2024-09-21T00:22:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.032+0000 7f23a34d4740 1 -- 172.21.15.27:0/1598644604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8dd44d0 msgr2=0x561fe91ab3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.032+0000 7f23a34d4740 1 --2- 172.21.15.27:0/1598644604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8dd44d0 0x561fe91ab3e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f239406a3b0 tx=0x7f23800046c0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.032+0000 7f23a34d4740 1 -- 172.21.15.27:0/1598644604 shutdown_connections 2024-09-21T00:22:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.032+0000 7f23a34d4740 1 --2- 172.21.15.27:0/1598644604 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fe8d891c0 0x561fe9224760 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.032+0000 7f23a34d4740 1 --2- 172.21.15.27:0/1598644604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fe8df51b0 0x561fe91ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.032+0000 7f23a34d4740 1 --2- 172.21.15.27:0/1598644604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fe8dd44d0 0x561fe91ab3e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.032+0000 7f237b400640 0 client.15528 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.032+0000 7f23a34d4740 1 --2- 172.21.15.27:0/1598644604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fe8dcc2f0 0x561fe91adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.032+0000 7f23a34d4740 1 -- 172.21.15.27:0/1598644604 >> 172.21.15.27:0/1598644604 conn(0x561fe8de9300 msgr2=0x561fe91b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.033+0000 7f23a34d4740 1 -- 172.21.15.27:0/1598644604 shutdown_connections 2024-09-21T00:22:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.033+0000 7f23a34d4740 1 -- 172.21.15.27:0/1598644604 wait complete. 2024-09-21T00:22:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.033+0000 7f23a34d4740 20 client.15528 trim_cache size 0 max 16384 2024-09-21T00:22:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:22:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:22:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:22:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:22:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.033+0000 7f239ae00640 1 -- 172.21.15.27:0/3140920656 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f237c0b66a0 msgr2=0x7f237c0b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.033+0000 7f239ae00640 1 --2- 172.21.15.27:0/3140920656 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f237c0b66a0 0x7f237c0b8bb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2380005f60 tx=0x7f23800096a0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.033+0000 7f239ae00640 1 -- 172.21.15.27:0/3140920656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2394109bc0 msgr2=0x7f2394076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.033+0000 7f239ae00640 1 --2- 172.21.15.27:0/3140920656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2394109bc0 0x7f2394076190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f238c005e30 tx=0x7f238c0317c0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.033+0000 7f239ae00640 1 -- 172.21.15.27:0/3140920656 shutdown_connections 2024-09-21T00:22:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.033+0000 7f239ae00640 1 --2- 172.21.15.27:0/3140920656 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f237c0b66a0 0x7f237c0b8bb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.033+0000 7f239ae00640 1 --2- 172.21.15.27:0/3140920656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2394109bc0 0x7f2394076190 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.033+0000 7f239ae00640 1 --2- 172.21.15.27:0/3140920656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2394109270 0x7f2394075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.033+0000 7f239ae00640 1 --2- 172.21.15.27:0/3140920656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2394103940 0x7f2394075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.033+0000 7f239ae00640 1 -- 172.21.15.27:0/3140920656 >> 172.21.15.27:0/3140920656 conn(0x7f23940ff3a0 msgr2=0x7f2394108810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.034+0000 7f239ae00640 1 -- 172.21.15.27:0/3140920656 shutdown_connections 2024-09-21T00:22:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:07.034+0000 7f239ae00640 1 -- 172.21.15.27:0/3140920656 wait complete. 2024-09-21T00:22:07.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:07 smithi089 ceph-mon[34481]: pgmap v399: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-09-21T00:22:07.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:07 smithi098 ceph-mon[34725]: pgmap v399: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-09-21T00:22:07.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:07 smithi027 ceph-mon[30745]: pgmap v399: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-09-21T00:22:09.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:09 smithi089 ceph-mon[34481]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-21T00:22:09.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:09 smithi098 ceph-mon[34725]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-21T00:22:09.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:09 smithi027 ceph-mon[30745]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-21T00:22:11.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:11 smithi089 ceph-mon[34481]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-21T00:22:11.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:11 smithi098 ceph-mon[34725]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-21T00:22:11.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:11 smithi027 ceph-mon[30745]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-21T00:22:13.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:13 smithi089 ceph-mon[34481]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-21T00:22:13.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:13 smithi098 ceph-mon[34725]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-21T00:22:13.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:13 smithi027 ceph-mon[30745]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-21T00:22:15.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:15 smithi089 ceph-mon[34481]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 21 op/s 2024-09-21T00:22:15.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:15 smithi098 ceph-mon[34725]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 21 op/s 2024-09-21T00:22:15.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:15 smithi027 ceph-mon[30745]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 21 op/s 2024-09-21T00:22:17.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:17 smithi089 ceph-mon[34481]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-21T00:22:17.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:17 smithi098 ceph-mon[34725]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-21T00:22:17.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:17 smithi027 ceph-mon[30745]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-21T00:22:19.086 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 14.5 2024-09-21T00:22:19.087 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:22:19.521 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:22:19.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:19 smithi089 ceph-mon[34481]: pgmap v405: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-21T00:22:19.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:19 smithi098 ceph-mon[34725]: pgmap v405: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-21T00:22:19.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:19 smithi027 ceph-mon[30745]: pgmap v405: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-21T00:22:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.255+0000 7f5289400640 1 -- 172.21.15.27:0/3431384555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5284103910 msgr2=0x7f5284103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.255+0000 7f5289400640 1 --2- 172.21.15.27:0/3431384555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5284103910 0x7f5284103d20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5274009900 tx=0x7f527402a410 comp rx=0 tx=0).stop 2024-09-21T00:22:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.255+0000 7f5289400640 1 -- 172.21.15.27:0/3431384555 shutdown_connections 2024-09-21T00:22:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.255+0000 7f5289400640 1 --2- 172.21.15.27:0/3431384555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5284109b90 0x7f52841103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.255+0000 7f5289400640 1 --2- 172.21.15.27:0/3431384555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5284109240 0x7f5284109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.255+0000 7f5289400640 1 --2- 172.21.15.27:0/3431384555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5284103910 0x7f5284103d20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.255+0000 7f5289400640 1 -- 172.21.15.27:0/3431384555 >> 172.21.15.27:0/3431384555 conn(0x7f52840ff390 msgr2=0x7f52841017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.255+0000 7f5289400640 1 -- 172.21.15.27:0/3431384555 shutdown_connections 2024-09-21T00:22:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.255+0000 7f5289400640 1 -- 172.21.15.27:0/3431384555 wait complete. 2024-09-21T00:22:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.256+0000 7f5289400640 1 Processor -- start 2024-09-21T00:22:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.256+0000 7f5289400640 1 -- start start 2024-09-21T00:22:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.257+0000 7f5289400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5284103910 0x7f5284075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.257+0000 7f5289400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5284109240 0x7f5284075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.257+0000 7f5289400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5284109b90 0x7f5284076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.257+0000 7f5289400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5284113850 con 0x7f5284109240 2024-09-21T00:22:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.257+0000 7f5289400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f52841136d0 con 0x7f5284103910 2024-09-21T00:22:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.257+0000 7f5289400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f52841139d0 con 0x7f5284109b90 2024-09-21T00:22:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.257+0000 7f5283400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5284109240 0x7f5284075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.257+0000 7f5283400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5284109240 0x7f5284075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50416/0 (socket says 172.21.15.27:50416) 2024-09-21T00:22:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.257+0000 7f5283400640 1 -- 172.21.15.27:0/453104314 learned_addr learned my addr 172.21.15.27:0/453104314 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.257+0000 7f5288a00640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5284109b90 0x7f5284076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.257+0000 7f5283e00640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5284103910 0x7f5284075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.258+0000 7f5283400640 1 -- 172.21.15.27:0/453104314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5284109b90 msgr2=0x7f5284076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.258+0000 7f5283400640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5284109b90 0x7f5284076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.258+0000 7f5283400640 1 -- 172.21.15.27:0/453104314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5284103910 msgr2=0x7f5284075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.258+0000 7f5283400640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5284103910 0x7f5284075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.258+0000 7f5283400640 1 -- 172.21.15.27:0/453104314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52841a8580 con 0x7f5284109240 2024-09-21T00:22:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.258+0000 7f5288a00640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5284109b90 0x7f5284076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:22:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.258+0000 7f5283e00640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5284103910 0x7f5284075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:22:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.258+0000 7f5283400640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5284109240 0x7f5284075c50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f527000e970 tx=0x7f527000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.258+0000 7f5280c00640 1 -- 172.21.15.27:0/453104314 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527000cd80 con 0x7f5284109240 2024-09-21T00:22:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.259+0000 7f5280c00640 1 -- 172.21.15.27:0/453104314 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52700035f0 con 0x7f5284109240 2024-09-21T00:22:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.259+0000 7f5289400640 1 -- 172.21.15.27:0/453104314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52841a8870 con 0x7f5284109240 2024-09-21T00:22:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.259+0000 7f5280c00640 1 -- 172.21.15.27:0/453104314 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270010690 con 0x7f5284109240 2024-09-21T00:22:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.259+0000 7f5289400640 1 -- 172.21.15.27:0/453104314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52841a8dc0 con 0x7f5284109240 2024-09-21T00:22:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.262+0000 7f5291bcb740 1 Processor -- start 2024-09-21T00:22:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.262+0000 7f5291bcb740 1 -- start start 2024-09-21T00:22:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.263+0000 7f5291bcb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197ed1b0 0x559519b91e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.263+0000 7f5291bcb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197c42f0 0x559519b92350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.263+0000 7f5291bcb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197cc4d0 0x559519b92890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.263+0000 7f5291bcb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559519b076b0 con 0x5595197cc4d0 2024-09-21T00:22:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.263+0000 7f5291bcb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559519af6ae0 con 0x5595197c42f0 2024-09-21T00:22:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.263+0000 7f5291bcb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559519afa8f0 con 0x5595197ed1b0 2024-09-21T00:22:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.263+0000 7f5283e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197c42f0 0x559519b92350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.263+0000 7f5280c00640 1 -- 172.21.15.27:0/453104314 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5270003aa0 con 0x7f5284109240 2024-09-21T00:22:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.263+0000 7f5283e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197c42f0 0x559519b92350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37630/0 (socket says 172.21.15.27:37630) 2024-09-21T00:22:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.263+0000 7f5283e00640 1 -- 172.21.15.27:0/230693764 learned_addr learned my addr 172.21.15.27:0/230693764 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.263+0000 7f5288a00640 1 --2- 172.21.15.27:0/230693764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197ed1b0 0x559519b91e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.263+0000 7f5283400640 1 --2- 172.21.15.27:0/230693764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197cc4d0 0x559519b92890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.263+0000 7f5283e00640 1 -- 172.21.15.27:0/230693764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197ed1b0 msgr2=0x559519b91e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.263+0000 7f5283e00640 1 --2- 172.21.15.27:0/230693764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197ed1b0 0x559519b91e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.263+0000 7f5283e00640 1 -- 172.21.15.27:0/230693764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197cc4d0 msgr2=0x559519b92890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.264+0000 7f5283e00640 1 --2- 172.21.15.27:0/230693764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197cc4d0 0x559519b92890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.264+0000 7f5283e00640 1 -- 172.21.15.27:0/230693764 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559519ac3780 con 0x5595197c42f0 2024-09-21T00:22:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.264+0000 7f5283400640 1 --2- 172.21.15.27:0/230693764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197cc4d0 0x559519b92890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:22:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.264+0000 7f5283e00640 1 --2- 172.21.15.27:0/230693764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197c42f0 0x559519b92350 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5274009900 tx=0x7f5274005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48fecc75fc48cd95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.264+0000 7f5279200640 1 -- 172.21.15.27:0/230693764 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274035040 con 0x5595197c42f0 2024-09-21T00:22:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.264+0000 7f5279200640 1 -- 172.21.15.27:0/230693764 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5274031720 con 0x5595197c42f0 2024-09-21T00:22:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.264+0000 7f5279200640 1 -- 172.21.15.27:0/230693764 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52740049d0 con 0x5595197c42f0 2024-09-21T00:22:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.265+0000 7f5280c00640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f52680b66a0 0x7f52680b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.265+0000 7f5280c00640 1 -- 172.21.15.27:0/453104314 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f52700e7db0 con 0x7f5284109240 2024-09-21T00:22:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.265+0000 7f5291bcb740 1 -- 172.21.15.27:0/230693764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197c42f0 msgr2=0x559519b92350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.265+0000 7f5291bcb740 1 --2- 172.21.15.27:0/230693764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197c42f0 0x559519b92350 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5274009900 tx=0x7f5274005e20 comp rx=0 tx=0).stop 2024-09-21T00:22:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.265+0000 7f5288a00640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f52680b66a0 0x7f52680b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.266+0000 7f5291bcb740 1 -- 172.21.15.27:0/230693764 shutdown_connections 2024-09-21T00:22:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.266+0000 7f5291bcb740 1 --2- 172.21.15.27:0/230693764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197cc4d0 0x559519b92890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.266+0000 7f5291bcb740 1 --2- 172.21.15.27:0/230693764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197c42f0 0x559519b92350 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.266+0000 7f5291bcb740 1 --2- 172.21.15.27:0/230693764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197ed1b0 0x559519b91e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.266+0000 7f5291bcb740 1 -- 172.21.15.27:0/230693764 >> 172.21.15.27:0/230693764 conn(0x5595197e1300 msgr2=0x559519b8fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.266+0000 7f5291bcb740 1 -- 172.21.15.27:0/230693764 shutdown_connections 2024-09-21T00:22:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.266+0000 7f5291bcb740 1 -- 172.21.15.27:0/230693764 wait complete. 2024-09-21T00:22:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.266+0000 7f5288a00640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f52680b66a0 0x7f52680b8bb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f527c009fd0 tx=0x7f527c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.266+0000 7f5291bcb740 1 -- start start 2024-09-21T00:22:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.266+0000 7f5291bcb740 1 Processor -- start 2024-09-21T00:22:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.267+0000 7f5291bcb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197c42f0 0x559519b9d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.267+0000 7f5291bcb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197cc4d0 0x559519b9d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.267+0000 7f5291bcb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197ed1b0 0x559519b9dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.267+0000 7f5291bcb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559519a2eab0 con 0x5595197c42f0 2024-09-21T00:22:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.267+0000 7f5291bcb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559519a9d570 con 0x5595197ed1b0 2024-09-21T00:22:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.267+0000 7f5291bcb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559519ba0530 con 0x5595197cc4d0 2024-09-21T00:22:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.267+0000 7f5291bcb740 10 client.0 _ll_register_callbacks cb 0x559519aac100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.267+0000 7f5283e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197c42f0 0x559519b9d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.267+0000 7f5283e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197c42f0 0x559519b9d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50444/0 (socket says 172.21.15.27:50444) 2024-09-21T00:22:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.267+0000 7f5283400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197cc4d0 0x559519b9d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.267+0000 7f5283e00640 1 -- 172.21.15.27:0/1158760279 learned_addr learned my addr 172.21.15.27:0/1158760279 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.268+0000 7f5288a00640 1 --2- 172.21.15.27:0/1158760279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197ed1b0 0x559519b9dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.268+0000 7f5283400640 1 -- 172.21.15.27:0/1158760279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197ed1b0 msgr2=0x559519b9dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.268+0000 7f5283400640 1 --2- 172.21.15.27:0/1158760279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197ed1b0 0x559519b9dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.268+0000 7f5283400640 1 -- 172.21.15.27:0/1158760279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197c42f0 msgr2=0x559519b9d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.268+0000 7f5283400640 1 --2- 172.21.15.27:0/1158760279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197c42f0 0x559519b9d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.268+0000 7f5283400640 1 -- 172.21.15.27:0/1158760279 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559519858790 con 0x5595197cc4d0 2024-09-21T00:22:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.268+0000 7f5288a00640 1 --2- 172.21.15.27:0/1158760279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197ed1b0 0x559519b9dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:22:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.268+0000 7f5283400640 1 --2- 172.21.15.27:0/1158760279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197cc4d0 0x559519b9d960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f52700e5110 tx=0x7f52700ad160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c703f2690c3329fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.268+0000 7f526d600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:22:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.268+0000 7f526d600640 1 -- 172.21.15.27:0/1158760279 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52700adf10 con 0x5595197cc4d0 2024-09-21T00:22:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.269+0000 7f5291bcb740 1 -- 172.21.15.27:0/1158760279 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5274031720 con 0x5595197cc4d0 2024-09-21T00:22:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.269+0000 7f526d600640 1 -- 172.21.15.27:0/1158760279 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f52700aeb40 con 0x5595197cc4d0 2024-09-21T00:22:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.269+0000 7f526d600640 1 -- 172.21.15.27:0/1158760279 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f52700b22f0 con 0x5595197cc4d0 2024-09-21T00:22:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.269+0000 7f5291bcb740 10 client.25540 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.269+0000 7f5291bcb740 1 -- 172.21.15.27:0/1158760279 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5595198d1620 con 0x5595197cc4d0 2024-09-21T00:22:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.269+0000 7f526d600640 1 -- 172.21.15.27:0/1158760279 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5274031720 con 0x5595197cc4d0 2024-09-21T00:22:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.269+0000 7f526d600640 1 -- 172.21.15.27:0/1158760279 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f52700b3480 con 0x5595197cc4d0 2024-09-21T00:22:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.270+0000 7f5291bcb740 10 client.25540 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.270+0000 7f5291bcb740 10 client.25540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:22:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.270+0000 7f5291bcb740 20 client.25540 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.270+0000 7f5291bcb740 1 --2- 172.21.15.27:0/1158760279 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5595197abd80 0x559519baf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.270+0000 7f5291bcb740 4 client.25540 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:22:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.270+0000 7f5291bcb740 1 -- 172.21.15.27:0/1158760279 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559519b09600 con 0x5595197abd80 2024-09-21T00:22:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.270+0000 7f5283e00640 1 --2- 172.21.15.27:0/1158760279 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5595197abd80 0x559519baf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.271+0000 7f5283e00640 1 --2- 172.21.15.27:0/1158760279 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5595197abd80 0x559519baf430 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbf9932a8c7617ca server_cookie=c83760ebb69de46f in_seq=0 out_seq=0 2024-09-21T00:22:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.271+0000 7f526d600640 10 client.25540 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.272+0000 7f526d600640 1 -- 172.21.15.27:0/1158760279 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559519b09600 con 0x5595197abd80 2024-09-21T00:22:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.272+0000 7f526d600640 10 client.25540 handle_command_reply: tid=0 2024-09-21T00:22:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.272+0000 7f5291bcb740 1 client.25540 shutdown 2024-09-21T00:22:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.272+0000 7f5291bcb740 1 -- 172.21.15.27:0/1158760279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197cc4d0 msgr2=0x559519b9d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.272+0000 7f5291bcb740 1 --2- 172.21.15.27:0/1158760279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197cc4d0 0x559519b9d960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f52700e5110 tx=0x7f52700ad160 comp rx=0 tx=0).stop 2024-09-21T00:22:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.272+0000 7f5291bcb740 1 -- 172.21.15.27:0/1158760279 shutdown_connections 2024-09-21T00:22:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.272+0000 7f5291bcb740 1 --2- 172.21.15.27:0/1158760279 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5595197abd80 0x559519baf430 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.272+0000 7f5291bcb740 1 --2- 172.21.15.27:0/1158760279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197ed1b0 0x559519b9dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.272+0000 7f5291bcb740 1 --2- 172.21.15.27:0/1158760279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197cc4d0 0x559519b9d960 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.272+0000 7f5291bcb740 1 --2- 172.21.15.27:0/1158760279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197c42f0 0x559519b9d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.272+0000 7f5291bcb740 1 -- 172.21.15.27:0/1158760279 >> 172.21.15.27:0/1158760279 conn(0x5595197e1300 msgr2=0x559519b92e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.272+0000 7f526d600640 0 client.25540 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.272+0000 7f5291bcb740 1 -- 172.21.15.27:0/1158760279 shutdown_connections 2024-09-21T00:22:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.272+0000 7f5291bcb740 1 -- 172.21.15.27:0/1158760279 wait complete. 2024-09-21T00:22:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.273+0000 7f5291bcb740 20 client.25540 trim_cache size 0 max 16384 2024-09-21T00:22:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.298+0000 7f5291bcb740 1 Processor -- start 2024-09-21T00:22:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.299+0000 7f5291bcb740 1 -- start start 2024-09-21T00:22:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.299+0000 7f5291bcb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197c42f0 0x559519b8ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.299+0000 7f5291bcb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197cc4d0 0x559519b9f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.299+0000 7f5291bcb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197ed1b0 0x559519b9a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.299+0000 7f5291bcb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f52841b57a0 con 0x5595197c42f0 2024-09-21T00:22:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.299+0000 7f5291bcb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f52841b3070 con 0x5595197ed1b0 2024-09-21T00:22:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.299+0000 7f5291bcb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f52841b0ec0 con 0x5595197cc4d0 2024-09-21T00:22:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.299+0000 7f5283400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197cc4d0 0x559519b9f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.299+0000 7f5283e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197c42f0 0x559519b8ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.300+0000 7f5283400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197cc4d0 0x559519b9f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56400/0 (socket says 172.21.15.27:56400) 2024-09-21T00:22:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.300+0000 7f5288a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197ed1b0 0x559519b9a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.300+0000 7f5283400640 1 -- 172.21.15.27:0/1441743947 learned_addr learned my addr 172.21.15.27:0/1441743947 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.300+0000 7f5283e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197c42f0 0x559519b8ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50460/0 (socket says 172.21.15.27:50460) 2024-09-21T00:22:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.300+0000 7f5283400640 1 -- 172.21.15.27:0/1441743947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197ed1b0 msgr2=0x559519b9a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.300+0000 7f5283400640 1 --2- 172.21.15.27:0/1441743947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197ed1b0 0x559519b9a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.300+0000 7f5283400640 1 -- 172.21.15.27:0/1441743947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197c42f0 msgr2=0x559519b8ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.300+0000 7f5283400640 1 --2- 172.21.15.27:0/1441743947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197c42f0 0x559519b8ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.300+0000 7f5283400640 1 -- 172.21.15.27:0/1441743947 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559519baf7a0 con 0x5595197cc4d0 2024-09-21T00:22:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.301+0000 7f5283400640 1 --2- 172.21.15.27:0/1441743947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197cc4d0 0x559519b9f050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f52700058c0 tx=0x7f5270000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9ae974a03e66ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.301+0000 7f526fe00640 1 -- 172.21.15.27:0/1441743947 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52700c8e60 con 0x5595197cc4d0 2024-09-21T00:22:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.301+0000 7f526fe00640 1 -- 172.21.15.27:0/1441743947 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52700adf10 con 0x5595197cc4d0 2024-09-21T00:22:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.301+0000 7f526fe00640 1 -- 172.21.15.27:0/1441743947 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52700b6770 con 0x5595197cc4d0 2024-09-21T00:22:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.302+0000 7f5291bcb740 1 -- 172.21.15.27:0/1441743947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197cc4d0 msgr2=0x559519b9f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.302+0000 7f5291bcb740 1 --2- 172.21.15.27:0/1441743947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197cc4d0 0x559519b9f050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f52700058c0 tx=0x7f5270000bc0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.302+0000 7f5291bcb740 1 -- 172.21.15.27:0/1441743947 shutdown_connections 2024-09-21T00:22:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.302+0000 7f5291bcb740 1 --2- 172.21.15.27:0/1441743947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197ed1b0 0x559519b9a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.302+0000 7f5291bcb740 1 --2- 172.21.15.27:0/1441743947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197cc4d0 0x559519b9f050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.302+0000 7f5291bcb740 1 --2- 172.21.15.27:0/1441743947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197c42f0 0x559519b8ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.302+0000 7f5291bcb740 1 -- 172.21.15.27:0/1441743947 >> 172.21.15.27:0/1441743947 conn(0x5595197e1300 msgr2=0x559519b9e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.302+0000 7f5291bcb740 1 -- 172.21.15.27:0/1441743947 shutdown_connections 2024-09-21T00:22:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.302+0000 7f5291bcb740 1 -- 172.21.15.27:0/1441743947 wait complete. 2024-09-21T00:22:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.303+0000 7f5291bcb740 1 -- start start 2024-09-21T00:22:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.303+0000 7f5291bcb740 1 Processor -- start 2024-09-21T00:22:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.303+0000 7f5291bcb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197c42f0 0x559519ba61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.303+0000 7f5291bcb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197cc4d0 0x559519ba3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.303+0000 7f5291bcb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197ed1b0 0x559519ba3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.304+0000 7f5291bcb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f52841b1530 con 0x5595197cc4d0 2024-09-21T00:22:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.304+0000 7f5291bcb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f52841b29a0 con 0x5595197c42f0 2024-09-21T00:22:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.304+0000 7f5291bcb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f52841b1c00 con 0x5595197ed1b0 2024-09-21T00:22:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.304+0000 7f5291bcb740 10 client.0 _ll_register_callbacks cb 0x7f52700ad6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.304+0000 7f5283400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197cc4d0 0x559519ba3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.304+0000 7f5283e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197c42f0 0x559519ba61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.304+0000 7f5283e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197c42f0 0x559519ba61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37662/0 (socket says 172.21.15.27:37662) 2024-09-21T00:22:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.304+0000 7f5283e00640 1 -- 172.21.15.27:0/2805551356 learned_addr learned my addr 172.21.15.27:0/2805551356 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.304+0000 7f5288a00640 1 --2- 172.21.15.27:0/2805551356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197ed1b0 0x559519ba3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.304+0000 7f5283e00640 1 -- 172.21.15.27:0/2805551356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197ed1b0 msgr2=0x559519ba3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.304+0000 7f5283e00640 1 --2- 172.21.15.27:0/2805551356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197ed1b0 0x559519ba3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.304+0000 7f5283e00640 1 -- 172.21.15.27:0/2805551356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197cc4d0 msgr2=0x559519ba3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.304+0000 7f5283e00640 1 --2- 172.21.15.27:0/2805551356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197cc4d0 0x559519ba3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.304+0000 7f5283e00640 1 -- 172.21.15.27:0/2805551356 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559519c18df0 con 0x5595197c42f0 2024-09-21T00:22:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.305+0000 7f5283e00640 1 --2- 172.21.15.27:0/2805551356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197c42f0 0x559519ba61f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f52740096c0 tx=0x7f5274005400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=582b5ec22e084cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.305+0000 7f526d600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:22:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.305+0000 7f526d600640 1 -- 172.21.15.27:0/2805551356 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274035040 con 0x5595197c42f0 2024-09-21T00:22:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.305+0000 7f5291bcb740 1 -- 172.21.15.27:0/2805551356 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52700adf10 con 0x5595197c42f0 2024-09-21T00:22:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.305+0000 7f526d600640 1 -- 172.21.15.27:0/2805551356 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f5274004040 con 0x5595197c42f0 2024-09-21T00:22:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.306+0000 7f526d600640 1 -- 172.21.15.27:0/2805551356 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5274031e40 con 0x5595197c42f0 2024-09-21T00:22:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.306+0000 7f5291bcb740 10 client.25568 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.306+0000 7f5291bcb740 1 -- 172.21.15.27:0/2805551356 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559519c19fc0 con 0x5595197c42f0 2024-09-21T00:22:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.306+0000 7f526d600640 1 -- 172.21.15.27:0/2805551356 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52700adf10 con 0x5595197c42f0 2024-09-21T00:22:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.306+0000 7f526d600640 1 -- 172.21.15.27:0/2805551356 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5274003aa0 con 0x5595197c42f0 2024-09-21T00:22:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.306+0000 7f5291bcb740 10 client.25568 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.306+0000 7f5291bcb740 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:22:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.306+0000 7f5291bcb740 20 client.25568 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.306+0000 7f5291bcb740 1 --2- 172.21.15.27:0/2805551356 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5595197811c0 0x559519c1c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.306+0000 7f5291bcb740 4 client.25568 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:22:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.306+0000 7f5291bcb740 1 -- 172.21.15.27:0/2805551356 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559519b050b0 con 0x5595197811c0 2024-09-21T00:22:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.307+0000 7f5283400640 1 --2- 172.21.15.27:0/2805551356 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5595197811c0 0x559519c1c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.307+0000 7f5283400640 1 --2- 172.21.15.27:0/2805551356 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5595197811c0 0x559519c1c360 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1c87920f9f2b0af server_cookie=5d1ed55c3219685b in_seq=0 out_seq=0 2024-09-21T00:22:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.307+0000 7f526d600640 10 client.25568 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.313+0000 7f526d600640 1 -- 172.21.15.27:0/2805551356 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559519b050b0 con 0x5595197811c0 2024-09-21T00:22:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.313+0000 7f526d600640 10 client.25568 handle_command_reply: tid=0 2024-09-21T00:22:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.313+0000 7f5291bcb740 1 client.25568 shutdown 2024-09-21T00:22:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.313+0000 7f5291bcb740 1 -- 172.21.15.27:0/2805551356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197c42f0 msgr2=0x559519ba61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.313+0000 7f5291bcb740 1 --2- 172.21.15.27:0/2805551356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197c42f0 0x559519ba61f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f52740096c0 tx=0x7f5274005400 comp rx=0 tx=0).stop 2024-09-21T00:22:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.313+0000 7f5291bcb740 1 -- 172.21.15.27:0/2805551356 shutdown_connections 2024-09-21T00:22:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.313+0000 7f5291bcb740 1 --2- 172.21.15.27:0/2805551356 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5595197811c0 0x559519c1c360 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.313+0000 7f5291bcb740 1 --2- 172.21.15.27:0/2805551356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595197ed1b0 0x559519ba3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.313+0000 7f5291bcb740 1 --2- 172.21.15.27:0/2805551356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595197cc4d0 0x559519ba3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.313+0000 7f5291bcb740 1 --2- 172.21.15.27:0/2805551356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595197c42f0 0x559519ba61f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.313+0000 7f526d600640 0 client.25568 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.313+0000 7f5291bcb740 1 -- 172.21.15.27:0/2805551356 >> 172.21.15.27:0/2805551356 conn(0x5595197e1300 msgr2=0x559519b8eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.313+0000 7f5291bcb740 1 -- 172.21.15.27:0/2805551356 shutdown_connections 2024-09-21T00:22:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.314+0000 7f5291bcb740 1 -- 172.21.15.27:0/2805551356 wait complete. 2024-09-21T00:22:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.314+0000 7f5291bcb740 20 client.25568 trim_cache size 0 max 16384 2024-09-21T00:22:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:22:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:22:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:22:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:22:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.315+0000 7f5289400640 1 -- 172.21.15.27:0/453104314 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f52680b66a0 msgr2=0x7f52680b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.315+0000 7f5289400640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f52680b66a0 0x7f52680b8bb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f527c009fd0 tx=0x7f527c009500 comp rx=0 tx=0).stop 2024-09-21T00:22:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.315+0000 7f5289400640 1 -- 172.21.15.27:0/453104314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5284109240 msgr2=0x7f5284075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.315+0000 7f5289400640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5284109240 0x7f5284075c50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f527000e970 tx=0x7f527000ee30 comp rx=0 tx=0).stop 2024-09-21T00:22:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.315+0000 7f5289400640 1 -- 172.21.15.27:0/453104314 shutdown_connections 2024-09-21T00:22:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.315+0000 7f5289400640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f52680b66a0 0x7f52680b8bb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.315+0000 7f5289400640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5284109b90 0x7f5284076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.315+0000 7f5289400640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5284109240 0x7f5284075c50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.315+0000 7f5289400640 1 --2- 172.21.15.27:0/453104314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5284103910 0x7f5284075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.315+0000 7f5289400640 1 -- 172.21.15.27:0/453104314 >> 172.21.15.27:0/453104314 conn(0x7f52840ff390 msgr2=0x7f52841017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.315+0000 7f5289400640 1 -- 172.21.15.27:0/453104314 shutdown_connections 2024-09-21T00:22:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:20.315+0000 7f5289400640 1 -- 172.21.15.27:0/453104314 wait complete. 2024-09-21T00:22:20.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.505+0000 7fb28be00640 1 -- 172.21.15.27:0/2243751906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb28c10dc20 msgr2=0x7fb28c110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:20.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.505+0000 7fb28be00640 1 --2- 172.21.15.27:0/2243751906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb28c10dc20 0x7fb28c110110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb284009be0 tx=0x7fb28402a6c0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.505+0000 7fb28be00640 1 -- 172.21.15.27:0/2243751906 shutdown_connections 2024-09-21T00:22:20.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.505+0000 7fb28be00640 1 --2- 172.21.15.27:0/2243751906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb28c10dc20 0x7fb28c110110 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.505+0000 7fb28be00640 1 --2- 172.21.15.27:0/2243751906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb28c105110 0x7fb28c10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.505+0000 7fb28be00640 1 --2- 172.21.15.27:0/2243751906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb28c1047c0 0x7fb28c104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.505+0000 7fb28be00640 1 -- 172.21.15.27:0/2243751906 >> 172.21.15.27:0/2243751906 conn(0x7fb28c0fe2c0 msgr2=0x7fb28c100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:20.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.505+0000 7fb28be00640 1 -- 172.21.15.27:0/2243751906 shutdown_connections 2024-09-21T00:22:20.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.505+0000 7fb28be00640 1 -- 172.21.15.27:0/2243751906 wait complete. 2024-09-21T00:22:20.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.506+0000 7fb28be00640 1 Processor -- start 2024-09-21T00:22:20.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.506+0000 7fb28be00640 1 -- start start 2024-09-21T00:22:20.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.507+0000 7fb28be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb28c1047c0 0x7fb28c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.507+0000 7fb28be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb28c105110 0x7fb28c19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.507+0000 7fb28be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb28c10dc20 0x7fb28c19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.507+0000 7fb28be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c113880 con 0x7fb28c105110 2024-09-21T00:22:20.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.507+0000 7fb28be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c113700 con 0x7fb28c10dc20 2024-09-21T00:22:20.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.507+0000 7fb28be00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c113a00 con 0x7fb28c1047c0 2024-09-21T00:22:20.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.507+0000 7fb28aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb28c1047c0 0x7fb28c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.507+0000 7fb28aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb28c1047c0 0x7fb28c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56426/0 (socket says 172.21.15.27:56426) 2024-09-21T00:22:20.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.507+0000 7fb28aa00640 1 -- 172.21.15.27:0/3583159221 learned_addr learned my addr 172.21.15.27:0/3583159221 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:20.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.507+0000 7fb28a000640 1 --2- 172.21.15.27:0/3583159221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb28c105110 0x7fb28c19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.508+0000 7fb28b400640 1 --2- 172.21.15.27:0/3583159221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb28c10dc20 0x7fb28c19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.508+0000 7fb28aa00640 1 -- 172.21.15.27:0/3583159221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb28c10dc20 msgr2=0x7fb28c19fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:20.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.508+0000 7fb28aa00640 1 --2- 172.21.15.27:0/3583159221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb28c10dc20 0x7fb28c19fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.508+0000 7fb28aa00640 1 -- 172.21.15.27:0/3583159221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb28c105110 msgr2=0x7fb28c19a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:20.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.508+0000 7fb28aa00640 1 --2- 172.21.15.27:0/3583159221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb28c105110 0x7fb28c19a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.508+0000 7fb28aa00640 1 -- 172.21.15.27:0/3583159221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28c1a01a0 con 0x7fb28c1047c0 2024-09-21T00:22:20.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.508+0000 7fb28aa00640 1 --2- 172.21.15.27:0/3583159221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb28c1047c0 0x7fb28c19a230 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00e9c0 tx=0x7fb27c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:20.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.509+0000 7fb283400640 1 -- 172.21.15.27:0/3583159221 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c00cde0 con 0x7fb28c1047c0 2024-09-21T00:22:20.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.509+0000 7fb283400640 1 -- 172.21.15.27:0/3583159221 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb27c0035f0 con 0x7fb28c1047c0 2024-09-21T00:22:20.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.509+0000 7fb28be00640 1 -- 172.21.15.27:0/3583159221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28c1a0490 con 0x7fb28c1047c0 2024-09-21T00:22:20.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.509+0000 7fb28be00640 1 -- 172.21.15.27:0/3583159221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb28c1023e0 con 0x7fb28c1047c0 2024-09-21T00:22:20.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.509+0000 7fb283400640 1 -- 172.21.15.27:0/3583159221 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c010690 con 0x7fb28c1047c0 2024-09-21T00:22:20.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.510+0000 7fb28be00640 1 -- 172.21.15.27:0/3583159221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb26c0052c0 con 0x7fb28c1047c0 2024-09-21T00:22:20.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.513+0000 7fb283400640 1 -- 172.21.15.27:0/3583159221 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb27c004000 con 0x7fb28c1047c0 2024-09-21T00:22:20.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.515+0000 7fb283400640 1 --2- 172.21.15.27:0/3583159221 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb2700b6700 0x7fb2700b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:20.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.515+0000 7fb283400640 1 -- 172.21.15.27:0/3583159221 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fb27c0e7c30 con 0x7fb28c1047c0 2024-09-21T00:22:20.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.515+0000 7fb28a000640 1 --2- 172.21.15.27:0/3583159221 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb2700b6700 0x7fb2700b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:20.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.516+0000 7fb28a000640 1 --2- 172.21.15.27:0/3583159221 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb2700b6700 0x7fb2700b8c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb28c1100e0 tx=0x7fb278005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:20.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.519+0000 7fb283400640 1 -- 172.21.15.27:0/3583159221 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb27c009510 con 0x7fb28c1047c0 2024-09-21T00:22:20.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.779+0000 7fb28be00640 1 -- 172.21.15.27:0/3583159221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb26c005880 con 0x7fb28c1047c0 2024-09-21T00:22:20.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.781+0000 7fb283400640 1 -- 172.21.15.27:0/3583159221 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11893 (secure 0 0 0) 0x7fb27c0addf0 con 0x7fb28c1047c0 2024-09-21T00:22:20.782 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:22:20.782 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":26,"btime":"2024-09-21T00:21:36:085073+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:21:35.264444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:22:20.783 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-09-21T00:22:20.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.784+0000 7fb28be00640 1 -- 172.21.15.27:0/3583159221 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb2700b6700 msgr2=0x7fb2700b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:20.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.784+0000 7fb28be00640 1 --2- 172.21.15.27:0/3583159221 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb2700b6700 0x7fb2700b8c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb28c1100e0 tx=0x7fb278005f90 comp rx=0 tx=0).stop 2024-09-21T00:22:20.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.784+0000 7fb28be00640 1 -- 172.21.15.27:0/3583159221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb28c1047c0 msgr2=0x7fb28c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:20.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.784+0000 7fb28be00640 1 --2- 172.21.15.27:0/3583159221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb28c1047c0 0x7fb28c19a230 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00e9c0 tx=0x7fb27c00ee80 comp rx=0 tx=0).stop 2024-09-21T00:22:20.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.784+0000 7fb28be00640 1 -- 172.21.15.27:0/3583159221 shutdown_connections 2024-09-21T00:22:20.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.784+0000 7fb28be00640 1 --2- 172.21.15.27:0/3583159221 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb2700b6700 0x7fb2700b8c10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.784+0000 7fb28be00640 1 --2- 172.21.15.27:0/3583159221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb28c10dc20 0x7fb28c19fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.784+0000 7fb28be00640 1 --2- 172.21.15.27:0/3583159221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb28c105110 0x7fb28c19a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.784+0000 7fb28be00640 1 --2- 172.21.15.27:0/3583159221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb28c1047c0 0x7fb28c19a230 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:20.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.784+0000 7fb28be00640 1 -- 172.21.15.27:0/3583159221 >> 172.21.15.27:0/3583159221 conn(0x7fb28c0fe2c0 msgr2=0x7fb28c107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:20.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.784+0000 7fb28be00640 1 -- 172.21.15.27:0/3583159221 shutdown_connections 2024-09-21T00:22:20.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:20.784+0000 7fb28be00640 1 -- 172.21.15.27:0/3583159221 wait complete. 2024-09-21T00:22:21.614 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0' 2024-09-21T00:22:21.614 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0 2024-09-21T00:22:21.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:21 smithi089 ceph-mon[34481]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T00:22:21.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:21 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3583159221' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:22:21.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:21 smithi098 ceph-mon[34725]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T00:22:21.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:21 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3583159221' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:22:21.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:21 smithi027 ceph-mon[30745]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T00:22:21.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:21 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3583159221' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:22:22.024 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:22:22.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.671+0000 7fe660e00640 1 -- 172.21.15.27:0/3392816669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c10b4f0 msgr2=0x7fe65c10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:22.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.671+0000 7fe660e00640 1 --2- 172.21.15.27:0/3392816669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c10b4f0 0x7fe65c10b900 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe648009900 tx=0x7fe64802a410 comp rx=0 tx=0).stop 2024-09-21T00:22:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.671+0000 7fe660e00640 1 -- 172.21.15.27:0/3392816669 shutdown_connections 2024-09-21T00:22:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.671+0000 7fe660e00640 1 --2- 172.21.15.27:0/3392816669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe65c10be40 0x7fe65c112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.671+0000 7fe660e00640 1 --2- 172.21.15.27:0/3392816669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c10b4f0 0x7fe65c10b900 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.671+0000 7fe660e00640 1 --2- 172.21.15.27:0/3392816669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c105bb0 0x7fe65c105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.671+0000 7fe660e00640 1 -- 172.21.15.27:0/3392816669 >> 172.21.15.27:0/3392816669 conn(0x7fe65c101640 msgr2=0x7fe65c103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.671+0000 7fe660e00640 1 -- 172.21.15.27:0/3392816669 shutdown_connections 2024-09-21T00:22:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.672+0000 7fe660e00640 1 -- 172.21.15.27:0/3392816669 wait complete. 2024-09-21T00:22:22.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.673+0000 7fe660e00640 1 Processor -- start 2024-09-21T00:22:22.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.673+0000 7fe660e00640 1 -- start start 2024-09-21T00:22:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.674+0000 7fe660e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c105bb0 0x7fe65c19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.674+0000 7fe660e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe65c10b4f0 0x7fe65c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.674+0000 7fe660e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c10be40 0x7fe65c1a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.674+0000 7fe660e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c115b50 con 0x7fe65c10be40 2024-09-21T00:22:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.674+0000 7fe660e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1159d0 con 0x7fe65c10b4f0 2024-09-21T00:22:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.674+0000 7fe660e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c115cd0 con 0x7fe65c105bb0 2024-09-21T00:22:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.674+0000 7fe65aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe65c10b4f0 0x7fe65c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.674+0000 7fe65aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe65c10b4f0 0x7fe65c19ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37694/0 (socket says 172.21.15.27:37694) 2024-09-21T00:22:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.674+0000 7fe65aa00640 1 -- 172.21.15.27:0/2660172279 learned_addr learned my addr 172.21.15.27:0/2660172279 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.674+0000 7fe65b400640 1 --2- 172.21.15.27:0/2660172279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c105bb0 0x7fe65c19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.675+0000 7fe65be00640 1 --2- 172.21.15.27:0/2660172279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c10be40 0x7fe65c1a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.675+0000 7fe65b400640 1 -- 172.21.15.27:0/2660172279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe65c10b4f0 msgr2=0x7fe65c19ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.675+0000 7fe65b400640 1 --2- 172.21.15.27:0/2660172279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe65c10b4f0 0x7fe65c19ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.675+0000 7fe65b400640 1 -- 172.21.15.27:0/2660172279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c10be40 msgr2=0x7fe65c1a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:22.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.675+0000 7fe65b400640 1 --2- 172.21.15.27:0/2660172279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c10be40 0x7fe65c1a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.675+0000 7fe65b400640 1 -- 172.21.15.27:0/2660172279 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65c1a2450 con 0x7fe65c105bb0 2024-09-21T00:22:22.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.675+0000 7fe65be00640 1 --2- 172.21.15.27:0/2660172279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c10be40 0x7fe65c1a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:22:22.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.676+0000 7fe65b400640 1 --2- 172.21.15.27:0/2660172279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c105bb0 0x7fe65c19c4e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00e9f0 tx=0x7fe64c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:22.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.676+0000 7fe653e00640 1 -- 172.21.15.27:0/2660172279 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c00ccc0 con 0x7fe65c105bb0 2024-09-21T00:22:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.676+0000 7fe653e00640 1 -- 172.21.15.27:0/2660172279 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe64c0108e0 con 0x7fe65c105bb0 2024-09-21T00:22:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.676+0000 7fe653e00640 1 -- 172.21.15.27:0/2660172279 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c018880 con 0x7fe65c105bb0 2024-09-21T00:22:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.676+0000 7fe660e00640 1 -- 172.21.15.27:0/2660172279 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65c1a2740 con 0x7fe65c105bb0 2024-09-21T00:22:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.677+0000 7fe660e00640 1 -- 172.21.15.27:0/2660172279 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65c1a8ea0 con 0x7fe65c105bb0 2024-09-21T00:22:22.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.681+0000 7fe6691c4740 1 Processor -- start 2024-09-21T00:22:22.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.682+0000 7fe6691c4740 1 -- start start 2024-09-21T00:22:22.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.682+0000 7fe6691c4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 0x56498896a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.682+0000 7fe6691c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564988599980 0x56498896aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.682+0000 7fe6691c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5649885a1b60 0x56498896af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.682+0000 7fe6691c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5649888dfca0 con 0x5649885a1b60 2024-09-21T00:22:22.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.682+0000 7fe6691c4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5649888cf0d0 con 0x5649885c2840 2024-09-21T00:22:22.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.682+0000 7fe6691c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5649888d2ee0 con 0x564988599980 2024-09-21T00:22:22.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.683+0000 7fe653e00640 1 -- 172.21.15.27:0/2660172279 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe64c010430 con 0x7fe65c105bb0 2024-09-21T00:22:22.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.683+0000 7fe65be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 0x56498896a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.683+0000 7fe65be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 0x56498896a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37698/0 (socket says 172.21.15.27:37698) 2024-09-21T00:22:22.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.683+0000 7fe65be00640 1 -- 172.21.15.27:0/3880024398 learned_addr learned my addr 172.21.15.27:0/3880024398 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:22.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.683+0000 7fe65aa00640 1 --2- 172.21.15.27:0/3880024398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5649885a1b60 0x56498896af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.684+0000 7fe65be00640 1 -- 172.21.15.27:0/3880024398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564988599980 msgr2=0x56498896aa50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:22:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.684+0000 7fe65be00640 1 --2- 172.21.15.27:0/3880024398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564988599980 0x56498896aa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.684+0000 7fe65be00640 1 -- 172.21.15.27:0/3880024398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5649885a1b60 msgr2=0x56498896af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.684+0000 7fe65be00640 1 --2- 172.21.15.27:0/3880024398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5649885a1b60 0x56498896af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.684+0000 7fe65be00640 1 -- 172.21.15.27:0/3880024398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564988881400 con 0x5649885c2840 2024-09-21T00:22:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.684+0000 7fe65be00640 1 --2- 172.21.15.27:0/3880024398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 0x56498896a510 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe654009ea0 tx=0x7fe654009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60c0777374c40bba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.685+0000 7fe650c00640 1 -- 172.21.15.27:0/3880024398 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe654004d20 con 0x5649885c2840 2024-09-21T00:22:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.685+0000 7fe650c00640 1 -- 172.21.15.27:0/3880024398 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6540054d0 con 0x5649885c2840 2024-09-21T00:22:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.685+0000 7fe650c00640 1 -- 172.21.15.27:0/3880024398 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6540117a0 con 0x5649885c2840 2024-09-21T00:22:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.686+0000 7fe653e00640 1 --2- 172.21.15.27:0/2660172279 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe6400b6770 0x7fe6400b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.686+0000 7fe65b400640 1 --2- 172.21.15.27:0/2660172279 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe6400b6770 0x7fe6400b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.686+0000 7fe6691c4740 1 -- 172.21.15.27:0/3880024398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 msgr2=0x56498896a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.686+0000 7fe6691c4740 1 --2- 172.21.15.27:0/3880024398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 0x56498896a510 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe654009ea0 tx=0x7fe654009f90 comp rx=0 tx=0).stop 2024-09-21T00:22:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.686+0000 7fe653e00640 1 -- 172.21.15.27:0/2660172279 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fe64c0e7f30 con 0x7fe65c105bb0 2024-09-21T00:22:22.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.687+0000 7fe6691c4740 1 -- 172.21.15.27:0/3880024398 shutdown_connections 2024-09-21T00:22:22.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.687+0000 7fe6691c4740 1 --2- 172.21.15.27:0/3880024398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5649885a1b60 0x56498896af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.687+0000 7fe6691c4740 1 --2- 172.21.15.27:0/3880024398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564988599980 0x56498896aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.687+0000 7fe6691c4740 1 --2- 172.21.15.27:0/3880024398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 0x56498896a510 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.687+0000 7fe6691c4740 1 -- 172.21.15.27:0/3880024398 >> 172.21.15.27:0/3880024398 conn(0x5649885b6990 msgr2=0x564988968400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:22.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.687+0000 7fe65b400640 1 --2- 172.21.15.27:0/2660172279 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe6400b6770 0x7fe6400b8c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x564988969190 tx=0x7fe64c0adb80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:22.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.687+0000 7fe6691c4740 1 -- 172.21.15.27:0/3880024398 shutdown_connections 2024-09-21T00:22:22.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.687+0000 7fe6691c4740 1 -- 172.21.15.27:0/3880024398 wait complete. 2024-09-21T00:22:22.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.688+0000 7fe6691c4740 1 -- start start 2024-09-21T00:22:22.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.688+0000 7fe6691c4740 1 Processor -- start 2024-09-21T00:22:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.689+0000 7fe6691c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564988599980 0x564988975bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.689+0000 7fe6691c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885a1b60 0x564988976110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.689+0000 7fe6691c4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 0x564988981fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.689+0000 7fe6691c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5649887307a0 con 0x564988599980 2024-09-21T00:22:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.689+0000 7fe6691c4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564988874a10 con 0x5649885c2840 2024-09-21T00:22:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.689+0000 7fe6691c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564988978cf0 con 0x5649885a1b60 2024-09-21T00:22:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.689+0000 7fe6691c4740 10 client.0 _ll_register_callbacks cb 0x564988884be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:22.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.689+0000 7fe65be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564988599980 0x564988975bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.689+0000 7fe65be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564988599980 0x564988975bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50518/0 (socket says 172.21.15.27:50518) 2024-09-21T00:22:22.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.690+0000 7fe65be00640 1 -- 172.21.15.27:0/2761008309 learned_addr learned my addr 172.21.15.27:0/2761008309 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:22.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.690+0000 7fe65aa00640 1 --2- 172.21.15.27:0/2761008309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885a1b60 0x564988976110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.690+0000 7fe65be00640 1 --2- 172.21.15.27:0/2761008309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 0x564988981fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.690+0000 7fe65be00640 1 -- 172.21.15.27:0/2761008309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885a1b60 msgr2=0x564988976110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:22.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.690+0000 7fe65be00640 1 --2- 172.21.15.27:0/2761008309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885a1b60 0x564988976110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.690+0000 7fe65be00640 1 -- 172.21.15.27:0/2761008309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 msgr2=0x564988981fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:22.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.690+0000 7fe65be00640 1 --2- 172.21.15.27:0/2761008309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 0x564988981fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.690+0000 7fe65be00640 1 -- 172.21.15.27:0/2761008309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56498863cb90 con 0x564988599980 2024-09-21T00:22:22.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.691+0000 7fe65be00640 1 --2- 172.21.15.27:0/2761008309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564988599980 0x564988975bd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe654009c30 tx=0x7fe654011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebcd408e3173cf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:22.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.691+0000 7fe645600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:22:22.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.691+0000 7fe645600640 1 -- 172.21.15.27:0/2761008309 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe654023040 con 0x564988599980 2024-09-21T00:22:22.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.691+0000 7fe645600640 1 -- 172.21.15.27:0/2761008309 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fe654004c40 con 0x564988599980 2024-09-21T00:22:22.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.691+0000 7fe6691c4740 1 -- 172.21.15.27:0/2761008309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6540054d0 con 0x564988599980 2024-09-21T00:22:22.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.692+0000 7fe645600640 1 -- 172.21.15.27:0/2761008309 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe654011520 con 0x564988599980 2024-09-21T00:22:22.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.692+0000 7fe6691c4740 10 client.15558 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:22.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.692+0000 7fe6691c4740 1 -- 172.21.15.27:0/2761008309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564988984a00 con 0x564988599980 2024-09-21T00:22:22.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.692+0000 7fe645600640 1 -- 172.21.15.27:0/2761008309 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6540054d0 con 0x564988599980 2024-09-21T00:22:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.692+0000 7fe645600640 1 -- 172.21.15.27:0/2761008309 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe654019550 con 0x564988599980 2024-09-21T00:22:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.692+0000 7fe6691c4740 10 client.15558 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.692+0000 7fe6691c4740 10 client.15558 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:22:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.692+0000 7fe6691c4740 20 client.15558 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.693+0000 7fe6691c4740 1 --2- 172.21.15.27:0/2761008309 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5649885811c0 0x564988987e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.693+0000 7fe6691c4740 4 client.15558 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:22:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.693+0000 7fe6691c4740 1 -- 172.21.15.27:0/2761008309 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649888e1bf0 con 0x5649885811c0 2024-09-21T00:22:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.693+0000 7fe65aa00640 1 --2- 172.21.15.27:0/2761008309 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5649885811c0 0x564988987e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.694+0000 7fe645600640 10 client.15558 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.694+0000 7fe65aa00640 1 --2- 172.21.15.27:0/2761008309 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5649885811c0 0x564988987e70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e11d9f213b1e129 server_cookie=f6e1f41e7d3ffbb1 in_seq=0 out_seq=0 2024-09-21T00:22:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.695+0000 7fe645600640 1 -- 172.21.15.27:0/2761008309 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5649888e1bf0 con 0x5649885811c0 2024-09-21T00:22:22.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.695+0000 7fe645600640 10 client.15558 handle_command_reply: tid=0 2024-09-21T00:22:22.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.695+0000 7fe6691c4740 1 client.15558 shutdown 2024-09-21T00:22:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.695+0000 7fe6691c4740 1 -- 172.21.15.27:0/2761008309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564988599980 msgr2=0x564988975bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.695+0000 7fe6691c4740 1 --2- 172.21.15.27:0/2761008309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564988599980 0x564988975bd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe654009c30 tx=0x7fe654011440 comp rx=0 tx=0).stop 2024-09-21T00:22:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.695+0000 7fe6691c4740 1 -- 172.21.15.27:0/2761008309 shutdown_connections 2024-09-21T00:22:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.695+0000 7fe6691c4740 1 --2- 172.21.15.27:0/2761008309 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5649885811c0 0x564988987e70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.695+0000 7fe6691c4740 1 --2- 172.21.15.27:0/2761008309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 0x564988981fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.695+0000 7fe6691c4740 1 --2- 172.21.15.27:0/2761008309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885a1b60 0x564988976110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.695+0000 7fe6691c4740 1 --2- 172.21.15.27:0/2761008309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564988599980 0x564988975bd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.695+0000 7fe6691c4740 1 -- 172.21.15.27:0/2761008309 >> 172.21.15.27:0/2761008309 conn(0x5649885b6990 msgr2=0x56498896d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.696+0000 7fe645600640 0 client.15558 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.696+0000 7fe6691c4740 1 -- 172.21.15.27:0/2761008309 shutdown_connections 2024-09-21T00:22:22.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.696+0000 7fe6691c4740 1 -- 172.21.15.27:0/2761008309 wait complete. 2024-09-21T00:22:22.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.696+0000 7fe6691c4740 20 client.15558 trim_cache size 0 max 16384 2024-09-21T00:22:22.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.715+0000 7fe6691c4740 1 Processor -- start 2024-09-21T00:22:22.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.715+0000 7fe6691c4740 1 -- start start 2024-09-21T00:22:22.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.716+0000 7fe6691c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564988599980 0x56498896bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.716+0000 7fe6691c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885a1b60 0x564988967c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.716+0000 7fe6691c4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 0x564988977ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.716+0000 7fe6691c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1b58b0 con 0x564988599980 2024-09-21T00:22:22.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.716+0000 7fe6691c4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1b3180 con 0x5649885c2840 2024-09-21T00:22:22.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.716+0000 7fe6691c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1b0fd0 con 0x5649885a1b60 2024-09-21T00:22:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.716+0000 7fe65aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885a1b60 0x564988967c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.716+0000 7fe65aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885a1b60 0x564988967c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56486/0 (socket says 172.21.15.27:56486) 2024-09-21T00:22:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.716+0000 7fe65aa00640 1 -- 172.21.15.27:0/1322574207 learned_addr learned my addr 172.21.15.27:0/1322574207 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.716+0000 7fe65be00640 1 --2- 172.21.15.27:0/1322574207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564988599980 0x56498896bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.717+0000 7fe65be00640 1 --2- 172.21.15.27:0/1322574207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 0x564988977ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.717+0000 7fe65aa00640 1 -- 172.21.15.27:0/1322574207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 msgr2=0x564988977ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.717+0000 7fe65aa00640 1 --2- 172.21.15.27:0/1322574207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 0x564988977ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.717+0000 7fe65aa00640 1 -- 172.21.15.27:0/1322574207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564988599980 msgr2=0x56498896bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.717+0000 7fe65aa00640 1 --2- 172.21.15.27:0/1322574207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564988599980 0x56498896bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.717+0000 7fe65aa00640 1 -- 172.21.15.27:0/1322574207 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56498896c390 con 0x5649885a1b60 2024-09-21T00:22:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.717+0000 7fe65be00640 1 --2- 172.21.15.27:0/1322574207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564988599980 0x56498896bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:22:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.717+0000 7fe65aa00640 1 --2- 172.21.15.27:0/1322574207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885a1b60 0x564988967c50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe64802a9d0 tx=0x7fe648005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74e59163c3c3f30f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.717+0000 7fe647e00640 1 -- 172.21.15.27:0/1322574207 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648049020 con 0x5649885a1b60 2024-09-21T00:22:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.718+0000 7fe647e00640 1 -- 172.21.15.27:0/1322574207 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6480046c0 con 0x5649885a1b60 2024-09-21T00:22:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.718+0000 7fe647e00640 1 -- 172.21.15.27:0/1322574207 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648035040 con 0x5649885a1b60 2024-09-21T00:22:22.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.719+0000 7fe6691c4740 1 -- 172.21.15.27:0/1322574207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885a1b60 msgr2=0x564988967c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:22.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.719+0000 7fe6691c4740 1 --2- 172.21.15.27:0/1322574207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885a1b60 0x564988967c50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe64802a9d0 tx=0x7fe648005a40 comp rx=0 tx=0).stop 2024-09-21T00:22:22.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.719+0000 7fe6691c4740 1 -- 172.21.15.27:0/1322574207 shutdown_connections 2024-09-21T00:22:22.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.719+0000 7fe6691c4740 1 --2- 172.21.15.27:0/1322574207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649885c2840 0x564988977ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.719+0000 7fe6691c4740 1 --2- 172.21.15.27:0/1322574207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885a1b60 0x564988967c50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.719+0000 7fe6691c4740 1 --2- 172.21.15.27:0/1322574207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564988599980 0x56498896bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.719+0000 7fe6691c4740 1 -- 172.21.15.27:0/1322574207 >> 172.21.15.27:0/1322574207 conn(0x5649885b6990 msgr2=0x56498896f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:22.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.720+0000 7fe6691c4740 1 -- 172.21.15.27:0/1322574207 shutdown_connections 2024-09-21T00:22:22.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.720+0000 7fe6691c4740 1 -- 172.21.15.27:0/1322574207 wait complete. 2024-09-21T00:22:22.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.720+0000 7fe6691c4740 1 -- start start 2024-09-21T00:22:22.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.721+0000 7fe6691c4740 1 Processor -- start 2024-09-21T00:22:22.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.722+0000 7fe6691c4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564988599980 0x56498897e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.722+0000 7fe6691c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5649885a1b60 0x56498897bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.722+0000 7fe6691c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885c2840 0x56498897c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.722+0000 7fe6691c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1b1640 con 0x5649885a1b60 2024-09-21T00:22:22.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.722+0000 7fe6691c4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1b2ab0 con 0x564988599980 2024-09-21T00:22:22.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.722+0000 7fe6691c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1b1d10 con 0x5649885c2840 2024-09-21T00:22:22.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.722+0000 7fe6691c4740 10 client.0 _ll_register_callbacks cb 0x7fe64c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:22.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.722+0000 7fe65aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5649885a1b60 0x56498897bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.723+0000 7fe65aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5649885a1b60 0x56498897bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50528/0 (socket says 172.21.15.27:50528) 2024-09-21T00:22:22.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.723+0000 7fe65aa00640 1 -- 172.21.15.27:0/2686550040 learned_addr learned my addr 172.21.15.27:0/2686550040 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:22.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.723+0000 7fe65be00640 1 --2- 172.21.15.27:0/2686550040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885c2840 0x56498897c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.723+0000 7fe65be00640 1 --2- 172.21.15.27:0/2686550040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564988599980 0x56498897e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.723+0000 7fe65be00640 1 -- 172.21.15.27:0/2686550040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885c2840 msgr2=0x56498897c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:22.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.723+0000 7fe65be00640 1 --2- 172.21.15.27:0/2686550040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885c2840 0x56498897c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.723+0000 7fe65be00640 1 -- 172.21.15.27:0/2686550040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5649885a1b60 msgr2=0x56498897bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:22.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.723+0000 7fe65be00640 1 --2- 172.21.15.27:0/2686550040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5649885a1b60 0x56498897bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.723+0000 7fe65be00640 1 -- 172.21.15.27:0/2686550040 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56498897ca10 con 0x564988599980 2024-09-21T00:22:22.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.724+0000 7fe65be00640 1 --2- 172.21.15.27:0/2686550040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564988599980 0x56498897e7d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe654019a70 tx=0x7fe65400fef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7446f9e26e64e27d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:22.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.724+0000 7fe645600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:22:22.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.724+0000 7fe645600640 1 -- 172.21.15.27:0/2686550040 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6540053e0 con 0x564988599980 2024-09-21T00:22:22.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.724+0000 7fe645600640 1 -- 172.21.15.27:0/2686550040 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fe654005580 con 0x564988599980 2024-09-21T00:22:22.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.724+0000 7fe6691c4740 1 -- 172.21.15.27:0/2686550040 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6480046c0 con 0x564988599980 2024-09-21T00:22:22.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.725+0000 7fe645600640 1 -- 172.21.15.27:0/2686550040 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe6540116c0 con 0x564988599980 2024-09-21T00:22:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.725+0000 7fe6691c4740 10 client.25592 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.725+0000 7fe6691c4740 1 -- 172.21.15.27:0/2686550040 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5649889fae50 con 0x564988599980 2024-09-21T00:22:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.725+0000 7fe645600640 1 -- 172.21.15.27:0/2686550040 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6480046c0 con 0x564988599980 2024-09-21T00:22:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.725+0000 7fe645600640 1 -- 172.21.15.27:0/2686550040 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe654037070 con 0x564988599980 2024-09-21T00:22:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.726+0000 7fe6691c4740 10 client.25592 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.726+0000 7fe6691c4740 10 client.25592 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:22:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.726+0000 7fe6691c4740 20 client.25592 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.726+0000 7fe6691c4740 1 --2- 172.21.15.27:0/2686550040 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x564988556570 0x5649889fd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:22.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.726+0000 7fe6691c4740 4 client.25592 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}] 2024-09-21T00:22:22.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.726+0000 7fe6691c4740 1 -- 172.21.15.27:0/2686550040 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}) -- 0x5649888dd6a0 con 0x564988556570 2024-09-21T00:22:22.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.726+0000 7fe65aa00640 1 --2- 172.21.15.27:0/2686550040 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x564988556570 0x5649889fd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:22.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.727+0000 7fe65aa00640 1 --2- 172.21.15.27:0/2686550040 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x564988556570 0x5649889fd4b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a5b72d30a51a6 server_cookie=55d7de0ac7072ed5 in_seq=0 out_seq=0 2024-09-21T00:22:22.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.727+0000 7fe645600640 10 client.25592 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:22.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.742+0000 7fe645600640 1 -- 172.21.15.27:0/2686550040 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5649888dd6a0 con 0x564988556570 2024-09-21T00:22:22.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.742+0000 7fe645600640 10 client.25592 handle_command_reply: tid=0 2024-09-21T00:22:22.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.742+0000 7fe6691c4740 1 client.25592 shutdown 2024-09-21T00:22:22.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.742+0000 7fe6691c4740 1 -- 172.21.15.27:0/2686550040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564988599980 msgr2=0x56498897e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:22.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.742+0000 7fe6691c4740 1 --2- 172.21.15.27:0/2686550040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564988599980 0x56498897e7d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe654019a70 tx=0x7fe65400fef0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.742+0000 7fe6691c4740 1 -- 172.21.15.27:0/2686550040 shutdown_connections 2024-09-21T00:22:22.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.742+0000 7fe6691c4740 1 --2- 172.21.15.27:0/2686550040 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x564988556570 0x5649889fd4b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.742+0000 7fe6691c4740 1 --2- 172.21.15.27:0/2686550040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5649885c2840 0x56498897c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.742+0000 7fe6691c4740 1 --2- 172.21.15.27:0/2686550040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5649885a1b60 0x56498897bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.742+0000 7fe6691c4740 1 --2- 172.21.15.27:0/2686550040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564988599980 0x56498897e7d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.742+0000 7fe645600640 0 client.25592 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:22.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.742+0000 7fe6691c4740 1 -- 172.21.15.27:0/2686550040 >> 172.21.15.27:0/2686550040 conn(0x5649885b6990 msgr2=0x564988977770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:22.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.743+0000 7fe6691c4740 1 -- 172.21.15.27:0/2686550040 shutdown_connections 2024-09-21T00:22:22.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.743+0000 7fe6691c4740 1 -- 172.21.15.27:0/2686550040 wait complete. 2024-09-21T00:22:22.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.743+0000 7fe6691c4740 20 client.25592 trim_cache size 0 max 16384 2024-09-21T00:22:22.745 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:22:22.745 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 26, 2024-09-21T00:22:22.745 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24761, 2024-09-21T00:22:22.745 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 2, 2024-09-21T00:22:22.745 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T00:22:22.745 INFO:teuthology.orchestra.run.smithi027.stdout: "e14cd8ec": { 2024-09-21T00:22:22.745 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 2, 2024-09-21T00:22:22.745 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T00:22:22.746 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:22:22.746 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T00:22:22.746 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:22:22.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:22:22.746 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T00:22:22.746 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 134.5, 2024-09-21T00:22:22.746 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T00:22:22.746 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T00:22:22.746 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T00:22:22.746 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:22:22.746 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T00:22:22.746 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:22:22.746 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:22:22.746 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:22:22.746 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:22:22.747 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:22:22.747 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:22:22.747 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:22:22.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.744+0000 7fe660e00640 1 -- 172.21.15.27:0/2660172279 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe6400b6770 msgr2=0x7fe6400b8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:22.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.744+0000 7fe660e00640 1 --2- 172.21.15.27:0/2660172279 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe6400b6770 0x7fe6400b8c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x564988969190 tx=0x7fe64c0adb80 comp rx=0 tx=0).stop 2024-09-21T00:22:22.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.744+0000 7fe660e00640 1 -- 172.21.15.27:0/2660172279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c105bb0 msgr2=0x7fe65c19c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:22.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.744+0000 7fe660e00640 1 --2- 172.21.15.27:0/2660172279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c105bb0 0x7fe65c19c4e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00e9f0 tx=0x7fe64c00eeb0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.744+0000 7fe660e00640 1 -- 172.21.15.27:0/2660172279 shutdown_connections 2024-09-21T00:22:22.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.744+0000 7fe660e00640 1 --2- 172.21.15.27:0/2660172279 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe6400b6770 0x7fe6400b8c80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.744+0000 7fe660e00640 1 --2- 172.21.15.27:0/2660172279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c10be40 0x7fe65c1a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.744+0000 7fe660e00640 1 --2- 172.21.15.27:0/2660172279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe65c10b4f0 0x7fe65c19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.744+0000 7fe660e00640 1 --2- 172.21.15.27:0/2660172279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c105bb0 0x7fe65c19c4e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:22.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.744+0000 7fe660e00640 1 -- 172.21.15.27:0/2660172279 >> 172.21.15.27:0/2660172279 conn(0x7fe65c101640 msgr2=0x7fe65c103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:22.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.745+0000 7fe660e00640 1 -- 172.21.15.27:0/2660172279 shutdown_connections 2024-09-21T00:22:22.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:22.745+0000 7fe660e00640 1 -- 172.21.15.27:0/2660172279 wait complete. 2024-09-21T00:22:23.588 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e14cd8ec', quiesce took 4.5 seconds. Will release after: 10.0 2024-09-21T00:22:23.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:23 smithi089 ceph-mon[34481]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:22:23.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:23 smithi098 ceph-mon[34725]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:22:23.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:23 smithi027 ceph-mon[30745]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:22:25.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:25 smithi089 ceph-mon[34481]: pgmap v408: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T00:22:25.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:25 smithi098 ceph-mon[34725]: pgmap v408: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T00:22:25.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:25 smithi027 ceph-mon[30745]: pgmap v408: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T00:22:27.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:27 smithi089 ceph-mon[34481]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-21T00:22:27.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:27 smithi098 ceph-mon[34725]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-21T00:22:27.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:27 smithi027 ceph-mon[30745]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-21T00:22:29.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:29 smithi089 ceph-mon[34481]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T00:22:29.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:29 smithi098 ceph-mon[34725]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T00:22:29.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:29 smithi027 ceph-mon[30745]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T00:22:31.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:31 smithi098 ceph-mon[34725]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T00:22:31.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:31 smithi027 ceph-mon[30745]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T00:22:31.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:31 smithi089 ceph-mon[34481]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T00:22:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.551+0000 7fa6cf200640 1 -- 172.21.15.27:0/139914954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d010b470 msgr2=0x7fa6d010b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.551+0000 7fa6cf200640 1 --2- 172.21.15.27:0/139914954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d010b470 0x7fa6d010b880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009900 tx=0x7fa6b802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.551+0000 7fa6cf200640 1 -- 172.21.15.27:0/139914954 shutdown_connections 2024-09-21T00:22:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.551+0000 7fa6cf200640 1 --2- 172.21.15.27:0/139914954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6d010bdc0 0x7fa6d01125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.551+0000 7fa6cf200640 1 --2- 172.21.15.27:0/139914954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d010b470 0x7fa6d010b880 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.551+0000 7fa6cf200640 1 --2- 172.21.15.27:0/139914954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6d0105b40 0x7fa6d0105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.551+0000 7fa6cf200640 1 -- 172.21.15.27:0/139914954 >> 172.21.15.27:0/139914954 conn(0x7fa6d0101580 msgr2=0x7fa6d01039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.551+0000 7fa6cf200640 1 -- 172.21.15.27:0/139914954 shutdown_connections 2024-09-21T00:22:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.551+0000 7fa6cf200640 1 -- 172.21.15.27:0/139914954 wait complete. 2024-09-21T00:22:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.553+0000 7fa6cf200640 1 Processor -- start 2024-09-21T00:22:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.553+0000 7fa6cf200640 1 -- start start 2024-09-21T00:22:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.554+0000 7fa6cf200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d0105b40 0x7fa6d019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.554+0000 7fa6cf200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6d010b470 0x7fa6d019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.554+0000 7fa6cf200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6d010bdc0 0x7fa6d01a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.554+0000 7fa6cf200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d0115ad0 con 0x7fa6d010b470 2024-09-21T00:22:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.554+0000 7fa6cf200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d0115950 con 0x7fa6d010bdc0 2024-09-21T00:22:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.554+0000 7fa6cf200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d0115c50 con 0x7fa6d0105b40 2024-09-21T00:22:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.554+0000 7fa6cde00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d0105b40 0x7fa6d019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.554+0000 7fa6cde00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d0105b40 0x7fa6d019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:50514/0 (socket says 172.21.15.27:50514) 2024-09-21T00:22:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.554+0000 7fa6c7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6d010b470 0x7fa6d019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.554+0000 7fa6cde00640 1 -- 172.21.15.27:0/255650723 learned_addr learned my addr 172.21.15.27:0/255650723 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.554+0000 7fa6ce800640 1 --2- 172.21.15.27:0/255650723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6d010bdc0 0x7fa6d01a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.555+0000 7fa6cde00640 1 -- 172.21.15.27:0/255650723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6d010bdc0 msgr2=0x7fa6d01a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.555+0000 7fa6cde00640 1 --2- 172.21.15.27:0/255650723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6d010bdc0 0x7fa6d01a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.555+0000 7fa6cde00640 1 -- 172.21.15.27:0/255650723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6d010b470 msgr2=0x7fa6d019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.555+0000 7fa6cde00640 1 --2- 172.21.15.27:0/255650723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6d010b470 0x7fa6d019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.555+0000 7fa6cde00640 1 -- 172.21.15.27:0/255650723 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d01a2420 con 0x7fa6d0105b40 2024-09-21T00:22:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.555+0000 7fa6cde00640 1 --2- 172.21.15.27:0/255650723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d0105b40 0x7fa6d019c4f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000e9f0 tx=0x7fa6c000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.555+0000 7fa6c6a00640 1 -- 172.21.15.27:0/255650723 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c000cd80 con 0x7fa6d0105b40 2024-09-21T00:22:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.555+0000 7fa6cf200640 1 -- 172.21.15.27:0/255650723 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d01a2710 con 0x7fa6d0105b40 2024-09-21T00:22:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.555+0000 7fa6c6a00640 1 -- 172.21.15.27:0/255650723 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c00054d0 con 0x7fa6d0105b40 2024-09-21T00:22:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.556+0000 7fa6c6a00640 1 -- 172.21.15.27:0/255650723 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0010690 con 0x7fa6d0105b40 2024-09-21T00:22:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.556+0000 7fa6cf200640 1 -- 172.21.15.27:0/255650723 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d01a8ea0 con 0x7fa6d0105b40 2024-09-21T00:22:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.559+0000 7fa6c6a00640 1 -- 172.21.15.27:0/255650723 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa6c0004e90 con 0x7fa6d0105b40 2024-09-21T00:22:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.560+0000 7fa6dc21c740 1 Processor -- start 2024-09-21T00:22:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.560+0000 7fa6dc21c740 1 -- start start 2024-09-21T00:22:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.561+0000 7fa6dc21c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2ed1b0 0x55700b691e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.561+0000 7fa6dc21c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2c42f0 0x55700b692350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.561+0000 7fa6dc21c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2cc4d0 0x55700b692890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.561+0000 7fa6dc21c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55700b6076b0 con 0x55700b2ed1b0 2024-09-21T00:22:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.561+0000 7fa6dc21c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55700b5f6ae0 con 0x55700b2cc4d0 2024-09-21T00:22:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.561+0000 7fa6dc21c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55700b5fa8f0 con 0x55700b2c42f0 2024-09-21T00:22:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.561+0000 7fa6ce800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2ed1b0 0x55700b691e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.561+0000 7fa6ce800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2ed1b0 0x55700b691e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59808/0 (socket says 172.21.15.27:59808) 2024-09-21T00:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.561+0000 7fa6ce800640 1 -- 172.21.15.27:0/2987587777 learned_addr learned my addr 172.21.15.27:0/2987587777 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.561+0000 7fa6c7e00640 1 --2- 172.21.15.27:0/2987587777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2cc4d0 0x55700b692890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.562+0000 7fa6cde00640 1 --2- 172.21.15.27:0/2987587777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2c42f0 0x55700b692350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.562+0000 7fa6ce800640 1 -- 172.21.15.27:0/2987587777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2c42f0 msgr2=0x55700b692350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.562+0000 7fa6ce800640 1 --2- 172.21.15.27:0/2987587777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2c42f0 0x55700b692350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.562+0000 7fa6ce800640 1 -- 172.21.15.27:0/2987587777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2cc4d0 msgr2=0x55700b692890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.562+0000 7fa6ce800640 1 --2- 172.21.15.27:0/2987587777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2cc4d0 0x55700b692890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.562+0000 7fa6ce800640 1 -- 172.21.15.27:0/2987587777 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55700b5c3780 con 0x55700b2ed1b0 2024-09-21T00:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.562+0000 7fa6c6a00640 1 --2- 172.21.15.27:0/255650723 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa6b40b66f0 0x7fa6b40b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.562+0000 7fa6cde00640 1 --2- 172.21.15.27:0/255650723 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa6b40b66f0 0x7fa6b40b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.562+0000 7fa6c6a00640 1 -- 172.21.15.27:0/255650723 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fa6c00e7f30 con 0x7fa6d0105b40 2024-09-21T00:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.563+0000 7fa6ce800640 1 --2- 172.21.15.27:0/2987587777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2ed1b0 0x55700b691e10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009fc0 tx=0x7fa6c8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ab9af72be7e6a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.563+0000 7fa6bf400640 1 -- 172.21.15.27:0/2987587777 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8003b70 con 0x55700b2ed1b0 2024-09-21T00:22:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.563+0000 7fa6bf400640 1 -- 172.21.15.27:0/2987587777 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c8003d10 con 0x55700b2ed1b0 2024-09-21T00:22:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.563+0000 7fa6bf400640 1 -- 172.21.15.27:0/2987587777 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8011a40 con 0x55700b2ed1b0 2024-09-21T00:22:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.564+0000 7fa6cde00640 1 --2- 172.21.15.27:0/255650723 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa6b40b66f0 0x7fa6b40b8c00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55700b690a90 tx=0x7fa6c0004a90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.564+0000 7fa6dc21c740 1 -- 172.21.15.27:0/2987587777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2ed1b0 msgr2=0x55700b691e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.564+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/2987587777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2ed1b0 0x55700b691e10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009fc0 tx=0x7fa6c8010860 comp rx=0 tx=0).stop 2024-09-21T00:22:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.565+0000 7fa6dc21c740 1 -- 172.21.15.27:0/2987587777 shutdown_connections 2024-09-21T00:22:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.565+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/2987587777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2cc4d0 0x55700b692890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.565+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/2987587777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2c42f0 0x55700b692350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.565+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/2987587777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2ed1b0 0x55700b691e10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.565+0000 7fa6dc21c740 1 -- 172.21.15.27:0/2987587777 >> 172.21.15.27:0/2987587777 conn(0x55700b2e1300 msgr2=0x55700b68fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.565+0000 7fa6dc21c740 1 -- 172.21.15.27:0/2987587777 shutdown_connections 2024-09-21T00:22:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.565+0000 7fa6dc21c740 1 -- 172.21.15.27:0/2987587777 wait complete. 2024-09-21T00:22:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.566+0000 7fa6dc21c740 1 -- start start 2024-09-21T00:22:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.566+0000 7fa6dc21c740 1 Processor -- start 2024-09-21T00:22:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.567+0000 7fa6dc21c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2c42f0 0x55700b69d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.567+0000 7fa6dc21c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2cc4d0 0x55700b69d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.567+0000 7fa6dc21c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2ed1b0 0x55700b6a9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.567+0000 7fa6dc21c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55700b52eab0 con 0x55700b2ed1b0 2024-09-21T00:22:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.567+0000 7fa6dc21c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55700b59d570 con 0x55700b2c42f0 2024-09-21T00:22:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.567+0000 7fa6dc21c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55700b6a0590 con 0x55700b2cc4d0 2024-09-21T00:22:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.567+0000 7fa6dc21c740 10 client.0 _ll_register_callbacks cb 0x55700b5ac100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.567+0000 7fa6ce800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2c42f0 0x55700b69d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.567+0000 7fa6c7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2cc4d0 0x55700b69d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.567+0000 7fa6c7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2cc4d0 0x55700b69d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:50526/0 (socket says 172.21.15.27:50526) 2024-09-21T00:22:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.567+0000 7fa6c7e00640 1 -- 172.21.15.27:0/3587125598 learned_addr learned my addr 172.21.15.27:0/3587125598 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.567+0000 7fa6ce800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2c42f0 0x55700b69d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:59684/0 (socket says 172.21.15.27:59684) 2024-09-21T00:22:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.568+0000 7fa6ce800640 1 --2- 172.21.15.27:0/3587125598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2ed1b0 0x55700b6a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.568+0000 7fa6c7e00640 1 -- 172.21.15.27:0/3587125598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2c42f0 msgr2=0x55700b69d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.568+0000 7fa6c7e00640 1 --2- 172.21.15.27:0/3587125598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2c42f0 0x55700b69d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.568+0000 7fa6c7e00640 1 -- 172.21.15.27:0/3587125598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2ed1b0 msgr2=0x55700b6a9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.568+0000 7fa6c7e00640 1 --2- 172.21.15.27:0/3587125598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2ed1b0 0x55700b6a9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.568+0000 7fa6c7e00640 1 -- 172.21.15.27:0/3587125598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55700b358790 con 0x55700b2cc4d0 2024-09-21T00:22:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.568+0000 7fa6ce800640 1 --2- 172.21.15.27:0/3587125598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2ed1b0 0x55700b6a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:22:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.568+0000 7fa6c7e00640 1 --2- 172.21.15.27:0/3587125598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2cc4d0 0x55700b69d9a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005ab0 tx=0x7fa6b8004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49f53676dbc1f5dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.568+0000 7fa6b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:22:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.569+0000 7fa6b3e00640 1 -- 172.21.15.27:0/3587125598 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8035040 con 0x55700b2cc4d0 2024-09-21T00:22:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.569+0000 7fa6b3e00640 1 -- 172.21.15.27:0/3587125598 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fa6b8005510 con 0x55700b2cc4d0 2024-09-21T00:22:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.569+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3587125598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6c8003d10 con 0x55700b2cc4d0 2024-09-21T00:22:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.569+0000 7fa6b3e00640 1 -- 172.21.15.27:0/3587125598 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa6b804f050 con 0x55700b2cc4d0 2024-09-21T00:22:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.569+0000 7fa6dc21c740 10 client.25579 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.569+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3587125598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55700b3d1620 con 0x55700b2cc4d0 2024-09-21T00:22:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.570+0000 7fa6b3e00640 1 -- 172.21.15.27:0/3587125598 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c8003d10 con 0x55700b2cc4d0 2024-09-21T00:22:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.571+0000 7fa6b3e00640 1 -- 172.21.15.27:0/3587125598 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa6b8004750 con 0x55700b2cc4d0 2024-09-21T00:22:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.571+0000 7fa6dc21c740 10 client.25579 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.571+0000 7fa6dc21c740 10 client.25579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:22:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.571+0000 7fa6dc21c740 20 client.25579 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.571+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/3587125598 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55700b2abd80 0x55700b6af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.571+0000 7fa6dc21c740 4 client.25579 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:22:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.571+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3587125598 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55700b609600 con 0x55700b2abd80 2024-09-21T00:22:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.572+0000 7fa6ce800640 1 --2- 172.21.15.27:0/3587125598 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55700b2abd80 0x55700b6af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.572+0000 7fa6b3e00640 10 client.25579 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:33.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.572+0000 7fa6ce800640 1 --2- 172.21.15.27:0/3587125598 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55700b2abd80 0x55700b6af590 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e246a70b8feae610 server_cookie=e9540a12f37663f7 in_seq=0 out_seq=0 2024-09-21T00:22:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.573+0000 7fa6b3e00640 1 -- 172.21.15.27:0/3587125598 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55700b609600 con 0x55700b2abd80 2024-09-21T00:22:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.573+0000 7fa6b3e00640 10 client.25579 handle_command_reply: tid=0 2024-09-21T00:22:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.574+0000 7fa6dc21c740 1 client.25579 shutdown 2024-09-21T00:22:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.574+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3587125598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2cc4d0 msgr2=0x55700b69d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.574+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/3587125598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2cc4d0 0x55700b69d9a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005ab0 tx=0x7fa6b8004440 comp rx=0 tx=0).stop 2024-09-21T00:22:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.574+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3587125598 shutdown_connections 2024-09-21T00:22:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.574+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/3587125598 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55700b2abd80 0x55700b6af590 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.574+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/3587125598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2ed1b0 0x55700b6a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.574+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/3587125598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2cc4d0 0x55700b69d9a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.574+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/3587125598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2c42f0 0x55700b69d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.574+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3587125598 >> 172.21.15.27:0/3587125598 conn(0x55700b2e1300 msgr2=0x55700b692e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.574+0000 7fa6b3e00640 0 client.25579 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.574+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3587125598 shutdown_connections 2024-09-21T00:22:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.574+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3587125598 wait complete. 2024-09-21T00:22:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.575+0000 7fa6dc21c740 20 client.25579 trim_cache size 0 max 16384 2024-09-21T00:22:33.590 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:22:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.605+0000 7fa6dc21c740 1 Processor -- start 2024-09-21T00:22:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.606+0000 7fa6dc21c740 1 -- start start 2024-09-21T00:22:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.606+0000 7fa6dc21c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2c42f0 0x55700b693800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.606+0000 7fa6dc21c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2cc4d0 0x55700b69f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.606+0000 7fa6dc21c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2ed1b0 0x55700b69a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.606+0000 7fa6dc21c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d01b56d0 con 0x55700b2c42f0 2024-09-21T00:22:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.606+0000 7fa6dc21c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d01b2fa0 con 0x55700b2cc4d0 2024-09-21T00:22:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.606+0000 7fa6dc21c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d01b0df0 con 0x55700b2ed1b0 2024-09-21T00:22:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.607+0000 7fa6c7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2cc4d0 0x55700b69f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.607+0000 7fa6ce800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2c42f0 0x55700b693800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.607+0000 7fa6c7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2cc4d0 0x55700b69f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:59700/0 (socket says 172.21.15.27:59700) 2024-09-21T00:22:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.607+0000 7fa6c7e00640 1 -- 172.21.15.27:0/2450614033 learned_addr learned my addr 172.21.15.27:0/2450614033 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.607+0000 7fa6ce800640 1 --2- 172.21.15.27:0/2450614033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2ed1b0 0x55700b69a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.607+0000 7fa6c7e00640 1 -- 172.21.15.27:0/2450614033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2ed1b0 msgr2=0x55700b69a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.607+0000 7fa6c7e00640 1 --2- 172.21.15.27:0/2450614033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2ed1b0 0x55700b69a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.607+0000 7fa6c7e00640 1 -- 172.21.15.27:0/2450614033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2c42f0 msgr2=0x55700b693800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.607+0000 7fa6c7e00640 1 --2- 172.21.15.27:0/2450614033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2c42f0 0x55700b693800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.607+0000 7fa6c7e00640 1 -- 172.21.15.27:0/2450614033 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55700b692c40 con 0x55700b2cc4d0 2024-09-21T00:22:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.607+0000 7fa6ce800640 1 --2- 172.21.15.27:0/2450614033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2ed1b0 0x55700b69a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:22:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.607+0000 7fa6c7e00640 1 --2- 172.21.15.27:0/2450614033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2cc4d0 0x55700b69f090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005d70 tx=0x7fa6b8005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6afd3256fdf4c823 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.608+0000 7fa6bea00640 1 -- 172.21.15.27:0/2450614033 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8035030 con 0x55700b2cc4d0 2024-09-21T00:22:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.608+0000 7fa6bea00640 1 -- 172.21.15.27:0/2450614033 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b8041070 con 0x55700b2cc4d0 2024-09-21T00:22:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.608+0000 7fa6bea00640 1 -- 172.21.15.27:0/2450614033 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8003b20 con 0x55700b2cc4d0 2024-09-21T00:22:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.609+0000 7fa6dc21c740 1 -- 172.21.15.27:0/2450614033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2cc4d0 msgr2=0x55700b69f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.609+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/2450614033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2cc4d0 0x55700b69f090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005d70 tx=0x7fa6b8005f00 comp rx=0 tx=0).stop 2024-09-21T00:22:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.609+0000 7fa6dc21c740 1 -- 172.21.15.27:0/2450614033 shutdown_connections 2024-09-21T00:22:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.609+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/2450614033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2ed1b0 0x55700b69a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.609+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/2450614033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2cc4d0 0x55700b69f090 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.609+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/2450614033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2c42f0 0x55700b693800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.609+0000 7fa6dc21c740 1 -- 172.21.15.27:0/2450614033 >> 172.21.15.27:0/2450614033 conn(0x55700b2e1300 msgr2=0x55700b69e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.609+0000 7fa6dc21c740 1 -- 172.21.15.27:0/2450614033 shutdown_connections 2024-09-21T00:22:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.610+0000 7fa6dc21c740 1 -- 172.21.15.27:0/2450614033 wait complete. 2024-09-21T00:22:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.610+0000 7fa6dc21c740 1 -- start start 2024-09-21T00:22:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.610+0000 7fa6dc21c740 1 Processor -- start 2024-09-21T00:22:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.611+0000 7fa6dc21c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2c42f0 0x55700b6a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.611+0000 7fa6dc21c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2cc4d0 0x55700b6a3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.611+0000 7fa6dc21c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2ed1b0 0x55700b717f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.611+0000 7fa6dc21c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d01b1460 con 0x55700b2cc4d0 2024-09-21T00:22:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.611+0000 7fa6dc21c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d01b28d0 con 0x55700b2c42f0 2024-09-21T00:22:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.611+0000 7fa6dc21c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d01b1b30 con 0x55700b2ed1b0 2024-09-21T00:22:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.611+0000 7fa6dc21c740 10 client.0 _ll_register_callbacks cb 0x7fa6b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.611+0000 7fa6ce800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2c42f0 0x55700b6a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.611+0000 7fa6ce800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2c42f0 0x55700b6a2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:59710/0 (socket says 172.21.15.27:59710) 2024-09-21T00:22:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.611+0000 7fa6ce800640 1 -- 172.21.15.27:0/3426382834 learned_addr learned my addr 172.21.15.27:0/3426382834 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.611+0000 7fa6ce800640 1 --2- 172.21.15.27:0/3426382834 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2ed1b0 0x55700b717f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.611+0000 7fa6c7e00640 1 --2- 172.21.15.27:0/3426382834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2cc4d0 0x55700b6a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.612+0000 7fa6ce800640 1 -- 172.21.15.27:0/3426382834 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2ed1b0 msgr2=0x55700b717f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.612+0000 7fa6ce800640 1 --2- 172.21.15.27:0/3426382834 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2ed1b0 0x55700b717f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.612+0000 7fa6ce800640 1 -- 172.21.15.27:0/3426382834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2cc4d0 msgr2=0x55700b6a3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.612+0000 7fa6ce800640 1 --2- 172.21.15.27:0/3426382834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2cc4d0 0x55700b6a3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.612+0000 7fa6ce800640 1 -- 172.21.15.27:0/3426382834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55700b718510 con 0x55700b2c42f0 2024-09-21T00:22:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.612+0000 7fa6c7e00640 1 --2- 172.21.15.27:0/3426382834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2cc4d0 0x55700b6a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:22:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.612+0000 7fa6ce800640 1 --2- 172.21.15.27:0/3426382834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2c42f0 0x55700b6a2fc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8011670 tx=0x7fa6c8010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa49ad848d1ac8ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.612+0000 7fa6b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:22:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.612+0000 7fa6b3e00640 1 -- 172.21.15.27:0/3426382834 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8012730 con 0x55700b2c42f0 2024-09-21T00:22:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.613+0000 7fa6b3e00640 1 -- 172.21.15.27:0/3426382834 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fa6c8023040 con 0x55700b2c42f0 2024-09-21T00:22:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.613+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3426382834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6b8041070 con 0x55700b2c42f0 2024-09-21T00:22:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.613+0000 7fa6b3e00640 1 -- 172.21.15.27:0/3426382834 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa6c80128d0 con 0x55700b2c42f0 2024-09-21T00:22:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.613+0000 7fa6dc21c740 10 client.25610 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.613+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3426382834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55700b7197a0 con 0x55700b2c42f0 2024-09-21T00:22:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.613+0000 7fa6b3e00640 1 -- 172.21.15.27:0/3426382834 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b8041070 con 0x55700b2c42f0 2024-09-21T00:22:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.613+0000 7fa6b3e00640 1 -- 172.21.15.27:0/3426382834 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa6c802db40 con 0x55700b2c42f0 2024-09-21T00:22:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.614+0000 7fa6dc21c740 10 client.25610 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.614+0000 7fa6dc21c740 10 client.25610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:22:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.614+0000 7fa6dc21c740 20 client.25610 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.614+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/3426382834 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55700b2811c0 0x55700b71bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.614+0000 7fa6dc21c740 4 client.25610 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:22:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.614+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3426382834 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55700b6050b0 con 0x55700b2811c0 2024-09-21T00:22:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.614+0000 7fa6c7e00640 1 --2- 172.21.15.27:0/3426382834 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55700b2811c0 0x55700b71bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.614+0000 7fa6c7e00640 1 --2- 172.21.15.27:0/3426382834 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55700b2811c0 0x55700b71bdf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d45bc15c569170b server_cookie=16d65d924bd95ced in_seq=0 out_seq=0 2024-09-21T00:22:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.614+0000 7fa6b3e00640 10 client.25610 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.622+0000 7fa6b3e00640 1 -- 172.21.15.27:0/3426382834 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55700b6050b0 con 0x55700b2811c0 2024-09-21T00:22:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.622+0000 7fa6b3e00640 10 client.25610 handle_command_reply: tid=0 2024-09-21T00:22:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.622+0000 7fa6dc21c740 1 client.25610 shutdown 2024-09-21T00:22:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.622+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3426382834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2c42f0 msgr2=0x55700b6a2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.622+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/3426382834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2c42f0 0x55700b6a2fc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8011670 tx=0x7fa6c8010860 comp rx=0 tx=0).stop 2024-09-21T00:22:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.623+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3426382834 shutdown_connections 2024-09-21T00:22:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.623+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/3426382834 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55700b2811c0 0x55700b71bdf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.623+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/3426382834 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55700b2ed1b0 0x55700b717f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.623+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/3426382834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55700b2cc4d0 0x55700b6a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.623+0000 7fa6dc21c740 1 --2- 172.21.15.27:0/3426382834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55700b2c42f0 0x55700b6a2fc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.623+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3426382834 >> 172.21.15.27:0/3426382834 conn(0x55700b2e1300 msgr2=0x55700b6935d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.623+0000 7fa6b3e00640 0 client.25610 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.623+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3426382834 shutdown_connections 2024-09-21T00:22:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.623+0000 7fa6dc21c740 1 -- 172.21.15.27:0/3426382834 wait complete. 2024-09-21T00:22:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.623+0000 7fa6dc21c740 20 client.25610 trim_cache size 0 max 16384 2024-09-21T00:22:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:22:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:22:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:22:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:22:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.624+0000 7fa6cf200640 1 -- 172.21.15.27:0/255650723 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa6b40b66f0 msgr2=0x7fa6b40b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.624+0000 7fa6cf200640 1 --2- 172.21.15.27:0/255650723 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa6b40b66f0 0x7fa6b40b8c00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55700b690a90 tx=0x7fa6c0004a90 comp rx=0 tx=0).stop 2024-09-21T00:22:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.624+0000 7fa6cf200640 1 -- 172.21.15.27:0/255650723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d0105b40 msgr2=0x7fa6d019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.624+0000 7fa6cf200640 1 --2- 172.21.15.27:0/255650723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d0105b40 0x7fa6d019c4f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000e9f0 tx=0x7fa6c000edf0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.624+0000 7fa6cf200640 1 -- 172.21.15.27:0/255650723 shutdown_connections 2024-09-21T00:22:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.624+0000 7fa6cf200640 1 --2- 172.21.15.27:0/255650723 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa6b40b66f0 0x7fa6b40b8c00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.624+0000 7fa6cf200640 1 --2- 172.21.15.27:0/255650723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6d010bdc0 0x7fa6d01a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.624+0000 7fa6cf200640 1 --2- 172.21.15.27:0/255650723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6d010b470 0x7fa6d019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.624+0000 7fa6cf200640 1 --2- 172.21.15.27:0/255650723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d0105b40 0x7fa6d019c4f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.624+0000 7fa6cf200640 1 -- 172.21.15.27:0/255650723 >> 172.21.15.27:0/255650723 conn(0x7fa6d0101580 msgr2=0x7fa6d01039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.624+0000 7fa6cf200640 1 -- 172.21.15.27:0/255650723 shutdown_connections 2024-09-21T00:22:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:33.624+0000 7fa6cf200640 1 -- 172.21.15.27:0/255650723 wait complete. 2024-09-21T00:22:33.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:33 smithi098 ceph-mon[34725]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T00:22:33.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:33 smithi027 ceph-mon[30745]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T00:22:33.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:33 smithi089 ceph-mon[34481]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T00:22:34.069 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:22:34.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.930+0000 7f4f17200640 1 -- 172.21.15.27:0/298353318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f1810be50 msgr2=0x7f4f18112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:34.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.930+0000 7f4f17200640 1 --2- 172.21.15.27:0/298353318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f1810be50 0x7f4f18112680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009ca0 tx=0x7f4f1002a8c0 comp rx=0 tx=0).stop 2024-09-21T00:22:34.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.931+0000 7f4f17200640 1 -- 172.21.15.27:0/298353318 shutdown_connections 2024-09-21T00:22:34.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.931+0000 7f4f17200640 1 --2- 172.21.15.27:0/298353318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f1810be50 0x7f4f18112680 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:34.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.931+0000 7f4f17200640 1 --2- 172.21.15.27:0/298353318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f1810b500 0x7f4f1810b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:34.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.931+0000 7f4f17200640 1 --2- 172.21.15.27:0/298353318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f18105bc0 0x7f4f18105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:34.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.931+0000 7f4f17200640 1 -- 172.21.15.27:0/298353318 >> 172.21.15.27:0/298353318 conn(0x7f4f18101630 msgr2=0x7f4f18103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:34.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.931+0000 7f4f17200640 1 -- 172.21.15.27:0/298353318 shutdown_connections 2024-09-21T00:22:34.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.931+0000 7f4f17200640 1 -- 172.21.15.27:0/298353318 wait complete. 2024-09-21T00:22:34.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.932+0000 7f4f17200640 1 Processor -- start 2024-09-21T00:22:34.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.932+0000 7f4f17200640 1 -- start start 2024-09-21T00:22:34.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.933+0000 7f4f17200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f18105bc0 0x7f4f1819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:34.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.933+0000 7f4f17200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f1810b500 0x7f4f1819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:34.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.933+0000 7f4f17200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f1810be50 0x7f4f181a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:34.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.933+0000 7f4f17200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18115b60 con 0x7f4f18105bc0 2024-09-21T00:22:34.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.933+0000 7f4f17200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181159e0 con 0x7f4f1810be50 2024-09-21T00:22:34.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.933+0000 7f4f17200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18115ce0 con 0x7f4f1810b500 2024-09-21T00:22:34.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.933+0000 7f4f15e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f18105bc0 0x7f4f1819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:34.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.933+0000 7f4f15e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f18105bc0 0x7f4f1819c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59852/0 (socket says 172.21.15.27:59852) 2024-09-21T00:22:34.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.933+0000 7f4f15e00640 1 -- 172.21.15.27:0/2999761273 learned_addr learned my addr 172.21.15.27:0/2999761273 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:34.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.933+0000 7f4f15400640 1 --2- 172.21.15.27:0/2999761273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f1810b500 0x7f4f1819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:34.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.933+0000 7f4f16800640 1 --2- 172.21.15.27:0/2999761273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f1810be50 0x7f4f181a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:34.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.934+0000 7f4f15e00640 1 -- 172.21.15.27:0/2999761273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f1810b500 msgr2=0x7f4f1819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:34.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.934+0000 7f4f15e00640 1 --2- 172.21.15.27:0/2999761273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f1810b500 0x7f4f1819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:34.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.934+0000 7f4f15e00640 1 -- 172.21.15.27:0/2999761273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f1810be50 msgr2=0x7f4f181a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:34.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.934+0000 7f4f15e00640 1 --2- 172.21.15.27:0/2999761273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f1810be50 0x7f4f181a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:34.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.934+0000 7f4f15e00640 1 -- 172.21.15.27:0/2999761273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f181a2400 con 0x7f4f18105bc0 2024-09-21T00:22:34.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.935+0000 7f4f15e00640 1 --2- 172.21.15.27:0/2999761273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f18105bc0 0x7f4f1819c490 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800ea60 tx=0x7f4f0800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:34.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.935+0000 7f4f0ea00640 1 -- 172.21.15.27:0/2999761273 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0800cde0 con 0x7f4f18105bc0 2024-09-21T00:22:34.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.935+0000 7f4f0ea00640 1 -- 172.21.15.27:0/2999761273 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f080035f0 con 0x7f4f18105bc0 2024-09-21T00:22:34.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.935+0000 7f4f17200640 1 -- 172.21.15.27:0/2999761273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f181a26f0 con 0x7f4f18105bc0 2024-09-21T00:22:34.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.935+0000 7f4f0ea00640 1 -- 172.21.15.27:0/2999761273 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08010690 con 0x7f4f18105bc0 2024-09-21T00:22:34.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.936+0000 7f4f17200640 1 -- 172.21.15.27:0/2999761273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f18077800 con 0x7f4f18105bc0 2024-09-21T00:22:34.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.937+0000 7f4f17200640 1 -- 172.21.15.27:0/2999761273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f18105fd0 con 0x7f4f18105bc0 2024-09-21T00:22:34.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.939+0000 7f4f0ea00640 1 -- 172.21.15.27:0/2999761273 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4f080037a0 con 0x7f4f18105bc0 2024-09-21T00:22:34.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.942+0000 7f4f0ea00640 1 --2- 172.21.15.27:0/2999761273 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4efc0b6700 0x7f4efc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:34.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.942+0000 7f4f0ea00640 1 -- 172.21.15.27:0/2999761273 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f4f080e7d60 con 0x7f4f18105bc0 2024-09-21T00:22:34.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.943+0000 7f4f15400640 1 --2- 172.21.15.27:0/2999761273 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4efc0b6700 0x7f4efc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:34.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.943+0000 7f4f15400640 1 --2- 172.21.15.27:0/2999761273 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4efc0b6700 0x7f4efc0b8c10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4f18104d30 tx=0x7f4f04005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:34.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:34.946+0000 7f4f0ea00640 1 -- 172.21.15.27:0/2999761273 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4f080adf20 con 0x7f4f18105bc0 2024-09-21T00:22:35.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:35.249+0000 7f4f17200640 1 -- 172.21.15.27:0/2999761273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f1819d5c0 con 0x7f4f18105bc0 2024-09-21T00:22:35.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:35.250+0000 7f4f0ea00640 1 -- 172.21.15.27:0/2999761273 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11893 (secure 0 0 0) 0x7f4f080b2300 con 0x7f4f18105bc0 2024-09-21T00:22:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:22:35.252 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":26,"btime":"2024-09-21T00:21:36:085073+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:21:35.264444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:22:35.252 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-09-21T00:22:35.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:35.253+0000 7f4f17200640 1 -- 172.21.15.27:0/2999761273 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4efc0b6700 msgr2=0x7f4efc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:35.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:35.253+0000 7f4f17200640 1 --2- 172.21.15.27:0/2999761273 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4efc0b6700 0x7f4efc0b8c10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4f18104d30 tx=0x7f4f04005f90 comp rx=0 tx=0).stop 2024-09-21T00:22:35.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:35.253+0000 7f4f17200640 1 -- 172.21.15.27:0/2999761273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f18105bc0 msgr2=0x7f4f1819c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:35.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:35.253+0000 7f4f17200640 1 --2- 172.21.15.27:0/2999761273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f18105bc0 0x7f4f1819c490 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800ea60 tx=0x7f4f0800ef20 comp rx=0 tx=0).stop 2024-09-21T00:22:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:35.253+0000 7f4f17200640 1 -- 172.21.15.27:0/2999761273 shutdown_connections 2024-09-21T00:22:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:35.253+0000 7f4f17200640 1 --2- 172.21.15.27:0/2999761273 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4efc0b6700 0x7f4efc0b8c10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:35.253+0000 7f4f17200640 1 --2- 172.21.15.27:0/2999761273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f1810be50 0x7f4f181a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:35.253+0000 7f4f17200640 1 --2- 172.21.15.27:0/2999761273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f1810b500 0x7f4f1819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:35.253+0000 7f4f17200640 1 --2- 172.21.15.27:0/2999761273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f18105bc0 0x7f4f1819c490 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:35.253+0000 7f4f17200640 1 -- 172.21.15.27:0/2999761273 >> 172.21.15.27:0/2999761273 conn(0x7f4f18101630 msgr2=0x7f4f1810ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:35.254+0000 7f4f17200640 1 -- 172.21.15.27:0/2999761273 shutdown_connections 2024-09-21T00:22:35.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:35.254+0000 7f4f17200640 1 -- 172.21.15.27:0/2999761273 wait complete. 2024-09-21T00:22:35.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:35 smithi027 ceph-mon[30745]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-21T00:22:35.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:35 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2999761273' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:22:35.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:35 smithi098 ceph-mon[34725]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-21T00:22:35.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:35 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2999761273' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:22:35.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:35 smithi089 ceph-mon[34481]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-21T00:22:35.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:35 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2999761273' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:22:36.121 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id e14cd8ec --release --await' 2024-09-21T00:22:36.121 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id e14cd8ec --release --await 2024-09-21T00:22:36.745 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:22:37.492 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:37 smithi027 ceph-mon[30745]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-21T00:22:37.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.528+0000 7fecb0a00640 1 -- 172.21.15.27:0/1593923090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecac10c020 msgr2=0x7fecac112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:37.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.528+0000 7fecb0a00640 1 --2- 172.21.15.27:0/1593923090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecac10c020 0x7fecac112850 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feca4009ca0 tx=0x7feca402a8b0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.529+0000 7fecb0a00640 1 -- 172.21.15.27:0/1593923090 shutdown_connections 2024-09-21T00:22:37.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.529+0000 7fecb0a00640 1 --2- 172.21.15.27:0/1593923090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecac10c020 0x7fecac112850 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.529+0000 7fecb0a00640 1 --2- 172.21.15.27:0/1593923090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecac10b6d0 0x7fecac10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.529+0000 7fecb0a00640 1 --2- 172.21.15.27:0/1593923090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecac105d90 0x7fecac1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.529+0000 7fecb0a00640 1 -- 172.21.15.27:0/1593923090 >> 172.21.15.27:0/1593923090 conn(0x7fecac101860 msgr2=0x7fecac103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:37.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.529+0000 7fecb0a00640 1 -- 172.21.15.27:0/1593923090 shutdown_connections 2024-09-21T00:22:37.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.529+0000 7fecb0a00640 1 -- 172.21.15.27:0/1593923090 wait complete. 2024-09-21T00:22:37.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.530+0000 7fecb0a00640 1 Processor -- start 2024-09-21T00:22:37.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.531+0000 7fecb0a00640 1 -- start start 2024-09-21T00:22:37.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.532+0000 7fecb0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecac105d90 0x7fecac19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.532+0000 7fecb0a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecac10b6d0 0x7fecac19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.532+0000 7fecb0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecac10c020 0x7fecac1a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.532+0000 7fecb0a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fecac115d30 con 0x7fecac105d90 2024-09-21T00:22:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.532+0000 7fecb0a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fecac115bb0 con 0x7fecac10c020 2024-09-21T00:22:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.532+0000 7fecb0a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fecac115eb0 con 0x7fecac10b6d0 2024-09-21T00:22:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.533+0000 7feca9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecac105d90 0x7fecac19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.533+0000 7feca9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecac105d90 0x7fecac19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59872/0 (socket says 172.21.15.27:59872) 2024-09-21T00:22:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.533+0000 7feca9e00640 1 -- 172.21.15.27:0/2995908124 learned_addr learned my addr 172.21.15.27:0/2995908124 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.533+0000 7fecaa800640 1 --2- 172.21.15.27:0/2995908124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecac10c020 0x7fecac1a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.533+0000 7feca9400640 1 --2- 172.21.15.27:0/2995908124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecac10b6d0 0x7fecac19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.533+0000 7feca9e00640 1 -- 172.21.15.27:0/2995908124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecac10b6d0 msgr2=0x7fecac19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:37.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.533+0000 7feca9e00640 1 --2- 172.21.15.27:0/2995908124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecac10b6d0 0x7fecac19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.533+0000 7feca9e00640 1 -- 172.21.15.27:0/2995908124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecac10c020 msgr2=0x7fecac1a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:37.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.533+0000 7feca9e00640 1 --2- 172.21.15.27:0/2995908124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecac10c020 0x7fecac1a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.534+0000 7feca9e00640 1 -- 172.21.15.27:0/2995908124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecac1a25d0 con 0x7fecac105d90 2024-09-21T00:22:37.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.534+0000 7feca9e00640 1 --2- 172.21.15.27:0/2995908124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecac105d90 0x7fecac19c660 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00e9f0 tx=0x7fec9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.534+0000 7feca2a00640 1 -- 172.21.15.27:0/2995908124 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c00cd80 con 0x7fecac105d90 2024-09-21T00:22:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.534+0000 7feca2a00640 1 -- 172.21.15.27:0/2995908124 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec9c0108e0 con 0x7fecac105d90 2024-09-21T00:22:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.534+0000 7fecb0a00640 1 -- 172.21.15.27:0/2995908124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecac1a28c0 con 0x7fecac105d90 2024-09-21T00:22:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.534+0000 7feca2a00640 1 -- 172.21.15.27:0/2995908124 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c018980 con 0x7fecac105d90 2024-09-21T00:22:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.535+0000 7fecb0a00640 1 -- 172.21.15.27:0/2995908124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecac1a9020 con 0x7fecac105d90 2024-09-21T00:22:37.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.539+0000 7feca2a00640 1 -- 172.21.15.27:0/2995908124 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fec9c010430 con 0x7fecac105d90 2024-09-21T00:22:37.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.539+0000 7fecb77c9740 1 Processor -- start 2024-09-21T00:22:37.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.540+0000 7fecb77c9740 1 -- start start 2024-09-21T00:22:37.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.540+0000 7fecb77c9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584163520 0x56258450a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.540+0000 7fecb77c9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x56258450a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.540+0000 7fecb77c9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584142840 0x56258450ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.540+0000 7fecb77c9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562584481290 con 0x562584163520 2024-09-21T00:22:37.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.540+0000 7fecb77c9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5625844706c0 con 0x562584142840 2024-09-21T00:22:37.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.540+0000 7fecb77c9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5625844744d0 con 0x56258413a660 2024-09-21T00:22:37.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.541+0000 7feca9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x56258450a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.541+0000 7feca9400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584142840 0x56258450ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.541+0000 7fecaa800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584163520 0x56258450a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.541+0000 7fecaa800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584163520 0x56258450a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59878/0 (socket says 172.21.15.27:59878) 2024-09-21T00:22:37.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.541+0000 7feca9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x56258450a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:50596/0 (socket says 172.21.15.27:50596) 2024-09-21T00:22:37.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.541+0000 7fecaa800640 1 -- 172.21.15.27:0/827974956 learned_addr learned my addr 172.21.15.27:0/827974956 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:37.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.542+0000 7feca2a00640 1 --2- 172.21.15.27:0/2995908124 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fec900b6680 0x7fec900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.542+0000 7fecaa800640 1 -- 172.21.15.27:0/827974956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 msgr2=0x56258450a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:37.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.542+0000 7fecaa800640 1 --2- 172.21.15.27:0/827974956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x56258450a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.542+0000 7fecaa800640 1 -- 172.21.15.27:0/827974956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584142840 msgr2=0x56258450ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:37.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.542+0000 7fecaa800640 1 --2- 172.21.15.27:0/827974956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584142840 0x56258450ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.542+0000 7fecaa800640 1 -- 172.21.15.27:0/827974956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625843fd460 con 0x562584163520 2024-09-21T00:22:37.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.542+0000 7feca2a00640 1 -- 172.21.15.27:0/2995908124 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fec9c0e7ef0 con 0x7fecac105d90 2024-09-21T00:22:37.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.542+0000 7feca9400640 1 --2- 172.21.15.27:0/827974956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584142840 0x56258450ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:22:37.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.543+0000 7fecaa800640 1 --2- 172.21.15.27:0/827974956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584163520 0x56258450a1c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7feca4009ca0 tx=0x7feca4030fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91e9ab02c04da31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:37.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.543+0000 7fecaa800640 1 --2- 172.21.15.27:0/2995908124 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fec900b6680 0x7fec900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.544+0000 7fec97000640 1 -- 172.21.15.27:0/827974956 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca404a040 con 0x562584163520 2024-09-21T00:22:37.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.544+0000 7fec97000640 1 -- 172.21.15.27:0/827974956 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feca4036030 con 0x562584163520 2024-09-21T00:22:37.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.544+0000 7fec97000640 1 -- 172.21.15.27:0/827974956 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4004440 con 0x562584163520 2024-09-21T00:22:37.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.545+0000 7fecb77c9740 1 -- 172.21.15.27:0/827974956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584163520 msgr2=0x56258450a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:37.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.545+0000 7fecb77c9740 1 --2- 172.21.15.27:0/827974956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584163520 0x56258450a1c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7feca4009ca0 tx=0x7feca4030fd0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.545+0000 7fecaa800640 1 --2- 172.21.15.27:0/2995908124 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fec900b6680 0x7fec900b8b90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feca4030e30 tx=0x7feca40046e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:37.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.545+0000 7fecb77c9740 1 -- 172.21.15.27:0/827974956 shutdown_connections 2024-09-21T00:22:37.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.545+0000 7fecb77c9740 1 --2- 172.21.15.27:0/827974956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584142840 0x56258450ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.545+0000 7fecb77c9740 1 --2- 172.21.15.27:0/827974956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x56258450a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.545+0000 7fecb77c9740 1 --2- 172.21.15.27:0/827974956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584163520 0x56258450a1c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.545+0000 7fecb77c9740 1 -- 172.21.15.27:0/827974956 >> 172.21.15.27:0/827974956 conn(0x562584157910 msgr2=0x5625845080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:37.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.545+0000 7fecb77c9740 1 -- 172.21.15.27:0/827974956 shutdown_connections 2024-09-21T00:22:37.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.545+0000 7fecb77c9740 1 -- 172.21.15.27:0/827974956 wait complete. 2024-09-21T00:22:37.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.546+0000 7fecb77c9740 1 -- start start 2024-09-21T00:22:37.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.546+0000 7fecb77c9740 1 Processor -- start 2024-09-21T00:22:37.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.547+0000 7fecb77c9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x56258451e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.547+0000 7fecb77c9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584142840 0x56258451eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.547+0000 7fecb77c9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584163520 0x56258452a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.547+0000 7fecb77c9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5625842d1bb0 con 0x562584142840 2024-09-21T00:22:37.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.547+0000 7fecb77c9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56258440b6c0 con 0x562584163520 2024-09-21T00:22:37.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.547+0000 7fecb77c9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562584521070 con 0x56258413a660 2024-09-21T00:22:37.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.547+0000 7fecb77c9740 10 client.0 _ll_register_callbacks cb 0x562584426490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:37.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.548+0000 7fecaa800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584163520 0x56258452a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.548+0000 7feca9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x56258451e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.548+0000 7fecaa800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584163520 0x56258452a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:59760/0 (socket says 172.21.15.27:59760) 2024-09-21T00:22:37.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.548+0000 7fecaa800640 1 -- 172.21.15.27:0/1807634140 learned_addr learned my addr 172.21.15.27:0/1807634140 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:37.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.548+0000 7feca9400640 1 --2- 172.21.15.27:0/1807634140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584142840 0x56258451eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.549+0000 7fecaa800640 1 -- 172.21.15.27:0/1807634140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 msgr2=0x56258451e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:37.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.549+0000 7fecaa800640 1 --2- 172.21.15.27:0/1807634140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x56258451e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.549+0000 7fecaa800640 1 -- 172.21.15.27:0/1807634140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584142840 msgr2=0x56258451eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:37.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.549+0000 7fecaa800640 1 --2- 172.21.15.27:0/1807634140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584142840 0x56258451eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.549+0000 7fecaa800640 1 -- 172.21.15.27:0/1807634140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625841ceb00 con 0x562584163520 2024-09-21T00:22:37.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.549+0000 7feca9e00640 1 --2- 172.21.15.27:0/1807634140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x56258451e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:22:37.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.549+0000 7fecaa800640 1 --2- 172.21.15.27:0/1807634140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584163520 0x56258452a3a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feca4004af0 tx=0x7feca4043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae77e33f47f3ef02 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:37.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.549+0000 7fec8b400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:22:37.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.550+0000 7fec8b400640 1 -- 172.21.15.27:0/1807634140 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca405f020 con 0x562584163520 2024-09-21T00:22:37.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.550+0000 7fec8b400640 1 -- 172.21.15.27:0/1807634140 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7feca4004340 con 0x562584163520 2024-09-21T00:22:37.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.550+0000 7fecb77c9740 1 -- 172.21.15.27:0/1807634140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca4036030 con 0x562584163520 2024-09-21T00:22:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.551+0000 7fec8b400640 1 -- 172.21.15.27:0/1807634140 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feca4057040 con 0x562584163520 2024-09-21T00:22:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.551+0000 7fecb77c9740 10 client.25628 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.551+0000 7fecb77c9740 1 -- 172.21.15.27:0/1807634140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562584245960 con 0x562584163520 2024-09-21T00:22:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.551+0000 7fec8b400640 1 -- 172.21.15.27:0/1807634140 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feca4036030 con 0x562584163520 2024-09-21T00:22:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.551+0000 7fec8b400640 1 -- 172.21.15.27:0/1807634140 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7feca405a030 con 0x562584163520 2024-09-21T00:22:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.551+0000 7fecb77c9740 10 client.25628 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.551+0000 7fecb77c9740 10 client.25628 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:22:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.551+0000 7fecb77c9740 20 client.25628 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.551+0000 7fecb77c9740 1 --2- 172.21.15.27:0/1807634140 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562584121ea0 0x562584530280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.551+0000 7fecb77c9740 4 client.25628 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:22:37.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.551+0000 7fecb77c9740 1 -- 172.21.15.27:0/1807634140 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625844831e0 con 0x562584121ea0 2024-09-21T00:22:37.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.552+0000 7feca9e00640 1 --2- 172.21.15.27:0/1807634140 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562584121ea0 0x562584530280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.553+0000 7feca9e00640 1 --2- 172.21.15.27:0/1807634140 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562584121ea0 0x562584530280 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da5aea3ba427d9d0 server_cookie=570d5a26feff642a in_seq=0 out_seq=0 2024-09-21T00:22:37.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.553+0000 7fec8b400640 10 client.25628 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:37.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.553+0000 7fec8b400640 1 -- 172.21.15.27:0/1807634140 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5625844831e0 con 0x562584121ea0 2024-09-21T00:22:37.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.553+0000 7fec8b400640 10 client.25628 handle_command_reply: tid=0 2024-09-21T00:22:37.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.554+0000 7fecb77c9740 1 client.25628 shutdown 2024-09-21T00:22:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.554+0000 7fecb77c9740 1 -- 172.21.15.27:0/1807634140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584163520 msgr2=0x56258452a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.554+0000 7fecb77c9740 1 --2- 172.21.15.27:0/1807634140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584163520 0x56258452a3a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feca4004af0 tx=0x7feca4043000 comp rx=0 tx=0).stop 2024-09-21T00:22:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.554+0000 7fecb77c9740 1 -- 172.21.15.27:0/1807634140 shutdown_connections 2024-09-21T00:22:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.554+0000 7fecb77c9740 1 --2- 172.21.15.27:0/1807634140 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562584121ea0 0x562584530280 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.554+0000 7fecb77c9740 1 --2- 172.21.15.27:0/1807634140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584163520 0x56258452a3a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.554+0000 7fecb77c9740 1 --2- 172.21.15.27:0/1807634140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584142840 0x56258451eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.554+0000 7fecb77c9740 1 --2- 172.21.15.27:0/1807634140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x56258451e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.554+0000 7fecb77c9740 1 -- 172.21.15.27:0/1807634140 >> 172.21.15.27:0/1807634140 conn(0x562584157910 msgr2=0x562584506650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.554+0000 7fec8b400640 0 client.25628 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.555+0000 7fecb77c9740 1 -- 172.21.15.27:0/1807634140 shutdown_connections 2024-09-21T00:22:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.555+0000 7fecb77c9740 1 -- 172.21.15.27:0/1807634140 wait complete. 2024-09-21T00:22:37.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.555+0000 7fecb77c9740 20 client.25628 trim_cache size 0 max 16384 2024-09-21T00:22:37.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.573+0000 7fecb77c9740 1 Processor -- start 2024-09-21T00:22:37.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.573+0000 7fecb77c9740 1 -- start start 2024-09-21T00:22:37.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.574+0000 7fecb77c9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56258413a660 0x562584520290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.574+0000 7fecb77c9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584142840 0x56258450b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.574+0000 7fecb77c9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562584163520 0x56258451b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.574+0000 7fecb77c9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1b1510 con 0x562584142840 2024-09-21T00:22:37.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.574+0000 7fecb77c9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1aede0 con 0x56258413a660 2024-09-21T00:22:37.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.574+0000 7fecb77c9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1acc30 con 0x562584163520 2024-09-21T00:22:37.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.574+0000 7feca9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56258413a660 0x562584520290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.574+0000 7fecaa800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562584163520 0x56258451b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.574+0000 7feca9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56258413a660 0x562584520290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:59764/0 (socket says 172.21.15.27:59764) 2024-09-21T00:22:37.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.574+0000 7feca9e00640 1 -- 172.21.15.27:0/3170125130 learned_addr learned my addr 172.21.15.27:0/3170125130 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:37.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.575+0000 7fecaa800640 1 -- 172.21.15.27:0/3170125130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56258413a660 msgr2=0x562584520290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:37.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.575+0000 7fecaa800640 1 --2- 172.21.15.27:0/3170125130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56258413a660 0x562584520290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.575+0000 7fecaa800640 1 -- 172.21.15.27:0/3170125130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584142840 msgr2=0x56258450b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:37.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.575+0000 7feca9400640 1 --2- 172.21.15.27:0/3170125130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584142840 0x56258450b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.575+0000 7fecaa800640 1 --2- 172.21.15.27:0/3170125130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584142840 0x56258450b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.575+0000 7fecaa800640 1 -- 172.21.15.27:0/3170125130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258451f670 con 0x562584163520 2024-09-21T00:22:37.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.575+0000 7feca9e00640 1 --2- 172.21.15.27:0/3170125130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56258413a660 0x562584520290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:22:37.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.576+0000 7fecaa800640 1 --2- 172.21.15.27:0/3170125130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562584163520 0x56258451b5a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feca402add0 tx=0x7feca402a8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40f99944019562e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:37.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.576+0000 7fec96600640 1 -- 172.21.15.27:0/3170125130 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4072070 con 0x562584163520 2024-09-21T00:22:37.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.576+0000 7fec96600640 1 -- 172.21.15.27:0/3170125130 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feca4036040 con 0x562584163520 2024-09-21T00:22:37.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.576+0000 7fec96600640 1 -- 172.21.15.27:0/3170125130 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca405f040 con 0x562584163520 2024-09-21T00:22:37.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.577+0000 7fecb77c9740 1 -- 172.21.15.27:0/3170125130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562584163520 msgr2=0x56258451b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:37.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.577+0000 7fecb77c9740 1 --2- 172.21.15.27:0/3170125130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562584163520 0x56258451b5a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feca402add0 tx=0x7feca402a8b0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.577+0000 7fecb77c9740 1 -- 172.21.15.27:0/3170125130 shutdown_connections 2024-09-21T00:22:37.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.577+0000 7fecb77c9740 1 --2- 172.21.15.27:0/3170125130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562584163520 0x56258451b5a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.577+0000 7fecb77c9740 1 --2- 172.21.15.27:0/3170125130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584142840 0x56258450b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.577+0000 7fecb77c9740 1 --2- 172.21.15.27:0/3170125130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56258413a660 0x562584520290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.577+0000 7fecb77c9740 1 -- 172.21.15.27:0/3170125130 >> 172.21.15.27:0/3170125130 conn(0x562584157910 msgr2=0x56258450b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:37.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.577+0000 7fecb77c9740 1 -- 172.21.15.27:0/3170125130 shutdown_connections 2024-09-21T00:22:37.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.578+0000 7fecb77c9740 1 -- 172.21.15.27:0/3170125130 wait complete. 2024-09-21T00:22:37.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.578+0000 7fecb77c9740 1 -- start start 2024-09-21T00:22:37.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.578+0000 7fecb77c9740 1 Processor -- start 2024-09-21T00:22:37.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.579+0000 7fecb77c9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x562584533b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.579+0000 7fecb77c9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584142840 0x562584530050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.579+0000 7fecb77c9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584163520 0x562584530680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.579+0000 7fecb77c9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1ad2a0 con 0x562584142840 2024-09-21T00:22:37.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.579+0000 7fecb77c9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1ae710 con 0x562584163520 2024-09-21T00:22:37.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.579+0000 7fecb77c9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1ad970 con 0x56258413a660 2024-09-21T00:22:37.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.579+0000 7fecb77c9740 10 client.0 _ll_register_callbacks cb 0x7feca4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:37.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.579+0000 7feca9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x562584533b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.579+0000 7feca9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x562584533b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:50620/0 (socket says 172.21.15.27:50620) 2024-09-21T00:22:37.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.579+0000 7feca9e00640 1 -- 172.21.15.27:0/2019843780 learned_addr learned my addr 172.21.15.27:0/2019843780 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:37.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.579+0000 7feca9400640 1 --2- 172.21.15.27:0/2019843780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584142840 0x562584530050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.580+0000 7fecaa800640 1 --2- 172.21.15.27:0/2019843780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584163520 0x562584530680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.580+0000 7feca9e00640 1 -- 172.21.15.27:0/2019843780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584163520 msgr2=0x562584530680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:37.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.580+0000 7feca9e00640 1 --2- 172.21.15.27:0/2019843780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584163520 0x562584530680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.580+0000 7feca9e00640 1 -- 172.21.15.27:0/2019843780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584142840 msgr2=0x562584530050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:37.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.580+0000 7feca9e00640 1 --2- 172.21.15.27:0/2019843780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584142840 0x562584530050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.580+0000 7feca9e00640 1 -- 172.21.15.27:0/2019843780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562584530e10 con 0x56258413a660 2024-09-21T00:22:37.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.580+0000 7feca9e00640 1 --2- 172.21.15.27:0/2019843780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x562584533b90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0a9ed0 tx=0x7fec9c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cd42c9d682ae3d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:37.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.580+0000 7fec8b400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:22:37.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.581+0000 7fec8b400640 1 -- 172.21.15.27:0/2019843780 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c0ae050 con 0x56258413a660 2024-09-21T00:22:37.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.581+0000 7fec8b400640 1 -- 172.21.15.27:0/2019843780 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fec9c0e54d0 con 0x56258413a660 2024-09-21T00:22:37.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.581+0000 7fecb77c9740 1 -- 172.21.15.27:0/2019843780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca4036040 con 0x56258413a660 2024-09-21T00:22:37.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.581+0000 7fec8b400640 1 -- 172.21.15.27:0/2019843780 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec9c0e56d0 con 0x56258413a660 2024-09-21T00:22:37.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.581+0000 7fecb77c9740 10 client.25609 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:37.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.581+0000 7fecb77c9740 1 -- 172.21.15.27:0/2019843780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56258459bae0 con 0x56258413a660 2024-09-21T00:22:37.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.581+0000 7fec8b400640 1 -- 172.21.15.27:0/2019843780 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feca4036040 con 0x56258413a660 2024-09-21T00:22:37.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.582+0000 7fec8b400640 1 -- 172.21.15.27:0/2019843780 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fec9c0a93c0 con 0x56258413a660 2024-09-21T00:22:37.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.582+0000 7fecb77c9740 10 client.25609 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:37.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.582+0000 7fecb77c9740 10 client.25609 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:22:37.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.582+0000 7fecb77c9740 20 client.25609 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:37.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.582+0000 7fecb77c9740 1 --2- 172.21.15.27:0/2019843780 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5625840f7250 0x56258459e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:37.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.582+0000 7fecb77c9740 4 client.25609 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e14cd8ec", "release": true, "await": true}] 2024-09-21T00:22:37.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.582+0000 7fecb77c9740 1 -- 172.21.15.27:0/2019843780 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e14cd8ec", "release": true, "await": true}) -- 0x56258447ec90 con 0x5625840f7250 2024-09-21T00:22:37.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.582+0000 7feca9400640 1 --2- 172.21.15.27:0/2019843780 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5625840f7250 0x56258459e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:37.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.583+0000 7feca9400640 1 --2- 172.21.15.27:0/2019843780 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5625840f7250 0x56258459e2a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95437d29ae9110cb server_cookie=5474ff9f48a1dfd6 in_seq=0 out_seq=0 2024-09-21T00:22:37.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.583+0000 7fec8b400640 10 client.25609 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:37.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.584+0000 7fec8b400640 1 -- 172.21.15.27:0/2019843780 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56258447ec90 con 0x5625840f7250 2024-09-21T00:22:37.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.584+0000 7fec8b400640 10 client.25609 handle_command_reply: tid=0 2024-09-21T00:22:37.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.584+0000 7fecb77c9740 1 client.25609 shutdown 2024-09-21T00:22:37.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.585+0000 7fecb77c9740 1 -- 172.21.15.27:0/2019843780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 msgr2=0x562584533b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:37.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.585+0000 7fecb77c9740 1 --2- 172.21.15.27:0/2019843780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x562584533b90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0a9ed0 tx=0x7fec9c000bc0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.585+0000 7fecb77c9740 1 -- 172.21.15.27:0/2019843780 shutdown_connections 2024-09-21T00:22:37.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.585+0000 7fecb77c9740 1 --2- 172.21.15.27:0/2019843780 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5625840f7250 0x56258459e2a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.585+0000 7fecb77c9740 1 --2- 172.21.15.27:0/2019843780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562584163520 0x562584530680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.585+0000 7fecb77c9740 1 --2- 172.21.15.27:0/2019843780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562584142840 0x562584530050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.585+0000 7fecb77c9740 1 --2- 172.21.15.27:0/2019843780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56258413a660 0x562584533b90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.585+0000 7fecb77c9740 1 -- 172.21.15.27:0/2019843780 >> 172.21.15.27:0/2019843780 conn(0x562584157910 msgr2=0x56258451d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:37.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.585+0000 7fec8b400640 0 client.25609 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:37.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.585+0000 7fecb77c9740 1 -- 172.21.15.27:0/2019843780 shutdown_connections 2024-09-21T00:22:37.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.586+0000 7fecb77c9740 1 -- 172.21.15.27:0/2019843780 wait complete. 2024-09-21T00:22:37.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.586+0000 7fecb77c9740 20 client.25609 trim_cache size 0 max 16384 2024-09-21T00:22:37.589 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:22:37.589 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 26, 2024-09-21T00:22:37.589 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24761, 2024-09-21T00:22:37.589 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 4, 2024-09-21T00:22:37.589 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T00:22:37.589 INFO:teuthology.orchestra.run.smithi027.stdout: "e14cd8ec": { 2024-09-21T00:22:37.589 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 4, 2024-09-21T00:22:37.589 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T00:22:37.589 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:22:37.589 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T00:22:37.589 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:22:37.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:22:37.590 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T00:22:37.590 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 134.5, 2024-09-21T00:22:37.590 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T00:22:37.590 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T00:22:37.590 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T00:22:37.590 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:22:37.590 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T00:22:37.590 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:22:37.590 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:22:37.591 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:22:37.591 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:22:37.591 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:22:37.591 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:22:37.591 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:22:37.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.587+0000 7fecb0a00640 1 -- 172.21.15.27:0/2995908124 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fec900b6680 msgr2=0x7fec900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:37.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.587+0000 7fecb0a00640 1 --2- 172.21.15.27:0/2995908124 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fec900b6680 0x7fec900b8b90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feca4030e30 tx=0x7feca40046e0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.587+0000 7fecb0a00640 1 -- 172.21.15.27:0/2995908124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecac105d90 msgr2=0x7fecac19c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:37.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.587+0000 7fecb0a00640 1 --2- 172.21.15.27:0/2995908124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecac105d90 0x7fecac19c660 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00e9f0 tx=0x7fec9c00eeb0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.587+0000 7fecb0a00640 1 -- 172.21.15.27:0/2995908124 shutdown_connections 2024-09-21T00:22:37.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.587+0000 7fecb0a00640 1 --2- 172.21.15.27:0/2995908124 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fec900b6680 0x7fec900b8b90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.587+0000 7fecb0a00640 1 --2- 172.21.15.27:0/2995908124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecac10c020 0x7fecac1a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.587+0000 7fecb0a00640 1 --2- 172.21.15.27:0/2995908124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecac10b6d0 0x7fecac19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.587+0000 7fecb0a00640 1 --2- 172.21.15.27:0/2995908124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecac105d90 0x7fecac19c660 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:37.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.587+0000 7fecb0a00640 1 -- 172.21.15.27:0/2995908124 >> 172.21.15.27:0/2995908124 conn(0x7fecac101860 msgr2=0x7fecac108c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:37.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.587+0000 7fecb0a00640 1 -- 172.21.15.27:0/2995908124 shutdown_connections 2024-09-21T00:22:37.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:22:37.588+0000 7fecb0a00640 1 -- 172.21.15.27:0/2995908124 wait complete. 2024-09-21T00:22:37.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:37 smithi098 ceph-mon[34725]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-21T00:22:37.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:37 smithi089 ceph-mon[34481]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-21T00:22:38.420 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e14cd8ec', total seconds elapsed: 19.3 2024-09-21T00:22:39.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:39 smithi098 ceph-mon[34725]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T00:22:39.781 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:39 smithi089 ceph-mon[34481]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T00:22:39.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:39 smithi027 ceph-mon[30745]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T00:22:41.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:41 smithi027 ceph-mon[30745]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-21T00:22:41.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:41 smithi089 ceph-mon[34481]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-21T00:22:41.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:41 smithi098 ceph-mon[34725]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-21T00:22:43.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:43 smithi027 ceph-mon[30745]: pgmap v417: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T00:22:43.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:43 smithi089 ceph-mon[34481]: pgmap v417: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T00:22:43.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:43 smithi098 ceph-mon[34725]: pgmap v417: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T00:22:45.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:45 smithi027 ceph-mon[30745]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T00:22:45.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:45 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:22:45.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:45 smithi089 ceph-mon[34481]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T00:22:45.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:45 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:22:45.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:45 smithi098 ceph-mon[34725]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T00:22:45.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:45 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:22:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:46 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:22:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:46 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:22:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:46 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:22:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.847+0000 7f0f90c00640 1 -- 172.21.15.27:0/406517934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f8c10ff00 msgr2=0x7f0f8c112320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.847+0000 7f0f90c00640 1 --2- 172.21.15.27:0/406517934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f8c10ff00 0x7f0f8c112320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009be0 tx=0x7f0f8402a6c0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.847+0000 7f0f90c00640 1 -- 172.21.15.27:0/406517934 shutdown_connections 2024-09-21T00:22:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.847+0000 7f0f90c00640 1 --2- 172.21.15.27:0/406517934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f8c10ff00 0x7f0f8c112320 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.847+0000 7f0f90c00640 1 --2- 172.21.15.27:0/406517934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f8c102ca0 0x7f0f8c10f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.847+0000 7f0f90c00640 1 --2- 172.21.15.27:0/406517934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f8c102350 0x7f0f8c102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.847+0000 7f0f90c00640 1 -- 172.21.15.27:0/406517934 >> 172.21.15.27:0/406517934 conn(0x7f0f8c0fe210 msgr2=0x7f0f8c100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.847+0000 7f0f90c00640 1 -- 172.21.15.27:0/406517934 shutdown_connections 2024-09-21T00:22:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.847+0000 7f0f90c00640 1 -- 172.21.15.27:0/406517934 wait complete. 2024-09-21T00:22:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.849+0000 7f0f90c00640 1 Processor -- start 2024-09-21T00:22:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.849+0000 7f0f90c00640 1 -- start start 2024-09-21T00:22:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.849+0000 7f0f90c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f8c102350 0x7f0f8c19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.849+0000 7f0f90c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f8c102ca0 0x7f0f8c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.849+0000 7f0f90c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f8c10ff00 0x7f0f8c1a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.849+0000 7f0f90c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c115a10 con 0x7f0f8c102350 2024-09-21T00:22:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.849+0000 7f0f90c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c115890 con 0x7f0f8c10ff00 2024-09-21T00:22:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.849+0000 7f0f90c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c115b90 con 0x7f0f8c102ca0 2024-09-21T00:22:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.850+0000 7f0f8be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f8c10ff00 0x7f0f8c1a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.850+0000 7f0f8b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f8c102350 0x7f0f8c19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.850+0000 7f0f8be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f8c10ff00 0x7f0f8c1a1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38254/0 (socket says 172.21.15.27:38254) 2024-09-21T00:22:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.850+0000 7f0f8be00640 1 -- 172.21.15.27:0/1546408649 learned_addr learned my addr 172.21.15.27:0/1546408649 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.850+0000 7f0f8aa00640 1 --2- 172.21.15.27:0/1546408649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f8c102ca0 0x7f0f8c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.850+0000 7f0f8be00640 1 -- 172.21.15.27:0/1546408649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f8c102ca0 msgr2=0x7f0f8c19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.850+0000 7f0f8be00640 1 --2- 172.21.15.27:0/1546408649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f8c102ca0 0x7f0f8c19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.850+0000 7f0f8be00640 1 -- 172.21.15.27:0/1546408649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f8c102350 msgr2=0x7f0f8c19c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.851+0000 7f0f8be00640 1 --2- 172.21.15.27:0/1546408649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f8c102350 0x7f0f8c19c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.851+0000 7f0f8be00640 1 -- 172.21.15.27:0/1546408649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8c1a2320 con 0x7f0f8c10ff00 2024-09-21T00:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.851+0000 7f0f8b400640 1 --2- 172.21.15.27:0/1546408649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f8c102350 0x7f0f8c19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.851+0000 7f0f8be00640 1 --2- 172.21.15.27:0/1546408649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f8c10ff00 0x7f0f8c1a1c20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009b30 tx=0x7f0f84003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.851+0000 7f0f83e00640 1 -- 172.21.15.27:0/1546408649 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84040020 con 0x7f0f8c10ff00 2024-09-21T00:22:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.851+0000 7f0f83e00640 1 -- 172.21.15.27:0/1546408649 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f84030970 con 0x7f0f8c10ff00 2024-09-21T00:22:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.851+0000 7f0f90c00640 1 -- 172.21.15.27:0/1546408649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8c1a25b0 con 0x7f0f8c10ff00 2024-09-21T00:22:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.852+0000 7f0f90c00640 1 -- 172.21.15.27:0/1546408649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f8c1a8da0 con 0x7f0f8c10ff00 2024-09-21T00:22:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.852+0000 7f0f83e00640 1 -- 172.21.15.27:0/1546408649 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84036040 con 0x7f0f8c10ff00 2024-09-21T00:22:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.855+0000 7f0f9924d740 1 Processor -- start 2024-09-21T00:22:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.857+0000 7f0f83e00640 1 -- 172.21.15.27:0/1546408649 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0f84043020 con 0x7f0f8c10ff00 2024-09-21T00:22:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.857+0000 7f0f9924d740 1 -- start start 2024-09-21T00:22:46.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.858+0000 7f0f9924d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe545a1b0 0x562fe57fee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.858+0000 7f0f9924d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe54312f0 0x562fe57ff350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.858+0000 7f0f9924d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54394d0 0x562fe57ff890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.858+0000 7f0f9924d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562fe57746b0 con 0x562fe54394d0 2024-09-21T00:22:46.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.858+0000 7f0f9924d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562fe5763ae0 con 0x562fe54312f0 2024-09-21T00:22:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.858+0000 7f0f9924d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562fe57678f0 con 0x562fe545a1b0 2024-09-21T00:22:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.858+0000 7f0f8aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe545a1b0 0x562fe57fee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.858+0000 7f0f8aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe545a1b0 0x562fe57fee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:35758/0 (socket says 172.21.15.27:35758) 2024-09-21T00:22:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.858+0000 7f0f8aa00640 1 -- 172.21.15.27:0/3598065644 learned_addr learned my addr 172.21.15.27:0/3598065644 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.858+0000 7f0f8b400640 1 --2- 172.21.15.27:0/3598065644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe54312f0 0x562fe57ff350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.858+0000 7f0f8aa00640 1 --2- 172.21.15.27:0/3598065644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54394d0 0x562fe57ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.859+0000 7f0f8aa00640 1 -- 172.21.15.27:0/3598065644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe545a1b0 msgr2=0x562fe57fee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.859+0000 7f0f8aa00640 1 --2- 172.21.15.27:0/3598065644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe545a1b0 0x562fe57fee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.859+0000 7f0f8aa00640 1 -- 172.21.15.27:0/3598065644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe54312f0 msgr2=0x562fe57ff350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.859+0000 7f0f8aa00640 1 --2- 172.21.15.27:0/3598065644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe54312f0 0x562fe57ff350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.859+0000 7f0f8aa00640 1 -- 172.21.15.27:0/3598065644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe5730780 con 0x562fe54394d0 2024-09-21T00:22:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.860+0000 7f0f8aa00640 1 --2- 172.21.15.27:0/3598065644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54394d0 0x562fe57ff890 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0f78011710 tx=0x7f0f78005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa6682acd698624 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.860+0000 7f0f83e00640 1 --2- 172.21.15.27:0/1546408649 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0f700b6670 0x7f0f700b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.860+0000 7f0f77e00640 1 -- 172.21.15.27:0/3598065644 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78011d50 con 0x562fe54394d0 2024-09-21T00:22:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.860+0000 7f0f77e00640 1 -- 172.21.15.27:0/3598065644 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f7800fcf0 con 0x562fe54394d0 2024-09-21T00:22:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.860+0000 7f0f77e00640 1 -- 172.21.15.27:0/3598065644 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7801ebb0 con 0x562fe54394d0 2024-09-21T00:22:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.861+0000 7f0f83e00640 1 -- 172.21.15.27:0/1546408649 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f0f841086f0 con 0x7f0f8c10ff00 2024-09-21T00:22:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.861+0000 7f0f8b400640 1 --2- 172.21.15.27:0/1546408649 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0f700b6670 0x7f0f700b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.862+0000 7f0f9924d740 1 -- 172.21.15.27:0/3598065644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54394d0 msgr2=0x562fe57ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.862+0000 7f0f9924d740 1 --2- 172.21.15.27:0/3598065644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54394d0 0x562fe57ff890 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0f78011710 tx=0x7f0f78005f90 comp rx=0 tx=0).stop 2024-09-21T00:22:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.862+0000 7f0f9924d740 1 -- 172.21.15.27:0/3598065644 shutdown_connections 2024-09-21T00:22:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.862+0000 7f0f9924d740 1 --2- 172.21.15.27:0/3598065644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54394d0 0x562fe57ff890 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.862+0000 7f0f9924d740 1 --2- 172.21.15.27:0/3598065644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe54312f0 0x562fe57ff350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.862+0000 7f0f9924d740 1 --2- 172.21.15.27:0/3598065644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe545a1b0 0x562fe57fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.862+0000 7f0f9924d740 1 -- 172.21.15.27:0/3598065644 >> 172.21.15.27:0/3598065644 conn(0x562fe544e300 msgr2=0x562fe57fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.862+0000 7f0f9924d740 1 -- 172.21.15.27:0/3598065644 shutdown_connections 2024-09-21T00:22:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.862+0000 7f0f8b400640 1 --2- 172.21.15.27:0/1546408649 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0f700b6670 0x7f0f700b8b80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x562fe57fda90 tx=0x7f0f7c009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.862+0000 7f0f9924d740 1 -- 172.21.15.27:0/3598065644 wait complete. 2024-09-21T00:22:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.863+0000 7f0f9924d740 1 -- start start 2024-09-21T00:22:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.864+0000 7f0f9924d740 1 Processor -- start 2024-09-21T00:22:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.864+0000 7f0f9924d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe54312f0 0x562fe580a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.864+0000 7f0f9924d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe545a1b0 0x562fe580aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.864+0000 7f0f9924d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe5418d80 0x562fe5816880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.864+0000 7f0f9924d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562fe569bab0 con 0x562fe545a1b0 2024-09-21T00:22:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.864+0000 7f0f9924d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562fe570a570 con 0x562fe5418d80 2024-09-21T00:22:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.864+0000 7f0f9924d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562fe580d590 con 0x562fe54312f0 2024-09-21T00:22:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.865+0000 7f0f9924d740 10 client.0 _ll_register_callbacks cb 0x562fe5719100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.865+0000 7f0f8be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe5418d80 0x562fe5816880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.865+0000 7f0f8be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe5418d80 0x562fe5816880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38278/0 (socket says 172.21.15.27:38278) 2024-09-21T00:22:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.865+0000 7f0f8be00640 1 -- 172.21.15.27:0/1936689185 learned_addr learned my addr 172.21.15.27:0/1936689185 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.865+0000 7f0f8b400640 1 --2- 172.21.15.27:0/1936689185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe54312f0 0x562fe580a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.865+0000 7f0f8aa00640 1 --2- 172.21.15.27:0/1936689185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe545a1b0 0x562fe580aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.865+0000 7f0f8b400640 1 -- 172.21.15.27:0/1936689185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe5418d80 msgr2=0x562fe5816880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.865+0000 7f0f8b400640 1 --2- 172.21.15.27:0/1936689185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe5418d80 0x562fe5816880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.865+0000 7f0f8b400640 1 -- 172.21.15.27:0/1936689185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe545a1b0 msgr2=0x562fe580aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.865+0000 7f0f8b400640 1 --2- 172.21.15.27:0/1936689185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe545a1b0 0x562fe580aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.866+0000 7f0f8b400640 1 -- 172.21.15.27:0/1936689185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe54c5790 con 0x562fe54312f0 2024-09-21T00:22:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.866+0000 7f0f8be00640 1 --2- 172.21.15.27:0/1936689185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe5418d80 0x562fe5816880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:22:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.866+0000 7f0f8aa00640 1 --2- 172.21.15.27:0/1936689185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe545a1b0 0x562fe580aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:22:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.866+0000 7f0f8b400640 1 --2- 172.21.15.27:0/1936689185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe54312f0 0x562fe580a530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c005fd0 tx=0x7f0f7c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa74efe4bc935a3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.866+0000 7f0f74c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:22:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.866+0000 7f0f74c00640 1 -- 172.21.15.27:0/1936689185 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c02e060 con 0x562fe54312f0 2024-09-21T00:22:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.867+0000 7f0f9924d740 1 -- 172.21.15.27:0/1936689185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f7800fcf0 con 0x562fe54312f0 2024-09-21T00:22:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.867+0000 7f0f74c00640 1 -- 172.21.15.27:0/1936689185 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f0f7c018030 con 0x562fe54312f0 2024-09-21T00:22:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.867+0000 7f0f74c00640 1 -- 172.21.15.27:0/1936689185 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f7c035050 con 0x562fe54312f0 2024-09-21T00:22:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.867+0000 7f0f9924d740 10 client.25621 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.867+0000 7f0f9924d740 1 -- 172.21.15.27:0/1936689185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562fe553e620 con 0x562fe54312f0 2024-09-21T00:22:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.867+0000 7f0f74c00640 1 -- 172.21.15.27:0/1936689185 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f7800fcf0 con 0x562fe54312f0 2024-09-21T00:22:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.868+0000 7f0f74c00640 1 -- 172.21.15.27:0/1936689185 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0f7c01c7e0 con 0x562fe54312f0 2024-09-21T00:22:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.868+0000 7f0f9924d740 10 client.25621 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.868+0000 7f0f9924d740 10 client.25621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:22:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.868+0000 7f0f9924d740 20 client.25621 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.868+0000 7f0f9924d740 1 --2- 172.21.15.27:0/1936689185 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562fe53ee1c0 0x562fe581c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.868+0000 7f0f9924d740 4 client.25621 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:22:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.868+0000 7f0f9924d740 1 -- 172.21.15.27:0/1936689185 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fe5776600 con 0x562fe53ee1c0 2024-09-21T00:22:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.869+0000 7f0f8aa00640 1 --2- 172.21.15.27:0/1936689185 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562fe53ee1c0 0x562fe581c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.869+0000 7f0f8aa00640 1 --2- 172.21.15.27:0/1936689185 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562fe53ee1c0 0x562fe581c680 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c27d99537dc764a0 server_cookie=7f30a2ed27706bdd in_seq=0 out_seq=0 2024-09-21T00:22:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.869+0000 7f0f74c00640 10 client.25621 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.870+0000 7f0f74c00640 1 -- 172.21.15.27:0/1936689185 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562fe5776600 con 0x562fe53ee1c0 2024-09-21T00:22:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.870+0000 7f0f74c00640 10 client.25621 handle_command_reply: tid=0 2024-09-21T00:22:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.870+0000 7f0f9924d740 1 client.25621 shutdown 2024-09-21T00:22:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.871+0000 7f0f9924d740 1 -- 172.21.15.27:0/1936689185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe54312f0 msgr2=0x562fe580a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.871+0000 7f0f9924d740 1 --2- 172.21.15.27:0/1936689185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe54312f0 0x562fe580a530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c005fd0 tx=0x7f0f7c004730 comp rx=0 tx=0).stop 2024-09-21T00:22:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.871+0000 7f0f9924d740 1 -- 172.21.15.27:0/1936689185 shutdown_connections 2024-09-21T00:22:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.871+0000 7f0f9924d740 1 --2- 172.21.15.27:0/1936689185 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562fe53ee1c0 0x562fe581c680 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.871+0000 7f0f9924d740 1 --2- 172.21.15.27:0/1936689185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe5418d80 0x562fe5816880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.871+0000 7f0f9924d740 1 --2- 172.21.15.27:0/1936689185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe545a1b0 0x562fe580aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.871+0000 7f0f9924d740 1 --2- 172.21.15.27:0/1936689185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe54312f0 0x562fe580a530 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.871+0000 7f0f9924d740 1 -- 172.21.15.27:0/1936689185 >> 172.21.15.27:0/1936689185 conn(0x562fe544e300 msgr2=0x562fe57ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.871+0000 7f0f74c00640 0 client.25621 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.871+0000 7f0f9924d740 1 -- 172.21.15.27:0/1936689185 shutdown_connections 2024-09-21T00:22:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.871+0000 7f0f9924d740 1 -- 172.21.15.27:0/1936689185 wait complete. 2024-09-21T00:22:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.872+0000 7f0f9924d740 20 client.25621 trim_cache size 0 max 16384 2024-09-21T00:22:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.902+0000 7f0f9924d740 1 Processor -- start 2024-09-21T00:22:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.902+0000 7f0f9924d740 1 -- start start 2024-09-21T00:22:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.903+0000 7f0f9924d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe5418d80 0x562fe5816780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.903+0000 7f0f9924d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54312f0 0x562fe57fc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.903+0000 7f0f9924d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe545a1b0 0x562fe57fcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.903+0000 7f0f9924d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c1b5660 con 0x562fe54312f0 2024-09-21T00:22:46.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.903+0000 7f0f9924d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c1b2f30 con 0x562fe545a1b0 2024-09-21T00:22:46.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.903+0000 7f0f9924d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c1b0d80 con 0x562fe5418d80 2024-09-21T00:22:46.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.903+0000 7f0f8b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe5418d80 0x562fe5816780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.903+0000 7f0f8be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe545a1b0 0x562fe57fcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.903+0000 7f0f8b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe5418d80 0x562fe5816780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:35786/0 (socket says 172.21.15.27:35786) 2024-09-21T00:22:46.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.903+0000 7f0f8b400640 1 -- 172.21.15.27:0/631225068 learned_addr learned my addr 172.21.15.27:0/631225068 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:46.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.903+0000 7f0f8aa00640 1 --2- 172.21.15.27:0/631225068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54312f0 0x562fe57fc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.904+0000 7f0f8be00640 1 -- 172.21.15.27:0/631225068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe5418d80 msgr2=0x562fe5816780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.904+0000 7f0f8be00640 1 --2- 172.21.15.27:0/631225068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe5418d80 0x562fe5816780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.904+0000 7f0f8be00640 1 -- 172.21.15.27:0/631225068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54312f0 msgr2=0x562fe57fc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.904+0000 7f0f8be00640 1 --2- 172.21.15.27:0/631225068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54312f0 0x562fe57fc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.904+0000 7f0f8be00640 1 -- 172.21.15.27:0/631225068 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe580cb10 con 0x562fe545a1b0 2024-09-21T00:22:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.904+0000 7f0f8be00640 1 --2- 172.21.15.27:0/631225068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe545a1b0 0x562fe57fcaf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f840ca720 tx=0x7f0f840cdb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f51f3239d1d2d31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.904+0000 7f0f77400640 1 -- 172.21.15.27:0/631225068 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f840ce850 con 0x562fe545a1b0 2024-09-21T00:22:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.904+0000 7f0f77400640 1 -- 172.21.15.27:0/631225068 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f840d2c30 con 0x562fe545a1b0 2024-09-21T00:22:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.904+0000 7f0f77400640 1 -- 172.21.15.27:0/631225068 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f840045a0 con 0x562fe545a1b0 2024-09-21T00:22:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.905+0000 7f0f9924d740 1 -- 172.21.15.27:0/631225068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe545a1b0 msgr2=0x562fe57fcaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.905+0000 7f0f9924d740 1 --2- 172.21.15.27:0/631225068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe545a1b0 0x562fe57fcaf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f840ca720 tx=0x7f0f840cdb60 comp rx=0 tx=0).stop 2024-09-21T00:22:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.906+0000 7f0f9924d740 1 -- 172.21.15.27:0/631225068 shutdown_connections 2024-09-21T00:22:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.906+0000 7f0f9924d740 1 --2- 172.21.15.27:0/631225068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe545a1b0 0x562fe57fcaf0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.906+0000 7f0f9924d740 1 --2- 172.21.15.27:0/631225068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54312f0 0x562fe57fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.906+0000 7f0f9924d740 1 --2- 172.21.15.27:0/631225068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe5418d80 0x562fe5816780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.906+0000 7f0f9924d740 1 -- 172.21.15.27:0/631225068 >> 172.21.15.27:0/631225068 conn(0x562fe544e300 msgr2=0x562fe5808d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.906+0000 7f0f9924d740 1 -- 172.21.15.27:0/631225068 shutdown_connections 2024-09-21T00:22:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.906+0000 7f0f9924d740 1 -- 172.21.15.27:0/631225068 wait complete. 2024-09-21T00:22:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.906+0000 7f0f9924d740 1 -- start start 2024-09-21T00:22:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.907+0000 7f0f9924d740 1 Processor -- start 2024-09-21T00:22:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.907+0000 7f0f9924d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe5418d80 0x562fe5813080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.907+0000 7f0f9924d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54312f0 0x562fe580e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.907+0000 7f0f9924d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe545a1b0 0x562fe580eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.907+0000 7f0f9924d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c1b13f0 con 0x562fe54312f0 2024-09-21T00:22:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.907+0000 7f0f9924d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c1b2860 con 0x562fe5418d80 2024-09-21T00:22:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.907+0000 7f0f9924d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c1b1ac0 con 0x562fe545a1b0 2024-09-21T00:22:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.908+0000 7f0f9924d740 10 client.0 _ll_register_callbacks cb 0x7f0f7c00fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:22:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.908+0000 7f0f8b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe5418d80 0x562fe5813080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.908+0000 7f0f8b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe5418d80 0x562fe5813080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38296/0 (socket says 172.21.15.27:38296) 2024-09-21T00:22:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.908+0000 7f0f8b400640 1 -- 172.21.15.27:0/2278163559 learned_addr learned my addr 172.21.15.27:0/2278163559 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:22:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.908+0000 7f0f8be00640 1 --2- 172.21.15.27:0/2278163559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe545a1b0 0x562fe580eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.908+0000 7f0f8aa00640 1 --2- 172.21.15.27:0/2278163559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54312f0 0x562fe580e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.908+0000 7f0f8b400640 1 -- 172.21.15.27:0/2278163559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe545a1b0 msgr2=0x562fe580eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.909+0000 7f0f8b400640 1 --2- 172.21.15.27:0/2278163559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe545a1b0 0x562fe580eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.909+0000 7f0f8b400640 1 -- 172.21.15.27:0/2278163559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54312f0 msgr2=0x562fe580e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:22:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.909+0000 7f0f8b400640 1 --2- 172.21.15.27:0/2278163559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54312f0 0x562fe580e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.909+0000 7f0f8b400640 1 -- 172.21.15.27:0/2278163559 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe580f300 con 0x562fe5418d80 2024-09-21T00:22:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.909+0000 7f0f8aa00640 1 --2- 172.21.15.27:0/2278163559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54312f0 0x562fe580e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:22:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.909+0000 7f0f8be00640 1 --2- 172.21.15.27:0/2278163559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe545a1b0 0x562fe580eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:22:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.909+0000 7f0f8b400640 1 --2- 172.21.15.27:0/2278163559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe5418d80 0x562fe5813080 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009530 tx=0x7f0f7c01e270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f160bf8447fbf388 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:22:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.909+0000 7f0f74c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:22:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.909+0000 7f0f74c00640 1 -- 172.21.15.27:0/2278163559 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00e3b0 con 0x562fe5418d80 2024-09-21T00:22:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.909+0000 7f0f74c00640 1 -- 172.21.15.27:0/2278163559 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f0f7c01c5b0 con 0x562fe5418d80 2024-09-21T00:22:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.909+0000 7f0f9924d740 1 -- 172.21.15.27:0/2278163559 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f840d2c30 con 0x562fe5418d80 2024-09-21T00:22:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.910+0000 7f0f74c00640 1 -- 172.21.15.27:0/2278163559 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f7c01c7b0 con 0x562fe5418d80 2024-09-21T00:22:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.910+0000 7f0f9924d740 10 client.25652 fetch_fsmap learned FSMap version 26 2024-09-21T00:22:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.910+0000 7f0f9924d740 1 -- 172.21.15.27:0/2278163559 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562fe588f8c0 con 0x562fe5418d80 2024-09-21T00:22:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.910+0000 7f0f74c00640 1 -- 172.21.15.27:0/2278163559 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f840d2c30 con 0x562fe5418d80 2024-09-21T00:22:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.911+0000 7f0f74c00640 1 -- 172.21.15.27:0/2278163559 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0f7c00e550 con 0x562fe5418d80 2024-09-21T00:22:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.911+0000 7f0f9924d740 10 client.25652 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:22:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.911+0000 7f0f9924d740 10 client.25652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:22:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.911+0000 7f0f9924d740 20 client.25652 populate_metadata read hostname 'smithi027' 2024-09-21T00:22:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.911+0000 7f0f9924d740 1 --2- 172.21.15.27:0/2278163559 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562fe5890090 0x562fe58924b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:22:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.911+0000 7f0f9924d740 4 client.25652 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:22:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.911+0000 7f0f9924d740 1 -- 172.21.15.27:0/2278163559 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562fe57720b0 con 0x562fe5890090 2024-09-21T00:22:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.911+0000 7f0f8aa00640 1 --2- 172.21.15.27:0/2278163559 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562fe5890090 0x562fe58924b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:22:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.912+0000 7f0f74c00640 10 client.25652 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.912+0000 7f0f8aa00640 1 --2- 172.21.15.27:0/2278163559 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562fe5890090 0x562fe58924b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0b73b36b846b34c server_cookie=175e5aee09ce6c6b in_seq=0 out_seq=0 2024-09-21T00:22:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.918+0000 7f0f74c00640 1 -- 172.21.15.27:0/2278163559 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562fe57720b0 con 0x562fe5890090 2024-09-21T00:22:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.918+0000 7f0f74c00640 10 client.25652 handle_command_reply: tid=0 2024-09-21T00:22:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.918+0000 7f0f9924d740 1 client.25652 shutdown 2024-09-21T00:22:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.919+0000 7f0f9924d740 1 -- 172.21.15.27:0/2278163559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe5418d80 msgr2=0x562fe5813080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.919+0000 7f0f9924d740 1 --2- 172.21.15.27:0/2278163559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe5418d80 0x562fe5813080 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009530 tx=0x7f0f7c01e270 comp rx=0 tx=0).stop 2024-09-21T00:22:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.919+0000 7f0f9924d740 1 -- 172.21.15.27:0/2278163559 shutdown_connections 2024-09-21T00:22:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.919+0000 7f0f9924d740 1 --2- 172.21.15.27:0/2278163559 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562fe5890090 0x562fe58924b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.919+0000 7f0f9924d740 1 --2- 172.21.15.27:0/2278163559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fe545a1b0 0x562fe580eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.919+0000 7f0f9924d740 1 --2- 172.21.15.27:0/2278163559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fe54312f0 0x562fe580e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.919+0000 7f0f9924d740 1 --2- 172.21.15.27:0/2278163559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fe5418d80 0x562fe5813080 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.919+0000 7f0f9924d740 1 -- 172.21.15.27:0/2278163559 >> 172.21.15.27:0/2278163559 conn(0x562fe544e300 msgr2=0x562fe57fc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.919+0000 7f0f74c00640 0 client.25652 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:22:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.919+0000 7f0f9924d740 1 -- 172.21.15.27:0/2278163559 shutdown_connections 2024-09-21T00:22:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.919+0000 7f0f9924d740 1 -- 172.21.15.27:0/2278163559 wait complete. 2024-09-21T00:22:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.919+0000 7f0f9924d740 20 client.25652 trim_cache size 0 max 16384 2024-09-21T00:22:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:22:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:22:46.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:22:46.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:22:46.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.920+0000 7f0f90c00640 1 -- 172.21.15.27:0/1546408649 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0f700b6670 msgr2=0x7f0f700b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:46.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.920+0000 7f0f90c00640 1 --2- 172.21.15.27:0/1546408649 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0f700b6670 0x7f0f700b8b80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x562fe57fda90 tx=0x7f0f7c009260 comp rx=0 tx=0).stop 2024-09-21T00:22:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.920+0000 7f0f90c00640 1 -- 172.21.15.27:0/1546408649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f8c10ff00 msgr2=0x7f0f8c1a1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:22:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.920+0000 7f0f90c00640 1 --2- 172.21.15.27:0/1546408649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f8c10ff00 0x7f0f8c1a1c20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009b30 tx=0x7f0f84003c10 comp rx=0 tx=0).stop 2024-09-21T00:22:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.921+0000 7f0f90c00640 1 -- 172.21.15.27:0/1546408649 shutdown_connections 2024-09-21T00:22:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.921+0000 7f0f90c00640 1 --2- 172.21.15.27:0/1546408649 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0f700b6670 0x7f0f700b8b80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.921+0000 7f0f90c00640 1 --2- 172.21.15.27:0/1546408649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f8c10ff00 0x7f0f8c1a1c20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.921+0000 7f0f90c00640 1 --2- 172.21.15.27:0/1546408649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f8c102ca0 0x7f0f8c19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.921+0000 7f0f90c00640 1 --2- 172.21.15.27:0/1546408649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f8c102350 0x7f0f8c19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:22:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.921+0000 7f0f90c00640 1 -- 172.21.15.27:0/1546408649 >> 172.21.15.27:0/1546408649 conn(0x7f0f8c0fe210 msgr2=0x7f0f8c1120a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:22:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.921+0000 7f0f90c00640 1 -- 172.21.15.27:0/1546408649 shutdown_connections 2024-09-21T00:22:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:22:46.921+0000 7f0f90c00640 1 -- 172.21.15.27:0/1546408649 wait complete. 2024-09-21T00:22:46.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:46 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:22:46.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:46 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:22:46.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:46 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:22:46.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:46 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:22:46.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:46 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:22:46.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:46 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:22:48.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:48 smithi089 ceph-mon[34481]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-21T00:22:48.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:48 smithi098 ceph-mon[34725]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-21T00:22:48.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:48 smithi027 ceph-mon[30745]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-21T00:22:50.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:50 smithi089 ceph-mon[34481]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:22:50.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:50 smithi098 ceph-mon[34725]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:22:50.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:50 smithi027 ceph-mon[30745]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T00:22:52.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:52 smithi089 ceph-mon[34481]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:22:52.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:52 smithi098 ceph-mon[34725]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:22:52.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:52 smithi027 ceph-mon[30745]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:22:54.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:54 smithi089 ceph-mon[34481]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:22:54.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:54 smithi098 ceph-mon[34725]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:22:54.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:54 smithi027 ceph-mon[30745]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:22:56.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:56 smithi089 ceph-mon[34481]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T00:22:56.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:56 smithi098 ceph-mon[34725]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T00:22:56.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:56 smithi027 ceph-mon[30745]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T00:22:58.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:22:58 smithi089 ceph-mon[34481]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:22:58.498 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:22:58 smithi098 ceph-mon[34725]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:22:58.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:22:58 smithi027 ceph-mon[30745]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:23:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.138+0000 7f1854600640 1 -- 172.21.15.27:0/3158324629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f184c10b630 msgr2=0x7f184c10ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.138+0000 7f1854600640 1 --2- 172.21.15.27:0/3158324629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f184c10b630 0x7f184c10ba40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1838009900 tx=0x7f183802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.138+0000 7f1854600640 1 -- 172.21.15.27:0/3158324629 shutdown_connections 2024-09-21T00:23:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.138+0000 7f1854600640 1 --2- 172.21.15.27:0/3158324629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c10bf80 0x7f184c1127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.138+0000 7f1854600640 1 --2- 172.21.15.27:0/3158324629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f184c10b630 0x7f184c10ba40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.138+0000 7f1854600640 1 --2- 172.21.15.27:0/3158324629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f184c105d00 0x7f184c106110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.138+0000 7f1854600640 1 -- 172.21.15.27:0/3158324629 >> 172.21.15.27:0/3158324629 conn(0x7f184c101720 msgr2=0x7f184c103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.138+0000 7f1854600640 1 -- 172.21.15.27:0/3158324629 shutdown_connections 2024-09-21T00:23:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.138+0000 7f1854600640 1 -- 172.21.15.27:0/3158324629 wait complete. 2024-09-21T00:23:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.139+0000 7f1854600640 1 Processor -- start 2024-09-21T00:23:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.139+0000 7f1854600640 1 -- start start 2024-09-21T00:23:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1854600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c105d00 0x7f184c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1854600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f184c10b630 0x7f184c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1854600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f184c10bf80 0x7f184c1a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1854600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f184c115c40 con 0x7f184c10b630 2024-09-21T00:23:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1854600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f184c115ac0 con 0x7f184c105d00 2024-09-21T00:23:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1854600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f184c115dc0 con 0x7f184c10bf80 2024-09-21T00:23:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1851c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c105d00 0x7f184c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1851c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c105d00 0x7f184c19c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37096/0 (socket says 172.21.15.27:37096) 2024-09-21T00:23:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1851c00640 1 -- 172.21.15.27:0/1342687989 learned_addr learned my addr 172.21.15.27:0/1342687989 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1851200640 1 --2- 172.21.15.27:0/1342687989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f184c10b630 0x7f184c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1852600640 1 --2- 172.21.15.27:0/1342687989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f184c10bf80 0x7f184c1a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1851c00640 1 -- 172.21.15.27:0/1342687989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f184c10bf80 msgr2=0x7f184c1a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1851c00640 1 --2- 172.21.15.27:0/1342687989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f184c10bf80 0x7f184c1a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1851c00640 1 -- 172.21.15.27:0/1342687989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f184c10b630 msgr2=0x7f184c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1851c00640 1 --2- 172.21.15.27:0/1342687989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f184c10b630 0x7f184c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.140+0000 7f1851c00640 1 -- 172.21.15.27:0/1342687989 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c1a25d0 con 0x7f184c105d00 2024-09-21T00:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.141+0000 7f1851c00640 1 --2- 172.21.15.27:0/1342687989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c105d00 0x7f184c19c6a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f184000e9f0 tx=0x7f184000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.141+0000 7f184a000640 1 -- 172.21.15.27:0/1342687989 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184000ccc0 con 0x7f184c105d00 2024-09-21T00:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.141+0000 7f1854600640 1 -- 172.21.15.27:0/1342687989 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184c1a28c0 con 0x7f184c105d00 2024-09-21T00:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.141+0000 7f184a000640 1 -- 172.21.15.27:0/1342687989 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f184000ce60 con 0x7f184c105d00 2024-09-21T00:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.141+0000 7f184a000640 1 -- 172.21.15.27:0/1342687989 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840010430 con 0x7f184c105d00 2024-09-21T00:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.141+0000 7f1854600640 1 -- 172.21.15.27:0/1342687989 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184c1a9050 con 0x7f184c105d00 2024-09-21T00:23:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.144+0000 7f185b812740 1 Processor -- start 2024-09-21T00:23:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.144+0000 7f185b812740 1 -- start start 2024-09-21T00:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.145+0000 7f185b812740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a40750a1b0 0x55a4078aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.145+0000 7f185b812740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4074e12f0 0x55a4078af170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.145+0000 7f185b812740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e94d0 0x55a4078af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.145+0000 7f185b812740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a4078246b0 con 0x55a40750a1b0 2024-09-21T00:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.145+0000 7f185b812740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a407813ae0 con 0x55a4074e12f0 2024-09-21T00:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.145+0000 7f185b812740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a4078178f0 con 0x55a4074e94d0 2024-09-21T00:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.145+0000 7f1852600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a40750a1b0 0x55a4078aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.145+0000 7f1851200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e94d0 0x55a4078af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.145+0000 7f1851200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e94d0 0x55a4078af6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39618/0 (socket says 172.21.15.27:39618) 2024-09-21T00:23:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.145+0000 7f1852600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a40750a1b0 0x55a4078aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60628/0 (socket says 172.21.15.27:60628) 2024-09-21T00:23:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.145+0000 7f1851200640 1 -- 172.21.15.27:0/2590466387 learned_addr learned my addr 172.21.15.27:0/2590466387 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.145+0000 7f184a000640 1 -- 172.21.15.27:0/1342687989 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1840010610 con 0x7f184c105d00 2024-09-21T00:23:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.146+0000 7f1851200640 1 -- 172.21.15.27:0/2590466387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4074e12f0 msgr2=0x55a4078af170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:23:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.146+0000 7f1851200640 1 --2- 172.21.15.27:0/2590466387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4074e12f0 0x55a4078af170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.146+0000 7f1851200640 1 -- 172.21.15.27:0/2590466387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a40750a1b0 msgr2=0x55a4078aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.146+0000 7f1851200640 1 --2- 172.21.15.27:0/2590466387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a40750a1b0 0x55a4078aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.146+0000 7f1851200640 1 -- 172.21.15.27:0/2590466387 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4077e0780 con 0x55a4074e94d0 2024-09-21T00:23:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.146+0000 7f1851200640 1 --2- 172.21.15.27:0/2590466387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e94d0 0x55a4078af6b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1838033000 tx=0x7f1838000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=664172d70cb0e848 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.147+0000 7f183ea00640 1 -- 172.21.15.27:0/2590466387 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838035040 con 0x55a4074e94d0 2024-09-21T00:23:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.147+0000 7f183ea00640 1 -- 172.21.15.27:0/2590466387 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1838004030 con 0x55a4074e94d0 2024-09-21T00:23:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.147+0000 7f183ea00640 1 -- 172.21.15.27:0/2590466387 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18380053d0 con 0x55a4074e94d0 2024-09-21T00:23:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.147+0000 7f184a000640 1 --2- 172.21.15.27:0/1342687989 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f18340b6680 0x7f18340b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.147+0000 7f185b812740 1 -- 172.21.15.27:0/2590466387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e94d0 msgr2=0x55a4078af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.147+0000 7f185b812740 1 --2- 172.21.15.27:0/2590466387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e94d0 0x55a4078af6b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1838033000 tx=0x7f1838000bc0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.148+0000 7f184a000640 1 -- 172.21.15.27:0/1342687989 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f18400e7f30 con 0x7f184c105d00 2024-09-21T00:23:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.148+0000 7f185b812740 1 -- 172.21.15.27:0/2590466387 shutdown_connections 2024-09-21T00:23:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.148+0000 7f185b812740 1 --2- 172.21.15.27:0/2590466387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e94d0 0x55a4078af6b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.148+0000 7f185b812740 1 --2- 172.21.15.27:0/2590466387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4074e12f0 0x55a4078af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.148+0000 7f185b812740 1 --2- 172.21.15.27:0/2590466387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a40750a1b0 0x55a4078aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.148+0000 7f185b812740 1 -- 172.21.15.27:0/2590466387 >> 172.21.15.27:0/2590466387 conn(0x55a4074fe300 msgr2=0x55a4078acb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.148+0000 7f1852600640 1 --2- 172.21.15.27:0/1342687989 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f18340b6680 0x7f18340b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.148+0000 7f185b812740 1 -- 172.21.15.27:0/2590466387 shutdown_connections 2024-09-21T00:23:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.148+0000 7f185b812740 1 -- 172.21.15.27:0/2590466387 wait complete. 2024-09-21T00:23:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.149+0000 7f185b812740 1 -- start start 2024-09-21T00:23:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.149+0000 7f1852600640 1 --2- 172.21.15.27:0/1342687989 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f18340b6680 0x7f18340b8b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1844005fd0 tx=0x7f1844009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.149+0000 7f185b812740 1 Processor -- start 2024-09-21T00:23:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.149+0000 7f185b812740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e12f0 0x55a4078ba260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.149+0000 7f185b812740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4074e94d0 0x55a4078ba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.149+0000 7f185b812740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a40750a1b0 0x55a4078bad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.149+0000 7f185b812740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a40774bab0 con 0x55a40750a1b0 2024-09-21T00:23:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.149+0000 7f185b812740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a4077ba570 con 0x55a4074e94d0 2024-09-21T00:23:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.149+0000 7f185b812740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a4078bd350 con 0x55a4074e12f0 2024-09-21T00:23:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.150+0000 7f185b812740 10 client.0 _ll_register_callbacks cb 0x55a4077c9100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:23:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.150+0000 7f1851c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e12f0 0x55a4078ba260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.150+0000 7f1851c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e12f0 0x55a4078ba260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39634/0 (socket says 172.21.15.27:39634) 2024-09-21T00:23:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.150+0000 7f1851c00640 1 -- 172.21.15.27:0/3319019220 learned_addr learned my addr 172.21.15.27:0/3319019220 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.150+0000 7f1851200640 1 --2- 172.21.15.27:0/3319019220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4074e94d0 0x55a4078ba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.150+0000 7f1852600640 1 --2- 172.21.15.27:0/3319019220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a40750a1b0 0x55a4078bad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.150+0000 7f1851c00640 1 -- 172.21.15.27:0/3319019220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4074e94d0 msgr2=0x55a4078ba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.150+0000 7f1851c00640 1 --2- 172.21.15.27:0/3319019220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4074e94d0 0x55a4078ba7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.150+0000 7f1851c00640 1 -- 172.21.15.27:0/3319019220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a40750a1b0 msgr2=0x55a4078bad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.150+0000 7f1851c00640 1 --2- 172.21.15.27:0/3319019220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a40750a1b0 0x55a4078bad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.150+0000 7f1851c00640 1 -- 172.21.15.27:0/3319019220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a407575790 con 0x55a4074e12f0 2024-09-21T00:23:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.150+0000 7f1851200640 1 --2- 172.21.15.27:0/3319019220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4074e94d0 0x55a4078ba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:23:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.150+0000 7f1851c00640 1 --2- 172.21.15.27:0/3319019220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e12f0 0x55a4078ba260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f184000c6a0 tx=0x7f18400ad3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55db610f94165ba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.150+0000 7f1833400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:23:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.151+0000 7f1833400640 1 -- 172.21.15.27:0/3319019220 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18400b2070 con 0x55a4074e12f0 2024-09-21T00:23:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.151+0000 7f1833400640 1 -- 172.21.15.27:0/3319019220 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f18400ace00 con 0x55a4074e12f0 2024-09-21T00:23:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.151+0000 7f185b812740 1 -- 172.21.15.27:0/3319019220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1838004030 con 0x55a4074e12f0 2024-09-21T00:23:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.151+0000 7f1833400640 1 -- 172.21.15.27:0/3319019220 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f18400ad4a0 con 0x55a4074e12f0 2024-09-21T00:23:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.151+0000 7f185b812740 10 client.25639 fetch_fsmap learned FSMap version 26 2024-09-21T00:23:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.151+0000 7f185b812740 1 -- 172.21.15.27:0/3319019220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a4075ee620 con 0x55a4074e12f0 2024-09-21T00:23:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.151+0000 7f1833400640 1 -- 172.21.15.27:0/3319019220 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1838004030 con 0x55a4074e12f0 2024-09-21T00:23:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.151+0000 7f1833400640 1 -- 172.21.15.27:0/3319019220 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f18400a9400 con 0x55a4074e12f0 2024-09-21T00:23:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.151+0000 7f185b812740 10 client.25639 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:23:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.151+0000 7f185b812740 10 client.25639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:23:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.151+0000 7f185b812740 20 client.25639 populate_metadata read hostname 'smithi027' 2024-09-21T00:23:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.151+0000 7f185b812740 1 --2- 172.21.15.27:0/3319019220 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55a4074c8d80 0x55a4078cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.151+0000 7f185b812740 4 client.25639 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:23:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.151+0000 7f185b812740 1 -- 172.21.15.27:0/3319019220 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a407826600 con 0x55a4074c8d80 2024-09-21T00:23:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.152+0000 7f1851200640 1 --2- 172.21.15.27:0/3319019220 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55a4074c8d80 0x55a4078cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.153+0000 7f1851200640 1 --2- 172.21.15.27:0/3319019220 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55a4074c8d80 0x55a4078cc380 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e71c99d27105152 server_cookie=76391ece674854b4 in_seq=0 out_seq=0 2024-09-21T00:23:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.153+0000 7f1833400640 10 client.25639 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.153+0000 7f1833400640 1 -- 172.21.15.27:0/3319019220 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a407826600 con 0x55a4074c8d80 2024-09-21T00:23:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.153+0000 7f1833400640 10 client.25639 handle_command_reply: tid=0 2024-09-21T00:23:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.154+0000 7f185b812740 1 client.25639 shutdown 2024-09-21T00:23:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.154+0000 7f185b812740 1 -- 172.21.15.27:0/3319019220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e12f0 msgr2=0x55a4078ba260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.154+0000 7f185b812740 1 --2- 172.21.15.27:0/3319019220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e12f0 0x55a4078ba260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f184000c6a0 tx=0x7f18400ad3a0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.154+0000 7f185b812740 1 -- 172.21.15.27:0/3319019220 shutdown_connections 2024-09-21T00:23:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.154+0000 7f185b812740 1 --2- 172.21.15.27:0/3319019220 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55a4074c8d80 0x55a4078cc380 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.154+0000 7f185b812740 1 --2- 172.21.15.27:0/3319019220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a40750a1b0 0x55a4078bad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.154+0000 7f185b812740 1 --2- 172.21.15.27:0/3319019220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4074e94d0 0x55a4078ba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.154+0000 7f185b812740 1 --2- 172.21.15.27:0/3319019220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e12f0 0x55a4078ba260 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.154+0000 7f1833400640 0 client.25639 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.154+0000 7f185b812740 1 -- 172.21.15.27:0/3319019220 >> 172.21.15.27:0/3319019220 conn(0x55a4074fe300 msgr2=0x55a4078afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.154+0000 7f185b812740 1 -- 172.21.15.27:0/3319019220 shutdown_connections 2024-09-21T00:23:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.154+0000 7f185b812740 1 -- 172.21.15.27:0/3319019220 wait complete. 2024-09-21T00:23:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.154+0000 7f185b812740 20 client.25639 trim_cache size 0 max 16384 2024-09-21T00:23:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.184+0000 7f185b812740 1 Processor -- start 2024-09-21T00:23:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.184+0000 7f185b812740 1 -- start start 2024-09-21T00:23:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.184+0000 7f185b812740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4074e12f0 0x55a4078abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.184+0000 7f185b812740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e94d0 0x55a4078c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.184+0000 7f185b812740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a40750a1b0 0x55a4078d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.184+0000 7f185b812740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f184c1b5850 con 0x55a4074e12f0 2024-09-21T00:23:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.184+0000 7f185b812740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f184c1b3120 con 0x55a40750a1b0 2024-09-21T00:23:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.184+0000 7f185b812740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f184c1b0f70 con 0x55a4074e94d0 2024-09-21T00:23:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.185+0000 7f1851c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4074e12f0 0x55a4078abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.185+0000 7f1851c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4074e12f0 0x55a4078abaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60650/0 (socket says 172.21.15.27:60650) 2024-09-21T00:23:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.185+0000 7f1851c00640 1 -- 172.21.15.27:0/21308868 learned_addr learned my addr 172.21.15.27:0/21308868 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.185+0000 7f1852600640 1 --2- 172.21.15.27:0/21308868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a40750a1b0 0x55a4078d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.185+0000 7f1851200640 1 --2- 172.21.15.27:0/21308868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e94d0 0x55a4078c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.185+0000 7f1851c00640 1 -- 172.21.15.27:0/21308868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e94d0 msgr2=0x55a4078c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.185+0000 7f1851c00640 1 --2- 172.21.15.27:0/21308868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e94d0 0x55a4078c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.185+0000 7f1851c00640 1 -- 172.21.15.27:0/21308868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a40750a1b0 msgr2=0x55a4078d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.185+0000 7f1851c00640 1 --2- 172.21.15.27:0/21308868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a40750a1b0 0x55a4078d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.185+0000 7f1851c00640 1 -- 172.21.15.27:0/21308868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4078cc6f0 con 0x55a4074e12f0 2024-09-21T00:23:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.185+0000 7f1851c00640 1 --2- 172.21.15.27:0/21308868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4074e12f0 0x55a4078abaa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f18400aac00 tx=0x7f18400ad410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674252ed499cf1ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.186+0000 7f183e000640 1 -- 172.21.15.27:0/21308868 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18400b2040 con 0x55a4074e12f0 2024-09-21T00:23:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.186+0000 7f183e000640 1 -- 172.21.15.27:0/21308868 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18400ae090 con 0x55a4074e12f0 2024-09-21T00:23:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.186+0000 7f183e000640 1 -- 172.21.15.27:0/21308868 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18400b73b0 con 0x55a4074e12f0 2024-09-21T00:23:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.187+0000 7f185b812740 1 -- 172.21.15.27:0/21308868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4074e12f0 msgr2=0x55a4078abaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.187+0000 7f185b812740 1 --2- 172.21.15.27:0/21308868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4074e12f0 0x55a4078abaa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f18400aac00 tx=0x7f18400ad410 comp rx=0 tx=0).stop 2024-09-21T00:23:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.187+0000 7f185b812740 1 -- 172.21.15.27:0/21308868 shutdown_connections 2024-09-21T00:23:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.187+0000 7f185b812740 1 --2- 172.21.15.27:0/21308868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a40750a1b0 0x55a4078d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.187+0000 7f185b812740 1 --2- 172.21.15.27:0/21308868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4074e94d0 0x55a4078c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.187+0000 7f185b812740 1 --2- 172.21.15.27:0/21308868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4074e12f0 0x55a4078abaa0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.187+0000 7f185b812740 1 -- 172.21.15.27:0/21308868 >> 172.21.15.27:0/21308868 conn(0x55a4074fe300 msgr2=0x55a4078c5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.187+0000 7f185b812740 1 -- 172.21.15.27:0/21308868 shutdown_connections 2024-09-21T00:23:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.187+0000 7f185b812740 1 -- 172.21.15.27:0/21308868 wait complete. 2024-09-21T00:23:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.187+0000 7f185b812740 1 -- start start 2024-09-21T00:23:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.188+0000 7f185b812740 1 Processor -- start 2024-09-21T00:23:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.188+0000 7f185b812740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4074e12f0 0x55a4078c0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.188+0000 7f185b812740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4074e94d0 0x55a4078c0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.188+0000 7f185b812740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a40750a1b0 0x55a4078c0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.188+0000 7f185b812740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f184c1b15e0 con 0x55a4074e94d0 2024-09-21T00:23:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.188+0000 7f185b812740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f184c1b2a50 con 0x55a4074e12f0 2024-09-21T00:23:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.188+0000 7f185b812740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f184c1b1cb0 con 0x55a40750a1b0 2024-09-21T00:23:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.188+0000 7f185b812740 10 client.0 _ll_register_callbacks cb 0x7f18400048f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:23:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.188+0000 7f1851200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4074e94d0 0x55a4078c0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.189+0000 7f1851200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4074e94d0 0x55a4078c0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60662/0 (socket says 172.21.15.27:60662) 2024-09-21T00:23:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.189+0000 7f1851200640 1 -- 172.21.15.27:0/2824254782 learned_addr learned my addr 172.21.15.27:0/2824254782 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.189+0000 7f1852600640 1 --2- 172.21.15.27:0/2824254782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a40750a1b0 0x55a4078c0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.189+0000 7f1851c00640 1 --2- 172.21.15.27:0/2824254782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4074e12f0 0x55a4078c0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.189+0000 7f1851200640 1 -- 172.21.15.27:0/2824254782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a40750a1b0 msgr2=0x55a4078c0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.189+0000 7f1851200640 1 --2- 172.21.15.27:0/2824254782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a40750a1b0 0x55a4078c0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.189+0000 7f1851200640 1 -- 172.21.15.27:0/2824254782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4074e12f0 msgr2=0x55a4078c0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.189+0000 7f1851200640 1 --2- 172.21.15.27:0/2824254782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4074e12f0 0x55a4078c0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.189+0000 7f1851200640 1 -- 172.21.15.27:0/2824254782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a407935ce0 con 0x55a4074e94d0 2024-09-21T00:23:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.189+0000 7f1851200640 1 --2- 172.21.15.27:0/2824254782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4074e94d0 0x55a4078c0a40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1838009900 tx=0x7f183802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05276f60ab8f6c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.189+0000 7f1833400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:23:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.190+0000 7f1833400640 1 -- 172.21.15.27:0/2824254782 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838059020 con 0x55a4074e94d0 2024-09-21T00:23:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.190+0000 7f1833400640 1 -- 172.21.15.27:0/2824254782 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f18380052c0 con 0x55a4074e94d0 2024-09-21T00:23:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.190+0000 7f185b812740 1 -- 172.21.15.27:0/2824254782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18400ae090 con 0x55a4074e94d0 2024-09-21T00:23:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.190+0000 7f1833400640 1 -- 172.21.15.27:0/2824254782 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1838064050 con 0x55a4074e94d0 2024-09-21T00:23:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.190+0000 7f185b812740 10 client.15636 fetch_fsmap learned FSMap version 26 2024-09-21T00:23:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.190+0000 7f185b812740 1 -- 172.21.15.27:0/2824254782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a407936eb0 con 0x55a4074e94d0 2024-09-21T00:23:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.190+0000 7f1833400640 1 -- 172.21.15.27:0/2824254782 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18400ae090 con 0x55a4074e94d0 2024-09-21T00:23:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.190+0000 7f1833400640 1 -- 172.21.15.27:0/2824254782 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f183804d030 con 0x55a4074e94d0 2024-09-21T00:23:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.191+0000 7f185b812740 10 client.15636 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:23:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.191+0000 7f185b812740 10 client.15636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:23:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.191+0000 7f185b812740 20 client.15636 populate_metadata read hostname 'smithi027' 2024-09-21T00:23:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.191+0000 7f185b812740 1 --2- 172.21.15.27:0/2824254782 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55a40749e1c0 0x55a4079392b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.191+0000 7f185b812740 4 client.15636 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:23:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.191+0000 7f185b812740 1 -- 172.21.15.27:0/2824254782 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a4078220b0 con 0x55a40749e1c0 2024-09-21T00:23:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.191+0000 7f1851c00640 1 --2- 172.21.15.27:0/2824254782 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55a40749e1c0 0x55a4079392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.192+0000 7f1851c00640 1 --2- 172.21.15.27:0/2824254782 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55a40749e1c0 0x55a4079392b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60a9c7d7d258b630 server_cookie=97db0790fa31ca39 in_seq=0 out_seq=0 2024-09-21T00:23:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.192+0000 7f1833400640 10 client.15636 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.197+0000 7f1833400640 1 -- 172.21.15.27:0/2824254782 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a4078220b0 con 0x55a40749e1c0 2024-09-21T00:23:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.197+0000 7f1833400640 10 client.15636 handle_command_reply: tid=0 2024-09-21T00:23:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.197+0000 7f185b812740 1 client.15636 shutdown 2024-09-21T00:23:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.198+0000 7f185b812740 1 -- 172.21.15.27:0/2824254782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4074e94d0 msgr2=0x55a4078c0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.198+0000 7f185b812740 1 --2- 172.21.15.27:0/2824254782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4074e94d0 0x55a4078c0a40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1838009900 tx=0x7f183802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.198+0000 7f185b812740 1 -- 172.21.15.27:0/2824254782 shutdown_connections 2024-09-21T00:23:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.198+0000 7f185b812740 1 --2- 172.21.15.27:0/2824254782 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55a40749e1c0 0x55a4079392b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.198+0000 7f185b812740 1 --2- 172.21.15.27:0/2824254782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a40750a1b0 0x55a4078c0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.198+0000 7f185b812740 1 --2- 172.21.15.27:0/2824254782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4074e94d0 0x55a4078c0a40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.198+0000 7f185b812740 1 --2- 172.21.15.27:0/2824254782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4074e12f0 0x55a4078c0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.198+0000 7f185b812740 1 -- 172.21.15.27:0/2824254782 >> 172.21.15.27:0/2824254782 conn(0x55a4074fe300 msgr2=0x55a4078ab870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.198+0000 7f1833400640 0 client.15636 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.198+0000 7f185b812740 1 -- 172.21.15.27:0/2824254782 shutdown_connections 2024-09-21T00:23:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.198+0000 7f185b812740 1 -- 172.21.15.27:0/2824254782 wait complete. 2024-09-21T00:23:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.198+0000 7f185b812740 20 client.15636 trim_cache size 0 max 16384 2024-09-21T00:23:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:23:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:23:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:23:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:23:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.199+0000 7f1854600640 1 -- 172.21.15.27:0/1342687989 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f18340b6680 msgr2=0x7f18340b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.199+0000 7f1854600640 1 --2- 172.21.15.27:0/1342687989 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f18340b6680 0x7f18340b8b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1844005fd0 tx=0x7f1844009500 comp rx=0 tx=0).stop 2024-09-21T00:23:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.199+0000 7f1854600640 1 -- 172.21.15.27:0/1342687989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c105d00 msgr2=0x7f184c19c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.199+0000 7f1854600640 1 --2- 172.21.15.27:0/1342687989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c105d00 0x7f184c19c6a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f184000e9f0 tx=0x7f184000edf0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.199+0000 7f1854600640 1 -- 172.21.15.27:0/1342687989 shutdown_connections 2024-09-21T00:23:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.199+0000 7f1854600640 1 --2- 172.21.15.27:0/1342687989 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f18340b6680 0x7f18340b8b90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.199+0000 7f1854600640 1 --2- 172.21.15.27:0/1342687989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f184c10bf80 0x7f184c1a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.199+0000 7f1854600640 1 --2- 172.21.15.27:0/1342687989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f184c10b630 0x7f184c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.199+0000 7f1854600640 1 --2- 172.21.15.27:0/1342687989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c105d00 0x7f184c19c6a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.200+0000 7f1854600640 1 -- 172.21.15.27:0/1342687989 >> 172.21.15.27:0/1342687989 conn(0x7f184c101720 msgr2=0x7f184c103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.200+0000 7f1854600640 1 -- 172.21.15.27:0/1342687989 shutdown_connections 2024-09-21T00:23:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:00.200+0000 7f1854600640 1 -- 172.21.15.27:0/1342687989 wait complete. 2024-09-21T00:23:00.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:00 smithi089 ceph-mon[34481]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:23:00.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:00 smithi098 ceph-mon[34725]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:23:00.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:00 smithi027 ceph-mon[30745]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:23:02.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:02 smithi027 ceph-mon[30745]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:23:02.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:02 smithi089 ceph-mon[34481]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:23:02.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:02 smithi098 ceph-mon[34725]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:23:04.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:04 smithi089 ceph-mon[34481]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-09-21T00:23:04.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:04 smithi098 ceph-mon[34725]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-09-21T00:23:04.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:04 smithi027 ceph-mon[30745]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-09-21T00:23:05.349 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.0 2024-09-21T00:23:05.349 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:23:05.762 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:23:06.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.460+0000 7fad88600640 1 -- 172.21.15.27:0/1822545768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad80075750 msgr2=0x7fad80075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:06.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.460+0000 7fad88600640 1 --2- 172.21.15.27:0/1822545768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad80075750 0x7fad80075b60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fad74009900 tx=0x7fad7402a410 comp rx=0 tx=0).stop 2024-09-21T00:23:06.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.461+0000 7fad88600640 1 -- 172.21.15.27:0/1822545768 shutdown_connections 2024-09-21T00:23:06.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.461+0000 7fad88600640 1 --2- 172.21.15.27:0/1822545768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad80110120 0x7fad80112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:06.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.461+0000 7fad88600640 1 --2- 172.21.15.27:0/1822545768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad800760a0 0x7fad8010fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:06.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.461+0000 7fad88600640 1 --2- 172.21.15.27:0/1822545768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad80075750 0x7fad80075b60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:06.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.461+0000 7fad88600640 1 -- 172.21.15.27:0/1822545768 >> 172.21.15.27:0/1822545768 conn(0x7fad8007b850 msgr2=0x7fad8007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:06.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.461+0000 7fad88600640 1 -- 172.21.15.27:0/1822545768 shutdown_connections 2024-09-21T00:23:06.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.461+0000 7fad88600640 1 -- 172.21.15.27:0/1822545768 wait complete. 2024-09-21T00:23:06.464 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:06 smithi027 ceph-mon[30745]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-21T00:23:06.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.462+0000 7fad88600640 1 Processor -- start 2024-09-21T00:23:06.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.462+0000 7fad88600640 1 -- start start 2024-09-21T00:23:06.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.463+0000 7fad88600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad80075750 0x7fad8019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:06.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.463+0000 7fad88600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad800760a0 0x7fad8019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:06.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.463+0000 7fad88600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad80110120 0x7fad801a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:06.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.463+0000 7fad88600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fad80115c90 con 0x7fad80075750 2024-09-21T00:23:06.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.463+0000 7fad88600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fad80115b10 con 0x7fad800760a0 2024-09-21T00:23:06.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.463+0000 7fad88600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fad80115e10 con 0x7fad80110120 2024-09-21T00:23:06.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.463+0000 7fad85c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad80075750 0x7fad8019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.463+0000 7fad85200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad800760a0 0x7fad8019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.463+0000 7fad85c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad80075750 0x7fad8019c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46560/0 (socket says 172.21.15.27:46560) 2024-09-21T00:23:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.463+0000 7fad85200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad800760a0 0x7fad8019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37398/0 (socket says 172.21.15.27:37398) 2024-09-21T00:23:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.463+0000 7fad85c00640 1 -- 172.21.15.27:0/1955830281 learned_addr learned my addr 172.21.15.27:0/1955830281 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.463+0000 7fad86600640 1 --2- 172.21.15.27:0/1955830281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad80110120 0x7fad801a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.464+0000 7fad85200640 1 -- 172.21.15.27:0/1955830281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad80110120 msgr2=0x7fad801a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:06.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.464+0000 7fad85200640 1 --2- 172.21.15.27:0/1955830281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad80110120 0x7fad801a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:06.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.464+0000 7fad85200640 1 -- 172.21.15.27:0/1955830281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad80075750 msgr2=0x7fad8019c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:06.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.464+0000 7fad85200640 1 --2- 172.21.15.27:0/1955830281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad80075750 0x7fad8019c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:06.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.464+0000 7fad85200640 1 -- 172.21.15.27:0/1955830281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad801a25f0 con 0x7fad800760a0 2024-09-21T00:23:06.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.464+0000 7fad85200640 1 --2- 172.21.15.27:0/1955830281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad800760a0 0x7fad8019cc20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fad800777d0 tx=0x7fad6c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:06.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.464+0000 7fad7e000640 1 -- 172.21.15.27:0/1955830281 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00cc40 con 0x7fad800760a0 2024-09-21T00:23:06.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.464+0000 7fad7e000640 1 -- 172.21.15.27:0/1955830281 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad6c00cde0 con 0x7fad800760a0 2024-09-21T00:23:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.464+0000 7fad88600640 1 -- 172.21.15.27:0/1955830281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad801a28e0 con 0x7fad800760a0 2024-09-21T00:23:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.464+0000 7fad7e000640 1 -- 172.21.15.27:0/1955830281 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c004a00 con 0x7fad800760a0 2024-09-21T00:23:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.464+0000 7fad88600640 1 -- 172.21.15.27:0/1955830281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad801a9040 con 0x7fad800760a0 2024-09-21T00:23:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.466+0000 7fad88600640 1 -- 172.21.15.27:0/1955830281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad80075b60 con 0x7fad800760a0 2024-09-21T00:23:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.468+0000 7fad7e000640 1 -- 172.21.15.27:0/1955830281 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fad6c004ba0 con 0x7fad800760a0 2024-09-21T00:23:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.470+0000 7fad7e000640 1 --2- 172.21.15.27:0/1955830281 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fad680b6700 0x7fad680b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.470+0000 7fad7e000640 1 -- 172.21.15.27:0/1955830281 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fad6c0e79d0 con 0x7fad800760a0 2024-09-21T00:23:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.470+0000 7fad85c00640 1 --2- 172.21.15.27:0/1955830281 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fad680b6700 0x7fad680b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:06.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.471+0000 7fad85c00640 1 --2- 172.21.15.27:0/1955830281 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fad680b6700 0x7fad680b8c10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fad74031440 tx=0x7fad740058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:06.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.474+0000 7fad7e000640 1 -- 172.21.15.27:0/1955830281 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fad6c0adb30 con 0x7fad800760a0 2024-09-21T00:23:06.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:06 smithi089 ceph-mon[34481]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-21T00:23:06.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:06 smithi098 ceph-mon[34725]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-21T00:23:06.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.715+0000 7fad88600640 1 -- 172.21.15.27:0/1955830281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad801a9300 con 0x7fad800760a0 2024-09-21T00:23:06.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.716+0000 7fad7e000640 1 -- 172.21.15.27:0/1955830281 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11893 (secure 0 0 0) 0x7fad6c0b1f10 con 0x7fad800760a0 2024-09-21T00:23:06.718 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:23:06.718 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":26,"btime":"2024-09-21T00:21:36:085073+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:21:35.264444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:23:06.718 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-09-21T00:23:06.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.719+0000 7fad88600640 1 -- 172.21.15.27:0/1955830281 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fad680b6700 msgr2=0x7fad680b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:06.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.719+0000 7fad88600640 1 --2- 172.21.15.27:0/1955830281 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fad680b6700 0x7fad680b8c10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fad74031440 tx=0x7fad740058d0 comp rx=0 tx=0).stop 2024-09-21T00:23:06.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.719+0000 7fad88600640 1 -- 172.21.15.27:0/1955830281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad800760a0 msgr2=0x7fad8019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:06.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.719+0000 7fad88600640 1 --2- 172.21.15.27:0/1955830281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad800760a0 0x7fad8019cc20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fad800777d0 tx=0x7fad6c00ee50 comp rx=0 tx=0).stop 2024-09-21T00:23:06.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.719+0000 7fad88600640 1 -- 172.21.15.27:0/1955830281 shutdown_connections 2024-09-21T00:23:06.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.719+0000 7fad88600640 1 --2- 172.21.15.27:0/1955830281 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fad680b6700 0x7fad680b8c10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:06.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.719+0000 7fad88600640 1 --2- 172.21.15.27:0/1955830281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad80110120 0x7fad801a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:06.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.719+0000 7fad88600640 1 --2- 172.21.15.27:0/1955830281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad800760a0 0x7fad8019cc20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:06.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.719+0000 7fad88600640 1 --2- 172.21.15.27:0/1955830281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad80075750 0x7fad8019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:06.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.719+0000 7fad88600640 1 -- 172.21.15.27:0/1955830281 >> 172.21.15.27:0/1955830281 conn(0x7fad8007b850 msgr2=0x7fad80101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:06.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.720+0000 7fad88600640 1 -- 172.21.15.27:0/1955830281 shutdown_connections 2024-09-21T00:23:06.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:06.720+0000 7fad88600640 1 -- 172.21.15.27:0/1955830281 wait complete. 2024-09-21T00:23:07.431 INFO:tasks.workunit.client.0.smithi027.stderr:Updating files: 56% (7115/12697) Updating files: 57% (7238/12697) Updating files: 58% (7365/12697) Updating files: 59% (7492/12697) Updating files: 60% (7619/12697) Updating files: 61% (7746/12697) Updating files: 62% (7873/12697) Updating files: 63% (8000/12697) Updating files: 64% (8127/12697) Updating files: 65% (8254/12697) Updating files: 66% (8381/12697) Updating files: 67% (8507/12697) Updating files: 68% (8634/12697) Updating files: 69% (8761/12697) Updating files: 70% (8888/12697) Updating files: 71% (9015/12697) Updating files: 72% (9142/12697) Updating files: 73% (9269/12697) Updating files: 74% (9396/12697) Updating files: 75% (9523/12697) Updating files: 76% (9650/12697) Updating files: 77% (9777/12697) Updating files: 78% (9904/12697) Updating files: 79% (10031/12697) Updating files: 80% (10158/12697) Updating files: 81% (10285/12697) Updating files: 82% (10412/12697) Updating files: 83% (10539/12697) Updating files: 84% (10666/12697) Updating files: 85% (10793/12697) Updating files: 86% (10920/12697) Updating files: 87% (11047/12697) Updating files: 88% (11174/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 91% (11655/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T00:23:07.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:07 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1955830281' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:23:07.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:07 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1955830281' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:23:07.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:07 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1955830281' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:23:07.616 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0' 2024-09-21T00:23:07.616 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0 2024-09-21T00:23:08.010 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:23:08.391 INFO:tasks.workunit.client.0.smithi027.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T00:23:08.391 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T00:23:08.391 INFO:tasks.workunit.client.0.smithi027.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T00:23:08.391 INFO:tasks.workunit.client.0.smithi027.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T00:23:08.391 INFO:tasks.workunit.client.0.smithi027.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T00:23:08.391 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T00:23:08.391 INFO:tasks.workunit.client.0.smithi027.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T00:23:08.391 INFO:tasks.workunit.client.0.smithi027.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T00:23:08.391 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T00:23:08.392 INFO:tasks.workunit.client.0.smithi027.stderr: git switch -c 2024-09-21T00:23:08.392 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T00:23:08.392 INFO:tasks.workunit.client.0.smithi027.stderr:Or undo this operation with: 2024-09-21T00:23:08.392 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T00:23:08.392 INFO:tasks.workunit.client.0.smithi027.stderr: git switch - 2024-09-21T00:23:08.392 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T00:23:08.392 INFO:tasks.workunit.client.0.smithi027.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T00:23:08.392 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T00:23:08.392 INFO:tasks.workunit.client.0.smithi027.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T00:23:08.404 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:08 smithi027 ceph-mon[30745]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-21T00:23:08.406 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T00:23:08.465 INFO:tasks.workunit.client.0.smithi027.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T00:23:08.470 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T00:23:08.470 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T00:23:08.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:08 smithi089 ceph-mon[34481]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-21T00:23:08.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:08 smithi098 ceph-mon[34725]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-21T00:23:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.756+0000 7f9951a00640 1 -- 172.21.15.27:0/2159437294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f994c078290 msgr2=0x7f994c114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.756+0000 7f9951a00640 1 --2- 172.21.15.27:0/2159437294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f994c078290 0x7f994c114200 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9938009860 tx=0x7f993802a340 comp rx=0 tx=0).stop 2024-09-21T00:23:08.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.756+0000 7f9951a00640 1 -- 172.21.15.27:0/2159437294 shutdown_connections 2024-09-21T00:23:08.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.756+0000 7f9951a00640 1 --2- 172.21.15.27:0/2159437294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c114740 0x7f994c116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.756+0000 7f9951a00640 1 --2- 172.21.15.27:0/2159437294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f994c078290 0x7f994c114200 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.756+0000 7f9951a00640 1 --2- 172.21.15.27:0/2159437294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f994c077940 0x7f994c077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.756+0000 7f9951a00640 1 -- 172.21.15.27:0/2159437294 >> 172.21.15.27:0/2159437294 conn(0x7f994c1017d0 msgr2=0x7f994c103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:08.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.756+0000 7f9951a00640 1 -- 172.21.15.27:0/2159437294 shutdown_connections 2024-09-21T00:23:08.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.756+0000 7f9951a00640 1 -- 172.21.15.27:0/2159437294 wait complete. 2024-09-21T00:23:08.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.757+0000 7f9951a00640 1 Processor -- start 2024-09-21T00:23:08.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.757+0000 7f9951a00640 1 -- start start 2024-09-21T00:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.758+0000 7f9951a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f994c077940 0x7f994c1a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.758+0000 7f9951a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f994c078290 0x7f994c1a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.758+0000 7f9951a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c114740 0x7f994c1a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.758+0000 7f9951a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f994c11a080 con 0x7f994c114740 2024-09-21T00:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.758+0000 7f9951a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f994c119f00 con 0x7f994c077940 2024-09-21T00:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.758+0000 7f9951a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f994c11a200 con 0x7f994c078290 2024-09-21T00:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.758+0000 7f994aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f994c077940 0x7f994c1a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.758+0000 7f994aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f994c077940 0x7f994c1a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37422/0 (socket says 172.21.15.27:37422) 2024-09-21T00:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.758+0000 7f994aa00640 1 -- 172.21.15.27:0/1630388406 learned_addr learned my addr 172.21.15.27:0/1630388406 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.758+0000 7f994a000640 1 --2- 172.21.15.27:0/1630388406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f994c078290 0x7f994c1a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.758+0000 7f994b400640 1 --2- 172.21.15.27:0/1630388406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c114740 0x7f994c1a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.758+0000 7f994aa00640 1 -- 172.21.15.27:0/1630388406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f994c078290 msgr2=0x7f994c1a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.758+0000 7f994aa00640 1 --2- 172.21.15.27:0/1630388406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f994c078290 0x7f994c1a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.758+0000 7f994aa00640 1 -- 172.21.15.27:0/1630388406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c114740 msgr2=0x7f994c1a6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:08.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.758+0000 7f994aa00640 1 --2- 172.21.15.27:0/1630388406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c114740 0x7f994c1a6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.759+0000 7f994aa00640 1 -- 172.21.15.27:0/1630388406 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c1a6a40 con 0x7f994c077940 2024-09-21T00:23:08.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.759+0000 7f994aa00640 1 --2- 172.21.15.27:0/1630388406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f994c077940 0x7f994c1a0ad0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f993c00e9f0 tx=0x7f993c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:08.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.759+0000 7f9943400640 1 -- 172.21.15.27:0/1630388406 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c00cd80 con 0x7f994c077940 2024-09-21T00:23:08.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.759+0000 7f9943400640 1 -- 172.21.15.27:0/1630388406 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f993c0054d0 con 0x7f994c077940 2024-09-21T00:23:08.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.759+0000 7f9951a00640 1 -- 172.21.15.27:0/1630388406 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994c1a6d30 con 0x7f994c077940 2024-09-21T00:23:08.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.759+0000 7f9943400640 1 -- 172.21.15.27:0/1630388406 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c010690 con 0x7f994c077940 2024-09-21T00:23:08.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.759+0000 7f9951a00640 1 -- 172.21.15.27:0/1630388406 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994c1ad490 con 0x7f994c077940 2024-09-21T00:23:08.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.763+0000 7f99587e2740 1 Processor -- start 2024-09-21T00:23:08.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.763+0000 7f99587e2740 1 -- start start 2024-09-21T00:23:08.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.763+0000 7f99587e2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 0x5584cd296350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.763+0000 7f99587e2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 0x5584cd296890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.763+0000 7f9943400640 1 -- 172.21.15.27:0/1630388406 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f993c004e90 con 0x7f994c077940 2024-09-21T00:23:08.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.763+0000 7f99587e2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd296dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.763+0000 7f99587e2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5584cd20bca0 con 0x5584cceee840 2024-09-21T00:23:08.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.763+0000 7f99587e2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5584cd1fb0d0 con 0x5584ccec5980 2024-09-21T00:23:08.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.763+0000 7f99587e2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5584cd1feee0 con 0x5584ccecdb60 2024-09-21T00:23:08.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.764+0000 7f994b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 0x5584cd296350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.764+0000 7f994b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 0x5584cd296350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46566/0 (socket says 172.21.15.27:46566) 2024-09-21T00:23:08.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.764+0000 7f994b400640 1 -- 172.21.15.27:0/3912155586 learned_addr learned my addr 172.21.15.27:0/3912155586 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:08.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.764+0000 7f994aa00640 1 --2- 172.21.15.27:0/3912155586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 0x5584cd296890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.764+0000 7f994a000640 1 --2- 172.21.15.27:0/3912155586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd296dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.764+0000 7f994b400640 1 -- 172.21.15.27:0/3912155586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 msgr2=0x5584cd296dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:08.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.764+0000 7f994b400640 1 --2- 172.21.15.27:0/3912155586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd296dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.764+0000 7f994b400640 1 -- 172.21.15.27:0/3912155586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 msgr2=0x5584cd296890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:08.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.764+0000 7f994b400640 1 --2- 172.21.15.27:0/3912155586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 0x5584cd296890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.764+0000 7f994b400640 1 -- 172.21.15.27:0/3912155586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584cd1ad400 con 0x5584cceee840 2024-09-21T00:23:08.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.765+0000 7f994b400640 1 --2- 172.21.15.27:0/3912155586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 0x5584cd296350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9944009f60 tx=0x7f9944009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8697bdfc8e53fb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:08.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.765+0000 7f9937e00640 1 -- 172.21.15.27:0/3912155586 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994401d070 con 0x5584cceee840 2024-09-21T00:23:08.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.765+0000 7f9937e00640 1 -- 172.21.15.27:0/3912155586 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f994400daa0 con 0x5584cceee840 2024-09-21T00:23:08.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.765+0000 7f9937e00640 1 -- 172.21.15.27:0/3912155586 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944018af0 con 0x5584cceee840 2024-09-21T00:23:08.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.765+0000 7f9943400640 1 --2- 172.21.15.27:0/1630388406 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f99300b6700 0x7f99300b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.766+0000 7f9943400640 1 -- 172.21.15.27:0/1630388406 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f993c0e7f30 con 0x7f994c077940 2024-09-21T00:23:08.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.766+0000 7f994aa00640 1 --2- 172.21.15.27:0/1630388406 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f99300b6700 0x7f99300b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.766+0000 7f99587e2740 1 -- 172.21.15.27:0/3912155586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 msgr2=0x5584cd296350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:08.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.766+0000 7f99587e2740 1 --2- 172.21.15.27:0/3912155586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 0x5584cd296350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9944009f60 tx=0x7f9944009f90 comp rx=0 tx=0).stop 2024-09-21T00:23:08.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.766+0000 7f99587e2740 1 -- 172.21.15.27:0/3912155586 shutdown_connections 2024-09-21T00:23:08.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.766+0000 7f99587e2740 1 --2- 172.21.15.27:0/3912155586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd296dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.766+0000 7f99587e2740 1 --2- 172.21.15.27:0/3912155586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 0x5584cd296890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.766+0000 7f99587e2740 1 --2- 172.21.15.27:0/3912155586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 0x5584cd296350 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.766+0000 7f99587e2740 1 -- 172.21.15.27:0/3912155586 >> 172.21.15.27:0/3912155586 conn(0x5584ccee2990 msgr2=0x5584cd294240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:08.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.766+0000 7f994aa00640 1 --2- 172.21.15.27:0/1630388406 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f99300b6700 0x7f99300b8c10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x5584cd294fd0 tx=0x7f993c0eb9e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:08.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.766+0000 7f99587e2740 1 -- 172.21.15.27:0/3912155586 shutdown_connections 2024-09-21T00:23:08.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.767+0000 7f99587e2740 1 -- 172.21.15.27:0/3912155586 wait complete. 2024-09-21T00:23:08.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.767+0000 7f99587e2740 1 -- start start 2024-09-21T00:23:08.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.767+0000 7f99587e2740 1 Processor -- start 2024-09-21T00:23:08.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.768+0000 7f99587e2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 0x5584cd2a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.768+0000 7f99587e2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd2a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.768+0000 7f99587e2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 0x5584cd2ade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.768+0000 7f99587e2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5584cd05c7a0 con 0x5584cceee840 2024-09-21T00:23:08.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.768+0000 7f99587e2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5584cd1a0a10 con 0x5584ccec5980 2024-09-21T00:23:08.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.768+0000 7f99587e2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5584cd2a4b30 con 0x5584ccecdb60 2024-09-21T00:23:08.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.768+0000 7f99587e2740 10 client.0 _ll_register_callbacks cb 0x5584cd1b0be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:23:08.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.768+0000 7f994a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd2a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.768+0000 7f994a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd2a1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:35420/0 (socket says 172.21.15.27:35420) 2024-09-21T00:23:08.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.768+0000 7f994a000640 1 -- 172.21.15.27:0/2081695832 learned_addr learned my addr 172.21.15.27:0/2081695832 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:08.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.768+0000 7f994b400640 1 --2- 172.21.15.27:0/2081695832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 0x5584cd2a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.768+0000 7f994b400640 1 --2- 172.21.15.27:0/2081695832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 0x5584cd2ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.769+0000 7f994a000640 1 -- 172.21.15.27:0/2081695832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 msgr2=0x5584cd2a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:08.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.769+0000 7f994a000640 1 --2- 172.21.15.27:0/2081695832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 0x5584cd2a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.769+0000 7f994a000640 1 -- 172.21.15.27:0/2081695832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 msgr2=0x5584cd2ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:08.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.769+0000 7f994a000640 1 --2- 172.21.15.27:0/2081695832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 0x5584cd2ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.769+0000 7f994a000640 1 -- 172.21.15.27:0/2081695832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584ccf68b90 con 0x5584ccecdb60 2024-09-21T00:23:08.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.769+0000 7f994b400640 1 --2- 172.21.15.27:0/2081695832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 0x5584cd2a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:23:08.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.769+0000 7f994a000640 1 --2- 172.21.15.27:0/2081695832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd2a1fb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9938009860 tx=0x7f9938005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5318b551aa282cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:08.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.769+0000 7f9934c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:23:08.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.770+0000 7f9934c00640 1 -- 172.21.15.27:0/2081695832 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938042070 con 0x5584ccecdb60 2024-09-21T00:23:08.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.770+0000 7f99587e2740 1 -- 172.21.15.27:0/2081695832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f994400daa0 con 0x5584ccecdb60 2024-09-21T00:23:08.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.770+0000 7f9934c00640 1 -- 172.21.15.27:0/2081695832 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f9938004240 con 0x5584ccecdb60 2024-09-21T00:23:08.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.770+0000 7f9934c00640 1 -- 172.21.15.27:0/2081695832 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f993804e050 con 0x5584ccecdb60 2024-09-21T00:23:08.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.770+0000 7f99587e2740 10 client.25663 fetch_fsmap learned FSMap version 26 2024-09-21T00:23:08.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.770+0000 7f99587e2740 1 -- 172.21.15.27:0/2081695832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5584cd2b08c0 con 0x5584ccecdb60 2024-09-21T00:23:08.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.770+0000 7f9934c00640 1 -- 172.21.15.27:0/2081695832 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f994400daa0 con 0x5584ccecdb60 2024-09-21T00:23:08.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.771+0000 7f9934c00640 1 -- 172.21.15.27:0/2081695832 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9938003b10 con 0x5584ccecdb60 2024-09-21T00:23:08.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.771+0000 7f99587e2740 10 client.25663 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:23:08.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.771+0000 7f99587e2740 10 client.25663 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:23:08.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.771+0000 7f99587e2740 20 client.25663 populate_metadata read hostname 'smithi027' 2024-09-21T00:23:08.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.771+0000 7f99587e2740 1 --2- 172.21.15.27:0/2081695832 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5584ccead1c0 0x5584cd2b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.771+0000 7f99587e2740 4 client.25663 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:23:08.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.771+0000 7f99587e2740 1 -- 172.21.15.27:0/2081695832 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584cd20dbf0 con 0x5584ccead1c0 2024-09-21T00:23:08.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.771+0000 7f994b400640 1 --2- 172.21.15.27:0/2081695832 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5584ccead1c0 0x5584cd2b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.772+0000 7f994b400640 1 --2- 172.21.15.27:0/2081695832 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5584ccead1c0 0x5584cd2b3d80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63416c5a2b5d0e12 server_cookie=2fa6e0c42b41f8d5 in_seq=0 out_seq=0 2024-09-21T00:23:08.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.772+0000 7f9934c00640 10 client.25663 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:08.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.773+0000 7f9934c00640 1 -- 172.21.15.27:0/2081695832 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5584cd20dbf0 con 0x5584ccead1c0 2024-09-21T00:23:08.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.773+0000 7f9934c00640 10 client.25663 handle_command_reply: tid=0 2024-09-21T00:23:08.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.773+0000 7f99587e2740 1 client.25663 shutdown 2024-09-21T00:23:08.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.773+0000 7f99587e2740 1 -- 172.21.15.27:0/2081695832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 msgr2=0x5584cd2a1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:08.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.773+0000 7f99587e2740 1 --2- 172.21.15.27:0/2081695832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd2a1fb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9938009860 tx=0x7f9938005e70 comp rx=0 tx=0).stop 2024-09-21T00:23:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.773+0000 7f99587e2740 1 -- 172.21.15.27:0/2081695832 shutdown_connections 2024-09-21T00:23:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.773+0000 7f99587e2740 1 --2- 172.21.15.27:0/2081695832 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5584ccead1c0 0x5584cd2b3d80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.773+0000 7f99587e2740 1 --2- 172.21.15.27:0/2081695832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 0x5584cd2ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.773+0000 7f99587e2740 1 --2- 172.21.15.27:0/2081695832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd2a1fb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.773+0000 7f99587e2740 1 --2- 172.21.15.27:0/2081695832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 0x5584cd2a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.773+0000 7f99587e2740 1 -- 172.21.15.27:0/2081695832 >> 172.21.15.27:0/2081695832 conn(0x5584ccee2990 msgr2=0x5584cd299530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.773+0000 7f9934c00640 0 client.25663 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.773+0000 7f99587e2740 1 -- 172.21.15.27:0/2081695832 shutdown_connections 2024-09-21T00:23:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.773+0000 7f99587e2740 1 -- 172.21.15.27:0/2081695832 wait complete. 2024-09-21T00:23:08.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.774+0000 7f99587e2740 20 client.25663 trim_cache size 0 max 16384 2024-09-21T00:23:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.792+0000 7f99587e2740 1 Processor -- start 2024-09-21T00:23:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.792+0000 7f99587e2740 1 -- start start 2024-09-21T00:23:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.792+0000 7f99587e2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 0x5584cd2a3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.792+0000 7f99587e2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd2add50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.792+0000 7f99587e2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 0x5584cd2930b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.792+0000 7f99587e2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b9c90 con 0x5584cceee840 2024-09-21T00:23:08.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.792+0000 7f99587e2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b7560 con 0x5584ccec5980 2024-09-21T00:23:08.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.792+0000 7f99587e2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b53b0 con 0x5584ccecdb60 2024-09-21T00:23:08.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.793+0000 7f994b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 0x5584cd2a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.793+0000 7f994b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 0x5584cd2a3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37436/0 (socket says 172.21.15.27:37436) 2024-09-21T00:23:08.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.793+0000 7f994b400640 1 -- 172.21.15.27:0/583993691 learned_addr learned my addr 172.21.15.27:0/583993691 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:08.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.793+0000 7f994a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd2add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.793+0000 7f994b400640 1 --2- 172.21.15.27:0/583993691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 0x5584cd2930b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.793+0000 7f994a000640 1 -- 172.21.15.27:0/583993691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 msgr2=0x5584cd2a3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:08.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.793+0000 7f994a000640 1 --2- 172.21.15.27:0/583993691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 0x5584cd2a3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.793+0000 7f994a000640 1 -- 172.21.15.27:0/583993691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 msgr2=0x5584cd2930b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:08.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.793+0000 7f994a000640 1 --2- 172.21.15.27:0/583993691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 0x5584cd2930b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.793+0000 7f994a000640 1 -- 172.21.15.27:0/583993691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584cd28fc50 con 0x5584ccecdb60 2024-09-21T00:23:08.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.793+0000 7f994a000640 1 --2- 172.21.15.27:0/583993691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd2add50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f993802a860 tx=0x7f9938030fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c812f1398981da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.794+0000 7f9937400640 1 -- 172.21.15.27:0/583993691 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938042040 con 0x5584ccecdb60 2024-09-21T00:23:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.794+0000 7f9937400640 1 -- 172.21.15.27:0/583993691 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9938030900 con 0x5584ccecdb60 2024-09-21T00:23:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.794+0000 7f9937400640 1 -- 172.21.15.27:0/583993691 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938037040 con 0x5584ccecdb60 2024-09-21T00:23:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.795+0000 7f99587e2740 1 -- 172.21.15.27:0/583993691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 msgr2=0x5584cd2add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.795+0000 7f99587e2740 1 --2- 172.21.15.27:0/583993691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd2add50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f993802a860 tx=0x7f9938030fc0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.795+0000 7f99587e2740 1 -- 172.21.15.27:0/583993691 shutdown_connections 2024-09-21T00:23:08.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.795+0000 7f99587e2740 1 --2- 172.21.15.27:0/583993691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584cceee840 0x5584cd2930b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.795+0000 7f99587e2740 1 --2- 172.21.15.27:0/583993691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd2add50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.795+0000 7f99587e2740 1 --2- 172.21.15.27:0/583993691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584ccec5980 0x5584cd2a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.795+0000 7f99587e2740 1 -- 172.21.15.27:0/583993691 >> 172.21.15.27:0/583993691 conn(0x5584ccee2990 msgr2=0x5584cd2929a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:08.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.795+0000 7f99587e2740 1 -- 172.21.15.27:0/583993691 shutdown_connections 2024-09-21T00:23:08.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.795+0000 7f99587e2740 1 -- 172.21.15.27:0/583993691 wait complete. 2024-09-21T00:23:08.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.795+0000 7f99587e2740 1 -- start start 2024-09-21T00:23:08.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.796+0000 7f99587e2740 1 Processor -- start 2024-09-21T00:23:08.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.796+0000 7f99587e2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584ccec5980 0x5584cd2aa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.796+0000 7f99587e2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd2a6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.796+0000 7f99587e2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584cceee840 0x5584cd2a6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.796+0000 7f99587e2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b5a20 con 0x5584ccec5980 2024-09-21T00:23:08.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.796+0000 7f99587e2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b6e90 con 0x5584cceee840 2024-09-21T00:23:08.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.796+0000 7f99587e2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b60f0 con 0x5584ccecdb60 2024-09-21T00:23:08.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.796+0000 7f99587e2740 10 client.0 _ll_register_callbacks cb 0x7f993c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:23:08.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.797+0000 7f994b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584ccec5980 0x5584cd2aa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.797+0000 7f994b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584ccec5980 0x5584cd2aa400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46596/0 (socket says 172.21.15.27:46596) 2024-09-21T00:23:08.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.797+0000 7f994b400640 1 -- 172.21.15.27:0/3203763117 learned_addr learned my addr 172.21.15.27:0/3203763117 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:08.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.797+0000 7f994a000640 1 --2- 172.21.15.27:0/3203763117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd2a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.797+0000 7f994b400640 1 --2- 172.21.15.27:0/3203763117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584cceee840 0x5584cd2a6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.797+0000 7f994b400640 1 -- 172.21.15.27:0/3203763117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 msgr2=0x5584cd2a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:08.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.797+0000 7f994b400640 1 --2- 172.21.15.27:0/3203763117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd2a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.797+0000 7f994b400640 1 -- 172.21.15.27:0/3203763117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584cceee840 msgr2=0x5584cd2a6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:08.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.797+0000 7f994b400640 1 --2- 172.21.15.27:0/3203763117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584cceee840 0x5584cd2a6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.797+0000 7f994b400640 1 -- 172.21.15.27:0/3203763117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584cd2a7440 con 0x5584ccec5980 2024-09-21T00:23:08.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.797+0000 7f994b400640 1 --2- 172.21.15.27:0/3203763117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584ccec5980 0x5584cd2aa400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9944018740 tx=0x7f9944009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37abab96f0a4f89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:08.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.797+0000 7f9934c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:23:08.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.798+0000 7f9934c00640 1 -- 172.21.15.27:0/3203763117 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944039060 con 0x5584ccec5980 2024-09-21T00:23:08.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.798+0000 7f9934c00640 1 -- 172.21.15.27:0/3203763117 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f9944004cd0 con 0x5584ccec5980 2024-09-21T00:23:08.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.798+0000 7f99587e2740 1 -- 172.21.15.27:0/3203763117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9938030900 con 0x5584ccec5980 2024-09-21T00:23:08.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.798+0000 7f9934c00640 1 -- 172.21.15.27:0/3203763117 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f994401d040 con 0x5584ccec5980 2024-09-21T00:23:08.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.798+0000 7f99587e2740 10 client.15660 fetch_fsmap learned FSMap version 26 2024-09-21T00:23:08.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.798+0000 7f99587e2740 1 -- 172.21.15.27:0/3203763117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5584cd31f4d0 con 0x5584ccec5980 2024-09-21T00:23:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.799+0000 7f9934c00640 1 -- 172.21.15.27:0/3203763117 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9938030900 con 0x5584ccec5980 2024-09-21T00:23:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.799+0000 7f9934c00640 1 -- 172.21.15.27:0/3203763117 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f994400b040 con 0x5584ccec5980 2024-09-21T00:23:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.799+0000 7f99587e2740 10 client.15660 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:23:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.799+0000 7f99587e2740 10 client.15660 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:23:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.799+0000 7f99587e2740 20 client.15660 populate_metadata read hostname 'smithi027' 2024-09-21T00:23:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.799+0000 7f99587e2740 1 --2- 172.21.15.27:0/3203763117 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5584cce82570 0x5584cd321970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.799+0000 7f99587e2740 4 client.15660 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}] 2024-09-21T00:23:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.799+0000 7f99587e2740 1 -- 172.21.15.27:0/3203763117 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}) -- 0x5584cd2096a0 con 0x5584cce82570 2024-09-21T00:23:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.799+0000 7f994a000640 1 --2- 172.21.15.27:0/3203763117 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5584cce82570 0x5584cd321970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.800+0000 7f994a000640 1 --2- 172.21.15.27:0/3203763117 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5584cce82570 0x5584cd321970 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b85e98abb396a21 server_cookie=b44d6c79bce68f67 in_seq=0 out_seq=0 2024-09-21T00:23:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.800+0000 7f9934c00640 10 client.15660 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:08.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.806+0000 7f9934c00640 1 -- 172.21.15.27:0/3203763117 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5584cd2096a0 con 0x5584cce82570 2024-09-21T00:23:08.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.806+0000 7f9934c00640 10 client.15660 handle_command_reply: tid=0 2024-09-21T00:23:08.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.806+0000 7f99587e2740 1 client.15660 shutdown 2024-09-21T00:23:08.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.807+0000 7f99587e2740 1 -- 172.21.15.27:0/3203763117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584ccec5980 msgr2=0x5584cd2aa400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:08.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.807+0000 7f99587e2740 1 --2- 172.21.15.27:0/3203763117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584ccec5980 0x5584cd2aa400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9944018740 tx=0x7f9944009f90 comp rx=0 tx=0).stop 2024-09-21T00:23:08.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.807+0000 7f99587e2740 1 -- 172.21.15.27:0/3203763117 shutdown_connections 2024-09-21T00:23:08.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.807+0000 7f99587e2740 1 --2- 172.21.15.27:0/3203763117 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5584cce82570 0x5584cd321970 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.807+0000 7f99587e2740 1 --2- 172.21.15.27:0/3203763117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584cceee840 0x5584cd2a6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.807+0000 7f99587e2740 1 --2- 172.21.15.27:0/3203763117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584ccecdb60 0x5584cd2a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.807+0000 7f99587e2740 1 --2- 172.21.15.27:0/3203763117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5584ccec5980 0x5584cd2aa400 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.807+0000 7f99587e2740 1 -- 172.21.15.27:0/3203763117 >> 172.21.15.27:0/3203763117 conn(0x5584ccee2990 msgr2=0x5584cd2929a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:08.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.807+0000 7f9934c00640 0 client.15660 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:08.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.807+0000 7f99587e2740 1 -- 172.21.15.27:0/3203763117 shutdown_connections 2024-09-21T00:23:08.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.807+0000 7f99587e2740 1 -- 172.21.15.27:0/3203763117 wait complete. 2024-09-21T00:23:08.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.807+0000 7f99587e2740 20 client.15660 trim_cache size 0 max 16384 2024-09-21T00:23:08.809 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:23:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 26, 2024-09-21T00:23:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24761, 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 6, 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "72a16756": { 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 6, 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 137.0, 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:23:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T00:23:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:23:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:23:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:23:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:23:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:23:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:23:08.811 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:23:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.808+0000 7f9951a00640 1 -- 172.21.15.27:0/1630388406 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f99300b6700 msgr2=0x7f99300b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.808+0000 7f9951a00640 1 --2- 172.21.15.27:0/1630388406 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f99300b6700 0x7f99300b8c10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x5584cd294fd0 tx=0x7f993c0eb9e0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.808+0000 7f9951a00640 1 -- 172.21.15.27:0/1630388406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f994c077940 msgr2=0x7f994c1a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.808+0000 7f9951a00640 1 --2- 172.21.15.27:0/1630388406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f994c077940 0x7f994c1a0ad0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f993c00e9f0 tx=0x7f993c00edf0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.808+0000 7f9951a00640 1 -- 172.21.15.27:0/1630388406 shutdown_connections 2024-09-21T00:23:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.808+0000 7f9951a00640 1 --2- 172.21.15.27:0/1630388406 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f99300b6700 0x7f99300b8c10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.808+0000 7f9951a00640 1 --2- 172.21.15.27:0/1630388406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c114740 0x7f994c1a6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.808+0000 7f9951a00640 1 --2- 172.21.15.27:0/1630388406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f994c078290 0x7f994c1a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.808+0000 7f9951a00640 1 --2- 172.21.15.27:0/1630388406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f994c077940 0x7f994c1a0ad0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.808+0000 7f9951a00640 1 -- 172.21.15.27:0/1630388406 >> 172.21.15.27:0/1630388406 conn(0x7f994c1017d0 msgr2=0x7f994c103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.808+0000 7f9951a00640 1 -- 172.21.15.27:0/1630388406 shutdown_connections 2024-09-21T00:23:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:08.808+0000 7f9951a00640 1 -- 172.21.15.27:0/1630388406 wait complete. 2024-09-21T00:23:09.835 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '72a16756', quiesce took 4.5 seconds. Will release after: 12.5 2024-09-21T00:23:10.039 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T00:23:10.131 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T00:23:10.199 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T00:23:10.203 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T00:23:10.204 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T00:23:10.271 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T00:23:10.278 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:23:10.278 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T00:23:10.315 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-09-21T00:23:10.357 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-09-21T00:23:10.357 DEBUG:teuthology.orchestra.run.smithi027:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-09-21T00:23:10.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:10 smithi089 ceph-mon[34481]: pgmap v430: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-21T00:23:10.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:10 smithi098 ceph-mon[34725]: pgmap v430: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-21T00:23:10.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:10 smithi027 ceph-mon[30745]: pgmap v430: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-21T00:23:12.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:12 smithi089 ceph-mon[34481]: pgmap v431: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-21T00:23:12.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:12 smithi098 ceph-mon[34725]: pgmap v431: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-21T00:23:12.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:12 smithi027 ceph-mon[30745]: pgmap v431: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-21T00:23:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.405+0000 7fa3a5a00640 1 -- 172.21.15.27:0/1448851305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3a0109250 msgr2=0x7fa3a0109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.405+0000 7fa3a5a00640 1 --2- 172.21.15.27:0/1448851305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3a0109250 0x7fa3a0109660 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa388009900 tx=0x7fa38802a410 comp rx=0 tx=0).stop 2024-09-21T00:23:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.405+0000 7fa3a5a00640 1 -- 172.21.15.27:0/1448851305 shutdown_connections 2024-09-21T00:23:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.405+0000 7fa3a5a00640 1 --2- 172.21.15.27:0/1448851305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3a0109ba0 0x7fa3a01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.405+0000 7fa3a5a00640 1 --2- 172.21.15.27:0/1448851305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3a0109250 0x7fa3a0109660 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.405+0000 7fa3a5a00640 1 --2- 172.21.15.27:0/1448851305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a0103920 0x7fa3a0103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.405+0000 7fa3a5a00640 1 -- 172.21.15.27:0/1448851305 >> 172.21.15.27:0/1448851305 conn(0x7fa3a00ff320 msgr2=0x7fa3a0101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.405+0000 7fa3a5a00640 1 -- 172.21.15.27:0/1448851305 shutdown_connections 2024-09-21T00:23:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.405+0000 7fa3a5a00640 1 -- 172.21.15.27:0/1448851305 wait complete. 2024-09-21T00:23:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.406+0000 7fa3a5a00640 1 Processor -- start 2024-09-21T00:23:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.407+0000 7fa3a5a00640 1 -- start start 2024-09-21T00:23:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.407+0000 7fa3a5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3a0103920 0x7fa3a019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.407+0000 7fa3a5a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3a0109250 0x7fa3a019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.407+0000 7fa3a5a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a0109ba0 0x7fa3a01a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.407+0000 7fa3a5a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01138b0 con 0x7fa3a0103920 2024-09-21T00:23:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.407+0000 7fa3a5a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0113730 con 0x7fa3a0109250 2024-09-21T00:23:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.407+0000 7fa3a5a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0113a30 con 0x7fa3a0109ba0 2024-09-21T00:23:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.407+0000 7fa39fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3a0103920 0x7fa3a019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.407+0000 7fa39fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3a0103920 0x7fa3a019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55580/0 (socket says 172.21.15.27:55580) 2024-09-21T00:23:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.407+0000 7fa39fe00640 1 -- 172.21.15.27:0/1274719285 learned_addr learned my addr 172.21.15.27:0/1274719285 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.407+0000 7fa397e00640 1 --2- 172.21.15.27:0/1274719285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3a0109250 0x7fa3a019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.407+0000 7fa3a5000640 1 --2- 172.21.15.27:0/1274719285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a0109ba0 0x7fa3a01a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.408+0000 7fa39fe00640 1 -- 172.21.15.27:0/1274719285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a0109ba0 msgr2=0x7fa3a01a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.408+0000 7fa39fe00640 1 --2- 172.21.15.27:0/1274719285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a0109ba0 0x7fa3a01a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.408+0000 7fa39fe00640 1 -- 172.21.15.27:0/1274719285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3a0109250 msgr2=0x7fa3a019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.408+0000 7fa39fe00640 1 --2- 172.21.15.27:0/1274719285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3a0109250 0x7fa3a019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.408+0000 7fa39fe00640 1 -- 172.21.15.27:0/1274719285 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a01a23e0 con 0x7fa3a0103920 2024-09-21T00:23:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.408+0000 7fa39fe00640 1 --2- 172.21.15.27:0/1274719285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3a0103920 0x7fa3a019c4b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa39000e9f0 tx=0x7fa39000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.408+0000 7fa39d600640 1 -- 172.21.15.27:0/1274719285 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39000cd80 con 0x7fa3a0103920 2024-09-21T00:23:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.409+0000 7fa39d600640 1 -- 172.21.15.27:0/1274719285 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3900054d0 con 0x7fa3a0103920 2024-09-21T00:23:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.409+0000 7fa3a5a00640 1 -- 172.21.15.27:0/1274719285 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a01a26d0 con 0x7fa3a0103920 2024-09-21T00:23:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.409+0000 7fa39d600640 1 -- 172.21.15.27:0/1274719285 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390010690 con 0x7fa3a0103920 2024-09-21T00:23:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.409+0000 7fa3a5a00640 1 -- 172.21.15.27:0/1274719285 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a01a8e60 con 0x7fa3a0103920 2024-09-21T00:23:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.412+0000 7fa3adfcc740 1 Processor -- start 2024-09-21T00:23:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.412+0000 7fa3adfcc740 1 -- start start 2024-09-21T00:23:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.412+0000 7fa3adfcc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643812021b0 0x5643815a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.412+0000 7fa3adfcc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811d92f0 0x5643815a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.412+0000 7fa3adfcc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643811e14d0 0x5643815a7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.412+0000 7fa3adfcc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56438151c6b0 con 0x5643811e14d0 2024-09-21T00:23:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.412+0000 7fa3adfcc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56438150bae0 con 0x5643812021b0 2024-09-21T00:23:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.412+0000 7fa3adfcc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56438150f8f0 con 0x5643811d92f0 2024-09-21T00:23:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.412+0000 7fa39d600640 1 -- 172.21.15.27:0/1274719285 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa390004e90 con 0x7fa3a0103920 2024-09-21T00:23:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.413+0000 7fa397e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643811e14d0 0x5643815a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.413+0000 7fa3a5000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643812021b0 0x5643815a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.413+0000 7fa3a5000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643812021b0 0x5643815a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57472/0 (socket says 172.21.15.27:57472) 2024-09-21T00:23:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.413+0000 7fa3a5000640 1 -- 172.21.15.27:0/3752368031 learned_addr learned my addr 172.21.15.27:0/3752368031 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.413+0000 7fa397e00640 1 --2- 172.21.15.27:0/3752368031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643811e14d0 0x5643815a7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55584/0 (socket says 172.21.15.27:55584) 2024-09-21T00:23:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.413+0000 7fa39fe00640 1 --2- 172.21.15.27:0/3752368031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811d92f0 0x5643815a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.413+0000 7fa3a5000640 1 -- 172.21.15.27:0/3752368031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811d92f0 msgr2=0x5643815a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.413+0000 7fa3a5000640 1 --2- 172.21.15.27:0/3752368031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811d92f0 0x5643815a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.413+0000 7fa3a5000640 1 -- 172.21.15.27:0/3752368031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643811e14d0 msgr2=0x5643815a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.413+0000 7fa3a5000640 1 --2- 172.21.15.27:0/3752368031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643811e14d0 0x5643815a7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.413+0000 7fa3a5000640 1 -- 172.21.15.27:0/3752368031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643814d8780 con 0x5643812021b0 2024-09-21T00:23:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.413+0000 7fa397e00640 1 --2- 172.21.15.27:0/3752368031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643811e14d0 0x5643815a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:23:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.414+0000 7fa3a5000640 1 --2- 172.21.15.27:0/3752368031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643812021b0 0x5643815a6e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa398011440 tx=0x7fa398011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d958642d0eddad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.414+0000 7fa395600640 1 -- 172.21.15.27:0/3752368031 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398004d20 con 0x5643812021b0 2024-09-21T00:23:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.414+0000 7fa395600640 1 -- 172.21.15.27:0/3752368031 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa398004050 con 0x5643812021b0 2024-09-21T00:23:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.414+0000 7fa395600640 1 -- 172.21.15.27:0/3752368031 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39800f740 con 0x5643812021b0 2024-09-21T00:23:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.414+0000 7fa39d600640 1 --2- 172.21.15.27:0/1274719285 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa3840b66f0 0x7fa3840b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.415+0000 7fa39d600640 1 -- 172.21.15.27:0/1274719285 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fa3900e7e80 con 0x7fa3a0103920 2024-09-21T00:23:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.415+0000 7fa39fe00640 1 --2- 172.21.15.27:0/1274719285 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa3840b66f0 0x7fa3840b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.415+0000 7fa3adfcc740 1 -- 172.21.15.27:0/3752368031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643812021b0 msgr2=0x5643815a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.415+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/3752368031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643812021b0 0x5643815a6e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa398011440 tx=0x7fa398011900 comp rx=0 tx=0).stop 2024-09-21T00:23:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.415+0000 7fa3adfcc740 1 -- 172.21.15.27:0/3752368031 shutdown_connections 2024-09-21T00:23:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.415+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/3752368031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643811e14d0 0x5643815a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.415+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/3752368031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811d92f0 0x5643815a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.415+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/3752368031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643812021b0 0x5643815a6e10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.415+0000 7fa3adfcc740 1 -- 172.21.15.27:0/3752368031 >> 172.21.15.27:0/3752368031 conn(0x5643811f6300 msgr2=0x5643815a4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.415+0000 7fa3adfcc740 1 -- 172.21.15.27:0/3752368031 shutdown_connections 2024-09-21T00:23:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.415+0000 7fa3adfcc740 1 -- 172.21.15.27:0/3752368031 wait complete. 2024-09-21T00:23:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.415+0000 7fa39fe00640 1 --2- 172.21.15.27:0/1274719285 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa3840b66f0 0x7fa3840b8c00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x5643815a5a90 tx=0x7fa390004a90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.416+0000 7fa3adfcc740 1 -- start start 2024-09-21T00:23:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.416+0000 7fa3adfcc740 1 Processor -- start 2024-09-21T00:23:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.416+0000 7fa3adfcc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811d92f0 0x5643815b2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.416+0000 7fa3adfcc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811e14d0 0x5643815b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.416+0000 7fa3adfcc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643812021b0 0x5643815be8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.416+0000 7fa3adfcc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564381443ab0 con 0x5643812021b0 2024-09-21T00:23:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.416+0000 7fa3adfcc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5643814b2570 con 0x5643811e14d0 2024-09-21T00:23:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.416+0000 7fa3adfcc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5643815b5590 con 0x5643811d92f0 2024-09-21T00:23:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.417+0000 7fa3adfcc740 10 client.0 _ll_register_callbacks cb 0x5643814c1100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:23:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.417+0000 7fa397e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811e14d0 0x5643815b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.417+0000 7fa397e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811e14d0 0x5643815b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57474/0 (socket says 172.21.15.27:57474) 2024-09-21T00:23:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.417+0000 7fa397e00640 1 -- 172.21.15.27:0/2127798887 learned_addr learned my addr 172.21.15.27:0/2127798887 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.417+0000 7fa3a5000640 1 --2- 172.21.15.27:0/2127798887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811d92f0 0x5643815b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.417+0000 7fa3a5000640 1 --2- 172.21.15.27:0/2127798887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643812021b0 0x5643815be8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.417+0000 7fa397e00640 1 -- 172.21.15.27:0/2127798887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811d92f0 msgr2=0x5643815b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.417+0000 7fa397e00640 1 --2- 172.21.15.27:0/2127798887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811d92f0 0x5643815b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.417+0000 7fa397e00640 1 -- 172.21.15.27:0/2127798887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643812021b0 msgr2=0x5643815be8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.417+0000 7fa397e00640 1 --2- 172.21.15.27:0/2127798887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643812021b0 0x5643815be8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.417+0000 7fa397e00640 1 -- 172.21.15.27:0/2127798887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56438126d790 con 0x5643811e14d0 2024-09-21T00:23:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.417+0000 7fa397e00640 1 --2- 172.21.15.27:0/2127798887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811e14d0 0x5643815b2990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa388000c00 tx=0x7fa3880043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a89572b9b4ead03c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.417+0000 7fa38e000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:23:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.418+0000 7fa38e000640 1 -- 172.21.15.27:0/2127798887 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388035040 con 0x5643811e14d0 2024-09-21T00:23:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.418+0000 7fa3adfcc740 1 -- 172.21.15.27:0/2127798887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa398004050 con 0x5643811e14d0 2024-09-21T00:23:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.418+0000 7fa38e000640 1 -- 172.21.15.27:0/2127798887 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fa388005450 con 0x5643811e14d0 2024-09-21T00:23:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.418+0000 7fa38e000640 1 -- 172.21.15.27:0/2127798887 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa388005650 con 0x5643811e14d0 2024-09-21T00:23:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.418+0000 7fa3adfcc740 10 client.25703 fetch_fsmap learned FSMap version 26 2024-09-21T00:23:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.418+0000 7fa3adfcc740 1 -- 172.21.15.27:0/2127798887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5643812e6620 con 0x5643811e14d0 2024-09-21T00:23:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.419+0000 7fa38e000640 1 -- 172.21.15.27:0/2127798887 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa398004050 con 0x5643811e14d0 2024-09-21T00:23:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.419+0000 7fa38e000640 1 -- 172.21.15.27:0/2127798887 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa388003a00 con 0x5643811e14d0 2024-09-21T00:23:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.419+0000 7fa3adfcc740 10 client.25703 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:23:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.419+0000 7fa3adfcc740 10 client.25703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:23:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.419+0000 7fa3adfcc740 20 client.25703 populate_metadata read hostname 'smithi027' 2024-09-21T00:23:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.419+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/2127798887 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5643811c0d80 0x5643815c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.419+0000 7fa3adfcc740 4 client.25703 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:23:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.419+0000 7fa3adfcc740 1 -- 172.21.15.27:0/2127798887 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56438151e600 con 0x5643811c0d80 2024-09-21T00:23:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.419+0000 7fa3a5000640 1 --2- 172.21.15.27:0/2127798887 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5643811c0d80 0x5643815c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.420+0000 7fa38e000640 10 client.25703 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.420+0000 7fa3a5000640 1 --2- 172.21.15.27:0/2127798887 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5643811c0d80 0x5643815c45f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93190341d1e81982 server_cookie=f0d46a50cfd1f795 in_seq=0 out_seq=0 2024-09-21T00:23:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.421+0000 7fa38e000640 1 -- 172.21.15.27:0/2127798887 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56438151e600 con 0x5643811c0d80 2024-09-21T00:23:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.421+0000 7fa38e000640 10 client.25703 handle_command_reply: tid=0 2024-09-21T00:23:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.421+0000 7fa3adfcc740 1 client.25703 shutdown 2024-09-21T00:23:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.421+0000 7fa3adfcc740 1 -- 172.21.15.27:0/2127798887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811e14d0 msgr2=0x5643815b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.421+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/2127798887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811e14d0 0x5643815b2990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa388000c00 tx=0x7fa3880043b0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.421+0000 7fa3adfcc740 1 -- 172.21.15.27:0/2127798887 shutdown_connections 2024-09-21T00:23:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.421+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/2127798887 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5643811c0d80 0x5643815c45f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.421+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/2127798887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643812021b0 0x5643815be8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.421+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/2127798887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811e14d0 0x5643815b2990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.421+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/2127798887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811d92f0 0x5643815b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.421+0000 7fa38e000640 0 client.25703 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.421+0000 7fa3adfcc740 1 -- 172.21.15.27:0/2127798887 >> 172.21.15.27:0/2127798887 conn(0x5643811f6300 msgr2=0x5643815a7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.421+0000 7fa3adfcc740 1 -- 172.21.15.27:0/2127798887 shutdown_connections 2024-09-21T00:23:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.421+0000 7fa3adfcc740 1 -- 172.21.15.27:0/2127798887 wait complete. 2024-09-21T00:23:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.421+0000 7fa3adfcc740 20 client.25703 trim_cache size 0 max 16384 2024-09-21T00:23:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.437+0000 7fa3adfcc740 1 Processor -- start 2024-09-21T00:23:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.437+0000 7fa3adfcc740 1 -- start start 2024-09-21T00:23:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.437+0000 7fa3adfcc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643811d92f0 0x5643815a8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.437+0000 7fa3adfcc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811e14d0 0x5643815b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.437+0000 7fa3adfcc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643812021b0 0x5643815af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.437+0000 7fa3adfcc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01b57c0 con 0x5643811d92f0 2024-09-21T00:23:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.437+0000 7fa3adfcc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01b3090 con 0x5643811e14d0 2024-09-21T00:23:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.437+0000 7fa3adfcc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01b0ee0 con 0x5643812021b0 2024-09-21T00:23:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.437+0000 7fa3a5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643811d92f0 0x5643815a8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.437+0000 7fa3a5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643811d92f0 0x5643815a8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55612/0 (socket says 172.21.15.27:55612) 2024-09-21T00:23:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.437+0000 7fa3a5000640 1 -- 172.21.15.27:0/2671014969 learned_addr learned my addr 172.21.15.27:0/2671014969 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.437+0000 7fa397e00640 1 --2- 172.21.15.27:0/2671014969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811e14d0 0x5643815b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.437+0000 7fa3a5000640 1 --2- 172.21.15.27:0/2671014969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643812021b0 0x5643815af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.438+0000 7fa3a5000640 1 -- 172.21.15.27:0/2671014969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643812021b0 msgr2=0x5643815af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.438+0000 7fa3a5000640 1 --2- 172.21.15.27:0/2671014969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643812021b0 0x5643815af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.438+0000 7fa3a5000640 1 -- 172.21.15.27:0/2671014969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811e14d0 msgr2=0x5643815b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.438+0000 7fa3a5000640 1 --2- 172.21.15.27:0/2671014969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811e14d0 0x5643815b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.438+0000 7fa3a5000640 1 -- 172.21.15.27:0/2671014969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643815b3710 con 0x5643811d92f0 2024-09-21T00:23:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.438+0000 7fa3a5000640 1 --2- 172.21.15.27:0/2671014969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643811d92f0 0x5643815a8800 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa39801b040 tx=0x7fa398009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d316cfea310040a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.438+0000 7fa394c00640 1 -- 172.21.15.27:0/2671014969 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39801d040 con 0x5643811d92f0 2024-09-21T00:23:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.438+0000 7fa394c00640 1 -- 172.21.15.27:0/2671014969 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa39801e040 con 0x5643811d92f0 2024-09-21T00:23:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.438+0000 7fa394c00640 1 -- 172.21.15.27:0/2671014969 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3980052b0 con 0x5643811d92f0 2024-09-21T00:23:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.439+0000 7fa3adfcc740 1 -- 172.21.15.27:0/2671014969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643811d92f0 msgr2=0x5643815a8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.439+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/2671014969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643811d92f0 0x5643815a8800 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa39801b040 tx=0x7fa398009500 comp rx=0 tx=0).stop 2024-09-21T00:23:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.439+0000 7fa3adfcc740 1 -- 172.21.15.27:0/2671014969 shutdown_connections 2024-09-21T00:23:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.439+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/2671014969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643812021b0 0x5643815af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.439+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/2671014969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811e14d0 0x5643815b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.439+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/2671014969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643811d92f0 0x5643815a8800 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.439+0000 7fa3adfcc740 1 -- 172.21.15.27:0/2671014969 >> 172.21.15.27:0/2671014969 conn(0x5643811f6300 msgr2=0x5643815b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.439+0000 7fa3adfcc740 1 -- 172.21.15.27:0/2671014969 shutdown_connections 2024-09-21T00:23:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.439+0000 7fa3adfcc740 1 -- 172.21.15.27:0/2671014969 wait complete. 2024-09-21T00:23:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.440+0000 7fa3adfcc740 1 -- start start 2024-09-21T00:23:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.440+0000 7fa3adfcc740 1 Processor -- start 2024-09-21T00:23:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.440+0000 7fa3adfcc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811d92f0 0x5643815b8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.440+0000 7fa3adfcc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811e14d0 0x5643815b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.440+0000 7fa3adfcc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643812021b0 0x564381626f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.440+0000 7fa3adfcc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01b1550 con 0x5643812021b0 2024-09-21T00:23:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.440+0000 7fa3adfcc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01b29c0 con 0x5643811d92f0 2024-09-21T00:23:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.440+0000 7fa3adfcc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01b1c20 con 0x5643811e14d0 2024-09-21T00:23:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.440+0000 7fa3adfcc740 10 client.0 _ll_register_callbacks cb 0x7fa388009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:23:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.441+0000 7fa3a5000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811d92f0 0x5643815b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.441+0000 7fa3a5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643812021b0 0x564381626f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.441+0000 7fa3a5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643812021b0 0x564381626f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55628/0 (socket says 172.21.15.27:55628) 2024-09-21T00:23:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.441+0000 7fa3a5000640 1 -- 172.21.15.27:0/1702972653 learned_addr learned my addr 172.21.15.27:0/1702972653 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.441+0000 7fa397e00640 1 --2- 172.21.15.27:0/1702972653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811e14d0 0x5643815b87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.441+0000 7fa3a5000640 1 -- 172.21.15.27:0/1702972653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811e14d0 msgr2=0x5643815b87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.441+0000 7fa3a5000640 1 --2- 172.21.15.27:0/1702972653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811e14d0 0x5643815b87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.441+0000 7fa3a5000640 1 -- 172.21.15.27:0/1702972653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811d92f0 msgr2=0x5643815b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.441+0000 7fa3a5000640 1 --2- 172.21.15.27:0/1702972653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811d92f0 0x5643815b8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.441+0000 7fa3a5000640 1 -- 172.21.15.27:0/1702972653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643816275d0 con 0x5643812021b0 2024-09-21T00:23:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.441+0000 7fa397e00640 1 --2- 172.21.15.27:0/1702972653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811e14d0 0x5643815b87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:23:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.442+0000 7fa3a5000640 1 --2- 172.21.15.27:0/1702972653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643812021b0 0x564381626f60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x5643815b7840 tx=0x7fa398011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa53e608cafaec83 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.442+0000 7fa38e000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:23:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.442+0000 7fa38e000640 1 -- 172.21.15.27:0/1702972653 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39801d070 con 0x5643812021b0 2024-09-21T00:23:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.442+0000 7fa38e000640 1 -- 172.21.15.27:0/1702972653 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fa398004550 con 0x5643812021b0 2024-09-21T00:23:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.442+0000 7fa3adfcc740 1 -- 172.21.15.27:0/1702972653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa39801e040 con 0x5643812021b0 2024-09-21T00:23:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.442+0000 7fa38e000640 1 -- 172.21.15.27:0/1702972653 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa39800f540 con 0x5643812021b0 2024-09-21T00:23:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.442+0000 7fa3adfcc740 10 client.15678 fetch_fsmap learned FSMap version 26 2024-09-21T00:23:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.442+0000 7fa3adfcc740 1 -- 172.21.15.27:0/1702972653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56438162e9d0 con 0x5643812021b0 2024-09-21T00:23:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.442+0000 7fa38e000640 1 -- 172.21.15.27:0/1702972653 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa39801e040 con 0x5643812021b0 2024-09-21T00:23:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.443+0000 7fa38e000640 1 -- 172.21.15.27:0/1702972653 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa39803f790 con 0x5643812021b0 2024-09-21T00:23:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.443+0000 7fa3adfcc740 10 client.15678 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:23:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.443+0000 7fa3adfcc740 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:23:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.443+0000 7fa3adfcc740 20 client.15678 populate_metadata read hostname 'smithi027' 2024-09-21T00:23:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.443+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/1702972653 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5643811961c0 0x564381630ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.443+0000 7fa3adfcc740 4 client.15678 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:23:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.443+0000 7fa3adfcc740 1 -- 172.21.15.27:0/1702972653 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56438151a0b0 con 0x5643811961c0 2024-09-21T00:23:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.443+0000 7fa397e00640 1 --2- 172.21.15.27:0/1702972653 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5643811961c0 0x564381630ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.444+0000 7fa397e00640 1 --2- 172.21.15.27:0/1702972653 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5643811961c0 0x564381630ec0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14930526ae22c908 server_cookie=54ea5fa47ab2f5c in_seq=0 out_seq=0 2024-09-21T00:23:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.444+0000 7fa38e000640 10 client.15678 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.449+0000 7fa38e000640 1 -- 172.21.15.27:0/1702972653 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56438151a0b0 con 0x5643811961c0 2024-09-21T00:23:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.449+0000 7fa38e000640 10 client.15678 handle_command_reply: tid=0 2024-09-21T00:23:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.449+0000 7fa3adfcc740 1 client.15678 shutdown 2024-09-21T00:23:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.450+0000 7fa3adfcc740 1 -- 172.21.15.27:0/1702972653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643812021b0 msgr2=0x564381626f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.450+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/1702972653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643812021b0 0x564381626f60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x5643815b7840 tx=0x7fa398011900 comp rx=0 tx=0).stop 2024-09-21T00:23:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.450+0000 7fa3adfcc740 1 -- 172.21.15.27:0/1702972653 shutdown_connections 2024-09-21T00:23:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.450+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/1702972653 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5643811961c0 0x564381630ec0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.450+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/1702972653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643812021b0 0x564381626f60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.450+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/1702972653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643811e14d0 0x5643815b87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.450+0000 7fa38e000640 0 client.15678 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.450+0000 7fa3adfcc740 1 --2- 172.21.15.27:0/1702972653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643811d92f0 0x5643815b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.450+0000 7fa3adfcc740 1 -- 172.21.15.27:0/1702972653 >> 172.21.15.27:0/1702972653 conn(0x5643811f6300 msgr2=0x5643815a3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.450+0000 7fa3adfcc740 1 -- 172.21.15.27:0/1702972653 shutdown_connections 2024-09-21T00:23:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.450+0000 7fa3adfcc740 1 -- 172.21.15.27:0/1702972653 wait complete. 2024-09-21T00:23:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.450+0000 7fa3adfcc740 20 client.15678 trim_cache size 0 max 16384 2024-09-21T00:23:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:23:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:23:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:23:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:23:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.451+0000 7fa3a5a00640 1 -- 172.21.15.27:0/1274719285 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa3840b66f0 msgr2=0x7fa3840b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.451+0000 7fa3a5a00640 1 --2- 172.21.15.27:0/1274719285 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa3840b66f0 0x7fa3840b8c00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x5643815a5a90 tx=0x7fa390004a90 comp rx=0 tx=0).stop 2024-09-21T00:23:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.451+0000 7fa3a5a00640 1 -- 172.21.15.27:0/1274719285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3a0103920 msgr2=0x7fa3a019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.451+0000 7fa3a5a00640 1 --2- 172.21.15.27:0/1274719285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3a0103920 0x7fa3a019c4b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa39000e9f0 tx=0x7fa39000edf0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.451+0000 7fa3a5a00640 1 -- 172.21.15.27:0/1274719285 shutdown_connections 2024-09-21T00:23:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.451+0000 7fa3a5a00640 1 --2- 172.21.15.27:0/1274719285 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fa3840b66f0 0x7fa3840b8c00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.451+0000 7fa3a5a00640 1 --2- 172.21.15.27:0/1274719285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a0109ba0 0x7fa3a01a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.451+0000 7fa3a5a00640 1 --2- 172.21.15.27:0/1274719285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3a0109250 0x7fa3a019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.451+0000 7fa3a5a00640 1 --2- 172.21.15.27:0/1274719285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3a0103920 0x7fa3a019c4b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.451+0000 7fa3a5a00640 1 -- 172.21.15.27:0/1274719285 >> 172.21.15.27:0/1274719285 conn(0x7fa3a00ff320 msgr2=0x7fa3a0108840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.451+0000 7fa3a5a00640 1 -- 172.21.15.27:0/1274719285 shutdown_connections 2024-09-21T00:23:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:13.451+0000 7fa3a5a00640 1 -- 172.21.15.27:0/1274719285 wait complete. 2024-09-21T00:23:14.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:14 smithi089 ceph-mon[34481]: pgmap v432: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-21T00:23:14.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:14 smithi098 ceph-mon[34725]: pgmap v432: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-21T00:23:14.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:14 smithi027 ceph-mon[30745]: pgmap v432: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-21T00:23:16.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:16 smithi089 ceph-mon[34481]: pgmap v433: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-09-21T00:23:16.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:16 smithi098 ceph-mon[34725]: pgmap v433: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-09-21T00:23:16.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:16 smithi027 ceph-mon[30745]: pgmap v433: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-09-21T00:23:18.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:18 smithi027 ceph-mon[30745]: pgmap v434: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-21T00:23:18.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:18 smithi089 ceph-mon[34481]: pgmap v434: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-21T00:23:18.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:18 smithi098 ceph-mon[34725]: pgmap v434: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-21T00:23:20.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:20 smithi089 ceph-mon[34481]: pgmap v435: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T00:23:20.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:20 smithi098 ceph-mon[34725]: pgmap v435: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T00:23:20.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:20 smithi027 ceph-mon[30745]: pgmap v435: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T00:23:22.335 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:23:22.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:22 smithi089 ceph-mon[34481]: pgmap v436: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T00:23:22.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:22 smithi098 ceph-mon[34725]: pgmap v436: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T00:23:22.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:22 smithi027 ceph-mon[30745]: pgmap v436: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T00:23:22.738 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:23:23.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.394+0000 7fcac5c00640 1 -- 172.21.15.27:0/2755139245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcac0109c40 msgr2=0x7fcac0110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:23.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.394+0000 7fcac5c00640 1 --2- 172.21.15.27:0/2755139245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcac0109c40 0x7fcac0110470 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009ca0 tx=0x7fcab802a8b0 comp rx=0 tx=0).stop 2024-09-21T00:23:23.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.394+0000 7fcac5c00640 1 -- 172.21.15.27:0/2755139245 shutdown_connections 2024-09-21T00:23:23.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.394+0000 7fcac5c00640 1 --2- 172.21.15.27:0/2755139245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcac0109c40 0x7fcac0110470 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:23.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.394+0000 7fcac5c00640 1 --2- 172.21.15.27:0/2755139245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcac01092f0 0x7fcac0109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:23.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.394+0000 7fcac5c00640 1 --2- 172.21.15.27:0/2755139245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac01039b0 0x7fcac0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:23.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.394+0000 7fcac5c00640 1 -- 172.21.15.27:0/2755139245 >> 172.21.15.27:0/2755139245 conn(0x7fcac00ff400 msgr2=0x7fcac0101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:23.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.394+0000 7fcac5c00640 1 -- 172.21.15.27:0/2755139245 shutdown_connections 2024-09-21T00:23:23.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.394+0000 7fcac5c00640 1 -- 172.21.15.27:0/2755139245 wait complete. 2024-09-21T00:23:23.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.395+0000 7fcac5c00640 1 Processor -- start 2024-09-21T00:23:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.395+0000 7fcac5c00640 1 -- start start 2024-09-21T00:23:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcac5c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcac01039b0 0x7fcac019c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcac5c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac01092f0 0x7fcac019ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcac5c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcac0109c40 0x7fcac01a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcac5c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0113900 con 0x7fcac01039b0 2024-09-21T00:23:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcac5c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0113780 con 0x7fcac01092f0 2024-09-21T00:23:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcac5c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0113a80 con 0x7fcac0109c40 2024-09-21T00:23:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcabf400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac01092f0 0x7fcac019ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcabfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcac01039b0 0x7fcac019c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcabf400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac01092f0 0x7fcac019ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40432/0 (socket says 172.21.15.27:40432) 2024-09-21T00:23:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcabf400640 1 -- 172.21.15.27:0/3385133790 learned_addr learned my addr 172.21.15.27:0/3385133790 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcac5200640 1 --2- 172.21.15.27:0/3385133790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcac0109c40 0x7fcac01a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcabf400640 1 -- 172.21.15.27:0/3385133790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcac0109c40 msgr2=0x7fcac01a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:23.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcabf400640 1 --2- 172.21.15.27:0/3385133790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcac0109c40 0x7fcac01a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:23.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcabf400640 1 -- 172.21.15.27:0/3385133790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcac01039b0 msgr2=0x7fcac019c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:23.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcabf400640 1 --2- 172.21.15.27:0/3385133790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcac01039b0 0x7fcac019c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:23.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.396+0000 7fcabf400640 1 -- 172.21.15.27:0/3385133790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac01a23e0 con 0x7fcac01092f0 2024-09-21T00:23:23.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.397+0000 7fcabf400640 1 --2- 172.21.15.27:0/3385133790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac01092f0 0x7fcac019ca10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00e970 tx=0x7fcaac00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:23.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.397+0000 7fcabcc00640 1 -- 172.21.15.27:0/3385133790 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac00cd80 con 0x7fcac01092f0 2024-09-21T00:23:23.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.397+0000 7fcac5c00640 1 -- 172.21.15.27:0/3385133790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac01a26d0 con 0x7fcac01092f0 2024-09-21T00:23:23.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.397+0000 7fcabcc00640 1 -- 172.21.15.27:0/3385133790 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaac0035f0 con 0x7fcac01092f0 2024-09-21T00:23:23.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.397+0000 7fcac5c00640 1 -- 172.21.15.27:0/3385133790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac01a8e30 con 0x7fcac01092f0 2024-09-21T00:23:23.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.397+0000 7fcabcc00640 1 -- 172.21.15.27:0/3385133790 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac010690 con 0x7fcac01092f0 2024-09-21T00:23:23.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.398+0000 7fcac5c00640 1 -- 172.21.15.27:0/3385133790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac0103dc0 con 0x7fcac01092f0 2024-09-21T00:23:23.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.401+0000 7fcabcc00640 1 -- 172.21.15.27:0/3385133790 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fcaac0037a0 con 0x7fcac01092f0 2024-09-21T00:23:23.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.403+0000 7fcabcc00640 1 --2- 172.21.15.27:0/3385133790 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcaa40b6700 0x7fcaa40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:23.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.403+0000 7fcabcc00640 1 -- 172.21.15.27:0/3385133790 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fcaac0e7da0 con 0x7fcac01092f0 2024-09-21T00:23:23.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.403+0000 7fcabfe00640 1 --2- 172.21.15.27:0/3385133790 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcaa40b6700 0x7fcaa40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:23.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.404+0000 7fcabcc00640 1 -- 172.21.15.27:0/3385133790 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcaac0adf90 con 0x7fcac01092f0 2024-09-21T00:23:23.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.404+0000 7fcabfe00640 1 --2- 172.21.15.27:0/3385133790 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcaa40b6700 0x7fcaa40b8c10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009bd0 tx=0x7fcab0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:23.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:23 smithi027 ceph-mon[30745]: pgmap v437: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-21T00:23:23.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.646+0000 7fcac5c00640 1 -- 172.21.15.27:0/3385133790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcac019d5a0 con 0x7fcac01092f0 2024-09-21T00:23:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.647+0000 7fcabcc00640 1 -- 172.21.15.27:0/3385133790 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11893 (secure 0 0 0) 0x7fcaac0b2370 con 0x7fcac01092f0 2024-09-21T00:23:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:23:23.649 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":26,"btime":"2024-09-21T00:21:36:085073+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:21:35.264444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:23:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-09-21T00:23:23.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.649+0000 7fcac5c00640 1 -- 172.21.15.27:0/3385133790 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcaa40b6700 msgr2=0x7fcaa40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:23.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.649+0000 7fcac5c00640 1 --2- 172.21.15.27:0/3385133790 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcaa40b6700 0x7fcaa40b8c10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009bd0 tx=0x7fcab0005f90 comp rx=0 tx=0).stop 2024-09-21T00:23:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.649+0000 7fcac5c00640 1 -- 172.21.15.27:0/3385133790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac01092f0 msgr2=0x7fcac019ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.649+0000 7fcac5c00640 1 --2- 172.21.15.27:0/3385133790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac01092f0 0x7fcac019ca10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00e970 tx=0x7fcaac00ee30 comp rx=0 tx=0).stop 2024-09-21T00:23:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.649+0000 7fcac5c00640 1 -- 172.21.15.27:0/3385133790 shutdown_connections 2024-09-21T00:23:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.649+0000 7fcac5c00640 1 --2- 172.21.15.27:0/3385133790 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcaa40b6700 0x7fcaa40b8c10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.649+0000 7fcac5c00640 1 --2- 172.21.15.27:0/3385133790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcac0109c40 0x7fcac01a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.649+0000 7fcac5c00640 1 --2- 172.21.15.27:0/3385133790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac01092f0 0x7fcac019ca10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.649+0000 7fcac5c00640 1 --2- 172.21.15.27:0/3385133790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcac01039b0 0x7fcac019c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.649+0000 7fcac5c00640 1 -- 172.21.15.27:0/3385133790 >> 172.21.15.27:0/3385133790 conn(0x7fcac00ff400 msgr2=0x7fcac0108940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.649+0000 7fcac5c00640 1 -- 172.21.15.27:0/3385133790 shutdown_connections 2024-09-21T00:23:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:23.649+0000 7fcac5c00640 1 -- 172.21.15.27:0/3385133790 wait complete. 2024-09-21T00:23:23.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:23 smithi089 ceph-mon[34481]: pgmap v437: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-21T00:23:23.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:23 smithi098 ceph-mon[34725]: pgmap v437: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-21T00:23:24.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:24 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3385133790' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:23:24.550 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id 72a16756 --release --await' 2024-09-21T00:23:24.551 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id 72a16756 --release --await 2024-09-21T00:23:24.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:24 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3385133790' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:23:24.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:24 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3385133790' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:23:24.937 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:23:25.381 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:25 smithi027 ceph-mon[30745]: pgmap v438: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-21T00:23:25.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.585+0000 7f1eb6600640 1 -- 172.21.15.27:0/4107795321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1eb0114740 msgr2=0x7f1eb0116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:25.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.585+0000 7f1eb6600640 1 --2- 172.21.15.27:0/4107795321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1eb0114740 0x7f1eb0116b60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009ca0 tx=0x7f1ea802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.585+0000 7f1eb6600640 1 -- 172.21.15.27:0/4107795321 shutdown_connections 2024-09-21T00:23:25.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.585+0000 7f1eb6600640 1 --2- 172.21.15.27:0/4107795321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1eb0114740 0x7f1eb0116b60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.585+0000 7f1eb6600640 1 --2- 172.21.15.27:0/4107795321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1eb0078290 0x7f1eb0114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.585+0000 7f1eb6600640 1 --2- 172.21.15.27:0/4107795321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb0077940 0x7f1eb0077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.585+0000 7f1eb6600640 1 -- 172.21.15.27:0/4107795321 >> 172.21.15.27:0/4107795321 conn(0x7f1eb0101810 msgr2=0x7f1eb0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:25.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.586+0000 7f1eb6600640 1 -- 172.21.15.27:0/4107795321 shutdown_connections 2024-09-21T00:23:25.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.586+0000 7f1eb6600640 1 -- 172.21.15.27:0/4107795321 wait complete. 2024-09-21T00:23:25.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.587+0000 7f1eb6600640 1 Processor -- start 2024-09-21T00:23:25.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.587+0000 7f1eb6600640 1 -- start start 2024-09-21T00:23:25.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.587+0000 7f1eb6600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1eb0077940 0x7f1eb01a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.587+0000 7f1eb6600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb0078290 0x7f1eb01a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.587+0000 7f1eb6600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1eb0114740 0x7f1eb01a6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.587+0000 7f1eb6600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb011a0d0 con 0x7f1eb0077940 2024-09-21T00:23:25.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.587+0000 7f1eb6600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb0119f50 con 0x7f1eb0078290 2024-09-21T00:23:25.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.587+0000 7f1eb6600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb011a250 con 0x7f1eb0114740 2024-09-21T00:23:25.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.588+0000 7f1eaf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1eb0077940 0x7f1eb01a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.588+0000 7f1eaf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1eb0077940 0x7f1eb01a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52080/0 (socket says 172.21.15.27:52080) 2024-09-21T00:23:25.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.588+0000 7f1eaf400640 1 -- 172.21.15.27:0/1506191584 learned_addr learned my addr 172.21.15.27:0/1506191584 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:25.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.588+0000 7f1ea6a00640 1 --2- 172.21.15.27:0/1506191584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb0078290 0x7f1eb01a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.588+0000 7f1eafe00640 1 --2- 172.21.15.27:0/1506191584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1eb0114740 0x7f1eb01a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.588+0000 7f1eaf400640 1 -- 172.21.15.27:0/1506191584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1eb0114740 msgr2=0x7f1eb01a6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:25.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.588+0000 7f1eaf400640 1 --2- 172.21.15.27:0/1506191584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1eb0114740 0x7f1eb01a6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.588+0000 7f1eaf400640 1 -- 172.21.15.27:0/1506191584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb0078290 msgr2=0x7f1eb01a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:25.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.588+0000 7f1eaf400640 1 --2- 172.21.15.27:0/1506191584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb0078290 0x7f1eb01a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.588+0000 7f1eaf400640 1 -- 172.21.15.27:0/1506191584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb01a6a70 con 0x7f1eb0077940 2024-09-21T00:23:25.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.588+0000 7f1ea6a00640 1 --2- 172.21.15.27:0/1506191584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb0078290 0x7f1eb01a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:23:25.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.589+0000 7f1eaf400640 1 --2- 172.21.15.27:0/1506191584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1eb0077940 0x7f1eb01a0b00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000ea60 tx=0x7f1ea000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:25.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.589+0000 7f1eacc00640 1 -- 172.21.15.27:0/1506191584 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea000cde0 con 0x7f1eb0077940 2024-09-21T00:23:25.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.589+0000 7f1eb6600640 1 -- 172.21.15.27:0/1506191584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb01a6d60 con 0x7f1eb0077940 2024-09-21T00:23:25.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.589+0000 7f1eacc00640 1 -- 172.21.15.27:0/1506191584 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea00035f0 con 0x7f1eb0077940 2024-09-21T00:23:25.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.589+0000 7f1eacc00640 1 -- 172.21.15.27:0/1506191584 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0010690 con 0x7f1eb0077940 2024-09-21T00:23:25.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.589+0000 7f1eb6600640 1 -- 172.21.15.27:0/1506191584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb01ad4c0 con 0x7f1eb0077940 2024-09-21T00:23:25.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.592+0000 7f1ebd28c740 1 Processor -- start 2024-09-21T00:23:25.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.592+0000 7f1ebd28c740 1 -- start start 2024-09-21T00:23:25.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.593+0000 7f1eacc00640 1 -- 172.21.15.27:0/1506191584 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1ea00037a0 con 0x7f1eb0077940 2024-09-21T00:23:25.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.593+0000 7f1ebd28c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95db520 0x55b9c99821c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.593+0000 7f1ebd28c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95b2660 0x55b9c9982700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.593+0000 7f1ebd28c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 0x55b9c9982c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.593+0000 7f1ebd28c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b9c98f9290 con 0x55b9c95db520 2024-09-21T00:23:25.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.593+0000 7f1ebd28c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b9c98e86c0 con 0x55b9c95b2660 2024-09-21T00:23:25.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.593+0000 7f1ebd28c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b9c98ec4d0 con 0x55b9c95ba840 2024-09-21T00:23:25.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.593+0000 7f1eafe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95db520 0x55b9c99821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.593+0000 7f1eaf400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95b2660 0x55b9c9982700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.593+0000 7f1eaf400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95b2660 0x55b9c9982700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40454/0 (socket says 172.21.15.27:40454) 2024-09-21T00:23:25.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.593+0000 7f1eaf400640 1 -- 172.21.15.27:0/772814464 learned_addr learned my addr 172.21.15.27:0/772814464 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:25.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.593+0000 7f1eafe00640 1 --2- 172.21.15.27:0/772814464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95db520 0x55b9c99821c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52088/0 (socket says 172.21.15.27:52088) 2024-09-21T00:23:25.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.594+0000 7f1ea6a00640 1 --2- 172.21.15.27:0/772814464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 0x55b9c9982c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.594+0000 7f1eaf400640 1 -- 172.21.15.27:0/772814464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 msgr2=0x55b9c9982c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.594+0000 7f1eaf400640 1 --2- 172.21.15.27:0/772814464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 0x55b9c9982c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.594+0000 7f1eaf400640 1 -- 172.21.15.27:0/772814464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95db520 msgr2=0x55b9c99821c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.594+0000 7f1eaf400640 1 --2- 172.21.15.27:0/772814464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95db520 0x55b9c99821c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.594+0000 7f1eaf400640 1 -- 172.21.15.27:0/772814464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c9875460 con 0x55b9c95b2660 2024-09-21T00:23:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.594+0000 7f1eaf400640 1 --2- 172.21.15.27:0/772814464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95b2660 0x55b9c9982700 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00e5090 tx=0x7f1ea00ee830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1141b3d99c3aed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.595+0000 7f1eacc00640 1 --2- 172.21.15.27:0/1506191584 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1e940b6680 0x7f1e940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.595+0000 7f1ea4c00640 1 -- 172.21.15.27:0/772814464 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00acfc0 con 0x55b9c95b2660 2024-09-21T00:23:25.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.595+0000 7f1eacc00640 1 -- 172.21.15.27:0/1506191584 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f1ea00e7df0 con 0x7f1eb0077940 2024-09-21T00:23:25.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.595+0000 7f1ea4c00640 1 -- 172.21.15.27:0/772814464 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea007edb0 con 0x55b9c95b2660 2024-09-21T00:23:25.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.595+0000 7f1ea4c00640 1 -- 172.21.15.27:0/772814464 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00ad1a0 con 0x55b9c95b2660 2024-09-21T00:23:25.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.595+0000 7f1eafe00640 1 --2- 172.21.15.27:0/1506191584 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1e940b6680 0x7f1e940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.596+0000 7f1ebd28c740 1 -- 172.21.15.27:0/772814464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95b2660 msgr2=0x55b9c9982700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:25.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.596+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/772814464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95b2660 0x55b9c9982700 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00e5090 tx=0x7f1ea00ee830 comp rx=0 tx=0).stop 2024-09-21T00:23:25.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.596+0000 7f1eafe00640 1 --2- 172.21.15.27:0/1506191584 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1e940b6680 0x7f1e940b8b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1ea802f500 tx=0x7f1ea802f310 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:25.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.596+0000 7f1ebd28c740 1 -- 172.21.15.27:0/772814464 shutdown_connections 2024-09-21T00:23:25.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.596+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/772814464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 0x55b9c9982c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.596+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/772814464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95b2660 0x55b9c9982700 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.596+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/772814464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95db520 0x55b9c99821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.596+0000 7f1ebd28c740 1 -- 172.21.15.27:0/772814464 >> 172.21.15.27:0/772814464 conn(0x55b9c95cf910 msgr2=0x55b9c99800b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:25.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.596+0000 7f1ebd28c740 1 -- 172.21.15.27:0/772814464 shutdown_connections 2024-09-21T00:23:25.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.596+0000 7f1ebd28c740 1 -- 172.21.15.27:0/772814464 wait complete. 2024-09-21T00:23:25.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.597+0000 7f1ebd28c740 1 -- start start 2024-09-21T00:23:25.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.597+0000 7f1ebd28c740 1 Processor -- start 2024-09-21T00:23:25.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.598+0000 7f1ebd28c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95b2660 0x55b9c998dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.598+0000 7f1ebd28c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 0x55b9c998e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.598+0000 7f1ebd28c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95db520 0x55b9c9999c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.598+0000 7f1ebd28c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b9c9749bb0 con 0x55b9c95b2660 2024-09-21T00:23:25.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.598+0000 7f1ebd28c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b9c98836c0 con 0x55b9c95db520 2024-09-21T00:23:25.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.598+0000 7f1ebd28c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b9c99909a0 con 0x55b9c95ba840 2024-09-21T00:23:25.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.598+0000 7f1ebd28c740 10 client.0 _ll_register_callbacks cb 0x55b9c989e490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:23:25.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.598+0000 7f1ea6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 0x55b9c998e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.598+0000 7f1eaf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95b2660 0x55b9c998dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.598+0000 7f1eafe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95db520 0x55b9c9999c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.598+0000 7f1ea6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 0x55b9c998e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:47494/0 (socket says 172.21.15.27:47494) 2024-09-21T00:23:25.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.598+0000 7f1ea6a00640 1 -- 172.21.15.27:0/573923611 learned_addr learned my addr 172.21.15.27:0/573923611 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:25.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.598+0000 7f1ea6a00640 1 -- 172.21.15.27:0/573923611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95db520 msgr2=0x55b9c9999c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:25.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.598+0000 7f1ea6a00640 1 --2- 172.21.15.27:0/573923611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95db520 0x55b9c9999c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.599+0000 7f1ea6a00640 1 -- 172.21.15.27:0/573923611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95b2660 msgr2=0x55b9c998dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:25.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.599+0000 7f1ea6a00640 1 --2- 172.21.15.27:0/573923611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95b2660 0x55b9c998dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.599+0000 7f1ea6a00640 1 -- 172.21.15.27:0/573923611 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c9646b00 con 0x55b9c95ba840 2024-09-21T00:23:25.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.599+0000 7f1eaf400640 1 --2- 172.21.15.27:0/573923611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95b2660 0x55b9c998dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:23:25.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.599+0000 7f1ea6a00640 1 --2- 172.21.15.27:0/573923611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 0x55b9c998e2e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1e98012d20 tx=0x7f1e9800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cb650f9782469e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:25.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.599+0000 7f1e9d600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:23:25.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.599+0000 7f1e9d600640 1 -- 172.21.15.27:0/573923611 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e980120d0 con 0x55b9c95ba840 2024-09-21T00:23:25.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.599+0000 7f1ebd28c740 1 -- 172.21.15.27:0/573923611 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ea007edb0 con 0x55b9c95ba840 2024-09-21T00:23:25.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.599+0000 7f1e9d600640 1 -- 172.21.15.27:0/573923611 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f1e980047a0 con 0x55b9c95ba840 2024-09-21T00:23:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.599+0000 7f1e9d600640 1 -- 172.21.15.27:0/573923611 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e98012270 con 0x55b9c95ba840 2024-09-21T00:23:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.599+0000 7f1ebd28c740 10 client.25705 fetch_fsmap learned FSMap version 26 2024-09-21T00:23:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.599+0000 7f1ebd28c740 1 -- 172.21.15.27:0/573923611 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b9c96bd960 con 0x55b9c95ba840 2024-09-21T00:23:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.600+0000 7f1e9d600640 1 -- 172.21.15.27:0/573923611 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea007edb0 con 0x55b9c95ba840 2024-09-21T00:23:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.600+0000 7f1e9d600640 1 -- 172.21.15.27:0/573923611 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1e98010550 con 0x55b9c95ba840 2024-09-21T00:23:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.600+0000 7f1ebd28c740 10 client.25705 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:23:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.600+0000 7f1ebd28c740 10 client.25705 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:23:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.600+0000 7f1ebd28c740 20 client.25705 populate_metadata read hostname 'smithi027' 2024-09-21T00:23:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.600+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/573923611 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b9c9599ea0 0x55b9c999fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.600+0000 7f1ebd28c740 4 client.25705 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:23:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.600+0000 7f1ebd28c740 1 -- 172.21.15.27:0/573923611 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9c98fb1e0 con 0x55b9c9599ea0 2024-09-21T00:23:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.601+0000 7f1eaf400640 1 --2- 172.21.15.27:0/573923611 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b9c9599ea0 0x55b9c999fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.601+0000 7f1eaf400640 1 --2- 172.21.15.27:0/573923611 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b9c9599ea0 0x55b9c999fcc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c377e1d1d938d40c server_cookie=fc5cb291e2302933 in_seq=0 out_seq=0 2024-09-21T00:23:25.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.601+0000 7f1e9d600640 10 client.25705 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:25.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.602+0000 7f1e9d600640 1 -- 172.21.15.27:0/573923611 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b9c98fb1e0 con 0x55b9c9599ea0 2024-09-21T00:23:25.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.602+0000 7f1e9d600640 10 client.25705 handle_command_reply: tid=0 2024-09-21T00:23:25.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.602+0000 7f1ebd28c740 1 client.25705 shutdown 2024-09-21T00:23:25.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.602+0000 7f1ebd28c740 1 -- 172.21.15.27:0/573923611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 msgr2=0x55b9c998e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:25.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.602+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/573923611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 0x55b9c998e2e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1e98012d20 tx=0x7f1e9800f790 comp rx=0 tx=0).stop 2024-09-21T00:23:25.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.602+0000 7f1ebd28c740 1 -- 172.21.15.27:0/573923611 shutdown_connections 2024-09-21T00:23:25.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.602+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/573923611 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b9c9599ea0 0x55b9c999fcc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.602+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/573923611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95db520 0x55b9c9999c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.602+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/573923611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 0x55b9c998e2e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.602+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/573923611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95b2660 0x55b9c998dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.602+0000 7f1e9d600640 0 client.25705 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:25.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.602+0000 7f1ebd28c740 1 -- 172.21.15.27:0/573923611 >> 172.21.15.27:0/573923611 conn(0x55b9c95cf910 msgr2=0x55b9c99853a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:25.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.603+0000 7f1ebd28c740 1 -- 172.21.15.27:0/573923611 shutdown_connections 2024-09-21T00:23:25.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.603+0000 7f1ebd28c740 1 -- 172.21.15.27:0/573923611 wait complete. 2024-09-21T00:23:25.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.603+0000 7f1ebd28c740 20 client.25705 trim_cache size 0 max 16384 2024-09-21T00:23:25.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.621+0000 7f1ebd28c740 1 Processor -- start 2024-09-21T00:23:25.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.621+0000 7f1ebd28c740 1 -- start start 2024-09-21T00:23:25.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.621+0000 7f1ebd28c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95b2660 0x55b9c9985c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.621+0000 7f1ebd28c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 0x55b9c9983900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.621+0000 7f1ebd28c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95db520 0x55b9c998ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.621+0000 7f1ebd28c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb01b9e60 con 0x55b9c95b2660 2024-09-21T00:23:25.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.621+0000 7f1ebd28c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb01b7730 con 0x55b9c95db520 2024-09-21T00:23:25.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.621+0000 7f1ebd28c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb01b5580 con 0x55b9c95ba840 2024-09-21T00:23:25.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.621+0000 7f1eaf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95b2660 0x55b9c9985c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.622+0000 7f1eaf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95b2660 0x55b9c9985c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52100/0 (socket says 172.21.15.27:52100) 2024-09-21T00:23:25.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.622+0000 7f1eaf400640 1 -- 172.21.15.27:0/2683037287 learned_addr learned my addr 172.21.15.27:0/2683037287 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:25.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.622+0000 7f1eafe00640 1 --2- 172.21.15.27:0/2683037287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95db520 0x55b9c998ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.622+0000 7f1ea6a00640 1 --2- 172.21.15.27:0/2683037287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 0x55b9c9983900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.622+0000 7f1eaf400640 1 -- 172.21.15.27:0/2683037287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 msgr2=0x55b9c9983900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:25.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.622+0000 7f1eaf400640 1 --2- 172.21.15.27:0/2683037287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 0x55b9c9983900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.622+0000 7f1eaf400640 1 -- 172.21.15.27:0/2683037287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95db520 msgr2=0x55b9c998ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:25.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.622+0000 7f1eaf400640 1 --2- 172.21.15.27:0/2683037287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95db520 0x55b9c998ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.622+0000 7f1eaf400640 1 -- 172.21.15.27:0/2683037287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c998ee60 con 0x55b9c95b2660 2024-09-21T00:23:25.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.622+0000 7f1eaf400640 1 --2- 172.21.15.27:0/2683037287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95b2660 0x55b9c9985c80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0003ed0 tx=0x7f1ea0009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80756cc8389e191f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:25.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.623+0000 7f1e9fe00640 1 -- 172.21.15.27:0/2683037287 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00ad390 con 0x55b9c95b2660 2024-09-21T00:23:25.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.623+0000 7f1e9fe00640 1 -- 172.21.15.27:0/2683037287 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea00adf50 con 0x55b9c95b2660 2024-09-21T00:23:25.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.623+0000 7f1e9fe00640 1 -- 172.21.15.27:0/2683037287 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00ad620 con 0x55b9c95b2660 2024-09-21T00:23:25.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.623+0000 7f1ebd28c740 1 -- 172.21.15.27:0/2683037287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95b2660 msgr2=0x55b9c9985c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:25.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.623+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/2683037287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95b2660 0x55b9c9985c80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0003ed0 tx=0x7f1ea0009490 comp rx=0 tx=0).stop 2024-09-21T00:23:25.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.624+0000 7f1ebd28c740 1 -- 172.21.15.27:0/2683037287 shutdown_connections 2024-09-21T00:23:25.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.624+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/2683037287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95db520 0x55b9c998ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.624+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/2683037287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95ba840 0x55b9c9983900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.624+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/2683037287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95b2660 0x55b9c9985c80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.624+0000 7f1ebd28c740 1 -- 172.21.15.27:0/2683037287 >> 172.21.15.27:0/2683037287 conn(0x55b9c95cf910 msgr2=0x55b9c99831f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:25.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.624+0000 7f1ebd28c740 1 -- 172.21.15.27:0/2683037287 shutdown_connections 2024-09-21T00:23:25.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.624+0000 7f1ebd28c740 1 -- 172.21.15.27:0/2683037287 wait complete. 2024-09-21T00:23:25.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.624+0000 7f1ebd28c740 1 -- start start 2024-09-21T00:23:25.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.625+0000 7f1ebd28c740 1 Processor -- start 2024-09-21T00:23:25.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.625+0000 7f1ebd28c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95b2660 0x55b9c99925b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.625+0000 7f1ebd28c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95ba840 0x55b9c9992b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.625+0000 7f1ebd28c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95db520 0x55b9c9a03650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.625+0000 7f1ebd28c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb01b5bf0 con 0x55b9c95ba840 2024-09-21T00:23:25.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.625+0000 7f1ebd28c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb01b7060 con 0x55b9c95b2660 2024-09-21T00:23:25.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.625+0000 7f1ebd28c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb01b62c0 con 0x55b9c95db520 2024-09-21T00:23:25.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.625+0000 7f1ebd28c740 10 client.0 _ll_register_callbacks cb 0x7f1e98009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:23:25.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.625+0000 7f1ea6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95ba840 0x55b9c9992b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.625+0000 7f1eafe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95db520 0x55b9c9a03650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.625+0000 7f1ea6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95ba840 0x55b9c9992b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52104/0 (socket says 172.21.15.27:52104) 2024-09-21T00:23:25.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.625+0000 7f1ea6a00640 1 -- 172.21.15.27:0/14374206 learned_addr learned my addr 172.21.15.27:0/14374206 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:25.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.626+0000 7f1eaf400640 1 --2- 172.21.15.27:0/14374206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95b2660 0x55b9c99925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.626+0000 7f1eafe00640 1 -- 172.21.15.27:0/14374206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95b2660 msgr2=0x55b9c99925b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:25.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.626+0000 7f1eafe00640 1 --2- 172.21.15.27:0/14374206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95b2660 0x55b9c99925b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.626+0000 7f1eafe00640 1 -- 172.21.15.27:0/14374206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95ba840 msgr2=0x55b9c9992b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:25.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.626+0000 7f1eafe00640 1 --2- 172.21.15.27:0/14374206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95ba840 0x55b9c9992b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.626+0000 7f1eafe00640 1 -- 172.21.15.27:0/14374206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c9a03cc0 con 0x55b9c95db520 2024-09-21T00:23:25.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.626+0000 7f1eaf400640 1 --2- 172.21.15.27:0/14374206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95b2660 0x55b9c99925b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:23:25.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.626+0000 7f1ea6a00640 1 --2- 172.21.15.27:0/14374206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95ba840 0x55b9c9992b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:23:25.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.626+0000 7f1eafe00640 1 --2- 172.21.15.27:0/14374206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95db520 0x55b9c9a03650 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8006010 tx=0x7f1ea80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f12233c23463474 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:25.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.626+0000 7f1e9d600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:23:25.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.627+0000 7f1e9d600640 1 -- 172.21.15.27:0/14374206 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8058060 con 0x55b9c95db520 2024-09-21T00:23:25.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.627+0000 7f1ebd28c740 1 -- 172.21.15.27:0/14374206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ea00adf50 con 0x55b9c95db520 2024-09-21T00:23:25.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.627+0000 7f1e9d600640 1 -- 172.21.15.27:0/14374206 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f1ea8004040 con 0x55b9c95db520 2024-09-21T00:23:25.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.627+0000 7f1e9d600640 1 -- 172.21.15.27:0/14374206 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1ea805f050 con 0x55b9c95db520 2024-09-21T00:23:25.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.627+0000 7f1ebd28c740 10 client.25711 fetch_fsmap learned FSMap version 26 2024-09-21T00:23:25.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.627+0000 7f1ebd28c740 1 -- 172.21.15.27:0/14374206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b9c9a0b120 con 0x55b9c95db520 2024-09-21T00:23:25.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.628+0000 7f1e9d600640 1 -- 172.21.15.27:0/14374206 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea00adf50 con 0x55b9c95db520 2024-09-21T00:23:25.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.628+0000 7f1e9d600640 1 -- 172.21.15.27:0/14374206 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1ea802fbf0 con 0x55b9c95db520 2024-09-21T00:23:25.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.628+0000 7f1ebd28c740 10 client.25711 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:23:25.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.628+0000 7f1ebd28c740 10 client.25711 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:23:25.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.628+0000 7f1ebd28c740 20 client.25711 populate_metadata read hostname 'smithi027' 2024-09-21T00:23:25.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.628+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/14374206 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b9c956f250 0x55b9c9a0d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:25.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.628+0000 7f1ebd28c740 4 client.25711 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "72a16756", "release": true, "await": true}] 2024-09-21T00:23:25.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.628+0000 7f1ebd28c740 1 -- 172.21.15.27:0/14374206 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "quiesce db", "set_id": "72a16756", "release": true, "await": true}) -- 0x55b9c98f6c90 con 0x55b9c956f250 2024-09-21T00:23:25.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.629+0000 7f1eaf400640 1 --2- 172.21.15.27:0/14374206 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b9c956f250 0x55b9c9a0d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:25.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.629+0000 7f1eaf400640 1 --2- 172.21.15.27:0/14374206 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b9c956f250 0x55b9c9a0d8c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4547dad84bc31819 server_cookie=1e2fc198a9dfa1d8 in_seq=0 out_seq=0 2024-09-21T00:23:25.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.629+0000 7f1e9d600640 10 client.25711 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:25.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.632+0000 7f1e9d600640 1 -- 172.21.15.27:0/14374206 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b9c98f6c90 con 0x55b9c956f250 2024-09-21T00:23:25.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.632+0000 7f1e9d600640 10 client.25711 handle_command_reply: tid=0 2024-09-21T00:23:25.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.632+0000 7f1ebd28c740 1 client.25711 shutdown 2024-09-21T00:23:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.633+0000 7f1ebd28c740 1 -- 172.21.15.27:0/14374206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95db520 msgr2=0x55b9c9a03650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.633+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/14374206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95db520 0x55b9c9a03650 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8006010 tx=0x7f1ea80095c0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.633+0000 7f1ebd28c740 1 -- 172.21.15.27:0/14374206 shutdown_connections 2024-09-21T00:23:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.633+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/14374206 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b9c956f250 0x55b9c9a0d8c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.633+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/14374206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b9c95db520 0x55b9c9a03650 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.633+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/14374206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b9c95ba840 0x55b9c9992b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.633+0000 7f1ebd28c740 1 --2- 172.21.15.27:0/14374206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b9c95b2660 0x55b9c99925b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.633+0000 7f1ebd28c740 1 -- 172.21.15.27:0/14374206 >> 172.21.15.27:0/14374206 conn(0x55b9c95cf910 msgr2=0x55b9c9983bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.633+0000 7f1e9d600640 0 client.25711 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.633+0000 7f1ebd28c740 1 -- 172.21.15.27:0/14374206 shutdown_connections 2024-09-21T00:23:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.633+0000 7f1ebd28c740 1 -- 172.21.15.27:0/14374206 wait complete. 2024-09-21T00:23:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.633+0000 7f1ebd28c740 20 client.25711 trim_cache size 0 max 16384 2024-09-21T00:23:25.637 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:23:25.637 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 26, 2024-09-21T00:23:25.637 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24761, 2024-09-21T00:23:25.637 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 8, 2024-09-21T00:23:25.637 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T00:23:25.637 INFO:teuthology.orchestra.run.smithi027.stdout: "72a16756": { 2024-09-21T00:23:25.637 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 8, 2024-09-21T00:23:25.637 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T00:23:25.637 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:23:25.637 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T00:23:25.637 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:23:25.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:23:25.638 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T00:23:25.638 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 137.0, 2024-09-21T00:23:25.638 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T00:23:25.638 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T00:23:25.638 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T00:23:25.638 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:23:25.638 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T00:23:25.638 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:23:25.638 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:23:25.638 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:23:25.638 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:23:25.638 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:23:25.638 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:23:25.638 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:23:25.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.634+0000 7f1eb6600640 1 -- 172.21.15.27:0/1506191584 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1e940b6680 msgr2=0x7f1e940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:25.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.634+0000 7f1eb6600640 1 --2- 172.21.15.27:0/1506191584 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1e940b6680 0x7f1e940b8b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1ea802f500 tx=0x7f1ea802f310 comp rx=0 tx=0).stop 2024-09-21T00:23:25.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.634+0000 7f1eb6600640 1 -- 172.21.15.27:0/1506191584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1eb0077940 msgr2=0x7f1eb01a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:25.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.634+0000 7f1eb6600640 1 --2- 172.21.15.27:0/1506191584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1eb0077940 0x7f1eb01a0b00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000ea60 tx=0x7f1ea000ef20 comp rx=0 tx=0).stop 2024-09-21T00:23:25.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.635+0000 7f1eb6600640 1 -- 172.21.15.27:0/1506191584 shutdown_connections 2024-09-21T00:23:25.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.635+0000 7f1eb6600640 1 --2- 172.21.15.27:0/1506191584 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1e940b6680 0x7f1e940b8b90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.635+0000 7f1eb6600640 1 --2- 172.21.15.27:0/1506191584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1eb0114740 0x7f1eb01a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.635+0000 7f1eb6600640 1 --2- 172.21.15.27:0/1506191584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb0078290 0x7f1eb01a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.635+0000 7f1eb6600640 1 --2- 172.21.15.27:0/1506191584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1eb0077940 0x7f1eb01a0b00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.635+0000 7f1eb6600640 1 -- 172.21.15.27:0/1506191584 >> 172.21.15.27:0/1506191584 conn(0x7f1eb0101810 msgr2=0x7f1eb0116760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.635+0000 7f1eb6600640 1 -- 172.21.15.27:0/1506191584 shutdown_connections 2024-09-21T00:23:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:25.635+0000 7f1eb6600640 1 -- 172.21.15.27:0/1506191584 wait complete. 2024-09-21T00:23:25.648 INFO:tasks.workunit.client.0.smithi027.stderr:+ echo 'getting blogbench' 2024-09-21T00:23:25.648 INFO:tasks.workunit.client.0.smithi027.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-21T00:23:25.649 INFO:tasks.workunit.client.0.smithi027.stdout:getting blogbench 2024-09-21T00:23:25.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:25 smithi089 ceph-mon[34481]: pgmap v438: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-21T00:23:25.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:25 smithi098 ceph-mon[34725]: pgmap v438: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-21T00:23:25.756 INFO:tasks.workunit.client.0.smithi027.stderr:--2024-09-21 00:23:25-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-21T00:23:25.757 INFO:tasks.workunit.client.0.smithi027.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-21T00:23:25.779 INFO:tasks.workunit.client.0.smithi027.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-21T00:23:25.814 INFO:tasks.workunit.client.0.smithi027.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-21T00:23:25.814 INFO:tasks.workunit.client.0.smithi027.stderr:Length: 103355 (101K) [application/octet-stream] 2024-09-21T00:23:25.816 INFO:tasks.workunit.client.0.smithi027.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-09-21T00:23:25.821 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T00:23:25.856 INFO:tasks.workunit.client.0.smithi027.stderr: 0K .......... .......... .......... .......... .......... 49% 1.27M 0s 2024-09-21T00:23:25.876 INFO:tasks.workunit.client.0.smithi027.stderr: 50K .......... .......... .......... .......... .......... 99% 2.35M 0s 2024-09-21T00:23:25.876 INFO:tasks.workunit.client.0.smithi027.stderr: 100K 100% 1.74T=0.06s 2024-09-21T00:23:25.876 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T00:23:25.877 INFO:tasks.workunit.client.0.smithi027.stderr:2024-09-21 00:23:25 (1.66 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-09-21T00:23:25.877 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T00:23:25.877 INFO:tasks.workunit.client.0.smithi027.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-09-21T00:23:25.940 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/ 2024-09-21T00:23:25.941 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/m4/ 2024-09-21T00:23:25.943 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-09-21T00:23:25.944 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/man/ 2024-09-21T00:23:25.945 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/man/blogbench.8 2024-09-21T00:23:25.947 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/man/Makefile.am 2024-09-21T00:23:25.948 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/man/Makefile.in 2024-09-21T00:23:25.949 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/ 2024-09-21T00:23:25.951 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/mysnprintf.c 2024-09-21T00:23:25.952 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/mysnprintf.h 2024-09-21T00:23:25.954 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/blogbench.c 2024-09-21T00:23:25.955 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/blogbench.h 2024-09-21T00:23:25.956 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/fakesnprintf.c 2024-09-21T00:23:25.957 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/fakesnprintf.h 2024-09-21T00:23:25.958 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/blogbench_p.h 2024-09-21T00:23:25.960 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-09-21T00:23:25.961 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-09-21T00:23:25.963 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/Makefile.am 2024-09-21T00:23:25.964 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/Makefile.in 2024-09-21T00:23:25.965 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/process.c 2024-09-21T00:23:25.966 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/globals.h 2024-09-21T00:23:25.968 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/commenter.c 2024-09-21T00:23:25.969 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/rewriter.c 2024-09-21T00:23:25.971 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/reader.c 2024-09-21T00:23:25.972 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/writer.c 2024-09-21T00:23:25.973 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/src/helpers.c 2024-09-21T00:23:25.975 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/NEWS 2024-09-21T00:23:25.976 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/compile 2024-09-21T00:23:25.977 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/depcomp 2024-09-21T00:23:25.978 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/aclocal.m4 2024-09-21T00:23:25.980 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/README 2024-09-21T00:23:25.981 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/configure 2024-09-21T00:23:25.985 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/configure.ac 2024-09-21T00:23:25.986 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/config.guess 2024-09-21T00:23:25.988 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/install-sh 2024-09-21T00:23:25.989 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/config.sub 2024-09-21T00:23:25.990 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/missing 2024-09-21T00:23:25.991 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/mkinstalldirs 2024-09-21T00:23:25.992 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/Makefile.am 2024-09-21T00:23:25.994 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/Makefile.in 2024-09-21T00:23:25.995 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/config.h.in 2024-09-21T00:23:25.996 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/AUTHORS 2024-09-21T00:23:25.997 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/INSTALL 2024-09-21T00:23:25.998 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/ChangeLog 2024-09-21T00:23:25.999 INFO:tasks.workunit.client.0.smithi027.stdout:blogbench-1.0/COPYING 2024-09-21T00:23:26.001 INFO:tasks.workunit.client.0.smithi027.stderr:+ cd blogbench-1.0/ 2024-09-21T00:23:26.001 INFO:tasks.workunit.client.0.smithi027.stdout:making blogbench 2024-09-21T00:23:26.002 INFO:tasks.workunit.client.0.smithi027.stderr:+ echo 'making blogbench' 2024-09-21T00:23:26.002 INFO:tasks.workunit.client.0.smithi027.stderr:+ ./configure 2024-09-21T00:23:26.239 INFO:tasks.workunit.client.0.smithi027.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-21T00:23:26.442 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '72a16756', total seconds elapsed: 21.1 2024-09-21T00:23:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.643+0000 7f4de3e00640 1 -- 172.21.15.27:0/438698435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4109ba0 msgr2=0x7f4de41103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.643+0000 7f4de3e00640 1 --2- 172.21.15.27:0/438698435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4109ba0 0x7f4de41103d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009ca0 tx=0x7f4ddc02a8c0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.644+0000 7f4de3e00640 1 -- 172.21.15.27:0/438698435 shutdown_connections 2024-09-21T00:23:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.644+0000 7f4de3e00640 1 --2- 172.21.15.27:0/438698435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4109ba0 0x7f4de41103d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.644+0000 7f4de3e00640 1 --2- 172.21.15.27:0/438698435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de4109250 0x7f4de4109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.644+0000 7f4de3e00640 1 --2- 172.21.15.27:0/438698435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4de4103920 0x7f4de4103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.644+0000 7f4de3e00640 1 -- 172.21.15.27:0/438698435 >> 172.21.15.27:0/438698435 conn(0x7f4de40ff340 msgr2=0x7f4de4101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.644+0000 7f4de3e00640 1 -- 172.21.15.27:0/438698435 shutdown_connections 2024-09-21T00:23:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.644+0000 7f4de3e00640 1 -- 172.21.15.27:0/438698435 wait complete. 2024-09-21T00:23:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.645+0000 7f4de3e00640 1 Processor -- start 2024-09-21T00:23:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.645+0000 7f4de3e00640 1 -- start start 2024-09-21T00:23:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.645+0000 7f4de3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4103920 0x7f4de419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.645+0000 7f4de3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4de4109250 0x7f4de419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.645+0000 7f4de3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de4109ba0 0x7f4de41a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.645+0000 7f4de3e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4de4113860 con 0x7f4de4103920 2024-09-21T00:23:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.645+0000 7f4de3e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4de41136e0 con 0x7f4de4109250 2024-09-21T00:23:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.645+0000 7f4de3e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4de41139e0 con 0x7f4de4109ba0 2024-09-21T00:23:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.645+0000 7f4de2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4103920 0x7f4de419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.646+0000 7f4de2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4103920 0x7f4de419c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52134/0 (socket says 172.21.15.27:52134) 2024-09-21T00:23:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.646+0000 7f4de2a00640 1 -- 172.21.15.27:0/4274677902 learned_addr learned my addr 172.21.15.27:0/4274677902 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.646+0000 7f4de2000640 1 --2- 172.21.15.27:0/4274677902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4de4109250 0x7f4de419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.646+0000 7f4de3400640 1 --2- 172.21.15.27:0/4274677902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de4109ba0 0x7f4de41a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.646+0000 7f4de2a00640 1 -- 172.21.15.27:0/4274677902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de4109ba0 msgr2=0x7f4de41a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.646+0000 7f4de2a00640 1 --2- 172.21.15.27:0/4274677902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de4109ba0 0x7f4de41a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.646+0000 7f4de2a00640 1 -- 172.21.15.27:0/4274677902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4de4109250 msgr2=0x7f4de419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.646+0000 7f4de2a00640 1 --2- 172.21.15.27:0/4274677902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4de4109250 0x7f4de419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.646+0000 7f4de2a00640 1 -- 172.21.15.27:0/4274677902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de41a2350 con 0x7f4de4103920 2024-09-21T00:23:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.646+0000 7f4de2000640 1 --2- 172.21.15.27:0/4274677902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4de4109250 0x7f4de419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:23:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.646+0000 7f4de2a00640 1 --2- 172.21.15.27:0/4274677902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4103920 0x7f4de419c420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400e9f0 tx=0x7f4dd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.647+0000 7f4ddb400640 1 -- 172.21.15.27:0/4274677902 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd400cd80 con 0x7f4de4103920 2024-09-21T00:23:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.647+0000 7f4ddb400640 1 -- 172.21.15.27:0/4274677902 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd40108e0 con 0x7f4de4103920 2024-09-21T00:23:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.647+0000 7f4de3e00640 1 -- 172.21.15.27:0/4274677902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de41a2640 con 0x7f4de4103920 2024-09-21T00:23:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.647+0000 7f4ddb400640 1 -- 172.21.15.27:0/4274677902 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4018980 con 0x7f4de4103920 2024-09-21T00:23:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.647+0000 7f4de3e00640 1 -- 172.21.15.27:0/4274677902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de41a8dd0 con 0x7f4de4103920 2024-09-21T00:23:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.650+0000 7f4df05cd740 1 Processor -- start 2024-09-21T00:23:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.650+0000 7f4df05cd740 1 -- start start 2024-09-21T00:23:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.650+0000 7f4df05cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 0x55c733c80e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.650+0000 7f4df05cd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338b32f0 0x55c733c81350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.650+0000 7f4df05cd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338bb4d0 0x55c733c81890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.650+0000 7f4ddb400640 1 -- 172.21.15.27:0/4274677902 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4dd4010430 con 0x7f4de4103920 2024-09-21T00:23:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.650+0000 7f4df05cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c733bf66b0 con 0x55c7338dc1b0 2024-09-21T00:23:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.650+0000 7f4df05cd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c733be5ae0 con 0x55c7338b32f0 2024-09-21T00:23:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.650+0000 7f4df05cd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c733be98f0 con 0x55c7338bb4d0 2024-09-21T00:23:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.651+0000 7f4de3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 0x55c733c80e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.651+0000 7f4de3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 0x55c733c80e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52136/0 (socket says 172.21.15.27:52136) 2024-09-21T00:23:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.651+0000 7f4de3400640 1 -- 172.21.15.27:0/3132614759 learned_addr learned my addr 172.21.15.27:0/3132614759 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.651+0000 7f4de2a00640 1 --2- 172.21.15.27:0/3132614759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338b32f0 0x55c733c81350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.651+0000 7f4de2000640 1 --2- 172.21.15.27:0/3132614759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338bb4d0 0x55c733c81890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.651+0000 7f4de3400640 1 -- 172.21.15.27:0/3132614759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338bb4d0 msgr2=0x55c733c81890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.651+0000 7f4de3400640 1 --2- 172.21.15.27:0/3132614759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338bb4d0 0x55c733c81890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.651+0000 7f4de3400640 1 -- 172.21.15.27:0/3132614759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338b32f0 msgr2=0x55c733c81350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.651+0000 7f4de3400640 1 --2- 172.21.15.27:0/3132614759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338b32f0 0x55c733c81350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.651+0000 7f4de3400640 1 -- 172.21.15.27:0/3132614759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c733bb2780 con 0x55c7338dc1b0 2024-09-21T00:23:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.651+0000 7f4de2a00640 1 --2- 172.21.15.27:0/3132614759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338b32f0 0x55c733c81350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.652+0000 7f4de3400640 1 --2- 172.21.15.27:0/3132614759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 0x55c733c80e10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc02f410 tx=0x7f4ddc02f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ca9a655b965d05 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.652+0000 7f4dcf400640 1 -- 172.21.15.27:0/3132614759 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc031cf0 con 0x55c7338dc1b0 2024-09-21T00:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.652+0000 7f4dcf400640 1 -- 172.21.15.27:0/3132614759 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ddc036030 con 0x55c7338dc1b0 2024-09-21T00:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.652+0000 7f4dcf400640 1 -- 172.21.15.27:0/3132614759 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc0042a0 con 0x55c7338dc1b0 2024-09-21T00:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.652+0000 7f4ddb400640 1 --2- 172.21.15.27:0/4274677902 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4dc80b6670 0x7f4dc80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.653+0000 7f4ddb400640 1 -- 172.21.15.27:0/4274677902 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f4dd40e7de0 con 0x7f4de4103920 2024-09-21T00:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.653+0000 7f4de2a00640 1 --2- 172.21.15.27:0/4274677902 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4dc80b6670 0x7f4dc80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.653+0000 7f4df05cd740 1 -- 172.21.15.27:0/3132614759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 msgr2=0x55c733c80e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.653+0000 7f4df05cd740 1 --2- 172.21.15.27:0/3132614759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 0x55c733c80e10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc02f410 tx=0x7f4ddc02f440 comp rx=0 tx=0).stop 2024-09-21T00:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.653+0000 7f4df05cd740 1 -- 172.21.15.27:0/3132614759 shutdown_connections 2024-09-21T00:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.653+0000 7f4df05cd740 1 --2- 172.21.15.27:0/3132614759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338bb4d0 0x55c733c81890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.653+0000 7f4df05cd740 1 --2- 172.21.15.27:0/3132614759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338b32f0 0x55c733c81350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.653+0000 7f4df05cd740 1 --2- 172.21.15.27:0/3132614759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 0x55c733c80e10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.653+0000 7f4df05cd740 1 -- 172.21.15.27:0/3132614759 >> 172.21.15.27:0/3132614759 conn(0x55c7338d0300 msgr2=0x55c733c7ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.653+0000 7f4df05cd740 1 -- 172.21.15.27:0/3132614759 shutdown_connections 2024-09-21T00:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.653+0000 7f4df05cd740 1 -- 172.21.15.27:0/3132614759 wait complete. 2024-09-21T00:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.653+0000 7f4de2a00640 1 --2- 172.21.15.27:0/4274677902 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4dc80b6670 0x7f4dc80b8b80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4dd40e5250 tx=0x7f4dd40eb930 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.654+0000 7f4df05cd740 1 -- start start 2024-09-21T00:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.654+0000 7f4df05cd740 1 Processor -- start 2024-09-21T00:23:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.654+0000 7f4df05cd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338b32f0 0x55c733c8c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.654+0000 7f4df05cd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338bb4d0 0x55c733c8c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.655+0000 7f4df05cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 0x55c733c8ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.655+0000 7f4df05cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c733b1dab0 con 0x55c7338dc1b0 2024-09-21T00:23:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.655+0000 7f4df05cd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c733b8c570 con 0x55c7338bb4d0 2024-09-21T00:23:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.655+0000 7f4df05cd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c733c8f3b0 con 0x55c7338b32f0 2024-09-21T00:23:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.655+0000 7f4df05cd740 10 client.0 _ll_register_callbacks cb 0x55c733b9b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:23:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.655+0000 7f4de2000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338bb4d0 0x55c733c8c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.655+0000 7f4de3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 0x55c733c8ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.655+0000 7f4de2000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338bb4d0 0x55c733c8c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40514/0 (socket says 172.21.15.27:40514) 2024-09-21T00:23:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.655+0000 7f4de2000640 1 -- 172.21.15.27:0/4194103279 learned_addr learned my addr 172.21.15.27:0/4194103279 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.655+0000 7f4de3400640 1 --2- 172.21.15.27:0/4194103279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338b32f0 0x55c733c8c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.655+0000 7f4de2000640 1 -- 172.21.15.27:0/4194103279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338b32f0 msgr2=0x55c733c8c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.655+0000 7f4de2000640 1 --2- 172.21.15.27:0/4194103279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338b32f0 0x55c733c8c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.655+0000 7f4de2000640 1 -- 172.21.15.27:0/4194103279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 msgr2=0x55c733c8ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.655+0000 7f4de2000640 1 --2- 172.21.15.27:0/4194103279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 0x55c733c8ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.655+0000 7f4de2000640 1 -- 172.21.15.27:0/4194103279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c733947790 con 0x55c7338bb4d0 2024-09-21T00:23:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.656+0000 7f4de2000640 1 --2- 172.21.15.27:0/4194103279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338bb4d0 0x55c733c8c8f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0012d20 tx=0x7f4dd000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13e03c117d8ea799 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.656+0000 7f4dc3e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:23:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.656+0000 7f4dc3e00640 1 -- 172.21.15.27:0/4194103279 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0012110 con 0x55c7338bb4d0 2024-09-21T00:23:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.656+0000 7f4df05cd740 1 -- 172.21.15.27:0/4194103279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ddc036030 con 0x55c7338bb4d0 2024-09-21T00:23:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.656+0000 7f4dc3e00640 1 -- 172.21.15.27:0/4194103279 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f4dd0004050 con 0x55c7338bb4d0 2024-09-21T00:23:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.656+0000 7f4dc3e00640 1 -- 172.21.15.27:0/4194103279 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4dd0025050 con 0x55c7338bb4d0 2024-09-21T00:23:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.656+0000 7f4df05cd740 10 client.25745 fetch_fsmap learned FSMap version 26 2024-09-21T00:23:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.656+0000 7f4df05cd740 1 -- 172.21.15.27:0/4194103279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c7339c0620 con 0x55c7338bb4d0 2024-09-21T00:23:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.656+0000 7f4dc3e00640 1 -- 172.21.15.27:0/4194103279 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ddc036030 con 0x55c7338bb4d0 2024-09-21T00:23:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.657+0000 7f4dc3e00640 1 -- 172.21.15.27:0/4194103279 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4dd0010430 con 0x55c7338bb4d0 2024-09-21T00:23:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.657+0000 7f4df05cd740 10 client.25745 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:23:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.657+0000 7f4df05cd740 10 client.25745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:23:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.657+0000 7f4df05cd740 20 client.25745 populate_metadata read hostname 'smithi027' 2024-09-21T00:23:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.657+0000 7f4df05cd740 1 --2- 172.21.15.27:0/4194103279 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c73389ad80 0x55c733c9e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.657+0000 7f4df05cd740 4 client.25745 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:23:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.657+0000 7f4df05cd740 1 -- 172.21.15.27:0/4194103279 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c733bf8600 con 0x55c73389ad80 2024-09-21T00:23:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.657+0000 7f4de3400640 1 --2- 172.21.15.27:0/4194103279 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c73389ad80 0x55c733c9e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.658+0000 7f4de3400640 1 --2- 172.21.15.27:0/4194103279 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c73389ad80 0x55c733c9e360 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6277cfe3c65e6e server_cookie=f817df536e321941 in_seq=0 out_seq=0 2024-09-21T00:23:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.658+0000 7f4dc3e00640 10 client.25745 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.658+0000 7f4dc3e00640 1 -- 172.21.15.27:0/4194103279 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c733bf8600 con 0x55c73389ad80 2024-09-21T00:23:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.658+0000 7f4dc3e00640 10 client.25745 handle_command_reply: tid=0 2024-09-21T00:23:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.658+0000 7f4df05cd740 1 client.25745 shutdown 2024-09-21T00:23:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.659+0000 7f4df05cd740 1 -- 172.21.15.27:0/4194103279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338bb4d0 msgr2=0x55c733c8c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.659+0000 7f4df05cd740 1 --2- 172.21.15.27:0/4194103279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338bb4d0 0x55c733c8c8f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0012d20 tx=0x7f4dd000f790 comp rx=0 tx=0).stop 2024-09-21T00:23:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.659+0000 7f4df05cd740 1 -- 172.21.15.27:0/4194103279 shutdown_connections 2024-09-21T00:23:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.659+0000 7f4df05cd740 1 --2- 172.21.15.27:0/4194103279 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c73389ad80 0x55c733c9e360 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.659+0000 7f4df05cd740 1 --2- 172.21.15.27:0/4194103279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 0x55c733c8ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.659+0000 7f4df05cd740 1 --2- 172.21.15.27:0/4194103279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338bb4d0 0x55c733c8c8f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.659+0000 7f4dc3e00640 0 client.25745 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.659+0000 7f4df05cd740 1 --2- 172.21.15.27:0/4194103279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338b32f0 0x55c733c8c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.659+0000 7f4df05cd740 1 -- 172.21.15.27:0/4194103279 >> 172.21.15.27:0/4194103279 conn(0x55c7338d0300 msgr2=0x55c733c81e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.659+0000 7f4df05cd740 1 -- 172.21.15.27:0/4194103279 shutdown_connections 2024-09-21T00:23:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.659+0000 7f4df05cd740 1 -- 172.21.15.27:0/4194103279 wait complete. 2024-09-21T00:23:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.659+0000 7f4df05cd740 20 client.25745 trim_cache size 0 max 16384 2024-09-21T00:23:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.678+0000 7f4df05cd740 1 Processor -- start 2024-09-21T00:23:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.678+0000 7f4df05cd740 1 -- start start 2024-09-21T00:23:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.678+0000 7f4df05cd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338b32f0 0x55c733c7dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.678+0000 7f4df05cd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338bb4d0 0x55c733c8dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.678+0000 7f4df05cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 0x55c733c89580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.678+0000 7f4df05cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4de41b56f0 con 0x55c7338dc1b0 2024-09-21T00:23:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.678+0000 7f4df05cd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4de41b2fc0 con 0x55c7338bb4d0 2024-09-21T00:23:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.678+0000 7f4df05cd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4de41b0e10 con 0x55c7338b32f0 2024-09-21T00:23:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.679+0000 7f4de3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338b32f0 0x55c733c7dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.679+0000 7f4de3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338b32f0 0x55c733c7dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:47566/0 (socket says 172.21.15.27:47566) 2024-09-21T00:23:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.679+0000 7f4de3400640 1 -- 172.21.15.27:0/914009957 learned_addr learned my addr 172.21.15.27:0/914009957 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.679+0000 7f4de2000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338bb4d0 0x55c733c8dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.679+0000 7f4de3400640 1 --2- 172.21.15.27:0/914009957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 0x55c733c89580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.679+0000 7f4de3400640 1 -- 172.21.15.27:0/914009957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338bb4d0 msgr2=0x55c733c8dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.679+0000 7f4de3400640 1 --2- 172.21.15.27:0/914009957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338bb4d0 0x55c733c8dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.679+0000 7f4de3400640 1 -- 172.21.15.27:0/914009957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 msgr2=0x55c733c89580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.679+0000 7f4de3400640 1 --2- 172.21.15.27:0/914009957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 0x55c733c89580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.679+0000 7f4de3400640 1 -- 172.21.15.27:0/914009957 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c733c83c20 con 0x55c7338b32f0 2024-09-21T00:23:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.679+0000 7f4de2000640 1 --2- 172.21.15.27:0/914009957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338bb4d0 0x55c733c8dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:23:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.680+0000 7f4de3400640 1 --2- 172.21.15.27:0/914009957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338b32f0 0x55c733c7dd20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009a60 tx=0x7f4ddc02f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18bc0655dd9831e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.680+0000 7f4dcea00640 1 -- 172.21.15.27:0/914009957 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc04a040 con 0x55c7338b32f0 2024-09-21T00:23:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.680+0000 7f4dcea00640 1 -- 172.21.15.27:0/914009957 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ddc036030 con 0x55c7338b32f0 2024-09-21T00:23:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.680+0000 7f4dcea00640 1 -- 172.21.15.27:0/914009957 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc004510 con 0x55c7338b32f0 2024-09-21T00:23:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.681+0000 7f4df05cd740 1 -- 172.21.15.27:0/914009957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338b32f0 msgr2=0x55c733c7dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.681+0000 7f4df05cd740 1 --2- 172.21.15.27:0/914009957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338b32f0 0x55c733c7dd20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009a60 tx=0x7f4ddc02f3a0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.681+0000 7f4df05cd740 1 -- 172.21.15.27:0/914009957 shutdown_connections 2024-09-21T00:23:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.681+0000 7f4df05cd740 1 --2- 172.21.15.27:0/914009957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338dc1b0 0x55c733c89580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.681+0000 7f4df05cd740 1 --2- 172.21.15.27:0/914009957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338bb4d0 0x55c733c8dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.681+0000 7f4df05cd740 1 --2- 172.21.15.27:0/914009957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338b32f0 0x55c733c7dd20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.681+0000 7f4df05cd740 1 -- 172.21.15.27:0/914009957 >> 172.21.15.27:0/914009957 conn(0x55c7338d0300 msgr2=0x55c733c8d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.681+0000 7f4df05cd740 1 -- 172.21.15.27:0/914009957 shutdown_connections 2024-09-21T00:23:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.681+0000 7f4df05cd740 1 -- 172.21.15.27:0/914009957 wait complete. 2024-09-21T00:23:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.681+0000 7f4df05cd740 1 -- start start 2024-09-21T00:23:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.682+0000 7f4df05cd740 1 Processor -- start 2024-09-21T00:23:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.682+0000 7f4df05cd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338b32f0 0x55c733c92700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.682+0000 7f4df05cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338bb4d0 0x55c733c92c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.682+0000 7f4df05cd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338dc1b0 0x55c733c93180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.682+0000 7f4df05cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4de41b1480 con 0x55c7338bb4d0 2024-09-21T00:23:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.682+0000 7f4df05cd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4de41b28f0 con 0x55c7338b32f0 2024-09-21T00:23:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.682+0000 7f4df05cd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4de41b1b50 con 0x55c7338dc1b0 2024-09-21T00:23:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.682+0000 7f4df05cd740 10 client.0 _ll_register_callbacks cb 0x7f4dd0009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:23:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.683+0000 7f4de3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338b32f0 0x55c733c92700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.683+0000 7f4de3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338b32f0 0x55c733c92700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40526/0 (socket says 172.21.15.27:40526) 2024-09-21T00:23:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.683+0000 7f4de3400640 1 -- 172.21.15.27:0/1009272130 learned_addr learned my addr 172.21.15.27:0/1009272130 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.683+0000 7f4de2000640 1 --2- 172.21.15.27:0/1009272130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338bb4d0 0x55c733c92c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.683+0000 7f4de3400640 1 --2- 172.21.15.27:0/1009272130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338dc1b0 0x55c733c93180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.683+0000 7f4de2000640 1 -- 172.21.15.27:0/1009272130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338dc1b0 msgr2=0x55c733c93180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.683+0000 7f4de2000640 1 --2- 172.21.15.27:0/1009272130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338dc1b0 0x55c733c93180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.683+0000 7f4de2000640 1 -- 172.21.15.27:0/1009272130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338b32f0 msgr2=0x55c733c92700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.683+0000 7f4de2000640 1 --2- 172.21.15.27:0/1009272130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338b32f0 0x55c733c92700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.683+0000 7f4de2000640 1 -- 172.21.15.27:0/1009272130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c733d07e50 con 0x55c7338bb4d0 2024-09-21T00:23:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.683+0000 7f4de3400640 1 --2- 172.21.15.27:0/1009272130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338b32f0 0x55c733c92700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:23:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.684+0000 7f4de2000640 1 --2- 172.21.15.27:0/1009272130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338bb4d0 0x55c733c92c40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00109f0 tx=0x7f4dd0012340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2c5ecb0d55fa57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.684+0000 7f4dc3e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:23:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.684+0000 7f4dc3e00640 1 -- 172.21.15.27:0/1009272130 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd002d070 con 0x55c7338bb4d0 2024-09-21T00:23:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.684+0000 7f4df05cd740 1 -- 172.21.15.27:0/1009272130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ddc036030 con 0x55c7338bb4d0 2024-09-21T00:23:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.684+0000 7f4dc3e00640 1 -- 172.21.15.27:0/1009272130 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f4dd0018040 con 0x55c7338bb4d0 2024-09-21T00:23:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.684+0000 7f4dc3e00640 1 -- 172.21.15.27:0/1009272130 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4dd0038050 con 0x55c7338bb4d0 2024-09-21T00:23:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.684+0000 7f4df05cd740 10 client.15720 fetch_fsmap learned FSMap version 26 2024-09-21T00:23:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.684+0000 7f4df05cd740 1 -- 172.21.15.27:0/1009272130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c733d090c0 con 0x55c7338bb4d0 2024-09-21T00:23:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.685+0000 7f4dc3e00640 1 -- 172.21.15.27:0/1009272130 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ddc036030 con 0x55c7338bb4d0 2024-09-21T00:23:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.685+0000 7f4dc3e00640 1 -- 172.21.15.27:0/1009272130 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4dd00035f0 con 0x55c7338bb4d0 2024-09-21T00:23:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.685+0000 7f4df05cd740 10 client.15720 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:23:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.685+0000 7f4df05cd740 10 client.15720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:23:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.685+0000 7f4df05cd740 20 client.15720 populate_metadata read hostname 'smithi027' 2024-09-21T00:23:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.685+0000 7f4df05cd740 1 --2- 172.21.15.27:0/1009272130 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c7338701c0 0x55c733d0b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.685+0000 7f4df05cd740 4 client.15720 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:23:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.685+0000 7f4df05cd740 1 -- 172.21.15.27:0/1009272130 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c733bf40b0 con 0x55c7338701c0 2024-09-21T00:23:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.685+0000 7f4de3400640 1 --2- 172.21.15.27:0/1009272130 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c7338701c0 0x55c733d0b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.686+0000 7f4de3400640 1 --2- 172.21.15.27:0/1009272130 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c7338701c0 0x55c733d0b850 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dcb904dc7b000c3 server_cookie=9a61071646aef9e7 in_seq=0 out_seq=0 2024-09-21T00:23:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.686+0000 7f4dc3e00640 10 client.15720 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.724+0000 7f4dc3e00640 1 -- 172.21.15.27:0/1009272130 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c733bf40b0 con 0x55c7338701c0 2024-09-21T00:23:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.724+0000 7f4dc3e00640 10 client.15720 handle_command_reply: tid=0 2024-09-21T00:23:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.724+0000 7f4df05cd740 1 client.15720 shutdown 2024-09-21T00:23:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.724+0000 7f4df05cd740 1 -- 172.21.15.27:0/1009272130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338bb4d0 msgr2=0x55c733c92c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.724+0000 7f4df05cd740 1 --2- 172.21.15.27:0/1009272130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338bb4d0 0x55c733c92c40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00109f0 tx=0x7f4dd0012340 comp rx=0 tx=0).stop 2024-09-21T00:23:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.724+0000 7f4df05cd740 1 -- 172.21.15.27:0/1009272130 shutdown_connections 2024-09-21T00:23:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.724+0000 7f4df05cd740 1 --2- 172.21.15.27:0/1009272130 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c7338701c0 0x55c733d0b850 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.724+0000 7f4df05cd740 1 --2- 172.21.15.27:0/1009272130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7338dc1b0 0x55c733c93180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.724+0000 7f4df05cd740 1 --2- 172.21.15.27:0/1009272130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c7338bb4d0 0x55c733c92c40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.724+0000 7f4dc3e00640 0 client.15720 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.724+0000 7f4df05cd740 1 --2- 172.21.15.27:0/1009272130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7338b32f0 0x55c733c92700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.724+0000 7f4df05cd740 1 -- 172.21.15.27:0/1009272130 >> 172.21.15.27:0/1009272130 conn(0x55c7338d0300 msgr2=0x55c733c7daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.725+0000 7f4df05cd740 1 -- 172.21.15.27:0/1009272130 shutdown_connections 2024-09-21T00:23:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.725+0000 7f4df05cd740 1 -- 172.21.15.27:0/1009272130 wait complete. 2024-09-21T00:23:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.725+0000 7f4df05cd740 20 client.15720 trim_cache size 0 max 16384 2024-09-21T00:23:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:23:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:23:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:23:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:23:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.725+0000 7f4de3e00640 1 -- 172.21.15.27:0/4274677902 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4dc80b6670 msgr2=0x7f4dc80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.725+0000 7f4de3e00640 1 --2- 172.21.15.27:0/4274677902 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4dc80b6670 0x7f4dc80b8b80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4dd40e5250 tx=0x7f4dd40eb930 comp rx=0 tx=0).stop 2024-09-21T00:23:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.725+0000 7f4de3e00640 1 -- 172.21.15.27:0/4274677902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4103920 msgr2=0x7f4de419c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.725+0000 7f4de3e00640 1 --2- 172.21.15.27:0/4274677902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4103920 0x7f4de419c420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400e9f0 tx=0x7f4dd400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.726+0000 7f4de3e00640 1 -- 172.21.15.27:0/4274677902 shutdown_connections 2024-09-21T00:23:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.726+0000 7f4de3e00640 1 --2- 172.21.15.27:0/4274677902 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4dc80b6670 0x7f4dc80b8b80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.726+0000 7f4de3e00640 1 --2- 172.21.15.27:0/4274677902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de4109ba0 0x7f4de41a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.726+0000 7f4de3e00640 1 --2- 172.21.15.27:0/4274677902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4de4109250 0x7f4de419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.726+0000 7f4de3e00640 1 --2- 172.21.15.27:0/4274677902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4103920 0x7f4de419c420 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.726+0000 7f4de3e00640 1 -- 172.21.15.27:0/4274677902 >> 172.21.15.27:0/4274677902 conn(0x7f4de40ff340 msgr2=0x7f4de4108880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.726+0000 7f4de3e00640 1 -- 172.21.15.27:0/4274677902 shutdown_connections 2024-09-21T00:23:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:26.726+0000 7f4de3e00640 1 -- 172.21.15.27:0/4274677902 wait complete. 2024-09-21T00:23:27.251 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether build environment is sane... yes 2024-09-21T00:23:27.268 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gawk... gawk 2024-09-21T00:23:27.293 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether make sets $(MAKE)... yes 2024-09-21T00:23:27.312 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gcc... gcc 2024-09-21T00:23:27.405 INFO:tasks.workunit.client.0.smithi027.stdout:checking for C compiler default output file name... a.out 2024-09-21T00:23:27.409 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether the C compiler works... yes 2024-09-21T00:23:27.412 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether we are cross compiling... no 2024-09-21T00:23:27.473 INFO:tasks.workunit.client.0.smithi027.stdout:checking for suffix of executables... 2024-09-21T00:23:27.537 INFO:tasks.workunit.client.0.smithi027.stdout:checking for suffix of object files... o 2024-09-21T00:23:27.597 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether we are using the GNU C compiler... yes 2024-09-21T00:23:27.659 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether gcc accepts -g... yes 2024-09-21T00:23:27.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:27 smithi089 ceph-mon[34481]: pgmap v439: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-21T00:23:27.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:27 smithi098 ceph-mon[34725]: pgmap v439: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-21T00:23:27.776 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gcc option to accept ANSI C... none needed 2024-09-21T00:23:27.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:27 smithi027 ceph-mon[30745]: pgmap v439: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-21T00:23:27.858 INFO:tasks.workunit.client.0.smithi027.stdout:checking for style of include used by make... GNU 2024-09-21T00:23:27.997 INFO:tasks.workunit.client.0.smithi027.stdout:checking dependency style of gcc... gcc3 2024-09-21T00:23:27.997 INFO:tasks.workunit.client.0.smithi027.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-21T00:23:28.093 INFO:tasks.workunit.client.0.smithi027.stdout:checking how to run the C preprocessor... gcc -E 2024-09-21T00:23:28.172 INFO:tasks.workunit.client.0.smithi027.stdout:checking for egrep... grep -E 2024-09-21T00:23:28.192 INFO:tasks.workunit.client.0.smithi027.stdout:checking for AIX... no 2024-09-21T00:23:28.297 INFO:tasks.workunit.client.0.smithi027.stdout:checking for library containing strerror... none required 2024-09-21T00:23:28.559 INFO:tasks.workunit.client.0.smithi027.stdout:checking for ANSI C header files... yes 2024-09-21T00:23:28.634 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-21T00:23:28.718 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/types.h... yes 2024-09-21T00:23:28.808 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/stat.h... yes 2024-09-21T00:23:28.897 INFO:tasks.workunit.client.0.smithi027.stdout:checking for stdlib.h... yes 2024-09-21T00:23:28.987 INFO:tasks.workunit.client.0.smithi027.stdout:checking for string.h... yes 2024-09-21T00:23:29.085 INFO:tasks.workunit.client.0.smithi027.stdout:checking for memory.h... yes 2024-09-21T00:23:29.177 INFO:tasks.workunit.client.0.smithi027.stdout:checking for strings.h... yes 2024-09-21T00:23:29.268 INFO:tasks.workunit.client.0.smithi027.stdout:checking for inttypes.h... yes 2024-09-21T00:23:29.364 INFO:tasks.workunit.client.0.smithi027.stdout:checking for stdint.h... yes 2024-09-21T00:23:29.460 INFO:tasks.workunit.client.0.smithi027.stdout:checking for unistd.h... yes 2024-09-21T00:23:29.484 INFO:tasks.workunit.client.0.smithi027.stdout:checking for memory.h... (cached) yes 2024-09-21T00:23:29.498 INFO:tasks.workunit.client.0.smithi027.stdout:checking for strings.h... (cached) yes 2024-09-21T00:23:29.513 INFO:tasks.workunit.client.0.smithi027.stdout:checking for unistd.h... (cached) yes 2024-09-21T00:23:29.603 INFO:tasks.workunit.client.0.smithi027.stdout:checking locale.h usability... yes 2024-09-21T00:23:29.641 INFO:tasks.workunit.client.0.smithi027.stdout:checking locale.h presence... yes 2024-09-21T00:23:29.643 INFO:tasks.workunit.client.0.smithi027.stdout:checking for locale.h... yes 2024-09-21T00:23:29.666 INFO:tasks.workunit.client.0.smithi027.stdout:checking for stdlib.h... (cached) yes 2024-09-21T00:23:29.680 INFO:tasks.workunit.client.0.smithi027.stdout:checking for string.h... (cached) yes 2024-09-21T00:23:29.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:29 smithi089 ceph-mon[34481]: pgmap v440: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-21T00:23:29.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:29 smithi098 ceph-mon[34725]: pgmap v440: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-21T00:23:29.802 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/socket.h usability... yes 2024-09-21T00:23:29.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:29 smithi027 ceph-mon[30745]: pgmap v440: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-21T00:23:29.843 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/socket.h presence... yes 2024-09-21T00:23:29.845 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/socket.h... yes 2024-09-21T00:23:29.938 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/time.h usability... yes 2024-09-21T00:23:29.977 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/time.h presence... yes 2024-09-21T00:23:29.979 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/time.h... yes 2024-09-21T00:23:30.073 INFO:tasks.workunit.client.0.smithi027.stdout:checking fcntl.h usability... yes 2024-09-21T00:23:30.111 INFO:tasks.workunit.client.0.smithi027.stdout:checking fcntl.h presence... yes 2024-09-21T00:23:30.113 INFO:tasks.workunit.client.0.smithi027.stdout:checking for fcntl.h... yes 2024-09-21T00:23:30.222 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/fcntl.h usability... yes 2024-09-21T00:23:30.261 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/fcntl.h presence... yes 2024-09-21T00:23:30.262 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/fcntl.h... yes 2024-09-21T00:23:30.396 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/param.h usability... yes 2024-09-21T00:23:30.440 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/param.h presence... yes 2024-09-21T00:23:30.442 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/param.h... yes 2024-09-21T00:23:30.526 INFO:tasks.workunit.client.0.smithi027.stdout:checking ioctl.h usability... no 2024-09-21T00:23:30.568 INFO:tasks.workunit.client.0.smithi027.stdout:checking ioctl.h presence... no 2024-09-21T00:23:30.569 INFO:tasks.workunit.client.0.smithi027.stdout:checking for ioctl.h... no 2024-09-21T00:23:30.658 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/ioctl.h usability... yes 2024-09-21T00:23:30.698 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/ioctl.h presence... yes 2024-09-21T00:23:30.700 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/ioctl.h... yes 2024-09-21T00:23:30.796 INFO:tasks.workunit.client.0.smithi027.stdout:checking return type of signal handlers... void 2024-09-21T00:23:30.895 INFO:tasks.workunit.client.0.smithi027.stdout:checking for size_t... yes 2024-09-21T00:23:31.025 INFO:tasks.workunit.client.0.smithi027.stdout:checking for dev_t... yes 2024-09-21T00:23:31.103 INFO:tasks.workunit.client.0.smithi027.stdout:checking for short... yes 2024-09-21T00:23:31.234 INFO:tasks.workunit.client.0.smithi027.stdout:checking size of short... 2 2024-09-21T00:23:31.319 INFO:tasks.workunit.client.0.smithi027.stdout:checking for int... yes 2024-09-21T00:23:31.433 INFO:tasks.workunit.client.0.smithi027.stdout:checking size of int... 4 2024-09-21T00:23:31.518 INFO:tasks.workunit.client.0.smithi027.stdout:checking for long... yes 2024-09-21T00:23:31.628 INFO:tasks.workunit.client.0.smithi027.stdout:checking size of long... 8 2024-09-21T00:23:31.713 INFO:tasks.workunit.client.0.smithi027.stdout:checking for long long... yes 2024-09-21T00:23:31.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:31 smithi089 ceph-mon[34481]: pgmap v441: 129 pgs: 129 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 102 KiB/s wr, 21 op/s 2024-09-21T00:23:31.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:31 smithi098 ceph-mon[34725]: pgmap v441: 129 pgs: 129 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 102 KiB/s wr, 21 op/s 2024-09-21T00:23:31.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:31 smithi027 ceph-mon[30745]: pgmap v441: 129 pgs: 129 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 102 KiB/s wr, 21 op/s 2024-09-21T00:23:31.828 INFO:tasks.workunit.client.0.smithi027.stdout:checking size of long long... 8 2024-09-21T00:23:31.911 INFO:tasks.workunit.client.0.smithi027.stdout:checking for ssize_t... yes 2024-09-21T00:23:31.989 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sig_atomic_t... yes 2024-09-21T00:23:32.090 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether gcc needs -traditional... no 2024-09-21T00:23:32.154 INFO:tasks.workunit.client.0.smithi027.stdout:checking for an ANSI C-conforming const... yes 2024-09-21T00:23:32.206 INFO:tasks.workunit.client.0.smithi027.stdout:checking for inline... inline 2024-09-21T00:23:32.301 INFO:tasks.workunit.client.0.smithi027.stdout:checking for working alloca.h... yes 2024-09-21T00:23:32.387 INFO:tasks.workunit.client.0.smithi027.stdout:checking for alloca... yes 2024-09-21T00:23:32.495 INFO:tasks.workunit.client.0.smithi027.stdout:checking for working memcmp... yes 2024-09-21T00:23:32.594 INFO:tasks.workunit.client.0.smithi027.stdout:checking for strftime... yes 2024-09-21T00:23:32.721 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-09-21T00:23:32.822 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether stat accepts an empty string... no 2024-09-21T00:23:32.921 INFO:tasks.workunit.client.0.smithi027.stdout:checking for vprintf... yes 2024-09-21T00:23:33.023 INFO:tasks.workunit.client.0.smithi027.stdout:checking for _doprnt... no 2024-09-21T00:23:33.075 INFO:tasks.workunit.client.0.smithi027.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-21T00:23:33.105 INFO:tasks.workunit.client.0.smithi027.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-21T00:23:33.222 INFO:tasks.workunit.client.0.smithi027.stdout:checking for the pthreads library -lpthreads... no 2024-09-21T00:23:33.321 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether pthreads work without any flags... yes 2024-09-21T00:23:33.407 INFO:tasks.workunit.client.0.smithi027.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-09-21T00:23:33.407 INFO:tasks.workunit.client.0.smithi027.stdout:checking if more special flags are required for pthreads... no 2024-09-21T00:23:33.407 INFO:tasks.workunit.client.0.smithi027.stdout:checking for cc_r... gcc 2024-09-21T00:23:33.525 INFO:tasks.workunit.client.0.smithi027.stdout:checking for setrlimit... yes 2024-09-21T00:23:33.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:33 smithi027 ceph-mon[30745]: pgmap v442: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 120 KiB/s wr, 29 op/s 2024-09-21T00:23:33.633 INFO:tasks.workunit.client.0.smithi027.stdout:checking for setproctitle... no 2024-09-21T00:23:33.737 INFO:tasks.workunit.client.0.smithi027.stdout:checking for getopt_long... yes 2024-09-21T00:23:33.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:33 smithi098 ceph-mon[34725]: pgmap v442: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 120 KiB/s wr, 29 op/s 2024-09-21T00:23:33.846 INFO:tasks.workunit.client.0.smithi027.stdout:checking for strtoull... yes 2024-09-21T00:23:33.957 INFO:tasks.workunit.client.0.smithi027.stdout:checking for strtoq... yes 2024-09-21T00:23:33.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:33 smithi089 ceph-mon[34481]: pgmap v442: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 120 KiB/s wr, 29 op/s 2024-09-21T00:23:34.068 INFO:tasks.workunit.client.0.smithi027.stdout:checking for snprintf... yes 2024-09-21T00:23:34.184 INFO:tasks.workunit.client.0.smithi027.stdout:checking for vsnprintf... yes 2024-09-21T00:23:34.294 INFO:tasks.workunit.client.0.smithi027.stdout:checking for vfprintf... yes 2024-09-21T00:23:34.405 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gethostname... yes 2024-09-21T00:23:34.516 INFO:tasks.workunit.client.0.smithi027.stdout:checking for setlocale... yes 2024-09-21T00:23:34.626 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sync... yes 2024-09-21T00:23:34.740 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether snprintf is C99 conformant... done 2024-09-21T00:23:34.762 INFO:tasks.workunit.client.0.smithi027.stdout:configure: creating ./config.status 2024-09-21T00:23:35.060 INFO:tasks.workunit.client.0.smithi027.stdout:config.status: creating Makefile 2024-09-21T00:23:35.100 INFO:tasks.workunit.client.0.smithi027.stdout:config.status: creating src/Makefile 2024-09-21T00:23:35.141 INFO:tasks.workunit.client.0.smithi027.stdout:config.status: creating man/Makefile 2024-09-21T00:23:35.162 INFO:tasks.workunit.client.0.smithi027.stdout:config.status: creating config.h 2024-09-21T00:23:35.253 INFO:tasks.workunit.client.0.smithi027.stdout:config.status: executing depfiles commands 2024-09-21T00:23:35.543 INFO:tasks.workunit.client.0.smithi027.stderr:+ make 2024-09-21T00:23:35.556 INFO:tasks.workunit.client.0.smithi027.stdout:make all-recursive 2024-09-21T00:23:35.559 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-21T00:23:35.565 INFO:tasks.workunit.client.0.smithi027.stdout:Making all in src 2024-09-21T00:23:35.575 INFO:tasks.workunit.client.0.smithi027.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-21T00:23:35.575 INFO:tasks.workunit.client.0.smithi027.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-09-21T00:23:35.575 INFO:tasks.workunit.client.0.smithi027.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-09-21T00:23:35.716 INFO:tasks.workunit.client.0.smithi027.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-09-21T00:23:35.717 INFO:tasks.workunit.client.0.smithi027.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-09-21T00:23:35.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:35 smithi098 ceph-mon[34725]: pgmap v443: 129 pgs: 129 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 179 KiB/s wr, 35 op/s 2024-09-21T00:23:35.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:35 smithi027 ceph-mon[30745]: pgmap v443: 129 pgs: 129 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 179 KiB/s wr, 35 op/s 2024-09-21T00:23:35.821 INFO:tasks.workunit.client.0.smithi027.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-09-21T00:23:35.821 INFO:tasks.workunit.client.0.smithi027.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-09-21T00:23:35.903 INFO:tasks.workunit.client.0.smithi027.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-09-21T00:23:35.903 INFO:tasks.workunit.client.0.smithi027.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-09-21T00:23:35.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:35 smithi089 ceph-mon[34481]: pgmap v443: 129 pgs: 129 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 179 KiB/s wr, 35 op/s 2024-09-21T00:23:35.984 INFO:tasks.workunit.client.0.smithi027.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-09-21T00:23:35.984 INFO:tasks.workunit.client.0.smithi027.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-09-21T00:23:36.066 INFO:tasks.workunit.client.0.smithi027.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-09-21T00:23:36.066 INFO:tasks.workunit.client.0.smithi027.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-09-21T00:23:36.141 INFO:tasks.workunit.client.0.smithi027.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-09-21T00:23:36.142 INFO:tasks.workunit.client.0.smithi027.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-09-21T00:23:36.249 INFO:tasks.workunit.client.0.smithi027.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-09-21T00:23:36.249 INFO:tasks.workunit.client.0.smithi027.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-09-21T00:23:36.308 INFO:tasks.workunit.client.0.smithi027.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-09-21T00:23:36.308 INFO:tasks.workunit.client.0.smithi027.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-09-21T00:23:36.370 INFO:tasks.workunit.client.0.smithi027.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-09-21T00:23:36.370 INFO:tasks.workunit.client.0.smithi027.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-09-21T00:23:36.437 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-09-21T00:23:36.469 INFO:tasks.workunit.client.0.smithi027.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-21T00:23:36.470 INFO:tasks.workunit.client.0.smithi027.stdout:Making all in man 2024-09-21T00:23:36.498 INFO:tasks.workunit.client.0.smithi027.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-21T00:23:36.498 INFO:tasks.workunit.client.0.smithi027.stdout:make[2]: Nothing to be done for 'all'. 2024-09-21T00:23:36.498 INFO:tasks.workunit.client.0.smithi027.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-21T00:23:36.501 INFO:tasks.workunit.client.0.smithi027.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-21T00:23:36.503 INFO:tasks.workunit.client.0.smithi027.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-21T00:23:36.504 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-21T00:23:36.505 INFO:tasks.workunit.client.0.smithi027.stderr:+ cd src 2024-09-21T00:23:36.505 INFO:tasks.workunit.client.0.smithi027.stderr:+ mkdir blogtest_in 2024-09-21T00:23:36.509 INFO:tasks.workunit.client.0.smithi027.stderr:+ echo 'running blogbench' 2024-09-21T00:23:36.510 INFO:tasks.workunit.client.0.smithi027.stdout:running blogbench 2024-09-21T00:23:36.510 INFO:tasks.workunit.client.0.smithi027.stderr:+ ./blogbench -d blogtest_in 2024-09-21T00:23:36.511 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T00:23:36.511 INFO:tasks.workunit.client.0.smithi027.stdout:Frequency = 10 secs 2024-09-21T00:23:36.511 INFO:tasks.workunit.client.0.smithi027.stdout:Scratch dir = [blogtest_in] 2024-09-21T00:23:37.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:37 smithi098 ceph-mon[34725]: pgmap v444: 129 pgs: 129 active+clean; 5.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 321 KiB/s wr, 43 op/s 2024-09-21T00:23:37.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:37 smithi027 ceph-mon[30745]: pgmap v444: 129 pgs: 129 active+clean; 5.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 321 KiB/s wr, 43 op/s 2024-09-21T00:23:37.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:37 smithi089 ceph-mon[34481]: pgmap v444: 129 pgs: 129 active+clean; 5.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 321 KiB/s wr, 43 op/s 2024-09-21T00:23:38.271 INFO:tasks.workunit.client.0.smithi027.stdout:Spawning 3 writers... 2024-09-21T00:23:38.271 INFO:tasks.workunit.client.0.smithi027.stdout:Spawning 1 rewriters... 2024-09-21T00:23:38.271 INFO:tasks.workunit.client.0.smithi027.stdout:Spawning 5 commenters... 2024-09-21T00:23:38.271 INFO:tasks.workunit.client.0.smithi027.stdout:Spawning 100 readers... 2024-09-21T00:23:38.278 INFO:tasks.workunit.client.0.smithi027.stdout:Benchmarking for 30 iterations. 2024-09-21T00:23:38.278 INFO:tasks.workunit.client.0.smithi027.stdout:The test will run during 5 minutes. 2024-09-21T00:23:38.278 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T00:23:38.278 INFO:tasks.workunit.client.0.smithi027.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-09-21T00:23:39.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:39 smithi098 ceph-mon[34725]: pgmap v445: 129 pgs: 129 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 378 KiB/s wr, 57 op/s 2024-09-21T00:23:39.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:39 smithi027 ceph-mon[30745]: pgmap v445: 129 pgs: 129 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 378 KiB/s wr, 57 op/s 2024-09-21T00:23:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.925+0000 7fca19c00640 1 -- 172.21.15.27:0/3221442538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca141093e0 msgr2=0x7fca141097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.925+0000 7fca19c00640 1 --2- 172.21.15.27:0/3221442538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca141093e0 0x7fca141097f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca00009900 tx=0x7fca0002a3f0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.925+0000 7fca19c00640 1 -- 172.21.15.27:0/3221442538 shutdown_connections 2024-09-21T00:23:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.925+0000 7fca19c00640 1 --2- 172.21.15.27:0/3221442538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca14109d30 0x7fca14110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.925+0000 7fca19c00640 1 --2- 172.21.15.27:0/3221442538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca141093e0 0x7fca141097f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.925+0000 7fca19c00640 1 --2- 172.21.15.27:0/3221442538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca14103ab0 0x7fca14103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.925+0000 7fca19c00640 1 -- 172.21.15.27:0/3221442538 >> 172.21.15.27:0/3221442538 conn(0x7fca140ff4f0 msgr2=0x7fca14101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.926+0000 7fca19c00640 1 -- 172.21.15.27:0/3221442538 shutdown_connections 2024-09-21T00:23:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.926+0000 7fca19c00640 1 -- 172.21.15.27:0/3221442538 wait complete. 2024-09-21T00:23:39.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.927+0000 7fca19c00640 1 Processor -- start 2024-09-21T00:23:39.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.927+0000 7fca19c00640 1 -- start start 2024-09-21T00:23:39.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.927+0000 7fca19c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca14103ab0 0x7fca14075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.927+0000 7fca19c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca141093e0 0x7fca14075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.927+0000 7fca19c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca14109d30 0x7fca14076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.927+0000 7fca19c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca14113a40 con 0x7fca14103ab0 2024-09-21T00:23:39.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.927+0000 7fca19c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fca141138c0 con 0x7fca14109d30 2024-09-21T00:23:39.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.927+0000 7fca19c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fca14113bc0 con 0x7fca141093e0 2024-09-21T00:23:39.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.927+0000 7fca12a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca14103ab0 0x7fca14075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.928+0000 7fca13400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca14109d30 0x7fca14076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.928+0000 7fca12a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca14103ab0 0x7fca14075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56568/0 (socket says 172.21.15.27:56568) 2024-09-21T00:23:39.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.928+0000 7fca12a00640 1 -- 172.21.15.27:0/1125701788 learned_addr learned my addr 172.21.15.27:0/1125701788 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:39.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.928+0000 7fca12000640 1 --2- 172.21.15.27:0/1125701788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca141093e0 0x7fca14075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.928+0000 7fca12a00640 1 -- 172.21.15.27:0/1125701788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca141093e0 msgr2=0x7fca14075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.928+0000 7fca12a00640 1 --2- 172.21.15.27:0/1125701788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca141093e0 0x7fca14075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.928+0000 7fca12a00640 1 -- 172.21.15.27:0/1125701788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca14109d30 msgr2=0x7fca14076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.928+0000 7fca12a00640 1 --2- 172.21.15.27:0/1125701788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca14109d30 0x7fca14076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.928+0000 7fca12a00640 1 -- 172.21.15.27:0/1125701788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca141a86f0 con 0x7fca14103ab0 2024-09-21T00:23:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.928+0000 7fca13400640 1 --2- 172.21.15.27:0/1125701788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca14109d30 0x7fca14076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:23:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.928+0000 7fca12a00640 1 --2- 172.21.15.27:0/1125701788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca14103ab0 0x7fca14075710 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fca0400e9f0 tx=0x7fca0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.929+0000 7fca0b400640 1 -- 172.21.15.27:0/1125701788 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0400cd80 con 0x7fca14103ab0 2024-09-21T00:23:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.929+0000 7fca19c00640 1 -- 172.21.15.27:0/1125701788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca141a89e0 con 0x7fca14103ab0 2024-09-21T00:23:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.929+0000 7fca0b400640 1 -- 172.21.15.27:0/1125701788 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca040108e0 con 0x7fca14103ab0 2024-09-21T00:23:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.929+0000 7fca0b400640 1 -- 172.21.15.27:0/1125701788 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04018980 con 0x7fca14103ab0 2024-09-21T00:23:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.929+0000 7fca19c00640 1 -- 172.21.15.27:0/1125701788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca141a8f50 con 0x7fca14103ab0 2024-09-21T00:23:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.933+0000 7fca20c12740 1 Processor -- start 2024-09-21T00:23:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.933+0000 7fca20c12740 1 -- start start 2024-09-21T00:23:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.933+0000 7fca20c12740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca20c12740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91002f0 0x5626c94ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca20c12740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 0x5626c94ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca20c12740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5626c94436b0 con 0x5626c91084d0 2024-09-21T00:23:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca20c12740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5626c9432ae0 con 0x5626c91291b0 2024-09-21T00:23:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca20c12740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5626c94368f0 con 0x5626c91002f0 2024-09-21T00:23:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca13400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca13400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:48566/0 (socket says 172.21.15.27:48566) 2024-09-21T00:23:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca13400640 1 -- 172.21.15.27:0/2824698250 learned_addr learned my addr 172.21.15.27:0/2824698250 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca12000640 1 --2- 172.21.15.27:0/2824698250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 0x5626c94ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca13400640 1 -- 172.21.15.27:0/2824698250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91002f0 msgr2=0x5626c94ce170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:23:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca13400640 1 --2- 172.21.15.27:0/2824698250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91002f0 0x5626c94ce170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca13400640 1 -- 172.21.15.27:0/2824698250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 msgr2=0x5626c94ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca13400640 1 --2- 172.21.15.27:0/2824698250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 0x5626c94ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca13400640 1 -- 172.21.15.27:0/2824698250 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c93ff780 con 0x5626c91291b0 2024-09-21T00:23:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca12000640 1 --2- 172.21.15.27:0/2824698250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 0x5626c94ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:23:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.934+0000 7fca0b400640 1 -- 172.21.15.27:0/1125701788 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fca04010430 con 0x7fca14103ab0 2024-09-21T00:23:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.935+0000 7fca13400640 1 --2- 172.21.15.27:0/2824698250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94cdc30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fca0c010780 tx=0x7fca0c010c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a70735f8f90223 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.935+0000 7fc9ffe00640 1 -- 172.21.15.27:0/2824698250 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c00ead0 con 0x5626c91291b0 2024-09-21T00:23:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.935+0000 7fc9ffe00640 1 -- 172.21.15.27:0/2824698250 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca0c010e90 con 0x5626c91291b0 2024-09-21T00:23:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.935+0000 7fc9ffe00640 1 -- 172.21.15.27:0/2824698250 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c011640 con 0x5626c91291b0 2024-09-21T00:23:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.936+0000 7fca20c12740 1 -- 172.21.15.27:0/2824698250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 msgr2=0x5626c94cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.936+0000 7fca20c12740 1 --2- 172.21.15.27:0/2824698250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94cdc30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fca0c010780 tx=0x7fca0c010c40 comp rx=0 tx=0).stop 2024-09-21T00:23:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.936+0000 7fca20c12740 1 -- 172.21.15.27:0/2824698250 shutdown_connections 2024-09-21T00:23:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.936+0000 7fca20c12740 1 --2- 172.21.15.27:0/2824698250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 0x5626c94ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.936+0000 7fca20c12740 1 --2- 172.21.15.27:0/2824698250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91002f0 0x5626c94ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.936+0000 7fca20c12740 1 --2- 172.21.15.27:0/2824698250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94cdc30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.936+0000 7fca20c12740 1 -- 172.21.15.27:0/2824698250 >> 172.21.15.27:0/2824698250 conn(0x5626c911d300 msgr2=0x5626c94cbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.936+0000 7fca20c12740 1 -- 172.21.15.27:0/2824698250 shutdown_connections 2024-09-21T00:23:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.936+0000 7fca20c12740 1 -- 172.21.15.27:0/2824698250 wait complete. 2024-09-21T00:23:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.936+0000 7fca0b400640 1 --2- 172.21.15.27:0/1125701788 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc9f80b65c0 0x7fc9f80b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.936+0000 7fca0b400640 1 -- 172.21.15.27:0/1125701788 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fca040e7d70 con 0x7fca14103ab0 2024-09-21T00:23:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.936+0000 7fca12000640 1 --2- 172.21.15.27:0/1125701788 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc9f80b65c0 0x7fc9f80b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.936+0000 7fca20c12740 1 -- start start 2024-09-21T00:23:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.937+0000 7fca20c12740 1 Processor -- start 2024-09-21T00:23:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.937+0000 7fca12000640 1 --2- 172.21.15.27:0/1125701788 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc9f80b65c0 0x7fc9f80b8ad0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fca0002a9e0 tx=0x7fca00005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.937+0000 7fca20c12740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91002f0 0x5626c94d92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.937+0000 7fca20c12740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 0x5626c94d9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.937+0000 7fca20c12740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91291b0 0x5626c94e5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.937+0000 7fca20c12740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5626c936aab0 con 0x5626c91084d0 2024-09-21T00:23:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.937+0000 7fca20c12740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5626c93d9570 con 0x5626c91002f0 2024-09-21T00:23:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.937+0000 7fca20c12740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5626c94dc3b0 con 0x5626c91291b0 2024-09-21T00:23:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.937+0000 7fca20c12740 10 client.0 _ll_register_callbacks cb 0x5626c93e8100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:23:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.938+0000 7fca12a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91002f0 0x5626c94d92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.938+0000 7fca12a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91002f0 0x5626c94d92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:48568/0 (socket says 172.21.15.27:48568) 2024-09-21T00:23:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.938+0000 7fca12a00640 1 -- 172.21.15.27:0/2159851566 learned_addr learned my addr 172.21.15.27:0/2159851566 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.938+0000 7fca12000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 0x5626c94d9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.938+0000 7fca13400640 1 --2- 172.21.15.27:0/2159851566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91291b0 0x5626c94e5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.938+0000 7fca12a00640 1 -- 172.21.15.27:0/2159851566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91291b0 msgr2=0x5626c94e5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.938+0000 7fca12a00640 1 --2- 172.21.15.27:0/2159851566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91291b0 0x5626c94e5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.938+0000 7fca12a00640 1 -- 172.21.15.27:0/2159851566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 msgr2=0x5626c94d9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.938+0000 7fca12a00640 1 --2- 172.21.15.27:0/2159851566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 0x5626c94d9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.938+0000 7fca12a00640 1 -- 172.21.15.27:0/2159851566 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c9194790 con 0x5626c91002f0 2024-09-21T00:23:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.938+0000 7fca12a00640 1 --2- 172.21.15.27:0/2159851566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91002f0 0x5626c94d92e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fca040e5190 tx=0x7fca040ad2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5918235b1b132b4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.938+0000 7fc9fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:23:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.939+0000 7fc9fcc00640 1 -- 172.21.15.27:0/2159851566 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca040ad3f0 con 0x5626c91002f0 2024-09-21T00:23:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.939+0000 7fc9fcc00640 1 -- 172.21.15.27:0/2159851566 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fca040a9230 con 0x5626c91002f0 2024-09-21T00:23:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.939+0000 7fca20c12740 1 -- 172.21.15.27:0/2159851566 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca0c010e90 con 0x5626c91002f0 2024-09-21T00:23:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.939+0000 7fc9fcc00640 1 -- 172.21.15.27:0/2159851566 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca040b2310 con 0x5626c91002f0 2024-09-21T00:23:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.939+0000 7fca20c12740 10 client.25763 fetch_fsmap learned FSMap version 26 2024-09-21T00:23:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.939+0000 7fca20c12740 1 -- 172.21.15.27:0/2159851566 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626c920d620 con 0x5626c91002f0 2024-09-21T00:23:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.939+0000 7fc9fcc00640 1 -- 172.21.15.27:0/2159851566 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca0c010e90 con 0x5626c91002f0 2024-09-21T00:23:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.939+0000 7fc9fcc00640 1 -- 172.21.15.27:0/2159851566 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fca040b34a0 con 0x5626c91002f0 2024-09-21T00:23:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.940+0000 7fca20c12740 10 client.25763 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:23:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.940+0000 7fca20c12740 10 client.25763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:23:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.940+0000 7fca20c12740 20 client.25763 populate_metadata read hostname 'smithi027' 2024-09-21T00:23:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.940+0000 7fca20c12740 1 --2- 172.21.15.27:0/2159851566 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5626c90e7d80 0x5626c94eb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.940+0000 7fca20c12740 4 client.25763 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:23:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.940+0000 7fca20c12740 1 -- 172.21.15.27:0/2159851566 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626c9445600 con 0x5626c90e7d80 2024-09-21T00:23:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.940+0000 7fca12000640 1 --2- 172.21.15.27:0/2159851566 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5626c90e7d80 0x5626c94eb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.940+0000 7fc9fcc00640 10 client.25763 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.940+0000 7fca12000640 1 --2- 172.21.15.27:0/2159851566 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5626c90e7d80 0x5626c94eb570 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42f0f9a62969673f server_cookie=5792b36fe23b8ecd in_seq=0 out_seq=0 2024-09-21T00:23:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.942+0000 7fc9fcc00640 1 -- 172.21.15.27:0/2159851566 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5626c9445600 con 0x5626c90e7d80 2024-09-21T00:23:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.943+0000 7fc9fcc00640 10 client.25763 handle_command_reply: tid=0 2024-09-21T00:23:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.943+0000 7fca20c12740 1 client.25763 shutdown 2024-09-21T00:23:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.943+0000 7fca20c12740 1 -- 172.21.15.27:0/2159851566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91002f0 msgr2=0x5626c94d92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.943+0000 7fca20c12740 1 --2- 172.21.15.27:0/2159851566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91002f0 0x5626c94d92e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fca040e5190 tx=0x7fca040ad2f0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.943+0000 7fca20c12740 1 -- 172.21.15.27:0/2159851566 shutdown_connections 2024-09-21T00:23:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.943+0000 7fca20c12740 1 --2- 172.21.15.27:0/2159851566 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5626c90e7d80 0x5626c94eb570 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.943+0000 7fca20c12740 1 --2- 172.21.15.27:0/2159851566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91291b0 0x5626c94e5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.943+0000 7fca20c12740 1 --2- 172.21.15.27:0/2159851566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 0x5626c94d9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.943+0000 7fca20c12740 1 --2- 172.21.15.27:0/2159851566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91002f0 0x5626c94d92e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.943+0000 7fca20c12740 1 -- 172.21.15.27:0/2159851566 >> 172.21.15.27:0/2159851566 conn(0x5626c911d300 msgr2=0x5626c94cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.943+0000 7fc9fcc00640 0 client.25763 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.943+0000 7fca20c12740 1 -- 172.21.15.27:0/2159851566 shutdown_connections 2024-09-21T00:23:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.943+0000 7fca20c12740 1 -- 172.21.15.27:0/2159851566 wait complete. 2024-09-21T00:23:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.943+0000 7fca20c12740 20 client.25763 trim_cache size 0 max 16384 2024-09-21T00:23:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.959+0000 7fca20c12740 1 Processor -- start 2024-09-21T00:23:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.959+0000 7fca20c12740 1 -- start start 2024-09-21T00:23:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.959+0000 7fca20c12740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91002f0 0x5626c94cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca20c12740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 0x5626c94e56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca20c12740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca20c12740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca141b5970 con 0x5626c91084d0 2024-09-21T00:23:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca20c12740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fca141b3240 con 0x5626c91291b0 2024-09-21T00:23:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca20c12740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fca141b1090 con 0x5626c91002f0 2024-09-21T00:23:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca12000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 0x5626c94e56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca13400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca12a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91002f0 0x5626c94cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca13400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:48580/0 (socket says 172.21.15.27:48580) 2024-09-21T00:23:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca13400640 1 -- 172.21.15.27:0/1953493790 learned_addr learned my addr 172.21.15.27:0/1953493790 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca13400640 1 -- 172.21.15.27:0/1953493790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91002f0 msgr2=0x5626c94cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca13400640 1 --2- 172.21.15.27:0/1953493790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91002f0 0x5626c94cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca13400640 1 -- 172.21.15.27:0/1953493790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 msgr2=0x5626c94e56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca13400640 1 --2- 172.21.15.27:0/1953493790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 0x5626c94e56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca13400640 1 -- 172.21.15.27:0/1953493790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c94cff30 con 0x5626c91291b0 2024-09-21T00:23:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.960+0000 7fca12000640 1 --2- 172.21.15.27:0/1953493790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 0x5626c94e56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:23:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.961+0000 7fca13400640 1 --2- 172.21.15.27:0/1953493790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94cb0f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fca0c006010 tx=0x7fca0c0049e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dff51c1daa40f0e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.961+0000 7fc9ff400640 1 -- 172.21.15.27:0/1953493790 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c012e60 con 0x5626c91291b0 2024-09-21T00:23:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.961+0000 7fc9ff400640 1 -- 172.21.15.27:0/1953493790 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca0c02c070 con 0x5626c91291b0 2024-09-21T00:23:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.961+0000 7fc9ff400640 1 -- 172.21.15.27:0/1953493790 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c0127c0 con 0x5626c91291b0 2024-09-21T00:23:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.962+0000 7fca20c12740 1 -- 172.21.15.27:0/1953493790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 msgr2=0x5626c94cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.962+0000 7fca20c12740 1 --2- 172.21.15.27:0/1953493790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94cb0f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fca0c006010 tx=0x7fca0c0049e0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.962+0000 7fca20c12740 1 -- 172.21.15.27:0/1953493790 shutdown_connections 2024-09-21T00:23:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.962+0000 7fca20c12740 1 --2- 172.21.15.27:0/1953493790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94cb0f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.962+0000 7fca20c12740 1 --2- 172.21.15.27:0/1953493790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91084d0 0x5626c94e56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.962+0000 7fca20c12740 1 --2- 172.21.15.27:0/1953493790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91002f0 0x5626c94cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.962+0000 7fca20c12740 1 -- 172.21.15.27:0/1953493790 >> 172.21.15.27:0/1953493790 conn(0x5626c911d300 msgr2=0x5626c94e4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.962+0000 7fca20c12740 1 -- 172.21.15.27:0/1953493790 shutdown_connections 2024-09-21T00:23:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.962+0000 7fca20c12740 1 -- 172.21.15.27:0/1953493790 wait complete. 2024-09-21T00:23:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.962+0000 7fca20c12740 1 -- start start 2024-09-21T00:23:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.963+0000 7fca20c12740 1 Processor -- start 2024-09-21T00:23:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.963+0000 7fca20c12740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91002f0 0x5626c94df620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.963+0000 7fca20c12740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91084d0 0x5626c94dfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.963+0000 7fca20c12740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94e00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.963+0000 7fca20c12740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca141b1700 con 0x5626c91002f0 2024-09-21T00:23:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.963+0000 7fca20c12740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fca141b2b70 con 0x5626c91291b0 2024-09-21T00:23:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.963+0000 7fca20c12740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fca141b1dd0 con 0x5626c91084d0 2024-09-21T00:23:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.963+0000 7fca20c12740 10 client.0 _ll_register_callbacks cb 0x7fca04009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:23:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.963+0000 7fca12a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91002f0 0x5626c94df620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.963+0000 7fca13400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94e00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.963+0000 7fca12a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91002f0 0x5626c94df620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56592/0 (socket says 172.21.15.27:56592) 2024-09-21T00:23:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.963+0000 7fca12a00640 1 -- 172.21.15.27:0/2239596439 learned_addr learned my addr 172.21.15.27:0/2239596439 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.963+0000 7fca12000640 1 --2- 172.21.15.27:0/2239596439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91084d0 0x5626c94dfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.964+0000 7fca12a00640 1 -- 172.21.15.27:0/2239596439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91084d0 msgr2=0x5626c94dfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.964+0000 7fca12a00640 1 --2- 172.21.15.27:0/2239596439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91084d0 0x5626c94dfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.964+0000 7fca12a00640 1 -- 172.21.15.27:0/2239596439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 msgr2=0x5626c94e00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.964+0000 7fca12a00640 1 --2- 172.21.15.27:0/2239596439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94e00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.964+0000 7fca12a00640 1 -- 172.21.15.27:0/2239596439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c9554f40 con 0x5626c91002f0 2024-09-21T00:23:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.964+0000 7fca12000640 1 --2- 172.21.15.27:0/2239596439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91084d0 0x5626c94dfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:23:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.964+0000 7fca13400640 1 --2- 172.21.15.27:0/2239596439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94e00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:23:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.964+0000 7fca12a00640 1 --2- 172.21.15.27:0/2239596439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91002f0 0x5626c94df620 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fca040a9eb0 tx=0x7fca0400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9bd0ce16e6204b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.964+0000 7fc9fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:23:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.964+0000 7fc9fcc00640 1 -- 172.21.15.27:0/2239596439 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca040ad3f0 con 0x5626c91002f0 2024-09-21T00:23:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.964+0000 7fca20c12740 1 -- 172.21.15.27:0/2239596439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca0c02c070 con 0x5626c91002f0 2024-09-21T00:23:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.964+0000 7fc9fcc00640 1 -- 172.21.15.27:0/2239596439 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fca040a9230 con 0x5626c91002f0 2024-09-21T00:23:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.965+0000 7fc9fcc00640 1 -- 172.21.15.27:0/2239596439 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca040b7250 con 0x5626c91002f0 2024-09-21T00:23:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.965+0000 7fca20c12740 10 client.15738 fetch_fsmap learned FSMap version 26 2024-09-21T00:23:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.965+0000 7fca20c12740 1 -- 172.21.15.27:0/2239596439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626c95561d0 con 0x5626c91002f0 2024-09-21T00:23:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.965+0000 7fc9fcc00640 1 -- 172.21.15.27:0/2239596439 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca0c02c070 con 0x5626c91002f0 2024-09-21T00:23:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.965+0000 7fc9fcc00640 1 -- 172.21.15.27:0/2239596439 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fca040c39e0 con 0x5626c91002f0 2024-09-21T00:23:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.965+0000 7fca20c12740 10 client.15738 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:23:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.965+0000 7fca20c12740 10 client.15738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:23:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.965+0000 7fca20c12740 20 client.15738 populate_metadata read hostname 'smithi027' 2024-09-21T00:23:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.965+0000 7fca20c12740 1 --2- 172.21.15.27:0/2239596439 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5626c90bd1c0 0x5626c95589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.965+0000 7fca20c12740 4 client.15738 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:23:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.965+0000 7fca20c12740 1 -- 172.21.15.27:0/2239596439 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626c94410b0 con 0x5626c90bd1c0 2024-09-21T00:23:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.965+0000 7fca12000640 1 --2- 172.21.15.27:0/2239596439 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5626c90bd1c0 0x5626c95589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.966+0000 7fca12000640 1 --2- 172.21.15.27:0/2239596439 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5626c90bd1c0 0x5626c95589c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=497467914ca674db server_cookie=94946431622caec2 in_seq=0 out_seq=0 2024-09-21T00:23:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:39.966+0000 7fc9fcc00640 10 client.15738 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:39.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:39 smithi089 ceph-mon[34481]: pgmap v445: 129 pgs: 129 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 378 KiB/s wr, 57 op/s 2024-09-21T00:23:41.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:41 smithi098 ceph-mon[34725]: pgmap v446: 129 pgs: 129 active+clean; 6.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 496 KiB/s wr, 59 op/s 2024-09-21T00:23:41.780 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:41 smithi089 ceph-mon[34481]: pgmap v446: 129 pgs: 129 active+clean; 6.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 496 KiB/s wr, 59 op/s 2024-09-21T00:23:41.824 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:41 smithi027 ceph-mon[30745]: pgmap v446: 129 pgs: 129 active+clean; 6.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 496 KiB/s wr, 59 op/s 2024-09-21T00:23:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.047+0000 7fc9fcc00640 1 -- 172.21.15.27:0/2239596439 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5626c94410b0 con 0x5626c90bd1c0 2024-09-21T00:23:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.047+0000 7fc9fcc00640 10 client.15738 handle_command_reply: tid=0 2024-09-21T00:23:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.047+0000 7fca20c12740 1 client.15738 shutdown 2024-09-21T00:23:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.048+0000 7fca20c12740 1 -- 172.21.15.27:0/2239596439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91002f0 msgr2=0x5626c94df620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.048+0000 7fca20c12740 1 --2- 172.21.15.27:0/2239596439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91002f0 0x5626c94df620 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fca040a9eb0 tx=0x7fca0400cf20 comp rx=0 tx=0).stop 2024-09-21T00:23:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.048+0000 7fca20c12740 1 -- 172.21.15.27:0/2239596439 shutdown_connections 2024-09-21T00:23:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.048+0000 7fca20c12740 1 --2- 172.21.15.27:0/2239596439 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5626c90bd1c0 0x5626c95589c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.048+0000 7fca20c12740 1 --2- 172.21.15.27:0/2239596439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626c91291b0 0x5626c94e00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.048+0000 7fca20c12740 1 --2- 172.21.15.27:0/2239596439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626c91084d0 0x5626c94dfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.048+0000 7fc9fcc00640 0 client.15738 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.048+0000 7fca20c12740 1 --2- 172.21.15.27:0/2239596439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626c91002f0 0x5626c94df620 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.048+0000 7fca20c12740 1 -- 172.21.15.27:0/2239596439 >> 172.21.15.27:0/2239596439 conn(0x5626c911d300 msgr2=0x5626c94e5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.048+0000 7fca20c12740 1 -- 172.21.15.27:0/2239596439 shutdown_connections 2024-09-21T00:23:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.048+0000 7fca20c12740 1 -- 172.21.15.27:0/2239596439 wait complete. 2024-09-21T00:23:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.048+0000 7fca20c12740 20 client.15738 trim_cache size 0 max 16384 2024-09-21T00:23:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:23:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:23:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:23:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:23:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.049+0000 7fca19c00640 1 -- 172.21.15.27:0/1125701788 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc9f80b65c0 msgr2=0x7fc9f80b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.049+0000 7fca19c00640 1 --2- 172.21.15.27:0/1125701788 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc9f80b65c0 0x7fc9f80b8ad0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fca0002a9e0 tx=0x7fca00005a20 comp rx=0 tx=0).stop 2024-09-21T00:23:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.049+0000 7fca19c00640 1 -- 172.21.15.27:0/1125701788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca14103ab0 msgr2=0x7fca14075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.049+0000 7fca19c00640 1 --2- 172.21.15.27:0/1125701788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca14103ab0 0x7fca14075710 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fca0400e9f0 tx=0x7fca0400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:23:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.049+0000 7fca19c00640 1 -- 172.21.15.27:0/1125701788 shutdown_connections 2024-09-21T00:23:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.049+0000 7fca19c00640 1 --2- 172.21.15.27:0/1125701788 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc9f80b65c0 0x7fc9f80b8ad0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.049+0000 7fca19c00640 1 --2- 172.21.15.27:0/1125701788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca14109d30 0x7fca14076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.049+0000 7fca19c00640 1 --2- 172.21.15.27:0/1125701788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca141093e0 0x7fca14075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.049+0000 7fca19c00640 1 --2- 172.21.15.27:0/1125701788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca14103ab0 0x7fca14075710 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.049+0000 7fca19c00640 1 -- 172.21.15.27:0/1125701788 >> 172.21.15.27:0/1125701788 conn(0x7fca140ff4f0 msgr2=0x7fca14101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.049+0000 7fca19c00640 1 -- 172.21.15.27:0/1125701788 shutdown_connections 2024-09-21T00:23:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:42.049+0000 7fca19c00640 1 -- 172.21.15.27:0/1125701788 wait complete. 2024-09-21T00:23:43.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:43 smithi027 ceph-mon[30745]: pgmap v447: 129 pgs: 129 active+clean; 10 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 787 KiB/s wr, 113 op/s 2024-09-21T00:23:43.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:43 smithi089 ceph-mon[34481]: pgmap v447: 129 pgs: 129 active+clean; 10 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 787 KiB/s wr, 113 op/s 2024-09-21T00:23:43.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:43 smithi098 ceph-mon[34725]: pgmap v447: 129 pgs: 129 active+clean; 10 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 787 KiB/s wr, 113 op/s 2024-09-21T00:23:45.826 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:45 smithi027 ceph-mon[30745]: pgmap v448: 129 pgs: 129 active+clean; 11 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 852 KiB/s wr, 201 op/s 2024-09-21T00:23:45.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:45 smithi089 ceph-mon[34481]: pgmap v448: 129 pgs: 129 active+clean; 11 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 852 KiB/s wr, 201 op/s 2024-09-21T00:23:45.998 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:45 smithi098 ceph-mon[34725]: pgmap v448: 129 pgs: 129 active+clean; 11 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 852 KiB/s wr, 201 op/s 2024-09-21T00:23:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:46 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:23:46.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:46 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:23:46.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:46 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:23:47.821 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:47 smithi027 ceph-mon[30745]: pgmap v449: 129 pgs: 129 active+clean; 16 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 1.6 MiB/s wr, 279 op/s 2024-09-21T00:23:47.822 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:47 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:23:47.822 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:47 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:23:47.822 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:47 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:23:47.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:47 smithi089 ceph-mon[34481]: pgmap v449: 129 pgs: 129 active+clean; 16 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 1.6 MiB/s wr, 279 op/s 2024-09-21T00:23:47.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:47 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:23:47.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:47 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:23:47.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:47 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:23:47.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:47 smithi098 ceph-mon[34725]: pgmap v449: 129 pgs: 129 active+clean; 16 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 1.6 MiB/s wr, 279 op/s 2024-09-21T00:23:47.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:47 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:23:47.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:47 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:23:47.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:47 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:23:48.280 INFO:tasks.workunit.client.0.smithi027.stdout: 14 288741 981 207641 874 65808 1036 2024-09-21T00:23:48.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:48 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:23:48.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:48 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:23:48.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:48 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:23:49.602 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:49 smithi027 ceph-mon[30745]: pgmap v450: 129 pgs: 129 active+clean; 25 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 314 op/s 2024-09-21T00:23:49.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:49 smithi089 ceph-mon[34481]: pgmap v450: 129 pgs: 129 active+clean; 25 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 314 op/s 2024-09-21T00:23:49.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:49 smithi098 ceph-mon[34725]: pgmap v450: 129 pgs: 129 active+clean; 25 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 314 op/s 2024-09-21T00:23:51.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:51 smithi027 ceph-mon[30745]: pgmap v451: 129 pgs: 129 active+clean; 33 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 326 op/s 2024-09-21T00:23:51.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:51 smithi089 ceph-mon[34481]: pgmap v451: 129 pgs: 129 active+clean; 33 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 326 op/s 2024-09-21T00:23:51.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:51 smithi098 ceph-mon[34725]: pgmap v451: 129 pgs: 129 active+clean; 33 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 326 op/s 2024-09-21T00:23:53.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:53 smithi089 ceph-mon[34481]: pgmap v452: 129 pgs: 129 active+clean; 42 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 357 op/s 2024-09-21T00:23:53.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:53 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:23:53.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:53 smithi098 ceph-mon[34725]: pgmap v452: 129 pgs: 129 active+clean; 42 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 357 op/s 2024-09-21T00:23:53.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:53 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:23:54.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:53 smithi027 ceph-mon[30745]: pgmap v452: 129 pgs: 129 active+clean; 42 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 357 op/s 2024-09-21T00:23:54.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:53 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:23:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.311+0000 7f03b4800640 1 -- 172.21.15.27:0/3177927773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03ac110080 msgr2=0x7f03ac1124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.311+0000 7f03b4800640 1 --2- 172.21.15.27:0/3177927773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03ac110080 0x7f03ac1124a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009ca0 tx=0x7f03a802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.311+0000 7f03b4800640 1 -- 172.21.15.27:0/3177927773 shutdown_connections 2024-09-21T00:23:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.311+0000 7f03b4800640 1 --2- 172.21.15.27:0/3177927773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03ac110080 0x7f03ac1124a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.311+0000 7f03b4800640 1 --2- 172.21.15.27:0/3177927773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03ac0760a0 0x7f03ac10fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.311+0000 7f03b4800640 1 --2- 172.21.15.27:0/3177927773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ac075750 0x7f03ac075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.311+0000 7f03b4800640 1 -- 172.21.15.27:0/3177927773 >> 172.21.15.27:0/3177927773 conn(0x7f03ac0ff4d0 msgr2=0x7f03ac101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.311+0000 7f03b4800640 1 -- 172.21.15.27:0/3177927773 shutdown_connections 2024-09-21T00:23:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.311+0000 7f03b4800640 1 -- 172.21.15.27:0/3177927773 wait complete. 2024-09-21T00:23:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.313+0000 7f03b4800640 1 Processor -- start 2024-09-21T00:23:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.313+0000 7f03b4800640 1 -- start start 2024-09-21T00:23:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.314+0000 7f03b4800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ac075750 0x7f03ac19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.314+0000 7f03b4800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03ac0760a0 0x7f03ac19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.314+0000 7f03b4800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03ac110080 0x7f03ac1a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.314+0000 7f03b4800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac115c40 con 0x7f03ac110080 2024-09-21T00:23:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.314+0000 7f03b4800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac115ac0 con 0x7f03ac075750 2024-09-21T00:23:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.314+0000 7f03b4800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac115dc0 con 0x7f03ac0760a0 2024-09-21T00:23:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.314+0000 7f03b1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ac075750 0x7f03ac19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.314+0000 7f03b1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ac075750 0x7f03ac19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:50292/0 (socket says 172.21.15.27:50292) 2024-09-21T00:23:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.314+0000 7f03b1e00640 1 -- 172.21.15.27:0/3834046099 learned_addr learned my addr 172.21.15.27:0/3834046099 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.314+0000 7f03b1400640 1 --2- 172.21.15.27:0/3834046099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03ac0760a0 0x7f03ac19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.315+0000 7f03b2800640 1 --2- 172.21.15.27:0/3834046099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03ac110080 0x7f03ac1a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.315+0000 7f03b1400640 1 -- 172.21.15.27:0/3834046099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ac075750 msgr2=0x7f03ac19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.315+0000 7f03b1400640 1 --2- 172.21.15.27:0/3834046099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ac075750 0x7f03ac19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.315+0000 7f03b1400640 1 -- 172.21.15.27:0/3834046099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03ac110080 msgr2=0x7f03ac1a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.315+0000 7f03b1400640 1 --2- 172.21.15.27:0/3834046099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03ac110080 0x7f03ac1a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.315+0000 7f03b1400640 1 -- 172.21.15.27:0/3834046099 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac1a2590 con 0x7f03ac0760a0 2024-09-21T00:23:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.315+0000 7f03b1400640 1 --2- 172.21.15.27:0/3834046099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03ac0760a0 0x7f03ac19cc00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f039c00e9c0 tx=0x7f039c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.316+0000 7f03a6a00640 1 -- 172.21.15.27:0/3834046099 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c00cd80 con 0x7f03ac0760a0 2024-09-21T00:23:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.316+0000 7f03a6a00640 1 -- 172.21.15.27:0/3834046099 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f039c0035f0 con 0x7f03ac0760a0 2024-09-21T00:23:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.316+0000 7f03b4800640 1 -- 172.21.15.27:0/3834046099 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac1a2880 con 0x7f03ac0760a0 2024-09-21T00:23:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.316+0000 7f03a6a00640 1 -- 172.21.15.27:0/3834046099 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c010670 con 0x7f03ac0760a0 2024-09-21T00:23:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.316+0000 7f03b4800640 1 -- 172.21.15.27:0/3834046099 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac1a9010 con 0x7f03ac0760a0 2024-09-21T00:23:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.321+0000 7f03a6a00640 1 -- 172.21.15.27:0/3834046099 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f039c004000 con 0x7f03ac0760a0 2024-09-21T00:23:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.321+0000 7f03bb871740 1 Processor -- start 2024-09-21T00:23:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.322+0000 7f03bb871740 1 -- start start 2024-09-21T00:23:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.322+0000 7f03bb871740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603732c51b0 0x560373669c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.322+0000 7f03bb871740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56037329c2f0 0x56037366a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.322+0000 7f03bb871740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732a44d0 0x56037366a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.322+0000 7f03bb871740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5603735df6b0 con 0x56037329c2f0 2024-09-21T00:23:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.322+0000 7f03bb871740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5603735ceae0 con 0x5603732c51b0 2024-09-21T00:23:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.322+0000 7f03bb871740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5603735d28f0 con 0x5603732a44d0 2024-09-21T00:23:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.323+0000 7f03b2800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603732c51b0 0x560373669c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.323+0000 7f03b2800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603732c51b0 0x560373669c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:50304/0 (socket says 172.21.15.27:50304) 2024-09-21T00:23:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.323+0000 7f03b2800640 1 -- 172.21.15.27:0/193861989 learned_addr learned my addr 172.21.15.27:0/193861989 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.323+0000 7f03b1400640 1 --2- 172.21.15.27:0/193861989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732a44d0 0x56037366a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.323+0000 7f03b1e00640 1 --2- 172.21.15.27:0/193861989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56037329c2f0 0x56037366a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.323+0000 7f03b2800640 1 -- 172.21.15.27:0/193861989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732a44d0 msgr2=0x56037366a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.323+0000 7f03b2800640 1 --2- 172.21.15.27:0/193861989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732a44d0 0x56037366a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.323+0000 7f03b2800640 1 -- 172.21.15.27:0/193861989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56037329c2f0 msgr2=0x56037366a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.323+0000 7f03b2800640 1 --2- 172.21.15.27:0/193861989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56037329c2f0 0x56037366a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.323+0000 7f03b2800640 1 -- 172.21.15.27:0/193861989 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56037359b780 con 0x5603732c51b0 2024-09-21T00:23:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.323+0000 7f03a6a00640 1 --2- 172.21.15.27:0/3834046099 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f03940b6700 0x7f03940b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.324+0000 7f03b2800640 1 --2- 172.21.15.27:0/193861989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603732c51b0 0x560373669c30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f03a8005e30 tx=0x7f03a8009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=849f269346c0b312 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.324+0000 7f03a6a00640 1 -- 172.21.15.27:0/3834046099 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f039c0e7d40 con 0x7f03ac0760a0 2024-09-21T00:23:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.324+0000 7f03b1e00640 1 --2- 172.21.15.27:0/3834046099 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f03940b6700 0x7f03940b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.324+0000 7f039b000640 1 -- 172.21.15.27:0/193861989 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8031cf0 con 0x5603732c51b0 2024-09-21T00:23:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.324+0000 7f039b000640 1 -- 172.21.15.27:0/193861989 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03a8036040 con 0x5603732c51b0 2024-09-21T00:23:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.324+0000 7f039b000640 1 -- 172.21.15.27:0/193861989 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a80042a0 con 0x5603732c51b0 2024-09-21T00:23:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.326+0000 7f03bb871740 1 -- 172.21.15.27:0/193861989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603732c51b0 msgr2=0x560373669c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.326+0000 7f03b1e00640 1 --2- 172.21.15.27:0/3834046099 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f03940b6700 0x7f03940b8c10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5603736688b0 tx=0x7f03a0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.326+0000 7f03bb871740 1 --2- 172.21.15.27:0/193861989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603732c51b0 0x560373669c30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f03a8005e30 tx=0x7f03a8009460 comp rx=0 tx=0).stop 2024-09-21T00:23:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.327+0000 7f03bb871740 1 -- 172.21.15.27:0/193861989 shutdown_connections 2024-09-21T00:23:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.327+0000 7f03bb871740 1 --2- 172.21.15.27:0/193861989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732a44d0 0x56037366a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.327+0000 7f03bb871740 1 --2- 172.21.15.27:0/193861989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56037329c2f0 0x56037366a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.327+0000 7f03bb871740 1 --2- 172.21.15.27:0/193861989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603732c51b0 0x560373669c30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.327+0000 7f03bb871740 1 -- 172.21.15.27:0/193861989 >> 172.21.15.27:0/193861989 conn(0x5603732b9300 msgr2=0x560373667b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.327+0000 7f03bb871740 1 -- 172.21.15.27:0/193861989 shutdown_connections 2024-09-21T00:23:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.327+0000 7f03bb871740 1 -- 172.21.15.27:0/193861989 wait complete. 2024-09-21T00:23:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.328+0000 7f03bb871740 1 -- start start 2024-09-21T00:23:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.329+0000 7f03bb871740 1 Processor -- start 2024-09-21T00:23:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.329+0000 7f03bb871740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56037329c2f0 0x56037367dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.329+0000 7f03bb871740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732a44d0 0x56037367e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.329+0000 7f03bb871740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732c51b0 0x560373689e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.329+0000 7f03bb871740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560373506ab0 con 0x5603732a44d0 2024-09-21T00:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.329+0000 7f03bb871740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560373575570 con 0x56037329c2f0 2024-09-21T00:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.329+0000 7f03bb871740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560373680b10 con 0x5603732c51b0 2024-09-21T00:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.330+0000 7f03b1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56037329c2f0 0x56037367dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.330+0000 7f03b1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56037329c2f0 0x56037367dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:50310/0 (socket says 172.21.15.27:50310) 2024-09-21T00:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.330+0000 7f03b1e00640 1 -- 172.21.15.27:0/174462601 learned_addr learned my addr 172.21.15.27:0/174462601 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.330+0000 7f03b2800640 1 --2- 172.21.15.27:0/174462601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732c51b0 0x560373689e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.330+0000 7f03b1e00640 1 -- 172.21.15.27:0/174462601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732c51b0 msgr2=0x560373689e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.330+0000 7f03b1e00640 1 --2- 172.21.15.27:0/174462601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732c51b0 0x560373689e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.330+0000 7f03b1e00640 1 -- 172.21.15.27:0/174462601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732a44d0 msgr2=0x56037367e010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.330+0000 7f03b1e00640 1 --2- 172.21.15.27:0/174462601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732a44d0 0x56037367e010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.330+0000 7f03b1e00640 1 -- 172.21.15.27:0/174462601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560373330790 con 0x56037329c2f0 2024-09-21T00:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.330+0000 7f03b1e00640 1 --2- 172.21.15.27:0/174462601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56037329c2f0 0x56037367dad0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f03a001e960 tx=0x7f03a0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e44791e29cb0fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.330+0000 7f03bb871740 10 client.0 _ll_register_callbacks cb 0x560373584100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:23:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.331+0000 7f038f400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:23:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.331+0000 7f03bb871740 1 -- 172.21.15.27:0/174462601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03a8036040 con 0x56037329c2f0 2024-09-21T00:23:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.331+0000 7f038f400640 1 -- 172.21.15.27:0/174462601 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a003f060 con 0x56037329c2f0 2024-09-21T00:23:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.331+0000 7f038f400640 1 -- 172.21.15.27:0/174462601 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f03a003a430 con 0x56037329c2f0 2024-09-21T00:23:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.331+0000 7f038f400640 1 -- 172.21.15.27:0/174462601 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03a003a630 con 0x56037329c2f0 2024-09-21T00:23:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.331+0000 7f03bb871740 10 client.25781 fetch_fsmap learned FSMap version 28 2024-09-21T00:23:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.331+0000 7f03bb871740 1 -- 172.21.15.27:0/174462601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5603733a9620 con 0x56037329c2f0 2024-09-21T00:23:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.332+0000 7f038f400640 1 -- 172.21.15.27:0/174462601 <== mon.2 v2:172.21.15.89:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f03a000f3c0 con 0x56037329c2f0 2024-09-21T00:23:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.332+0000 7f03bb871740 10 client.25781 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:23:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.332+0000 7f03bb871740 10 client.25781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:23:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.332+0000 7f03bb871740 20 client.25781 populate_metadata read hostname 'smithi027' 2024-09-21T00:23:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.332+0000 7f03bb871740 1 --2- 172.21.15.27:0/174462601 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560373283d80 0x56037368fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.332+0000 7f03bb871740 4 client.25781 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:23:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.332+0000 7f03bb871740 1 -- 172.21.15.27:0/174462601 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603735e1600 con 0x560373283d80 2024-09-21T00:23:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.332+0000 7f03b1400640 1 --2- 172.21.15.27:0/174462601 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560373283d80 0x56037368fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.334+0000 7f038f400640 10 client.25781 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.334+0000 7f03b1400640 1 --2- 172.21.15.27:0/174462601 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560373283d80 0x56037368fab0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=223ec68505bc17df server_cookie=97440cdb8c4270a7 in_seq=0 out_seq=0 2024-09-21T00:23:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.338+0000 7f038f400640 1 -- 172.21.15.27:0/174462601 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5603735e1600 con 0x560373283d80 2024-09-21T00:23:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.338+0000 7f038f400640 10 client.25781 handle_command_reply: tid=0 2024-09-21T00:23:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.338+0000 7f03bb871740 1 client.25781 shutdown 2024-09-21T00:23:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.338+0000 7f03bb871740 1 -- 172.21.15.27:0/174462601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56037329c2f0 msgr2=0x56037367dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.338+0000 7f03bb871740 1 --2- 172.21.15.27:0/174462601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56037329c2f0 0x56037367dad0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f03a001e960 tx=0x7f03a0004730 comp rx=0 tx=0).stop 2024-09-21T00:23:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.339+0000 7f03bb871740 1 -- 172.21.15.27:0/174462601 shutdown_connections 2024-09-21T00:23:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.339+0000 7f03bb871740 1 --2- 172.21.15.27:0/174462601 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560373283d80 0x56037368fab0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.339+0000 7f03bb871740 1 --2- 172.21.15.27:0/174462601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732c51b0 0x560373689e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.339+0000 7f03bb871740 1 --2- 172.21.15.27:0/174462601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732a44d0 0x56037367e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.339+0000 7f038f400640 0 client.25781 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.339+0000 7f03bb871740 1 --2- 172.21.15.27:0/174462601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56037329c2f0 0x56037367dad0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.339+0000 7f03bb871740 1 -- 172.21.15.27:0/174462601 >> 172.21.15.27:0/174462601 conn(0x5603732b9300 msgr2=0x56037366cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.339+0000 7f03bb871740 1 -- 172.21.15.27:0/174462601 shutdown_connections 2024-09-21T00:23:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.339+0000 7f03bb871740 1 -- 172.21.15.27:0/174462601 wait complete. 2024-09-21T00:23:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.339+0000 7f03bb871740 20 client.25781 trim_cache size 0 max 16384 2024-09-21T00:23:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.356+0000 7f03bb871740 1 Processor -- start 2024-09-21T00:23:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.356+0000 7f03bb871740 1 -- start start 2024-09-21T00:23:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.356+0000 7f03bb871740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56037329c2f0 0x560373678e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.356+0000 7f03bb871740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732a44d0 0x560373689d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.356+0000 7f03bb871740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603732c51b0 0x56037367aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.356+0000 7f03bb871740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac075080 con 0x5603732a44d0 2024-09-21T00:23:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.356+0000 7f03bb871740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac072950 con 0x5603732c51b0 2024-09-21T00:23:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.356+0000 7f03bb871740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac0707a0 con 0x56037329c2f0 2024-09-21T00:23:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.356+0000 7f03b1e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56037329c2f0 0x560373678e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.356+0000 7f03b1e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56037329c2f0 0x560373678e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:45804/0 (socket says 172.21.15.27:45804) 2024-09-21T00:23:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.356+0000 7f03b1e00640 1 -- 172.21.15.27:0/2804520434 learned_addr learned my addr 172.21.15.27:0/2804520434 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.356+0000 7f03b2800640 1 --2- 172.21.15.27:0/2804520434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603732c51b0 0x56037367aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.357+0000 7f03b1e00640 1 -- 172.21.15.27:0/2804520434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603732c51b0 msgr2=0x56037367aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.357+0000 7f03b1e00640 1 --2- 172.21.15.27:0/2804520434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603732c51b0 0x56037367aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.357+0000 7f03b1e00640 1 -- 172.21.15.27:0/2804520434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732a44d0 msgr2=0x560373689d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.357+0000 7f03b1400640 1 --2- 172.21.15.27:0/2804520434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732a44d0 0x560373689d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.357+0000 7f03b1e00640 1 --2- 172.21.15.27:0/2804520434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732a44d0 0x560373689d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.357+0000 7f03b1e00640 1 -- 172.21.15.27:0/2804520434 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603736684b0 con 0x56037329c2f0 2024-09-21T00:23:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.357+0000 7f03b2800640 1 --2- 172.21.15.27:0/2804520434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603732c51b0 0x56037367aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:23:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.357+0000 7f03b1400640 1 --2- 172.21.15.27:0/2804520434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732a44d0 0x560373689d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:23:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.357+0000 7f03b1e00640 1 --2- 172.21.15.27:0/2804520434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56037329c2f0 0x560373678e80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009bd0 tx=0x7f03a0037000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cac14ffb87e2599 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.357+0000 7f039a600640 1 -- 172.21.15.27:0/2804520434 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a003f040 con 0x56037329c2f0 2024-09-21T00:23:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.358+0000 7f039a600640 1 -- 172.21.15.27:0/2804520434 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03a8036040 con 0x56037329c2f0 2024-09-21T00:23:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.358+0000 7f039a600640 1 -- 172.21.15.27:0/2804520434 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a00055c0 con 0x56037329c2f0 2024-09-21T00:23:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.358+0000 7f03bb871740 1 -- 172.21.15.27:0/2804520434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56037329c2f0 msgr2=0x560373678e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.358+0000 7f03bb871740 1 --2- 172.21.15.27:0/2804520434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56037329c2f0 0x560373678e80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009bd0 tx=0x7f03a0037000 comp rx=0 tx=0).stop 2024-09-21T00:23:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.359+0000 7f03bb871740 1 -- 172.21.15.27:0/2804520434 shutdown_connections 2024-09-21T00:23:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.359+0000 7f03bb871740 1 --2- 172.21.15.27:0/2804520434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603732c51b0 0x56037367aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.359+0000 7f03bb871740 1 --2- 172.21.15.27:0/2804520434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732a44d0 0x560373689d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.359+0000 7f03bb871740 1 --2- 172.21.15.27:0/2804520434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56037329c2f0 0x560373678e80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.359+0000 7f03bb871740 1 -- 172.21.15.27:0/2804520434 >> 172.21.15.27:0/2804520434 conn(0x5603732b9300 msgr2=0x560373678770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.359+0000 7f03bb871740 1 -- 172.21.15.27:0/2804520434 shutdown_connections 2024-09-21T00:23:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.359+0000 7f03bb871740 1 -- 172.21.15.27:0/2804520434 wait complete. 2024-09-21T00:23:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.359+0000 7f03bb871740 1 -- start start 2024-09-21T00:23:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.360+0000 7f03bb871740 1 Processor -- start 2024-09-21T00:23:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.360+0000 7f03bb871740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56037329c2f0 0x560373686490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.360+0000 7f03bb871740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732a44d0 0x560373681930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.360+0000 7f03bb871740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732c51b0 0x560373681e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.360+0000 7f03bb871740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac070e10 con 0x5603732c51b0 2024-09-21T00:23:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.360+0000 7f03bb871740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac072280 con 0x56037329c2f0 2024-09-21T00:23:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.360+0000 7f03bb871740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac0714e0 con 0x5603732a44d0 2024-09-21T00:23:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.360+0000 7f03bb871740 10 client.0 _ll_register_callbacks cb 0x7f03a0009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:23:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.361+0000 7f03b2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732c51b0 0x560373681e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.361+0000 7f03b2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732c51b0 0x560373681e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52392/0 (socket says 172.21.15.27:52392) 2024-09-21T00:23:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.361+0000 7f03b2800640 1 -- 172.21.15.27:0/1816528865 learned_addr learned my addr 172.21.15.27:0/1816528865 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.361+0000 7f03b1400640 1 --2- 172.21.15.27:0/1816528865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732a44d0 0x560373681930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.361+0000 7f03b1e00640 1 --2- 172.21.15.27:0/1816528865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56037329c2f0 0x560373686490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.361+0000 7f03b2800640 1 -- 172.21.15.27:0/1816528865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732a44d0 msgr2=0x560373681930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.361+0000 7f03b2800640 1 --2- 172.21.15.27:0/1816528865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732a44d0 0x560373681930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.361+0000 7f03b2800640 1 -- 172.21.15.27:0/1816528865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56037329c2f0 msgr2=0x560373686490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.361+0000 7f03b2800640 1 --2- 172.21.15.27:0/1816528865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56037329c2f0 0x560373686490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.361+0000 7f03b2800640 1 -- 172.21.15.27:0/1816528865 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603736824e0 con 0x5603732c51b0 2024-09-21T00:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.361+0000 7f03b1400640 1 --2- 172.21.15.27:0/1816528865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732a44d0 0x560373681930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.361+0000 7f03b1e00640 1 --2- 172.21.15.27:0/1816528865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56037329c2f0 0x560373686490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.362+0000 7f03b2800640 1 --2- 172.21.15.27:0/1816528865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732c51b0 0x560373681e70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f03a802a900 tx=0x7f03a8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7af1fe6b5f53761 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.362+0000 7f038f400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.362+0000 7f038f400640 1 -- 172.21.15.27:0/1816528865 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8053020 con 0x5603732c51b0 2024-09-21T00:23:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.362+0000 7f03bb871740 1 -- 172.21.15.27:0/1816528865 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03a8036040 con 0x5603732c51b0 2024-09-21T00:23:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.362+0000 7f038f400640 1 -- 172.21.15.27:0/1816528865 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f03a8031cf0 con 0x5603732c51b0 2024-09-21T00:23:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.363+0000 7f038f400640 1 -- 172.21.15.27:0/1816528865 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03a805e050 con 0x5603732c51b0 2024-09-21T00:23:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.363+0000 7f03bb871740 10 client.15750 fetch_fsmap learned FSMap version 28 2024-09-21T00:23:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.363+0000 7f03bb871740 1 -- 172.21.15.27:0/1816528865 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5603736fa300 con 0x5603732c51b0 2024-09-21T00:23:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.363+0000 7f038f400640 1 -- 172.21.15.27:0/1816528865 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03a8036040 con 0x5603732c51b0 2024-09-21T00:23:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.364+0000 7f038f400640 1 -- 172.21.15.27:0/1816528865 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f03a802fc20 con 0x5603732c51b0 2024-09-21T00:23:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.364+0000 7f03bb871740 10 client.15750 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:23:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.364+0000 7f03bb871740 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:23:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.364+0000 7f03bb871740 20 client.15750 populate_metadata read hostname 'smithi027' 2024-09-21T00:23:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.364+0000 7f03bb871740 1 --2- 172.21.15.27:0/1816528865 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5603732591c0 0x5603736fc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.364+0000 7f03bb871740 4 client.15750 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:23:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.364+0000 7f03bb871740 1 -- 172.21.15.27:0/1816528865 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603735dd0b0 con 0x5603732591c0 2024-09-21T00:23:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.364+0000 7f03b1e00640 1 --2- 172.21.15.27:0/1816528865 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5603732591c0 0x5603736fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.365+0000 7f03b1e00640 1 --2- 172.21.15.27:0/1816528865 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5603732591c0 0x5603736fc700 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9fc65914af14a77 server_cookie=7709d639f1795d1d in_seq=0 out_seq=0 2024-09-21T00:23:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:55.365+0000 7f038f400640 10 client.15750 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:55.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:55 smithi027 ceph-mon[30745]: pgmap v453: 129 pgs: 129 active+clean; 52 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 351 op/s 2024-09-21T00:23:55.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:55 smithi089 ceph-mon[34481]: pgmap v453: 129 pgs: 129 active+clean; 52 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 351 op/s 2024-09-21T00:23:55.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:55 smithi098 ceph-mon[34725]: pgmap v453: 129 pgs: 129 active+clean; 52 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 351 op/s 2024-09-21T00:23:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.174+0000 7f038f400640 1 -- 172.21.15.27:0/1816528865 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603735dd0b0 con 0x5603732591c0 2024-09-21T00:23:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.174+0000 7f038f400640 10 client.15750 handle_command_reply: tid=0 2024-09-21T00:23:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.174+0000 7f03bb871740 1 client.15750 shutdown 2024-09-21T00:23:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.175+0000 7f03bb871740 1 -- 172.21.15.27:0/1816528865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732c51b0 msgr2=0x560373681e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.175+0000 7f03bb871740 1 --2- 172.21.15.27:0/1816528865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732c51b0 0x560373681e70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f03a802a900 tx=0x7f03a8003970 comp rx=0 tx=0).stop 2024-09-21T00:23:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.175+0000 7f03bb871740 1 -- 172.21.15.27:0/1816528865 shutdown_connections 2024-09-21T00:23:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.175+0000 7f03bb871740 1 --2- 172.21.15.27:0/1816528865 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5603732591c0 0x5603736fc700 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.175+0000 7f03bb871740 1 --2- 172.21.15.27:0/1816528865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603732c51b0 0x560373681e70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.175+0000 7f03bb871740 1 --2- 172.21.15.27:0/1816528865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603732a44d0 0x560373681930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.175+0000 7f03bb871740 1 --2- 172.21.15.27:0/1816528865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56037329c2f0 0x560373686490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.175+0000 7f03bb871740 1 -- 172.21.15.27:0/1816528865 >> 172.21.15.27:0/1816528865 conn(0x5603732b9300 msgr2=0x5603736660e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.175+0000 7f038f400640 0 client.15750 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:23:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.175+0000 7f03bb871740 1 -- 172.21.15.27:0/1816528865 shutdown_connections 2024-09-21T00:23:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.176+0000 7f03bb871740 1 -- 172.21.15.27:0/1816528865 wait complete. 2024-09-21T00:23:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.176+0000 7f03bb871740 20 client.15750 trim_cache size 0 max 16384 2024-09-21T00:23:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:23:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:23:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:23:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:23:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.177+0000 7f03b4800640 1 -- 172.21.15.27:0/3834046099 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f03940b6700 msgr2=0x7f03940b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.177+0000 7f03b4800640 1 --2- 172.21.15.27:0/3834046099 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f03940b6700 0x7f03940b8c10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5603736688b0 tx=0x7f03a0005f90 comp rx=0 tx=0).stop 2024-09-21T00:23:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.177+0000 7f03b4800640 1 -- 172.21.15.27:0/3834046099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03ac0760a0 msgr2=0x7f03ac19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.177+0000 7f03b4800640 1 --2- 172.21.15.27:0/3834046099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03ac0760a0 0x7f03ac19cc00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f039c00e9c0 tx=0x7f039c00ee80 comp rx=0 tx=0).stop 2024-09-21T00:23:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.178+0000 7f03b4800640 1 -- 172.21.15.27:0/3834046099 shutdown_connections 2024-09-21T00:23:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.178+0000 7f03b4800640 1 --2- 172.21.15.27:0/3834046099 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f03940b6700 0x7f03940b8c10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.178+0000 7f03b4800640 1 --2- 172.21.15.27:0/3834046099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03ac110080 0x7f03ac1a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.178+0000 7f03b4800640 1 --2- 172.21.15.27:0/3834046099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03ac0760a0 0x7f03ac19cc00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.178+0000 7f03b4800640 1 --2- 172.21.15.27:0/3834046099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ac075750 0x7f03ac19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.178+0000 7f03b4800640 1 -- 172.21.15.27:0/3834046099 >> 172.21.15.27:0/3834046099 conn(0x7f03ac0ff4d0 msgr2=0x7f03ac107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.178+0000 7f03b4800640 1 -- 172.21.15.27:0/3834046099 shutdown_connections 2024-09-21T00:23:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:23:56.178+0000 7f03b4800640 1 -- 172.21.15.27:0/3834046099 wait complete. 2024-09-21T00:23:57.827 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:57 smithi027 ceph-mon[30745]: pgmap v454: 129 pgs: 129 active+clean; 55 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.8 MiB/s wr, 365 op/s 2024-09-21T00:23:57.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:57 smithi089 ceph-mon[34481]: pgmap v454: 129 pgs: 129 active+clean; 55 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.8 MiB/s wr, 365 op/s 2024-09-21T00:23:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:57 smithi098 ceph-mon[34725]: pgmap v454: 129 pgs: 129 active+clean; 55 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.8 MiB/s wr, 365 op/s 2024-09-21T00:23:58.015 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T00:23:58.015 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T00:23:58.015 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:23:58.289 INFO:tasks.workunit.client.0.smithi027.stdout: 19 168802 335 129566 442 64107 791 2024-09-21T00:23:59.004 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:23:59.756 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:23:59 smithi027 ceph-mon[30745]: pgmap v455: 129 pgs: 129 active+clean; 47 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 495 op/s 2024-09-21T00:23:59.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.921+0000 7faba5e00640 1 -- 172.21.15.27:0/2147764491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba010a590 msgr2=0x7faba010a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:59.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.921+0000 7faba5e00640 1 --2- 172.21.15.27:0/2147764491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba010a590 0x7faba010a9a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab90009900 tx=0x7fab9002a440 comp rx=0 tx=0).stop 2024-09-21T00:23:59.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.921+0000 7faba5e00640 1 -- 172.21.15.27:0/2147764491 shutdown_connections 2024-09-21T00:23:59.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.921+0000 7faba5e00640 1 --2- 172.21.15.27:0/2147764491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faba0116640 0x7faba0118b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:59.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.921+0000 7faba5e00640 1 --2- 172.21.15.27:0/2147764491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faba010aee0 0x7faba0115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:59.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.921+0000 7faba5e00640 1 --2- 172.21.15.27:0/2147764491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba010a590 0x7faba010a9a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:59.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.921+0000 7faba5e00640 1 -- 172.21.15.27:0/2147764491 >> 172.21.15.27:0/2147764491 conn(0x7faba0070ec0 msgr2=0x7faba00712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:23:59.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.921+0000 7faba5e00640 1 -- 172.21.15.27:0/2147764491 shutdown_connections 2024-09-21T00:23:59.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.921+0000 7faba5e00640 1 -- 172.21.15.27:0/2147764491 wait complete. 2024-09-21T00:23:59.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.925+0000 7faba5e00640 1 Processor -- start 2024-09-21T00:23:59.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.928+0000 7faba5e00640 1 -- start start 2024-09-21T00:23:59.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.929+0000 7faba5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba010a590 0x7faba01a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:59.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.929+0000 7fab9ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba010a590 0x7faba01a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:59.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.930+0000 7fab9ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba010a590 0x7faba01a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:50348/0 (socket says 172.21.15.27:50348) 2024-09-21T00:23:59.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.930+0000 7fab9ea00640 1 -- 172.21.15.27:0/1897778702 learned_addr learned my addr 172.21.15.27:0/1897778702 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:23:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.930+0000 7faba5e00640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faba010aee0 0x7faba01a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.930+0000 7faba5e00640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faba0116640 0x7faba01a8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.930+0000 7faba5e00640 1 -- 172.21.15.27:0/1897778702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faba011c410 con 0x7faba0116640 2024-09-21T00:23:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.930+0000 7faba5e00640 1 -- 172.21.15.27:0/1897778702 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7faba011c290 con 0x7faba010a590 2024-09-21T00:23:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.930+0000 7faba5e00640 1 -- 172.21.15.27:0/1897778702 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faba011c590 con 0x7faba010aee0 2024-09-21T00:23:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.931+0000 7fab9e000640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faba010aee0 0x7faba01a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.931+0000 7fab9f400640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faba0116640 0x7faba01a8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.931+0000 7fab9ea00640 1 -- 172.21.15.27:0/1897778702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faba010aee0 msgr2=0x7faba01a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.931+0000 7fab9ea00640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faba010aee0 0x7faba01a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.931+0000 7fab9ea00640 1 -- 172.21.15.27:0/1897778702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faba0116640 msgr2=0x7faba01a8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:23:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.931+0000 7fab9ea00640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faba0116640 0x7faba01a8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:23:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.931+0000 7fab9ea00640 1 -- 172.21.15.27:0/1897778702 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba01a8e80 con 0x7faba010a590 2024-09-21T00:23:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.931+0000 7fab9f400640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faba0116640 0x7faba01a8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:23:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.931+0000 7fab9e000640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faba010aee0 0x7faba01a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:23:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.932+0000 7fab9ea00640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba010a590 0x7faba01a18e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab9002aa30 tx=0x7fab90003de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.932+0000 7faba4a00640 1 -- 172.21.15.27:0/1897778702 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9003f070 con 0x7faba010a590 2024-09-21T00:23:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.932+0000 7faba4a00640 1 -- 172.21.15.27:0/1897778702 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab90031d60 con 0x7faba010a590 2024-09-21T00:23:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.932+0000 7faba4a00640 1 -- 172.21.15.27:0/1897778702 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90035030 con 0x7faba010a590 2024-09-21T00:23:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.933+0000 7faba5e00640 1 -- 172.21.15.27:0/1897778702 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba01a9110 con 0x7faba010a590 2024-09-21T00:23:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.933+0000 7faba5e00640 1 -- 172.21.15.27:0/1897778702 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba01a95a0 con 0x7faba010a590 2024-09-21T00:23:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.935+0000 7faba5e00640 1 -- 172.21.15.27:0/1897778702 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba010b5e0 con 0x7faba010a590 2024-09-21T00:23:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.941+0000 7faba4a00640 1 -- 172.21.15.27:0/1897778702 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fab90042070 con 0x7faba010a590 2024-09-21T00:23:59.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.946+0000 7faba4a00640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fab840b6700 0x7fab840b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:23:59.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.947+0000 7faba4a00640 1 -- 172.21.15.27:0/1897778702 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fab90108ad0 con 0x7faba010a590 2024-09-21T00:23:59.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.947+0000 7fab9e000640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fab840b6700 0x7fab840b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:23:59.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.947+0000 7faba4a00640 1 -- 172.21.15.27:0/1897778702 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fab90049070 con 0x7faba010a590 2024-09-21T00:23:59.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:23:59.953+0000 7fab9e000640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fab840b6700 0x7fab840b8c10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faba01182f0 tx=0x7fab88008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:23:59.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:23:59 smithi089 ceph-mon[34481]: pgmap v455: 129 pgs: 129 active+clean; 47 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 495 op/s 2024-09-21T00:23:59.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:23:59 smithi098 ceph-mon[34725]: pgmap v455: 129 pgs: 129 active+clean; 47 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 495 op/s 2024-09-21T00:24:00.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:00.464+0000 7faba5e00640 1 -- 172.21.15.27:0/1897778702 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faba01a98c0 con 0x7faba010a590 2024-09-21T00:24:00.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:00.465+0000 7faba4a00640 1 -- 172.21.15.27:0/1897778702 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11896 (secure 0 0 0) 0x7fab900cec30 con 0x7faba010a590 2024-09-21T00:24:00.476 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:24:00.484 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"btime":"2024-09-21T00:23:52:563301+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:23:52.207999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:24:00.497 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-09-21T00:24:00.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:00.482+0000 7faba5e00640 1 -- 172.21.15.27:0/1897778702 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fab840b6700 msgr2=0x7fab840b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:00.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:00.482+0000 7faba5e00640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fab840b6700 0x7fab840b8c10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faba01182f0 tx=0x7fab88008040 comp rx=0 tx=0).stop 2024-09-21T00:24:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:00.482+0000 7faba5e00640 1 -- 172.21.15.27:0/1897778702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba010a590 msgr2=0x7faba01a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:00.482+0000 7faba5e00640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba010a590 0x7faba01a18e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab9002aa30 tx=0x7fab90003de0 comp rx=0 tx=0).stop 2024-09-21T00:24:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:00.482+0000 7faba5e00640 1 -- 172.21.15.27:0/1897778702 shutdown_connections 2024-09-21T00:24:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:00.482+0000 7faba5e00640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fab840b6700 0x7fab840b8c10 secure :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faba01182f0 tx=0x7fab88008040 comp rx=0 tx=0).stop 2024-09-21T00:24:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:00.482+0000 7faba5e00640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faba0116640 0x7faba01a8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:00.482+0000 7faba5e00640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faba010aee0 0x7faba01a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:00.482+0000 7faba5e00640 1 --2- 172.21.15.27:0/1897778702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba010a590 0x7faba01a18e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:00.482+0000 7faba5e00640 1 -- 172.21.15.27:0/1897778702 >> 172.21.15.27:0/1897778702 conn(0x7faba0070ec0 msgr2=0x7faba010dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:00.483+0000 7faba5e00640 1 -- 172.21.15.27:0/1897778702 shutdown_connections 2024-09-21T00:24:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:00.483+0000 7faba5e00640 1 -- 172.21.15.27:0/1897778702 wait complete. 2024-09-21T00:24:00.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:00 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1897778702' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:24:00.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:00 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1897778702' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:24:00.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:00 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1897778702' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:24:01.682 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T00:24:01.683 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T00:24:01.780 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:01 smithi027 ceph-mon[30745]: pgmap v456: 129 pgs: 129 active+clean; 39 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 600 op/s 2024-09-21T00:24:01.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:01 smithi089 ceph-mon[34481]: pgmap v456: 129 pgs: 129 active+clean; 39 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 600 op/s 2024-09-21T00:24:01.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:01 smithi098 ceph-mon[34725]: pgmap v456: 129 pgs: 129 active+clean; 39 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 600 op/s 2024-09-21T00:24:02.648 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:24:03.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.562+0000 7fdf4a600640 1 -- 172.21.15.27:0/2054006118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf44076100 msgr2=0x7fdf4410f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:03.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.562+0000 7fdf4a600640 1 --2- 172.21.15.27:0/2054006118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf44076100 0x7fdf4410f9e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009900 tx=0x7fdf3002a410 comp rx=0 tx=0).stop 2024-09-21T00:24:03.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.562+0000 7fdf4a600640 1 -- 172.21.15.27:0/2054006118 shutdown_connections 2024-09-21T00:24:03.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.562+0000 7fdf4a600640 1 --2- 172.21.15.27:0/2054006118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4410ff20 0x7fdf44112360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.562+0000 7fdf4a600640 1 --2- 172.21.15.27:0/2054006118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf44076100 0x7fdf4410f9e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.562+0000 7fdf4a600640 1 --2- 172.21.15.27:0/2054006118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf440757b0 0x7fdf44075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.562+0000 7fdf4a600640 1 -- 172.21.15.27:0/2054006118 >> 172.21.15.27:0/2054006118 conn(0x7fdf440ff430 msgr2=0x7fdf44101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:03.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.562+0000 7fdf4a600640 1 -- 172.21.15.27:0/2054006118 shutdown_connections 2024-09-21T00:24:03.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.563+0000 7fdf4a600640 1 -- 172.21.15.27:0/2054006118 wait complete. 2024-09-21T00:24:03.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.564+0000 7fdf4a600640 1 Processor -- start 2024-09-21T00:24:03.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.564+0000 7fdf4a600640 1 -- start start 2024-09-21T00:24:03.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.565+0000 7fdf4a600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf440757b0 0x7fdf4419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.565+0000 7fdf4a600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf44076100 0x7fdf4419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.565+0000 7fdf4a600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4410ff20 0x7fdf441a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.565+0000 7fdf4a600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44115ad0 con 0x7fdf44076100 2024-09-21T00:24:03.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.565+0000 7fdf4a600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44115950 con 0x7fdf440757b0 2024-09-21T00:24:03.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.565+0000 7fdf4a600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44115c50 con 0x7fdf4410ff20 2024-09-21T00:24:03.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.565+0000 7fdf49200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf440757b0 0x7fdf4419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.565+0000 7fdf43e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf44076100 0x7fdf4419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.565+0000 7fdf49200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf440757b0 0x7fdf4419c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53918/0 (socket says 172.21.15.27:53918) 2024-09-21T00:24:03.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.565+0000 7fdf49200640 1 -- 172.21.15.27:0/2824249113 learned_addr learned my addr 172.21.15.27:0/2824249113 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:03.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.565+0000 7fdf49c00640 1 --2- 172.21.15.27:0/2824249113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4410ff20 0x7fdf441a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.566+0000 7fdf49200640 1 -- 172.21.15.27:0/2824249113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4410ff20 msgr2=0x7fdf441a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:03.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.566+0000 7fdf49200640 1 --2- 172.21.15.27:0/2824249113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4410ff20 0x7fdf441a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.566+0000 7fdf49200640 1 -- 172.21.15.27:0/2824249113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf44076100 msgr2=0x7fdf4419cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:03.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.566+0000 7fdf49200640 1 --2- 172.21.15.27:0/2824249113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf44076100 0x7fdf4419cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.566+0000 7fdf49200640 1 -- 172.21.15.27:0/2824249113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf441a2490 con 0x7fdf440757b0 2024-09-21T00:24:03.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.566+0000 7fdf49200640 1 --2- 172.21.15.27:0/2824249113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf440757b0 0x7fdf4419c520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800e9f0 tx=0x7fdf3800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:03.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.566+0000 7fdf41600640 1 -- 172.21.15.27:0/2824249113 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3800cd80 con 0x7fdf440757b0 2024-09-21T00:24:03.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.566+0000 7fdf41600640 1 -- 172.21.15.27:0/2824249113 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf380108e0 con 0x7fdf440757b0 2024-09-21T00:24:03.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.566+0000 7fdf41600640 1 -- 172.21.15.27:0/2824249113 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf38018800 con 0x7fdf440757b0 2024-09-21T00:24:03.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.567+0000 7fdf4a600640 1 -- 172.21.15.27:0/2824249113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf441a2780 con 0x7fdf440757b0 2024-09-21T00:24:03.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.567+0000 7fdf4a600640 1 -- 172.21.15.27:0/2824249113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf441a8ee0 con 0x7fdf440757b0 2024-09-21T00:24:03.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.570+0000 7fdf527dd740 1 Processor -- start 2024-09-21T00:24:03.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.571+0000 7fdf527dd740 1 -- start start 2024-09-21T00:24:03.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.571+0000 7fdf527dd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 0x555be6a2e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.571+0000 7fdf527dd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be665d980 0x555be6a2ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.571+0000 7fdf527dd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a2ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.571+0000 7fdf527dd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555be69a3ca0 con 0x555be6686840 2024-09-21T00:24:03.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.571+0000 7fdf527dd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555be69930d0 con 0x555be6665b60 2024-09-21T00:24:03.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.571+0000 7fdf527dd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555be6996ee0 con 0x555be665d980 2024-09-21T00:24:03.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.571+0000 7fdf49c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 0x555be6a2e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.572+0000 7fdf43e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a2ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.572+0000 7fdf49c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 0x555be6a2e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33066/0 (socket says 172.21.15.27:33066) 2024-09-21T00:24:03.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.572+0000 7fdf43e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a2ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53920/0 (socket says 172.21.15.27:53920) 2024-09-21T00:24:03.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.572+0000 7fdf49c00640 1 -- 172.21.15.27:0/2213423425 learned_addr learned my addr 172.21.15.27:0/2213423425 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:03.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.572+0000 7fdf43e00640 1 -- 172.21.15.27:0/2213423425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be665d980 msgr2=0x555be6a2ea50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:24:03.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.572+0000 7fdf43e00640 1 --2- 172.21.15.27:0/2213423425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be665d980 0x555be6a2ea50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.572+0000 7fdf43e00640 1 -- 172.21.15.27:0/2213423425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 msgr2=0x555be6a2e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:03.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.572+0000 7fdf43e00640 1 --2- 172.21.15.27:0/2213423425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 0x555be6a2e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.572+0000 7fdf43e00640 1 -- 172.21.15.27:0/2213423425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be6945400 con 0x555be6665b60 2024-09-21T00:24:03.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.572+0000 7fdf43e00640 1 --2- 172.21.15.27:0/2213423425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a2ef90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdf30006010 tx=0x7fdf30032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cce26d21176b89d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:03.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.572+0000 7fdf41600640 1 -- 172.21.15.27:0/2824249113 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fdf38010430 con 0x7fdf440757b0 2024-09-21T00:24:03.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.573+0000 7fdf36000640 1 -- 172.21.15.27:0/2213423425 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30040070 con 0x555be6665b60 2024-09-21T00:24:03.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.573+0000 7fdf36000640 1 -- 172.21.15.27:0/2213423425 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf30004030 con 0x555be6665b60 2024-09-21T00:24:03.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.573+0000 7fdf36000640 1 -- 172.21.15.27:0/2213423425 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30035040 con 0x555be6665b60 2024-09-21T00:24:03.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.574+0000 7fdf527dd740 1 -- 172.21.15.27:0/2213423425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 msgr2=0x555be6a2ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:03.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.574+0000 7fdf527dd740 1 --2- 172.21.15.27:0/2213423425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a2ef90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdf30006010 tx=0x7fdf30032b20 comp rx=0 tx=0).stop 2024-09-21T00:24:03.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.574+0000 7fdf527dd740 1 -- 172.21.15.27:0/2213423425 shutdown_connections 2024-09-21T00:24:03.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.574+0000 7fdf527dd740 1 --2- 172.21.15.27:0/2213423425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a2ef90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.574+0000 7fdf527dd740 1 --2- 172.21.15.27:0/2213423425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be665d980 0x555be6a2ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.574+0000 7fdf527dd740 1 --2- 172.21.15.27:0/2213423425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 0x555be6a2e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.574+0000 7fdf527dd740 1 -- 172.21.15.27:0/2213423425 >> 172.21.15.27:0/2213423425 conn(0x555be667a990 msgr2=0x555be6a2c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:03.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.574+0000 7fdf527dd740 1 -- 172.21.15.27:0/2213423425 shutdown_connections 2024-09-21T00:24:03.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.574+0000 7fdf527dd740 1 -- 172.21.15.27:0/2213423425 wait complete. 2024-09-21T00:24:03.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.575+0000 7fdf41600640 1 --2- 172.21.15.27:0/2824249113 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdf2c0b6600 0x7fdf2c0b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.575+0000 7fdf41600640 1 -- 172.21.15.27:0/2824249113 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fdf380e7d30 con 0x7fdf440757b0 2024-09-21T00:24:03.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.575+0000 7fdf43e00640 1 --2- 172.21.15.27:0/2824249113 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdf2c0b6600 0x7fdf2c0b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.575+0000 7fdf527dd740 1 -- start start 2024-09-21T00:24:03.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.575+0000 7fdf527dd740 1 Processor -- start 2024-09-21T00:24:03.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.576+0000 7fdf43e00640 1 --2- 172.21.15.27:0/2824249113 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdf2c0b6600 0x7fdf2c0b8b10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdf30005f70 tx=0x7fdf30000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:03.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.576+0000 7fdf527dd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be665d980 0x555be6a39bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.576+0000 7fdf527dd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a3a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.576+0000 7fdf527dd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 0x555be6a45f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.576+0000 7fdf527dd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555be67f47a0 con 0x555be6686840 2024-09-21T00:24:03.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.576+0000 7fdf527dd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555be6938a10 con 0x555be6665b60 2024-09-21T00:24:03.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.576+0000 7fdf527dd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555be6a3cc90 con 0x555be665d980 2024-09-21T00:24:03.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.577+0000 7fdf527dd740 10 client.0 _ll_register_callbacks cb 0x555be6948be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:24:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.577+0000 7fdf49c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 0x555be6a45f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.577+0000 7fdf49c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 0x555be6a45f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33074/0 (socket says 172.21.15.27:33074) 2024-09-21T00:24:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.577+0000 7fdf49c00640 1 -- 172.21.15.27:0/2006486317 learned_addr learned my addr 172.21.15.27:0/2006486317 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.577+0000 7fdf49200640 1 --2- 172.21.15.27:0/2006486317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be665d980 0x555be6a39bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.577+0000 7fdf43e00640 1 --2- 172.21.15.27:0/2006486317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a3a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.577+0000 7fdf49c00640 1 -- 172.21.15.27:0/2006486317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be665d980 msgr2=0x555be6a39bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.577+0000 7fdf49c00640 1 --2- 172.21.15.27:0/2006486317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be665d980 0x555be6a39bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.577+0000 7fdf49c00640 1 -- 172.21.15.27:0/2006486317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 msgr2=0x555be6a3a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.577+0000 7fdf49c00640 1 --2- 172.21.15.27:0/2006486317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a3a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.577+0000 7fdf49c00640 1 -- 172.21.15.27:0/2006486317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be6700b90 con 0x555be6686840 2024-09-21T00:24:03.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.577+0000 7fdf49200640 1 --2- 172.21.15.27:0/2006486317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be665d980 0x555be6a39bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:24:03.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.577+0000 7fdf43e00640 1 --2- 172.21.15.27:0/2006486317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a3a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:24:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.578+0000 7fdf49c00640 1 --2- 172.21.15.27:0/2006486317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 0x555be6a45f80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c011d10 tx=0x7fdf3c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62fb9a535ce7479 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.578+0000 7fdf2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:24:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.578+0000 7fdf2aa00640 1 -- 172.21.15.27:0/2006486317 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c00fe20 con 0x555be6686840 2024-09-21T00:24:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.578+0000 7fdf2aa00640 1 -- 172.21.15.27:0/2006486317 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fdf3c017050 con 0x555be6686840 2024-09-21T00:24:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.578+0000 7fdf527dd740 1 -- 172.21.15.27:0/2006486317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf30004030 con 0x555be6686840 2024-09-21T00:24:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.580+0000 7fdf2aa00640 1 -- 172.21.15.27:0/2006486317 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdf3c00e5c0 con 0x555be6686840 2024-09-21T00:24:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.580+0000 7fdf527dd740 10 client.15762 fetch_fsmap learned FSMap version 28 2024-09-21T00:24:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.580+0000 7fdf527dd740 1 -- 172.21.15.27:0/2006486317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555be6a599d0 con 0x555be6686840 2024-09-21T00:24:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.580+0000 7fdf2aa00640 1 -- 172.21.15.27:0/2006486317 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf30004030 con 0x555be6686840 2024-09-21T00:24:03.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.580+0000 7fdf2aa00640 1 -- 172.21.15.27:0/2006486317 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdf3c022030 con 0x555be6686840 2024-09-21T00:24:03.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.580+0000 7fdf527dd740 10 client.15762 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:24:03.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.580+0000 7fdf527dd740 10 client.15762 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:24:03.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.580+0000 7fdf527dd740 20 client.15762 populate_metadata read hostname 'smithi027' 2024-09-21T00:24:03.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.580+0000 7fdf527dd740 1 --2- 172.21.15.27:0/2006486317 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x555be66451c0 0x555be6a5d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.580+0000 7fdf527dd740 4 client.15762 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:24:03.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.580+0000 7fdf527dd740 1 -- 172.21.15.27:0/2006486317 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555be69a5bf0 con 0x555be66451c0 2024-09-21T00:24:03.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.580+0000 7fdf49200640 1 --2- 172.21.15.27:0/2006486317 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x555be66451c0 0x555be6a5d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.581+0000 7fdf49200640 1 --2- 172.21.15.27:0/2006486317 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x555be66451c0 0x555be6a5d090 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74ac6c23f56cb85 server_cookie=8e24007202a4d44c in_seq=0 out_seq=0 2024-09-21T00:24:03.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.581+0000 7fdf2aa00640 10 client.15762 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:03.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.583+0000 7fdf2aa00640 1 -- 172.21.15.27:0/2006486317 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555be69a5bf0 con 0x555be66451c0 2024-09-21T00:24:03.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.583+0000 7fdf2aa00640 10 client.15762 handle_command_reply: tid=0 2024-09-21T00:24:03.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.583+0000 7fdf527dd740 1 client.15762 shutdown 2024-09-21T00:24:03.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.584+0000 7fdf527dd740 1 -- 172.21.15.27:0/2006486317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 msgr2=0x555be6a45f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:03.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.584+0000 7fdf527dd740 1 --2- 172.21.15.27:0/2006486317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 0x555be6a45f80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c011d10 tx=0x7fdf3c00f790 comp rx=0 tx=0).stop 2024-09-21T00:24:03.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.584+0000 7fdf527dd740 1 -- 172.21.15.27:0/2006486317 shutdown_connections 2024-09-21T00:24:03.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.584+0000 7fdf527dd740 1 --2- 172.21.15.27:0/2006486317 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x555be66451c0 0x555be6a5d090 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.584+0000 7fdf527dd740 1 --2- 172.21.15.27:0/2006486317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 0x555be6a45f80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.584+0000 7fdf527dd740 1 --2- 172.21.15.27:0/2006486317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a3a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.584+0000 7fdf527dd740 1 --2- 172.21.15.27:0/2006486317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be665d980 0x555be6a39bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.584+0000 7fdf527dd740 1 -- 172.21.15.27:0/2006486317 >> 172.21.15.27:0/2006486317 conn(0x555be667a990 msgr2=0x555be6a316f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:03.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.584+0000 7fdf2aa00640 0 client.15762 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:03.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.584+0000 7fdf527dd740 1 -- 172.21.15.27:0/2006486317 shutdown_connections 2024-09-21T00:24:03.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.584+0000 7fdf527dd740 1 -- 172.21.15.27:0/2006486317 wait complete. 2024-09-21T00:24:03.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.584+0000 7fdf527dd740 20 client.15762 trim_cache size 0 max 16384 2024-09-21T00:24:03.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.604+0000 7fdf527dd740 1 Processor -- start 2024-09-21T00:24:03.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.604+0000 7fdf527dd740 1 -- start start 2024-09-21T00:24:03.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.604+0000 7fdf527dd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be665d980 0x555be6a3b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.604+0000 7fdf527dd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a38430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.604+0000 7fdf527dd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be6686840 0x555be6a2b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.604+0000 7fdf527dd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441ac920 con 0x555be665d980 2024-09-21T00:24:03.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.604+0000 7fdf527dd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441acff0 con 0x555be6665b60 2024-09-21T00:24:03.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.604+0000 7fdf527dd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441aa8c0 con 0x555be6686840 2024-09-21T00:24:03.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.605+0000 7fdf43e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a38430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.605+0000 7fdf43e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a38430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53938/0 (socket says 172.21.15.27:53938) 2024-09-21T00:24:03.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.605+0000 7fdf43e00640 1 -- 172.21.15.27:0/1032203820 learned_addr learned my addr 172.21.15.27:0/1032203820 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:03.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.605+0000 7fdf49200640 1 --2- 172.21.15.27:0/1032203820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be665d980 0x555be6a3b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.605+0000 7fdf49c00640 1 --2- 172.21.15.27:0/1032203820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be6686840 0x555be6a2b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.605+0000 7fdf43e00640 1 -- 172.21.15.27:0/1032203820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be6686840 msgr2=0x555be6a2b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:03.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.605+0000 7fdf43e00640 1 --2- 172.21.15.27:0/1032203820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be6686840 0x555be6a2b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.605+0000 7fdf43e00640 1 -- 172.21.15.27:0/1032203820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be665d980 msgr2=0x555be6a3b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:03.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.605+0000 7fdf43e00640 1 --2- 172.21.15.27:0/1032203820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be665d980 0x555be6a3b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.605+0000 7fdf43e00640 1 -- 172.21.15.27:0/1032203820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be6a30800 con 0x555be6665b60 2024-09-21T00:24:03.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.606+0000 7fdf43e00640 1 --2- 172.21.15.27:0/1032203820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a38430 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf30033000 tx=0x7fdf30031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c20be4432d419be6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:03.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.606+0000 7fdf35600640 1 -- 172.21.15.27:0/1032203820 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30040040 con 0x555be6665b60 2024-09-21T00:24:03.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.606+0000 7fdf35600640 1 -- 172.21.15.27:0/1032203820 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf3005e070 con 0x555be6665b60 2024-09-21T00:24:03.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.606+0000 7fdf35600640 1 -- 172.21.15.27:0/1032203820 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30004a40 con 0x555be6665b60 2024-09-21T00:24:03.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.607+0000 7fdf527dd740 1 -- 172.21.15.27:0/1032203820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 msgr2=0x555be6a38430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:03.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.607+0000 7fdf527dd740 1 --2- 172.21.15.27:0/1032203820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a38430 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf30033000 tx=0x7fdf30031400 comp rx=0 tx=0).stop 2024-09-21T00:24:03.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.607+0000 7fdf527dd740 1 -- 172.21.15.27:0/1032203820 shutdown_connections 2024-09-21T00:24:03.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.607+0000 7fdf527dd740 1 --2- 172.21.15.27:0/1032203820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be6686840 0x555be6a2b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.607+0000 7fdf527dd740 1 --2- 172.21.15.27:0/1032203820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be6665b60 0x555be6a38430 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.607+0000 7fdf527dd740 1 --2- 172.21.15.27:0/1032203820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be665d980 0x555be6a3b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.607+0000 7fdf527dd740 1 -- 172.21.15.27:0/1032203820 >> 172.21.15.27:0/1032203820 conn(0x555be667a990 msgr2=0x555be6a2ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:03.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.607+0000 7fdf527dd740 1 -- 172.21.15.27:0/1032203820 shutdown_connections 2024-09-21T00:24:03.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.612+0000 7fdf527dd740 1 -- 172.21.15.27:0/1032203820 wait complete. 2024-09-21T00:24:03.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.612+0000 7fdf527dd740 1 -- start start 2024-09-21T00:24:03.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.612+0000 7fdf527dd740 1 Processor -- start 2024-09-21T00:24:03.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.613+0000 7fdf527dd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be665d980 0x555be6a3df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.613+0000 7fdf527dd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be6665b60 0x555be6a3e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.613+0000 7fdf527dd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 0x555be6a3e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.613+0000 7fdf527dd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdf440752c0 con 0x555be6686840 2024-09-21T00:24:03.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.613+0000 7fdf527dd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441aa1f0 con 0x555be665d980 2024-09-21T00:24:03.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.613+0000 7fdf527dd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441a9450 con 0x555be6665b60 2024-09-21T00:24:03.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.613+0000 7fdf527dd740 10 client.0 _ll_register_callbacks cb 0x7fdf3c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:24:03.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.613+0000 7fdf49200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be665d980 0x555be6a3df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.613+0000 7fdf49200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be665d980 0x555be6a3df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53950/0 (socket says 172.21.15.27:53950) 2024-09-21T00:24:03.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.613+0000 7fdf49200640 1 -- 172.21.15.27:0/341825483 learned_addr learned my addr 172.21.15.27:0/341825483 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:03.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.614+0000 7fdf43e00640 1 --2- 172.21.15.27:0/341825483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be6665b60 0x555be6a3e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.614+0000 7fdf49c00640 1 --2- 172.21.15.27:0/341825483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 0x555be6a3e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.614+0000 7fdf49200640 1 -- 172.21.15.27:0/341825483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be6665b60 msgr2=0x555be6a3e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:03.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.614+0000 7fdf49200640 1 --2- 172.21.15.27:0/341825483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be6665b60 0x555be6a3e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.614+0000 7fdf49200640 1 -- 172.21.15.27:0/341825483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 msgr2=0x555be6a3e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:03.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.614+0000 7fdf49200640 1 --2- 172.21.15.27:0/341825483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 0x555be6a3e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:03.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.614+0000 7fdf49200640 1 -- 172.21.15.27:0/341825483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be6ac6b00 con 0x555be665d980 2024-09-21T00:24:03.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.614+0000 7fdf49200640 1 --2- 172.21.15.27:0/341825483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be665d980 0x555be6a3df70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf380ad5a0 tx=0x7fdf38009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d132b1c769a3e13e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:03.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.614+0000 7fdf2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:24:03.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.615+0000 7fdf2aa00640 1 -- 172.21.15.27:0/341825483 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf380d2020 con 0x555be665d980 2024-09-21T00:24:03.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.615+0000 7fdf2aa00640 1 -- 172.21.15.27:0/341825483 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fdf380931c0 con 0x555be665d980 2024-09-21T00:24:03.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.615+0000 7fdf527dd740 1 -- 172.21.15.27:0/341825483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf3005e070 con 0x555be665d980 2024-09-21T00:24:03.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.615+0000 7fdf2aa00640 1 -- 172.21.15.27:0/341825483 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdf380ad3b0 con 0x555be665d980 2024-09-21T00:24:03.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.615+0000 7fdf527dd740 10 client.25808 fetch_fsmap learned FSMap version 28 2024-09-21T00:24:03.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.615+0000 7fdf527dd740 1 -- 172.21.15.27:0/341825483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555be6ac7dc0 con 0x555be665d980 2024-09-21T00:24:03.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.616+0000 7fdf2aa00640 1 -- 172.21.15.27:0/341825483 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf3005e070 con 0x555be665d980 2024-09-21T00:24:03.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.616+0000 7fdf2aa00640 1 -- 172.21.15.27:0/341825483 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdf380a93f0 con 0x555be665d980 2024-09-21T00:24:03.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.616+0000 7fdf527dd740 10 client.25808 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:24:03.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.616+0000 7fdf527dd740 10 client.25808 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:24:03.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.616+0000 7fdf527dd740 20 client.25808 populate_metadata read hostname 'smithi027' 2024-09-21T00:24:03.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.616+0000 7fdf527dd740 1 --2- 172.21.15.27:0/341825483 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x555be661a570 0x555be6aca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:03.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.616+0000 7fdf527dd740 4 client.25808 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T00:24:03.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.616+0000 7fdf527dd740 1 -- 172.21.15.27:0/341825483 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x555be69a16a0 con 0x555be661a570 2024-09-21T00:24:03.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.616+0000 7fdf43e00640 1 --2- 172.21.15.27:0/341825483 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x555be661a570 0x555be6aca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:03.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.617+0000 7fdf43e00640 1 --2- 172.21.15.27:0/341825483 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x555be661a570 0x555be6aca4d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b8c3f46faec5fe7 server_cookie=3c38cd5af981ae60 in_seq=0 out_seq=0 2024-09-21T00:24:03.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:03.617+0000 7fdf2aa00640 10 client.25808 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:03.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:03 smithi089 ceph-mon[34481]: pgmap v457: 129 pgs: 129 active+clean; 45 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 659 op/s 2024-09-21T00:24:03.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:03 smithi098 ceph-mon[34725]: pgmap v457: 129 pgs: 129 active+clean; 45 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 659 op/s 2024-09-21T00:24:04.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:03 smithi027 ceph-mon[30745]: pgmap v457: 129 pgs: 129 active+clean; 45 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 659 op/s 2024-09-21T00:24:05.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:05 smithi089 ceph-mon[34481]: pgmap v458: 129 pgs: 129 active+clean; 50 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 693 op/s 2024-09-21T00:24:05.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:05 smithi098 ceph-mon[34725]: pgmap v458: 129 pgs: 129 active+clean; 50 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 693 op/s 2024-09-21T00:24:06.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:05 smithi027 ceph-mon[30745]: pgmap v458: 129 pgs: 129 active+clean; 50 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 693 op/s 2024-09-21T00:24:07.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:07 smithi089 ceph-mon[34481]: pgmap v459: 129 pgs: 129 active+clean; 54 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 712 op/s 2024-09-21T00:24:07.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:07 smithi098 ceph-mon[34725]: pgmap v459: 129 pgs: 129 active+clean; 54 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 712 op/s 2024-09-21T00:24:08.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:07 smithi027 ceph-mon[30745]: pgmap v459: 129 pgs: 129 active+clean; 54 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 712 op/s 2024-09-21T00:24:08.289 INFO:tasks.workunit.client.0.smithi027.stdout: 31 201511 775 156302 757 91444 638 2024-09-21T00:24:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.389+0000 7fe019000640 1 -- 172.21.15.27:0/1448111337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe014105d00 msgr2=0x7fe014106110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.389+0000 7fe019000640 1 --2- 172.21.15.27:0/1448111337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe014105d00 0x7fe014106110 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe004009900 tx=0x7fe00402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.389+0000 7fe019000640 1 -- 172.21.15.27:0/1448111337 shutdown_connections 2024-09-21T00:24:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.389+0000 7fe019000640 1 --2- 172.21.15.27:0/1448111337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe01410bf80 0x7fe0141127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.389+0000 7fe019000640 1 --2- 172.21.15.27:0/1448111337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe01410b630 0x7fe01410ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.389+0000 7fe019000640 1 --2- 172.21.15.27:0/1448111337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe014105d00 0x7fe014106110 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.389+0000 7fe019000640 1 -- 172.21.15.27:0/1448111337 >> 172.21.15.27:0/1448111337 conn(0x7fe014101720 msgr2=0x7fe014103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.389+0000 7fe019000640 1 -- 172.21.15.27:0/1448111337 shutdown_connections 2024-09-21T00:24:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.390+0000 7fe019000640 1 -- 172.21.15.27:0/1448111337 wait complete. 2024-09-21T00:24:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.391+0000 7fe019000640 1 Processor -- start 2024-09-21T00:24:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.391+0000 7fe019000640 1 -- start start 2024-09-21T00:24:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.392+0000 7fe019000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe014105d00 0x7fe01419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.392+0000 7fe019000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe01410b630 0x7fe01419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.392+0000 7fe019000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe01410bf80 0x7fe0141a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.392+0000 7fe019000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe014115c90 con 0x7fe01410bf80 2024-09-21T00:24:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.392+0000 7fe019000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe014115b10 con 0x7fe014105d00 2024-09-21T00:24:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.392+0000 7fe019000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe014115e10 con 0x7fe01410b630 2024-09-21T00:24:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.392+0000 7fe011400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe01410b630 0x7fe01419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.392+0000 7fe011e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe014105d00 0x7fe01419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.392+0000 7fe011400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe01410b630 0x7fe01419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:48896/0 (socket says 172.21.15.27:48896) 2024-09-21T00:24:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.392+0000 7fe011400640 1 -- 172.21.15.27:0/73198081 learned_addr learned my addr 172.21.15.27:0/73198081 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.392+0000 7fe011e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe014105d00 0x7fe01419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53978/0 (socket says 172.21.15.27:53978) 2024-09-21T00:24:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.392+0000 7fe012800640 1 --2- 172.21.15.27:0/73198081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe01410bf80 0x7fe0141a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe011400640 1 -- 172.21.15.27:0/73198081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe014105d00 msgr2=0x7fe01419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe011400640 1 --2- 172.21.15.27:0/73198081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe014105d00 0x7fe01419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe011400640 1 -- 172.21.15.27:0/73198081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe01410bf80 msgr2=0x7fe0141a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe011400640 1 --2- 172.21.15.27:0/73198081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe01410bf80 0x7fe0141a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe011400640 1 -- 172.21.15.27:0/73198081 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0141a25a0 con 0x7fe01410b630 2024-09-21T00:24:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe011400640 1 --2- 172.21.15.27:0/73198081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe01410b630 0x7fe01419cc10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe00000e9f0 tx=0x7fe00000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe00aa00640 1 -- 172.21.15.27:0/73198081 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00000ccc0 con 0x7fe01410b630 2024-09-21T00:24:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe019000640 1 -- 172.21.15.27:0/73198081 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0141a2890 con 0x7fe01410b630 2024-09-21T00:24:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe00aa00640 1 -- 172.21.15.27:0/73198081 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0000108e0 con 0x7fe01410b630 2024-09-21T00:24:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe00aa00640 1 -- 172.21.15.27:0/73198081 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000018860 con 0x7fe01410b630 2024-09-21T00:24:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe019000640 1 -- 172.21.15.27:0/73198081 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0141a9020 con 0x7fe01410b630 2024-09-21T00:24:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe020036740 1 Processor -- start 2024-09-21T00:24:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe020036740 1 -- start start 2024-09-21T00:24:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe00aa00640 1 -- 172.21.15.27:0/73198081 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe000010430 con 0x7fe01410b630 2024-09-21T00:24:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe020036740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef226c1b0 0x557ef2610c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe020036740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef22432f0 0x557ef2611170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe020036740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 0x557ef26116b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe020036740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557ef25866b0 con 0x557ef226c1b0 2024-09-21T00:24:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe020036740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557ef2575ae0 con 0x557ef22432f0 2024-09-21T00:24:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe020036740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557ef25798f0 con 0x557ef224b4d0 2024-09-21T00:24:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe011e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef22432f0 0x557ef2611170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe011e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef22432f0 0x557ef2611170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53990/0 (socket says 172.21.15.27:53990) 2024-09-21T00:24:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe011e00640 1 -- 172.21.15.27:0/3603927559 learned_addr learned my addr 172.21.15.27:0/3603927559 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.393+0000 7fe012800640 1 --2- 172.21.15.27:0/3603927559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef226c1b0 0x557ef2610c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011e00640 1 -- 172.21.15.27:0/3603927559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 msgr2=0x557ef26116b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:24:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011e00640 1 --2- 172.21.15.27:0/3603927559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 0x557ef26116b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011e00640 1 -- 172.21.15.27:0/3603927559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef226c1b0 msgr2=0x557ef2610c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011e00640 1 --2- 172.21.15.27:0/3603927559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef226c1b0 0x557ef2610c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011e00640 1 -- 172.21.15.27:0/3603927559 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ef2542780 con 0x557ef22432f0 2024-09-21T00:24:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011e00640 1 --2- 172.21.15.27:0/3603927559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef22432f0 0x557ef2611170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe004009900 tx=0x7fe004004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fe1a06e0c5308ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fdfff400640 1 -- 172.21.15.27:0/3603927559 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004040070 con 0x557ef22432f0 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fdfff400640 1 -- 172.21.15.27:0/3603927559 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe004005500 con 0x557ef22432f0 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fdfff400640 1 -- 172.21.15.27:0/3603927559 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004035040 con 0x557ef22432f0 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 -- 172.21.15.27:0/3603927559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef22432f0 msgr2=0x557ef2611170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 --2- 172.21.15.27:0/3603927559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef22432f0 0x557ef2611170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe004009900 tx=0x7fe004004040 comp rx=0 tx=0).stop 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe00aa00640 1 --2- 172.21.15.27:0/73198081 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdff80b6680 0x7fdff80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 -- 172.21.15.27:0/3603927559 shutdown_connections 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 --2- 172.21.15.27:0/3603927559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 0x557ef26116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 --2- 172.21.15.27:0/3603927559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef22432f0 0x557ef2611170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 --2- 172.21.15.27:0/3603927559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef226c1b0 0x557ef2610c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 -- 172.21.15.27:0/3603927559 >> 172.21.15.27:0/3603927559 conn(0x557ef2260300 msgr2=0x557ef260eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 -- 172.21.15.27:0/3603927559 shutdown_connections 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe00aa00640 1 -- 172.21.15.27:0/73198081 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fe0000e7db0 con 0x7fe01410b630 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe012800640 1 --2- 172.21.15.27:0/73198081 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdff80b6680 0x7fdff80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 -- 172.21.15.27:0/3603927559 wait complete. 2024-09-21T00:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 -- start start 2024-09-21T00:24:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe012800640 1 --2- 172.21.15.27:0/73198081 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdff80b6680 0x7fdff80b8b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe00c005fd0 tx=0x7fe00c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 Processor -- start 2024-09-21T00:24:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef22432f0 0x557ef261c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 0x557ef261c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef226c1b0 0x557ef261cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557ef24adab0 con 0x557ef22432f0 2024-09-21T00:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557ef251c570 con 0x557ef226c1b0 2024-09-21T00:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557ef261f350 con 0x557ef224b4d0 2024-09-21T00:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 10 client.0 _ll_register_callbacks cb 0x557ef252b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe012800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef226c1b0 0x557ef261cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 0x557ef261c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 0x557ef261c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:48906/0 (socket says 172.21.15.27:48906) 2024-09-21T00:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011400640 1 -- 172.21.15.27:0/1811855125 learned_addr learned my addr 172.21.15.27:0/1811855125 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe012800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef226c1b0 0x557ef261cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53996/0 (socket says 172.21.15.27:53996) 2024-09-21T00:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011e00640 1 --2- 172.21.15.27:0/1811855125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef22432f0 0x557ef261c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011400640 1 -- 172.21.15.27:0/1811855125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef226c1b0 msgr2=0x557ef261cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011400640 1 --2- 172.21.15.27:0/1811855125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef226c1b0 0x557ef261cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011400640 1 -- 172.21.15.27:0/1811855125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef22432f0 msgr2=0x557ef261c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011400640 1 --2- 172.21.15.27:0/1811855125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef22432f0 0x557ef261c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011400640 1 -- 172.21.15.27:0/1811855125 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ef22d7790 con 0x557ef224b4d0 2024-09-21T00:24:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe011400640 1 --2- 172.21.15.27:0/1811855125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 0x557ef261c820 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe0000ad850 tx=0x7fe00000ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=828456d4f2b22858 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fdff3e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:24:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fdff3e00640 1 -- 172.21.15.27:0/1811855125 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0000ad030 con 0x557ef224b4d0 2024-09-21T00:24:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fe020036740 1 -- 172.21.15.27:0/1811855125 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe004005500 con 0x557ef224b4d0 2024-09-21T00:24:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.399+0000 7fdff3e00640 1 -- 172.21.15.27:0/1811855125 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fe0000ac590 con 0x557ef224b4d0 2024-09-21T00:24:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.404+0000 7fdff3e00640 1 -- 172.21.15.27:0/1811855125 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe000010720 con 0x557ef224b4d0 2024-09-21T00:24:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.404+0000 7fe020036740 10 client.25789 fetch_fsmap learned FSMap version 28 2024-09-21T00:24:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.404+0000 7fe020036740 1 -- 172.21.15.27:0/1811855125 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ef2350620 con 0x557ef224b4d0 2024-09-21T00:24:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.404+0000 7fdff3e00640 1 -- 172.21.15.27:0/1811855125 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe004005500 con 0x557ef224b4d0 2024-09-21T00:24:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.404+0000 7fdff3e00640 1 -- 172.21.15.27:0/1811855125 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe0000b3480 con 0x557ef224b4d0 2024-09-21T00:24:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.404+0000 7fe020036740 10 client.25789 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:24:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.404+0000 7fe020036740 10 client.25789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:24:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.404+0000 7fe020036740 20 client.25789 populate_metadata read hostname 'smithi027' 2024-09-21T00:24:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.404+0000 7fe020036740 1 --2- 172.21.15.27:0/1811855125 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557ef222ad80 0x557ef262e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.404+0000 7fe020036740 4 client.25789 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:24:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.404+0000 7fe020036740 1 -- 172.21.15.27:0/1811855125 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ef2588600 con 0x557ef222ad80 2024-09-21T00:24:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.405+0000 7fe011e00640 1 --2- 172.21.15.27:0/1811855125 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557ef222ad80 0x557ef262e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.405+0000 7fe011e00640 1 --2- 172.21.15.27:0/1811855125 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557ef222ad80 0x557ef262e390 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17100f44e2751241 server_cookie=4008e47c8f7f09c5 in_seq=0 out_seq=0 2024-09-21T00:24:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.405+0000 7fdff3e00640 10 client.25789 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.407+0000 7fdff3e00640 1 -- 172.21.15.27:0/1811855125 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557ef2588600 con 0x557ef222ad80 2024-09-21T00:24:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.408+0000 7fdff3e00640 10 client.25789 handle_command_reply: tid=0 2024-09-21T00:24:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.408+0000 7fe020036740 1 client.25789 shutdown 2024-09-21T00:24:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.408+0000 7fe020036740 1 -- 172.21.15.27:0/1811855125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 msgr2=0x557ef261c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.408+0000 7fe020036740 1 --2- 172.21.15.27:0/1811855125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 0x557ef261c820 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe0000ad850 tx=0x7fe00000ce60 comp rx=0 tx=0).stop 2024-09-21T00:24:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.408+0000 7fe020036740 1 -- 172.21.15.27:0/1811855125 shutdown_connections 2024-09-21T00:24:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.408+0000 7fe020036740 1 --2- 172.21.15.27:0/1811855125 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557ef222ad80 0x557ef262e390 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.408+0000 7fe020036740 1 --2- 172.21.15.27:0/1811855125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef226c1b0 0x557ef261cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.408+0000 7fdff3e00640 0 client.25789 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.408+0000 7fe020036740 1 --2- 172.21.15.27:0/1811855125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 0x557ef261c820 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.408+0000 7fe020036740 1 --2- 172.21.15.27:0/1811855125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef22432f0 0x557ef261c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.408+0000 7fe020036740 1 -- 172.21.15.27:0/1811855125 >> 172.21.15.27:0/1811855125 conn(0x557ef2260300 msgr2=0x557ef2611c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.408+0000 7fe020036740 1 -- 172.21.15.27:0/1811855125 shutdown_connections 2024-09-21T00:24:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.408+0000 7fe020036740 1 -- 172.21.15.27:0/1811855125 wait complete. 2024-09-21T00:24:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.408+0000 7fe020036740 20 client.25789 trim_cache size 0 max 16384 2024-09-21T00:24:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.424+0000 7fe020036740 1 Processor -- start 2024-09-21T00:24:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.424+0000 7fe020036740 1 -- start start 2024-09-21T00:24:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.424+0000 7fe020036740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef22432f0 0x557ef2612620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.424+0000 7fe020036740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 0x557ef261df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.424+0000 7fe020036740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef226c1b0 0x557ef260e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.424+0000 7fe020036740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141b5950 con 0x557ef226c1b0 2024-09-21T00:24:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.424+0000 7fe020036740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141b3220 con 0x557ef22432f0 2024-09-21T00:24:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.424+0000 7fe020036740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141b1070 con 0x557ef224b4d0 2024-09-21T00:24:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.424+0000 7fe011e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef22432f0 0x557ef2612620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe011400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 0x557ef261df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe011e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef22432f0 0x557ef2612620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54004/0 (socket says 172.21.15.27:54004) 2024-09-21T00:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe011e00640 1 -- 172.21.15.27:0/4118600249 learned_addr learned my addr 172.21.15.27:0/4118600249 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe012800640 1 --2- 172.21.15.27:0/4118600249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef226c1b0 0x557ef260e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe011e00640 1 -- 172.21.15.27:0/4118600249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 msgr2=0x557ef261df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe011e00640 1 --2- 172.21.15.27:0/4118600249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 0x557ef261df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe011e00640 1 -- 172.21.15.27:0/4118600249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef226c1b0 msgr2=0x557ef260e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe011e00640 1 --2- 172.21.15.27:0/4118600249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef226c1b0 0x557ef260e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe011e00640 1 -- 172.21.15.27:0/4118600249 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ef2613160 con 0x557ef22432f0 2024-09-21T00:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe011400640 1 --2- 172.21.15.27:0/4118600249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 0x557ef261df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe011e00640 1 --2- 172.21.15.27:0/4118600249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef22432f0 0x557ef2612620 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe004031440 tx=0x7fe004000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44c273fb6191cea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fdffea00640 1 -- 172.21.15.27:0/4118600249 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004040040 con 0x557ef22432f0 2024-09-21T00:24:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fdffea00640 1 -- 172.21.15.27:0/4118600249 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe004004970 con 0x557ef22432f0 2024-09-21T00:24:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fdffea00640 1 -- 172.21.15.27:0/4118600249 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004035040 con 0x557ef22432f0 2024-09-21T00:24:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 -- 172.21.15.27:0/4118600249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef22432f0 msgr2=0x557ef2612620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 --2- 172.21.15.27:0/4118600249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef22432f0 0x557ef2612620 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe004031440 tx=0x7fe004000bc0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 -- 172.21.15.27:0/4118600249 shutdown_connections 2024-09-21T00:24:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 --2- 172.21.15.27:0/4118600249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef226c1b0 0x557ef260e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 --2- 172.21.15.27:0/4118600249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef224b4d0 0x557ef261df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 --2- 172.21.15.27:0/4118600249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef22432f0 0x557ef2612620 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 -- 172.21.15.27:0/4118600249 >> 172.21.15.27:0/4118600249 conn(0x557ef2260300 msgr2=0x557ef261d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 -- 172.21.15.27:0/4118600249 shutdown_connections 2024-09-21T00:24:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 -- 172.21.15.27:0/4118600249 wait complete. 2024-09-21T00:24:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 -- start start 2024-09-21T00:24:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 Processor -- start 2024-09-21T00:24:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef22432f0 0x557ef2625000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef224b4d0 0x557ef2620520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef226c1b0 0x557ef2620c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141b16e0 con 0x557ef22432f0 2024-09-21T00:24:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141b2b50 con 0x557ef224b4d0 2024-09-21T00:24:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141b1db0 con 0x557ef226c1b0 2024-09-21T00:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe020036740 10 client.0 _ll_register_callbacks cb 0x7fe0000056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe011400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef224b4d0 0x557ef2620520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe011e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef22432f0 0x557ef2625000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe011400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef224b4d0 0x557ef2620520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54010/0 (socket says 172.21.15.27:54010) 2024-09-21T00:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.425+0000 7fe011400640 1 -- 172.21.15.27:0/3970593810 learned_addr learned my addr 172.21.15.27:0/3970593810 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe012800640 1 --2- 172.21.15.27:0/3970593810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef226c1b0 0x557ef2620c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe011400640 1 -- 172.21.15.27:0/3970593810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef226c1b0 msgr2=0x557ef2620c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe011400640 1 --2- 172.21.15.27:0/3970593810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef226c1b0 0x557ef2620c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe011400640 1 -- 172.21.15.27:0/3970593810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef22432f0 msgr2=0x557ef2625000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe011400640 1 --2- 172.21.15.27:0/3970593810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef22432f0 0x557ef2625000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe011400640 1 -- 172.21.15.27:0/3970593810 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ef2697d70 con 0x557ef224b4d0 2024-09-21T00:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe011400640 1 --2- 172.21.15.27:0/3970593810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef224b4d0 0x557ef2620520 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe0000b38a0 tx=0x7fe00001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52c183d37266b11f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fdff3e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:24:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fdff3e00640 1 -- 172.21.15.27:0/3970593810 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0000c7060 con 0x557ef224b4d0 2024-09-21T00:24:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe020036740 1 -- 172.21.15.27:0/3970593810 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe004004970 con 0x557ef224b4d0 2024-09-21T00:24:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fdff3e00640 1 -- 172.21.15.27:0/3970593810 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fe0000b69a0 con 0x557ef224b4d0 2024-09-21T00:24:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fdff3e00640 1 -- 172.21.15.27:0/3970593810 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe0000ad030 con 0x557ef224b4d0 2024-09-21T00:24:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe020036740 10 client.25826 fetch_fsmap learned FSMap version 28 2024-09-21T00:24:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe020036740 1 -- 172.21.15.27:0/3970593810 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ef2698f10 con 0x557ef224b4d0 2024-09-21T00:24:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fdff3e00640 1 -- 172.21.15.27:0/3970593810 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe004004970 con 0x557ef224b4d0 2024-09-21T00:24:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fdff3e00640 1 -- 172.21.15.27:0/3970593810 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe000018a00 con 0x557ef224b4d0 2024-09-21T00:24:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe020036740 10 client.25826 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:24:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe020036740 10 client.25826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:24:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe020036740 20 client.25826 populate_metadata read hostname 'smithi027' 2024-09-21T00:24:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe020036740 1 --2- 172.21.15.27:0/3970593810 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557ef22001c0 0x557ef269b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe020036740 4 client.25826 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:24:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.428+0000 7fe020036740 1 -- 172.21.15.27:0/3970593810 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ef25840b0 con 0x557ef22001c0 2024-09-21T00:24:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.452+0000 7fe011e00640 1 --2- 172.21.15.27:0/3970593810 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557ef22001c0 0x557ef269b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.464+0000 7fe011e00640 1 --2- 172.21.15.27:0/3970593810 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557ef22001c0 0x557ef269b6a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a15c0d7b3f848942 server_cookie=66baa83db60dcbe5 in_seq=0 out_seq=0 2024-09-21T00:24:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:09.464+0000 7fdff3e00640 10 client.25826 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:09.883 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:09 smithi027 ceph-mon[30745]: pgmap v460: 129 pgs: 129 active+clean; 66 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 689 op/s 2024-09-21T00:24:09.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:09 smithi089 ceph-mon[34481]: pgmap v460: 129 pgs: 129 active+clean; 66 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 689 op/s 2024-09-21T00:24:09.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:09 smithi098 ceph-mon[34725]: pgmap v460: 129 pgs: 129 active+clean; 66 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 689 op/s 2024-09-21T00:24:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.867+0000 7fdff3e00640 1 -- 172.21.15.27:0/3970593810 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ef25840b0 con 0x557ef22001c0 2024-09-21T00:24:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.867+0000 7fdff3e00640 10 client.25826 handle_command_reply: tid=0 2024-09-21T00:24:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.867+0000 7fe020036740 1 client.25826 shutdown 2024-09-21T00:24:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.867+0000 7fe020036740 1 -- 172.21.15.27:0/3970593810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef224b4d0 msgr2=0x557ef2620520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.867+0000 7fe020036740 1 --2- 172.21.15.27:0/3970593810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef224b4d0 0x557ef2620520 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe0000b38a0 tx=0x7fe00001f000 comp rx=0 tx=0).stop 2024-09-21T00:24:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.867+0000 7fe020036740 1 -- 172.21.15.27:0/3970593810 shutdown_connections 2024-09-21T00:24:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.867+0000 7fe020036740 1 --2- 172.21.15.27:0/3970593810 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557ef22001c0 0x557ef269b6a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.867+0000 7fe020036740 1 --2- 172.21.15.27:0/3970593810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ef226c1b0 0x557ef2620c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.867+0000 7fe020036740 1 --2- 172.21.15.27:0/3970593810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557ef224b4d0 0x557ef2620520 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.867+0000 7fdff3e00640 0 client.25826 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.867+0000 7fe020036740 1 --2- 172.21.15.27:0/3970593810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ef22432f0 0x557ef2625000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.867+0000 7fe020036740 1 -- 172.21.15.27:0/3970593810 >> 172.21.15.27:0/3970593810 conn(0x557ef2260300 msgr2=0x557ef26123f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.867+0000 7fe020036740 1 -- 172.21.15.27:0/3970593810 shutdown_connections 2024-09-21T00:24:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.868+0000 7fe020036740 1 -- 172.21.15.27:0/3970593810 wait complete. 2024-09-21T00:24:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.868+0000 7fe020036740 20 client.25826 trim_cache size 0 max 16384 2024-09-21T00:24:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:24:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:24:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:24:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:24:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.868+0000 7fe019000640 1 -- 172.21.15.27:0/73198081 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdff80b6680 msgr2=0x7fdff80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.868+0000 7fe019000640 1 --2- 172.21.15.27:0/73198081 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdff80b6680 0x7fdff80b8b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe00c005fd0 tx=0x7fe00c009500 comp rx=0 tx=0).stop 2024-09-21T00:24:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.868+0000 7fe019000640 1 -- 172.21.15.27:0/73198081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe01410b630 msgr2=0x7fe01419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.868+0000 7fe019000640 1 --2- 172.21.15.27:0/73198081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe01410b630 0x7fe01419cc10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe00000e9f0 tx=0x7fe00000eeb0 comp rx=0 tx=0).stop 2024-09-21T00:24:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.868+0000 7fe019000640 1 -- 172.21.15.27:0/73198081 shutdown_connections 2024-09-21T00:24:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.868+0000 7fe019000640 1 --2- 172.21.15.27:0/73198081 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdff80b6680 0x7fdff80b8b90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.868+0000 7fe019000640 1 --2- 172.21.15.27:0/73198081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe01410bf80 0x7fe0141a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.868+0000 7fe019000640 1 --2- 172.21.15.27:0/73198081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe01410b630 0x7fe01419cc10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.868+0000 7fe019000640 1 --2- 172.21.15.27:0/73198081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe014105d00 0x7fe01419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.868+0000 7fe019000640 1 -- 172.21.15.27:0/73198081 >> 172.21.15.27:0/73198081 conn(0x7fe014101720 msgr2=0x7fe014103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.868+0000 7fe019000640 1 -- 172.21.15.27:0/73198081 shutdown_connections 2024-09-21T00:24:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:11.868+0000 7fe019000640 1 -- 172.21.15.27:0/73198081 wait complete. 2024-09-21T00:24:11.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:11 smithi089 ceph-mon[34481]: pgmap v461: 129 pgs: 129 active+clean; 76 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 713 op/s 2024-09-21T00:24:11.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:11 smithi098 ceph-mon[34725]: pgmap v461: 129 pgs: 129 active+clean; 76 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 713 op/s 2024-09-21T00:24:12.014 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:11 smithi027 ceph-mon[30745]: pgmap v461: 129 pgs: 129 active+clean; 76 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 713 op/s 2024-09-21T00:24:13.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:13 smithi027 ceph-mon[30745]: pgmap v462: 129 pgs: 129 active+clean; 89 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 801 op/s 2024-09-21T00:24:13.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:13 smithi089 ceph-mon[34481]: pgmap v462: 129 pgs: 129 active+clean; 89 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 801 op/s 2024-09-21T00:24:13.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:13 smithi098 ceph-mon[34725]: pgmap v462: 129 pgs: 129 active+clean; 89 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 801 op/s 2024-09-21T00:24:15.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:15 smithi027 ceph-mon[30745]: pgmap v463: 129 pgs: 129 active+clean; 93 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.6 MiB/s wr, 1.10k op/s 2024-09-21T00:24:15.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:15 smithi089 ceph-mon[34481]: pgmap v463: 129 pgs: 129 active+clean; 93 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.6 MiB/s wr, 1.10k op/s 2024-09-21T00:24:15.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:15 smithi098 ceph-mon[34725]: pgmap v463: 129 pgs: 129 active+clean; 93 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.6 MiB/s wr, 1.10k op/s 2024-09-21T00:24:17.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.485+0000 7fdf2aa00640 1 -- 172.21.15.27:0/341825483 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555be69a16a0 con 0x555be661a570 2024-09-21T00:24:17.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.485+0000 7fdf2aa00640 10 client.25808 handle_command_reply: tid=0 2024-09-21T00:24:17.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.485+0000 7fdf527dd740 1 client.25808 shutdown 2024-09-21T00:24:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.486+0000 7fdf527dd740 1 -- 172.21.15.27:0/341825483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be665d980 msgr2=0x555be6a3df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.486+0000 7fdf527dd740 1 --2- 172.21.15.27:0/341825483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be665d980 0x555be6a3df70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf380ad5a0 tx=0x7fdf38009590 comp rx=0 tx=0).stop 2024-09-21T00:24:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.486+0000 7fdf527dd740 1 -- 172.21.15.27:0/341825483 shutdown_connections 2024-09-21T00:24:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.486+0000 7fdf527dd740 1 --2- 172.21.15.27:0/341825483 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x555be661a570 0x555be6aca4d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.486+0000 7fdf527dd740 1 --2- 172.21.15.27:0/341825483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555be6686840 0x555be6a3e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.486+0000 7fdf527dd740 1 --2- 172.21.15.27:0/341825483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555be6665b60 0x555be6a3e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.486+0000 7fdf527dd740 1 --2- 172.21.15.27:0/341825483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555be665d980 0x555be6a3df70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.486+0000 7fdf2aa00640 0 client.25808 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.486+0000 7fdf527dd740 1 -- 172.21.15.27:0/341825483 >> 172.21.15.27:0/341825483 conn(0x555be667a990 msgr2=0x555be6a31fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.486+0000 7fdf527dd740 1 -- 172.21.15.27:0/341825483 shutdown_connections 2024-09-21T00:24:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.486+0000 7fdf527dd740 1 -- 172.21.15.27:0/341825483 wait complete. 2024-09-21T00:24:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.486+0000 7fdf527dd740 20 client.25808 trim_cache size 0 max 16384 2024-09-21T00:24:17.489 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:24:17.489 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 28, 2024-09-21T00:24:17.489 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24761, 2024-09-21T00:24:17.489 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 10, 2024-09-21T00:24:17.489 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T00:24:17.489 INFO:teuthology.orchestra.run.smithi027.stdout: "d8b515dc": { 2024-09-21T00:24:17.489 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 10, 2024-09-21T00:24:17.489 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T00:24:17.489 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:24:17.489 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T00:24:17.489 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:24:17.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:24:17.489 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T00:24:17.490 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 147.8, 2024-09-21T00:24:17.490 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T00:24:17.490 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T00:24:17.490 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T00:24:17.490 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:24:17.490 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T00:24:17.490 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:24:17.490 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:24:17.490 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:24:17.490 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:24:17.490 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:24:17.490 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:24:17.490 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:24:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.487+0000 7fdf4a600640 1 -- 172.21.15.27:0/2824249113 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdf2c0b6600 msgr2=0x7fdf2c0b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.487+0000 7fdf4a600640 1 --2- 172.21.15.27:0/2824249113 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdf2c0b6600 0x7fdf2c0b8b10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdf30005f70 tx=0x7fdf30000bc0 comp rx=0 tx=0).stop 2024-09-21T00:24:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.487+0000 7fdf4a600640 1 -- 172.21.15.27:0/2824249113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf440757b0 msgr2=0x7fdf4419c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.487+0000 7fdf4a600640 1 --2- 172.21.15.27:0/2824249113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf440757b0 0x7fdf4419c520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800e9f0 tx=0x7fdf3800eeb0 comp rx=0 tx=0).stop 2024-09-21T00:24:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.487+0000 7fdf4a600640 1 -- 172.21.15.27:0/2824249113 shutdown_connections 2024-09-21T00:24:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.487+0000 7fdf4a600640 1 --2- 172.21.15.27:0/2824249113 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fdf2c0b6600 0x7fdf2c0b8b10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.487+0000 7fdf4a600640 1 --2- 172.21.15.27:0/2824249113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4410ff20 0x7fdf441a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.487+0000 7fdf4a600640 1 --2- 172.21.15.27:0/2824249113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf44076100 0x7fdf4419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.487+0000 7fdf4a600640 1 --2- 172.21.15.27:0/2824249113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf440757b0 0x7fdf4419c520 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.487+0000 7fdf4a600640 1 -- 172.21.15.27:0/2824249113 >> 172.21.15.27:0/2824249113 conn(0x7fdf440ff430 msgr2=0x7fdf441089b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.487+0000 7fdf4a600640 1 -- 172.21.15.27:0/2824249113 shutdown_connections 2024-09-21T00:24:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:17.487+0000 7fdf4a600640 1 -- 172.21.15.27:0/2824249113 wait complete. 2024-09-21T00:24:17.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:17 smithi027 ceph-mon[30745]: pgmap v464: 129 pgs: 129 active+clean; 101 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 1.20k op/s 2024-09-21T00:24:17.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:17 smithi098 ceph-mon[34725]: pgmap v464: 129 pgs: 129 active+clean; 101 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 1.20k op/s 2024-09-21T00:24:18.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:17 smithi089 ceph-mon[34481]: pgmap v464: 129 pgs: 129 active+clean; 101 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 1.20k op/s 2024-09-21T00:24:18.290 INFO:tasks.workunit.client.0.smithi027.stdout: 31 651 0 338 0 88 0 2024-09-21T00:24:18.877 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd8b515dc', quiesce took 20.9 seconds. Will release after: 6.900000000000002 2024-09-21T00:24:19.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:19 smithi098 ceph-mon[34725]: pgmap v465: 129 pgs: 129 active+clean; 130 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.8 MiB/s wr, 1.30k op/s 2024-09-21T00:24:20.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:19 smithi027 ceph-mon[30745]: pgmap v465: 129 pgs: 129 active+clean; 130 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.8 MiB/s wr, 1.30k op/s 2024-09-21T00:24:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:19 smithi089 ceph-mon[34481]: pgmap v465: 129 pgs: 129 active+clean; 130 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.8 MiB/s wr, 1.30k op/s 2024-09-21T00:24:21.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:21 smithi098 ceph-mon[34725]: pgmap v466: 129 pgs: 129 active+clean; 145 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.1 MiB/s wr, 1.29k op/s 2024-09-21T00:24:22.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:21 smithi027 ceph-mon[30745]: pgmap v466: 129 pgs: 129 active+clean; 145 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.1 MiB/s wr, 1.29k op/s 2024-09-21T00:24:22.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:21 smithi089 ceph-mon[34481]: pgmap v466: 129 pgs: 129 active+clean; 145 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.1 MiB/s wr, 1.29k op/s 2024-09-21T00:24:23.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:23 smithi098 ceph-mon[34725]: pgmap v467: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.4 MiB/s wr, 1.08k op/s 2024-09-21T00:24:24.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:23 smithi027 ceph-mon[30745]: pgmap v467: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.4 MiB/s wr, 1.08k op/s 2024-09-21T00:24:24.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:23 smithi089 ceph-mon[34481]: pgmap v467: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.4 MiB/s wr, 1.08k op/s 2024-09-21T00:24:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.096+0000 7f565f400640 1 -- 172.21.15.27:0/1510615662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5658104920 msgr2=0x7f5658104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.096+0000 7f565f400640 1 --2- 172.21.15.27:0/1510615662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5658104920 0x7f5658104d30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f564c009900 tx=0x7f564c02a3f0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.096+0000 7f565f400640 1 -- 172.21.15.27:0/1510615662 shutdown_connections 2024-09-21T00:24:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.096+0000 7f565f400640 1 --2- 172.21.15.27:0/1510615662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f565810dd80 0x7f5658110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.096+0000 7f565f400640 1 --2- 172.21.15.27:0/1510615662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5658105270 0x7f565810d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.096+0000 7f565f400640 1 --2- 172.21.15.27:0/1510615662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5658104920 0x7f5658104d30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.096+0000 7f565f400640 1 -- 172.21.15.27:0/1510615662 >> 172.21.15.27:0/1510615662 conn(0x7f565807a710 msgr2=0x7f565807ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.097+0000 7f565f400640 1 -- 172.21.15.27:0/1510615662 shutdown_connections 2024-09-21T00:24:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.097+0000 7f565f400640 1 -- 172.21.15.27:0/1510615662 wait complete. 2024-09-21T00:24:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.098+0000 7f565f400640 1 Processor -- start 2024-09-21T00:24:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.098+0000 7f565f400640 1 -- start start 2024-09-21T00:24:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.098+0000 7f565f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5658104920 0x7f565819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.098+0000 7f565f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5658105270 0x7f565819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.098+0000 7f565f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f565810dd80 0x7f565819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.098+0000 7f565f400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5658113a50 con 0x7f5658105270 2024-09-21T00:24:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.098+0000 7f565f400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f56581138d0 con 0x7f5658104920 2024-09-21T00:24:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.098+0000 7f565f400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5658113bd0 con 0x7f565810dd80 2024-09-21T00:24:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.098+0000 7f565ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5658104920 0x7f565819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.098+0000 7f565ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5658104920 0x7f565819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49476/0 (socket says 172.21.15.27:49476) 2024-09-21T00:24:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.099+0000 7f565ca00640 1 -- 172.21.15.27:0/376892458 learned_addr learned my addr 172.21.15.27:0/376892458 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.099+0000 7f5657e00640 1 --2- 172.21.15.27:0/376892458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5658105270 0x7f565819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.099+0000 7f565d400640 1 --2- 172.21.15.27:0/376892458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f565810dd80 0x7f565819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.099+0000 7f565ca00640 1 -- 172.21.15.27:0/376892458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f565810dd80 msgr2=0x7f565819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.099+0000 7f565ca00640 1 --2- 172.21.15.27:0/376892458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f565810dd80 0x7f565819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.099+0000 7f565ca00640 1 -- 172.21.15.27:0/376892458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5658105270 msgr2=0x7f565819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.099+0000 7f565ca00640 1 --2- 172.21.15.27:0/376892458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5658105270 0x7f565819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.099+0000 7f565ca00640 1 -- 172.21.15.27:0/376892458 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56581a03c0 con 0x7f5658104920 2024-09-21T00:24:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.099+0000 7f565ca00640 1 --2- 172.21.15.27:0/376892458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5658104920 0x7f565819a490 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f564c005ab0 tx=0x7f564c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.099+0000 7f5655600640 1 -- 172.21.15.27:0/376892458 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c03f070 con 0x7f5658104920 2024-09-21T00:24:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.100+0000 7f5655600640 1 -- 172.21.15.27:0/376892458 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f564c031b50 con 0x7f5658104920 2024-09-21T00:24:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.100+0000 7f5655600640 1 -- 172.21.15.27:0/376892458 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c035030 con 0x7f5658104920 2024-09-21T00:24:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.100+0000 7f565f400640 1 -- 172.21.15.27:0/376892458 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56581a0650 con 0x7f5658104920 2024-09-21T00:24:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.100+0000 7f565f400640 1 -- 172.21.15.27:0/376892458 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56581a6e40 con 0x7f5658104920 2024-09-21T00:24:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.103+0000 7f5666444740 1 Processor -- start 2024-09-21T00:24:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.103+0000 7f5666444740 1 -- start start 2024-09-21T00:24:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.103+0000 7f5666444740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c41091b0 0x5613c44b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.103+0000 7f5666444740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c40e02f0 0x5613c44b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.103+0000 7f5666444740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.103+0000 7f5666444740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5613c44236b0 con 0x5613c41091b0 2024-09-21T00:24:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.103+0000 7f5666444740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5613c4412ae0 con 0x5613c40e84d0 2024-09-21T00:24:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.103+0000 7f5666444740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5613c44168f0 con 0x5613c40e02f0 2024-09-21T00:24:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.103+0000 7f5655600640 1 -- 172.21.15.27:0/376892458 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f564c042070 con 0x7f5658104920 2024-09-21T00:24:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.104+0000 7f565d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c41091b0 0x5613c44b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.104+0000 7f5657e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.104+0000 7f565d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c41091b0 0x5613c44b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51918/0 (socket says 172.21.15.27:51918) 2024-09-21T00:24:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.104+0000 7f5657e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44b2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49490/0 (socket says 172.21.15.27:49490) 2024-09-21T00:24:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.104+0000 7f565d400640 1 -- 172.21.15.27:0/1411110780 learned_addr learned my addr 172.21.15.27:0/1411110780 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.104+0000 7f565ca00640 1 --2- 172.21.15.27:0/1411110780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c40e02f0 0x5613c44b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.104+0000 7f5657e00640 1 -- 172.21.15.27:0/1411110780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c40e02f0 msgr2=0x5613c44b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.104+0000 7f5657e00640 1 --2- 172.21.15.27:0/1411110780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c40e02f0 0x5613c44b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.104+0000 7f5657e00640 1 -- 172.21.15.27:0/1411110780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c41091b0 msgr2=0x5613c44b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.104+0000 7f5657e00640 1 --2- 172.21.15.27:0/1411110780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c41091b0 0x5613c44b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.104+0000 7f5657e00640 1 -- 172.21.15.27:0/1411110780 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613c43df780 con 0x5613c40e84d0 2024-09-21T00:24:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.104+0000 7f565d400640 1 --2- 172.21.15.27:0/1411110780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c41091b0 0x5613c44b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:24:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.104+0000 7f5657e00640 1 --2- 172.21.15.27:0/1411110780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44b2ac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f565806b500 tx=0x7f5644011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ddc66808a2b1b94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.105+0000 7f5649c00640 1 -- 172.21.15.27:0/1411110780 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564401e070 con 0x5613c40e84d0 2024-09-21T00:24:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.105+0000 7f5649c00640 1 -- 172.21.15.27:0/1411110780 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5644009d70 con 0x5613c40e84d0 2024-09-21T00:24:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.105+0000 7f5649c00640 1 -- 172.21.15.27:0/1411110780 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644018730 con 0x5613c40e84d0 2024-09-21T00:24:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.105+0000 7f5655600640 1 --2- 172.21.15.27:0/376892458 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f56400b6680 0x7f56400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.105+0000 7f5655600640 1 -- 172.21.15.27:0/376892458 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f564c108130 con 0x7f5658104920 2024-09-21T00:24:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.106+0000 7f565d400640 1 --2- 172.21.15.27:0/376892458 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f56400b6680 0x7f56400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.106+0000 7f5666444740 1 -- 172.21.15.27:0/1411110780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 msgr2=0x5613c44b2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.106+0000 7f5666444740 1 --2- 172.21.15.27:0/1411110780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44b2ac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f565806b500 tx=0x7f5644011810 comp rx=0 tx=0).stop 2024-09-21T00:24:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.106+0000 7f5666444740 1 -- 172.21.15.27:0/1411110780 shutdown_connections 2024-09-21T00:24:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.106+0000 7f5666444740 1 --2- 172.21.15.27:0/1411110780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44b2ac0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.106+0000 7f5666444740 1 --2- 172.21.15.27:0/1411110780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c40e02f0 0x5613c44b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.106+0000 7f5666444740 1 --2- 172.21.15.27:0/1411110780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c41091b0 0x5613c44b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.106+0000 7f5666444740 1 -- 172.21.15.27:0/1411110780 >> 172.21.15.27:0/1411110780 conn(0x5613c40fd300 msgr2=0x5613c44aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.106+0000 7f5666444740 1 -- 172.21.15.27:0/1411110780 shutdown_connections 2024-09-21T00:24:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.106+0000 7f5666444740 1 -- 172.21.15.27:0/1411110780 wait complete. 2024-09-21T00:24:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.106+0000 7f565d400640 1 --2- 172.21.15.27:0/376892458 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f56400b6680 0x7f56400b8b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f56500117b0 tx=0x7f5650009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.107+0000 7f5666444740 1 -- start start 2024-09-21T00:24:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.107+0000 7f5666444740 1 Processor -- start 2024-09-21T00:24:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.107+0000 7f5666444740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c40e02f0 0x5613c44bd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.107+0000 7f5666444740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44bdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.107+0000 7f5666444740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c41091b0 0x5613c44c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.107+0000 7f5666444740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5613c434aab0 con 0x5613c40e02f0 2024-09-21T00:24:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.107+0000 7f5666444740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5613c43b9570 con 0x5613c40e84d0 2024-09-21T00:24:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.107+0000 7f5666444740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5613c44c07c0 con 0x5613c41091b0 2024-09-21T00:24:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.108+0000 7f5666444740 10 client.0 _ll_register_callbacks cb 0x5613c43c8100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:24:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.108+0000 7f5657e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44bdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.108+0000 7f565ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c40e02f0 0x5613c44bd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.108+0000 7f5657e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44bdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49496/0 (socket says 172.21.15.27:49496) 2024-09-21T00:24:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.108+0000 7f5657e00640 1 -- 172.21.15.27:0/1711024872 learned_addr learned my addr 172.21.15.27:0/1711024872 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.108+0000 7f565d400640 1 --2- 172.21.15.27:0/1711024872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c41091b0 0x5613c44c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.108+0000 7f5657e00640 1 -- 172.21.15.27:0/1711024872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c41091b0 msgr2=0x5613c44c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.108+0000 7f5657e00640 1 --2- 172.21.15.27:0/1711024872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c41091b0 0x5613c44c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.108+0000 7f5657e00640 1 -- 172.21.15.27:0/1711024872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c40e02f0 msgr2=0x5613c44bd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.108+0000 7f5657e00640 1 --2- 172.21.15.27:0/1711024872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c40e02f0 0x5613c44bd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.108+0000 7f5657e00640 1 -- 172.21.15.27:0/1711024872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613c4174790 con 0x5613c40e84d0 2024-09-21T00:24:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.108+0000 7f565d400640 1 --2- 172.21.15.27:0/1711024872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c41091b0 0x5613c44c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:24:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.109+0000 7f5657e00640 1 --2- 172.21.15.27:0/1711024872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44bdbe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f56440097b0 tx=0x7f56440056b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfabd24c57355 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.109+0000 7f563e000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:24:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.109+0000 7f563e000640 1 -- 172.21.15.27:0/1711024872 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644026020 con 0x5613c40e84d0 2024-09-21T00:24:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.109+0000 7f5666444740 1 -- 172.21.15.27:0/1711024872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5644009d70 con 0x5613c40e84d0 2024-09-21T00:24:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.109+0000 7f563e000640 1 -- 172.21.15.27:0/1711024872 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f56440035f0 con 0x5613c40e84d0 2024-09-21T00:24:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.109+0000 7f563e000640 1 -- 172.21.15.27:0/1711024872 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5644031050 con 0x5613c40e84d0 2024-09-21T00:24:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.109+0000 7f5666444740 10 client.25838 fetch_fsmap learned FSMap version 28 2024-09-21T00:24:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.109+0000 7f5666444740 1 -- 172.21.15.27:0/1711024872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5613c41ed620 con 0x5613c40e84d0 2024-09-21T00:24:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.109+0000 7f563e000640 1 -- 172.21.15.27:0/1711024872 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5644009d70 con 0x5613c40e84d0 2024-09-21T00:24:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.110+0000 7f563e000640 1 -- 172.21.15.27:0/1711024872 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5644021070 con 0x5613c40e84d0 2024-09-21T00:24:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.110+0000 7f5666444740 10 client.25838 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:24:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.110+0000 7f5666444740 10 client.25838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:24:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.110+0000 7f5666444740 20 client.25838 populate_metadata read hostname 'smithi027' 2024-09-21T00:24:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.110+0000 7f5666444740 1 --2- 172.21.15.27:0/1711024872 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5613c40c7d80 0x5613c44cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.110+0000 7f5666444740 4 client.25838 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:24:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.110+0000 7f5666444740 1 -- 172.21.15.27:0/1711024872 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613c4425600 con 0x5613c40c7d80 2024-09-21T00:24:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.110+0000 7f565ca00640 1 --2- 172.21.15.27:0/1711024872 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5613c40c7d80 0x5613c44cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.111+0000 7f565ca00640 1 --2- 172.21.15.27:0/1711024872 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5613c40c7d80 0x5613c44cf6b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d38d2c0c64670e3 server_cookie=3064e7881fa63a40 in_seq=0 out_seq=0 2024-09-21T00:24:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.111+0000 7f563e000640 10 client.25838 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.112+0000 7f563e000640 1 -- 172.21.15.27:0/1711024872 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5613c4425600 con 0x5613c40c7d80 2024-09-21T00:24:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.112+0000 7f563e000640 10 client.25838 handle_command_reply: tid=0 2024-09-21T00:24:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.112+0000 7f5666444740 1 client.25838 shutdown 2024-09-21T00:24:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.112+0000 7f5666444740 1 -- 172.21.15.27:0/1711024872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 msgr2=0x5613c44bdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.112+0000 7f5666444740 1 --2- 172.21.15.27:0/1711024872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44bdbe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f56440097b0 tx=0x7f56440056b0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.112+0000 7f5666444740 1 -- 172.21.15.27:0/1711024872 shutdown_connections 2024-09-21T00:24:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.112+0000 7f5666444740 1 --2- 172.21.15.27:0/1711024872 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5613c40c7d80 0x5613c44cf6b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.112+0000 7f5666444740 1 --2- 172.21.15.27:0/1711024872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c41091b0 0x5613c44c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.112+0000 7f5666444740 1 --2- 172.21.15.27:0/1711024872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44bdbe0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.112+0000 7f563e000640 0 client.25838 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.112+0000 7f5666444740 1 --2- 172.21.15.27:0/1711024872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c40e02f0 0x5613c44bd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.112+0000 7f5666444740 1 -- 172.21.15.27:0/1711024872 >> 172.21.15.27:0/1711024872 conn(0x5613c40fd300 msgr2=0x5613c44b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.113+0000 7f5666444740 1 -- 172.21.15.27:0/1711024872 shutdown_connections 2024-09-21T00:24:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.113+0000 7f5666444740 1 -- 172.21.15.27:0/1711024872 wait complete. 2024-09-21T00:24:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.113+0000 7f5666444740 20 client.25838 trim_cache size 0 max 16384 2024-09-21T00:24:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.129+0000 7f5666444740 1 Processor -- start 2024-09-21T00:24:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.129+0000 7f5666444740 1 -- start start 2024-09-21T00:24:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.129+0000 7f5666444740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c40e02f0 0x5613c44b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.129+0000 7f5666444740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.129+0000 7f5666444740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c41091b0 0x5613c44ba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.129+0000 7f5666444740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f56581b1550 con 0x5613c41091b0 2024-09-21T00:24:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.129+0000 7f5666444740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f56581aee20 con 0x5613c40e84d0 2024-09-21T00:24:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.129+0000 7f5666444740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f56581acc70 con 0x5613c40e02f0 2024-09-21T00:24:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.129+0000 7f565d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c41091b0 0x5613c44ba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.129+0000 7f5657e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.129+0000 7f5657e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49508/0 (socket says 172.21.15.27:49508) 2024-09-21T00:24:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.129+0000 7f5657e00640 1 -- 172.21.15.27:0/242238258 learned_addr learned my addr 172.21.15.27:0/242238258 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.130+0000 7f565ca00640 1 --2- 172.21.15.27:0/242238258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c40e02f0 0x5613c44b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.130+0000 7f5657e00640 1 -- 172.21.15.27:0/242238258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c40e02f0 msgr2=0x5613c44b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.130+0000 7f5657e00640 1 --2- 172.21.15.27:0/242238258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c40e02f0 0x5613c44b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.130+0000 7f5657e00640 1 -- 172.21.15.27:0/242238258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c41091b0 msgr2=0x5613c44ba870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.130+0000 7f5657e00640 1 --2- 172.21.15.27:0/242238258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c41091b0 0x5613c44ba870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.130+0000 7f5657e00640 1 -- 172.21.15.27:0/242238258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613c44be960 con 0x5613c40e84d0 2024-09-21T00:24:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.130+0000 7f565ca00640 1 --2- 172.21.15.27:0/242238258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c40e02f0 0x5613c44b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:24:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.130+0000 7f5657e00640 1 --2- 172.21.15.27:0/242238258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44c99e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5644005450 tx=0x7f5644001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c3f136a912bccf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.130+0000 7f5649200640 1 -- 172.21.15.27:0/242238258 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564403a070 con 0x5613c40e84d0 2024-09-21T00:24:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.130+0000 7f5649200640 1 -- 172.21.15.27:0/242238258 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5644009d70 con 0x5613c40e84d0 2024-09-21T00:24:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.130+0000 7f5649200640 1 -- 172.21.15.27:0/242238258 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644026030 con 0x5613c40e84d0 2024-09-21T00:24:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.131+0000 7f5666444740 1 -- 172.21.15.27:0/242238258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 msgr2=0x5613c44c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.131+0000 7f5666444740 1 --2- 172.21.15.27:0/242238258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44c99e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5644005450 tx=0x7f5644001e50 comp rx=0 tx=0).stop 2024-09-21T00:24:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.131+0000 7f5666444740 1 -- 172.21.15.27:0/242238258 shutdown_connections 2024-09-21T00:24:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.131+0000 7f5666444740 1 --2- 172.21.15.27:0/242238258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c41091b0 0x5613c44ba870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.131+0000 7f5666444740 1 --2- 172.21.15.27:0/242238258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e84d0 0x5613c44c99e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.131+0000 7f5666444740 1 --2- 172.21.15.27:0/242238258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c40e02f0 0x5613c44b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.131+0000 7f5666444740 1 -- 172.21.15.27:0/242238258 >> 172.21.15.27:0/242238258 conn(0x5613c40fd300 msgr2=0x5613c44b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.131+0000 7f5666444740 1 -- 172.21.15.27:0/242238258 shutdown_connections 2024-09-21T00:24:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.131+0000 7f5666444740 1 -- 172.21.15.27:0/242238258 wait complete. 2024-09-21T00:24:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.132+0000 7f5666444740 1 -- start start 2024-09-21T00:24:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.132+0000 7f5666444740 1 Processor -- start 2024-09-21T00:24:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.132+0000 7f5666444740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e02f0 0x5613c44c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f5666444740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c40e84d0 0x5613c44c3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f5666444740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c41091b0 0x5613c4537ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f5666444740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f56581ad2e0 con 0x5613c40e84d0 2024-09-21T00:24:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f5666444740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f56581ae750 con 0x5613c40e02f0 2024-09-21T00:24:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f5666444740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f56581ad9b0 con 0x5613c41091b0 2024-09-21T00:24:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f5666444740 10 client.0 _ll_register_callbacks cb 0x7f564401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:24:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f565ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e02f0 0x5613c44c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f5657e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c40e84d0 0x5613c44c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f565ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e02f0 0x5613c44c2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49522/0 (socket says 172.21.15.27:49522) 2024-09-21T00:24:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f565ca00640 1 -- 172.21.15.27:0/3824293948 learned_addr learned my addr 172.21.15.27:0/3824293948 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f565d400640 1 --2- 172.21.15.27:0/3824293948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c41091b0 0x5613c4537ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f565ca00640 1 -- 172.21.15.27:0/3824293948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c41091b0 msgr2=0x5613c4537ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f565ca00640 1 --2- 172.21.15.27:0/3824293948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c41091b0 0x5613c4537ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f565ca00640 1 -- 172.21.15.27:0/3824293948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c40e84d0 msgr2=0x5613c44c3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f565ca00640 1 --2- 172.21.15.27:0/3824293948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c40e84d0 0x5613c44c3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f565ca00640 1 -- 172.21.15.27:0/3824293948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613c4538530 con 0x5613c40e02f0 2024-09-21T00:24:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.133+0000 7f565ca00640 1 --2- 172.21.15.27:0/3824293948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e02f0 0x5613c44c2e80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f564c0ca210 tx=0x7f564c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ae3a884450be7a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.134+0000 7f563e000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:24:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.134+0000 7f563e000640 1 -- 172.21.15.27:0/3824293948 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c049040 con 0x5613c40e02f0 2024-09-21T00:24:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.134+0000 7f563e000640 1 -- 172.21.15.27:0/3824293948 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f564c0c9eb0 con 0x5613c40e02f0 2024-09-21T00:24:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.134+0000 7f5666444740 1 -- 172.21.15.27:0/3824293948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5644009d70 con 0x5613c40e02f0 2024-09-21T00:24:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.134+0000 7f563e000640 1 -- 172.21.15.27:0/3824293948 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f564c0c93b0 con 0x5613c40e02f0 2024-09-21T00:24:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.134+0000 7f5666444740 10 client.25844 fetch_fsmap learned FSMap version 28 2024-09-21T00:24:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.134+0000 7f5666444740 1 -- 172.21.15.27:0/3824293948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5613c4539750 con 0x5613c40e02f0 2024-09-21T00:24:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.134+0000 7f563e000640 1 -- 172.21.15.27:0/3824293948 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5644009d70 con 0x5613c40e02f0 2024-09-21T00:24:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.134+0000 7f563e000640 1 -- 172.21.15.27:0/3824293948 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f564c0f10b0 con 0x5613c40e02f0 2024-09-21T00:24:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.135+0000 7f5666444740 10 client.25844 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:24:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.135+0000 7f5666444740 10 client.25844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:24:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.135+0000 7f5666444740 20 client.25844 populate_metadata read hostname 'smithi027' 2024-09-21T00:24:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.135+0000 7f5666444740 1 --2- 172.21.15.27:0/3824293948 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5613c409d1c0 0x5613c453bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.135+0000 7f5666444740 4 client.25844 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:24:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.135+0000 7f5666444740 1 -- 172.21.15.27:0/3824293948 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613c44210b0 con 0x5613c409d1c0 2024-09-21T00:24:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.135+0000 7f5657e00640 1 --2- 172.21.15.27:0/3824293948 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5613c409d1c0 0x5613c453bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.135+0000 7f5657e00640 1 --2- 172.21.15.27:0/3824293948 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5613c409d1c0 0x5613c453bf20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c805e1cad44141b2 server_cookie=58986e621c8d6fcd in_seq=0 out_seq=0 2024-09-21T00:24:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.135+0000 7f563e000640 10 client.25844 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.464+0000 7f563e000640 1 -- 172.21.15.27:0/3824293948 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613c44210b0 con 0x5613c409d1c0 2024-09-21T00:24:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.464+0000 7f563e000640 10 client.25844 handle_command_reply: tid=0 2024-09-21T00:24:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.464+0000 7f5666444740 1 client.25844 shutdown 2024-09-21T00:24:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.465+0000 7f5666444740 1 -- 172.21.15.27:0/3824293948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e02f0 msgr2=0x5613c44c2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.465+0000 7f5666444740 1 --2- 172.21.15.27:0/3824293948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e02f0 0x5613c44c2e80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f564c0ca210 tx=0x7f564c009260 comp rx=0 tx=0).stop 2024-09-21T00:24:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.465+0000 7f5666444740 1 -- 172.21.15.27:0/3824293948 shutdown_connections 2024-09-21T00:24:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.465+0000 7f5666444740 1 --2- 172.21.15.27:0/3824293948 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5613c409d1c0 0x5613c453bf20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.465+0000 7f5666444740 1 --2- 172.21.15.27:0/3824293948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613c41091b0 0x5613c4537ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.465+0000 7f5666444740 1 --2- 172.21.15.27:0/3824293948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613c40e84d0 0x5613c44c3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.465+0000 7f5666444740 1 --2- 172.21.15.27:0/3824293948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613c40e02f0 0x5613c44c2e80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.465+0000 7f5666444740 1 -- 172.21.15.27:0/3824293948 >> 172.21.15.27:0/3824293948 conn(0x5613c40fd300 msgr2=0x5613c44beb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.465+0000 7f563e000640 0 client.25844 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.465+0000 7f5666444740 1 -- 172.21.15.27:0/3824293948 shutdown_connections 2024-09-21T00:24:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.465+0000 7f5666444740 1 -- 172.21.15.27:0/3824293948 wait complete. 2024-09-21T00:24:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.465+0000 7f5666444740 20 client.25844 trim_cache size 0 max 16384 2024-09-21T00:24:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:24:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:24:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:24:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:24:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.466+0000 7f565f400640 1 -- 172.21.15.27:0/376892458 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f56400b6680 msgr2=0x7f56400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.466+0000 7f565f400640 1 --2- 172.21.15.27:0/376892458 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f56400b6680 0x7f56400b8b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f56500117b0 tx=0x7f5650009500 comp rx=0 tx=0).stop 2024-09-21T00:24:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.466+0000 7f565f400640 1 -- 172.21.15.27:0/376892458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5658104920 msgr2=0x7f565819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.466+0000 7f565f400640 1 --2- 172.21.15.27:0/376892458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5658104920 0x7f565819a490 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f564c005ab0 tx=0x7f564c003b90 comp rx=0 tx=0).stop 2024-09-21T00:24:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.466+0000 7f565f400640 1 -- 172.21.15.27:0/376892458 shutdown_connections 2024-09-21T00:24:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.466+0000 7f565f400640 1 --2- 172.21.15.27:0/376892458 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f56400b6680 0x7f56400b8b90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.466+0000 7f565f400640 1 --2- 172.21.15.27:0/376892458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f565810dd80 0x7f565819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.466+0000 7f565f400640 1 --2- 172.21.15.27:0/376892458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5658105270 0x7f565819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.466+0000 7f565f400640 1 --2- 172.21.15.27:0/376892458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5658104920 0x7f565819a490 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.466+0000 7f565f400640 1 -- 172.21.15.27:0/376892458 >> 172.21.15.27:0/376892458 conn(0x7f565807a710 msgr2=0x7f56580ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.466+0000 7f565f400640 1 -- 172.21.15.27:0/376892458 shutdown_connections 2024-09-21T00:24:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:25.467+0000 7f565f400640 1 -- 172.21.15.27:0/376892458 wait complete. 2024-09-21T00:24:25.778 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:24:26.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:25 smithi027 ceph-mon[30745]: pgmap v468: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.2 MiB/s wr, 863 op/s 2024-09-21T00:24:26.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:25 smithi089 ceph-mon[34481]: pgmap v468: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.2 MiB/s wr, 863 op/s 2024-09-21T00:24:26.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:25 smithi098 ceph-mon[34725]: pgmap v468: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.2 MiB/s wr, 863 op/s 2024-09-21T00:24:26.246 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:24:27.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.078+0000 7f3208e00640 1 -- 172.21.15.27:0/2802212653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3204105110 msgr2=0x7f320410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:27.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.078+0000 7f3208e00640 1 --2- 172.21.15.27:0/2802212653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3204105110 0x7f320410d660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f31f400fb10 tx=0x7f31f402e5b0 comp rx=0 tx=0).stop 2024-09-21T00:24:27.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.078+0000 7f3208e00640 1 -- 172.21.15.27:0/2802212653 shutdown_connections 2024-09-21T00:24:27.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.078+0000 7f3208e00640 1 --2- 172.21.15.27:0/2802212653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f320410dc20 0x7f3204110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:27.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.078+0000 7f3208e00640 1 --2- 172.21.15.27:0/2802212653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3204105110 0x7f320410d660 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:27.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.078+0000 7f3208e00640 1 --2- 172.21.15.27:0/2802212653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32041047c0 0x7f3204104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:27.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.078+0000 7f3208e00640 1 -- 172.21.15.27:0/2802212653 >> 172.21.15.27:0/2802212653 conn(0x7f32040fe2c0 msgr2=0x7f3204100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:27.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.079+0000 7f3208e00640 1 -- 172.21.15.27:0/2802212653 shutdown_connections 2024-09-21T00:24:27.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.079+0000 7f3208e00640 1 -- 172.21.15.27:0/2802212653 wait complete. 2024-09-21T00:24:27.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.083+0000 7f3208e00640 1 Processor -- start 2024-09-21T00:24:27.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.084+0000 7f3208e00640 1 -- start start 2024-09-21T00:24:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.087+0000 7f3208e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32041047c0 0x7f32041980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.088+0000 7f3208e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3204105110 0x7f3204198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.088+0000 7f3208e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320410dc20 0x7f320419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.091+0000 7f3203400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32041047c0 0x7f32041980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.091+0000 7f3203400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32041047c0 0x7f32041980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:51248/0 (socket says 172.21.15.27:51248) 2024-09-21T00:24:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.092+0000 7f3203e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320410dc20 0x7f320419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.092+0000 7f3203400640 1 -- 172.21.15.27:0/563173032 learned_addr learned my addr 172.21.15.27:0/563173032 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.092+0000 7f3208e00640 1 -- 172.21.15.27:0/563173032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f32041116c0 con 0x7f320410dc20 2024-09-21T00:24:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.092+0000 7f3208e00640 1 -- 172.21.15.27:0/563173032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3204111540 con 0x7f3204105110 2024-09-21T00:24:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.092+0000 7f3208e00640 1 -- 172.21.15.27:0/563173032 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3204111840 con 0x7f32041047c0 2024-09-21T00:24:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.093+0000 7f3203e00640 1 -- 172.21.15.27:0/563173032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32041047c0 msgr2=0x7f32041980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:27.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.093+0000 7f3203e00640 1 --2- 172.21.15.27:0/563173032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32041047c0 0x7f32041980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:27.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.093+0000 7f3203e00640 1 -- 172.21.15.27:0/563173032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3204105110 msgr2=0x7f3204198690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:24:27.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.093+0000 7f3203e00640 1 --2- 172.21.15.27:0/563173032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3204105110 0x7f3204198690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:27.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.093+0000 7f3203e00640 1 -- 172.21.15.27:0/563173032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320419e060 con 0x7f320410dc20 2024-09-21T00:24:27.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.093+0000 7f3203400640 1 --2- 172.21.15.27:0/563173032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32041047c0 0x7f32041980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:24:27.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.096+0000 7f3203e00640 1 --2- 172.21.15.27:0/563173032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320410dc20 0x7f320419d930 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f31fc016fd0 tx=0x7f31fc00ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:27.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.098+0000 7f31fbe00640 1 -- 172.21.15.27:0/563173032 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc004d20 con 0x7f320410dc20 2024-09-21T00:24:27.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.098+0000 7f31fbe00640 1 -- 172.21.15.27:0/563173032 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31fc015e20 con 0x7f320410dc20 2024-09-21T00:24:27.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.098+0000 7f31fbe00640 1 -- 172.21.15.27:0/563173032 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0054c0 con 0x7f320410dc20 2024-09-21T00:24:27.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.098+0000 7f3208e00640 1 -- 172.21.15.27:0/563173032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320419e350 con 0x7f320410dc20 2024-09-21T00:24:27.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.098+0000 7f3208e00640 1 -- 172.21.15.27:0/563173032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32041a4ab0 con 0x7f320410dc20 2024-09-21T00:24:27.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.106+0000 7f3208e00640 1 -- 172.21.15.27:0/563173032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e4005290 con 0x7f320410dc20 2024-09-21T00:24:27.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.112+0000 7f31fbe00640 1 -- 172.21.15.27:0/563173032 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f31fc005660 con 0x7f320410dc20 2024-09-21T00:24:27.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.118+0000 7f31fbe00640 1 --2- 172.21.15.27:0/563173032 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f31e80b67b0 0x7f31e80b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:27.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.119+0000 7f31fbe00640 1 -- 172.21.15.27:0/563173032 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f31fc0f0660 con 0x7f320410dc20 2024-09-21T00:24:27.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.119+0000 7f3203400640 1 --2- 172.21.15.27:0/563173032 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f31e80b67b0 0x7f31e80b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:27.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.120+0000 7f31fbe00640 1 -- 172.21.15.27:0/563173032 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f31fc0f0a80 con 0x7f320410dc20 2024-09-21T00:24:27.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.121+0000 7f3203400640 1 --2- 172.21.15.27:0/563173032 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f31e80b67b0 0x7f31e80b8cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009c10 tx=0x7f31f0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:27.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.620+0000 7f3208e00640 1 -- 172.21.15.27:0/563173032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31e4005580 con 0x7f320410dc20 2024-09-21T00:24:27.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.622+0000 7f31fbe00640 1 -- 172.21.15.27:0/563173032 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11896 (secure 0 0 0) 0x7f31fc0b67c0 con 0x7f320410dc20 2024-09-21T00:24:27.634 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:24:27.635 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"btime":"2024-09-21T00:23:52:563301+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:23:52.207999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:24:27.635 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-09-21T00:24:27.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.630+0000 7f3208e00640 1 -- 172.21.15.27:0/563173032 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f31e80b67b0 msgr2=0x7f31e80b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:27.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.630+0000 7f3208e00640 1 --2- 172.21.15.27:0/563173032 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f31e80b67b0 0x7f31e80b8cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009c10 tx=0x7f31f0005f90 comp rx=0 tx=0).stop 2024-09-21T00:24:27.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.630+0000 7f3208e00640 1 -- 172.21.15.27:0/563173032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320410dc20 msgr2=0x7f320419d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:27.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.630+0000 7f3208e00640 1 --2- 172.21.15.27:0/563173032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320410dc20 0x7f320419d930 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f31fc016fd0 tx=0x7f31fc00ca50 comp rx=0 tx=0).stop 2024-09-21T00:24:27.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.631+0000 7f3208e00640 1 -- 172.21.15.27:0/563173032 shutdown_connections 2024-09-21T00:24:27.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.631+0000 7f3208e00640 1 --2- 172.21.15.27:0/563173032 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f31e80b67b0 0x7f31e80b8cc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:27.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.631+0000 7f3208e00640 1 --2- 172.21.15.27:0/563173032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320410dc20 0x7f320419d930 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:27.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.631+0000 7f3208e00640 1 --2- 172.21.15.27:0/563173032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3204105110 0x7f3204198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:27.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.631+0000 7f3208e00640 1 --2- 172.21.15.27:0/563173032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32041047c0 0x7f32041980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:27.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.631+0000 7f3208e00640 1 -- 172.21.15.27:0/563173032 >> 172.21.15.27:0/563173032 conn(0x7f32040fe2c0 msgr2=0x7f3204107780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:27.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.632+0000 7f3208e00640 1 -- 172.21.15.27:0/563173032 shutdown_connections 2024-09-21T00:24:27.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:27.632+0000 7f3208e00640 1 -- 172.21.15.27:0/563173032 wait complete. 2024-09-21T00:24:28.029 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:27 smithi027 ceph-mon[30745]: pgmap v469: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 505 op/s 2024-09-21T00:24:28.030 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:27 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/563173032' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:24:28.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:27 smithi089 ceph-mon[34481]: pgmap v469: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 505 op/s 2024-09-21T00:24:28.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:27 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/563173032' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:24:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:27 smithi098 ceph-mon[34725]: pgmap v469: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 505 op/s 2024-09-21T00:24:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:27 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/563173032' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:24:28.297 INFO:tasks.workunit.client.0.smithi027.stdout: 31 144028 0 106656 0 52786 0 2024-09-21T00:24:28.731 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id d8b515dc --release --await' 2024-09-21T00:24:28.732 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id d8b515dc --release --await 2024-09-21T00:24:29.647 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:24:30.046 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:29 smithi027 ceph-mon[30745]: pgmap v470: 129 pgs: 129 active+clean; 140 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 348 op/s 2024-09-21T00:24:30.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:29 smithi089 ceph-mon[34481]: pgmap v470: 129 pgs: 129 active+clean; 140 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 348 op/s 2024-09-21T00:24:30.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:29 smithi098 ceph-mon[34725]: pgmap v470: 129 pgs: 129 active+clean; 140 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 348 op/s 2024-09-21T00:24:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.385+0000 7fcfc6200640 1 -- 172.21.15.27:0/1641038515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfc0103b70 msgr2=0x7fcfc0103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.385+0000 7fcfc6200640 1 --2- 172.21.15.27:0/1641038515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfc0103b70 0x7fcfc0103f80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009900 tx=0x7fcfb002a3f0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.385+0000 7fcfc6200640 1 -- 172.21.15.27:0/1641038515 shutdown_connections 2024-09-21T00:24:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.385+0000 7fcfc6200640 1 --2- 172.21.15.27:0/1641038515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc0109e00 0x7fcfc0110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.385+0000 7fcfc6200640 1 --2- 172.21.15.27:0/1641038515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc01094b0 0x7fcfc01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.385+0000 7fcfc6200640 1 --2- 172.21.15.27:0/1641038515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfc0103b70 0x7fcfc0103f80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.385+0000 7fcfc6200640 1 -- 172.21.15.27:0/1641038515 >> 172.21.15.27:0/1641038515 conn(0x7fcfc00ff5e0 msgr2=0x7fcfc0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:30.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.385+0000 7fcfc6200640 1 -- 172.21.15.27:0/1641038515 shutdown_connections 2024-09-21T00:24:30.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.386+0000 7fcfc6200640 1 -- 172.21.15.27:0/1641038515 wait complete. 2024-09-21T00:24:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.387+0000 7fcfc6200640 1 Processor -- start 2024-09-21T00:24:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.387+0000 7fcfc6200640 1 -- start start 2024-09-21T00:24:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.387+0000 7fcfc6200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfc0103b70 0x7fcfc019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.387+0000 7fcfc6200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc01094b0 0x7fcfc019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.387+0000 7fcfc6200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc0109e00 0x7fcfc01a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.387+0000 7fcfc6200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0113ac0 con 0x7fcfc0103b70 2024-09-21T00:24:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.387+0000 7fcfc6200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0113940 con 0x7fcfc0109e00 2024-09-21T00:24:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.387+0000 7fcfc6200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0113c40 con 0x7fcfc01094b0 2024-09-21T00:24:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.388+0000 7fcfbf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfc0103b70 0x7fcfc019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.388+0000 7fcfbf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfc0103b70 0x7fcfc019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51974/0 (socket says 172.21.15.27:51974) 2024-09-21T00:24:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.388+0000 7fcfbf400640 1 -- 172.21.15.27:0/2850573961 learned_addr learned my addr 172.21.15.27:0/2850573961 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.388+0000 7fcfbfe00640 1 --2- 172.21.15.27:0/2850573961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc0109e00 0x7fcfc01a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.388+0000 7fcfbea00640 1 --2- 172.21.15.27:0/2850573961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc01094b0 0x7fcfc019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.388+0000 7fcfbfe00640 1 -- 172.21.15.27:0/2850573961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc01094b0 msgr2=0x7fcfc019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:30.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.388+0000 7fcfbfe00640 1 --2- 172.21.15.27:0/2850573961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc01094b0 0x7fcfc019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.388+0000 7fcfbfe00640 1 -- 172.21.15.27:0/2850573961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfc0103b70 msgr2=0x7fcfc019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.388+0000 7fcfbfe00640 1 --2- 172.21.15.27:0/2850573961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfc0103b70 0x7fcfc019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.388+0000 7fcfbfe00640 1 -- 172.21.15.27:0/2850573961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc01a2640 con 0x7fcfc0109e00 2024-09-21T00:24:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.389+0000 7fcfbfe00640 1 --2- 172.21.15.27:0/2850573961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc0109e00 0x7fcfc01a1f10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009fd0 tx=0x7fcfb800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.389+0000 7fcfb7e00640 1 -- 172.21.15.27:0/2850573961 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800ce20 con 0x7fcfc0109e00 2024-09-21T00:24:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.389+0000 7fcfc6200640 1 -- 172.21.15.27:0/2850573961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc01a2930 con 0x7fcfc0109e00 2024-09-21T00:24:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.389+0000 7fcfb7e00640 1 -- 172.21.15.27:0/2850573961 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb8010430 con 0x7fcfc0109e00 2024-09-21T00:24:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.389+0000 7fcfc6200640 1 -- 172.21.15.27:0/2850573961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc01a9090 con 0x7fcfc0109e00 2024-09-21T00:24:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.389+0000 7fcfb7e00640 1 -- 172.21.15.27:0/2850573961 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800f620 con 0x7fcfc0109e00 2024-09-21T00:24:30.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.392+0000 7fcfcd01f740 1 Processor -- start 2024-09-21T00:24:30.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.393+0000 7fcfb7e00640 1 -- 172.21.15.27:0/2850573961 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fcfb80108e0 con 0x7fcfc0109e00 2024-09-21T00:24:30.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.393+0000 7fcfcd01f740 1 -- start start 2024-09-21T00:24:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.394+0000 7fcfcd01f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e76520 0x5654c121d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.394+0000 7fcfcd01f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e4d660 0x5654c121d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.394+0000 7fcfcd01f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c121dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.394+0000 7fcfcd01f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5654c1194290 con 0x5654c0e4d660 2024-09-21T00:24:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.394+0000 7fcfcd01f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5654c11836c0 con 0x5654c0e55840 2024-09-21T00:24:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.394+0000 7fcfcd01f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5654c11874d0 con 0x5654c0e76520 2024-09-21T00:24:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.394+0000 7fcfbea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c121dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.394+0000 7fcfbf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e4d660 0x5654c121d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.394+0000 7fcfbea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c121dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49574/0 (socket says 172.21.15.27:49574) 2024-09-21T00:24:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.394+0000 7fcfbea00640 1 -- 172.21.15.27:0/1656549004 learned_addr learned my addr 172.21.15.27:0/1656549004 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.394+0000 7fcfbea00640 1 --2- 172.21.15.27:0/1656549004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e76520 0x5654c121d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.395+0000 7fcfbea00640 1 -- 172.21.15.27:0/1656549004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e76520 msgr2=0x5654c121d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.395+0000 7fcfbea00640 1 --2- 172.21.15.27:0/1656549004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e76520 0x5654c121d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.395+0000 7fcfbea00640 1 -- 172.21.15.27:0/1656549004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e4d660 msgr2=0x5654c121d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.395+0000 7fcfbea00640 1 --2- 172.21.15.27:0/1656549004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e4d660 0x5654c121d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.395+0000 7fcfbea00640 1 -- 172.21.15.27:0/1656549004 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654c1110460 con 0x5654c0e55840 2024-09-21T00:24:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.395+0000 7fcfbea00640 1 --2- 172.21.15.27:0/1656549004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c121dc40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcfac011710 tx=0x7fcfac005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2ee2abef8bacc87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.395+0000 7fcfb7e00640 1 --2- 172.21.15.27:0/2850573961 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcfa40b6680 0x7fcfa40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.395+0000 7fcfb7e00640 1 -- 172.21.15.27:0/2850573961 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fcfb80e7e30 con 0x7fcfc0109e00 2024-09-21T00:24:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.395+0000 7fcfabe00640 1 -- 172.21.15.27:0/1656549004 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac005020 con 0x5654c0e55840 2024-09-21T00:24:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.395+0000 7fcfbf400640 1 --2- 172.21.15.27:0/2850573961 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcfa40b6680 0x7fcfa40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.395+0000 7fcfabe00640 1 -- 172.21.15.27:0/1656549004 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfac00f840 con 0x5654c0e55840 2024-09-21T00:24:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.395+0000 7fcfabe00640 1 -- 172.21.15.27:0/1656549004 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac004a00 con 0x5654c0e55840 2024-09-21T00:24:30.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.397+0000 7fcfcd01f740 1 -- 172.21.15.27:0/1656549004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 msgr2=0x5654c121dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:30.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.397+0000 7fcfbf400640 1 --2- 172.21.15.27:0/2850573961 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcfa40b6680 0x7fcfa40b8b90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0031440 tx=0x7fcfb00058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:30.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.397+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/1656549004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c121dc40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcfac011710 tx=0x7fcfac005850 comp rx=0 tx=0).stop 2024-09-21T00:24:30.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.397+0000 7fcfcd01f740 1 -- 172.21.15.27:0/1656549004 shutdown_connections 2024-09-21T00:24:30.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.397+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/1656549004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c121dc40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.397+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/1656549004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e4d660 0x5654c121d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.397+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/1656549004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e76520 0x5654c121d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.397+0000 7fcfcd01f740 1 -- 172.21.15.27:0/1656549004 >> 172.21.15.27:0/1656549004 conn(0x5654c0e6a910 msgr2=0x5654c121b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:30.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.397+0000 7fcfcd01f740 1 -- 172.21.15.27:0/1656549004 shutdown_connections 2024-09-21T00:24:30.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.397+0000 7fcfcd01f740 1 -- 172.21.15.27:0/1656549004 wait complete. 2024-09-21T00:24:30.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.398+0000 7fcfcd01f740 1 -- start start 2024-09-21T00:24:30.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.398+0000 7fcfcd01f740 1 Processor -- start 2024-09-21T00:24:30.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.399+0000 7fcfcd01f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e4d660 0x5654c1228e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.399+0000 7fcfcd01f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e55840 0x5654c1229420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.399+0000 7fcfcd01f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e76520 0x5654c1234c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.399+0000 7fcfcd01f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5654c0fe4bb0 con 0x5654c0e55840 2024-09-21T00:24:30.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.399+0000 7fcfcd01f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5654c111e6c0 con 0x5654c0e4d660 2024-09-21T00:24:30.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.399+0000 7fcfcd01f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5654c122b9a0 con 0x5654c0e76520 2024-09-21T00:24:30.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.399+0000 7fcfcd01f740 10 client.0 _ll_register_callbacks cb 0x5654c1139490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:24:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.399+0000 7fcfbf400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e4d660 0x5654c1228e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.399+0000 7fcfbf400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e4d660 0x5654c1228e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49590/0 (socket says 172.21.15.27:49590) 2024-09-21T00:24:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.399+0000 7fcfbf400640 1 -- 172.21.15.27:0/649306791 learned_addr learned my addr 172.21.15.27:0/649306791 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.399+0000 7fcfbea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e55840 0x5654c1229420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.399+0000 7fcfbfe00640 1 --2- 172.21.15.27:0/649306791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e76520 0x5654c1234c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.400+0000 7fcfbf400640 1 -- 172.21.15.27:0/649306791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e76520 msgr2=0x5654c1234c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:30.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.400+0000 7fcfbf400640 1 --2- 172.21.15.27:0/649306791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e76520 0x5654c1234c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.400+0000 7fcfbf400640 1 -- 172.21.15.27:0/649306791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e55840 msgr2=0x5654c1229420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:30.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.400+0000 7fcfbf400640 1 --2- 172.21.15.27:0/649306791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e55840 0x5654c1229420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.400+0000 7fcfbf400640 1 -- 172.21.15.27:0/649306791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654c0ee1b00 con 0x5654c0e4d660 2024-09-21T00:24:30.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.400+0000 7fcfbf400640 1 --2- 172.21.15.27:0/649306791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e4d660 0x5654c1228e20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009a20 tx=0x7fcfb0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0fd02eca1a45f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:30.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.400+0000 7fcfa8c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:24:30.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.401+0000 7fcfa8c00640 1 -- 172.21.15.27:0/649306791 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb004d030 con 0x5654c0e4d660 2024-09-21T00:24:30.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.401+0000 7fcfcd01f740 1 -- 172.21.15.27:0/649306791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfac00f840 con 0x5654c0e4d660 2024-09-21T00:24:30.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.401+0000 7fcfa8c00640 1 -- 172.21.15.27:0/649306791 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fcfb0003800 con 0x5654c0e4d660 2024-09-21T00:24:30.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.401+0000 7fcfa8c00640 1 -- 172.21.15.27:0/649306791 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcfb005a050 con 0x5654c0e4d660 2024-09-21T00:24:30.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.401+0000 7fcfcd01f740 10 client.25859 fetch_fsmap learned FSMap version 28 2024-09-21T00:24:30.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.401+0000 7fcfcd01f740 1 -- 172.21.15.27:0/649306791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654c0f58960 con 0x5654c0e4d660 2024-09-21T00:24:30.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.402+0000 7fcfa8c00640 1 -- 172.21.15.27:0/649306791 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfac00f840 con 0x5654c0e4d660 2024-09-21T00:24:30.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.402+0000 7fcfa8c00640 1 -- 172.21.15.27:0/649306791 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcfb003d970 con 0x5654c0e4d660 2024-09-21T00:24:30.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.402+0000 7fcfcd01f740 10 client.25859 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:24:30.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.402+0000 7fcfcd01f740 10 client.25859 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:24:30.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.402+0000 7fcfcd01f740 20 client.25859 populate_metadata read hostname 'smithi027' 2024-09-21T00:24:30.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.402+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/649306791 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5654c0e34ea0 0x5654c123aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.402+0000 7fcfcd01f740 4 client.25859 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:24:30.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.402+0000 7fcfcd01f740 1 -- 172.21.15.27:0/649306791 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654c11961e0 con 0x5654c0e34ea0 2024-09-21T00:24:30.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.402+0000 7fcfbea00640 1 --2- 172.21.15.27:0/649306791 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5654c0e34ea0 0x5654c123aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.403+0000 7fcfa8c00640 10 client.25859 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:30.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.403+0000 7fcfbea00640 1 --2- 172.21.15.27:0/649306791 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5654c0e34ea0 0x5654c123aee0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52ba49563d61d1f6 server_cookie=2fce371ed7d1804c in_seq=0 out_seq=0 2024-09-21T00:24:30.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.404+0000 7fcfa8c00640 1 -- 172.21.15.27:0/649306791 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5654c11961e0 con 0x5654c0e34ea0 2024-09-21T00:24:30.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.404+0000 7fcfa8c00640 10 client.25859 handle_command_reply: tid=0 2024-09-21T00:24:30.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.404+0000 7fcfcd01f740 1 client.25859 shutdown 2024-09-21T00:24:30.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.405+0000 7fcfcd01f740 1 -- 172.21.15.27:0/649306791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e4d660 msgr2=0x5654c1228e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:30.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.405+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/649306791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e4d660 0x5654c1228e20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009a20 tx=0x7fcfb0009260 comp rx=0 tx=0).stop 2024-09-21T00:24:30.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.405+0000 7fcfcd01f740 1 -- 172.21.15.27:0/649306791 shutdown_connections 2024-09-21T00:24:30.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.405+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/649306791 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5654c0e34ea0 0x5654c123aee0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.405+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/649306791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e76520 0x5654c1234c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.405+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/649306791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e55840 0x5654c1229420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.405+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/649306791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e4d660 0x5654c1228e20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.405+0000 7fcfcd01f740 1 -- 172.21.15.27:0/649306791 >> 172.21.15.27:0/649306791 conn(0x5654c0e6a910 msgr2=0x5654c1220350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:30.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.405+0000 7fcfa8c00640 0 client.25859 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:30.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.405+0000 7fcfcd01f740 1 -- 172.21.15.27:0/649306791 shutdown_connections 2024-09-21T00:24:30.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.405+0000 7fcfcd01f740 1 -- 172.21.15.27:0/649306791 wait complete. 2024-09-21T00:24:30.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.405+0000 7fcfcd01f740 20 client.25859 trim_cache size 0 max 16384 2024-09-21T00:24:30.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.424+0000 7fcfcd01f740 1 Processor -- start 2024-09-21T00:24:30.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.424+0000 7fcfcd01f740 1 -- start start 2024-09-21T00:24:30.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.424+0000 7fcfcd01f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e4d660 0x5654c1223280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.424+0000 7fcfcd01f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c121a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.424+0000 7fcfcd01f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e76520 0x5654c121aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.424+0000 7fcfcd01f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01b5930 con 0x5654c0e4d660 2024-09-21T00:24:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.424+0000 7fcfcd01f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01b3200 con 0x5654c0e55840 2024-09-21T00:24:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.424+0000 7fcfcd01f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01b1050 con 0x5654c0e76520 2024-09-21T00:24:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.425+0000 7fcfbea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c121a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.425+0000 7fcfbea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c121a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49606/0 (socket says 172.21.15.27:49606) 2024-09-21T00:24:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.425+0000 7fcfbea00640 1 -- 172.21.15.27:0/3233611001 learned_addr learned my addr 172.21.15.27:0/3233611001 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.425+0000 7fcfbfe00640 1 --2- 172.21.15.27:0/3233611001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e76520 0x5654c121aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.425+0000 7fcfbea00640 1 -- 172.21.15.27:0/3233611001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e76520 msgr2=0x5654c121aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.425+0000 7fcfbea00640 1 --2- 172.21.15.27:0/3233611001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e76520 0x5654c121aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.425+0000 7fcfbea00640 1 -- 172.21.15.27:0/3233611001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e4d660 msgr2=0x5654c1223280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.425+0000 7fcfbea00640 1 --2- 172.21.15.27:0/3233611001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e4d660 0x5654c1223280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.425+0000 7fcfbea00640 1 -- 172.21.15.27:0/3233611001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654c12244b0 con 0x5654c0e55840 2024-09-21T00:24:30.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.426+0000 7fcfbea00640 1 --2- 172.21.15.27:0/3233611001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c121a660 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0116e0 tx=0x7fcfac005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0bc934c15e6410a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:30.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.426+0000 7fcfab400640 1 -- 172.21.15.27:0/3233611001 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac00f840 con 0x5654c0e55840 2024-09-21T00:24:30.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.426+0000 7fcfab400640 1 -- 172.21.15.27:0/3233611001 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfac00fb90 con 0x5654c0e55840 2024-09-21T00:24:30.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.426+0000 7fcfab400640 1 -- 172.21.15.27:0/3233611001 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac003970 con 0x5654c0e55840 2024-09-21T00:24:30.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.427+0000 7fcfcd01f740 1 -- 172.21.15.27:0/3233611001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 msgr2=0x5654c121a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:30.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.427+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/3233611001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c121a660 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0116e0 tx=0x7fcfac005f90 comp rx=0 tx=0).stop 2024-09-21T00:24:30.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.427+0000 7fcfcd01f740 1 -- 172.21.15.27:0/3233611001 shutdown_connections 2024-09-21T00:24:30.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.427+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/3233611001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e76520 0x5654c121aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.427+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/3233611001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c121a660 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.427+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/3233611001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e4d660 0x5654c1223280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.427+0000 7fcfcd01f740 1 -- 172.21.15.27:0/3233611001 >> 172.21.15.27:0/3233611001 conn(0x5654c0e6a910 msgr2=0x5654c12274a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:30.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.428+0000 7fcfcd01f740 1 -- 172.21.15.27:0/3233611001 shutdown_connections 2024-09-21T00:24:30.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.428+0000 7fcfcd01f740 1 -- 172.21.15.27:0/3233611001 wait complete. 2024-09-21T00:24:30.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.428+0000 7fcfcd01f740 1 -- start start 2024-09-21T00:24:30.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.428+0000 7fcfcd01f740 1 Processor -- start 2024-09-21T00:24:30.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.429+0000 7fcfcd01f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e4d660 0x5654c122d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.429+0000 7fcfcd01f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c122daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.429+0000 7fcfcd01f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e76520 0x5654c12a7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.429+0000 7fcfcd01f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01b16c0 con 0x5654c0e76520 2024-09-21T00:24:30.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.429+0000 7fcfcd01f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01b2b30 con 0x5654c0e55840 2024-09-21T00:24:30.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.429+0000 7fcfcd01f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01b1d90 con 0x5654c0e4d660 2024-09-21T00:24:30.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.429+0000 7fcfcd01f740 10 client.0 _ll_register_callbacks cb 0x7fcfb0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:24:30.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.429+0000 7fcfbf400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e4d660 0x5654c122d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.429+0000 7fcfbf400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e4d660 0x5654c122d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:51312/0 (socket says 172.21.15.27:51312) 2024-09-21T00:24:30.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.429+0000 7fcfbf400640 1 -- 172.21.15.27:0/2569990421 learned_addr learned my addr 172.21.15.27:0/2569990421 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:30.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.430+0000 7fcfbfe00640 1 --2- 172.21.15.27:0/2569990421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e76520 0x5654c12a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.430+0000 7fcfbea00640 1 --2- 172.21.15.27:0/2569990421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c122daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.430+0000 7fcfbfe00640 1 -- 172.21.15.27:0/2569990421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e4d660 msgr2=0x5654c122d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:30.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.430+0000 7fcfbfe00640 1 --2- 172.21.15.27:0/2569990421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e4d660 0x5654c122d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.430+0000 7fcfbfe00640 1 -- 172.21.15.27:0/2569990421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 msgr2=0x5654c122daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:30.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.430+0000 7fcfbfe00640 1 --2- 172.21.15.27:0/2569990421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c122daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.430+0000 7fcfbfe00640 1 -- 172.21.15.27:0/2569990421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654c12a76c0 con 0x5654c0e76520 2024-09-21T00:24:30.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.431+0000 7fcfbfe00640 1 --2- 172.21.15.27:0/2569990421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e76520 0x5654c12a7080 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80e5190 tx=0x7fcfb80aa010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb1f15ba3f54c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:30.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.431+0000 7fcfa8c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:24:30.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.431+0000 7fcfa8c00640 1 -- 172.21.15.27:0/2569990421 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8098070 con 0x5654c0e76520 2024-09-21T00:24:30.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.431+0000 7fcfa8c00640 1 -- 172.21.15.27:0/2569990421 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fcfb80a91e0 con 0x5654c0e76520 2024-09-21T00:24:30.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.431+0000 7fcfcd01f740 1 -- 172.21.15.27:0/2569990421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfac00fb90 con 0x5654c0e76520 2024-09-21T00:24:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.432+0000 7fcfa8c00640 1 -- 172.21.15.27:0/2569990421 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcfb80adf90 con 0x5654c0e76520 2024-09-21T00:24:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.432+0000 7fcfcd01f740 10 client.15825 fetch_fsmap learned FSMap version 28 2024-09-21T00:24:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.432+0000 7fcfcd01f740 1 -- 172.21.15.27:0/2569990421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654c12aea90 con 0x5654c0e76520 2024-09-21T00:24:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.432+0000 7fcfa8c00640 1 -- 172.21.15.27:0/2569990421 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfac00fb90 con 0x5654c0e76520 2024-09-21T00:24:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.432+0000 7fcfa8c00640 1 -- 172.21.15.27:0/2569990421 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcfb801f020 con 0x5654c0e76520 2024-09-21T00:24:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.432+0000 7fcfcd01f740 10 client.15825 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:24:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.432+0000 7fcfcd01f740 10 client.15825 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:24:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.433+0000 7fcfcd01f740 20 client.15825 populate_metadata read hostname 'smithi027' 2024-09-21T00:24:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.433+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/2569990421 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5654c0e0a250 0x5654c12b1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.433+0000 7fcfcd01f740 4 client.15825 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d8b515dc", "release": true, "await": true}] 2024-09-21T00:24:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.433+0000 7fcfcd01f740 1 -- 172.21.15.27:0/2569990421 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d8b515dc", "release": true, "await": true}) -- 0x5654c1191c90 con 0x5654c0e0a250 2024-09-21T00:24:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.433+0000 7fcfbf400640 1 --2- 172.21.15.27:0/2569990421 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5654c0e0a250 0x5654c12b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.434+0000 7fcfbf400640 1 --2- 172.21.15.27:0/2569990421 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5654c0e0a250 0x5654c12b1330 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=138fa0a8a6baffec server_cookie=5ed57cd628994161 in_seq=0 out_seq=0 2024-09-21T00:24:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.434+0000 7fcfa8c00640 10 client.15825 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:30.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.657+0000 7fcfa8c00640 1 -- 172.21.15.27:0/2569990421 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5654c1191c90 con 0x5654c0e0a250 2024-09-21T00:24:30.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.657+0000 7fcfa8c00640 10 client.15825 handle_command_reply: tid=0 2024-09-21T00:24:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.657+0000 7fcfcd01f740 1 client.15825 shutdown 2024-09-21T00:24:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.658+0000 7fcfcd01f740 1 -- 172.21.15.27:0/2569990421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e76520 msgr2=0x5654c12a7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.658+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/2569990421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e76520 0x5654c12a7080 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80e5190 tx=0x7fcfb80aa010 comp rx=0 tx=0).stop 2024-09-21T00:24:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.658+0000 7fcfcd01f740 1 -- 172.21.15.27:0/2569990421 shutdown_connections 2024-09-21T00:24:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.658+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/2569990421 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5654c0e0a250 0x5654c12b1330 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.658+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/2569990421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654c0e76520 0x5654c12a7080 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.658+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/2569990421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654c0e55840 0x5654c122daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.658+0000 7fcfcd01f740 1 --2- 172.21.15.27:0/2569990421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5654c0e4d660 0x5654c122d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.658+0000 7fcfcd01f740 1 -- 172.21.15.27:0/2569990421 >> 172.21.15.27:0/2569990421 conn(0x5654c0e6a910 msgr2=0x5654c121e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.658+0000 7fcfa8c00640 0 client.15825 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.658+0000 7fcfcd01f740 1 -- 172.21.15.27:0/2569990421 shutdown_connections 2024-09-21T00:24:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.658+0000 7fcfcd01f740 1 -- 172.21.15.27:0/2569990421 wait complete. 2024-09-21T00:24:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.658+0000 7fcfcd01f740 20 client.15825 trim_cache size 0 max 16384 2024-09-21T00:24:30.661 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:24:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 28, 2024-09-21T00:24:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24761, 2024-09-21T00:24:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 12, 2024-09-21T00:24:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T00:24:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: "d8b515dc": { 2024-09-21T00:24:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 12, 2024-09-21T00:24:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T00:24:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:24:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T00:24:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:24:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:24:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T00:24:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 147.8, 2024-09-21T00:24:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T00:24:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T00:24:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T00:24:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:24:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T00:24:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:24:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:24:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:24:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:24:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:24:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:24:30.663 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:24:30.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.659+0000 7fcfc6200640 1 -- 172.21.15.27:0/2850573961 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcfa40b6680 msgr2=0x7fcfa40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:30.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.659+0000 7fcfc6200640 1 --2- 172.21.15.27:0/2850573961 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcfa40b6680 0x7fcfa40b8b90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0031440 tx=0x7fcfb00058d0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.659+0000 7fcfc6200640 1 -- 172.21.15.27:0/2850573961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc0109e00 msgr2=0x7fcfc01a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:30.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.659+0000 7fcfc6200640 1 --2- 172.21.15.27:0/2850573961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc0109e00 0x7fcfc01a1f10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009fd0 tx=0x7fcfb800ef10 comp rx=0 tx=0).stop 2024-09-21T00:24:30.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.659+0000 7fcfc6200640 1 -- 172.21.15.27:0/2850573961 shutdown_connections 2024-09-21T00:24:30.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.660+0000 7fcfc6200640 1 --2- 172.21.15.27:0/2850573961 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcfa40b6680 0x7fcfa40b8b90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.660+0000 7fcfc6200640 1 --2- 172.21.15.27:0/2850573961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc0109e00 0x7fcfc01a1f10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.660+0000 7fcfc6200640 1 --2- 172.21.15.27:0/2850573961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc01094b0 0x7fcfc019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.660+0000 7fcfc6200640 1 --2- 172.21.15.27:0/2850573961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfc0103b70 0x7fcfc019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:30.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.660+0000 7fcfc6200640 1 -- 172.21.15.27:0/2850573961 >> 172.21.15.27:0/2850573961 conn(0x7fcfc00ff5e0 msgr2=0x7fcfc010e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:30.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.661+0000 7fcfc6200640 1 -- 172.21.15.27:0/2850573961 shutdown_connections 2024-09-21T00:24:30.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:30.661+0000 7fcfc6200640 1 -- 172.21.15.27:0/2850573961 wait complete. 2024-09-21T00:24:31.520 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd8b515dc', total seconds elapsed: 33.5 2024-09-21T00:24:31.520 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T00:24:31.520 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:24:31.939 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:24:32.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:31 smithi027 ceph-mon[30745]: pgmap v471: 129 pgs: 129 active+clean; 132 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-09-21T00:24:32.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:31 smithi089 ceph-mon[34481]: pgmap v471: 129 pgs: 129 active+clean; 132 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-09-21T00:24:32.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:31 smithi098 ceph-mon[34725]: pgmap v471: 129 pgs: 129 active+clean; 132 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-09-21T00:24:32.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.703+0000 7f3319a00640 1 -- 172.21.15.27:0/1555507654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3314076080 msgr2=0x7f331410fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.703+0000 7f3319a00640 1 --2- 172.21.15.27:0/1555507654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3314076080 0x7f331410fa30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3300009860 tx=0x7f330002a340 comp rx=0 tx=0).stop 2024-09-21T00:24:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.704+0000 7f3319a00640 1 -- 172.21.15.27:0/1555507654 shutdown_connections 2024-09-21T00:24:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.704+0000 7f3319a00640 1 --2- 172.21.15.27:0/1555507654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f331410ff70 0x7f3314112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.704+0000 7f3319a00640 1 --2- 172.21.15.27:0/1555507654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3314076080 0x7f331410fa30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.704+0000 7f3319a00640 1 --2- 172.21.15.27:0/1555507654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3314075730 0x7f3314075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.704+0000 7f3319a00640 1 -- 172.21.15.27:0/1555507654 >> 172.21.15.27:0/1555507654 conn(0x7f33140ff3e0 msgr2=0x7f3314101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.704+0000 7f3319a00640 1 -- 172.21.15.27:0/1555507654 shutdown_connections 2024-09-21T00:24:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.704+0000 7f3319a00640 1 -- 172.21.15.27:0/1555507654 wait complete. 2024-09-21T00:24:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.706+0000 7f3319a00640 1 Processor -- start 2024-09-21T00:24:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.709+0000 7f3319a00640 1 -- start start 2024-09-21T00:24:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.709+0000 7f3319a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3314075730 0x7f331419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.709+0000 7f3319a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3314076080 0x7f331419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.710+0000 7f3319a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f331410ff70 0x7f33141a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.710+0000 7f3319a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3314115ab0 con 0x7f331410ff70 2024-09-21T00:24:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.710+0000 7f3319a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3314115930 con 0x7f3314075730 2024-09-21T00:24:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.710+0000 7f3319a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3314115c30 con 0x7f3314076080 2024-09-21T00:24:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.710+0000 7f3313e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3314075730 0x7f331419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.710+0000 7f3313e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3314075730 0x7f331419c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49640/0 (socket says 172.21.15.27:49640) 2024-09-21T00:24:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.710+0000 7f3313e00640 1 -- 172.21.15.27:0/3315453475 learned_addr learned my addr 172.21.15.27:0/3315453475 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.710+0000 7f3313400640 1 --2- 172.21.15.27:0/3315453475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3314076080 0x7f331419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.711+0000 7f3313400640 1 -- 172.21.15.27:0/3315453475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3314075730 msgr2=0x7f331419c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.711+0000 7f3313400640 1 --2- 172.21.15.27:0/3315453475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3314075730 0x7f331419c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.711+0000 7f3313400640 1 -- 172.21.15.27:0/3315453475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f331410ff70 msgr2=0x7f33141a1d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:24:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.711+0000 7f3313400640 1 --2- 172.21.15.27:0/3315453475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f331410ff70 0x7f33141a1d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.711+0000 7f3313400640 1 -- 172.21.15.27:0/3315453475 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33141a2490 con 0x7f3314076080 2024-09-21T00:24:32.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.712+0000 7f3313400640 1 --2- 172.21.15.27:0/3315453475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3314076080 0x7f331419cac0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f331406c580 tx=0x7f3300033ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:32.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.712+0000 7f3310c00640 1 -- 172.21.15.27:0/3315453475 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330003f020 con 0x7f3314076080 2024-09-21T00:24:32.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.713+0000 7f3310c00640 1 -- 172.21.15.27:0/3315453475 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3300005500 con 0x7f3314076080 2024-09-21T00:24:32.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.713+0000 7f3319a00640 1 -- 172.21.15.27:0/3315453475 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33141a2720 con 0x7f3314076080 2024-09-21T00:24:32.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.713+0000 7f3310c00640 1 -- 172.21.15.27:0/3315453475 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300037040 con 0x7f3314076080 2024-09-21T00:24:32.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.713+0000 7f3319a00640 1 -- 172.21.15.27:0/3315453475 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33141035c0 con 0x7f3314076080 2024-09-21T00:24:32.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.715+0000 7f3319a00640 1 -- 172.21.15.27:0/3315453475 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3314076c40 con 0x7f3314076080 2024-09-21T00:24:32.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.729+0000 7f3310c00640 1 -- 172.21.15.27:0/3315453475 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3300042020 con 0x7f3314076080 2024-09-21T00:24:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.745+0000 7f3310c00640 1 --2- 172.21.15.27:0/3315453475 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f32f80b6700 0x7f32f80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.746+0000 7f3310c00640 1 -- 172.21.15.27:0/3315453475 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f33001076b0 con 0x7f3314076080 2024-09-21T00:24:32.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.747+0000 7f3310c00640 1 -- 172.21.15.27:0/3315453475 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3300049020 con 0x7f3314076080 2024-09-21T00:24:32.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.748+0000 7f3313e00640 1 --2- 172.21.15.27:0/3315453475 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f32f80b6700 0x7f32f80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:32.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:32.751+0000 7f3313e00640 1 --2- 172.21.15.27:0/3315453475 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f32f80b6700 0x7f32f80b8c10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3304005fd0 tx=0x7f330400b4a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:33.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:33.260+0000 7f3319a00640 1 -- 172.21.15.27:0/3315453475 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3314075b40 con 0x7f3314076080 2024-09-21T00:24:33.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:33.261+0000 7f3310c00640 1 -- 172.21.15.27:0/3315453475 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11896 (secure 0 0 0) 0x7f33000cd870 con 0x7f3314076080 2024-09-21T00:24:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:24:33.268 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-09-21T00:24:33.269 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"btime":"2024-09-21T00:23:52:563301+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:23:52.207999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:24:33.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:33.268+0000 7f3319a00640 1 -- 172.21.15.27:0/3315453475 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f32f80b6700 msgr2=0x7f32f80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:33.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:33.268+0000 7f3319a00640 1 --2- 172.21.15.27:0/3315453475 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f32f80b6700 0x7f32f80b8c10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3304005fd0 tx=0x7f330400b4a0 comp rx=0 tx=0).stop 2024-09-21T00:24:33.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:33.268+0000 7f3319a00640 1 -- 172.21.15.27:0/3315453475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3314076080 msgr2=0x7f331419cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:33.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:33.268+0000 7f3319a00640 1 --2- 172.21.15.27:0/3315453475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3314076080 0x7f331419cac0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f331406c580 tx=0x7f3300033ce0 comp rx=0 tx=0).stop 2024-09-21T00:24:33.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:33.271+0000 7f3319a00640 1 -- 172.21.15.27:0/3315453475 shutdown_connections 2024-09-21T00:24:33.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:33.271+0000 7f3319a00640 1 --2- 172.21.15.27:0/3315453475 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f32f80b6700 0x7f32f80b8c10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:33.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:33.271+0000 7f3319a00640 1 --2- 172.21.15.27:0/3315453475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f331410ff70 0x7f33141a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:33.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:33.271+0000 7f3319a00640 1 --2- 172.21.15.27:0/3315453475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3314076080 0x7f331419cac0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:33.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:33.271+0000 7f3319a00640 1 --2- 172.21.15.27:0/3315453475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3314075730 0x7f331419c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:33.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:33.271+0000 7f3319a00640 1 -- 172.21.15.27:0/3315453475 >> 172.21.15.27:0/3315453475 conn(0x7f33140ff3e0 msgr2=0x7f33140ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:33.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:33.274+0000 7f3319a00640 1 -- 172.21.15.27:0/3315453475 shutdown_connections 2024-09-21T00:24:33.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:33.275+0000 7f3319a00640 1 -- 172.21.15.27:0/3315453475 wait complete. 2024-09-21T00:24:34.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:33 smithi027 ceph-mon[30745]: pgmap v472: 129 pgs: 129 active+clean; 133 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 791 KiB/s wr, 110 op/s 2024-09-21T00:24:34.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:33 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3315453475' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:24:34.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:33 smithi089 ceph-mon[34481]: pgmap v472: 129 pgs: 129 active+clean; 133 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 791 KiB/s wr, 110 op/s 2024-09-21T00:24:34.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:33 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3315453475' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:24:34.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:33 smithi098 ceph-mon[34725]: pgmap v472: 129 pgs: 129 active+clean; 133 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 791 KiB/s wr, 110 op/s 2024-09-21T00:24:34.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:33 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3315453475' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:24:34.259 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T00:24:34.259 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T00:24:34.841 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:24:35.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.645+0000 7fcd66200640 1 -- 172.21.15.27:0/2669643312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd6010c010 msgr2=0x7fcd60112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:35.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.645+0000 7fcd66200640 1 --2- 172.21.15.27:0/2669643312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd6010c010 0x7fcd60112840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009ca0 tx=0x7fcd5802a8b0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.646+0000 7fcd66200640 1 -- 172.21.15.27:0/2669643312 shutdown_connections 2024-09-21T00:24:35.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.646+0000 7fcd66200640 1 --2- 172.21.15.27:0/2669643312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd6010c010 0x7fcd60112840 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.646+0000 7fcd66200640 1 --2- 172.21.15.27:0/2669643312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd6010b6c0 0x7fcd6010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.646+0000 7fcd66200640 1 --2- 172.21.15.27:0/2669643312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd60105d80 0x7fcd60106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.646+0000 7fcd66200640 1 -- 172.21.15.27:0/2669643312 >> 172.21.15.27:0/2669643312 conn(0x7fcd601017d0 msgr2=0x7fcd60103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:35.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.646+0000 7fcd66200640 1 -- 172.21.15.27:0/2669643312 shutdown_connections 2024-09-21T00:24:35.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.647+0000 7fcd66200640 1 -- 172.21.15.27:0/2669643312 wait complete. 2024-09-21T00:24:35.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.648+0000 7fcd66200640 1 Processor -- start 2024-09-21T00:24:35.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.649+0000 7fcd66200640 1 -- start start 2024-09-21T00:24:35.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.649+0000 7fcd66200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60105d80 0x7fcd6019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.649+0000 7fcd66200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd6010b6c0 0x7fcd6019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.650+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60105d80 0x7fcd6019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.650+0000 7fcd66200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd6010c010 0x7fcd601a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.650+0000 7fcd66200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60115cd0 con 0x7fcd6010b6c0 2024-09-21T00:24:35.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.650+0000 7fcd66200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60115b50 con 0x7fcd60105d80 2024-09-21T00:24:35.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.650+0000 7fcd66200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60115e50 con 0x7fcd6010c010 2024-09-21T00:24:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.650+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60105d80 0x7fcd6019c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37524/0 (socket says 172.21.15.27:37524) 2024-09-21T00:24:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.650+0000 7fcd5f400640 1 -- 172.21.15.27:0/807332918 learned_addr learned my addr 172.21.15.27:0/807332918 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.650+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/807332918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd6010c010 0x7fcd601a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.650+0000 7fcd5ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd6010b6c0 0x7fcd6019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.651+0000 7fcd5f400640 1 -- 172.21.15.27:0/807332918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd6010c010 msgr2=0x7fcd601a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.651+0000 7fcd5f400640 1 --2- 172.21.15.27:0/807332918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd6010c010 0x7fcd601a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.651+0000 7fcd5f400640 1 -- 172.21.15.27:0/807332918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd6010b6c0 msgr2=0x7fcd6019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.651+0000 7fcd5f400640 1 --2- 172.21.15.27:0/807332918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd6010b6c0 0x7fcd6019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.651+0000 7fcd5f400640 1 -- 172.21.15.27:0/807332918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd601a25b0 con 0x7fcd60105d80 2024-09-21T00:24:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.651+0000 7fcd5f400640 1 --2- 172.21.15.27:0/807332918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60105d80 0x7fcd6019c640 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000e9f0 tx=0x7fcd5000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.652+0000 7fcd57e00640 1 -- 172.21.15.27:0/807332918 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5000cd80 con 0x7fcd60105d80 2024-09-21T00:24:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.652+0000 7fcd66200640 1 -- 172.21.15.27:0/807332918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd601a28a0 con 0x7fcd60105d80 2024-09-21T00:24:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.652+0000 7fcd66200640 1 -- 172.21.15.27:0/807332918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd601a9000 con 0x7fcd60105d80 2024-09-21T00:24:35.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.652+0000 7fcd57e00640 1 -- 172.21.15.27:0/807332918 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd500108e0 con 0x7fcd60105d80 2024-09-21T00:24:35.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.652+0000 7fcd57e00640 1 -- 172.21.15.27:0/807332918 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd50018860 con 0x7fcd60105d80 2024-09-21T00:24:35.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.657+0000 7fcd6cea4740 1 Processor -- start 2024-09-21T00:24:35.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.658+0000 7fcd6cea4740 1 -- start start 2024-09-21T00:24:35.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.658+0000 7fcd6cea4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e92840 0x5610e923a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.658+0000 7fcd57e00640 1 -- 172.21.15.27:0/807332918 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fcd50018a00 con 0x7fcd60105d80 2024-09-21T00:24:35.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.658+0000 7fcd6cea4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e69980 0x5610e923a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.658+0000 7fcd6cea4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e71b60 0x5610e923add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.659+0000 7fcd6cea4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5610e91afca0 con 0x5610e8e71b60 2024-09-21T00:24:35.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.659+0000 7fcd6cea4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5610e919f0d0 con 0x5610e8e69980 2024-09-21T00:24:35.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.659+0000 7fcd6cea4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5610e91a2ee0 con 0x5610e8e92840 2024-09-21T00:24:35.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.659+0000 7fcd5fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e92840 0x5610e923a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.659+0000 7fcd5fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e92840 0x5610e923a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:47038/0 (socket says 172.21.15.27:47038) 2024-09-21T00:24:35.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.659+0000 7fcd5fe00640 1 -- 172.21.15.27:0/1227014735 learned_addr learned my addr 172.21.15.27:0/1227014735 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:35.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.660+0000 7fcd5fe00640 1 -- 172.21.15.27:0/1227014735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e69980 msgr2=0x5610e923a890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:24:35.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.660+0000 7fcd5f400640 1 --2- 172.21.15.27:0/1227014735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e69980 0x5610e923a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.660+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/1227014735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e69980 0x5610e923a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.660+0000 7fcd5fe00640 1 -- 172.21.15.27:0/1227014735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e71b60 msgr2=0x5610e923add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:35.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.660+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/1227014735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e71b60 0x5610e923add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.660+0000 7fcd5fe00640 1 -- 172.21.15.27:0/1227014735 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e9151400 con 0x5610e8e92840 2024-09-21T00:24:35.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.661+0000 7fcd5f400640 1 --2- 172.21.15.27:0/1227014735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e69980 0x5610e923a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:24:35.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.661+0000 7fcd57e00640 1 --2- 172.21.15.27:0/807332918 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcd440b6700 0x7fcd440b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.661+0000 7fcd57e00640 1 -- 172.21.15.27:0/807332918 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fcd500e7bb0 con 0x7fcd60105d80 2024-09-21T00:24:35.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.661+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/1227014735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e92840 0x5610e923a350 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009800 tx=0x7fcd58009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be6566ed6cfe19a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:35.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.662+0000 7fcd5f400640 1 --2- 172.21.15.27:0/807332918 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcd440b6700 0x7fcd440b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.662+0000 7fcd4be00640 1 -- 172.21.15.27:0/1227014735 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd580305a0 con 0x5610e8e92840 2024-09-21T00:24:35.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.662+0000 7fcd4be00640 1 -- 172.21.15.27:0/1227014735 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd58036030 con 0x5610e8e92840 2024-09-21T00:24:35.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.663+0000 7fcd6cea4740 1 -- 172.21.15.27:0/1227014735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e92840 msgr2=0x5610e923a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:35.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.663+0000 7fcd5f400640 1 --2- 172.21.15.27:0/807332918 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcd440b6700 0x7fcd440b8c10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcd500e5020 tx=0x7fcd50000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:35.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.663+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/1227014735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e92840 0x5610e923a350 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009800 tx=0x7fcd58009830 comp rx=0 tx=0).stop 2024-09-21T00:24:35.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.662+0000 7fcd4be00640 1 -- 172.21.15.27:0/1227014735 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58004280 con 0x5610e8e92840 2024-09-21T00:24:35.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.664+0000 7fcd6cea4740 1 -- 172.21.15.27:0/1227014735 shutdown_connections 2024-09-21T00:24:35.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.664+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/1227014735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e71b60 0x5610e923add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.664+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/1227014735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e69980 0x5610e923a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.664+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/1227014735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e92840 0x5610e923a350 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.664+0000 7fcd6cea4740 1 -- 172.21.15.27:0/1227014735 >> 172.21.15.27:0/1227014735 conn(0x5610e8e86990 msgr2=0x5610e9238240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:35.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.664+0000 7fcd6cea4740 1 -- 172.21.15.27:0/1227014735 shutdown_connections 2024-09-21T00:24:35.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.664+0000 7fcd6cea4740 1 -- 172.21.15.27:0/1227014735 wait complete. 2024-09-21T00:24:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.665+0000 7fcd6cea4740 1 -- start start 2024-09-21T00:24:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.665+0000 7fcd6cea4740 1 Processor -- start 2024-09-21T00:24:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.666+0000 7fcd6cea4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e69980 0x5610e92458d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.666+0000 7fcd6cea4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e71b60 0x5610e9245e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.666+0000 7fcd6cea4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e92840 0x5610e9246350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.666+0000 7fcd6cea4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5610e90007a0 con 0x5610e8e71b60 2024-09-21T00:24:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.666+0000 7fcd6cea4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5610e9144a10 con 0x5610e8e92840 2024-09-21T00:24:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.666+0000 7fcd6cea4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5610e9248b20 con 0x5610e8e69980 2024-09-21T00:24:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.666+0000 7fcd6cea4740 10 client.0 _ll_register_callbacks cb 0x5610e9154be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:24:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.666+0000 7fcd5ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e71b60 0x5610e9245e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.666+0000 7fcd5ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e71b60 0x5610e9245e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55130/0 (socket says 172.21.15.27:55130) 2024-09-21T00:24:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.666+0000 7fcd5ea00640 1 -- 172.21.15.27:0/3518579676 learned_addr learned my addr 172.21.15.27:0/3518579676 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.666+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/3518579676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e69980 0x5610e92458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.667+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/3518579676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e92840 0x5610e9246350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.667+0000 7fcd5ea00640 1 -- 172.21.15.27:0/3518579676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e69980 msgr2=0x5610e92458d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.667+0000 7fcd5ea00640 1 --2- 172.21.15.27:0/3518579676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e69980 0x5610e92458d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.667+0000 7fcd5ea00640 1 -- 172.21.15.27:0/3518579676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e92840 msgr2=0x5610e9246350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.667+0000 7fcd5ea00640 1 --2- 172.21.15.27:0/3518579676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e92840 0x5610e9246350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.667+0000 7fcd5ea00640 1 -- 172.21.15.27:0/3518579676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e8f0cb90 con 0x5610e8e71b60 2024-09-21T00:24:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.667+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/3518579676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e92840 0x5610e9246350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:24:35.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.668+0000 7fcd5ea00640 1 --2- 172.21.15.27:0/3518579676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e71b60 0x5610e9245e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0116c0 tx=0x7fcd4c011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c033c3300bdb5184 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:35.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.668+0000 7fcd48c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:24:35.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.668+0000 7fcd48c00640 1 -- 172.21.15.27:0/3518579676 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c00fcb0 con 0x5610e8e71b60 2024-09-21T00:24:35.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.668+0000 7fcd6cea4740 1 -- 172.21.15.27:0/3518579676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd58036030 con 0x5610e8e71b60 2024-09-21T00:24:35.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.669+0000 7fcd48c00640 1 -- 172.21.15.27:0/3518579676 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fcd4c00fab0 con 0x5610e8e71b60 2024-09-21T00:24:35.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.669+0000 7fcd48c00640 1 -- 172.21.15.27:0/3518579676 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd4c00fcb0 con 0x5610e8e71b60 2024-09-21T00:24:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.669+0000 7fcd6cea4740 10 client.15837 fetch_fsmap learned FSMap version 28 2024-09-21T00:24:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.669+0000 7fcd6cea4740 1 -- 172.21.15.27:0/3518579676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610e92547c0 con 0x5610e8e71b60 2024-09-21T00:24:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.670+0000 7fcd48c00640 1 -- 172.21.15.27:0/3518579676 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd58036030 con 0x5610e8e71b60 2024-09-21T00:24:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.670+0000 7fcd48c00640 1 -- 172.21.15.27:0/3518579676 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcd4c0223f0 con 0x5610e8e71b60 2024-09-21T00:24:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.670+0000 7fcd6cea4740 10 client.15837 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:24:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.670+0000 7fcd6cea4740 10 client.15837 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:24:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.670+0000 7fcd6cea4740 20 client.15837 populate_metadata read hostname 'smithi027' 2024-09-21T00:24:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.670+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/3518579676 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5610e8e511c0 0x5610e9257d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.670+0000 7fcd6cea4740 4 client.15837 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:24:35.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.670+0000 7fcd6cea4740 1 -- 172.21.15.27:0/3518579676 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610e91b1bf0 con 0x5610e8e511c0 2024-09-21T00:24:35.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.671+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/3518579676 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5610e8e511c0 0x5610e9257d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.671+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/3518579676 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5610e8e511c0 0x5610e9257d80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7182f28bd1f97d26 server_cookie=4943fc6cf7d41042 in_seq=0 out_seq=0 2024-09-21T00:24:35.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.671+0000 7fcd48c00640 10 client.15837 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:35.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.673+0000 7fcd48c00640 1 -- 172.21.15.27:0/3518579676 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5610e91b1bf0 con 0x5610e8e511c0 2024-09-21T00:24:35.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.673+0000 7fcd48c00640 10 client.15837 handle_command_reply: tid=0 2024-09-21T00:24:35.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.673+0000 7fcd6cea4740 1 client.15837 shutdown 2024-09-21T00:24:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.674+0000 7fcd6cea4740 1 -- 172.21.15.27:0/3518579676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e71b60 msgr2=0x5610e9245e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.674+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/3518579676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e71b60 0x5610e9245e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0116c0 tx=0x7fcd4c011b80 comp rx=0 tx=0).stop 2024-09-21T00:24:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.674+0000 7fcd6cea4740 1 -- 172.21.15.27:0/3518579676 shutdown_connections 2024-09-21T00:24:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.674+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/3518579676 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5610e8e511c0 0x5610e9257d80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.674+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/3518579676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e92840 0x5610e9246350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.674+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/3518579676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e71b60 0x5610e9245e10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.674+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/3518579676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e69980 0x5610e92458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.674+0000 7fcd48c00640 0 client.15837 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:35.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.674+0000 7fcd6cea4740 1 -- 172.21.15.27:0/3518579676 >> 172.21.15.27:0/3518579676 conn(0x5610e8e86990 msgr2=0x5610e92369a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:35.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.674+0000 7fcd6cea4740 1 -- 172.21.15.27:0/3518579676 shutdown_connections 2024-09-21T00:24:35.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.674+0000 7fcd6cea4740 1 -- 172.21.15.27:0/3518579676 wait complete. 2024-09-21T00:24:35.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.674+0000 7fcd6cea4740 20 client.15837 trim_cache size 0 max 16384 2024-09-21T00:24:35.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.712+0000 7fcd6cea4740 1 Processor -- start 2024-09-21T00:24:35.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.712+0000 7fcd6cea4740 1 -- start start 2024-09-21T00:24:35.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.712+0000 7fcd6cea4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e69980 0x5610e923ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.712+0000 7fcd6cea4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e71b60 0x5610e9242ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.712+0000 7fcd6cea4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e92840 0x5610e9243000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.712+0000 7fcd6cea4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd400128f0 con 0x5610e8e69980 2024-09-21T00:24:35.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.712+0000 7fcd6cea4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcd400101c0 con 0x5610e8e92840 2024-09-21T00:24:35.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.712+0000 7fcd6cea4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4000e010 con 0x5610e8e71b60 2024-09-21T00:24:35.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.713+0000 7fcd5ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e71b60 0x5610e9242ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.713+0000 7fcd5fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e69980 0x5610e923ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.713+0000 7fcd5ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e71b60 0x5610e9242ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:47058/0 (socket says 172.21.15.27:47058) 2024-09-21T00:24:35.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.713+0000 7fcd5ea00640 1 -- 172.21.15.27:0/2144957663 learned_addr learned my addr 172.21.15.27:0/2144957663 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:35.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.713+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/2144957663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e92840 0x5610e9243000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.713+0000 7fcd5fe00640 1 -- 172.21.15.27:0/2144957663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e71b60 msgr2=0x5610e9242ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:35.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.713+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/2144957663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e71b60 0x5610e9242ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.713+0000 7fcd5fe00640 1 -- 172.21.15.27:0/2144957663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e92840 msgr2=0x5610e9243000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:35.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.713+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/2144957663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e92840 0x5610e9243000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.713+0000 7fcd5fe00640 1 -- 172.21.15.27:0/2144957663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e92467f0 con 0x5610e8e69980 2024-09-21T00:24:35.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.714+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/2144957663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e69980 0x5610e923ba90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcd58005e30 tx=0x7fcd58030fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92cc840d16d9edd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:35.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.714+0000 7fcd4b400640 1 -- 172.21.15.27:0/2144957663 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58030800 con 0x5610e8e69980 2024-09-21T00:24:35.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.714+0000 7fcd4b400640 1 -- 172.21.15.27:0/2144957663 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd58036030 con 0x5610e8e69980 2024-09-21T00:24:35.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.714+0000 7fcd4b400640 1 -- 172.21.15.27:0/2144957663 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58004670 con 0x5610e8e69980 2024-09-21T00:24:35.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.715+0000 7fcd6cea4740 1 -- 172.21.15.27:0/2144957663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e69980 msgr2=0x5610e923ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:35.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.715+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/2144957663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e69980 0x5610e923ba90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcd58005e30 tx=0x7fcd58030fe0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.715+0000 7fcd6cea4740 1 -- 172.21.15.27:0/2144957663 shutdown_connections 2024-09-21T00:24:35.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.715+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/2144957663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e92840 0x5610e9243000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.715+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/2144957663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e71b60 0x5610e9242ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.715+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/2144957663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e69980 0x5610e923ba90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.715+0000 7fcd6cea4740 1 -- 172.21.15.27:0/2144957663 >> 172.21.15.27:0/2144957663 conn(0x5610e8e86990 msgr2=0x5610e923b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:35.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.716+0000 7fcd6cea4740 1 -- 172.21.15.27:0/2144957663 shutdown_connections 2024-09-21T00:24:35.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.716+0000 7fcd6cea4740 1 -- 172.21.15.27:0/2144957663 wait complete. 2024-09-21T00:24:35.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.716+0000 7fcd6cea4740 1 -- start start 2024-09-21T00:24:35.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.717+0000 7fcd6cea4740 1 Processor -- start 2024-09-21T00:24:35.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.717+0000 7fcd6cea4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e69980 0x5610e9249d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.717+0000 7fcd6cea4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e71b60 0x5610e924a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.718+0000 7fcd6cea4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e92840 0x5610e924a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.718+0000 7fcd5fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e69980 0x5610e9249d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.718+0000 7fcd6cea4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4000e680 con 0x5610e8e92840 2024-09-21T00:24:35.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.718+0000 7fcd6cea4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4000faf0 con 0x5610e8e71b60 2024-09-21T00:24:35.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.718+0000 7fcd6cea4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4000ed50 con 0x5610e8e69980 2024-09-21T00:24:35.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.718+0000 7fcd5fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e69980 0x5610e9249d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:47062/0 (socket says 172.21.15.27:47062) 2024-09-21T00:24:35.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.718+0000 7fcd5fe00640 1 -- 172.21.15.27:0/1051767232 learned_addr learned my addr 172.21.15.27:0/1051767232 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:35.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.718+0000 7fcd6cea4740 10 client.0 _ll_register_callbacks cb 0x7fcd4c009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:24:35.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.718+0000 7fcd5ea00640 1 --2- 172.21.15.27:0/1051767232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e71b60 0x5610e924a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.718+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/1051767232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e92840 0x5610e924a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.718+0000 7fcd5ea00640 1 -- 172.21.15.27:0/1051767232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e69980 msgr2=0x5610e9249d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:35.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.719+0000 7fcd5ea00640 1 --2- 172.21.15.27:0/1051767232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e69980 0x5610e9249d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.719+0000 7fcd5ea00640 1 -- 172.21.15.27:0/1051767232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e92840 msgr2=0x5610e924a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:35.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.719+0000 7fcd5ea00640 1 --2- 172.21.15.27:0/1051767232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e92840 0x5610e924a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:35.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.719+0000 7fcd5ea00640 1 -- 172.21.15.27:0/1051767232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e92c1760 con 0x5610e8e71b60 2024-09-21T00:24:35.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.719+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/1051767232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e92840 0x5610e924a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:24:35.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.719+0000 7fcd5ea00640 1 --2- 172.21.15.27:0/1051767232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e71b60 0x5610e924a260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c000c00 tx=0x7fcd4c022d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe71e395d467b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:35.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.719+0000 7fcd48c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:24:35.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.720+0000 7fcd48c00640 1 -- 172.21.15.27:0/1051767232 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c00fe50 con 0x5610e8e71b60 2024-09-21T00:24:35.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.720+0000 7fcd48c00640 1 -- 172.21.15.27:0/1051767232 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fcd4c00fab0 con 0x5610e8e71b60 2024-09-21T00:24:35.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.720+0000 7fcd6cea4740 1 -- 172.21.15.27:0/1051767232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd58036030 con 0x5610e8e71b60 2024-09-21T00:24:35.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.720+0000 7fcd48c00640 1 -- 172.21.15.27:0/1051767232 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd4c005250 con 0x5610e8e71b60 2024-09-21T00:24:35.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.720+0000 7fcd6cea4740 10 client.25883 fetch_fsmap learned FSMap version 28 2024-09-21T00:24:35.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.720+0000 7fcd6cea4740 1 -- 172.21.15.27:0/1051767232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610e92c28e0 con 0x5610e8e71b60 2024-09-21T00:24:35.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.720+0000 7fcd48c00640 1 -- 172.21.15.27:0/1051767232 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd58036030 con 0x5610e8e71b60 2024-09-21T00:24:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.722+0000 7fcd48c00640 1 -- 172.21.15.27:0/1051767232 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcd4c003c20 con 0x5610e8e71b60 2024-09-21T00:24:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.722+0000 7fcd6cea4740 10 client.25883 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:24:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.722+0000 7fcd6cea4740 10 client.25883 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:24:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.722+0000 7fcd6cea4740 20 client.25883 populate_metadata read hostname 'smithi027' 2024-09-21T00:24:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.722+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/1051767232 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5610e8e26570 0x5610e92c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.722+0000 7fcd6cea4740 4 client.25883 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T00:24:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.722+0000 7fcd6cea4740 1 -- 172.21.15.27:0/1051767232 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x5610e91ad6a0 con 0x5610e8e26570 2024-09-21T00:24:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.722+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/1051767232 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5610e8e26570 0x5610e92c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.723+0000 7fcd5fe00640 1 --2- 172.21.15.27:0/1051767232 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5610e8e26570 0x5610e92c51c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=622ca59239a33f26 server_cookie=92f0ad88a2616539 in_seq=0 out_seq=0 2024-09-21T00:24:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:35.723+0000 7fcd48c00640 10 client.25883 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:36.047 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:35 smithi098 ceph-mon[34725]: pgmap v473: 129 pgs: 129 active+clean; 138 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 866 KiB/s wr, 101 op/s 2024-09-21T00:24:36.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:35 smithi027 ceph-mon[30745]: pgmap v473: 129 pgs: 129 active+clean; 138 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 866 KiB/s wr, 101 op/s 2024-09-21T00:24:36.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:35 smithi089 ceph-mon[34481]: pgmap v473: 129 pgs: 129 active+clean; 138 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 866 KiB/s wr, 101 op/s 2024-09-21T00:24:37.859 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:37 smithi027 ceph-mon[30745]: pgmap v474: 129 pgs: 129 active+clean; 138 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 846 KiB/s wr, 92 op/s 2024-09-21T00:24:38.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:37 smithi089 ceph-mon[34481]: pgmap v474: 129 pgs: 129 active+clean; 138 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 846 KiB/s wr, 92 op/s 2024-09-21T00:24:38.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:37 smithi098 ceph-mon[34725]: pgmap v474: 129 pgs: 129 active+clean; 138 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 846 KiB/s wr, 92 op/s 2024-09-21T00:24:38.298 INFO:tasks.workunit.client.0.smithi027.stdout: 41 173778 551 133469 287 62766 409 2024-09-21T00:24:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 -- 172.21.15.27:0/1406223469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ca8076100 msgr2=0x7f4ca810f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 --2- 172.21.15.27:0/1406223469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ca8076100 0x7f4ca810f920 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009860 tx=0x7f4c9402a340 comp rx=0 tx=0).stop 2024-09-21T00:24:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 -- 172.21.15.27:0/1406223469 shutdown_connections 2024-09-21T00:24:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 --2- 172.21.15.27:0/1406223469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ca810fe60 0x7f4ca81122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 --2- 172.21.15.27:0/1406223469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ca8076100 0x7f4ca810f920 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 --2- 172.21.15.27:0/1406223469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ca80757b0 0x7f4ca8075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 -- 172.21.15.27:0/1406223469 >> 172.21.15.27:0/1406223469 conn(0x7f4ca80ff390 msgr2=0x7f4ca81017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 -- 172.21.15.27:0/1406223469 shutdown_connections 2024-09-21T00:24:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 -- 172.21.15.27:0/1406223469 wait complete. 2024-09-21T00:24:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 Processor -- start 2024-09-21T00:24:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 -- start start 2024-09-21T00:24:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ca80757b0 0x7f4ca819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ca8076100 0x7f4ca819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ca810fe60 0x7f4ca81a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca8115a30 con 0x7f4ca80757b0 2024-09-21T00:24:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81158b0 con 0x7f4ca810fe60 2024-09-21T00:24:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4caee00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca8115bb0 con 0x7f4ca8076100 2024-09-21T00:24:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4cada00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ca80757b0 0x7f4ca819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4cada00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ca80757b0 0x7f4ca819c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55166/0 (socket says 172.21.15.27:55166) 2024-09-21T00:24:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4cada00640 1 -- 172.21.15.27:0/3851485742 learned_addr learned my addr 172.21.15.27:0/3851485742 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4cad000640 1 --2- 172.21.15.27:0/3851485742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ca8076100 0x7f4ca819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.661+0000 7f4cae400640 1 --2- 172.21.15.27:0/3851485742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ca810fe60 0x7f4ca81a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.667+0000 7f4cae400640 1 -- 172.21.15.27:0/3851485742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ca8076100 msgr2=0x7f4ca819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.667+0000 7f4cae400640 1 --2- 172.21.15.27:0/3851485742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ca8076100 0x7f4ca819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.667+0000 7f4cae400640 1 -- 172.21.15.27:0/3851485742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ca80757b0 msgr2=0x7f4ca819c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.667+0000 7f4cae400640 1 --2- 172.21.15.27:0/3851485742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ca80757b0 0x7f4ca819c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.667+0000 7f4cae400640 1 -- 172.21.15.27:0/3851485742 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca81a23c0 con 0x7f4ca810fe60 2024-09-21T00:24:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.667+0000 7f4cada00640 1 --2- 172.21.15.27:0/3851485742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ca80757b0 0x7f4ca819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:24:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.667+0000 7f4cae400640 1 --2- 172.21.15.27:0/3851485742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ca810fe60 0x7f4ca81a1cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009fd0 tx=0x7f4ca000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.667+0000 7f4ca6000640 1 -- 172.21.15.27:0/3851485742 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca000ce20 con 0x7f4ca810fe60 2024-09-21T00:24:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.667+0000 7f4ca6000640 1 -- 172.21.15.27:0/3851485742 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca0005020 con 0x7f4ca810fe60 2024-09-21T00:24:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.667+0000 7f4ca6000640 1 -- 172.21.15.27:0/3851485742 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0010690 con 0x7f4ca810fe60 2024-09-21T00:24:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.667+0000 7f4caee00640 1 -- 172.21.15.27:0/3851485742 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca81a26b0 con 0x7f4ca810fe60 2024-09-21T00:24:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.667+0000 7f4caee00640 1 -- 172.21.15.27:0/3851485742 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca81a8e40 con 0x7f4ca810fe60 2024-09-21T00:24:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cb748c740 1 Processor -- start 2024-09-21T00:24:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4ca6000640 1 -- 172.21.15.27:0/3851485742 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4ca00054d0 con 0x7f4ca810fe60 2024-09-21T00:24:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cb748c740 1 -- start start 2024-09-21T00:24:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cb748c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee441b0 0x556bbf1e8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cb748c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee1b2f0 0x556bbf1e9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cb748c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee234d0 0x556bbf1e9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cb748c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556bbf15e6b0 con 0x556bbee234d0 2024-09-21T00:24:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cb748c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556bbf14dae0 con 0x556bbee1b2f0 2024-09-21T00:24:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cb748c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556bbf1518f0 con 0x556bbee441b0 2024-09-21T00:24:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cada00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee1b2f0 0x556bbf1e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cada00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee1b2f0 0x556bbf1e9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37586/0 (socket says 172.21.15.27:37586) 2024-09-21T00:24:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cada00640 1 -- 172.21.15.27:0/3846052848 learned_addr learned my addr 172.21.15.27:0/3846052848 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cad000640 1 --2- 172.21.15.27:0/3846052848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee441b0 0x556bbf1e8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cada00640 1 -- 172.21.15.27:0/3846052848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee441b0 msgr2=0x556bbf1e8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cada00640 1 --2- 172.21.15.27:0/3846052848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee441b0 0x556bbf1e8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cad000640 1 --2- 172.21.15.27:0/3846052848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee234d0 0x556bbf1e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cada00640 1 -- 172.21.15.27:0/3846052848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee234d0 msgr2=0x556bbf1e9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cada00640 1 --2- 172.21.15.27:0/3846052848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee234d0 0x556bbf1e9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cada00640 1 -- 172.21.15.27:0/3846052848 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bbf11a780 con 0x556bbee1b2f0 2024-09-21T00:24:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cad000640 1 --2- 172.21.15.27:0/3846052848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee234d0 0x556bbf1e9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:24:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cada00640 1 --2- 172.21.15.27:0/3846052848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee1b2f0 0x556bbf1e9350 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0115c0 tx=0x7f4c9c011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7df5af7fa1d50473 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4ca6000640 1 --2- 172.21.15.27:0/3851485742 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4c900b6670 0x7f4c900b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4c9aa00640 1 -- 172.21.15.27:0/3846052848 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c00f6c0 con 0x556bbee1b2f0 2024-09-21T00:24:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4c9aa00640 1 -- 172.21.15.27:0/3846052848 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c9c00fd10 con 0x556bbee1b2f0 2024-09-21T00:24:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4c9aa00640 1 -- 172.21.15.27:0/3846052848 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c019e50 con 0x556bbee1b2f0 2024-09-21T00:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4ca6000640 1 -- 172.21.15.27:0/3851485742 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f4ca00e8190 con 0x7f4ca810fe60 2024-09-21T00:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.670+0000 7f4cad000640 1 --2- 172.21.15.27:0/3851485742 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4c900b6670 0x7f4c900b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 -- 172.21.15.27:0/3846052848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee1b2f0 msgr2=0x556bbf1e9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 --2- 172.21.15.27:0/3846052848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee1b2f0 0x556bbf1e9350 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0115c0 tx=0x7f4c9c011a80 comp rx=0 tx=0).stop 2024-09-21T00:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cad000640 1 --2- 172.21.15.27:0/3851485742 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4c900b6670 0x7f4c900b8b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4c94035040 tx=0x7f4c94030340 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 -- 172.21.15.27:0/3846052848 shutdown_connections 2024-09-21T00:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 --2- 172.21.15.27:0/3846052848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee234d0 0x556bbf1e9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 --2- 172.21.15.27:0/3846052848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee1b2f0 0x556bbf1e9350 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 --2- 172.21.15.27:0/3846052848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee441b0 0x556bbf1e8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 -- 172.21.15.27:0/3846052848 >> 172.21.15.27:0/3846052848 conn(0x556bbee38300 msgr2=0x556bbf1e6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 -- 172.21.15.27:0/3846052848 shutdown_connections 2024-09-21T00:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 -- 172.21.15.27:0/3846052848 wait complete. 2024-09-21T00:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 -- start start 2024-09-21T00:24:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 Processor -- start 2024-09-21T00:24:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 0x556bbf1fcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee234d0 0x556bbf1fd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee441b0 0x556bbf1fd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556bbf085ab0 con 0x556bbee1b2f0 2024-09-21T00:24:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556bbf0f4570 con 0x556bbee441b0 2024-09-21T00:24:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556bbf1ffd50 con 0x556bbee234d0 2024-09-21T00:24:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 10 client.0 _ll_register_callbacks cb 0x556bbf103100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:24:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cae400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee441b0 0x556bbf1fd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cae400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee441b0 0x556bbf1fd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37592/0 (socket says 172.21.15.27:37592) 2024-09-21T00:24:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cae400640 1 -- 172.21.15.27:0/2368519639 learned_addr learned my addr 172.21.15.27:0/2368519639 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cada00640 1 --2- 172.21.15.27:0/2368519639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 0x556bbf1fcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cad000640 1 --2- 172.21.15.27:0/2368519639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee234d0 0x556bbf1fd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cae400640 1 -- 172.21.15.27:0/2368519639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee234d0 msgr2=0x556bbf1fd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cae400640 1 --2- 172.21.15.27:0/2368519639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee234d0 0x556bbf1fd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cae400640 1 -- 172.21.15.27:0/2368519639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 msgr2=0x556bbf1fcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cae400640 1 --2- 172.21.15.27:0/2368519639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 0x556bbf1fcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cae400640 1 -- 172.21.15.27:0/2368519639 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bbeeaf790 con 0x556bbee441b0 2024-09-21T00:24:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cada00640 1 --2- 172.21.15.27:0/2368519639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 0x556bbf1fcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:24:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cae400640 1 --2- 172.21.15.27:0/2368519639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee441b0 0x556bbf1fd6a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00adbe0 tx=0x7f4ca00a93c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dccb55065f4c5a59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4c8f400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:24:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4c8f400640 1 -- 172.21.15.27:0/2368519639 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca00a9af0 con 0x556bbee441b0 2024-09-21T00:24:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4c8f400640 1 -- 172.21.15.27:0/2368519639 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f4ca00aceb0 con 0x556bbee441b0 2024-09-21T00:24:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 -- 172.21.15.27:0/2368519639 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c9c00fd10 con 0x556bbee441b0 2024-09-21T00:24:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4c8f400640 1 -- 172.21.15.27:0/2368519639 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4ca00d1310 con 0x556bbee441b0 2024-09-21T00:24:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 10 client.25895 fetch_fsmap learned FSMap version 28 2024-09-21T00:24:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 -- 172.21.15.27:0/2368519639 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556bbef28620 con 0x556bbee441b0 2024-09-21T00:24:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4c8f400640 1 -- 172.21.15.27:0/2368519639 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c9c00fd10 con 0x556bbee441b0 2024-09-21T00:24:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4c8f400640 1 -- 172.21.15.27:0/2368519639 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4ca00a95a0 con 0x556bbee441b0 2024-09-21T00:24:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 10 client.25895 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:24:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 10 client.25895 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:24:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 20 client.25895 populate_metadata read hostname 'smithi027' 2024-09-21T00:24:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 --2- 172.21.15.27:0/2368519639 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x556bbee02d80 0x556bbf20ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 4 client.25895 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:24:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.675+0000 7f4cb748c740 1 -- 172.21.15.27:0/2368519639 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bbf160600 con 0x556bbee02d80 2024-09-21T00:24:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.680+0000 7f4cada00640 1 --2- 172.21.15.27:0/2368519639 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x556bbee02d80 0x556bbf20ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.680+0000 7f4cada00640 1 --2- 172.21.15.27:0/2368519639 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x556bbee02d80 0x556bbf20ee00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca3f692ae95f24e server_cookie=d0a305ffbb34e0da in_seq=0 out_seq=0 2024-09-21T00:24:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.680+0000 7f4c8f400640 10 client.25895 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.682+0000 7f4c8f400640 1 -- 172.21.15.27:0/2368519639 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556bbf160600 con 0x556bbee02d80 2024-09-21T00:24:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.682+0000 7f4c8f400640 10 client.25895 handle_command_reply: tid=0 2024-09-21T00:24:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.682+0000 7f4cb748c740 1 client.25895 shutdown 2024-09-21T00:24:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.682+0000 7f4cb748c740 1 -- 172.21.15.27:0/2368519639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee441b0 msgr2=0x556bbf1fd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.682+0000 7f4cb748c740 1 --2- 172.21.15.27:0/2368519639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee441b0 0x556bbf1fd6a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00adbe0 tx=0x7f4ca00a93c0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.682+0000 7f4cb748c740 1 -- 172.21.15.27:0/2368519639 shutdown_connections 2024-09-21T00:24:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.682+0000 7f4cb748c740 1 --2- 172.21.15.27:0/2368519639 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x556bbee02d80 0x556bbf20ee00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.682+0000 7f4cb748c740 1 --2- 172.21.15.27:0/2368519639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee441b0 0x556bbf1fd6a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.682+0000 7f4cb748c740 1 --2- 172.21.15.27:0/2368519639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee234d0 0x556bbf1fd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.682+0000 7f4cb748c740 1 --2- 172.21.15.27:0/2368519639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 0x556bbf1fcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.682+0000 7f4cb748c740 1 -- 172.21.15.27:0/2368519639 >> 172.21.15.27:0/2368519639 conn(0x556bbee38300 msgr2=0x556bbf1e5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.682+0000 7f4c8f400640 0 client.25895 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.682+0000 7f4cb748c740 1 -- 172.21.15.27:0/2368519639 shutdown_connections 2024-09-21T00:24:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.682+0000 7f4cb748c740 1 -- 172.21.15.27:0/2368519639 wait complete. 2024-09-21T00:24:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.682+0000 7f4cb748c740 20 client.25895 trim_cache size 0 max 16384 2024-09-21T00:24:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 Processor -- start 2024-09-21T00:24:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 -- start start 2024-09-21T00:24:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 0x556bbf1e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee234d0 0x556bbf1f8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee441b0 0x556bbf1fd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81b1330 con 0x556bbee1b2f0 2024-09-21T00:24:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81aec00 con 0x556bbee234d0 2024-09-21T00:24:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81acab0 con 0x556bbee441b0 2024-09-21T00:24:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cad000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee234d0 0x556bbf1f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cada00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 0x556bbf1e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cad000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee234d0 0x556bbf1f8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37596/0 (socket says 172.21.15.27:37596) 2024-09-21T00:24:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cad000640 1 -- 172.21.15.27:0/3522303206 learned_addr learned my addr 172.21.15.27:0/3522303206 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cae400640 1 --2- 172.21.15.27:0/3522303206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee441b0 0x556bbf1fd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cae400640 1 -- 172.21.15.27:0/3522303206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee234d0 msgr2=0x556bbf1f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cae400640 1 --2- 172.21.15.27:0/3522303206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee234d0 0x556bbf1f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cae400640 1 -- 172.21.15.27:0/3522303206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 msgr2=0x556bbf1e69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cae400640 1 --2- 172.21.15.27:0/3522303206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 0x556bbf1e69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cae400640 1 -- 172.21.15.27:0/3522303206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bbf1ebc50 con 0x556bbee441b0 2024-09-21T00:24:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cada00640 1 --2- 172.21.15.27:0/3522303206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 0x556bbf1e69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:24:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cae400640 1 --2- 172.21.15.27:0/3522303206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee441b0 0x556bbf1fd100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00aaec0 tx=0x7f4ca00a94f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15e3898143235c0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4c9a000640 1 -- 172.21.15.27:0/3522303206 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca00ad760 con 0x556bbee441b0 2024-09-21T00:24:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4c9a000640 1 -- 172.21.15.27:0/3522303206 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca00b2730 con 0x556bbee441b0 2024-09-21T00:24:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4c9a000640 1 -- 172.21.15.27:0/3522303206 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca00b21b0 con 0x556bbee441b0 2024-09-21T00:24:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 -- 172.21.15.27:0/3522303206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee441b0 msgr2=0x556bbf1fd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 --2- 172.21.15.27:0/3522303206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee441b0 0x556bbf1fd100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00aaec0 tx=0x7f4ca00a94f0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 -- 172.21.15.27:0/3522303206 shutdown_connections 2024-09-21T00:24:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 --2- 172.21.15.27:0/3522303206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee441b0 0x556bbf1fd100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 --2- 172.21.15.27:0/3522303206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee234d0 0x556bbf1f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 --2- 172.21.15.27:0/3522303206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 0x556bbf1e69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 -- 172.21.15.27:0/3522303206 >> 172.21.15.27:0/3522303206 conn(0x556bbee38300 msgr2=0x556bbf1f7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 -- 172.21.15.27:0/3522303206 shutdown_connections 2024-09-21T00:24:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 -- 172.21.15.27:0/3522303206 wait complete. 2024-09-21T00:24:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 -- start start 2024-09-21T00:24:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 Processor -- start 2024-09-21T00:24:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 0x556bbf205610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee234d0 0x556bbf200ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee441b0 0x556bbf200ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81ad100 con 0x556bbee1b2f0 2024-09-21T00:24:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81ae530 con 0x556bbee441b0 2024-09-21T00:24:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81ad790 con 0x556bbee234d0 2024-09-21T00:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cb748c740 10 client.0 _ll_register_callbacks cb 0x7f4ca00ad610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cada00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 0x556bbf205610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cad000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee234d0 0x556bbf200ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cad000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee234d0 0x556bbf200ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:47102/0 (socket says 172.21.15.27:47102) 2024-09-21T00:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cae400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee441b0 0x556bbf200ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cad000640 1 -- 172.21.15.27:0/3326433360 learned_addr learned my addr 172.21.15.27:0/3326433360 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cae400640 1 -- 172.21.15.27:0/3326433360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee234d0 msgr2=0x556bbf200ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cae400640 1 --2- 172.21.15.27:0/3326433360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee234d0 0x556bbf200ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cae400640 1 -- 172.21.15.27:0/3326433360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 msgr2=0x556bbf205610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cae400640 1 --2- 172.21.15.27:0/3326433360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 0x556bbf205610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cae400640 1 -- 172.21.15.27:0/3326433360 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bbf20f390 con 0x556bbee441b0 2024-09-21T00:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.700+0000 7f4cad000640 1 --2- 172.21.15.27:0/3326433360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee234d0 0x556bbf200ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4cae400640 1 --2- 172.21.15.27:0/3326433360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee441b0 0x556bbf200ff0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00a99a0 tx=0x7f4ca00aa2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d53fa33b34fb13d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4c8f400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:24:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4c8f400640 1 -- 172.21.15.27:0/3326433360 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca00ad760 con 0x556bbee441b0 2024-09-21T00:24:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4cb748c740 1 -- 172.21.15.27:0/3326433360 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ca00b2730 con 0x556bbee441b0 2024-09-21T00:24:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4c8f400640 1 -- 172.21.15.27:0/3326433360 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f4ca00aceb0 con 0x556bbee441b0 2024-09-21T00:24:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4c8f400640 1 -- 172.21.15.27:0/3326433360 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4ca00b7670 con 0x556bbee441b0 2024-09-21T00:24:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4cb748c740 10 client.25901 fetch_fsmap learned FSMap version 28 2024-09-21T00:24:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4cb748c740 1 -- 172.21.15.27:0/3326433360 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556bbf279390 con 0x556bbee441b0 2024-09-21T00:24:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4c8f400640 1 -- 172.21.15.27:0/3326433360 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca00b2730 con 0x556bbee441b0 2024-09-21T00:24:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4c8f400640 1 -- 172.21.15.27:0/3326433360 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4ca00b38c0 con 0x556bbee441b0 2024-09-21T00:24:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4cb748c740 10 client.25901 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:24:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4cb748c740 10 client.25901 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:24:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4cb748c740 20 client.25901 populate_metadata read hostname 'smithi027' 2024-09-21T00:24:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4cb748c740 1 --2- 172.21.15.27:0/3326433360 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x556bbedd81c0 0x556bbf27b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4cb748c740 4 client.25901 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:24:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4cb748c740 1 -- 172.21.15.27:0/3326433360 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556bbf15c0b0 con 0x556bbedd81c0 2024-09-21T00:24:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4cada00640 1 --2- 172.21.15.27:0/3326433360 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x556bbedd81c0 0x556bbf27b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4c8f400640 10 client.25901 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:38.705+0000 7f4cada00640 1 --2- 172.21.15.27:0/3326433360 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x556bbedd81c0 0x556bbf27b820 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d158cc2e742a382b server_cookie=3d184722956c3b16 in_seq=0 out_seq=0 2024-09-21T00:24:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.182+0000 7f4c8f400640 1 -- 172.21.15.27:0/3326433360 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556bbf15c0b0 con 0x556bbedd81c0 2024-09-21T00:24:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.182+0000 7f4c8f400640 10 client.25901 handle_command_reply: tid=0 2024-09-21T00:24:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.182+0000 7f4cb748c740 1 client.25901 shutdown 2024-09-21T00:24:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.182+0000 7f4cb748c740 1 -- 172.21.15.27:0/3326433360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee441b0 msgr2=0x556bbf200ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.182+0000 7f4cb748c740 1 --2- 172.21.15.27:0/3326433360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee441b0 0x556bbf200ff0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00a99a0 tx=0x7f4ca00aa2a0 comp rx=0 tx=0).stop 2024-09-21T00:24:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.182+0000 7f4cb748c740 1 -- 172.21.15.27:0/3326433360 shutdown_connections 2024-09-21T00:24:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.182+0000 7f4cb748c740 1 --2- 172.21.15.27:0/3326433360 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x556bbedd81c0 0x556bbf27b820 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.182+0000 7f4cb748c740 1 --2- 172.21.15.27:0/3326433360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bbee441b0 0x556bbf200ff0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.182+0000 7f4cb748c740 1 --2- 172.21.15.27:0/3326433360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556bbee234d0 0x556bbf200ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.182+0000 7f4cb748c740 1 --2- 172.21.15.27:0/3326433360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556bbee1b2f0 0x556bbf205610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.182+0000 7f4cb748c740 1 -- 172.21.15.27:0/3326433360 >> 172.21.15.27:0/3326433360 conn(0x556bbee38300 msgr2=0x556bbf1e6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.182+0000 7f4c8f400640 0 client.25901 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.182+0000 7f4cb748c740 1 -- 172.21.15.27:0/3326433360 shutdown_connections 2024-09-21T00:24:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.182+0000 7f4cb748c740 1 -- 172.21.15.27:0/3326433360 wait complete. 2024-09-21T00:24:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.182+0000 7f4cb748c740 20 client.25901 trim_cache size 0 max 16384 2024-09-21T00:24:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:24:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:24:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:24:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:24:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.184+0000 7f4caee00640 1 -- 172.21.15.27:0/3851485742 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4c900b6670 msgr2=0x7f4c900b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.184+0000 7f4caee00640 1 --2- 172.21.15.27:0/3851485742 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4c900b6670 0x7f4c900b8b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4c94035040 tx=0x7f4c94030340 comp rx=0 tx=0).stop 2024-09-21T00:24:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.184+0000 7f4caee00640 1 -- 172.21.15.27:0/3851485742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ca810fe60 msgr2=0x7f4ca81a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.184+0000 7f4caee00640 1 --2- 172.21.15.27:0/3851485742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ca810fe60 0x7f4ca81a1cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009fd0 tx=0x7f4ca000ef10 comp rx=0 tx=0).stop 2024-09-21T00:24:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.184+0000 7f4caee00640 1 -- 172.21.15.27:0/3851485742 shutdown_connections 2024-09-21T00:24:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.184+0000 7f4caee00640 1 --2- 172.21.15.27:0/3851485742 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4c900b6670 0x7f4c900b8b80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.184+0000 7f4caee00640 1 --2- 172.21.15.27:0/3851485742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ca810fe60 0x7f4ca81a1cc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.184+0000 7f4caee00640 1 --2- 172.21.15.27:0/3851485742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ca8076100 0x7f4ca819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.184+0000 7f4caee00640 1 --2- 172.21.15.27:0/3851485742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ca80757b0 0x7f4ca819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.184+0000 7f4caee00640 1 -- 172.21.15.27:0/3851485742 >> 172.21.15.27:0/3851485742 conn(0x7f4ca80ff390 msgr2=0x7f4ca8109a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.184+0000 7f4caee00640 1 -- 172.21.15.27:0/3851485742 shutdown_connections 2024-09-21T00:24:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:39.184+0000 7f4caee00640 1 -- 172.21.15.27:0/3851485742 wait complete. 2024-09-21T00:24:40.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:39 smithi027 ceph-mon[30745]: pgmap v475: 129 pgs: 129 active+clean; 140 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 990 KiB/s wr, 99 op/s 2024-09-21T00:24:40.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:39 smithi089 ceph-mon[34481]: pgmap v475: 129 pgs: 129 active+clean; 140 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 990 KiB/s wr, 99 op/s 2024-09-21T00:24:40.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:39 smithi098 ceph-mon[34725]: pgmap v475: 129 pgs: 129 active+clean; 140 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 990 KiB/s wr, 99 op/s 2024-09-21T00:24:42.034 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:41 smithi027 ceph-mon[30745]: pgmap v476: 129 pgs: 129 active+clean; 142 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.0 MiB/s wr, 168 op/s 2024-09-21T00:24:42.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:41 smithi089 ceph-mon[34481]: pgmap v476: 129 pgs: 129 active+clean; 142 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.0 MiB/s wr, 168 op/s 2024-09-21T00:24:42.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:41 smithi098 ceph-mon[34725]: pgmap v476: 129 pgs: 129 active+clean; 142 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.0 MiB/s wr, 168 op/s 2024-09-21T00:24:44.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:43 smithi027 ceph-mon[30745]: pgmap v477: 129 pgs: 129 active+clean; 144 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 303 op/s 2024-09-21T00:24:44.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:43 smithi089 ceph-mon[34481]: pgmap v477: 129 pgs: 129 active+clean; 144 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 303 op/s 2024-09-21T00:24:44.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:43 smithi098 ceph-mon[34725]: pgmap v477: 129 pgs: 129 active+clean; 144 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 303 op/s 2024-09-21T00:24:45.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:45 smithi027 ceph-mon[30745]: pgmap v478: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 441 op/s 2024-09-21T00:24:46.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:45 smithi089 ceph-mon[34481]: pgmap v478: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 441 op/s 2024-09-21T00:24:46.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:45 smithi098 ceph-mon[34725]: pgmap v478: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 441 op/s 2024-09-21T00:24:48.060 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:47 smithi027 ceph-mon[30745]: pgmap v479: 129 pgs: 129 active+clean; 153 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.2 MiB/s wr, 452 op/s 2024-09-21T00:24:48.061 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:47 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:24:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.058+0000 7fcd48c00640 1 -- 172.21.15.27:0/1051767232 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5610e91ad6a0 con 0x5610e8e26570 2024-09-21T00:24:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.058+0000 7fcd48c00640 10 client.25883 handle_command_reply: tid=0 2024-09-21T00:24:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.058+0000 7fcd6cea4740 1 client.25883 shutdown 2024-09-21T00:24:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.059+0000 7fcd6cea4740 1 -- 172.21.15.27:0/1051767232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e71b60 msgr2=0x5610e924a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.059+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/1051767232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e71b60 0x5610e924a260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c000c00 tx=0x7fcd4c022d00 comp rx=0 tx=0).stop 2024-09-21T00:24:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.059+0000 7fcd6cea4740 1 -- 172.21.15.27:0/1051767232 shutdown_connections 2024-09-21T00:24:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.059+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/1051767232 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5610e8e26570 0x5610e92c51c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.059+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/1051767232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610e8e92840 0x5610e924a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.059+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/1051767232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610e8e71b60 0x5610e924a260 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.059+0000 7fcd6cea4740 1 --2- 172.21.15.27:0/1051767232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610e8e69980 0x5610e9249d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.059+0000 7fcd6cea4740 1 -- 172.21.15.27:0/1051767232 >> 172.21.15.27:0/1051767232 conn(0x5610e8e86990 msgr2=0x5610e923bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.059+0000 7fcd48c00640 0 client.25883 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.059+0000 7fcd6cea4740 1 -- 172.21.15.27:0/1051767232 shutdown_connections 2024-09-21T00:24:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.060+0000 7fcd6cea4740 1 -- 172.21.15.27:0/1051767232 wait complete. 2024-09-21T00:24:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.060+0000 7fcd6cea4740 20 client.25883 trim_cache size 0 max 16384 2024-09-21T00:24:48.063 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:24:48.063 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 28, 2024-09-21T00:24:48.063 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24761, 2024-09-21T00:24:48.063 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 14, 2024-09-21T00:24:48.063 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T00:24:48.063 INFO:teuthology.orchestra.run.smithi027.stdout: "15ed622a": { 2024-09-21T00:24:48.063 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 14, 2024-09-21T00:24:48.063 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T00:24:48.063 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:24:48.063 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 147.8, 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:24:48.064 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:24:48.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.061+0000 7fcd66200640 1 -- 172.21.15.27:0/807332918 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcd440b6700 msgr2=0x7fcd440b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:48.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.061+0000 7fcd66200640 1 --2- 172.21.15.27:0/807332918 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcd440b6700 0x7fcd440b8c10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcd500e5020 tx=0x7fcd50000bc0 comp rx=0 tx=0).stop 2024-09-21T00:24:48.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.061+0000 7fcd66200640 1 -- 172.21.15.27:0/807332918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60105d80 msgr2=0x7fcd6019c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.061+0000 7fcd66200640 1 --2- 172.21.15.27:0/807332918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60105d80 0x7fcd6019c640 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000e9f0 tx=0x7fcd5000eeb0 comp rx=0 tx=0).stop 2024-09-21T00:24:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.061+0000 7fcd66200640 1 -- 172.21.15.27:0/807332918 shutdown_connections 2024-09-21T00:24:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.061+0000 7fcd66200640 1 --2- 172.21.15.27:0/807332918 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fcd440b6700 0x7fcd440b8c10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.061+0000 7fcd66200640 1 --2- 172.21.15.27:0/807332918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd6010c010 0x7fcd601a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.061+0000 7fcd66200640 1 --2- 172.21.15.27:0/807332918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd6010b6c0 0x7fcd6019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.061+0000 7fcd66200640 1 --2- 172.21.15.27:0/807332918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60105d80 0x7fcd6019c640 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.061+0000 7fcd66200640 1 -- 172.21.15.27:0/807332918 >> 172.21.15.27:0/807332918 conn(0x7fcd601017d0 msgr2=0x7fcd60103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.062+0000 7fcd66200640 1 -- 172.21.15.27:0/807332918 shutdown_connections 2024-09-21T00:24:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:24:48.062+0000 7fcd66200640 1 -- 172.21.15.27:0/807332918 wait complete. 2024-09-21T00:24:48.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:47 smithi089 ceph-mon[34481]: pgmap v479: 129 pgs: 129 active+clean; 153 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.2 MiB/s wr, 452 op/s 2024-09-21T00:24:48.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:47 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:24:48.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:47 smithi098 ceph-mon[34725]: pgmap v479: 129 pgs: 129 active+clean; 153 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.2 MiB/s wr, 452 op/s 2024-09-21T00:24:48.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:47 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:24:48.298 INFO:tasks.workunit.client.0.smithi027.stdout: 41 1648 0 1172 0 171 1 2024-09-21T00:24:49.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:48 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:24:49.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:48 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:24:49.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:48 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:24:49.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:48 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:24:49.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:48 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:24:49.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:48 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:24:49.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:48 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:24:49.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:48 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:24:49.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:48 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:24:50.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:49 smithi027 ceph-mon[30745]: pgmap v480: 129 pgs: 129 active+clean; 159 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.7 MiB/s wr, 518 op/s 2024-09-21T00:24:50.125 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '15ed622a', quiesce took 18.6 seconds. Will release after: 9.2 2024-09-21T00:24:50.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:49 smithi089 ceph-mon[34481]: pgmap v480: 129 pgs: 129 active+clean; 159 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.7 MiB/s wr, 518 op/s 2024-09-21T00:24:50.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:49 smithi098 ceph-mon[34725]: pgmap v480: 129 pgs: 129 active+clean; 159 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.7 MiB/s wr, 518 op/s 2024-09-21T00:24:52.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:51 smithi089 ceph-mon[34481]: pgmap v481: 129 pgs: 129 active+clean; 160 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 2.6 MiB/s wr, 509 op/s 2024-09-21T00:24:52.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:51 smithi098 ceph-mon[34725]: pgmap v481: 129 pgs: 129 active+clean; 160 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 2.6 MiB/s wr, 509 op/s 2024-09-21T00:24:52.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:51 smithi027 ceph-mon[30745]: pgmap v481: 129 pgs: 129 active+clean; 160 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 2.6 MiB/s wr, 509 op/s 2024-09-21T00:24:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.381+0000 7f374f200640 1 -- 172.21.15.27:0/2922462734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3750109b90 msgr2=0x7f37501103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.381+0000 7f374f200640 1 --2- 172.21.15.27:0/2922462734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3750109b90 0x7f37501103c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3748009ca0 tx=0x7f374802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.381+0000 7f374f200640 1 -- 172.21.15.27:0/2922462734 shutdown_connections 2024-09-21T00:24:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.381+0000 7f374f200640 1 --2- 172.21.15.27:0/2922462734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3750109b90 0x7f37501103c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.381+0000 7f374f200640 1 --2- 172.21.15.27:0/2922462734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3750109240 0x7f3750109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.381+0000 7f374f200640 1 --2- 172.21.15.27:0/2922462734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3750103910 0x7f3750103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.381+0000 7f374f200640 1 -- 172.21.15.27:0/2922462734 >> 172.21.15.27:0/2922462734 conn(0x7f37500ff390 msgr2=0x7f37501017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.381+0000 7f374f200640 1 -- 172.21.15.27:0/2922462734 shutdown_connections 2024-09-21T00:24:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.381+0000 7f374f200640 1 -- 172.21.15.27:0/2922462734 wait complete. 2024-09-21T00:24:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.382+0000 7f374f200640 1 Processor -- start 2024-09-21T00:24:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.382+0000 7f374f200640 1 -- start start 2024-09-21T00:24:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374f200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3750103910 0x7f375019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3750109240 0x7f375019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3750109b90 0x7f37501a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37501138a0 con 0x7f3750109b90 2024-09-21T00:24:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374f200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3750113720 con 0x7f3750109240 2024-09-21T00:24:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374f200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3750113a20 con 0x7f3750103910 2024-09-21T00:24:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3750109b90 0x7f37501a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3750109b90 0x7f37501a1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46254/0 (socket says 172.21.15.27:46254) 2024-09-21T00:24:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374e800640 1 -- 172.21.15.27:0/3564358626 learned_addr learned my addr 172.21.15.27:0/3564358626 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374d400640 1 --2- 172.21.15.27:0/3564358626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3750109240 0x7f375019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374de00640 1 --2- 172.21.15.27:0/3564358626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3750103910 0x7f375019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374d400640 1 -- 172.21.15.27:0/3564358626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3750103910 msgr2=0x7f375019c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374d400640 1 --2- 172.21.15.27:0/3564358626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3750103910 0x7f375019c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374d400640 1 -- 172.21.15.27:0/3564358626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3750109b90 msgr2=0x7f37501a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374d400640 1 --2- 172.21.15.27:0/3564358626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3750109b90 0x7f37501a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.383+0000 7f374d400640 1 -- 172.21.15.27:0/3564358626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37501a23d0 con 0x7f3750109240 2024-09-21T00:24:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.384+0000 7f374d400640 1 --2- 172.21.15.27:0/3564358626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3750109240 0x7f375019ca40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f375006c660 tx=0x7f373c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.384+0000 7f3746a00640 1 -- 172.21.15.27:0/3564358626 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c00cc40 con 0x7f3750109240 2024-09-21T00:24:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.384+0000 7f374f200640 1 -- 172.21.15.27:0/3564358626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37501a26c0 con 0x7f3750109240 2024-09-21T00:24:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.384+0000 7f3746a00640 1 -- 172.21.15.27:0/3564358626 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f373c00cde0 con 0x7f3750109240 2024-09-21T00:24:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.384+0000 7f3746a00640 1 -- 172.21.15.27:0/3564358626 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c004a00 con 0x7f3750109240 2024-09-21T00:24:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.384+0000 7f374f200640 1 -- 172.21.15.27:0/3564358626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37501a8e50 con 0x7f3750109240 2024-09-21T00:24:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.387+0000 7f375bdf7740 1 Processor -- start 2024-09-21T00:24:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.387+0000 7f375bdf7740 1 -- start start 2024-09-21T00:24:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.388+0000 7f375bdf7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8b051b0 0x5635c8ea9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.388+0000 7f375bdf7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 0x5635c8eaa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.388+0000 7f375bdf7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8ae44d0 0x5635c8eaa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.388+0000 7f375bdf7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5635c8e1f6b0 con 0x5635c8b051b0 2024-09-21T00:24:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.388+0000 7f375bdf7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5635c8e0eae0 con 0x5635c8ae44d0 2024-09-21T00:24:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.388+0000 7f375bdf7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5635c8e128f0 con 0x5635c8adc2f0 2024-09-21T00:24:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.388+0000 7f374e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8b051b0 0x5635c8ea9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.388+0000 7f374e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8b051b0 0x5635c8ea9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46264/0 (socket says 172.21.15.27:46264) 2024-09-21T00:24:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.388+0000 7f374e800640 1 -- 172.21.15.27:0/1388152372 learned_addr learned my addr 172.21.15.27:0/1388152372 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.388+0000 7f374de00640 1 --2- 172.21.15.27:0/1388152372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 0x5635c8eaa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.388+0000 7f3746a00640 1 -- 172.21.15.27:0/3564358626 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f373c004ba0 con 0x7f3750109240 2024-09-21T00:24:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.388+0000 7f374e800640 1 -- 172.21.15.27:0/1388152372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 msgr2=0x5635c8eaa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.388+0000 7f374e800640 1 --2- 172.21.15.27:0/1388152372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 0x5635c8eaa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.388+0000 7f374e800640 1 -- 172.21.15.27:0/1388152372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8ae44d0 msgr2=0x5635c8eaa890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:24:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.389+0000 7f374e800640 1 --2- 172.21.15.27:0/1388152372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8ae44d0 0x5635c8eaa890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.389+0000 7f374e800640 1 -- 172.21.15.27:0/1388152372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635c8ddb780 con 0x5635c8b051b0 2024-09-21T00:24:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.389+0000 7f374de00640 1 --2- 172.21.15.27:0/1388152372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 0x5635c8eaa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:24:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.389+0000 7f374e800640 1 --2- 172.21.15.27:0/1388152372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8b051b0 0x5635c8ea9e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f37480094c0 tx=0x7f3748009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84e6cdf486555e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.389+0000 7f373b400640 1 -- 172.21.15.27:0/1388152372 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748031db0 con 0x5635c8b051b0 2024-09-21T00:24:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.389+0000 7f373b400640 1 -- 172.21.15.27:0/1388152372 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3748036030 con 0x5635c8b051b0 2024-09-21T00:24:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.389+0000 7f373b400640 1 -- 172.21.15.27:0/1388152372 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37480042a0 con 0x5635c8b051b0 2024-09-21T00:24:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.390+0000 7f375bdf7740 1 -- 172.21.15.27:0/1388152372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8b051b0 msgr2=0x5635c8ea9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.390+0000 7f375bdf7740 1 --2- 172.21.15.27:0/1388152372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8b051b0 0x5635c8ea9e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f37480094c0 tx=0x7f3748009a10 comp rx=0 tx=0).stop 2024-09-21T00:24:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.390+0000 7f375bdf7740 1 -- 172.21.15.27:0/1388152372 shutdown_connections 2024-09-21T00:24:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.390+0000 7f375bdf7740 1 --2- 172.21.15.27:0/1388152372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8ae44d0 0x5635c8eaa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.390+0000 7f375bdf7740 1 --2- 172.21.15.27:0/1388152372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 0x5635c8eaa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.390+0000 7f375bdf7740 1 --2- 172.21.15.27:0/1388152372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8b051b0 0x5635c8ea9e10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.390+0000 7f375bdf7740 1 -- 172.21.15.27:0/1388152372 >> 172.21.15.27:0/1388152372 conn(0x5635c8af9300 msgr2=0x5635c8ea7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.390+0000 7f375bdf7740 1 -- 172.21.15.27:0/1388152372 shutdown_connections 2024-09-21T00:24:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.390+0000 7f375bdf7740 1 -- 172.21.15.27:0/1388152372 wait complete. 2024-09-21T00:24:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.390+0000 7f3746a00640 1 --2- 172.21.15.27:0/3564358626 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f37340b6580 0x7f37340b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.391+0000 7f3746a00640 1 -- 172.21.15.27:0/3564358626 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f373c0e7a90 con 0x7f3750109240 2024-09-21T00:24:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.391+0000 7f374de00640 1 --2- 172.21.15.27:0/3564358626 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f37340b6580 0x7f37340b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.391+0000 7f375bdf7740 1 -- start start 2024-09-21T00:24:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.391+0000 7f375bdf7740 1 Processor -- start 2024-09-21T00:24:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.391+0000 7f374de00640 1 --2- 172.21.15.27:0/3564358626 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f37340b6580 0x7f37340b8a90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5635c8ea8a90 tx=0x7f3740005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f375bdf7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8adc2f0 0x5635c8eb53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f375bdf7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8ae44d0 0x5635c8eb5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f375bdf7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8b051b0 0x5635c8eb5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f375bdf7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5635c8d46ab0 con 0x5635c8b051b0 2024-09-21T00:24:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f375bdf7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5635c8db5570 con 0x5635c8adc2f0 2024-09-21T00:24:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f375bdf7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5635c8eb83b0 con 0x5635c8ae44d0 2024-09-21T00:24:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f375bdf7740 10 client.0 _ll_register_callbacks cb 0x5635c8dc4100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:24:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f374de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8adc2f0 0x5635c8eb53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f374de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8adc2f0 0x5635c8eb53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39506/0 (socket says 172.21.15.27:39506) 2024-09-21T00:24:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f374e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8b051b0 0x5635c8eb5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f374de00640 1 -- 172.21.15.27:0/1587609183 learned_addr learned my addr 172.21.15.27:0/1587609183 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f374d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8ae44d0 0x5635c8eb5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f374de00640 1 -- 172.21.15.27:0/1587609183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8ae44d0 msgr2=0x5635c8eb5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f374de00640 1 --2- 172.21.15.27:0/1587609183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8ae44d0 0x5635c8eb5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f374de00640 1 -- 172.21.15.27:0/1587609183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8b051b0 msgr2=0x5635c8eb5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f374de00640 1 --2- 172.21.15.27:0/1587609183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8b051b0 0x5635c8eb5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.392+0000 7f374de00640 1 -- 172.21.15.27:0/1587609183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635c8b70790 con 0x5635c8adc2f0 2024-09-21T00:24:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.393+0000 7f374de00640 1 --2- 172.21.15.27:0/1587609183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8adc2f0 0x5635c8eb53c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3740004620 tx=0x7f3740004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50826dbc03dc4586 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.393+0000 7f372fe00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:24:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.393+0000 7f372fe00640 1 -- 172.21.15.27:0/1587609183 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740029440 con 0x5635c8adc2f0 2024-09-21T00:24:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.393+0000 7f375bdf7740 1 -- 172.21.15.27:0/1587609183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3748036030 con 0x5635c8adc2f0 2024-09-21T00:24:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.393+0000 7f372fe00640 1 -- 172.21.15.27:0/1587609183 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f374001f430 con 0x5635c8adc2f0 2024-09-21T00:24:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.393+0000 7f372fe00640 1 -- 172.21.15.27:0/1587609183 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f374001f630 con 0x5635c8adc2f0 2024-09-21T00:24:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.393+0000 7f375bdf7740 10 client.25910 fetch_fsmap learned FSMap version 28 2024-09-21T00:24:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.393+0000 7f375bdf7740 1 -- 172.21.15.27:0/1587609183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5635c8be9620 con 0x5635c8adc2f0 2024-09-21T00:24:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.394+0000 7f372fe00640 1 -- 172.21.15.27:0/1587609183 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3748036030 con 0x5635c8adc2f0 2024-09-21T00:24:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.394+0000 7f372fe00640 1 -- 172.21.15.27:0/1587609183 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3740027070 con 0x5635c8adc2f0 2024-09-21T00:24:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.394+0000 7f375bdf7740 10 client.25910 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:24:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.394+0000 7f375bdf7740 10 client.25910 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:24:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.394+0000 7f375bdf7740 20 client.25910 populate_metadata read hostname 'smithi027' 2024-09-21T00:24:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.394+0000 7f375bdf7740 1 --2- 172.21.15.27:0/1587609183 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5635c8ac3d80 0x5635c8ec7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.394+0000 7f375bdf7740 4 client.25910 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:24:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.394+0000 7f375bdf7740 1 -- 172.21.15.27:0/1587609183 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635c8e21600 con 0x5635c8ac3d80 2024-09-21T00:24:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.394+0000 7f374d400640 1 --2- 172.21.15.27:0/1587609183 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5635c8ac3d80 0x5635c8ec7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.395+0000 7f374d400640 1 --2- 172.21.15.27:0/1587609183 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5635c8ac3d80 0x5635c8ec7420 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1ea8fa37550f373 server_cookie=b8885a231dba68 in_seq=0 out_seq=0 2024-09-21T00:24:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.395+0000 7f372fe00640 10 client.25910 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.396+0000 7f372fe00640 1 -- 172.21.15.27:0/1587609183 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5635c8e21600 con 0x5635c8ac3d80 2024-09-21T00:24:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.396+0000 7f372fe00640 10 client.25910 handle_command_reply: tid=0 2024-09-21T00:24:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.396+0000 7f375bdf7740 1 client.25910 shutdown 2024-09-21T00:24:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.396+0000 7f375bdf7740 1 -- 172.21.15.27:0/1587609183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8adc2f0 msgr2=0x5635c8eb53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.396+0000 7f375bdf7740 1 --2- 172.21.15.27:0/1587609183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8adc2f0 0x5635c8eb53c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3740004620 tx=0x7f3740004730 comp rx=0 tx=0).stop 2024-09-21T00:24:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.396+0000 7f375bdf7740 1 -- 172.21.15.27:0/1587609183 shutdown_connections 2024-09-21T00:24:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.396+0000 7f375bdf7740 1 --2- 172.21.15.27:0/1587609183 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5635c8ac3d80 0x5635c8ec7420 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.396+0000 7f375bdf7740 1 --2- 172.21.15.27:0/1587609183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8b051b0 0x5635c8eb5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.396+0000 7f375bdf7740 1 --2- 172.21.15.27:0/1587609183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8ae44d0 0x5635c8eb5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.396+0000 7f372fe00640 0 client.25910 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.396+0000 7f375bdf7740 1 --2- 172.21.15.27:0/1587609183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8adc2f0 0x5635c8eb53c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.396+0000 7f375bdf7740 1 -- 172.21.15.27:0/1587609183 >> 172.21.15.27:0/1587609183 conn(0x5635c8af9300 msgr2=0x5635c8eaae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.396+0000 7f375bdf7740 1 -- 172.21.15.27:0/1587609183 shutdown_connections 2024-09-21T00:24:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.396+0000 7f375bdf7740 1 -- 172.21.15.27:0/1587609183 wait complete. 2024-09-21T00:24:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.397+0000 7f375bdf7740 20 client.25910 trim_cache size 0 max 16384 2024-09-21T00:24:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.412+0000 7f375bdf7740 1 Processor -- start 2024-09-21T00:24:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.412+0000 7f375bdf7740 1 -- start start 2024-09-21T00:24:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.412+0000 7f375bdf7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 0x5635c8eab550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.412+0000 7f375bdf7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8ae44d0 0x5635c8eb2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.412+0000 7f375bdf7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8b051b0 0x5635c8eb2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.412+0000 7f375bdf7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37501b56a0 con 0x5635c8ae44d0 2024-09-21T00:24:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.412+0000 7f375bdf7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f37501b2f70 con 0x5635c8b051b0 2024-09-21T00:24:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.412+0000 7f375bdf7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37501b0dc0 con 0x5635c8adc2f0 2024-09-21T00:24:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.413+0000 7f374d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8ae44d0 0x5635c8eb2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.413+0000 7f374d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8ae44d0 0x5635c8eb2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46278/0 (socket says 172.21.15.27:46278) 2024-09-21T00:24:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.413+0000 7f374d400640 1 -- 172.21.15.27:0/2042839380 learned_addr learned my addr 172.21.15.27:0/2042839380 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.413+0000 7f374de00640 1 --2- 172.21.15.27:0/2042839380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 0x5635c8eab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.413+0000 7f374e800640 1 --2- 172.21.15.27:0/2042839380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8b051b0 0x5635c8eb2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.413+0000 7f374d400640 1 -- 172.21.15.27:0/2042839380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 msgr2=0x5635c8eab550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.413+0000 7f374d400640 1 --2- 172.21.15.27:0/2042839380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 0x5635c8eab550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.413+0000 7f374d400640 1 -- 172.21.15.27:0/2042839380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8b051b0 msgr2=0x5635c8eb2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.413+0000 7f374d400640 1 --2- 172.21.15.27:0/2042839380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8b051b0 0x5635c8eb2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.413+0000 7f374d400640 1 -- 172.21.15.27:0/2042839380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635c8eb79a0 con 0x5635c8ae44d0 2024-09-21T00:24:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.414+0000 7f374d400640 1 --2- 172.21.15.27:0/2042839380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8ae44d0 0x5635c8eb2590 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f373c0ad480 tx=0x7f373c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9476b45ca89ddc11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.414+0000 7f373aa00640 1 -- 172.21.15.27:0/2042839380 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c01c040 con 0x5635c8ae44d0 2024-09-21T00:24:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.414+0000 7f373aa00640 1 -- 172.21.15.27:0/2042839380 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f373c0d4030 con 0x5635c8ae44d0 2024-09-21T00:24:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.414+0000 7f373aa00640 1 -- 172.21.15.27:0/2042839380 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c0a8e40 con 0x5635c8ae44d0 2024-09-21T00:24:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.414+0000 7f375bdf7740 1 -- 172.21.15.27:0/2042839380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8ae44d0 msgr2=0x5635c8eb2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.414+0000 7f375bdf7740 1 --2- 172.21.15.27:0/2042839380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8ae44d0 0x5635c8eb2590 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f373c0ad480 tx=0x7f373c009f60 comp rx=0 tx=0).stop 2024-09-21T00:24:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.415+0000 7f375bdf7740 1 -- 172.21.15.27:0/2042839380 shutdown_connections 2024-09-21T00:24:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.415+0000 7f375bdf7740 1 --2- 172.21.15.27:0/2042839380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8b051b0 0x5635c8eb2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.415+0000 7f375bdf7740 1 --2- 172.21.15.27:0/2042839380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8ae44d0 0x5635c8eb2590 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.415+0000 7f375bdf7740 1 --2- 172.21.15.27:0/2042839380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 0x5635c8eab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.415+0000 7f375bdf7740 1 -- 172.21.15.27:0/2042839380 >> 172.21.15.27:0/2042839380 conn(0x5635c8af9300 msgr2=0x5635c8eaae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.415+0000 7f375bdf7740 1 -- 172.21.15.27:0/2042839380 shutdown_connections 2024-09-21T00:24:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.415+0000 7f375bdf7740 1 -- 172.21.15.27:0/2042839380 wait complete. 2024-09-21T00:24:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.415+0000 7f375bdf7740 1 -- start start 2024-09-21T00:24:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.415+0000 7f375bdf7740 1 Processor -- start 2024-09-21T00:24:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.416+0000 7f375bdf7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 0x5635c8ebdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.416+0000 7f375bdf7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8ae44d0 0x5635c8ebb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.416+0000 7f375bdf7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8b051b0 0x5635c8ebb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.416+0000 7f375bdf7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37501b1430 con 0x5635c8ae44d0 2024-09-21T00:24:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.416+0000 7f375bdf7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f37501b28a0 con 0x5635c8b051b0 2024-09-21T00:24:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.416+0000 7f375bdf7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37501b1b00 con 0x5635c8adc2f0 2024-09-21T00:24:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.416+0000 7f375bdf7740 10 client.0 _ll_register_callbacks cb 0x7f3740009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:24:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.416+0000 7f374de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 0x5635c8ebdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.416+0000 7f374d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8ae44d0 0x5635c8ebb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.416+0000 7f374de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 0x5635c8ebdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:46432/0 (socket says 172.21.15.27:46432) 2024-09-21T00:24:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.416+0000 7f374de00640 1 -- 172.21.15.27:0/824569272 learned_addr learned my addr 172.21.15.27:0/824569272 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:24:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.416+0000 7f374e800640 1 --2- 172.21.15.27:0/824569272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8b051b0 0x5635c8ebb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.417+0000 7f374de00640 1 -- 172.21.15.27:0/824569272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8b051b0 msgr2=0x5635c8ebb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.417+0000 7f374de00640 1 --2- 172.21.15.27:0/824569272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8b051b0 0x5635c8ebb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.417+0000 7f374de00640 1 -- 172.21.15.27:0/824569272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8ae44d0 msgr2=0x5635c8ebb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:24:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.417+0000 7f374de00640 1 --2- 172.21.15.27:0/824569272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8ae44d0 0x5635c8ebb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.417+0000 7f374de00640 1 -- 172.21.15.27:0/824569272 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635c8ebbac0 con 0x5635c8adc2f0 2024-09-21T00:24:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.417+0000 7f374de00640 1 --2- 172.21.15.27:0/824569272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 0x5635c8ebdb80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3740029cf0 tx=0x7f3740012d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ce1d310e884520 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:24:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.417+0000 7f372fe00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:24:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.417+0000 7f372fe00640 1 -- 172.21.15.27:0/824569272 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374003c030 con 0x5635c8adc2f0 2024-09-21T00:24:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.417+0000 7f375bdf7740 1 -- 172.21.15.27:0/824569272 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f373c0d4030 con 0x5635c8adc2f0 2024-09-21T00:24:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.417+0000 7f372fe00640 1 -- 172.21.15.27:0/824569272 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f374001f430 con 0x5635c8adc2f0 2024-09-21T00:24:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.417+0000 7f372fe00640 1 -- 172.21.15.27:0/824569272 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3740003e50 con 0x5635c8adc2f0 2024-09-21T00:24:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.417+0000 7f375bdf7740 10 client.25891 fetch_fsmap learned FSMap version 28 2024-09-21T00:24:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.417+0000 7f375bdf7740 1 -- 172.21.15.27:0/824569272 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5635c8f31ba0 con 0x5635c8adc2f0 2024-09-21T00:24:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.418+0000 7f372fe00640 1 -- 172.21.15.27:0/824569272 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f373c0d4030 con 0x5635c8adc2f0 2024-09-21T00:24:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.418+0000 7f372fe00640 1 -- 172.21.15.27:0/824569272 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3740027050 con 0x5635c8adc2f0 2024-09-21T00:24:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.418+0000 7f375bdf7740 10 client.25891 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:24:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.418+0000 7f375bdf7740 10 client.25891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:24:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.418+0000 7f375bdf7740 20 client.25891 populate_metadata read hostname 'smithi027' 2024-09-21T00:24:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.418+0000 7f375bdf7740 1 --2- 172.21.15.27:0/824569272 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5635c8a991c0 0x5635c8f343f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:24:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.418+0000 7f375bdf7740 4 client.25891 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:24:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.418+0000 7f375bdf7740 1 -- 172.21.15.27:0/824569272 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5635c8e1d0b0 con 0x5635c8a991c0 2024-09-21T00:24:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.418+0000 7f374d400640 1 --2- 172.21.15.27:0/824569272 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5635c8a991c0 0x5635c8f343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:24:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.419+0000 7f374d400640 1 --2- 172.21.15.27:0/824569272 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5635c8a991c0 0x5635c8f343f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d1895c49dc09073 server_cookie=814551c256e23838 in_seq=0 out_seq=0 2024-09-21T00:24:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.419+0000 7f372fe00640 10 client.25891 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.500+0000 7f372fe00640 1 -- 172.21.15.27:0/824569272 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5635c8e1d0b0 con 0x5635c8a991c0 2024-09-21T00:24:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.500+0000 7f372fe00640 10 client.25891 handle_command_reply: tid=0 2024-09-21T00:24:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.500+0000 7f375bdf7740 1 client.25891 shutdown 2024-09-21T00:24:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.501+0000 7f375bdf7740 1 -- 172.21.15.27:0/824569272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 msgr2=0x5635c8ebdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.501+0000 7f375bdf7740 1 --2- 172.21.15.27:0/824569272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 0x5635c8ebdb80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3740029cf0 tx=0x7f3740012d70 comp rx=0 tx=0).stop 2024-09-21T00:24:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.501+0000 7f375bdf7740 1 -- 172.21.15.27:0/824569272 shutdown_connections 2024-09-21T00:24:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.501+0000 7f375bdf7740 1 --2- 172.21.15.27:0/824569272 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5635c8a991c0 0x5635c8f343f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.501+0000 7f375bdf7740 1 --2- 172.21.15.27:0/824569272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5635c8b051b0 0x5635c8ebb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.501+0000 7f372fe00640 0 client.25891 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:24:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.501+0000 7f375bdf7740 1 --2- 172.21.15.27:0/824569272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635c8ae44d0 0x5635c8ebb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.501+0000 7f375bdf7740 1 --2- 172.21.15.27:0/824569272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5635c8adc2f0 0x5635c8ebdb80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.501+0000 7f375bdf7740 1 -- 172.21.15.27:0/824569272 >> 172.21.15.27:0/824569272 conn(0x5635c8af9300 msgr2=0x5635c8eb6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.501+0000 7f375bdf7740 1 -- 172.21.15.27:0/824569272 shutdown_connections 2024-09-21T00:24:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.501+0000 7f375bdf7740 1 -- 172.21.15.27:0/824569272 wait complete. 2024-09-21T00:24:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.501+0000 7f375bdf7740 20 client.25891 trim_cache size 0 max 16384 2024-09-21T00:24:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:24:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:24:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:24:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:24:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.502+0000 7f374f200640 1 -- 172.21.15.27:0/3564358626 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f37340b6580 msgr2=0x7f37340b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.502+0000 7f374f200640 1 --2- 172.21.15.27:0/3564358626 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f37340b6580 0x7f37340b8a90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5635c8ea8a90 tx=0x7f3740005f90 comp rx=0 tx=0).stop 2024-09-21T00:24:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.502+0000 7f374f200640 1 -- 172.21.15.27:0/3564358626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3750109240 msgr2=0x7f375019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:24:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.502+0000 7f374f200640 1 --2- 172.21.15.27:0/3564358626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3750109240 0x7f375019ca40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f375006c660 tx=0x7f373c00ee50 comp rx=0 tx=0).stop 2024-09-21T00:24:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.502+0000 7f374f200640 1 -- 172.21.15.27:0/3564358626 shutdown_connections 2024-09-21T00:24:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.502+0000 7f374f200640 1 --2- 172.21.15.27:0/3564358626 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f37340b6580 0x7f37340b8a90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.502+0000 7f374f200640 1 --2- 172.21.15.27:0/3564358626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3750109b90 0x7f37501a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.502+0000 7f374f200640 1 --2- 172.21.15.27:0/3564358626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3750109240 0x7f375019ca40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.502+0000 7f374f200640 1 --2- 172.21.15.27:0/3564358626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3750103910 0x7f375019c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:24:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.502+0000 7f374f200640 1 -- 172.21.15.27:0/3564358626 >> 172.21.15.27:0/3564358626 conn(0x7f37500ff390 msgr2=0x7f3750108830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:24:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.502+0000 7f374f200640 1 -- 172.21.15.27:0/3564358626 shutdown_connections 2024-09-21T00:24:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:24:52.502+0000 7f374f200640 1 -- 172.21.15.27:0/3564358626 wait complete. 2024-09-21T00:24:54.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:53 smithi089 ceph-mon[34481]: pgmap v482: 129 pgs: 129 active+clean; 160 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 466 op/s 2024-09-21T00:24:54.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:53 smithi098 ceph-mon[34725]: pgmap v482: 129 pgs: 129 active+clean; 160 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 466 op/s 2024-09-21T00:24:54.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:53 smithi027 ceph-mon[30745]: pgmap v482: 129 pgs: 129 active+clean; 160 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 466 op/s 2024-09-21T00:24:56.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:55 smithi089 ceph-mon[34481]: pgmap v483: 129 pgs: 129 active+clean; 162 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 365 op/s 2024-09-21T00:24:56.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:55 smithi098 ceph-mon[34725]: pgmap v483: 129 pgs: 129 active+clean; 162 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 365 op/s 2024-09-21T00:24:56.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:55 smithi027 ceph-mon[30745]: pgmap v483: 129 pgs: 129 active+clean; 162 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 365 op/s 2024-09-21T00:24:58.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:57 smithi089 ceph-mon[34481]: pgmap v484: 129 pgs: 129 active+clean; 162 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-09-21T00:24:58.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:57 smithi098 ceph-mon[34725]: pgmap v484: 129 pgs: 129 active+clean; 162 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-09-21T00:24:58.298 INFO:tasks.workunit.client.0.smithi027.stdout: 41 35886 0 25381 0 8668 0 2024-09-21T00:24:58.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:57 smithi027 ceph-mon[30745]: pgmap v484: 129 pgs: 129 active+clean; 162 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-09-21T00:24:59.326 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:24:59.729 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:25:00.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:24:59 smithi027 ceph-mon[30745]: pgmap v485: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 842 KiB/s wr, 207 op/s 2024-09-21T00:25:00.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:24:59 smithi089 ceph-mon[34481]: pgmap v485: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 842 KiB/s wr, 207 op/s 2024-09-21T00:25:00.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:24:59 smithi098 ceph-mon[34725]: pgmap v485: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 842 KiB/s wr, 207 op/s 2024-09-21T00:25:00.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.345+0000 7f449d400640 1 -- 172.21.15.27:0/871857073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4498105300 msgr2=0x7f449810d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:00.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.345+0000 7f449d400640 1 --2- 172.21.15.27:0/871857073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4498105300 0x7f449810d850 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4480009900 tx=0x7f448002a410 comp rx=0 tx=0).stop 2024-09-21T00:25:00.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.345+0000 7f449d400640 1 -- 172.21.15.27:0/871857073 shutdown_connections 2024-09-21T00:25:00.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.345+0000 7f449d400640 1 --2- 172.21.15.27:0/871857073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f449810de10 0x7f4498110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:00.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.345+0000 7f449d400640 1 --2- 172.21.15.27:0/871857073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4498105300 0x7f449810d850 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:00.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.345+0000 7f449d400640 1 --2- 172.21.15.27:0/871857073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44981049b0 0x7f4498104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:00.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.345+0000 7f449d400640 1 -- 172.21.15.27:0/871857073 >> 172.21.15.27:0/871857073 conn(0x7f4498079760 msgr2=0x7f4498079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:00.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.345+0000 7f449d400640 1 -- 172.21.15.27:0/871857073 shutdown_connections 2024-09-21T00:25:00.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.345+0000 7f449d400640 1 -- 172.21.15.27:0/871857073 wait complete. 2024-09-21T00:25:00.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.346+0000 7f449d400640 1 Processor -- start 2024-09-21T00:25:00.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.346+0000 7f449d400640 1 -- start start 2024-09-21T00:25:00.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.347+0000 7f449d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44981049b0 0x7f449819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:00.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.347+0000 7f449d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4498105300 0x7f449819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:00.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.347+0000 7f449d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f449810de10 0x7f44981a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:00.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.347+0000 7f449d400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4498113a70 con 0x7f4498105300 2024-09-21T00:25:00.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.347+0000 7f449d400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f44981138f0 con 0x7f44981049b0 2024-09-21T00:25:00.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.347+0000 7f449d400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4498113bf0 con 0x7f449810de10 2024-09-21T00:25:00.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.347+0000 7f448de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4498105300 0x7f449819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:00.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.347+0000 7f4496800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44981049b0 0x7f449819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:00.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.347+0000 7f448de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4498105300 0x7f449819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57106/0 (socket says 172.21.15.27:57106) 2024-09-21T00:25:00.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.347+0000 7f448de00640 1 -- 172.21.15.27:0/2877693256 learned_addr learned my addr 172.21.15.27:0/2877693256 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:00.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.347+0000 7f4497200640 1 --2- 172.21.15.27:0/2877693256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f449810de10 0x7f44981a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:00.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.348+0000 7f4496800640 1 -- 172.21.15.27:0/2877693256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f449810de10 msgr2=0x7f44981a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:00.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.348+0000 7f4496800640 1 --2- 172.21.15.27:0/2877693256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f449810de10 0x7f44981a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:00.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.348+0000 7f4496800640 1 -- 172.21.15.27:0/2877693256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4498105300 msgr2=0x7f449819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:00.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.348+0000 7f4496800640 1 --2- 172.21.15.27:0/2877693256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4498105300 0x7f449819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:00.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.348+0000 7f4496800640 1 -- 172.21.15.27:0/2877693256 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44981a2630 con 0x7f44981049b0 2024-09-21T00:25:00.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.348+0000 7f4496800640 1 --2- 172.21.15.27:0/2877693256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44981049b0 0x7f449819c6c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f448800e9f0 tx=0x7f448800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:00.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.348+0000 7f448fe00640 1 -- 172.21.15.27:0/2877693256 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448800ccc0 con 0x7f44981049b0 2024-09-21T00:25:00.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.348+0000 7f449d400640 1 -- 172.21.15.27:0/2877693256 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44981a2920 con 0x7f44981049b0 2024-09-21T00:25:00.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.348+0000 7f448fe00640 1 -- 172.21.15.27:0/2877693256 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4488005020 con 0x7f44981049b0 2024-09-21T00:25:00.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.348+0000 7f448fe00640 1 -- 172.21.15.27:0/2877693256 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4488004a00 con 0x7f44981049b0 2024-09-21T00:25:00.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.349+0000 7f449d400640 1 -- 172.21.15.27:0/2877693256 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44981a9080 con 0x7f44981049b0 2024-09-21T00:25:00.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.350+0000 7f449d400640 1 -- 172.21.15.27:0/2877693256 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f449806c2c0 con 0x7f44981049b0 2024-09-21T00:25:00.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.352+0000 7f448fe00640 1 -- 172.21.15.27:0/2877693256 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4488004ba0 con 0x7f44981049b0 2024-09-21T00:25:00.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.354+0000 7f448fe00640 1 --2- 172.21.15.27:0/2877693256 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f447c0b6700 0x7f447c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:00.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.354+0000 7f448fe00640 1 -- 172.21.15.27:0/2877693256 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f44880e7a90 con 0x7f44981049b0 2024-09-21T00:25:00.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.354+0000 7f448de00640 1 --2- 172.21.15.27:0/2877693256 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f447c0b6700 0x7f447c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:00.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.355+0000 7f448de00640 1 --2- 172.21.15.27:0/2877693256 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f447c0b6700 0x7f447c0b8c10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f449810fd80 tx=0x7f4480009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:00.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.356+0000 7f448fe00640 1 -- 172.21.15.27:0/2877693256 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f44880adc80 con 0x7f44981049b0 2024-09-21T00:25:00.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.595+0000 7f449d400640 1 -- 172.21.15.27:0/2877693256 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4498104dc0 con 0x7f44981049b0 2024-09-21T00:25:00.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.596+0000 7f448fe00640 1 -- 172.21.15.27:0/2877693256 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11896 (secure 0 0 0) 0x7f44880b2060 con 0x7f44981049b0 2024-09-21T00:25:00.598 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:25:00.598 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"btime":"2024-09-21T00:23:52:563301+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:23:52.207999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:25:00.598 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-09-21T00:25:00.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.599+0000 7f449d400640 1 -- 172.21.15.27:0/2877693256 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f447c0b6700 msgr2=0x7f447c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:00.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.599+0000 7f449d400640 1 --2- 172.21.15.27:0/2877693256 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f447c0b6700 0x7f447c0b8c10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f449810fd80 tx=0x7f4480009560 comp rx=0 tx=0).stop 2024-09-21T00:25:00.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.599+0000 7f449d400640 1 -- 172.21.15.27:0/2877693256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44981049b0 msgr2=0x7f449819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:00.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.599+0000 7f449d400640 1 --2- 172.21.15.27:0/2877693256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44981049b0 0x7f449819c6c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f448800e9f0 tx=0x7f448800eeb0 comp rx=0 tx=0).stop 2024-09-21T00:25:00.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.600+0000 7f449d400640 1 -- 172.21.15.27:0/2877693256 shutdown_connections 2024-09-21T00:25:00.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.600+0000 7f449d400640 1 --2- 172.21.15.27:0/2877693256 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f447c0b6700 0x7f447c0b8c10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:00.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.600+0000 7f449d400640 1 --2- 172.21.15.27:0/2877693256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f449810de10 0x7f44981a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:00.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.600+0000 7f449d400640 1 --2- 172.21.15.27:0/2877693256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4498105300 0x7f449819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:00.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.600+0000 7f449d400640 1 --2- 172.21.15.27:0/2877693256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44981049b0 0x7f449819c6c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:00.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.600+0000 7f449d400640 1 -- 172.21.15.27:0/2877693256 >> 172.21.15.27:0/2877693256 conn(0x7f4498079760 msgr2=0x7f44981005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:00.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.600+0000 7f449d400640 1 -- 172.21.15.27:0/2877693256 shutdown_connections 2024-09-21T00:25:00.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:00.600+0000 7f449d400640 1 -- 172.21.15.27:0/2877693256 wait complete. 2024-09-21T00:25:01.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:00 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2877693256' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:25:01.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:00 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2877693256' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:25:01.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:00 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2877693256' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:25:01.516 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id 15ed622a --release --await' 2024-09-21T00:25:01.516 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id 15ed622a --release --await 2024-09-21T00:25:01.906 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:25:02.215 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:01 smithi027 ceph-mon[30745]: pgmap v486: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 321 KiB/s wr, 132 op/s 2024-09-21T00:25:02.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:01 smithi089 ceph-mon[34481]: pgmap v486: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 321 KiB/s wr, 132 op/s 2024-09-21T00:25:02.243 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:01 smithi098 ceph-mon[34725]: pgmap v486: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 321 KiB/s wr, 132 op/s 2024-09-21T00:25:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.558+0000 7f5cc8000640 1 -- 172.21.15.27:0/604065942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc0078290 msgr2=0x7f5cc0114210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.558+0000 7f5cc8000640 1 --2- 172.21.15.27:0/604065942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc0078290 0x7f5cc0114210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009900 tx=0x7f5cac02a3f0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.558+0000 7f5cc8000640 1 -- 172.21.15.27:0/604065942 shutdown_connections 2024-09-21T00:25:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.558+0000 7f5cc8000640 1 --2- 172.21.15.27:0/604065942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc0114750 0x7f5cc0116b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.558+0000 7f5cc8000640 1 --2- 172.21.15.27:0/604065942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc0078290 0x7f5cc0114210 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.558+0000 7f5cc8000640 1 --2- 172.21.15.27:0/604065942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cc0077940 0x7f5cc0077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.558+0000 7f5cc8000640 1 -- 172.21.15.27:0/604065942 >> 172.21.15.27:0/604065942 conn(0x7f5cc0101860 msgr2=0x7f5cc0103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.558+0000 7f5cc8000640 1 -- 172.21.15.27:0/604065942 shutdown_connections 2024-09-21T00:25:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.558+0000 7f5cc8000640 1 -- 172.21.15.27:0/604065942 wait complete. 2024-09-21T00:25:02.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.559+0000 7f5cc8000640 1 Processor -- start 2024-09-21T00:25:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.560+0000 7f5cc8000640 1 -- start start 2024-09-21T00:25:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.560+0000 7f5cc8000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc0077940 0x7f5cc01a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.560+0000 7f5cc8000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc0078290 0x7f5cc01a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.560+0000 7f5cc8000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cc0114750 0x7f5cc01a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.560+0000 7f5cc8000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc011a090 con 0x7f5cc0077940 2024-09-21T00:25:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.560+0000 7f5cc8000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc0119f10 con 0x7f5cc0078290 2024-09-21T00:25:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.560+0000 7f5cc8000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc011a210 con 0x7f5cc0114750 2024-09-21T00:25:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.560+0000 7f5cc5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc0077940 0x7f5cc01a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.560+0000 7f5cc5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc0077940 0x7f5cc01a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57120/0 (socket says 172.21.15.27:57120) 2024-09-21T00:25:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.560+0000 7f5cc5600640 1 -- 172.21.15.27:0/329599540 learned_addr learned my addr 172.21.15.27:0/329599540 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.561+0000 7f5cc6000640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cc0114750 0x7f5cc01a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.561+0000 7f5cc4c00640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc0078290 0x7f5cc01a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.561+0000 7f5cc5600640 1 -- 172.21.15.27:0/329599540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cc0114750 msgr2=0x7f5cc01a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:02.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.561+0000 7f5cc5600640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cc0114750 0x7f5cc01a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.561+0000 7f5cc5600640 1 -- 172.21.15.27:0/329599540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc0078290 msgr2=0x7f5cc01a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:02.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.561+0000 7f5cc5600640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc0078290 0x7f5cc01a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.561+0000 7f5cc5600640 1 -- 172.21.15.27:0/329599540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc01a6a50 con 0x7f5cc0077940 2024-09-21T00:25:02.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.561+0000 7f5cc6000640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cc0114750 0x7f5cc01a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:02.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.561+0000 7f5cc4c00640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc0078290 0x7f5cc01a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:02.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.561+0000 7f5cc5600640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc0077940 0x7f5cc01a0ae0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400e9f0 tx=0x7f5cb400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:02.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.562+0000 7f5cbe000640 1 -- 172.21.15.27:0/329599540 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb400cd80 con 0x7f5cc0077940 2024-09-21T00:25:02.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.562+0000 7f5cbe000640 1 -- 172.21.15.27:0/329599540 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb40054d0 con 0x7f5cc0077940 2024-09-21T00:25:02.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.562+0000 7f5cbe000640 1 -- 172.21.15.27:0/329599540 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4010690 con 0x7f5cc0077940 2024-09-21T00:25:02.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.562+0000 7f5cc8000640 1 -- 172.21.15.27:0/329599540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc01a6d40 con 0x7f5cc0077940 2024-09-21T00:25:02.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.562+0000 7f5cc8000640 1 -- 172.21.15.27:0/329599540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc01ad4a0 con 0x7f5cc0077940 2024-09-21T00:25:02.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.565+0000 7f5ccedba740 1 Processor -- start 2024-09-21T00:25:02.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.566+0000 7f5ccedba740 1 -- start start 2024-09-21T00:25:02.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.566+0000 7f5ccedba740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858c7520 0x55f285c6e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.566+0000 7f5ccedba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f28589e660 0x55f285c6e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.566+0000 7f5ccedba740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2858a6840 0x55f285c6ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.566+0000 7f5ccedba740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f285be5290 con 0x55f2858a6840 2024-09-21T00:25:02.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.566+0000 7f5ccedba740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f285bd46c0 con 0x55f2858c7520 2024-09-21T00:25:02.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.566+0000 7f5ccedba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f285bd84d0 con 0x55f28589e660 2024-09-21T00:25:02.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.566+0000 7f5cbe000640 1 -- 172.21.15.27:0/329599540 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5cb4004e90 con 0x7f5cc0077940 2024-09-21T00:25:02.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.566+0000 7f5cc4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2858a6840 0x55f285c6ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.566+0000 7f5cc6000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858c7520 0x55f285c6e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.566+0000 7f5cc4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2858a6840 0x55f285c6ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57126/0 (socket says 172.21.15.27:57126) 2024-09-21T00:25:02.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.566+0000 7f5cc4c00640 1 -- 172.21.15.27:0/831880023 learned_addr learned my addr 172.21.15.27:0/831880023 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:02.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.567+0000 7f5cc6000640 1 -- 172.21.15.27:0/831880023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f28589e660 msgr2=0x55f285c6e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:02.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.567+0000 7f5cc6000640 1 --2- 172.21.15.27:0/831880023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f28589e660 0x55f285c6e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.567+0000 7f5cc6000640 1 -- 172.21.15.27:0/831880023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2858a6840 msgr2=0x55f285c6ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:02.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.567+0000 7f5cc6000640 1 --2- 172.21.15.27:0/831880023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2858a6840 0x55f285c6ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.567+0000 7f5cc6000640 1 -- 172.21.15.27:0/831880023 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f285b61460 con 0x55f2858c7520 2024-09-21T00:25:02.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.567+0000 7f5cc6000640 1 --2- 172.21.15.27:0/831880023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858c7520 0x55f285c6e1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009f90 tx=0x7f5cb8010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc65e05fd77a34c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:02.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.567+0000 7f5cb2600640 1 -- 172.21.15.27:0/831880023 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8011700 con 0x55f2858c7520 2024-09-21T00:25:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.568+0000 7f5cb2600640 1 -- 172.21.15.27:0/831880023 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb8011d50 con 0x55f2858c7520 2024-09-21T00:25:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.568+0000 7f5cb2600640 1 -- 172.21.15.27:0/831880023 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb801ce40 con 0x55f2858c7520 2024-09-21T00:25:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.568+0000 7f5ccedba740 1 -- 172.21.15.27:0/831880023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858c7520 msgr2=0x55f285c6e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.568+0000 7f5ccedba740 1 --2- 172.21.15.27:0/831880023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858c7520 0x55f285c6e1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009f90 tx=0x7f5cb8010860 comp rx=0 tx=0).stop 2024-09-21T00:25:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.569+0000 7f5cbe000640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5ca80b6700 0x7f5ca80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.569+0000 7f5cbe000640 1 -- 172.21.15.27:0/329599540 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f5cb40e7f30 con 0x7f5cc0077940 2024-09-21T00:25:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.569+0000 7f5ccedba740 1 -- 172.21.15.27:0/831880023 shutdown_connections 2024-09-21T00:25:02.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.569+0000 7f5ccedba740 1 --2- 172.21.15.27:0/831880023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2858a6840 0x55f285c6ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.569+0000 7f5ccedba740 1 --2- 172.21.15.27:0/831880023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f28589e660 0x55f285c6e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.569+0000 7f5ccedba740 1 --2- 172.21.15.27:0/831880023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858c7520 0x55f285c6e1c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.569+0000 7f5ccedba740 1 -- 172.21.15.27:0/831880023 >> 172.21.15.27:0/831880023 conn(0x55f2858bb910 msgr2=0x55f285c6c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:02.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.569+0000 7f5ccedba740 1 -- 172.21.15.27:0/831880023 shutdown_connections 2024-09-21T00:25:02.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.569+0000 7f5cc6000640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5ca80b6700 0x7f5ca80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.569+0000 7f5ccedba740 1 -- 172.21.15.27:0/831880023 wait complete. 2024-09-21T00:25:02.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.570+0000 7f5cc6000640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5ca80b6700 0x7f5ca80b8c10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5cb801cb20 tx=0x7f5cb8013040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:02.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.570+0000 7f5ccedba740 1 -- start start 2024-09-21T00:25:02.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.570+0000 7f5ccedba740 1 Processor -- start 2024-09-21T00:25:02.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5ccedba740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f28589e660 0x55f285c79d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5ccedba740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2858a6840 0x55f285c7a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5ccedba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858c7520 0x55f285c85cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5ccedba740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f285a35bb0 con 0x55f2858a6840 2024-09-21T00:25:02.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5ccedba740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f285b6f6c0 con 0x55f28589e660 2024-09-21T00:25:02.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5ccedba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f285c7c9a0 con 0x55f2858c7520 2024-09-21T00:25:02.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5ccedba740 10 client.0 _ll_register_callbacks cb 0x55f285b8a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:25:02.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5cc4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2858a6840 0x55f285c7a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5cc5600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f28589e660 0x55f285c79d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5cc4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2858a6840 0x55f285c7a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57136/0 (socket says 172.21.15.27:57136) 2024-09-21T00:25:02.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5cc4c00640 1 -- 172.21.15.27:0/825512948 learned_addr learned my addr 172.21.15.27:0/825512948 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:02.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5cc6000640 1 --2- 172.21.15.27:0/825512948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858c7520 0x55f285c85cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5cc5600640 1 -- 172.21.15.27:0/825512948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858c7520 msgr2=0x55f285c85cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:02.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5cc5600640 1 --2- 172.21.15.27:0/825512948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858c7520 0x55f285c85cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5cc5600640 1 -- 172.21.15.27:0/825512948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2858a6840 msgr2=0x55f285c7a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:02.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5cc5600640 1 --2- 172.21.15.27:0/825512948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2858a6840 0x55f285c7a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.571+0000 7f5cc5600640 1 -- 172.21.15.27:0/825512948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f285932b00 con 0x55f28589e660 2024-09-21T00:25:02.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.572+0000 7f5cc4c00640 1 --2- 172.21.15.27:0/825512948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2858a6840 0x55f285c7a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:02.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.572+0000 7f5cc6000640 1 --2- 172.21.15.27:0/825512948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858c7520 0x55f285c85cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:25:02.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.572+0000 7f5cc5600640 1 --2- 172.21.15.27:0/825512948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f28589e660 0x55f285c79d70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40e5290 tx=0x7f5cb40048d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c864e3068d59516c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:02.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.572+0000 7f5ca6a00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:25:02.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.572+0000 7f5ca6a00640 1 -- 172.21.15.27:0/825512948 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4005180 con 0x55f28589e660 2024-09-21T00:25:02.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.572+0000 7f5ca6a00640 1 -- 172.21.15.27:0/825512948 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f5cb40ace00 con 0x55f28589e660 2024-09-21T00:25:02.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.572+0000 7f5ccedba740 1 -- 172.21.15.27:0/825512948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb8011d50 con 0x55f28589e660 2024-09-21T00:25:02.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.572+0000 7f5ca6a00640 1 -- 172.21.15.27:0/825512948 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5cb40ad4a0 con 0x55f28589e660 2024-09-21T00:25:02.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.572+0000 7f5ccedba740 10 client.25934 fetch_fsmap learned FSMap version 28 2024-09-21T00:25:02.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.572+0000 7f5ccedba740 1 -- 172.21.15.27:0/825512948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2859a9960 con 0x55f28589e660 2024-09-21T00:25:02.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.573+0000 7f5ca6a00640 1 -- 172.21.15.27:0/825512948 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb8011d50 con 0x55f28589e660 2024-09-21T00:25:02.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.573+0000 7f5ca6a00640 1 -- 172.21.15.27:0/825512948 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5cb40a92e0 con 0x55f28589e660 2024-09-21T00:25:02.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.573+0000 7f5ccedba740 10 client.25934 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:25:02.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.573+0000 7f5ccedba740 10 client.25934 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:25:02.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.573+0000 7f5ccedba740 20 client.25934 populate_metadata read hostname 'smithi027' 2024-09-21T00:25:02.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.573+0000 7f5ccedba740 1 --2- 172.21.15.27:0/825512948 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f285885ea0 0x55f285c8bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.573+0000 7f5ccedba740 4 client.25934 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:25:02.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.573+0000 7f5ccedba740 1 -- 172.21.15.27:0/825512948 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f285be71e0 con 0x55f285885ea0 2024-09-21T00:25:02.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.573+0000 7f5cc4c00640 1 --2- 172.21.15.27:0/825512948 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f285885ea0 0x55f285c8bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.574+0000 7f5cc4c00640 1 --2- 172.21.15.27:0/825512948 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f285885ea0 0x55f285c8bce0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=270ff359f4241162 server_cookie=cc8eff62ce08ddef in_seq=0 out_seq=0 2024-09-21T00:25:02.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.574+0000 7f5ca6a00640 10 client.25934 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:02.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.575+0000 7f5ca6a00640 1 -- 172.21.15.27:0/825512948 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f285be71e0 con 0x55f285885ea0 2024-09-21T00:25:02.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.575+0000 7f5ca6a00640 10 client.25934 handle_command_reply: tid=0 2024-09-21T00:25:02.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.575+0000 7f5ccedba740 1 client.25934 shutdown 2024-09-21T00:25:02.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.575+0000 7f5ccedba740 1 -- 172.21.15.27:0/825512948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f28589e660 msgr2=0x55f285c79d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:02.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.575+0000 7f5ccedba740 1 --2- 172.21.15.27:0/825512948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f28589e660 0x55f285c79d70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40e5290 tx=0x7f5cb40048d0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.575+0000 7f5ccedba740 1 -- 172.21.15.27:0/825512948 shutdown_connections 2024-09-21T00:25:02.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.575+0000 7f5ccedba740 1 --2- 172.21.15.27:0/825512948 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f285885ea0 0x55f285c8bce0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.575+0000 7f5ccedba740 1 --2- 172.21.15.27:0/825512948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858c7520 0x55f285c85cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.575+0000 7f5ccedba740 1 --2- 172.21.15.27:0/825512948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2858a6840 0x55f285c7a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.575+0000 7f5ccedba740 1 --2- 172.21.15.27:0/825512948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f28589e660 0x55f285c79d70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.575+0000 7f5ccedba740 1 -- 172.21.15.27:0/825512948 >> 172.21.15.27:0/825512948 conn(0x55f2858bb910 msgr2=0x55f285c71350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:02.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.575+0000 7f5ca6a00640 0 client.25934 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:02.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.576+0000 7f5ccedba740 1 -- 172.21.15.27:0/825512948 shutdown_connections 2024-09-21T00:25:02.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.576+0000 7f5ccedba740 1 -- 172.21.15.27:0/825512948 wait complete. 2024-09-21T00:25:02.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.576+0000 7f5ccedba740 20 client.25934 trim_cache size 0 max 16384 2024-09-21T00:25:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.594+0000 7f5ccedba740 1 Processor -- start 2024-09-21T00:25:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.594+0000 7f5ccedba740 1 -- start start 2024-09-21T00:25:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.595+0000 7f5ccedba740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f28589e660 0x55f285c85bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.595+0000 7f5ccedba740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858a6840 0x55f285c6f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.595+0000 7f5ccedba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858c7520 0x55f285c76d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.595+0000 7f5ccedba740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01b9d50 con 0x55f28589e660 2024-09-21T00:25:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.595+0000 7f5ccedba740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01b7620 con 0x55f2858a6840 2024-09-21T00:25:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.595+0000 7f5ccedba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01b5470 con 0x55f2858c7520 2024-09-21T00:25:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.595+0000 7f5cc5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f28589e660 0x55f285c85bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.595+0000 7f5cc4c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858a6840 0x55f285c6f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.595+0000 7f5cc6000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858c7520 0x55f285c76d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.595+0000 7f5cc4c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858a6840 0x55f285c6f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45236/0 (socket says 172.21.15.27:45236) 2024-09-21T00:25:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.595+0000 7f5cc4c00640 1 -- 172.21.15.27:0/1060123544 learned_addr learned my addr 172.21.15.27:0/1060123544 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.596+0000 7f5cc5600640 1 -- 172.21.15.27:0/1060123544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858c7520 msgr2=0x55f285c76d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:02.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.596+0000 7f5cc5600640 1 --2- 172.21.15.27:0/1060123544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858c7520 0x55f285c76d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.596+0000 7f5cc5600640 1 -- 172.21.15.27:0/1060123544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858a6840 msgr2=0x55f285c6f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:02.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.596+0000 7f5cc5600640 1 --2- 172.21.15.27:0/1060123544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858a6840 0x55f285c6f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.596+0000 7f5cc5600640 1 -- 172.21.15.27:0/1060123544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f285c704b0 con 0x55f28589e660 2024-09-21T00:25:02.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.596+0000 7f5cc6000640 1 --2- 172.21.15.27:0/1060123544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858c7520 0x55f285c76d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:02.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.596+0000 7f5cc5600640 1 --2- 172.21.15.27:0/1060123544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f28589e660 0x55f285c85bc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40aac00 tx=0x7f5cb40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784ef8e2b031de45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:02.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.596+0000 7f5cb1c00640 1 -- 172.21.15.27:0/1060123544 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4005180 con 0x55f28589e660 2024-09-21T00:25:02.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.596+0000 7f5cb1c00640 1 -- 172.21.15.27:0/1060123544 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb40b2470 con 0x55f28589e660 2024-09-21T00:25:02.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.596+0000 7f5cb1c00640 1 -- 172.21.15.27:0/1060123544 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40ad2e0 con 0x55f28589e660 2024-09-21T00:25:02.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.598+0000 7f5ccedba740 1 -- 172.21.15.27:0/1060123544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f28589e660 msgr2=0x55f285c85bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:02.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.598+0000 7f5ccedba740 1 --2- 172.21.15.27:0/1060123544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f28589e660 0x55f285c85bc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40aac00 tx=0x7f5cb40057c0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.598+0000 7f5ccedba740 1 -- 172.21.15.27:0/1060123544 shutdown_connections 2024-09-21T00:25:02.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.598+0000 7f5ccedba740 1 --2- 172.21.15.27:0/1060123544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858c7520 0x55f285c76d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.598+0000 7f5ccedba740 1 --2- 172.21.15.27:0/1060123544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858a6840 0x55f285c6f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.598+0000 7f5ccedba740 1 --2- 172.21.15.27:0/1060123544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f28589e660 0x55f285c85bc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.598+0000 7f5ccedba740 1 -- 172.21.15.27:0/1060123544 >> 172.21.15.27:0/1060123544 conn(0x55f2858bb910 msgr2=0x55f285c6f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:02.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.598+0000 7f5ccedba740 1 -- 172.21.15.27:0/1060123544 shutdown_connections 2024-09-21T00:25:02.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.598+0000 7f5ccedba740 1 -- 172.21.15.27:0/1060123544 wait complete. 2024-09-21T00:25:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.598+0000 7f5ccedba740 1 -- start start 2024-09-21T00:25:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.599+0000 7f5ccedba740 1 Processor -- start 2024-09-21T00:25:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.599+0000 7f5ccedba740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f28589e660 0x55f285c82440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.599+0000 7f5ccedba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858a6840 0x55f285c7e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.599+0000 7f5ccedba740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858c7520 0x55f285c7ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.599+0000 7f5ccedba740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01b5ae0 con 0x55f28589e660 2024-09-21T00:25:02.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.599+0000 7f5ccedba740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01b6f50 con 0x55f2858c7520 2024-09-21T00:25:02.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.599+0000 7f5ccedba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01b61b0 con 0x55f2858a6840 2024-09-21T00:25:02.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.599+0000 7f5ccedba740 10 client.0 _ll_register_callbacks cb 0x7f5cb400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:25:02.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.599+0000 7f5cc5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f28589e660 0x55f285c82440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.600+0000 7f5cc5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f28589e660 0x55f285c82440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57154/0 (socket says 172.21.15.27:57154) 2024-09-21T00:25:02.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.600+0000 7f5cc5600640 1 -- 172.21.15.27:0/446228423 learned_addr learned my addr 172.21.15.27:0/446228423 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:02.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.600+0000 7f5cc4c00640 1 --2- 172.21.15.27:0/446228423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858a6840 0x55f285c7e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.600+0000 7f5cc6000640 1 --2- 172.21.15.27:0/446228423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858c7520 0x55f285c7ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.600+0000 7f5cc5600640 1 -- 172.21.15.27:0/446228423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858a6840 msgr2=0x55f285c7e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:02.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.600+0000 7f5cc5600640 1 --2- 172.21.15.27:0/446228423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858a6840 0x55f285c7e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.600+0000 7f5cc5600640 1 -- 172.21.15.27:0/446228423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858c7520 msgr2=0x55f285c7ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:02.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.600+0000 7f5cc5600640 1 --2- 172.21.15.27:0/446228423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858c7520 0x55f285c7ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.600+0000 7f5cc5600640 1 -- 172.21.15.27:0/446228423 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f285c7f610 con 0x55f28589e660 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.600+0000 7f5cc5600640 1 --2- 172.21.15.27:0/446228423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f28589e660 0x55f285c82440 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40e8220 tx=0x7f5cb40ad1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875e9a100a310f06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.600+0000 7f5ca6a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.601+0000 7f5ca6a00640 1 -- 172.21.15.27:0/446228423 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40a9830 con 0x55f28589e660 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.601+0000 7f5ca6a00640 1 -- 172.21.15.27:0/446228423 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f5cb40b21c0 con 0x55f28589e660 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.601+0000 7f5ccedba740 1 -- 172.21.15.27:0/446228423 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f285be2c90 con 0x55f28589e660 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.601+0000 7f5ca6a00640 1 -- 172.21.15.27:0/446228423 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5cb40ad2e0 con 0x55f28589e660 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.601+0000 7f5ccedba740 10 client.15894 fetch_fsmap learned FSMap version 28 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.601+0000 7f5ccedba740 1 -- 172.21.15.27:0/446228423 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f285cf7670 con 0x55f28589e660 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.601+0000 7f5ca6a00640 1 -- 172.21.15.27:0/446228423 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f285be2c90 con 0x55f28589e660 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.601+0000 7f5ca6a00640 1 -- 172.21.15.27:0/446228423 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5cb40b37e0 con 0x55f28589e660 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.602+0000 7f5ccedba740 10 client.15894 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.602+0000 7f5ccedba740 10 client.15894 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.602+0000 7f5ccedba740 20 client.15894 populate_metadata read hostname 'smithi027' 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.602+0000 7f5ccedba740 1 --2- 172.21.15.27:0/446228423 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f28585b250 0x55f285cf9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.602+0000 7f5ccedba740 4 client.15894 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "15ed622a", "release": true, "await": true}] 2024-09-21T00:25:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.602+0000 7f5ccedba740 1 -- 172.21.15.27:0/446228423 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "quiesce db", "set_id": "15ed622a", "release": true, "await": true}) -- 0x55f285be00c0 con 0x55f28585b250 2024-09-21T00:25:02.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.602+0000 7f5cc4c00640 1 --2- 172.21.15.27:0/446228423 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f28585b250 0x55f285cf9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:02.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.603+0000 7f5cc4c00640 1 --2- 172.21.15.27:0/446228423 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f28585b250 0x55f285cf9c70 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2c453ffe3bfb0af server_cookie=66b9275ff0ad335d in_seq=0 out_seq=0 2024-09-21T00:25:02.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.603+0000 7f5ca6a00640 10 client.15894 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.823+0000 7f5ca6a00640 1 -- 172.21.15.27:0/446228423 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f285be00c0 con 0x55f28585b250 2024-09-21T00:25:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.823+0000 7f5ca6a00640 10 client.15894 handle_command_reply: tid=0 2024-09-21T00:25:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.824+0000 7f5ccedba740 1 client.15894 shutdown 2024-09-21T00:25:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.824+0000 7f5ccedba740 1 -- 172.21.15.27:0/446228423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f28589e660 msgr2=0x55f285c82440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.824+0000 7f5ccedba740 1 --2- 172.21.15.27:0/446228423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f28589e660 0x55f285c82440 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40e8220 tx=0x7f5cb40ad1c0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.824+0000 7f5ccedba740 1 -- 172.21.15.27:0/446228423 shutdown_connections 2024-09-21T00:25:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.824+0000 7f5ccedba740 1 --2- 172.21.15.27:0/446228423 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55f28585b250 0x55f285cf9c70 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.824+0000 7f5ccedba740 1 --2- 172.21.15.27:0/446228423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f2858c7520 0x55f285c7ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.824+0000 7f5ccedba740 1 --2- 172.21.15.27:0/446228423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2858a6840 0x55f285c7e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.824+0000 7f5ccedba740 1 --2- 172.21.15.27:0/446228423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f28589e660 0x55f285c82440 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.824+0000 7f5ccedba740 1 -- 172.21.15.27:0/446228423 >> 172.21.15.27:0/446228423 conn(0x55f2858bb910 msgr2=0x55f285c78750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:02.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.824+0000 7f5ca6a00640 0 client.15894 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:02.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.824+0000 7f5ccedba740 1 -- 172.21.15.27:0/446228423 shutdown_connections 2024-09-21T00:25:02.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.824+0000 7f5ccedba740 1 -- 172.21.15.27:0/446228423 wait complete. 2024-09-21T00:25:02.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.825+0000 7f5ccedba740 20 client.15894 trim_cache size 0 max 16384 2024-09-21T00:25:02.827 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:25:02.827 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 28, 2024-09-21T00:25:02.827 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24761, 2024-09-21T00:25:02.827 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 16, 2024-09-21T00:25:02.827 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: "15ed622a": { 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 16, 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 147.8, 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:25:02.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:25:02.829 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:25:02.829 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:25:02.829 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:25:02.829 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:25:02.829 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:25:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.826+0000 7f5cc8000640 1 -- 172.21.15.27:0/329599540 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5ca80b6700 msgr2=0x7f5ca80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.826+0000 7f5cc8000640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5ca80b6700 0x7f5ca80b8c10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5cb801cb20 tx=0x7f5cb8013040 comp rx=0 tx=0).stop 2024-09-21T00:25:02.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.826+0000 7f5cc8000640 1 -- 172.21.15.27:0/329599540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc0077940 msgr2=0x7f5cc01a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:02.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.826+0000 7f5cc8000640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc0077940 0x7f5cc01a0ae0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400e9f0 tx=0x7f5cb400edf0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.826+0000 7f5cc8000640 1 -- 172.21.15.27:0/329599540 shutdown_connections 2024-09-21T00:25:02.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.826+0000 7f5cc8000640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5ca80b6700 0x7f5ca80b8c10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.826+0000 7f5cc8000640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cc0114750 0x7f5cc01a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.826+0000 7f5cc8000640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc0078290 0x7f5cc01a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.826+0000 7f5cc8000640 1 --2- 172.21.15.27:0/329599540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc0077940 0x7f5cc01a0ae0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:02.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.826+0000 7f5cc8000640 1 -- 172.21.15.27:0/329599540 >> 172.21.15.27:0/329599540 conn(0x7f5cc0101860 msgr2=0x7f5cc0116810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:02.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.826+0000 7f5cc8000640 1 -- 172.21.15.27:0/329599540 shutdown_connections 2024-09-21T00:25:02.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:25:02.826+0000 7f5cc8000640 1 -- 172.21.15.27:0/329599540 wait complete. 2024-09-21T00:25:03.900 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '15ed622a', total seconds elapsed: 32.4 2024-09-21T00:25:04.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:03 smithi027 ceph-mon[30745]: pgmap v487: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 256 KiB/s wr, 112 op/s 2024-09-21T00:25:04.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:03 smithi089 ceph-mon[34481]: pgmap v487: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 256 KiB/s wr, 112 op/s 2024-09-21T00:25:04.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:03 smithi098 ceph-mon[34725]: pgmap v487: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 256 KiB/s wr, 112 op/s 2024-09-21T00:25:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.721+0000 7fe3d7e00640 1 -- 172.21.15.27:0/399055583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d810bd90 msgr2=0x7fe3d81125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.721+0000 7fe3d7e00640 1 --2- 172.21.15.27:0/399055583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d810bd90 0x7fe3d81125c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009860 tx=0x7fe3d002a340 comp rx=0 tx=0).stop 2024-09-21T00:25:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.721+0000 7fe3d7e00640 1 -- 172.21.15.27:0/399055583 shutdown_connections 2024-09-21T00:25:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.721+0000 7fe3d7e00640 1 --2- 172.21.15.27:0/399055583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d810bd90 0x7fe3d81125c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.721+0000 7fe3d7e00640 1 --2- 172.21.15.27:0/399055583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3d810b440 0x7fe3d810b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.721+0000 7fe3d7e00640 1 --2- 172.21.15.27:0/399055583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3d8105b10 0x7fe3d8105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.721+0000 7fe3d7e00640 1 -- 172.21.15.27:0/399055583 >> 172.21.15.27:0/399055583 conn(0x7fe3d8101510 msgr2=0x7fe3d8103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.721+0000 7fe3d7e00640 1 -- 172.21.15.27:0/399055583 shutdown_connections 2024-09-21T00:25:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.721+0000 7fe3d7e00640 1 -- 172.21.15.27:0/399055583 wait complete. 2024-09-21T00:25:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.722+0000 7fe3d7e00640 1 Processor -- start 2024-09-21T00:25:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.722+0000 7fe3d7e00640 1 -- start start 2024-09-21T00:25:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.723+0000 7fe3d7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3d8105b10 0x7fe3d819c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.723+0000 7fe3d7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3d810b440 0x7fe3d819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.723+0000 7fe3d7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d810bd90 0x7fe3d81a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.723+0000 7fe3d7e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d8115a50 con 0x7fe3d810b440 2024-09-21T00:25:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.723+0000 7fe3d7e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81158d0 con 0x7fe3d8105b10 2024-09-21T00:25:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.723+0000 7fe3d7e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d8115bd0 con 0x7fe3d810bd90 2024-09-21T00:25:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.723+0000 7fe3d6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3d8105b10 0x7fe3d819c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.723+0000 7fe3d7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d810bd90 0x7fe3d81a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.723+0000 7fe3ce000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3d810b440 0x7fe3d819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.723+0000 7fe3d6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3d8105b10 0x7fe3d819c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39850/0 (socket says 172.21.15.27:39850) 2024-09-21T00:25:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.723+0000 7fe3d6a00640 1 -- 172.21.15.27:0/1836594979 learned_addr learned my addr 172.21.15.27:0/1836594979 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.724+0000 7fe3d6a00640 1 -- 172.21.15.27:0/1836594979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d810bd90 msgr2=0x7fe3d81a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.724+0000 7fe3d6a00640 1 --2- 172.21.15.27:0/1836594979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d810bd90 0x7fe3d81a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.724+0000 7fe3d6a00640 1 -- 172.21.15.27:0/1836594979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3d810b440 msgr2=0x7fe3d819c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.724+0000 7fe3d6a00640 1 --2- 172.21.15.27:0/1836594979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3d810b440 0x7fe3d819c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.724+0000 7fe3d6a00640 1 -- 172.21.15.27:0/1836594979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d81a2300 con 0x7fe3d8105b10 2024-09-21T00:25:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.724+0000 7fe3d6a00640 1 --2- 172.21.15.27:0/1836594979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3d8105b10 0x7fe3d819c3d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800ebc0 tx=0x7fe3c800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.724+0000 7fe3cfe00640 1 -- 172.21.15.27:0/1836594979 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8010630 con 0x7fe3d8105b10 2024-09-21T00:25:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.724+0000 7fe3cfe00640 1 -- 172.21.15.27:0/1836594979 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c8010c80 con 0x7fe3d8105b10 2024-09-21T00:25:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.724+0000 7fe3d7e00640 1 -- 172.21.15.27:0/1836594979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d81a25f0 con 0x7fe3d8105b10 2024-09-21T00:25:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.724+0000 7fe3cfe00640 1 -- 172.21.15.27:0/1836594979 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8018b60 con 0x7fe3d8105b10 2024-09-21T00:25:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.724+0000 7fe3d7e00640 1 -- 172.21.15.27:0/1836594979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d81a8d80 con 0x7fe3d8105b10 2024-09-21T00:25:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.727+0000 7fe3e4a48740 1 Processor -- start 2024-09-21T00:25:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.727+0000 7fe3e4a48740 1 -- start start 2024-09-21T00:25:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.728+0000 7fe3e4a48740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fabc21b0 0x5588faf66e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.728+0000 7fe3e4a48740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 0x5588faf67350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.728+0000 7fe3e4a48740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588faba14d0 0x5588faf67890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.728+0000 7fe3e4a48740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5588faedc6b0 con 0x5588fab992f0 2024-09-21T00:25:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.728+0000 7fe3e4a48740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5588faecbae0 con 0x5588faba14d0 2024-09-21T00:25:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.728+0000 7fe3e4a48740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5588faecf8f0 con 0x5588fabc21b0 2024-09-21T00:25:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.728+0000 7fe3cfe00640 1 -- 172.21.15.27:0/1836594979 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe3c8018d00 con 0x7fe3d8105b10 2024-09-21T00:25:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.728+0000 7fe3ce000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588faba14d0 0x5588faf67890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.728+0000 7fe3ce000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588faba14d0 0x5588faf67890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39864/0 (socket says 172.21.15.27:39864) 2024-09-21T00:25:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.728+0000 7fe3ce000640 1 -- 172.21.15.27:0/337555205 learned_addr learned my addr 172.21.15.27:0/337555205 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.728+0000 7fe3d7400640 1 --2- 172.21.15.27:0/337555205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fabc21b0 0x5588faf66e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.729+0000 7fe3d6a00640 1 --2- 172.21.15.27:0/337555205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 0x5588faf67350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.729+0000 7fe3ce000640 1 -- 172.21.15.27:0/337555205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fabc21b0 msgr2=0x5588faf66e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.729+0000 7fe3ce000640 1 --2- 172.21.15.27:0/337555205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fabc21b0 0x5588faf66e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.729+0000 7fe3ce000640 1 -- 172.21.15.27:0/337555205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 msgr2=0x5588faf67350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.729+0000 7fe3ce000640 1 --2- 172.21.15.27:0/337555205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 0x5588faf67350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.729+0000 7fe3ce000640 1 -- 172.21.15.27:0/337555205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588fae98780 con 0x5588faba14d0 2024-09-21T00:25:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.729+0000 7fe3d7400640 1 --2- 172.21.15.27:0/337555205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fabc21b0 0x5588faf66e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.729+0000 7fe3ce000640 1 --2- 172.21.15.27:0/337555205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588faba14d0 0x5588faf67890 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009c60 tx=0x7fe3c0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9041aea24f5c01fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.729+0000 7fe3c7e00640 1 -- 172.21.15.27:0/337555205 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0004220 con 0x5588faba14d0 2024-09-21T00:25:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.729+0000 7fe3c7e00640 1 -- 172.21.15.27:0/337555205 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c0009e10 con 0x5588faba14d0 2024-09-21T00:25:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.729+0000 7fe3c7e00640 1 -- 172.21.15.27:0/337555205 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0019770 con 0x5588faba14d0 2024-09-21T00:25:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.730+0000 7fe3cfe00640 1 --2- 172.21.15.27:0/1836594979 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe3bc0b66f0 0x7fe3bc0b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.730+0000 7fe3cfe00640 1 -- 172.21.15.27:0/1836594979 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fe3c80e8080 con 0x7fe3d8105b10 2024-09-21T00:25:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.730+0000 7fe3e4a48740 1 -- 172.21.15.27:0/337555205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588faba14d0 msgr2=0x5588faf67890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.730+0000 7fe3d7400640 1 --2- 172.21.15.27:0/1836594979 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe3bc0b66f0 0x7fe3bc0b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.730+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/337555205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588faba14d0 0x5588faf67890 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009c60 tx=0x7fe3c0011810 comp rx=0 tx=0).stop 2024-09-21T00:25:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.731+0000 7fe3e4a48740 1 -- 172.21.15.27:0/337555205 shutdown_connections 2024-09-21T00:25:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.731+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/337555205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588faba14d0 0x5588faf67890 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.731+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/337555205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 0x5588faf67350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.731+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/337555205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fabc21b0 0x5588faf66e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.731+0000 7fe3e4a48740 1 -- 172.21.15.27:0/337555205 >> 172.21.15.27:0/337555205 conn(0x5588fabb6300 msgr2=0x5588faf64d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.731+0000 7fe3e4a48740 1 -- 172.21.15.27:0/337555205 shutdown_connections 2024-09-21T00:25:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.731+0000 7fe3e4a48740 1 -- 172.21.15.27:0/337555205 wait complete. 2024-09-21T00:25:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.731+0000 7fe3d7400640 1 --2- 172.21.15.27:0/1836594979 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe3bc0b66f0 0x7fe3bc0b8c00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00303f0 tx=0x7fe3d002c040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.732+0000 7fe3e4a48740 1 -- start start 2024-09-21T00:25:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.732+0000 7fe3e4a48740 1 Processor -- start 2024-09-21T00:25:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.733+0000 7fe3e4a48740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 0x5588faf72420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.733+0000 7fe3e4a48740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588faba14d0 0x5588faf72960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.733+0000 7fe3d6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 0x5588faf72420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.733+0000 7fe3d6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 0x5588faf72420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54968/0 (socket says 172.21.15.27:54968) 2024-09-21T00:25:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.734+0000 7fe3d6a00640 1 -- 172.21.15.27:0/4143125901 learned_addr learned my addr 172.21.15.27:0/4143125901 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.734+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/4143125901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fabc21b0 0x5588faf72ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.734+0000 7fe3e4a48740 1 -- 172.21.15.27:0/4143125901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5588fae03ab0 con 0x5588fab992f0 2024-09-21T00:25:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.734+0000 7fe3e4a48740 1 -- 172.21.15.27:0/4143125901 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5588fae72570 con 0x5588faba14d0 2024-09-21T00:25:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.734+0000 7fe3e4a48740 1 -- 172.21.15.27:0/4143125901 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5588faf75590 con 0x5588fabc21b0 2024-09-21T00:25:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.734+0000 7fe3ce000640 1 --2- 172.21.15.27:0/4143125901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588faba14d0 0x5588faf72960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.734+0000 7fe3d7400640 1 --2- 172.21.15.27:0/4143125901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fabc21b0 0x5588faf72ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.734+0000 7fe3e4a48740 10 client.0 _ll_register_callbacks cb 0x5588fae81100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:25:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.734+0000 7fe3ce000640 1 -- 172.21.15.27:0/4143125901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fabc21b0 msgr2=0x5588faf72ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.734+0000 7fe3ce000640 1 --2- 172.21.15.27:0/4143125901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fabc21b0 0x5588faf72ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.734+0000 7fe3ce000640 1 -- 172.21.15.27:0/4143125901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 msgr2=0x5588faf72420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.734+0000 7fe3ce000640 1 --2- 172.21.15.27:0/4143125901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 0x5588faf72420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.734+0000 7fe3ce000640 1 -- 172.21.15.27:0/4143125901 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588fac2d790 con 0x5588faba14d0 2024-09-21T00:25:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.735+0000 7fe3ce000640 1 --2- 172.21.15.27:0/4143125901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588faba14d0 0x5588faf72960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009530 tx=0x7fe3c0003d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=183e589ee95f973 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.735+0000 7fe3c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:25:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.737+0000 7fe3c4c00640 1 -- 172.21.15.27:0/4143125901 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0027070 con 0x5588faba14d0 2024-09-21T00:25:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.737+0000 7fe3c4c00640 1 -- 172.21.15.27:0/4143125901 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fe3c0003a00 con 0x5588faba14d0 2024-09-21T00:25:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.737+0000 7fe3e4a48740 1 -- 172.21.15.27:0/4143125901 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c0009e10 con 0x5588faba14d0 2024-09-21T00:25:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.738+0000 7fe3c4c00640 1 -- 172.21.15.27:0/4143125901 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3c0033050 con 0x5588faba14d0 2024-09-21T00:25:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.738+0000 7fe3e4a48740 10 client.25952 fetch_fsmap learned FSMap version 28 2024-09-21T00:25:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.738+0000 7fe3e4a48740 1 -- 172.21.15.27:0/4143125901 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588faca6620 con 0x5588faba14d0 2024-09-21T00:25:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.738+0000 7fe3c4c00640 1 -- 172.21.15.27:0/4143125901 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c0009e10 con 0x5588faba14d0 2024-09-21T00:25:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.738+0000 7fe3c4c00640 1 -- 172.21.15.27:0/4143125901 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe3c0004bc0 con 0x5588faba14d0 2024-09-21T00:25:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.739+0000 7fe3e4a48740 10 client.25952 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:25:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.739+0000 7fe3e4a48740 10 client.25952 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:25:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.739+0000 7fe3e4a48740 20 client.25952 populate_metadata read hostname 'smithi027' 2024-09-21T00:25:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.739+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/4143125901 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5588fab80d80 0x5588faf84450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.739+0000 7fe3e4a48740 4 client.25952 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:25:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.739+0000 7fe3e4a48740 1 -- 172.21.15.27:0/4143125901 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588faede600 con 0x5588fab80d80 2024-09-21T00:25:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.739+0000 7fe3d6a00640 1 --2- 172.21.15.27:0/4143125901 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5588fab80d80 0x5588faf84450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.740+0000 7fe3d6a00640 1 --2- 172.21.15.27:0/4143125901 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5588fab80d80 0x5588faf84450 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed0fa0000568054d server_cookie=1e70557548343557 in_seq=0 out_seq=0 2024-09-21T00:25:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.740+0000 7fe3c4c00640 10 client.25952 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.741+0000 7fe3c4c00640 1 -- 172.21.15.27:0/4143125901 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5588faede600 con 0x5588fab80d80 2024-09-21T00:25:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.741+0000 7fe3c4c00640 10 client.25952 handle_command_reply: tid=0 2024-09-21T00:25:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.741+0000 7fe3e4a48740 1 client.25952 shutdown 2024-09-21T00:25:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.742+0000 7fe3e4a48740 1 -- 172.21.15.27:0/4143125901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588faba14d0 msgr2=0x5588faf72960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.742+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/4143125901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588faba14d0 0x5588faf72960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009530 tx=0x7fe3c0003d00 comp rx=0 tx=0).stop 2024-09-21T00:25:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.742+0000 7fe3e4a48740 1 -- 172.21.15.27:0/4143125901 shutdown_connections 2024-09-21T00:25:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.742+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/4143125901 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5588fab80d80 0x5588faf84450 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.742+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/4143125901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fabc21b0 0x5588faf72ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.742+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/4143125901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588faba14d0 0x5588faf72960 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.742+0000 7fe3c4c00640 0 client.25952 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.742+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/4143125901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 0x5588faf72420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.742+0000 7fe3e4a48740 1 -- 172.21.15.27:0/4143125901 >> 172.21.15.27:0/4143125901 conn(0x5588fabb6300 msgr2=0x5588faf67e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.742+0000 7fe3e4a48740 1 -- 172.21.15.27:0/4143125901 shutdown_connections 2024-09-21T00:25:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.742+0000 7fe3e4a48740 1 -- 172.21.15.27:0/4143125901 wait complete. 2024-09-21T00:25:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.742+0000 7fe3e4a48740 20 client.25952 trim_cache size 0 max 16384 2024-09-21T00:25:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.759+0000 7fe3e4a48740 1 Processor -- start 2024-09-21T00:25:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.759+0000 7fe3e4a48740 1 -- start start 2024-09-21T00:25:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.759+0000 7fe3e4a48740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fab992f0 0x5588faf74050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3e4a48740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588faba14d0 0x5588faf6f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3e4a48740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588fabc21b0 0x5588faf6fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3e4a48740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b5740 con 0x5588faba14d0 2024-09-21T00:25:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3e4a48740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b3010 con 0x5588fabc21b0 2024-09-21T00:25:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3e4a48740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b0e60 con 0x5588fab992f0 2024-09-21T00:25:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3d7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588fabc21b0 0x5588faf6fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3ce000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588faba14d0 0x5588faf6f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3d7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588fabc21b0 0x5588faf6fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39882/0 (socket says 172.21.15.27:39882) 2024-09-21T00:25:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3d7400640 1 -- 172.21.15.27:0/638384725 learned_addr learned my addr 172.21.15.27:0/638384725 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3d6a00640 1 --2- 172.21.15.27:0/638384725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fab992f0 0x5588faf74050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3d7400640 1 -- 172.21.15.27:0/638384725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fab992f0 msgr2=0x5588faf74050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3d7400640 1 --2- 172.21.15.27:0/638384725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fab992f0 0x5588faf74050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3d7400640 1 -- 172.21.15.27:0/638384725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588faba14d0 msgr2=0x5588faf6f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3d7400640 1 --2- 172.21.15.27:0/638384725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588faba14d0 0x5588faf6f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3d7400640 1 -- 172.21.15.27:0/638384725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588faf6df50 con 0x5588fabc21b0 2024-09-21T00:25:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.760+0000 7fe3ce000640 1 --2- 172.21.15.27:0/638384725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588faba14d0 0x5588faf6f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.761+0000 7fe3d7400640 1 --2- 172.21.15.27:0/638384725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588fabc21b0 0x5588faf6fb30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00057c0 tx=0x7fe3d003d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a3b31ed5fa953b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.761+0000 7fe3c7400640 1 -- 172.21.15.27:0/638384725 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0037040 con 0x5588fabc21b0 2024-09-21T00:25:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.761+0000 7fe3c7400640 1 -- 172.21.15.27:0/638384725 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3d0043030 con 0x5588fabc21b0 2024-09-21T00:25:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.761+0000 7fe3c7400640 1 -- 172.21.15.27:0/638384725 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d003d870 con 0x5588fabc21b0 2024-09-21T00:25:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.762+0000 7fe3e4a48740 1 -- 172.21.15.27:0/638384725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588fabc21b0 msgr2=0x5588faf6fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.762+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/638384725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588fabc21b0 0x5588faf6fb30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00057c0 tx=0x7fe3d003d000 comp rx=0 tx=0).stop 2024-09-21T00:25:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.762+0000 7fe3e4a48740 1 -- 172.21.15.27:0/638384725 shutdown_connections 2024-09-21T00:25:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.762+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/638384725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588fabc21b0 0x5588faf6fb30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.762+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/638384725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588faba14d0 0x5588faf6f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.762+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/638384725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588fab992f0 0x5588faf74050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.762+0000 7fe3e4a48740 1 -- 172.21.15.27:0/638384725 >> 172.21.15.27:0/638384725 conn(0x5588fabb6300 msgr2=0x5588faf73940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.762+0000 7fe3e4a48740 1 -- 172.21.15.27:0/638384725 shutdown_connections 2024-09-21T00:25:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.762+0000 7fe3e4a48740 1 -- 172.21.15.27:0/638384725 wait complete. 2024-09-21T00:25:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.762+0000 7fe3e4a48740 1 -- start start 2024-09-21T00:25:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.763+0000 7fe3e4a48740 1 Processor -- start 2024-09-21T00:25:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.763+0000 7fe3e4a48740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 0x5588faf78370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.763+0000 7fe3e4a48740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588faba14d0 0x5588faf78950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.763+0000 7fe3e4a48740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588fabc21b0 0x5588fafe6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.763+0000 7fe3e4a48740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b14d0 con 0x5588fab992f0 2024-09-21T00:25:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.763+0000 7fe3e4a48740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b2940 con 0x5588fabc21b0 2024-09-21T00:25:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.763+0000 7fe3e4a48740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b1ba0 con 0x5588faba14d0 2024-09-21T00:25:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.763+0000 7fe3e4a48740 10 client.0 _ll_register_callbacks cb 0x7fe3c80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:25:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.763+0000 7fe3d6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 0x5588faf78370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.763+0000 7fe3d7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588fabc21b0 0x5588fafe6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.764+0000 7fe3d7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588fabc21b0 0x5588fafe6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39892/0 (socket says 172.21.15.27:39892) 2024-09-21T00:25:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.764+0000 7fe3d7400640 1 -- 172.21.15.27:0/35873693 learned_addr learned my addr 172.21.15.27:0/35873693 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.764+0000 7fe3ce000640 1 --2- 172.21.15.27:0/35873693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588faba14d0 0x5588faf78950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.764+0000 7fe3ce000640 1 -- 172.21.15.27:0/35873693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588fabc21b0 msgr2=0x5588fafe6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.764+0000 7fe3ce000640 1 --2- 172.21.15.27:0/35873693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588fabc21b0 0x5588fafe6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.764+0000 7fe3ce000640 1 -- 172.21.15.27:0/35873693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 msgr2=0x5588faf78370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.764+0000 7fe3ce000640 1 --2- 172.21.15.27:0/35873693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 0x5588faf78370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.764+0000 7fe3ce000640 1 -- 172.21.15.27:0/35873693 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588fafe74d0 con 0x5588faba14d0 2024-09-21T00:25:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.764+0000 7fe3d7400640 1 --2- 172.21.15.27:0/35873693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588fabc21b0 0x5588fafe6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.764+0000 7fe3d6a00640 1 --2- 172.21.15.27:0/35873693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 0x5588faf78370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.764+0000 7fe3ce000640 1 --2- 172.21.15.27:0/35873693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588faba14d0 0x5588faf78950 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0000c00 tx=0x7fe3c0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5baee849016de4be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.764+0000 7fe3c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:25:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.765+0000 7fe3c4c00640 1 -- 172.21.15.27:0/35873693 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0027040 con 0x5588faba14d0 2024-09-21T00:25:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.765+0000 7fe3c4c00640 1 -- 172.21.15.27:0/35873693 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fe3c0019d30 con 0x5588faba14d0 2024-09-21T00:25:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.765+0000 7fe3e4a48740 1 -- 172.21.15.27:0/35873693 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d0043030 con 0x5588faba14d0 2024-09-21T00:25:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.765+0000 7fe3c4c00640 1 -- 172.21.15.27:0/35873693 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3c0003d70 con 0x5588faba14d0 2024-09-21T00:25:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.765+0000 7fe3e4a48740 10 client.25927 fetch_fsmap learned FSMap version 28 2024-09-21T00:25:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.765+0000 7fe3e4a48740 1 -- 172.21.15.27:0/35873693 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588fafee8c0 con 0x5588faba14d0 2024-09-21T00:25:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.765+0000 7fe3c4c00640 1 -- 172.21.15.27:0/35873693 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3d0043030 con 0x5588faba14d0 2024-09-21T00:25:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.765+0000 7fe3c4c00640 1 -- 172.21.15.27:0/35873693 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe3c0005560 con 0x5588faba14d0 2024-09-21T00:25:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.766+0000 7fe3e4a48740 10 client.25927 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:25:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.766+0000 7fe3e4a48740 10 client.25927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:25:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.766+0000 7fe3e4a48740 20 client.25927 populate_metadata read hostname 'smithi027' 2024-09-21T00:25:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.766+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/35873693 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5588fab561c0 0x5588faff10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.766+0000 7fe3e4a48740 4 client.25927 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:25:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.766+0000 7fe3e4a48740 1 -- 172.21.15.27:0/35873693 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588faeda0b0 con 0x5588fab561c0 2024-09-21T00:25:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.766+0000 7fe3d6a00640 1 --2- 172.21.15.27:0/35873693 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5588fab561c0 0x5588faff10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.766+0000 7fe3d6a00640 1 --2- 172.21.15.27:0/35873693 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5588fab561c0 0x5588faff10d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b946a2d4b5ea42 server_cookie=f4fda6cc1db06867 in_seq=0 out_seq=0 2024-09-21T00:25:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:05.766+0000 7fe3c4c00640 10 client.25927 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:06.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:05 smithi089 ceph-mon[34481]: pgmap v488: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 250 KiB/s wr, 86 op/s 2024-09-21T00:25:06.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:05 smithi098 ceph-mon[34725]: pgmap v488: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 250 KiB/s wr, 86 op/s 2024-09-21T00:25:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:25:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:25:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:25:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:25:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.226+0000 7fe3c4c00640 1 -- 172.21.15.27:0/35873693 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588faeda0b0 con 0x5588fab561c0 2024-09-21T00:25:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.226+0000 7fe3c4c00640 10 client.25927 handle_command_reply: tid=0 2024-09-21T00:25:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.236+0000 7fe3e4a48740 1 client.25927 shutdown 2024-09-21T00:25:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.237+0000 7fe3e4a48740 1 -- 172.21.15.27:0/35873693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588faba14d0 msgr2=0x5588faf78950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.237+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/35873693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588faba14d0 0x5588faf78950 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0000c00 tx=0x7fe3c0009260 comp rx=0 tx=0).stop 2024-09-21T00:25:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.237+0000 7fe3e4a48740 1 -- 172.21.15.27:0/35873693 shutdown_connections 2024-09-21T00:25:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.237+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/35873693 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5588fab561c0 0x5588faff10d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.237+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/35873693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5588fabc21b0 0x5588fafe6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.237+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/35873693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588faba14d0 0x5588faf78950 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.237+0000 7fe3e4a48740 1 --2- 172.21.15.27:0/35873693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588fab992f0 0x5588faf78370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.237+0000 7fe3e4a48740 1 -- 172.21.15.27:0/35873693 >> 172.21.15.27:0/35873693 conn(0x5588fabb6300 msgr2=0x5588faf67e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.237+0000 7fe3c4c00640 0 client.25927 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.238+0000 7fe3e4a48740 1 -- 172.21.15.27:0/35873693 shutdown_connections 2024-09-21T00:25:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.238+0000 7fe3e4a48740 1 -- 172.21.15.27:0/35873693 wait complete. 2024-09-21T00:25:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.238+0000 7fe3e4a48740 20 client.25927 trim_cache size 0 max 16384 2024-09-21T00:25:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.262+0000 7fe3d7e00640 1 -- 172.21.15.27:0/1836594979 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe3bc0b66f0 msgr2=0x7fe3bc0b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.262+0000 7fe3d7e00640 1 --2- 172.21.15.27:0/1836594979 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe3bc0b66f0 0x7fe3bc0b8c00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00303f0 tx=0x7fe3d002c040 comp rx=0 tx=0).stop 2024-09-21T00:25:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.262+0000 7fe3d7e00640 1 -- 172.21.15.27:0/1836594979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3d8105b10 msgr2=0x7fe3d819c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.262+0000 7fe3d7e00640 1 --2- 172.21.15.27:0/1836594979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3d8105b10 0x7fe3d819c3d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800ebc0 tx=0x7fe3c800c6a0 comp rx=0 tx=0).stop 2024-09-21T00:25:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.263+0000 7fe3d7e00640 1 -- 172.21.15.27:0/1836594979 shutdown_connections 2024-09-21T00:25:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.263+0000 7fe3d7e00640 1 --2- 172.21.15.27:0/1836594979 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe3bc0b66f0 0x7fe3bc0b8c00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.263+0000 7fe3d7e00640 1 --2- 172.21.15.27:0/1836594979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d810bd90 0x7fe3d81a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.263+0000 7fe3d7e00640 1 --2- 172.21.15.27:0/1836594979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3d810b440 0x7fe3d819c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.263+0000 7fe3d7e00640 1 --2- 172.21.15.27:0/1836594979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3d8105b10 0x7fe3d819c3d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.263+0000 7fe3d7e00640 1 -- 172.21.15.27:0/1836594979 >> 172.21.15.27:0/1836594979 conn(0x7fe3d8101510 msgr2=0x7fe3d810aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.271+0000 7fe3d7e00640 1 -- 172.21.15.27:0/1836594979 shutdown_connections 2024-09-21T00:25:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:06.271+0000 7fe3d7e00640 1 -- 172.21.15.27:0/1836594979 wait complete. 2024-09-21T00:25:06.325 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:05 smithi027 ceph-mon[30745]: pgmap v488: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 250 KiB/s wr, 86 op/s 2024-09-21T00:25:08.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:07 smithi089 ceph-mon[34481]: pgmap v489: 129 pgs: 129 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 337 KiB/s wr, 78 op/s 2024-09-21T00:25:08.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:07 smithi098 ceph-mon[34725]: pgmap v489: 129 pgs: 129 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 337 KiB/s wr, 78 op/s 2024-09-21T00:25:08.319 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:07 smithi027 ceph-mon[30745]: pgmap v489: 129 pgs: 129 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 337 KiB/s wr, 78 op/s 2024-09-21T00:25:08.326 INFO:tasks.workunit.client.0.smithi027.stdout: 46 237380 506 162449 142 74909 438 2024-09-21T00:25:10.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:09 smithi089 ceph-mon[34481]: pgmap v490: 129 pgs: 129 active+clean; 160 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 584 KiB/s wr, 140 op/s 2024-09-21T00:25:10.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:09 smithi098 ceph-mon[34725]: pgmap v490: 129 pgs: 129 active+clean; 160 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 584 KiB/s wr, 140 op/s 2024-09-21T00:25:10.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:09 smithi027 ceph-mon[30745]: pgmap v490: 129 pgs: 129 active+clean; 160 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 584 KiB/s wr, 140 op/s 2024-09-21T00:25:12.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:11 smithi089 ceph-mon[34481]: pgmap v491: 129 pgs: 129 active+clean; 165 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.0 MiB/s wr, 228 op/s 2024-09-21T00:25:12.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:11 smithi098 ceph-mon[34725]: pgmap v491: 129 pgs: 129 active+clean; 165 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.0 MiB/s wr, 228 op/s 2024-09-21T00:25:12.321 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:11 smithi027 ceph-mon[30745]: pgmap v491: 129 pgs: 129 active+clean; 165 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.0 MiB/s wr, 228 op/s 2024-09-21T00:25:14.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:13 smithi027 ceph-mon[30745]: pgmap v492: 129 pgs: 129 active+clean; 178 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 300 op/s 2024-09-21T00:25:14.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:13 smithi089 ceph-mon[34481]: pgmap v492: 129 pgs: 129 active+clean; 178 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 300 op/s 2024-09-21T00:25:14.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:13 smithi098 ceph-mon[34725]: pgmap v492: 129 pgs: 129 active+clean; 178 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 300 op/s 2024-09-21T00:25:16.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:15 smithi089 ceph-mon[34481]: pgmap v493: 129 pgs: 129 active+clean; 180 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 345 op/s 2024-09-21T00:25:16.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:15 smithi098 ceph-mon[34725]: pgmap v493: 129 pgs: 129 active+clean; 180 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 345 op/s 2024-09-21T00:25:16.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:15 smithi027 ceph-mon[30745]: pgmap v493: 129 pgs: 129 active+clean; 180 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 345 op/s 2024-09-21T00:25:18.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:17 smithi027 ceph-mon[30745]: pgmap v494: 129 pgs: 129 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 365 op/s 2024-09-21T00:25:18.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:17 smithi089 ceph-mon[34481]: pgmap v494: 129 pgs: 129 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 365 op/s 2024-09-21T00:25:18.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:17 smithi098 ceph-mon[34725]: pgmap v494: 129 pgs: 129 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 365 op/s 2024-09-21T00:25:18.323 INFO:tasks.workunit.client.0.smithi027.stdout: 69 360637 1115 235919 1165 97082 513 2024-09-21T00:25:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:19 smithi089 ceph-mon[34481]: pgmap v495: 129 pgs: 129 active+clean; 192 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 359 op/s 2024-09-21T00:25:20.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:19 smithi098 ceph-mon[34725]: pgmap v495: 129 pgs: 129 active+clean; 192 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 359 op/s 2024-09-21T00:25:20.331 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:19 smithi027 ceph-mon[30745]: pgmap v495: 129 pgs: 129 active+clean; 192 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 359 op/s 2024-09-21T00:25:22.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:22 smithi089 ceph-mon[34481]: pgmap v496: 129 pgs: 129 active+clean; 204 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 333 op/s 2024-09-21T00:25:22.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:22 smithi098 ceph-mon[34725]: pgmap v496: 129 pgs: 129 active+clean; 204 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 333 op/s 2024-09-21T00:25:22.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:22 smithi027 ceph-mon[30745]: pgmap v496: 129 pgs: 129 active+clean; 204 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 333 op/s 2024-09-21T00:25:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.932+0000 7f8557800640 1 -- 172.21.15.27:0/2342331031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8550111ca0 msgr2=0x7f855010a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.932+0000 7f8557800640 1 --2- 172.21.15.27:0/2342331031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8550111ca0 0x7f855010a3c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f853c009900 tx=0x7f853c02a3f0 comp rx=0 tx=0).stop 2024-09-21T00:25:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.934+0000 7f8557800640 1 -- 172.21.15.27:0/2342331031 shutdown_connections 2024-09-21T00:25:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.934+0000 7f8557800640 1 --2- 172.21.15.27:0/2342331031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f855011cba0 0x7f855011efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.934+0000 7f8557800640 1 --2- 172.21.15.27:0/2342331031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f855010a900 0x7f855010ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.934+0000 7f8557800640 1 --2- 172.21.15.27:0/2342331031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8550111ca0 0x7f855010a3c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.934+0000 7f8557800640 1 -- 172.21.15.27:0/2342331031 >> 172.21.15.27:0/2342331031 conn(0x7f855006e920 msgr2=0x7f855006ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.934+0000 7f8557800640 1 -- 172.21.15.27:0/2342331031 shutdown_connections 2024-09-21T00:25:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.934+0000 7f8557800640 1 -- 172.21.15.27:0/2342331031 wait complete. 2024-09-21T00:25:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.941+0000 7f8557800640 1 Processor -- start 2024-09-21T00:25:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.942+0000 7f8557800640 1 -- start start 2024-09-21T00:25:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.943+0000 7f8557800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f855010a900 0x7f85501a9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.943+0000 7f8557800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8550111ca0 0x7f85501a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.943+0000 7f8554e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f855010a900 0x7f85501a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.944+0000 7f8554e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f855010a900 0x7f85501a9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:51216/0 (socket says 172.21.15.27:51216) 2024-09-21T00:25:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.944+0000 7f8554e00640 1 -- 172.21.15.27:0/2264566215 learned_addr learned my addr 172.21.15.27:0/2264566215 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.944+0000 7f854fe00640 1 --2- 172.21.15.27:0/2264566215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8550111ca0 0x7f85501a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.947+0000 7f8557800640 1 --2- 172.21.15.27:0/2264566215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f855011cba0 0x7f85501ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.947+0000 7f8557800640 1 -- 172.21.15.27:0/2264566215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8550122630 con 0x7f855011cba0 2024-09-21T00:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.947+0000 7f8557800640 1 -- 172.21.15.27:0/2264566215 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f85501224b0 con 0x7f855010a900 2024-09-21T00:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.947+0000 7f8557800640 1 -- 172.21.15.27:0/2264566215 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f85501227b0 con 0x7f8550111ca0 2024-09-21T00:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.947+0000 7f8554e00640 1 -- 172.21.15.27:0/2264566215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8550111ca0 msgr2=0x7f85501a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.947+0000 7f8554e00640 1 --2- 172.21.15.27:0/2264566215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8550111ca0 0x7f85501a95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.947+0000 7f8554e00640 1 -- 172.21.15.27:0/2264566215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f855011cba0 msgr2=0x7f85501ae880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:25:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.947+0000 7f8554e00640 1 --2- 172.21.15.27:0/2264566215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f855011cba0 0x7f85501ae880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.947+0000 7f8554e00640 1 -- 172.21.15.27:0/2264566215 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85501aef80 con 0x7f855010a900 2024-09-21T00:25:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.948+0000 7f8554e00640 1 --2- 172.21.15.27:0/2264566215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f855010a900 0x7f85501a9050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f853c0057c0 tx=0x7f853c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.949+0000 7f854d600640 1 -- 172.21.15.27:0/2264566215 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c040070 con 0x7f855010a900 2024-09-21T00:25:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.949+0000 7f8557800640 1 -- 172.21.15.27:0/2264566215 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85501af210 con 0x7f855010a900 2024-09-21T00:25:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.949+0000 7f8557800640 1 -- 172.21.15.27:0/2264566215 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85501b5a00 con 0x7f855010a900 2024-09-21T00:25:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.957+0000 7f854d600640 1 -- 172.21.15.27:0/2264566215 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f853c0316d0 con 0x7f855010a900 2024-09-21T00:25:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.957+0000 7f854d600640 1 -- 172.21.15.27:0/2264566215 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c035020 con 0x7f855010a900 2024-09-21T00:25:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.968+0000 7f855e7df740 1 Processor -- start 2024-09-21T00:25:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.968+0000 7f855e7df740 1 -- start start 2024-09-21T00:25:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.971+0000 7f855e7df740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8931b0 0x55827fc37e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.971+0000 7f855e7df740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f86a2f0 0x55827fc38350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.971+0000 7f855e7df740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8724d0 0x55827fc38890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.972+0000 7f855e7df740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55827fbad6b0 con 0x55827f8931b0 2024-09-21T00:25:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.972+0000 7f855e7df740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55827fb9cae0 con 0x55827f8724d0 2024-09-21T00:25:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.972+0000 7f855e7df740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55827fba08f0 con 0x55827f86a2f0 2024-09-21T00:25:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.973+0000 7f854fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8724d0 0x55827fc38890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.973+0000 7f854fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8724d0 0x55827fc38890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53562/0 (socket says 172.21.15.27:53562) 2024-09-21T00:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.973+0000 7f854fe00640 1 -- 172.21.15.27:0/2790974560 learned_addr learned my addr 172.21.15.27:0/2790974560 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.973+0000 7f8555800640 1 --2- 172.21.15.27:0/2790974560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8931b0 0x55827fc37e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.974+0000 7f854d600640 1 -- 172.21.15.27:0/2264566215 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f853c043070 con 0x7f855010a900 2024-09-21T00:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.974+0000 7f8555800640 1 -- 172.21.15.27:0/2790974560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f86a2f0 msgr2=0x55827fc38350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.974+0000 7f8555800640 1 --2- 172.21.15.27:0/2790974560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f86a2f0 0x55827fc38350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.974+0000 7f8555800640 1 -- 172.21.15.27:0/2790974560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8724d0 msgr2=0x55827fc38890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.974+0000 7f8555800640 1 --2- 172.21.15.27:0/2790974560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8724d0 0x55827fc38890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.974+0000 7f8555800640 1 -- 172.21.15.27:0/2790974560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55827fb69780 con 0x55827f8931b0 2024-09-21T00:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.975+0000 7f8555800640 1 --2- 172.21.15.27:0/2790974560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8931b0 0x55827fc37e10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8540009980 tx=0x7f8540009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cf1c71b36cf86a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.976+0000 7f8546000640 1 -- 172.21.15.27:0/2790974560 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540004230 con 0x55827f8931b0 2024-09-21T00:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.977+0000 7f8546000640 1 -- 172.21.15.27:0/2790974560 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85400043d0 con 0x55827f8931b0 2024-09-21T00:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.977+0000 7f8546000640 1 -- 172.21.15.27:0/2790974560 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540010430 con 0x55827f8931b0 2024-09-21T00:25:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.978+0000 7f854d600640 1 --2- 172.21.15.27:0/2264566215 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f85340b6840 0x7f85340b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.980+0000 7f855e7df740 1 -- 172.21.15.27:0/2790974560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8931b0 msgr2=0x55827fc37e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.980+0000 7f855e7df740 1 --2- 172.21.15.27:0/2790974560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8931b0 0x55827fc37e10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8540009980 tx=0x7f8540009e40 comp rx=0 tx=0).stop 2024-09-21T00:25:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.984+0000 7f8554e00640 1 --2- 172.21.15.27:0/2264566215 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f85340b6840 0x7f85340b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.987+0000 7f855e7df740 1 -- 172.21.15.27:0/2790974560 shutdown_connections 2024-09-21T00:25:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.987+0000 7f855e7df740 1 --2- 172.21.15.27:0/2790974560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8724d0 0x55827fc38890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.988+0000 7f855e7df740 1 --2- 172.21.15.27:0/2790974560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f86a2f0 0x55827fc38350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.988+0000 7f855e7df740 1 --2- 172.21.15.27:0/2790974560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8931b0 0x55827fc37e10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.988+0000 7f855e7df740 1 -- 172.21.15.27:0/2790974560 >> 172.21.15.27:0/2790974560 conn(0x55827f887300 msgr2=0x55827fc35d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.989+0000 7f854d600640 1 -- 172.21.15.27:0/2264566215 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f853c108420 con 0x7f855010a900 2024-09-21T00:25:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.990+0000 7f8554e00640 1 --2- 172.21.15.27:0/2264566215 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f85340b6840 0x7f85340b8d50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55827fc36a90 tx=0x7f853c10ba90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.990+0000 7f855e7df740 1 -- 172.21.15.27:0/2790974560 shutdown_connections 2024-09-21T00:25:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.991+0000 7f855e7df740 1 -- 172.21.15.27:0/2790974560 wait complete. 2024-09-21T00:25:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.993+0000 7f855e7df740 1 -- start start 2024-09-21T00:25:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.993+0000 7f855e7df740 1 Processor -- start 2024-09-21T00:25:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.998+0000 7f855e7df740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f86a2f0 0x55827fc433f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.999+0000 7f8555800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f86a2f0 0x55827fc433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.999+0000 7f8555800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f86a2f0 0x55827fc433f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:43904/0 (socket says 172.21.15.27:43904) 2024-09-21T00:25:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.999+0000 7f8555800640 1 -- 172.21.15.27:0/2867497445 learned_addr learned my addr 172.21.15.27:0/2867497445 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:22.999+0000 7f855e7df740 1 --2- 172.21.15.27:0/2867497445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8724d0 0x55827fc43930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.001+0000 7f854fe00640 1 --2- 172.21.15.27:0/2867497445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8724d0 0x55827fc43930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.002+0000 7f855e7df740 1 --2- 172.21.15.27:0/2867497445 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8931b0 0x55827fc43e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.002+0000 7f855e7df740 1 -- 172.21.15.27:0/2867497445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55827fad4ab0 con 0x55827f8724d0 2024-09-21T00:25:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.002+0000 7f855e7df740 1 -- 172.21.15.27:0/2867497445 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55827fb43570 con 0x55827f8931b0 2024-09-21T00:25:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.002+0000 7f855e7df740 1 -- 172.21.15.27:0/2867497445 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55827fc575a0 con 0x55827f86a2f0 2024-09-21T00:25:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.004+0000 7f854fe00640 1 -- 172.21.15.27:0/2867497445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f86a2f0 msgr2=0x55827fc433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.004+0000 7f854fe00640 1 --2- 172.21.15.27:0/2867497445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f86a2f0 0x55827fc433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.004+0000 7f854fe00640 1 -- 172.21.15.27:0/2867497445 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8931b0 msgr2=0x55827fc43e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.004+0000 7f854fe00640 1 --2- 172.21.15.27:0/2867497445 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8931b0 0x55827fc43e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.004+0000 7f854fe00640 1 -- 172.21.15.27:0/2867497445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55827f8fe790 con 0x55827f8724d0 2024-09-21T00:25:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.004+0000 7f855e7df740 10 client.0 _ll_register_callbacks cb 0x55827fb52100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:25:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.005+0000 7f855e7df740 1 -- 172.21.15.27:0/2867497445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85400043d0 con 0x55827f8724d0 2024-09-21T00:25:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.005+0000 7f854fe00640 1 --2- 172.21.15.27:0/2867497445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8724d0 0x55827fc43930 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8548005f10 tx=0x7f8548004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcadc6d736ff33a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.005+0000 7f853aa00640 10 client.15921 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:25:23.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.006+0000 7f853aa00640 1 -- 172.21.15.27:0/2867497445 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548004d20 con 0x55827f8724d0 2024-09-21T00:25:23.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.006+0000 7f853aa00640 1 -- 172.21.15.27:0/2867497445 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f8548013a10 con 0x55827f8724d0 2024-09-21T00:25:23.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.007+0000 7f853aa00640 1 -- 172.21.15.27:0/2867497445 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8548013c70 con 0x55827f8724d0 2024-09-21T00:25:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.011+0000 7f855e7df740 10 client.15921 fetch_fsmap learned FSMap version 28 2024-09-21T00:25:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.011+0000 7f855e7df740 1 -- 172.21.15.27:0/2867497445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55827f977620 con 0x55827f8724d0 2024-09-21T00:25:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.013+0000 7f853aa00640 1 -- 172.21.15.27:0/2867497445 <== mon.0 v2:172.21.15.27:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f854801a020 con 0x55827f8724d0 2024-09-21T00:25:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.013+0000 7f855e7df740 10 client.15921 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:25:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.013+0000 7f855e7df740 10 client.15921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:25:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.013+0000 7f855e7df740 20 client.15921 populate_metadata read hostname 'smithi027' 2024-09-21T00:25:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.013+0000 7f855e7df740 1 --2- 172.21.15.27:0/2867497445 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827f851d80 0x55827fc66ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.013+0000 7f855e7df740 4 client.15921 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:25:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.013+0000 7f855e7df740 1 -- 172.21.15.27:0/2867497445 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55827fbaf600 con 0x55827f851d80 2024-09-21T00:25:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.014+0000 7f8555800640 1 --2- 172.21.15.27:0/2867497445 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827f851d80 0x55827fc66ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.040+0000 7f8555800640 1 --2- 172.21.15.27:0/2867497445 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827f851d80 0x55827fc66ac0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9361cb09a565ddf server_cookie=1ff7742bddfa7b29 in_seq=0 out_seq=0 2024-09-21T00:25:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.044+0000 7f853aa00640 10 client.15921 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.053+0000 7f853aa00640 1 -- 172.21.15.27:0/2867497445 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55827fbaf600 con 0x55827f851d80 2024-09-21T00:25:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.053+0000 7f853aa00640 10 client.15921 handle_command_reply: tid=0 2024-09-21T00:25:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.053+0000 7f855e7df740 1 client.15921 shutdown 2024-09-21T00:25:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.057+0000 7f855e7df740 1 -- 172.21.15.27:0/2867497445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8724d0 msgr2=0x55827fc43930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.057+0000 7f855e7df740 1 --2- 172.21.15.27:0/2867497445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8724d0 0x55827fc43930 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8548005f10 tx=0x7f8548004750 comp rx=0 tx=0).stop 2024-09-21T00:25:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.060+0000 7f855e7df740 1 -- 172.21.15.27:0/2867497445 shutdown_connections 2024-09-21T00:25:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.062+0000 7f855e7df740 1 --2- 172.21.15.27:0/2867497445 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827f851d80 0x55827fc66ac0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.062+0000 7f855e7df740 1 --2- 172.21.15.27:0/2867497445 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8931b0 0x55827fc43e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.062+0000 7f855e7df740 1 --2- 172.21.15.27:0/2867497445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8724d0 0x55827fc43930 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.062+0000 7f855e7df740 1 --2- 172.21.15.27:0/2867497445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f86a2f0 0x55827fc433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.062+0000 7f855e7df740 1 -- 172.21.15.27:0/2867497445 >> 172.21.15.27:0/2867497445 conn(0x55827f887300 msgr2=0x55827fc38e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.062+0000 7f853aa00640 0 client.15921 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.064+0000 7f855e7df740 1 -- 172.21.15.27:0/2867497445 shutdown_connections 2024-09-21T00:25:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.064+0000 7f855e7df740 1 -- 172.21.15.27:0/2867497445 wait complete. 2024-09-21T00:25:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.064+0000 7f855e7df740 20 client.15921 trim_cache size 0 max 16384 2024-09-21T00:25:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.235+0000 7f855e7df740 1 Processor -- start 2024-09-21T00:25:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.239+0000 7f855e7df740 1 -- start start 2024-09-21T00:25:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.239+0000 7f855e7df740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f86a2f0 0x55827fc34d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.241+0000 7f855e7df740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8724d0 0x55827fc60820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.241+0000 7f855e7df740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f8931b0 0x55827fc45020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.241+0000 7f8555800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f86a2f0 0x55827fc34d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.241+0000 7f8555800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f86a2f0 0x55827fc34d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53578/0 (socket says 172.21.15.27:53578) 2024-09-21T00:25:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.241+0000 7f8555800640 1 -- 172.21.15.27:0/668831485 learned_addr learned my addr 172.21.15.27:0/668831485 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.241+0000 7f855e7df740 1 -- 172.21.15.27:0/668831485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f85300127f0 con 0x55827f8724d0 2024-09-21T00:25:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.241+0000 7f855e7df740 1 -- 172.21.15.27:0/668831485 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f85300100c0 con 0x55827f86a2f0 2024-09-21T00:25:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.241+0000 7f855e7df740 1 -- 172.21.15.27:0/668831485 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f853000df10 con 0x55827f8931b0 2024-09-21T00:25:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.241+0000 7f8555800640 1 --2- 172.21.15.27:0/668831485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f8931b0 0x55827fc45020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.242+0000 7f8555800640 1 -- 172.21.15.27:0/668831485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f8931b0 msgr2=0x55827fc45020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.242+0000 7f8555800640 1 --2- 172.21.15.27:0/668831485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f8931b0 0x55827fc45020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.242+0000 7f8555800640 1 -- 172.21.15.27:0/668831485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8724d0 msgr2=0x55827fc60820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.242+0000 7f8555800640 1 --2- 172.21.15.27:0/668831485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8724d0 0x55827fc60820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.242+0000 7f8555800640 1 -- 172.21.15.27:0/668831485 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55827fc30000 con 0x55827f86a2f0 2024-09-21T00:25:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.242+0000 7f8555800640 1 --2- 172.21.15.27:0/668831485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f86a2f0 0x55827fc34d20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8540009980 tx=0x7f8540003b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce0dbd979800b824 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.246+0000 7f8545600640 1 -- 172.21.15.27:0/668831485 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854001d040 con 0x55827f86a2f0 2024-09-21T00:25:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.246+0000 7f8545600640 1 -- 172.21.15.27:0/668831485 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8540005250 con 0x55827f86a2f0 2024-09-21T00:25:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.246+0000 7f8545600640 1 -- 172.21.15.27:0/668831485 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540004580 con 0x55827f86a2f0 2024-09-21T00:25:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.250+0000 7f855e7df740 1 -- 172.21.15.27:0/668831485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f86a2f0 msgr2=0x55827fc34d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.250+0000 7f855e7df740 1 --2- 172.21.15.27:0/668831485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f86a2f0 0x55827fc34d20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8540009980 tx=0x7f8540003b60 comp rx=0 tx=0).stop 2024-09-21T00:25:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.257+0000 7f855e7df740 1 -- 172.21.15.27:0/668831485 shutdown_connections 2024-09-21T00:25:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.257+0000 7f855e7df740 1 --2- 172.21.15.27:0/668831485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f8931b0 0x55827fc45020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.257+0000 7f855e7df740 1 --2- 172.21.15.27:0/668831485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8724d0 0x55827fc60820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.257+0000 7f855e7df740 1 --2- 172.21.15.27:0/668831485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f86a2f0 0x55827fc34d20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.257+0000 7f855e7df740 1 -- 172.21.15.27:0/668831485 >> 172.21.15.27:0/668831485 conn(0x55827f887300 msgr2=0x55827fc44910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.263+0000 7f855e7df740 1 -- 172.21.15.27:0/668831485 shutdown_connections 2024-09-21T00:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.272+0000 7f855e7df740 1 -- 172.21.15.27:0/668831485 wait complete. 2024-09-21T00:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.273+0000 7f855e7df740 1 -- start start 2024-09-21T00:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.274+0000 7f855e7df740 1 Processor -- start 2024-09-21T00:25:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.276+0000 7f855e7df740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f86a2f0 0x55827fc5a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.276+0000 7f855e7df740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8724d0 0x55827fc5a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.277+0000 7f854fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8724d0 0x55827fc5a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.277+0000 7f854fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8724d0 0x55827fc5a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53580/0 (socket says 172.21.15.27:53580) 2024-09-21T00:25:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.277+0000 7f855e7df740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8931b0 0x55827fccf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.277+0000 7f854fe00640 1 -- 172.21.15.27:0/1969585321 learned_addr learned my addr 172.21.15.27:0/1969585321 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.277+0000 7f855e7df740 1 -- 172.21.15.27:0/1969585321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f853000e580 con 0x55827f8931b0 2024-09-21T00:25:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.277+0000 7f855e7df740 1 -- 172.21.15.27:0/1969585321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f853000f9f0 con 0x55827f8724d0 2024-09-21T00:25:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.277+0000 7f855e7df740 1 -- 172.21.15.27:0/1969585321 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f853000ec50 con 0x55827f86a2f0 2024-09-21T00:25:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.277+0000 7f855e7df740 10 client.0 _ll_register_callbacks cb 0x7f8548009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:25:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.278+0000 7f854fe00640 1 -- 172.21.15.27:0/1969585321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f86a2f0 msgr2=0x55827fc5a0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:25:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.278+0000 7f854fe00640 1 --2- 172.21.15.27:0/1969585321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f86a2f0 0x55827fc5a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.278+0000 7f854fe00640 1 -- 172.21.15.27:0/1969585321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8931b0 msgr2=0x55827fccf4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.278+0000 7f854fe00640 1 --2- 172.21.15.27:0/1969585321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8931b0 0x55827fccf4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.278+0000 7f854fe00640 1 -- 172.21.15.27:0/1969585321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55827fccfb00 con 0x55827f8724d0 2024-09-21T00:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.282+0000 7f853aa00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:25:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.283+0000 7f854fe00640 1 --2- 172.21.15.27:0/1969585321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8724d0 0x55827fc5a640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8548004790 tx=0x7f8548004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a97f615f2dcce7a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.286+0000 7f853aa00640 1 -- 172.21.15.27:0/1969585321 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854801c4c0 con 0x55827f8724d0 2024-09-21T00:25:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.287+0000 7f855e7df740 1 -- 172.21.15.27:0/1969585321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8540005250 con 0x55827f8724d0 2024-09-21T00:25:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.287+0000 7f853aa00640 1 -- 172.21.15.27:0/1969585321 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f8548002030 con 0x55827f8724d0 2024-09-21T00:25:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.289+0000 7f853aa00640 1 -- 172.21.15.27:0/1969585321 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f854801c4c0 con 0x55827f8724d0 2024-09-21T00:25:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.294+0000 7f855e7df740 10 client.25973 fetch_fsmap learned FSMap version 28 2024-09-21T00:25:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.294+0000 7f855e7df740 1 -- 172.21.15.27:0/1969585321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55827fcd0d70 con 0x55827f8724d0 2024-09-21T00:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.299+0000 7f853aa00640 1 -- 172.21.15.27:0/1969585321 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8540005250 con 0x55827f8724d0 2024-09-21T00:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.300+0000 7f853aa00640 1 -- 172.21.15.27:0/1969585321 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f854801a040 con 0x55827f8724d0 2024-09-21T00:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.300+0000 7f855e7df740 10 client.25973 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:25:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.300+0000 7f855e7df740 10 client.25973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:25:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.300+0000 7f855e7df740 20 client.25973 populate_metadata read hostname 'smithi027' 2024-09-21T00:25:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.300+0000 7f855e7df740 1 --2- 172.21.15.27:0/1969585321 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827f8271c0 0x55827fcd3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.300+0000 7f855e7df740 4 client.25973 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:25:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.300+0000 7f855e7df740 1 -- 172.21.15.27:0/1969585321 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55827fbab0b0 con 0x55827f8271c0 2024-09-21T00:25:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.301+0000 7f8555800640 1 --2- 172.21.15.27:0/1969585321 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827f8271c0 0x55827fcd3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.303+0000 7f8555800640 1 --2- 172.21.15.27:0/1969585321 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827f8271c0 0x55827fcd3560 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4040fe5e54f4e9ad server_cookie=6ec655c9833ca29e in_seq=0 out_seq=0 2024-09-21T00:25:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:23.303+0000 7f853aa00640 10 client.25973 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:24.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:24 smithi027 ceph-mon[30745]: pgmap v497: 129 pgs: 129 active+clean; 218 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 300 op/s 2024-09-21T00:25:24.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:24 smithi089 ceph-mon[34481]: pgmap v497: 129 pgs: 129 active+clean; 218 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 300 op/s 2024-09-21T00:25:24.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:24 smithi098 ceph-mon[34725]: pgmap v497: 129 pgs: 129 active+clean; 218 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 300 op/s 2024-09-21T00:25:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.370+0000 7f853aa00640 1 -- 172.21.15.27:0/1969585321 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55827fbab0b0 con 0x55827f8271c0 2024-09-21T00:25:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.370+0000 7f853aa00640 10 client.25973 handle_command_reply: tid=0 2024-09-21T00:25:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.370+0000 7f855e7df740 1 client.25973 shutdown 2024-09-21T00:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.370+0000 7f855e7df740 1 -- 172.21.15.27:0/1969585321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8724d0 msgr2=0x55827fc5a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.370+0000 7f855e7df740 1 --2- 172.21.15.27:0/1969585321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8724d0 0x55827fc5a640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8548004790 tx=0x7f8548004ab0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.370+0000 7f855e7df740 1 -- 172.21.15.27:0/1969585321 shutdown_connections 2024-09-21T00:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.370+0000 7f855e7df740 1 --2- 172.21.15.27:0/1969585321 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827f8271c0 0x55827fcd3560 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.370+0000 7f855e7df740 1 --2- 172.21.15.27:0/1969585321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827f8931b0 0x55827fccf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.370+0000 7f855e7df740 1 --2- 172.21.15.27:0/1969585321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827f8724d0 0x55827fc5a640 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.370+0000 7f855e7df740 1 --2- 172.21.15.27:0/1969585321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827f86a2f0 0x55827fc5a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.370+0000 7f855e7df740 1 -- 172.21.15.27:0/1969585321 >> 172.21.15.27:0/1969585321 conn(0x55827f887300 msgr2=0x55827fc34af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.370+0000 7f853aa00640 0 client.25973 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.371+0000 7f855e7df740 1 -- 172.21.15.27:0/1969585321 shutdown_connections 2024-09-21T00:25:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.375+0000 7f855e7df740 1 -- 172.21.15.27:0/1969585321 wait complete. 2024-09-21T00:25:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.375+0000 7f855e7df740 20 client.25973 trim_cache size 0 max 16384 2024-09-21T00:25:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:25:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:25:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:25:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:25:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.379+0000 7f8557800640 1 -- 172.21.15.27:0/2264566215 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f85340b6840 msgr2=0x7f85340b8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.379+0000 7f8557800640 1 --2- 172.21.15.27:0/2264566215 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f85340b6840 0x7f85340b8d50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55827fc36a90 tx=0x7f853c10ba90 comp rx=0 tx=0).stop 2024-09-21T00:25:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.379+0000 7f8557800640 1 -- 172.21.15.27:0/2264566215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f855010a900 msgr2=0x7f85501a9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.379+0000 7f8557800640 1 --2- 172.21.15.27:0/2264566215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f855010a900 0x7f85501a9050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f853c0057c0 tx=0x7f853c0045e0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.387+0000 7f8557800640 1 -- 172.21.15.27:0/2264566215 shutdown_connections 2024-09-21T00:25:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.387+0000 7f8557800640 1 --2- 172.21.15.27:0/2264566215 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f85340b6840 0x7f85340b8d50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.387+0000 7f8557800640 1 --2- 172.21.15.27:0/2264566215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f855011cba0 0x7f85501ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.387+0000 7f8557800640 1 --2- 172.21.15.27:0/2264566215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8550111ca0 0x7f85501a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.387+0000 7f8557800640 1 --2- 172.21.15.27:0/2264566215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f855010a900 0x7f85501a9050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.387+0000 7f8557800640 1 -- 172.21.15.27:0/2264566215 >> 172.21.15.27:0/2264566215 conn(0x7f855006e920 msgr2=0x7f8550110920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.391+0000 7f8557800640 1 -- 172.21.15.27:0/2264566215 shutdown_connections 2024-09-21T00:25:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:25.391+0000 7f8557800640 1 -- 172.21.15.27:0/2264566215 wait complete. 2024-09-21T00:25:26.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:26 smithi027 ceph-mon[30745]: pgmap v498: 129 pgs: 129 active+clean; 228 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 289 op/s 2024-09-21T00:25:26.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:26 smithi089 ceph-mon[34481]: pgmap v498: 129 pgs: 129 active+clean; 228 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 289 op/s 2024-09-21T00:25:26.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:26 smithi098 ceph-mon[34725]: pgmap v498: 129 pgs: 129 active+clean; 228 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 289 op/s 2024-09-21T00:25:28.330 INFO:tasks.workunit.client.0.smithi027.stdout: 84 429064 783 281605 836 103102 858 2024-09-21T00:25:28.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:28 smithi089 ceph-mon[34481]: pgmap v499: 129 pgs: 129 active+clean; 238 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.3 MiB/s wr, 569 op/s 2024-09-21T00:25:28.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:28 smithi098 ceph-mon[34725]: pgmap v499: 129 pgs: 129 active+clean; 238 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.3 MiB/s wr, 569 op/s 2024-09-21T00:25:28.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:28 smithi027 ceph-mon[30745]: pgmap v499: 129 pgs: 129 active+clean; 238 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.3 MiB/s wr, 569 op/s 2024-09-21T00:25:30.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:30 smithi089 ceph-mon[34481]: pgmap v500: 129 pgs: 129 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.6 MiB/s wr, 793 op/s 2024-09-21T00:25:30.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:30 smithi098 ceph-mon[34725]: pgmap v500: 129 pgs: 129 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.6 MiB/s wr, 793 op/s 2024-09-21T00:25:30.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:30 smithi027 ceph-mon[30745]: pgmap v500: 129 pgs: 129 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.6 MiB/s wr, 793 op/s 2024-09-21T00:25:32.320 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:32 smithi027 ceph-mon[30745]: pgmap v501: 129 pgs: 129 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.3 MiB/s wr, 1.07k op/s 2024-09-21T00:25:32.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:32 smithi089 ceph-mon[34481]: pgmap v501: 129 pgs: 129 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.3 MiB/s wr, 1.07k op/s 2024-09-21T00:25:32.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:32 smithi098 ceph-mon[34725]: pgmap v501: 129 pgs: 129 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.3 MiB/s wr, 1.07k op/s 2024-09-21T00:25:34.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:34 smithi089 ceph-mon[34481]: pgmap v502: 129 pgs: 129 active+clean; 254 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.8 MiB/s wr, 1.06k op/s 2024-09-21T00:25:34.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:34 smithi098 ceph-mon[34725]: pgmap v502: 129 pgs: 129 active+clean; 254 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.8 MiB/s wr, 1.06k op/s 2024-09-21T00:25:34.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:34 smithi027 ceph-mon[30745]: pgmap v502: 129 pgs: 129 active+clean; 254 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.8 MiB/s wr, 1.06k op/s 2024-09-21T00:25:36.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:36 smithi089 ceph-mon[34481]: pgmap v503: 129 pgs: 129 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 1.04k op/s 2024-09-21T00:25:36.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:36 smithi098 ceph-mon[34725]: pgmap v503: 129 pgs: 129 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 1.04k op/s 2024-09-21T00:25:36.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:36 smithi027 ceph-mon[30745]: pgmap v503: 129 pgs: 129 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 1.04k op/s 2024-09-21T00:25:38.330 INFO:tasks.workunit.client.0.smithi027.stdout: 105 401252 1040 261639 1115 115957 936 2024-09-21T00:25:38.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:38 smithi089 ceph-mon[34481]: pgmap v504: 129 pgs: 129 active+clean; 277 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 1.06k op/s 2024-09-21T00:25:38.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:38 smithi098 ceph-mon[34725]: pgmap v504: 129 pgs: 129 active+clean; 277 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 1.06k op/s 2024-09-21T00:25:38.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:38 smithi027 ceph-mon[30745]: pgmap v504: 129 pgs: 129 active+clean; 277 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 1.06k op/s 2024-09-21T00:25:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.082+0000 7f32daa00640 1 -- 172.21.15.27:0/2630446425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d4111ea0 msgr2=0x7f32d410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.082+0000 7f32daa00640 1 --2- 172.21.15.27:0/2630446425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d4111ea0 0x7f32d410a3c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009860 tx=0x7f32bc02a3d0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.082+0000 7f32daa00640 1 -- 172.21.15.27:0/2630446425 shutdown_connections 2024-09-21T00:25:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.082+0000 7f32daa00640 1 --2- 172.21.15.27:0/2630446425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d411cba0 0x7f32d411f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.082+0000 7f32daa00640 1 --2- 172.21.15.27:0/2630446425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32d410a900 0x7f32d410ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.082+0000 7f32daa00640 1 --2- 172.21.15.27:0/2630446425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d4111ea0 0x7f32d410a3c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.082+0000 7f32daa00640 1 -- 172.21.15.27:0/2630446425 >> 172.21.15.27:0/2630446425 conn(0x7f32d4070d90 msgr2=0x7f32d40711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.084+0000 7f32daa00640 1 -- 172.21.15.27:0/2630446425 shutdown_connections 2024-09-21T00:25:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.084+0000 7f32daa00640 1 -- 172.21.15.27:0/2630446425 wait complete. 2024-09-21T00:25:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.085+0000 7f32daa00640 1 Processor -- start 2024-09-21T00:25:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.086+0000 7f32daa00640 1 -- start start 2024-09-21T00:25:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.087+0000 7f32daa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32d410a900 0x7f32d41a9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.087+0000 7f32daa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d4111ea0 0x7f32d41a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.087+0000 7f32daa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d411cba0 0x7f32d41aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.087+0000 7f32daa00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f32d4122830 con 0x7f32d411cba0 2024-09-21T00:25:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.087+0000 7f32daa00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f32d41226b0 con 0x7f32d4111ea0 2024-09-21T00:25:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.087+0000 7f32daa00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32d41229b0 con 0x7f32d410a900 2024-09-21T00:25:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.087+0000 7f32d3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d4111ea0 0x7f32d41a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.087+0000 7f32d3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d4111ea0 0x7f32d41a97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:50018/0 (socket says 172.21.15.27:50018) 2024-09-21T00:25:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.087+0000 7f32d3400640 1 -- 172.21.15.27:0/455443363 learned_addr learned my addr 172.21.15.27:0/455443363 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.088+0000 7f32d3e00640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32d410a900 0x7f32d41a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.088+0000 7f32d3400640 1 -- 172.21.15.27:0/455443363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32d410a900 msgr2=0x7f32d41a9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.088+0000 7f32d3400640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32d410a900 0x7f32d41a9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.088+0000 7f32d8a00640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d411cba0 0x7f32d41aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.088+0000 7f32d3400640 1 -- 172.21.15.27:0/455443363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d411cba0 msgr2=0x7f32d41aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.088+0000 7f32d3400640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d411cba0 0x7f32d41aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.088+0000 7f32d3400640 1 -- 172.21.15.27:0/455443363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d41af0d0 con 0x7f32d4111ea0 2024-09-21T00:25:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.089+0000 7f32d3e00640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32d410a900 0x7f32d41a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.089+0000 7f32d8a00640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d411cba0 0x7f32d41aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:25:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.089+0000 7f32d3400640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d4111ea0 0x7f32d41a97d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f32c400d770 tx=0x7f32c400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.090+0000 7f32d0c00640 1 -- 172.21.15.27:0/455443363 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4017070 con 0x7f32d4111ea0 2024-09-21T00:25:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.090+0000 7f32d0c00640 1 -- 172.21.15.27:0/455443363 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32c4009c30 con 0x7f32d4111ea0 2024-09-21T00:25:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.090+0000 7f32d0c00640 1 -- 172.21.15.27:0/455443363 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c400f620 con 0x7f32d4111ea0 2024-09-21T00:25:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.090+0000 7f32daa00640 1 -- 172.21.15.27:0/455443363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d41af3c0 con 0x7f32d4111ea0 2024-09-21T00:25:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.090+0000 7f32daa00640 1 -- 172.21.15.27:0/455443363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d41af800 con 0x7f32d4111ea0 2024-09-21T00:25:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.099+0000 7f32e19e9740 1 Processor -- start 2024-09-21T00:25:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.099+0000 7f32d0c00640 1 -- 172.21.15.27:0/455443363 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f32c401a070 con 0x7f32d4111ea0 2024-09-21T00:25:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.100+0000 7f32e19e9740 1 -- start start 2024-09-21T00:25:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.100+0000 7f32e19e9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7ae31b0 0x560ef7e87c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.100+0000 7f32e19e9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7aba2f0 0x560ef7e88170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.100+0000 7f32e19e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ac24d0 0x560ef7e886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.100+0000 7f32e19e9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560ef7dfd6b0 con 0x560ef7ae31b0 2024-09-21T00:25:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.100+0000 7f32e19e9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560ef7decae0 con 0x560ef7aba2f0 2024-09-21T00:25:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.100+0000 7f32e19e9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560ef7df08f0 con 0x560ef7ac24d0 2024-09-21T00:25:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.101+0000 7f32d3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7aba2f0 0x560ef7e88170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.101+0000 7f32d3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7aba2f0 0x560ef7e88170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:50026/0 (socket says 172.21.15.27:50026) 2024-09-21T00:25:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.101+0000 7f32d3e00640 1 -- 172.21.15.27:0/1082523672 learned_addr learned my addr 172.21.15.27:0/1082523672 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.101+0000 7f32d3400640 1 --2- 172.21.15.27:0/1082523672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ac24d0 0x560ef7e886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.102+0000 7f32d8a00640 1 --2- 172.21.15.27:0/1082523672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7ae31b0 0x560ef7e87c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.103+0000 7f32d3e00640 1 -- 172.21.15.27:0/1082523672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ac24d0 msgr2=0x560ef7e886b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.103+0000 7f32d3e00640 1 --2- 172.21.15.27:0/1082523672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ac24d0 0x560ef7e886b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.103+0000 7f32d3e00640 1 -- 172.21.15.27:0/1082523672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7ae31b0 msgr2=0x560ef7e87c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.103+0000 7f32d3e00640 1 --2- 172.21.15.27:0/1082523672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7ae31b0 0x560ef7e87c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.103+0000 7f32d3e00640 1 -- 172.21.15.27:0/1082523672 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ef7db9780 con 0x560ef7aba2f0 2024-09-21T00:25:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.104+0000 7f32d3e00640 1 --2- 172.21.15.27:0/1082523672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7aba2f0 0x560ef7e88170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f32bc030710 tx=0x7f32bc030740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2fd81e856c3f970 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.104+0000 7f32d8a00640 1 --2- 172.21.15.27:0/1082523672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7ae31b0 0x560ef7e87c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.104+0000 7f32d0c00640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f32b80b6850 0x7f32b80b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.104+0000 7f32d3400640 1 --2- 172.21.15.27:0/1082523672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ac24d0 0x560ef7e886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.104+0000 7f32d8a00640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f32b80b6850 0x7f32b80b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.105+0000 7f32d0c00640 1 -- 172.21.15.27:0/455443363 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f32c40e7460 con 0x7f32d4111ea0 2024-09-21T00:25:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.105+0000 7f32c9200640 1 -- 172.21.15.27:0/1082523672 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc030c00 con 0x560ef7aba2f0 2024-09-21T00:25:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.105+0000 7f32c9200640 1 -- 172.21.15.27:0/1082523672 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32bc040030 con 0x560ef7aba2f0 2024-09-21T00:25:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.105+0000 7f32c9200640 1 -- 172.21.15.27:0/1082523672 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc004c60 con 0x560ef7aba2f0 2024-09-21T00:25:39.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.107+0000 7f32e19e9740 1 -- 172.21.15.27:0/1082523672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7aba2f0 msgr2=0x560ef7e88170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:39.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.107+0000 7f32e19e9740 1 --2- 172.21.15.27:0/1082523672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7aba2f0 0x560ef7e88170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f32bc030710 tx=0x7f32bc030740 comp rx=0 tx=0).stop 2024-09-21T00:25:39.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.107+0000 7f32d8a00640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f32b80b6850 0x7f32b80b8d60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f32cc005fd0 tx=0x7f32cc008570 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.108+0000 7f32e19e9740 1 -- 172.21.15.27:0/1082523672 shutdown_connections 2024-09-21T00:25:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.108+0000 7f32e19e9740 1 --2- 172.21.15.27:0/1082523672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ac24d0 0x560ef7e886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.108+0000 7f32e19e9740 1 --2- 172.21.15.27:0/1082523672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7aba2f0 0x560ef7e88170 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.108+0000 7f32e19e9740 1 --2- 172.21.15.27:0/1082523672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7ae31b0 0x560ef7e87c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.108+0000 7f32e19e9740 1 -- 172.21.15.27:0/1082523672 >> 172.21.15.27:0/1082523672 conn(0x560ef7ad7300 msgr2=0x560ef7e85b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.108+0000 7f32e19e9740 1 -- 172.21.15.27:0/1082523672 shutdown_connections 2024-09-21T00:25:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.109+0000 7f32e19e9740 1 -- 172.21.15.27:0/1082523672 wait complete. 2024-09-21T00:25:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.109+0000 7f32e19e9740 1 -- start start 2024-09-21T00:25:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.110+0000 7f32e19e9740 1 Processor -- start 2024-09-21T00:25:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.112+0000 7f32e19e9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 0x560ef7e93200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.112+0000 7f32e19e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ac24d0 0x560ef7e93740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.112+0000 7f32e19e9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ae31b0 0x560ef7e93c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.113+0000 7f32d3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ac24d0 0x560ef7e93740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.113+0000 7f32d3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ac24d0 0x560ef7e93740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41740/0 (socket says 172.21.15.27:41740) 2024-09-21T00:25:39.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.113+0000 7f32d3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 0x560ef7e93200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.113+0000 7f32d3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 0x560ef7e93200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39254/0 (socket says 172.21.15.27:39254) 2024-09-21T00:25:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.115+0000 7f32d3400640 1 -- 172.21.15.27:0/3122655598 learned_addr learned my addr 172.21.15.27:0/3122655598 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.115+0000 7f32e19e9740 1 -- 172.21.15.27:0/3122655598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560ef7d24ab0 con 0x560ef7aba2f0 2024-09-21T00:25:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.115+0000 7f32e19e9740 1 -- 172.21.15.27:0/3122655598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560ef7d93570 con 0x560ef7ae31b0 2024-09-21T00:25:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.115+0000 7f32e19e9740 1 -- 172.21.15.27:0/3122655598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560ef7e96350 con 0x560ef7ac24d0 2024-09-21T00:25:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.115+0000 7f32e19e9740 10 client.0 _ll_register_callbacks cb 0x560ef7da2100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:25:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.116+0000 7f32d3400640 1 -- 172.21.15.27:0/3122655598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ae31b0 msgr2=0x560ef7e93c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.116+0000 7f32d8a00640 1 --2- 172.21.15.27:0/3122655598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ae31b0 0x560ef7e93c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.116+0000 7f32d3400640 1 --2- 172.21.15.27:0/3122655598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ae31b0 0x560ef7e93c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.116+0000 7f32d3400640 1 -- 172.21.15.27:0/3122655598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 msgr2=0x560ef7e93200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.116+0000 7f32d3400640 1 --2- 172.21.15.27:0/3122655598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 0x560ef7e93200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.116+0000 7f32d3400640 1 -- 172.21.15.27:0/3122655598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ef7b4e790 con 0x560ef7ac24d0 2024-09-21T00:25:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.116+0000 7f32d3e00640 1 --2- 172.21.15.27:0/3122655598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 0x560ef7e93200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.117+0000 7f32d8a00640 1 --2- 172.21.15.27:0/3122655598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ae31b0 0x560ef7e93c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:25:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.119+0000 7f32d3400640 1 --2- 172.21.15.27:0/3122655598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ac24d0 0x560ef7e93740 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f32c400d740 tx=0x7f32c40a95d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2e61eda1851d16c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.119+0000 7f32c1600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:25:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.119+0000 7f32c1600640 1 -- 172.21.15.27:0/3122655598 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4021050 con 0x560ef7ac24d0 2024-09-21T00:25:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.119+0000 7f32c1600640 1 -- 172.21.15.27:0/3122655598 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f32c40ac330 con 0x560ef7ac24d0 2024-09-21T00:25:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.120+0000 7f32e19e9740 1 -- 172.21.15.27:0/3122655598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32bc040030 con 0x560ef7ac24d0 2024-09-21T00:25:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.120+0000 7f32c1600640 1 -- 172.21.15.27:0/3122655598 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32c40ac630 con 0x560ef7ac24d0 2024-09-21T00:25:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.121+0000 7f32e19e9740 10 client.25951 fetch_fsmap learned FSMap version 28 2024-09-21T00:25:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.121+0000 7f32e19e9740 1 -- 172.21.15.27:0/3122655598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560ef7bc7620 con 0x560ef7ac24d0 2024-09-21T00:25:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.121+0000 7f32c1600640 1 -- 172.21.15.27:0/3122655598 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32bc040030 con 0x560ef7ac24d0 2024-09-21T00:25:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.122+0000 7f32c1600640 1 -- 172.21.15.27:0/3122655598 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f32c40abb90 con 0x560ef7ac24d0 2024-09-21T00:25:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.122+0000 7f32e19e9740 10 client.25951 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:25:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.122+0000 7f32e19e9740 10 client.25951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:25:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.122+0000 7f32e19e9740 20 client.25951 populate_metadata read hostname 'smithi027' 2024-09-21T00:25:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.123+0000 7f32e19e9740 1 --2- 172.21.15.27:0/3122655598 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560ef7aa1d80 0x560ef7ea5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.123+0000 7f32e19e9740 4 client.25951 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:25:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.123+0000 7f32e19e9740 1 -- 172.21.15.27:0/3122655598 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ef7dff600 con 0x560ef7aa1d80 2024-09-21T00:25:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.123+0000 7f32d3e00640 1 --2- 172.21.15.27:0/3122655598 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560ef7aa1d80 0x560ef7ea5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.124+0000 7f32c1600640 10 client.25951 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.124+0000 7f32d3e00640 1 --2- 172.21.15.27:0/3122655598 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560ef7aa1d80 0x560ef7ea5920 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5709b5042070bf08 server_cookie=1f1cc9e8b9a4f9eb in_seq=0 out_seq=0 2024-09-21T00:25:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.126+0000 7f32c1600640 1 -- 172.21.15.27:0/3122655598 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560ef7dff600 con 0x560ef7aa1d80 2024-09-21T00:25:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.126+0000 7f32c1600640 10 client.25951 handle_command_reply: tid=0 2024-09-21T00:25:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.126+0000 7f32e19e9740 1 client.25951 shutdown 2024-09-21T00:25:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.127+0000 7f32e19e9740 1 -- 172.21.15.27:0/3122655598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ac24d0 msgr2=0x560ef7e93740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.127+0000 7f32e19e9740 1 --2- 172.21.15.27:0/3122655598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ac24d0 0x560ef7e93740 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f32c400d740 tx=0x7f32c40a95d0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.127+0000 7f32e19e9740 1 -- 172.21.15.27:0/3122655598 shutdown_connections 2024-09-21T00:25:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.128+0000 7f32e19e9740 1 --2- 172.21.15.27:0/3122655598 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560ef7aa1d80 0x560ef7ea5920 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.128+0000 7f32e19e9740 1 --2- 172.21.15.27:0/3122655598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ae31b0 0x560ef7e93c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.128+0000 7f32e19e9740 1 --2- 172.21.15.27:0/3122655598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ac24d0 0x560ef7e93740 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.128+0000 7f32e19e9740 1 --2- 172.21.15.27:0/3122655598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 0x560ef7e93200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.128+0000 7f32e19e9740 1 -- 172.21.15.27:0/3122655598 >> 172.21.15.27:0/3122655598 conn(0x560ef7ad7300 msgr2=0x560ef7e88c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.128+0000 7f32c1600640 0 client.25951 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.128+0000 7f32e19e9740 1 -- 172.21.15.27:0/3122655598 shutdown_connections 2024-09-21T00:25:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.129+0000 7f32e19e9740 1 -- 172.21.15.27:0/3122655598 wait complete. 2024-09-21T00:25:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.129+0000 7f32e19e9740 20 client.25951 trim_cache size 0 max 16384 2024-09-21T00:25:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.172+0000 7f32e19e9740 1 Processor -- start 2024-09-21T00:25:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.172+0000 7f32e19e9740 1 -- start start 2024-09-21T00:25:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.173+0000 7f32e19e9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 0x560ef7e84bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.173+0000 7f32e19e9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ac24d0 0x560ef7e950e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.173+0000 7f32e19e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ae31b0 0x560ef7e89370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.173+0000 7f32e19e9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0012870 con 0x560ef7aba2f0 2024-09-21T00:25:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.173+0000 7f32e19e9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0010140 con 0x560ef7ac24d0 2024-09-21T00:25:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.173+0000 7f32e19e9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32b000df50 con 0x560ef7ae31b0 2024-09-21T00:25:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.174+0000 7f32d3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ac24d0 0x560ef7e950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.174+0000 7f32d3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ac24d0 0x560ef7e950e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:50052/0 (socket says 172.21.15.27:50052) 2024-09-21T00:25:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.174+0000 7f32d3400640 1 -- 172.21.15.27:0/1584421047 learned_addr learned my addr 172.21.15.27:0/1584421047 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.175+0000 7f32d3400640 1 -- 172.21.15.27:0/1584421047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ae31b0 msgr2=0x560ef7e89370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:25:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.175+0000 7f32d3400640 1 --2- 172.21.15.27:0/1584421047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ae31b0 0x560ef7e89370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.175+0000 7f32d3400640 1 -- 172.21.15.27:0/1584421047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 msgr2=0x560ef7e84bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.175+0000 7f32d3400640 1 --2- 172.21.15.27:0/1584421047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 0x560ef7e84bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.175+0000 7f32d3400640 1 -- 172.21.15.27:0/1584421047 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ef7e7fd50 con 0x560ef7ac24d0 2024-09-21T00:25:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.176+0000 7f32d3400640 1 --2- 172.21.15.27:0/1584421047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ac24d0 0x560ef7e950e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f32c400d740 tx=0x7f32c4010c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f460f7c542d57c66 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.177+0000 7f32c3e00640 1 -- 172.21.15.27:0/1584421047 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c40ac630 con 0x560ef7ac24d0 2024-09-21T00:25:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.178+0000 7f32c3e00640 1 -- 172.21.15.27:0/1584421047 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32c40ac9d0 con 0x560ef7ac24d0 2024-09-21T00:25:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.178+0000 7f32c3e00640 1 -- 172.21.15.27:0/1584421047 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c40b19a0 con 0x560ef7ac24d0 2024-09-21T00:25:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.179+0000 7f32e19e9740 1 -- 172.21.15.27:0/1584421047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ac24d0 msgr2=0x560ef7e950e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.179+0000 7f32e19e9740 1 --2- 172.21.15.27:0/1584421047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ac24d0 0x560ef7e950e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f32c400d740 tx=0x7f32c4010c70 comp rx=0 tx=0).stop 2024-09-21T00:25:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.179+0000 7f32e19e9740 1 -- 172.21.15.27:0/1584421047 shutdown_connections 2024-09-21T00:25:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.179+0000 7f32e19e9740 1 --2- 172.21.15.27:0/1584421047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ae31b0 0x560ef7e89370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.179+0000 7f32e19e9740 1 --2- 172.21.15.27:0/1584421047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ac24d0 0x560ef7e950e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.179+0000 7f32e19e9740 1 --2- 172.21.15.27:0/1584421047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 0x560ef7e84bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.179+0000 7f32e19e9740 1 -- 172.21.15.27:0/1584421047 >> 172.21.15.27:0/1584421047 conn(0x560ef7ad7300 msgr2=0x560ef7e88c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.180+0000 7f32e19e9740 1 -- 172.21.15.27:0/1584421047 shutdown_connections 2024-09-21T00:25:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.180+0000 7f32e19e9740 1 -- 172.21.15.27:0/1584421047 wait complete. 2024-09-21T00:25:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.181+0000 7f32e19e9740 1 -- start start 2024-09-21T00:25:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.181+0000 7f32e19e9740 1 Processor -- start 2024-09-21T00:25:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.182+0000 7f32e19e9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 0x560ef7ea5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.183+0000 7f32d3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 0x560ef7ea5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.183+0000 7f32e19e9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ac24d0 0x560ef7ea6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.183+0000 7f32d3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 0x560ef7ea5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39268/0 (socket says 172.21.15.27:39268) 2024-09-21T00:25:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.183+0000 7f32e19e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ae31b0 0x560ef7ea6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.183+0000 7f32e19e9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f32b000e600 con 0x560ef7aba2f0 2024-09-21T00:25:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.183+0000 7f32e19e9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f32b000fa70 con 0x560ef7ac24d0 2024-09-21T00:25:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.183+0000 7f32e19e9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32b000ecd0 con 0x560ef7ae31b0 2024-09-21T00:25:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.183+0000 7f32e19e9740 10 client.0 _ll_register_callbacks cb 0x7f32c40e4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:25:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.183+0000 7f32d3e00640 1 -- 172.21.15.27:0/809487634 learned_addr learned my addr 172.21.15.27:0/809487634 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:25:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.183+0000 7f32d8a00640 1 --2- 172.21.15.27:0/809487634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ae31b0 0x560ef7ea6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.183+0000 7f32d3400640 1 --2- 172.21.15.27:0/809487634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ac24d0 0x560ef7ea6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.184+0000 7f32d8a00640 1 -- 172.21.15.27:0/809487634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ac24d0 msgr2=0x560ef7ea6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.184+0000 7f32d8a00640 1 --2- 172.21.15.27:0/809487634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ac24d0 0x560ef7ea6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.184+0000 7f32d8a00640 1 -- 172.21.15.27:0/809487634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 msgr2=0x560ef7ea5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.184+0000 7f32d8a00640 1 --2- 172.21.15.27:0/809487634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 0x560ef7ea5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.184+0000 7f32d8a00640 1 -- 172.21.15.27:0/809487634 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ef7eaaf80 con 0x560ef7ae31b0 2024-09-21T00:25:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.185+0000 7f32d3e00640 1 --2- 172.21.15.27:0/809487634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 0x560ef7ea5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.185+0000 7f32d8a00640 1 --2- 172.21.15.27:0/809487634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ae31b0 0x560ef7ea6980 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f32cc010d50 tx=0x7f32cc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe9f327790dbd601 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.185+0000 7f32c1600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:25:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.186+0000 7f32c1600640 1 -- 172.21.15.27:0/809487634 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc01d620 con 0x560ef7ae31b0 2024-09-21T00:25:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.186+0000 7f32c1600640 1 -- 172.21.15.27:0/809487634 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f32cc01d7c0 con 0x560ef7ae31b0 2024-09-21T00:25:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.186+0000 7f32e19e9740 1 -- 172.21.15.27:0/809487634 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32c40ac9d0 con 0x560ef7ae31b0 2024-09-21T00:25:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.186+0000 7f32c1600640 1 -- 172.21.15.27:0/809487634 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32cc01d9c0 con 0x560ef7ae31b0 2024-09-21T00:25:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.186+0000 7f32e19e9740 10 client.25954 fetch_fsmap learned FSMap version 28 2024-09-21T00:25:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.186+0000 7f32e19e9740 1 -- 172.21.15.27:0/809487634 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560ef7f21550 con 0x560ef7ae31b0 2024-09-21T00:25:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.187+0000 7f32c1600640 1 -- 172.21.15.27:0/809487634 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32c40ac9d0 con 0x560ef7ae31b0 2024-09-21T00:25:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.187+0000 7f32c1600640 1 -- 172.21.15.27:0/809487634 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f32cc01ab90 con 0x560ef7ae31b0 2024-09-21T00:25:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.187+0000 7f32e19e9740 10 client.25954 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:25:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.188+0000 7f32e19e9740 10 client.25954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:25:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.188+0000 7f32e19e9740 20 client.25954 populate_metadata read hostname 'smithi027' 2024-09-21T00:25:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.188+0000 7f32e19e9740 1 --2- 172.21.15.27:0/809487634 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560ef7a771c0 0x560ef7f23dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.188+0000 7f32e19e9740 4 client.25954 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:25:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.188+0000 7f32e19e9740 1 -- 172.21.15.27:0/809487634 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ef7dfb0b0 con 0x560ef7a771c0 2024-09-21T00:25:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.188+0000 7f32d3e00640 1 --2- 172.21.15.27:0/809487634 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560ef7a771c0 0x560ef7f23dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.189+0000 7f32d3e00640 1 --2- 172.21.15.27:0/809487634 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560ef7a771c0 0x560ef7f23dd0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cef517e93e3541c server_cookie=dd3953f04889c86e in_seq=0 out_seq=0 2024-09-21T00:25:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:39.193+0000 7f32c1600640 10 client.25954 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:40.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:40 smithi027 ceph-mon[30745]: pgmap v505: 129 pgs: 129 active+clean; 287 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 801 op/s 2024-09-21T00:25:40.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:40 smithi089 ceph-mon[34481]: pgmap v505: 129 pgs: 129 active+clean; 287 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 801 op/s 2024-09-21T00:25:40.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:40 smithi098 ceph-mon[34725]: pgmap v505: 129 pgs: 129 active+clean; 287 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 801 op/s 2024-09-21T00:25:42.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:42 smithi027 ceph-mon[30745]: pgmap v506: 129 pgs: 129 active+clean; 304 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 681 op/s 2024-09-21T00:25:42.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:42 smithi089 ceph-mon[34481]: pgmap v506: 129 pgs: 129 active+clean; 304 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 681 op/s 2024-09-21T00:25:42.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:42 smithi098 ceph-mon[34725]: pgmap v506: 129 pgs: 129 active+clean; 304 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 681 op/s 2024-09-21T00:25:44.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:44 smithi098 ceph-mon[34725]: pgmap v507: 129 pgs: 129 active+clean; 327 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 612 op/s 2024-09-21T00:25:44.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:44 smithi027 ceph-mon[30745]: pgmap v507: 129 pgs: 129 active+clean; 327 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 612 op/s 2024-09-21T00:25:44.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:44 smithi089 ceph-mon[34481]: pgmap v507: 129 pgs: 129 active+clean; 327 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 612 op/s 2024-09-21T00:25:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.866+0000 7f32c1600640 1 -- 172.21.15.27:0/809487634 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560ef7dfb0b0 con 0x560ef7a771c0 2024-09-21T00:25:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.866+0000 7f32c1600640 10 client.25954 handle_command_reply: tid=0 2024-09-21T00:25:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.866+0000 7f32e19e9740 1 client.25954 shutdown 2024-09-21T00:25:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.874+0000 7f32e19e9740 1 -- 172.21.15.27:0/809487634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ae31b0 msgr2=0x560ef7ea6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.874+0000 7f32e19e9740 1 --2- 172.21.15.27:0/809487634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ae31b0 0x560ef7ea6980 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f32cc010d50 tx=0x7f32cc004a40 comp rx=0 tx=0).stop 2024-09-21T00:25:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.874+0000 7f32e19e9740 1 -- 172.21.15.27:0/809487634 shutdown_connections 2024-09-21T00:25:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.874+0000 7f32e19e9740 1 --2- 172.21.15.27:0/809487634 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560ef7a771c0 0x560ef7f23dd0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.874+0000 7f32e19e9740 1 --2- 172.21.15.27:0/809487634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ef7ae31b0 0x560ef7ea6980 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.874+0000 7f32e19e9740 1 --2- 172.21.15.27:0/809487634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ef7ac24d0 0x560ef7ea6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.874+0000 7f32e19e9740 1 --2- 172.21.15.27:0/809487634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560ef7aba2f0 0x560ef7ea5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.874+0000 7f32c1600640 0 client.25954 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:25:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.874+0000 7f32e19e9740 1 -- 172.21.15.27:0/809487634 >> 172.21.15.27:0/809487634 conn(0x560ef7ad7300 msgr2=0x560ef7e94e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.875+0000 7f32e19e9740 1 -- 172.21.15.27:0/809487634 shutdown_connections 2024-09-21T00:25:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.875+0000 7f32e19e9740 1 -- 172.21.15.27:0/809487634 wait complete. 2024-09-21T00:25:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.875+0000 7f32e19e9740 20 client.25954 trim_cache size 0 max 16384 2024-09-21T00:25:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:25:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:25:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:25:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:25:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.882+0000 7f32daa00640 1 -- 172.21.15.27:0/455443363 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f32b80b6850 msgr2=0x7f32b80b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.882+0000 7f32daa00640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f32b80b6850 0x7f32b80b8d60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f32cc005fd0 tx=0x7f32cc008570 comp rx=0 tx=0).stop 2024-09-21T00:25:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.882+0000 7f32daa00640 1 -- 172.21.15.27:0/455443363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d4111ea0 msgr2=0x7f32d41a97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.882+0000 7f32daa00640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d4111ea0 0x7f32d41a97d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f32c400d770 tx=0x7f32c400dc30 comp rx=0 tx=0).stop 2024-09-21T00:25:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.882+0000 7f32daa00640 1 -- 172.21.15.27:0/455443363 shutdown_connections 2024-09-21T00:25:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.882+0000 7f32daa00640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f32b80b6850 0x7f32b80b8d60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.882+0000 7f32daa00640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d411cba0 0x7f32d41aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.882+0000 7f32daa00640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d4111ea0 0x7f32d41a97d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.882+0000 7f32daa00640 1 --2- 172.21.15.27:0/455443363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32d410a900 0x7f32d41a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.882+0000 7f32daa00640 1 -- 172.21.15.27:0/455443363 >> 172.21.15.27:0/455443363 conn(0x7f32d4070d90 msgr2=0x7f32d411b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.886+0000 7f32daa00640 1 -- 172.21.15.27:0/455443363 shutdown_connections 2024-09-21T00:25:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:25:44.886+0000 7f32daa00640 1 -- 172.21.15.27:0/455443363 wait complete. 2024-09-21T00:25:45.283 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:45 smithi027 ceph-mon[30745]: pgmap v508: 129 pgs: 129 active+clean; 346 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.3 MiB/s wr, 954 op/s 2024-09-21T00:25:45.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:45 smithi089 ceph-mon[34481]: pgmap v508: 129 pgs: 129 active+clean; 346 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.3 MiB/s wr, 954 op/s 2024-09-21T00:25:45.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:45 smithi098 ceph-mon[34725]: pgmap v508: 129 pgs: 129 active+clean; 346 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.3 MiB/s wr, 954 op/s 2024-09-21T00:25:47.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:47 smithi027 ceph-mon[30745]: pgmap v509: 129 pgs: 129 active+clean; 341 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.4 MiB/s wr, 1.27k op/s 2024-09-21T00:25:47.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:47 smithi089 ceph-mon[34481]: pgmap v509: 129 pgs: 129 active+clean; 341 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.4 MiB/s wr, 1.27k op/s 2024-09-21T00:25:47.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:47 smithi098 ceph-mon[34725]: pgmap v509: 129 pgs: 129 active+clean; 341 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.4 MiB/s wr, 1.27k op/s 2024-09-21T00:25:48.330 INFO:tasks.workunit.client.0.smithi027.stdout: 123 307612 953 204262 851 86368 504 2024-09-21T00:25:49.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:49 smithi027 ceph-mon[30745]: pgmap v510: 129 pgs: 129 active+clean; 344 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.9 MiB/s wr, 1.22k op/s 2024-09-21T00:25:49.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:49 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:25:49.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:49 smithi089 ceph-mon[34481]: pgmap v510: 129 pgs: 129 active+clean; 344 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.9 MiB/s wr, 1.22k op/s 2024-09-21T00:25:49.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:49 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:25:49.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:49 smithi098 ceph-mon[34725]: pgmap v510: 129 pgs: 129 active+clean; 344 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.9 MiB/s wr, 1.22k op/s 2024-09-21T00:25:49.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:49 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:25:50.819 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:50 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:25:50.819 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:50 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:25:50.819 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:50 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:25:50.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:50 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:25:50.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:50 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:25:50.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:50 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:25:50.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:50 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:25:50.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:50 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:25:50.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:50 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:25:51.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:51 smithi089 ceph-mon[34481]: pgmap v511: 129 pgs: 129 active+clean; 341 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.5 MiB/s wr, 1.20k op/s 2024-09-21T00:25:51.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:51 smithi098 ceph-mon[34725]: pgmap v511: 129 pgs: 129 active+clean; 341 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.5 MiB/s wr, 1.20k op/s 2024-09-21T00:25:52.087 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:51 smithi027 ceph-mon[30745]: pgmap v511: 129 pgs: 129 active+clean; 341 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.5 MiB/s wr, 1.20k op/s 2024-09-21T00:25:53.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:53 smithi089 ceph-mon[34481]: pgmap v512: 129 pgs: 129 active+clean; 345 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 1.07k op/s 2024-09-21T00:25:53.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:53 smithi098 ceph-mon[34725]: pgmap v512: 129 pgs: 129 active+clean; 345 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 1.07k op/s 2024-09-21T00:25:54.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:53 smithi027 ceph-mon[30745]: pgmap v512: 129 pgs: 129 active+clean; 345 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 1.07k op/s 2024-09-21T00:25:55.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:55 smithi089 ceph-mon[34481]: pgmap v513: 129 pgs: 129 active+clean; 347 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 842 op/s 2024-09-21T00:25:55.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:55 smithi098 ceph-mon[34725]: pgmap v513: 129 pgs: 129 active+clean; 347 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 842 op/s 2024-09-21T00:25:56.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:55 smithi027 ceph-mon[30745]: pgmap v513: 129 pgs: 129 active+clean; 347 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 842 op/s 2024-09-21T00:25:57.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:57 smithi089 ceph-mon[34481]: pgmap v514: 129 pgs: 129 active+clean; 362 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 528 op/s 2024-09-21T00:25:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:57 smithi098 ceph-mon[34725]: pgmap v514: 129 pgs: 129 active+clean; 362 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 528 op/s 2024-09-21T00:25:58.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:57 smithi027 ceph-mon[30745]: pgmap v514: 129 pgs: 129 active+clean; 362 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 528 op/s 2024-09-21T00:25:58.331 INFO:tasks.workunit.client.0.smithi027.stdout: 143 535155 918 353874 968 142576 818 2024-09-21T00:25:59.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:25:59 smithi089 ceph-mon[34481]: pgmap v515: 129 pgs: 129 active+clean; 373 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 218 op/s 2024-09-21T00:25:59.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:25:59 smithi098 ceph-mon[34725]: pgmap v515: 129 pgs: 129 active+clean; 373 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 218 op/s 2024-09-21T00:26:00.074 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:25:59 smithi027 ceph-mon[30745]: pgmap v515: 129 pgs: 129 active+clean; 373 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 218 op/s 2024-09-21T00:26:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.064+0000 7fd2eaa00640 1 -- 172.21.15.27:0/3493070999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2e410fe50 msgr2=0x7fd2e4110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.067+0000 7fd2eaa00640 1 --2- 172.21.15.27:0/3493070999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2e410fe50 0x7fd2e4110260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000bb30 tx=0x7fd2d002e6b0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.070+0000 7fd2eaa00640 1 -- 172.21.15.27:0/3493070999 shutdown_connections 2024-09-21T00:26:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.070+0000 7fd2eaa00640 1 --2- 172.21.15.27:0/3493070999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2e41107a0 0x7fd2e4116fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.070+0000 7fd2eaa00640 1 --2- 172.21.15.27:0/3493070999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2e410fe50 0x7fd2e4110260 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.072+0000 7fd2eaa00640 1 --2- 172.21.15.27:0/3493070999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e410a660 0x7fd2e410aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.072+0000 7fd2eaa00640 1 -- 172.21.15.27:0/3493070999 >> 172.21.15.27:0/3493070999 conn(0x7fd2e4070d90 msgr2=0x7fd2e40711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.074+0000 7fd2eaa00640 1 -- 172.21.15.27:0/3493070999 shutdown_connections 2024-09-21T00:26:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.077+0000 7fd2eaa00640 1 -- 172.21.15.27:0/3493070999 wait complete. 2024-09-21T00:26:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.099+0000 7fd2eaa00640 1 Processor -- start 2024-09-21T00:26:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.113+0000 7fd2eaa00640 1 -- start start 2024-09-21T00:26:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.114+0000 7fd2eaa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e410a660 0x7fd2e41a92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.127+0000 7fd2e3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e410a660 0x7fd2e41a92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.129+0000 7fd2e3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e410a660 0x7fd2e41a92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41164/0 (socket says 172.21.15.27:41164) 2024-09-21T00:26:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.131+0000 7fd2e3e00640 1 -- 172.21.15.27:0/725555371 learned_addr learned my addr 172.21.15.27:0/725555371 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.133+0000 7fd2eaa00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2e410fe50 0x7fd2e41a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.134+0000 7fd2eaa00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2e41107a0 0x7fd2e41aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.139+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2e41107a0 0x7fd2e41aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.139+0000 7fd2eaa00640 1 -- 172.21.15.27:0/725555371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e411a020 con 0x7fd2e410a660 2024-09-21T00:26:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.139+0000 7fd2eaa00640 1 -- 172.21.15.27:0/725555371 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4119ea0 con 0x7fd2e41107a0 2024-09-21T00:26:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.139+0000 7fd2eaa00640 1 -- 172.21.15.27:0/725555371 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e411a1a0 con 0x7fd2e410fe50 2024-09-21T00:26:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.139+0000 7fd2e3400640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2e410fe50 0x7fd2e41a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.140+0000 7fd2e3e00640 1 -- 172.21.15.27:0/725555371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2e410fe50 msgr2=0x7fd2e41a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.140+0000 7fd2e3e00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2e410fe50 0x7fd2e41a9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.140+0000 7fd2e3e00640 1 -- 172.21.15.27:0/725555371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2e41107a0 msgr2=0x7fd2e41aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.140+0000 7fd2e3e00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2e41107a0 0x7fd2e41aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.140+0000 7fd2e3e00640 1 -- 172.21.15.27:0/725555371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e41af150 con 0x7fd2e410a660 2024-09-21T00:26:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.140+0000 7fd2e3e00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e410a660 0x7fd2e41a92b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400c890 tx=0x7fd2d400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.141+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2e41107a0 0x7fd2e41aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.141+0000 7fd2e0c00640 1 -- 172.21.15.27:0/725555371 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4004220 con 0x7fd2e410a660 2024-09-21T00:26:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.142+0000 7fd2e0c00640 1 -- 172.21.15.27:0/725555371 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2d40043c0 con 0x7fd2e410a660 2024-09-21T00:26:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.142+0000 7fd2e0c00640 1 -- 172.21.15.27:0/725555371 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4004a00 con 0x7fd2e410a660 2024-09-21T00:26:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.143+0000 7fd2eaa00640 1 -- 172.21.15.27:0/725555371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e41af440 con 0x7fd2e410a660 2024-09-21T00:26:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.144+0000 7fd2eaa00640 1 -- 172.21.15.27:0/725555371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e41a07a0 con 0x7fd2e410a660 2024-09-21T00:26:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.158+0000 7fd2f1a7c740 1 Processor -- start 2024-09-21T00:26:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.174+0000 7fd2e0c00640 1 -- 172.21.15.27:0/725555371 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fd2d4004c60 con 0x7fd2e410a660 2024-09-21T00:26:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.177+0000 7fd2f1a7c740 1 -- start start 2024-09-21T00:26:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.177+0000 7fd2f1a7c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644e131b0 0x5576451d9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.190+0000 7fd2f1a7c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644dea2f0 0x5576451da180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.190+0000 7fd2f1a7c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644df24d0 0x5576451da6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.190+0000 7fd2f1a7c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55764512d6b0 con 0x557644e131b0 2024-09-21T00:26:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.190+0000 7fd2f1a7c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55764511cae0 con 0x557644df24d0 2024-09-21T00:26:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.190+0000 7fd2f1a7c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5576451208f0 con 0x557644dea2f0 2024-09-21T00:26:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.192+0000 7fd2e8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644e131b0 0x5576451d9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.193+0000 7fd2e8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644e131b0 0x5576451d9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41166/0 (socket says 172.21.15.27:41166) 2024-09-21T00:26:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.193+0000 7fd2e8a00640 1 -- 172.21.15.27:0/2566868142 learned_addr learned my addr 172.21.15.27:0/2566868142 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.194+0000 7fd2e3e00640 1 --2- 172.21.15.27:0/2566868142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644dea2f0 0x5576451da180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.195+0000 7fd2e8a00640 1 -- 172.21.15.27:0/2566868142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644dea2f0 msgr2=0x5576451da180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.195+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/2566868142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644dea2f0 0x5576451da180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.195+0000 7fd2e8a00640 1 -- 172.21.15.27:0/2566868142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644df24d0 msgr2=0x5576451da6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:26:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.195+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/2566868142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644df24d0 0x5576451da6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.195+0000 7fd2e8a00640 1 -- 172.21.15.27:0/2566868142 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576450e9780 con 0x557644e131b0 2024-09-21T00:26:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.197+0000 7fd2e0c00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd2c80b6800 0x7fd2c80b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.202+0000 7fd2e3e00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd2c80b6800 0x7fd2c80b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.203+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/2566868142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644e131b0 0x5576451d9c40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc007ec0 tx=0x7fd2dc00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51e13acdec6500b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.214+0000 7fd2e0c00640 1 -- 172.21.15.27:0/725555371 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fd2d40079f0 con 0x7fd2e410a660 2024-09-21T00:26:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.217+0000 7fd2d9200640 1 -- 172.21.15.27:0/2566868142 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc004d20 con 0x557644e131b0 2024-09-21T00:26:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.218+0000 7fd2d9200640 1 -- 172.21.15.27:0/2566868142 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2dc00fae0 con 0x557644e131b0 2024-09-21T00:26:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.218+0000 7fd2d9200640 1 -- 172.21.15.27:0/2566868142 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc005650 con 0x557644e131b0 2024-09-21T00:26:01.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.221+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/2566868142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644e131b0 msgr2=0x5576451d9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.221+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/2566868142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644e131b0 0x5576451d9c40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc007ec0 tx=0x7fd2dc00d710 comp rx=0 tx=0).stop 2024-09-21T00:26:01.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.228+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/2566868142 shutdown_connections 2024-09-21T00:26:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.230+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/2566868142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644df24d0 0x5576451da6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.230+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/2566868142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644dea2f0 0x5576451da180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.230+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/2566868142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644e131b0 0x5576451d9c40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.230+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/2566868142 >> 172.21.15.27:0/2566868142 conn(0x557644e07300 msgr2=0x5576451d7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.232+0000 7fd2e3e00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd2c80b6800 0x7fd2c80b8d10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5576451d88c0 tx=0x7fd2d40e9040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.242+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/2566868142 shutdown_connections 2024-09-21T00:26:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.245+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/2566868142 wait complete. 2024-09-21T00:26:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.245+0000 7fd2f1a7c740 1 -- start start 2024-09-21T00:26:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.246+0000 7fd2f1a7c740 1 Processor -- start 2024-09-21T00:26:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.256+0000 7fd2f1a7c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644dea2f0 0x5576451e5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.258+0000 7fd2e8a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644dea2f0 0x5576451e5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.258+0000 7fd2e8a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644dea2f0 0x5576451e5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:58186/0 (socket says 172.21.15.27:58186) 2024-09-21T00:26:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.258+0000 7fd2f1a7c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644df24d0 0x5576451e5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.259+0000 7fd2e3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644df24d0 0x5576451e5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.261+0000 7fd2e3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644df24d0 0x5576451e5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41168/0 (socket says 172.21.15.27:41168) 2024-09-21T00:26:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.261+0000 7fd2e8a00640 1 -- 172.21.15.27:0/384053933 learned_addr learned my addr 172.21.15.27:0/384053933 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.261+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/384053933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644e131b0 0x5576451f1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.261+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/384053933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557645054ab0 con 0x557644df24d0 2024-09-21T00:26:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.261+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/384053933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5576450c3570 con 0x557644e131b0 2024-09-21T00:26:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.261+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/384053933 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5576451e83c0 con 0x557644dea2f0 2024-09-21T00:26:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.262+0000 7fd2f1a7c740 10 client.0 _ll_register_callbacks cb 0x5576450d2100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:26:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.264+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/384053933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644e131b0 0x5576451f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.265+0000 7fd2e8a00640 1 -- 172.21.15.27:0/384053933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644e131b0 msgr2=0x5576451f1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.265+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/384053933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644e131b0 0x5576451f1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.265+0000 7fd2e8a00640 1 -- 172.21.15.27:0/384053933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644df24d0 msgr2=0x5576451e5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.265+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/384053933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644df24d0 0x5576451e5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.267+0000 7fd2e8a00640 1 -- 172.21.15.27:0/384053933 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557644e7e790 con 0x557644dea2f0 2024-09-21T00:26:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.268+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/384053933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644dea2f0 0x5576451e5320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0073f0 tx=0x7fd2dc007fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4dff30ae21eaf7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.285+0000 7fd2cd600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:26:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.285+0000 7fd2cd600640 1 -- 172.21.15.27:0/384053933 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc01f040 con 0x557644dea2f0 2024-09-21T00:26:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.285+0000 7fd2cd600640 1 -- 172.21.15.27:0/384053933 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fd2dc0057e0 con 0x557644dea2f0 2024-09-21T00:26:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.285+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/384053933 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2dc00fae0 con 0x557644dea2f0 2024-09-21T00:26:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.287+0000 7fd2cd600640 1 -- 172.21.15.27:0/384053933 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd2dc028470 con 0x557644dea2f0 2024-09-21T00:26:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.287+0000 7fd2f1a7c740 10 client.25963 fetch_fsmap learned FSMap version 28 2024-09-21T00:26:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.287+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/384053933 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557644ef7620 con 0x557644dea2f0 2024-09-21T00:26:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.287+0000 7fd2cd600640 1 -- 172.21.15.27:0/384053933 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2dc028ac0 con 0x557644dea2f0 2024-09-21T00:26:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.299+0000 7fd2cd600640 1 -- 172.21.15.27:0/384053933 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd2dc025040 con 0x557644dea2f0 2024-09-21T00:26:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.300+0000 7fd2f1a7c740 10 client.25963 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:26:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.300+0000 7fd2f1a7c740 10 client.25963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:26:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.300+0000 7fd2f1a7c740 20 client.25963 populate_metadata read hostname 'smithi027' 2024-09-21T00:26:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.300+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/384053933 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557644dd1d80 0x5576451f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.300+0000 7fd2f1a7c740 4 client.25963 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:26:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.300+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/384053933 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd2dc00fae0 con 0x557644dd1d80 2024-09-21T00:26:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.307+0000 7fd2e3400640 1 --2- 172.21.15.27:0/384053933 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557644dd1d80 0x5576451f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.313+0000 7fd2e3400640 1 --2- 172.21.15.27:0/384053933 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557644dd1d80 0x5576451f77d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f28a76b8f6e73e6 server_cookie=85a80ec0a22319c9 in_seq=0 out_seq=0 2024-09-21T00:26:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.313+0000 7fd2cd600640 10 client.25963 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:26:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.326+0000 7fd2cd600640 1 -- 172.21.15.27:0/384053933 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fd2dc00fae0 con 0x557644dd1d80 2024-09-21T00:26:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.329+0000 7fd2cd600640 10 client.25963 handle_command_reply: tid=0 2024-09-21T00:26:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.334+0000 7fd2f1a7c740 1 client.25963 shutdown 2024-09-21T00:26:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.388+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/384053933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644dea2f0 msgr2=0x5576451e5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.388+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/384053933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644dea2f0 0x5576451e5320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0073f0 tx=0x7fd2dc007fb0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.416+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/384053933 shutdown_connections 2024-09-21T00:26:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.416+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/384053933 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557644dd1d80 0x5576451f77d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.417+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/384053933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644e131b0 0x5576451f1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.417+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/384053933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644df24d0 0x5576451e5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.417+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/384053933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644dea2f0 0x5576451e5320 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.417+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/384053933 >> 172.21.15.27:0/384053933 conn(0x557644e07300 msgr2=0x5576451dac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.422+0000 7fd2cd600640 0 client.25963 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:26:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.427+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/384053933 shutdown_connections 2024-09-21T00:26:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.443+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/384053933 wait complete. 2024-09-21T00:26:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.454+0000 7fd2f1a7c740 20 client.25963 trim_cache size 0 max 16384 2024-09-21T00:26:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.963+0000 7fd2f1a7c740 1 Processor -- start 2024-09-21T00:26:01.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:01 smithi089 ceph-mon[34481]: pgmap v516: 129 pgs: 129 active+clean; 397 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 279 op/s 2024-09-21T00:26:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.988+0000 7fd2f1a7c740 1 -- start start 2024-09-21T00:26:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.989+0000 7fd2f1a7c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644dea2f0 0x5576451e5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.989+0000 7fd2f1a7c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644df24d0 0x5576451d7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.989+0000 7fd2f1a7c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644e131b0 0x5576451f1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.989+0000 7fd2f1a7c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41b9eb0 con 0x557644e131b0 2024-09-21T00:26:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.989+0000 7fd2f1a7c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41b7780 con 0x557644dea2f0 2024-09-21T00:26:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.989+0000 7fd2f1a7c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41a88e0 con 0x557644df24d0 2024-09-21T00:26:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.990+0000 7fd2e8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644e131b0 0x5576451f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.991+0000 7fd2e8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644e131b0 0x5576451f1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41174/0 (socket says 172.21.15.27:41174) 2024-09-21T00:26:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.991+0000 7fd2e8a00640 1 -- 172.21.15.27:0/539239108 learned_addr learned my addr 172.21.15.27:0/539239108 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.991+0000 7fd2e8a00640 1 -- 172.21.15.27:0/539239108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644df24d0 msgr2=0x5576451d7a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:26:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.991+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/539239108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644df24d0 0x5576451d7a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.991+0000 7fd2e8a00640 1 -- 172.21.15.27:0/539239108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644dea2f0 msgr2=0x5576451e5800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:26:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.991+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/539239108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644dea2f0 0x5576451e5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:01.991+0000 7fd2e8a00640 1 -- 172.21.15.27:0/539239108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576451e65e0 con 0x557644e131b0 2024-09-21T00:26:01.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:01 smithi098 ceph-mon[34725]: pgmap v516: 129 pgs: 129 active+clean; 397 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 279 op/s 2024-09-21T00:26:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.004+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/539239108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644e131b0 0x5576451f1610 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc007ec0 tx=0x7fd2dc005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f30c159f738c4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.015+0000 7fd2cfe00640 1 -- 172.21.15.27:0/539239108 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc037070 con 0x557644e131b0 2024-09-21T00:26:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.015+0000 7fd2cfe00640 1 -- 172.21.15.27:0/539239108 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2dc028ac0 con 0x557644e131b0 2024-09-21T00:26:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.015+0000 7fd2cfe00640 1 -- 172.21.15.27:0/539239108 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc028470 con 0x557644e131b0 2024-09-21T00:26:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.019+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/539239108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644e131b0 msgr2=0x5576451f1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.019+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/539239108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644e131b0 0x5576451f1610 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc007ec0 tx=0x7fd2dc005a50 comp rx=0 tx=0).stop 2024-09-21T00:26:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.023+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/539239108 shutdown_connections 2024-09-21T00:26:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.024+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/539239108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644e131b0 0x5576451f1610 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.024+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/539239108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644df24d0 0x5576451d7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.024+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/539239108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644dea2f0 0x5576451e5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.024+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/539239108 >> 172.21.15.27:0/539239108 conn(0x557644e07300 msgr2=0x5576451d7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:02.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.056+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/539239108 shutdown_connections 2024-09-21T00:26:02.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.056+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/539239108 wait complete. 2024-09-21T00:26:02.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.057+0000 7fd2f1a7c740 1 -- start start 2024-09-21T00:26:02.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.057+0000 7fd2f1a7c740 1 Processor -- start 2024-09-21T00:26:02.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:01 smithi027 ceph-mon[30745]: pgmap v516: 129 pgs: 129 active+clean; 397 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 279 op/s 2024-09-21T00:26:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.069+0000 7fd2f1a7c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644dea2f0 0x5576451edc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.075+0000 7fd2f1a7c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644df24d0 0x5576451e90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.075+0000 7fd2f1a7c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644e131b0 0x5576451e9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.075+0000 7fd2f1a7c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41b5c60 con 0x557644dea2f0 2024-09-21T00:26:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.075+0000 7fd2f1a7c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41b70b0 con 0x557644e131b0 2024-09-21T00:26:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.075+0000 7fd2f1a7c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41b6310 con 0x557644df24d0 2024-09-21T00:26:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.075+0000 7fd2f1a7c740 10 client.0 _ll_register_callbacks cb 0x7fd2dc007d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:26:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.078+0000 7fd2e3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644df24d0 0x5576451e90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.079+0000 7fd2e3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644df24d0 0x5576451e90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:58198/0 (socket says 172.21.15.27:58198) 2024-09-21T00:26:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.079+0000 7fd2e3400640 1 -- 172.21.15.27:0/777448812 learned_addr learned my addr 172.21.15.27:0/777448812 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.080+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/777448812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644e131b0 0x5576451e9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.080+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/777448812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644dea2f0 0x5576451edc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.080+0000 7fd2e3400640 1 -- 172.21.15.27:0/777448812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644e131b0 msgr2=0x5576451e9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.080+0000 7fd2e3400640 1 --2- 172.21.15.27:0/777448812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644e131b0 0x5576451e9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.080+0000 7fd2e3400640 1 -- 172.21.15.27:0/777448812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644dea2f0 msgr2=0x5576451edc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.080+0000 7fd2e3400640 1 --2- 172.21.15.27:0/777448812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644dea2f0 0x5576451edc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.080+0000 7fd2e3400640 1 -- 172.21.15.27:0/777448812 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576451e9b70 con 0x557644df24d0 2024-09-21T00:26:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.088+0000 7fd2e3400640 1 --2- 172.21.15.27:0/777448812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644df24d0 0x5576451e90c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd2d003a040 tx=0x7fd2d00112d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14c59b339f8a8e58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.091+0000 7fd2cd600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:26:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.091+0000 7fd2cd600640 1 -- 172.21.15.27:0/777448812 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d003c040 con 0x557644df24d0 2024-09-21T00:26:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.091+0000 7fd2cd600640 1 -- 172.21.15.27:0/777448812 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7fd2d0034040 con 0x557644df24d0 2024-09-21T00:26:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.108+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/777448812 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2dc028ac0 con 0x557644df24d0 2024-09-21T00:26:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.112+0000 7fd2cd600640 1 -- 172.21.15.27:0/777448812 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd2d000b040 con 0x557644df24d0 2024-09-21T00:26:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.116+0000 7fd2f1a7c740 10 client.25966 fetch_fsmap learned FSMap version 28 2024-09-21T00:26:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.116+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/777448812 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557645261f50 con 0x557644df24d0 2024-09-21T00:26:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.124+0000 7fd2cd600640 1 -- 172.21.15.27:0/777448812 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2dc028ac0 con 0x557644df24d0 2024-09-21T00:26:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.124+0000 7fd2cd600640 1 -- 172.21.15.27:0/777448812 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd2d0037940 con 0x557644df24d0 2024-09-21T00:26:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.125+0000 7fd2f1a7c740 10 client.25966 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:26:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.125+0000 7fd2f1a7c740 10 client.25966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:26:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.125+0000 7fd2f1a7c740 20 client.25966 populate_metadata read hostname 'smithi027' 2024-09-21T00:26:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.125+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/777448812 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557644da71c0 0x557645264680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.125+0000 7fd2f1a7c740 4 client.25966 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:26:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.125+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/777448812 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55764512f600 con 0x557644da71c0 2024-09-21T00:26:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.126+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/777448812 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557644da71c0 0x557645264680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.128+0000 7fd2e8a00640 1 --2- 172.21.15.27:0/777448812 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557644da71c0 0x557645264680 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5810f34321633cf2 server_cookie=687a09f626617685 in_seq=0 out_seq=0 2024-09-21T00:26:02.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:02.139+0000 7fd2cd600640 10 client.25966 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:26:03.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:03 smithi027 ceph-mon[30745]: pgmap v517: 129 pgs: 129 active+clean; 411 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.3 MiB/s wr, 292 op/s 2024-09-21T00:26:03.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:03 smithi089 ceph-mon[34481]: pgmap v517: 129 pgs: 129 active+clean; 411 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.3 MiB/s wr, 292 op/s 2024-09-21T00:26:03.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:03 smithi098 ceph-mon[34725]: pgmap v517: 129 pgs: 129 active+clean; 411 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.3 MiB/s wr, 292 op/s 2024-09-21T00:26:05.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:05 smithi027 ceph-mon[30745]: pgmap v518: 129 pgs: 129 active+clean; 438 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.2 MiB/s wr, 371 op/s 2024-09-21T00:26:05.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:05 smithi089 ceph-mon[34481]: pgmap v518: 129 pgs: 129 active+clean; 438 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.2 MiB/s wr, 371 op/s 2024-09-21T00:26:05.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:05 smithi098 ceph-mon[34725]: pgmap v518: 129 pgs: 129 active+clean; 438 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.2 MiB/s wr, 371 op/s 2024-09-21T00:26:07.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:07 smithi089 ceph-mon[34481]: pgmap v519: 129 pgs: 129 active+clean; 482 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 522 op/s 2024-09-21T00:26:07.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:07 smithi098 ceph-mon[34725]: pgmap v519: 129 pgs: 129 active+clean; 482 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 522 op/s 2024-09-21T00:26:08.071 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:07 smithi027 ceph-mon[30745]: pgmap v519: 129 pgs: 129 active+clean; 482 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 522 op/s 2024-09-21T00:26:08.337 INFO:tasks.workunit.client.0.smithi027.stdout: 151 433059 402 284609 404 127762 491 2024-09-21T00:26:09.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:09 smithi089 ceph-mon[34481]: pgmap v520: 129 pgs: 129 active+clean; 494 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 524 op/s 2024-09-21T00:26:09.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:09 smithi098 ceph-mon[34725]: pgmap v520: 129 pgs: 129 active+clean; 494 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 524 op/s 2024-09-21T00:26:10.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:09 smithi027 ceph-mon[30745]: pgmap v520: 129 pgs: 129 active+clean; 494 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 524 op/s 2024-09-21T00:26:11.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:11 smithi098 ceph-mon[34725]: pgmap v521: 129 pgs: 129 active+clean; 508 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 883 op/s 2024-09-21T00:26:12.073 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:11 smithi027 ceph-mon[30745]: pgmap v521: 129 pgs: 129 active+clean; 508 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 883 op/s 2024-09-21T00:26:12.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:11 smithi089 ceph-mon[34481]: pgmap v521: 129 pgs: 129 active+clean; 508 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 883 op/s 2024-09-21T00:26:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.448+0000 7fd2cd600640 1 -- 172.21.15.27:0/777448812 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55764512f600 con 0x557644da71c0 2024-09-21T00:26:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.448+0000 7fd2cd600640 10 client.25966 handle_command_reply: tid=0 2024-09-21T00:26:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.448+0000 7fd2f1a7c740 1 client.25966 shutdown 2024-09-21T00:26:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.449+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/777448812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644df24d0 msgr2=0x5576451e90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.449+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/777448812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644df24d0 0x5576451e90c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd2d003a040 tx=0x7fd2d00112d0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.449+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/777448812 shutdown_connections 2024-09-21T00:26:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.449+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/777448812 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557644da71c0 0x557645264680 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.449+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/777448812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557644e131b0 0x5576451e9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.449+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/777448812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557644df24d0 0x5576451e90c0 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd2d003a040 tx=0x7fd2d00112d0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.449+0000 7fd2f1a7c740 1 --2- 172.21.15.27:0/777448812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557644dea2f0 0x5576451edc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.449+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/777448812 >> 172.21.15.27:0/777448812 conn(0x557644e07300 msgr2=0x5576451e67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.449+0000 7fd2cd600640 0 client.25966 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:26:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.450+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/777448812 shutdown_connections 2024-09-21T00:26:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.450+0000 7fd2f1a7c740 1 -- 172.21.15.27:0/777448812 wait complete. 2024-09-21T00:26:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.453+0000 7fd2f1a7c740 20 client.25966 trim_cache size 0 max 16384 2024-09-21T00:26:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:26:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:26:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:26:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:26:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.457+0000 7fd2eaa00640 1 -- 172.21.15.27:0/725555371 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd2c80b6800 msgr2=0x7fd2c80b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.457+0000 7fd2eaa00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd2c80b6800 0x7fd2c80b8d10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5576451d88c0 tx=0x7fd2d40e9040 comp rx=0 tx=0).stop 2024-09-21T00:26:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.457+0000 7fd2eaa00640 1 -- 172.21.15.27:0/725555371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e410a660 msgr2=0x7fd2e41a92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.457+0000 7fd2eaa00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e410a660 0x7fd2e41a92b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400c890 tx=0x7fd2d400cd50 comp rx=0 tx=0).stop 2024-09-21T00:26:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.457+0000 7fd2eaa00640 1 -- 172.21.15.27:0/725555371 shutdown_connections 2024-09-21T00:26:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.457+0000 7fd2eaa00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd2c80b6800 0x7fd2c80b8d10 secure :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5576451d88c0 tx=0x7fd2d40e9040 comp rx=0 tx=0).stop 2024-09-21T00:26:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.458+0000 7fd2eaa00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2e41107a0 0x7fd2e41aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.458+0000 7fd2eaa00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2e410fe50 0x7fd2e41a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.458+0000 7fd2eaa00640 1 --2- 172.21.15.27:0/725555371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e410a660 0x7fd2e41a92b0 secure :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400c890 tx=0x7fd2d400cd50 comp rx=0 tx=0).stop 2024-09-21T00:26:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.458+0000 7fd2eaa00640 1 -- 172.21.15.27:0/725555371 >> 172.21.15.27:0/725555371 conn(0x7fd2e4070d90 msgr2=0x7fd2e410df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.458+0000 7fd2eaa00640 1 -- 172.21.15.27:0/725555371 shutdown_connections 2024-09-21T00:26:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:12.459+0000 7fd2eaa00640 1 -- 172.21.15.27:0/725555371 wait complete. 2024-09-21T00:26:13.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:13 smithi098 ceph-mon[34725]: pgmap v522: 129 pgs: 129 active+clean; 513 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-09-21T00:26:14.030 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:13 smithi089 ceph-mon[34481]: pgmap v522: 129 pgs: 129 active+clean; 513 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-09-21T00:26:14.081 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:13 smithi027 ceph-mon[30745]: pgmap v522: 129 pgs: 129 active+clean; 513 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-09-21T00:26:15.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:15 smithi098 ceph-mon[34725]: pgmap v523: 129 pgs: 129 active+clean; 512 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.4 MiB/s wr, 1.22k op/s 2024-09-21T00:26:16.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:15 smithi027 ceph-mon[30745]: pgmap v523: 129 pgs: 129 active+clean; 512 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.4 MiB/s wr, 1.22k op/s 2024-09-21T00:26:16.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:15 smithi089 ceph-mon[34481]: pgmap v523: 129 pgs: 129 active+clean; 512 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.4 MiB/s wr, 1.22k op/s 2024-09-21T00:26:17.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:17 smithi027 ceph-mon[30745]: pgmap v524: 129 pgs: 129 active+clean; 491 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.4 MiB/s wr, 1.22k op/s 2024-09-21T00:26:18.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:17 smithi089 ceph-mon[34481]: pgmap v524: 129 pgs: 129 active+clean; 491 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.4 MiB/s wr, 1.22k op/s 2024-09-21T00:26:18.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:17 smithi098 ceph-mon[34725]: pgmap v524: 129 pgs: 129 active+clean; 491 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.4 MiB/s wr, 1.22k op/s 2024-09-21T00:26:18.339 INFO:tasks.workunit.client.0.smithi027.stdout: 173 373287 1399 244654 1224 100838 140 2024-09-21T00:26:20.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:19 smithi027 ceph-mon[30745]: pgmap v525: 129 pgs: 129 active+clean; 489 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 1.07k op/s 2024-09-21T00:26:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:19 smithi089 ceph-mon[34481]: pgmap v525: 129 pgs: 129 active+clean; 489 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 1.07k op/s 2024-09-21T00:26:20.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:19 smithi098 ceph-mon[34725]: pgmap v525: 129 pgs: 129 active+clean; 489 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 1.07k op/s 2024-09-21T00:26:22.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:21 smithi027 ceph-mon[30745]: pgmap v526: 129 pgs: 129 active+clean; 508 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 1.09k op/s 2024-09-21T00:26:22.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:21 smithi089 ceph-mon[34481]: pgmap v526: 129 pgs: 129 active+clean; 508 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 1.09k op/s 2024-09-21T00:26:22.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:21 smithi098 ceph-mon[34725]: pgmap v526: 129 pgs: 129 active+clean; 508 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 1.09k op/s 2024-09-21T00:26:24.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:23 smithi027 ceph-mon[30745]: pgmap v527: 129 pgs: 129 active+clean; 520 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 738 op/s 2024-09-21T00:26:24.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:23 smithi089 ceph-mon[34481]: pgmap v527: 129 pgs: 129 active+clean; 520 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 738 op/s 2024-09-21T00:26:24.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:23 smithi098 ceph-mon[34725]: pgmap v527: 129 pgs: 129 active+clean; 520 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 738 op/s 2024-09-21T00:26:25.794 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:25 smithi027 ceph-mon[30745]: pgmap v528: 129 pgs: 129 active+clean; 543 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 548 op/s 2024-09-21T00:26:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:25.991+0000 7f6f4f200640 1 -- 172.21.15.27:0/1659828998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f5010a900 msgr2=0x7f6f5010ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:25.991+0000 7f6f4f200640 1 --2- 172.21.15.27:0/1659828998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f5010a900 0x7f6f5010ad10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6f4801acb0 tx=0x7f6f4803b900 comp rx=0 tx=0).stop 2024-09-21T00:26:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:25.991+0000 7f6f4f200640 1 -- 172.21.15.27:0/1659828998 shutdown_connections 2024-09-21T00:26:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:25.991+0000 7f6f4f200640 1 --2- 172.21.15.27:0/1659828998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f5011cba0 0x7f6f5011efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:25.991+0000 7f6f4f200640 1 --2- 172.21.15.27:0/1659828998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f5010a900 0x7f6f5010ad10 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6f4801acb0 tx=0x7f6f4803b900 comp rx=0 tx=0).stop 2024-09-21T00:26:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:25.991+0000 7f6f4f200640 1 --2- 172.21.15.27:0/1659828998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f50111c20 0x7f6f5010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:25.991+0000 7f6f4f200640 1 -- 172.21.15.27:0/1659828998 >> 172.21.15.27:0/1659828998 conn(0x7f6f5006e8f0 msgr2=0x7f6f5006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:25.995+0000 7f6f4f200640 1 -- 172.21.15.27:0/1659828998 shutdown_connections 2024-09-21T00:26:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:25.995+0000 7f6f4f200640 1 -- 172.21.15.27:0/1659828998 wait complete. 2024-09-21T00:26:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:25.999+0000 7f6f4f200640 1 Processor -- start 2024-09-21T00:26:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:25.999+0000 7f6f4f200640 1 -- start start 2024-09-21T00:26:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.001+0000 7f6f4f200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f50111c20 0x7f6f501188f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.001+0000 7f6f4de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f50111c20 0x7f6f501188f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.001+0000 7f6f4de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f50111c20 0x7f6f501188f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39582/0 (socket says 172.21.15.27:39582) 2024-09-21T00:26:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.001+0000 7f6f4f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f5011cba0 0x7f6f50118e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.002+0000 7f6f4d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f5011cba0 0x7f6f50118e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.002+0000 7f6f4d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f5011cba0 0x7f6f50118e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40288/0 (socket says 172.21.15.27:40288) 2024-09-21T00:26:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.002+0000 7f6f4de00640 1 -- 172.21.15.27:0/3244398157 learned_addr learned my addr 172.21.15.27:0/3244398157 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.002+0000 7f6f4f200640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f50119370 0x7f6f501acb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.002+0000 7f6f4f200640 1 -- 172.21.15.27:0/3244398157 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6f50114cf0 con 0x7f6f5011cba0 2024-09-21T00:26:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.002+0000 7f6f4f200640 1 -- 172.21.15.27:0/3244398157 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6f50114b70 con 0x7f6f50119370 2024-09-21T00:26:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.002+0000 7f6f4f200640 1 -- 172.21.15.27:0/3244398157 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6f50114e70 con 0x7f6f50111c20 2024-09-21T00:26:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.003+0000 7f6f4e800640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f50119370 0x7f6f501acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.003+0000 7f6f4d400640 1 -- 172.21.15.27:0/3244398157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f50111c20 msgr2=0x7f6f501188f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.003+0000 7f6f4d400640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f50111c20 0x7f6f501188f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.003+0000 7f6f4d400640 1 -- 172.21.15.27:0/3244398157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f50119370 msgr2=0x7f6f501acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.003+0000 7f6f4d400640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f50119370 0x7f6f501acb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.003+0000 7f6f4d400640 1 -- 172.21.15.27:0/3244398157 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f501ad070 con 0x7f6f5011cba0 2024-09-21T00:26:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.004+0000 7f6f4d400640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f5011cba0 0x7f6f50118e30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6f5010e630 tx=0x7f6f48003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.022+0000 7f6f46a00640 1 -- 172.21.15.27:0/3244398157 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f48045040 con 0x7f6f5011cba0 2024-09-21T00:26:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.022+0000 7f6f46a00640 1 -- 172.21.15.27:0/3244398157 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f48040eb0 con 0x7f6f5011cba0 2024-09-21T00:26:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.022+0000 7f6f46a00640 1 -- 172.21.15.27:0/3244398157 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f48056480 con 0x7f6f5011cba0 2024-09-21T00:26:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.023+0000 7f6f4f200640 1 -- 172.21.15.27:0/3244398157 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f501ad2a0 con 0x7f6f5011cba0 2024-09-21T00:26:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.025+0000 7f6f4f200640 1 -- 172.21.15.27:0/3244398157 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f501ad6d0 con 0x7f6f5011cba0 2024-09-21T00:26:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.059+0000 7f6f46a00640 1 -- 172.21.15.27:0/3244398157 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6f48048020 con 0x7f6f5011cba0 2024-09-21T00:26:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.068+0000 7f6f5c061740 1 Processor -- start 2024-09-21T00:26:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.069+0000 7f6f46a00640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6f340b6890 0x7f6f340b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.069+0000 7f6f46a00640 1 -- 172.21.15.27:0/3244398157 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f6f48119dc0 con 0x7f6f5011cba0 2024-09-21T00:26:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.074+0000 7f6f4e800640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6f340b6890 0x7f6f340b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.077+0000 7f6f4e800640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6f340b6890 0x7f6f340b8da0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00b830 tx=0x7f6f3c008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.083+0000 7f6f5c061740 1 -- start start 2024-09-21T00:26:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.134+0000 7f6f5c061740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e53e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.134+0000 7f6f5c061740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 0x5645b1e54350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.134+0000 7f6f5c061740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 0x5645b1e54890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.134+0000 7f6f5c061740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5645b1dc96b0 con 0x5645b1a862f0 2024-09-21T00:26:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.134+0000 7f6f5c061740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5645b1db8ae0 con 0x5645b1aaf1b0 2024-09-21T00:26:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.134+0000 7f6f5c061740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5645b1dbc8f0 con 0x5645b1a8e4d0 2024-09-21T00:26:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.141+0000 7f6f4d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 0x5645b1e54350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.141+0000 7f6f4d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 0x5645b1e54350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40304/0 (socket says 172.21.15.27:40304) 2024-09-21T00:26:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.141+0000 7f6f4d400640 1 -- 172.21.15.27:0/344302740 learned_addr learned my addr 172.21.15.27:0/344302740 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.158+0000 7f6f4de00640 1 --2- 172.21.15.27:0/344302740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e53e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.159+0000 7f6f4e800640 1 --2- 172.21.15.27:0/344302740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 0x5645b1e54890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.162+0000 7f6f4d400640 1 -- 172.21.15.27:0/344302740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 msgr2=0x5645b1e54890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.163+0000 7f6f4d400640 1 --2- 172.21.15.27:0/344302740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 0x5645b1e54890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.163+0000 7f6f4d400640 1 -- 172.21.15.27:0/344302740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 msgr2=0x5645b1e53e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.163+0000 7f6f4d400640 1 --2- 172.21.15.27:0/344302740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e53e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.163+0000 7f6f4d400640 1 -- 172.21.15.27:0/344302740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b1d85780 con 0x5645b1a862f0 2024-09-21T00:26:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.163+0000 7f6f4d400640 1 --2- 172.21.15.27:0/344302740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 0x5645b1e54350 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6f48002230 tx=0x7f6f480db7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d1417fb7053f56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.168+0000 7f6f4e800640 1 --2- 172.21.15.27:0/344302740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 0x5645b1e54890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.169+0000 7f6f3b400640 1 -- 172.21.15.27:0/344302740 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f480e4070 con 0x5645b1a862f0 2024-09-21T00:26:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.169+0000 7f6f3b400640 1 -- 172.21.15.27:0/344302740 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f480df330 con 0x5645b1a862f0 2024-09-21T00:26:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.169+0000 7f6f3b400640 1 -- 172.21.15.27:0/344302740 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f480e9240 con 0x5645b1a862f0 2024-09-21T00:26:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.171+0000 7f6f5c061740 1 -- 172.21.15.27:0/344302740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 msgr2=0x5645b1e54350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.171+0000 7f6f5c061740 1 --2- 172.21.15.27:0/344302740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 0x5645b1e54350 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6f48002230 tx=0x7f6f480db7c0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.183+0000 7f6f4de00640 1 --2- 172.21.15.27:0/344302740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e53e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.196+0000 7f6f5c061740 1 -- 172.21.15.27:0/344302740 shutdown_connections 2024-09-21T00:26:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.196+0000 7f6f5c061740 1 --2- 172.21.15.27:0/344302740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 0x5645b1e54890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.196+0000 7f6f5c061740 1 --2- 172.21.15.27:0/344302740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 0x5645b1e54350 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.196+0000 7f6f5c061740 1 --2- 172.21.15.27:0/344302740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e53e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.196+0000 7f6f5c061740 1 -- 172.21.15.27:0/344302740 >> 172.21.15.27:0/344302740 conn(0x5645b1aa3300 msgr2=0x5645b1e51d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.202+0000 7f6f5c061740 1 -- 172.21.15.27:0/344302740 shutdown_connections 2024-09-21T00:26:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.202+0000 7f6f5c061740 1 -- 172.21.15.27:0/344302740 wait complete. 2024-09-21T00:26:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.203+0000 7f6f5c061740 1 -- start start 2024-09-21T00:26:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.203+0000 7f6f5c061740 1 Processor -- start 2024-09-21T00:26:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.212+0000 7f6f5c061740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 0x5645b1e5f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.216+0000 7f6f5c061740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 0x5645b1e5f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.217+0000 7f6f4d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 0x5645b1e5f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.217+0000 7f6f4d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 0x5645b1e5f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39596/0 (socket says 172.21.15.27:39596) 2024-09-21T00:26:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.217+0000 7f6f4de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 0x5645b1e5f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.217+0000 7f6f4de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 0x5645b1e5f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40314/0 (socket says 172.21.15.27:40314) 2024-09-21T00:26:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.217+0000 7f6f4d400640 1 -- 172.21.15.27:0/26427243 learned_addr learned my addr 172.21.15.27:0/26427243 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.217+0000 7f6f5c061740 1 --2- 172.21.15.27:0/26427243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e5fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:25 smithi089 ceph-mon[34481]: pgmap v528: 129 pgs: 129 active+clean; 543 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 548 op/s 2024-09-21T00:26:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.218+0000 7f6f4e800640 1 --2- 172.21.15.27:0/26427243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e5fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.218+0000 7f6f5c061740 1 -- 172.21.15.27:0/26427243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5645b1cf0ab0 con 0x5645b1a862f0 2024-09-21T00:26:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.218+0000 7f6f5c061740 1 -- 172.21.15.27:0/26427243 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5645b1d5f570 con 0x5645b1aaf1b0 2024-09-21T00:26:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.218+0000 7f6f5c061740 1 -- 172.21.15.27:0/26427243 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5645b1e62350 con 0x5645b1a8e4d0 2024-09-21T00:26:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.219+0000 7f6f4e800640 1 -- 172.21.15.27:0/26427243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 msgr2=0x5645b1e5f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.219+0000 7f6f4e800640 1 --2- 172.21.15.27:0/26427243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 0x5645b1e5f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.219+0000 7f6f4e800640 1 -- 172.21.15.27:0/26427243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 msgr2=0x5645b1e5f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.219+0000 7f6f4e800640 1 --2- 172.21.15.27:0/26427243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 0x5645b1e5f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.219+0000 7f6f4e800640 1 -- 172.21.15.27:0/26427243 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645b1b1a790 con 0x5645b1aaf1b0 2024-09-21T00:26:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.220+0000 7f6f5c061740 10 client.0 _ll_register_callbacks cb 0x5645b1d6e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:26:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.220+0000 7f6f5c061740 1 -- 172.21.15.27:0/26427243 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f480df330 con 0x5645b1aaf1b0 2024-09-21T00:26:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.228+0000 7f6f4e800640 1 --2- 172.21.15.27:0/26427243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e5fe50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00b610 tx=0x7f6f3c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=930dd70c970df17f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.229+0000 7f6f2fe00640 10 client.25997 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:26:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.240+0000 7f6f2fe00640 1 -- 172.21.15.27:0/26427243 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c014e40 con 0x5645b1aaf1b0 2024-09-21T00:26:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.240+0000 7f6f2fe00640 1 -- 172.21.15.27:0/26427243 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f6f3c0059c0 con 0x5645b1aaf1b0 2024-09-21T00:26:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.240+0000 7f6f2fe00640 1 -- 172.21.15.27:0/26427243 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f3c005c00 con 0x5645b1aaf1b0 2024-09-21T00:26:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.243+0000 7f6f5c061740 10 client.25997 fetch_fsmap learned FSMap version 28 2024-09-21T00:26:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.243+0000 7f6f5c061740 1 -- 172.21.15.27:0/26427243 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5645b1b93620 con 0x5645b1aaf1b0 2024-09-21T00:26:26.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:25 smithi098 ceph-mon[34725]: pgmap v528: 129 pgs: 129 active+clean; 543 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 548 op/s 2024-09-21T00:26:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.245+0000 7f6f2fe00640 1 -- 172.21.15.27:0/26427243 <== mon.2 v2:172.21.15.89:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6f3c013070 con 0x5645b1aaf1b0 2024-09-21T00:26:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.250+0000 7f6f5c061740 10 client.25997 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:26:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.250+0000 7f6f5c061740 10 client.25997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:26:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.250+0000 7f6f5c061740 20 client.25997 populate_metadata read hostname 'smithi027' 2024-09-21T00:26:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.250+0000 7f6f5c061740 1 --2- 172.21.15.27:0/26427243 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5645b1a6dd80 0x5645b1e70fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.250+0000 7f6f5c061740 4 client.25997 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:26:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.250+0000 7f6f5c061740 1 -- 172.21.15.27:0/26427243 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645b1dcb600 con 0x5645b1a6dd80 2024-09-21T00:26:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.252+0000 7f6f4de00640 1 --2- 172.21.15.27:0/26427243 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5645b1a6dd80 0x5645b1e70fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.265+0000 7f6f4de00640 1 --2- 172.21.15.27:0/26427243 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5645b1a6dd80 0x5645b1e70fa0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f28132c6367aec0 server_cookie=edba5ca7954c44e7 in_seq=0 out_seq=0 2024-09-21T00:26:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.265+0000 7f6f2fe00640 10 client.25997 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:26:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.266+0000 7f6f2fe00640 1 -- 172.21.15.27:0/26427243 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5645b1dcb600 con 0x5645b1a6dd80 2024-09-21T00:26:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.266+0000 7f6f2fe00640 10 client.25997 handle_command_reply: tid=0 2024-09-21T00:26:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.270+0000 7f6f5c061740 1 client.25997 shutdown 2024-09-21T00:26:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.284+0000 7f6f5c061740 1 -- 172.21.15.27:0/26427243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 msgr2=0x5645b1e5fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.284+0000 7f6f5c061740 1 --2- 172.21.15.27:0/26427243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e5fe50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00b610 tx=0x7f6f3c0046c0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.284+0000 7f6f5c061740 1 -- 172.21.15.27:0/26427243 shutdown_connections 2024-09-21T00:26:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.284+0000 7f6f5c061740 1 --2- 172.21.15.27:0/26427243 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5645b1a6dd80 0x5645b1e70fa0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.284+0000 7f6f5c061740 1 --2- 172.21.15.27:0/26427243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e5fe50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.284+0000 7f6f5c061740 1 --2- 172.21.15.27:0/26427243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 0x5645b1e5f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.284+0000 7f6f5c061740 1 --2- 172.21.15.27:0/26427243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 0x5645b1e5f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.284+0000 7f6f5c061740 1 -- 172.21.15.27:0/26427243 >> 172.21.15.27:0/26427243 conn(0x5645b1aa3300 msgr2=0x5645b1e54e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.284+0000 7f6f2fe00640 0 client.25997 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:26:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.285+0000 7f6f5c061740 1 -- 172.21.15.27:0/26427243 shutdown_connections 2024-09-21T00:26:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.291+0000 7f6f5c061740 1 -- 172.21.15.27:0/26427243 wait complete. 2024-09-21T00:26:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.293+0000 7f6f5c061740 20 client.25997 trim_cache size 0 max 16384 2024-09-21T00:26:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.671+0000 7f6f5c061740 1 Processor -- start 2024-09-21T00:26:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.692+0000 7f6f5c061740 1 -- start start 2024-09-21T00:26:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.692+0000 7f6f5c061740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 0x5645b1e55500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.692+0000 7f6f5c061740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 0x5645b1e5c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.692+0000 7f6f5c061740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e5cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.693+0000 7f6f4d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 0x5645b1e5c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.693+0000 7f6f4d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 0x5645b1e5c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39600/0 (socket says 172.21.15.27:39600) 2024-09-21T00:26:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.693+0000 7f6f5c061740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6f501ba1d0 con 0x5645b1a862f0 2024-09-21T00:26:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.693+0000 7f6f5c061740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6f501b7aa0 con 0x5645b1aaf1b0 2024-09-21T00:26:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.693+0000 7f6f5c061740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6f501b58f0 con 0x5645b1a8e4d0 2024-09-21T00:26:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.694+0000 7f6f4e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e5cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.694+0000 7f6f4e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e5cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37624/0 (socket says 172.21.15.27:37624) 2024-09-21T00:26:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.694+0000 7f6f4e800640 1 -- 172.21.15.27:0/3663899412 learned_addr learned my addr 172.21.15.27:0/3663899412 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.694+0000 7f6f4e800640 1 -- 172.21.15.27:0/3663899412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 msgr2=0x5645b1e5c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.695+0000 7f6f4e800640 1 --2- 172.21.15.27:0/3663899412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 0x5645b1e5c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.695+0000 7f6f4e800640 1 -- 172.21.15.27:0/3663899412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 msgr2=0x5645b1e55500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.695+0000 7f6f4e800640 1 --2- 172.21.15.27:0/3663899412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 0x5645b1e55500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.695+0000 7f6f4e800640 1 -- 172.21.15.27:0/3663899412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b1e568e0 con 0x5645b1aaf1b0 2024-09-21T00:26:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.695+0000 7f6f4e800640 1 --2- 172.21.15.27:0/3663899412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e5cae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00d630 tx=0x7f6f3c01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16868b8a5c4c8e4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.705+0000 7f6f3aa00640 1 -- 172.21.15.27:0/3663899412 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c036070 con 0x5645b1aaf1b0 2024-09-21T00:26:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.705+0000 7f6f3aa00640 1 -- 172.21.15.27:0/3663899412 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f480df330 con 0x5645b1aaf1b0 2024-09-21T00:26:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.705+0000 7f6f3aa00640 1 -- 172.21.15.27:0/3663899412 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c0285f0 con 0x5645b1aaf1b0 2024-09-21T00:26:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.716+0000 7f6f5c061740 1 -- 172.21.15.27:0/3663899412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 msgr2=0x5645b1e5cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.716+0000 7f6f5c061740 1 --2- 172.21.15.27:0/3663899412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e5cae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00d630 tx=0x7f6f3c01d000 comp rx=0 tx=0).stop 2024-09-21T00:26:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.722+0000 7f6f5c061740 1 -- 172.21.15.27:0/3663899412 shutdown_connections 2024-09-21T00:26:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.722+0000 7f6f5c061740 1 --2- 172.21.15.27:0/3663899412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e5cae0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.723+0000 7f6f5c061740 1 --2- 172.21.15.27:0/3663899412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a8e4d0 0x5645b1e5c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.723+0000 7f6f5c061740 1 --2- 172.21.15.27:0/3663899412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a862f0 0x5645b1e55500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.724+0000 7f6f5c061740 1 -- 172.21.15.27:0/3663899412 >> 172.21.15.27:0/3663899412 conn(0x5645b1aa3300 msgr2=0x5645b1e54df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.734+0000 7f6f5c061740 1 -- 172.21.15.27:0/3663899412 shutdown_connections 2024-09-21T00:26:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.750+0000 7f6f5c061740 1 -- 172.21.15.27:0/3663899412 wait complete. 2024-09-21T00:26:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.750+0000 7f6f5c061740 1 -- start start 2024-09-21T00:26:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.751+0000 7f6f5c061740 1 Processor -- start 2024-09-21T00:26:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.755+0000 7f6f5c061740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a862f0 0x5645b1e67ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.755+0000 7f6f5c061740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a8e4d0 0x5645b1e653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.755+0000 7f6f5c061740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e65ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.757+0000 7f6f5c061740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6f501b5f60 con 0x5645b1a8e4d0 2024-09-21T00:26:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.758+0000 7f6f4e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e65ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.758+0000 7f6f4e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e65ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37628/0 (socket says 172.21.15.27:37628) 2024-09-21T00:26:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.758+0000 7f6f4e800640 1 -- 172.21.15.27:0/3017677389 learned_addr learned my addr 172.21.15.27:0/3017677389 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.758+0000 7f6f5c061740 1 -- 172.21.15.27:0/3017677389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6f501b73d0 con 0x5645b1aaf1b0 2024-09-21T00:26:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.758+0000 7f6f5c061740 1 -- 172.21.15.27:0/3017677389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6f501b6630 con 0x5645b1a862f0 2024-09-21T00:26:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.763+0000 7f6f5c061740 10 client.0 _ll_register_callbacks cb 0x7f6f3c00bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:26:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.763+0000 7f6f4de00640 1 --2- 172.21.15.27:0/3017677389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a862f0 0x5645b1e67ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.763+0000 7f6f4d400640 1 --2- 172.21.15.27:0/3017677389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a8e4d0 0x5645b1e653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.771+0000 7f6f4e800640 1 -- 172.21.15.27:0/3017677389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a862f0 msgr2=0x5645b1e67ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.771+0000 7f6f4e800640 1 --2- 172.21.15.27:0/3017677389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a862f0 0x5645b1e67ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.771+0000 7f6f4e800640 1 -- 172.21.15.27:0/3017677389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a8e4d0 msgr2=0x5645b1e653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.771+0000 7f6f4e800640 1 --2- 172.21.15.27:0/3017677389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a8e4d0 0x5645b1e653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.771+0000 7f6f4e800640 1 -- 172.21.15.27:0/3017677389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645b1eda9b0 con 0x5645b1aaf1b0 2024-09-21T00:26:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.772+0000 7f6f4d400640 1 --2- 172.21.15.27:0/3017677389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a8e4d0 0x5645b1e653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.772+0000 7f6f4e800640 1 --2- 172.21.15.27:0/3017677389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e65ad0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c014c90 tx=0x7f6f3c006690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a2205ec16de28bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.772+0000 7f6f2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:26:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.772+0000 7f6f2fe00640 1 -- 172.21.15.27:0/3017677389 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c03f060 con 0x5645b1aaf1b0 2024-09-21T00:26:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.773+0000 7f6f2fe00640 1 -- 172.21.15.27:0/3017677389 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f6f3c0127f0 con 0x5645b1aaf1b0 2024-09-21T00:26:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.778+0000 7f6f5c061740 1 -- 172.21.15.27:0/3017677389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f480df330 con 0x5645b1aaf1b0 2024-09-21T00:26:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.785+0000 7f6f2fe00640 1 -- 172.21.15.27:0/3017677389 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f3c036040 con 0x5645b1aaf1b0 2024-09-21T00:26:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.786+0000 7f6f5c061740 10 client.26003 fetch_fsmap learned FSMap version 28 2024-09-21T00:26:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.786+0000 7f6f5c061740 1 -- 172.21.15.27:0/3017677389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5645b1edba30 con 0x5645b1aaf1b0 2024-09-21T00:26:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.791+0000 7f6f2fe00640 1 -- 172.21.15.27:0/3017677389 <== mon.2 v2:172.21.15.89:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6f3c0129f0 con 0x5645b1aaf1b0 2024-09-21T00:26:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.791+0000 7f6f5c061740 10 client.26003 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T00:26:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.791+0000 7f6f5c061740 10 client.26003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:26:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.791+0000 7f6f5c061740 20 client.26003 populate_metadata read hostname 'smithi027' 2024-09-21T00:26:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.791+0000 7f6f5c061740 1 --2- 172.21.15.27:0/3017677389 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5645b1a431c0 0x5645b1eddee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.791+0000 7f6f5c061740 4 client.26003 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:26:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.791+0000 7f6f5c061740 1 -- 172.21.15.27:0/3017677389 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645b1dc70b0 con 0x5645b1a431c0 2024-09-21T00:26:26.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.800+0000 7f6f4de00640 1 --2- 172.21.15.27:0/3017677389 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5645b1a431c0 0x5645b1eddee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.812+0000 7f6f4de00640 1 --2- 172.21.15.27:0/3017677389 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5645b1a431c0 0x5645b1eddee0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5085f3235224cbf5 server_cookie=cc88b19792673e95 in_seq=0 out_seq=0 2024-09-21T00:26:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:26.812+0000 7f6f2fe00640 10 client.26003 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:26:28.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:27 smithi027 ceph-mon[30745]: pgmap v529: 129 pgs: 129 active+clean; 585 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.9 MiB/s wr, 504 op/s 2024-09-21T00:26:28.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:27 smithi089 ceph-mon[34481]: pgmap v529: 129 pgs: 129 active+clean; 585 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.9 MiB/s wr, 504 op/s 2024-09-21T00:26:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:27 smithi098 ceph-mon[34725]: pgmap v529: 129 pgs: 129 active+clean; 585 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.9 MiB/s wr, 504 op/s 2024-09-21T00:26:28.342 INFO:tasks.workunit.client.0.smithi027.stdout: 185 452390 674 298735 576 106320 338 2024-09-21T00:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.017+0000 7f6f45600640 1 -- 172.21.15.27:0/3244398157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f5011cba0 msgr2=0x7f6f50118e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.017+0000 7f6f45600640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f5011cba0 0x7f6f50118e30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6f5010e630 tx=0x7f6f48003ba0 comp rx=0 tx=0).stop 2024-09-21T00:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.017+0000 7f6f45600640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f50111c20 0x7f6f280056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.017+0000 7f6f45600640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f50119370 0x7f6f280081c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.017+0000 7f6f45600640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f28008c20 0x7f6f2800b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.017+0000 7f6f45600640 1 -- 172.21.15.27:0/3244398157 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2800bc00 con 0x7f6f28008c20 2024-09-21T00:26:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.017+0000 7f6f45600640 1 -- 172.21.15.27:0/3244398157 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2800bd80 con 0x7f6f50119370 2024-09-21T00:26:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.017+0000 7f6f45600640 1 -- 172.21.15.27:0/3244398157 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2800bf80 con 0x7f6f50111c20 2024-09-21T00:26:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.025+0000 7f6f4de00640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f28008c20 0x7f6f2800b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.034+0000 7f6f4d400640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f50119370 0x7f6f280081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.034+0000 7f6f4d400640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f50111c20 0x7f6f280056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.042+0000 7f6f4de00640 1 -- 172.21.15.27:0/3244398157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f50111c20 msgr2=0x7f6f280056f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.042+0000 7f6f4de00640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f50111c20 0x7f6f280056f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.042+0000 7f6f4de00640 1 -- 172.21.15.27:0/3244398157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f50119370 msgr2=0x7f6f280081c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.042+0000 7f6f4de00640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f50119370 0x7f6f280081c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.042+0000 7f6f4de00640 1 -- 172.21.15.27:0/3244398157 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=4+}) -- 0x7f6f2800c2a0 con 0x7f6f28008c20 2024-09-21T00:26:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.043+0000 7f6f4de00640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f28008c20 0x7f6f2800b100 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x5645b1e716e0 tx=0x7f6f4801a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:29.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.046+0000 7f6f46a00640 1 -- 172.21.15.27:0/3244398157 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4000f040 con 0x7f6f28008c20 2024-09-21T00:26:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.046+0000 7f6f46a00640 1 -- 172.21.15.27:0/3244398157 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5645b1dc70b0 con 0x7f6f28008c20 2024-09-21T00:26:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:29.078+0000 7f6f46a00640 1 -- 172.21.15.27:0/3244398157 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6f400143b0 con 0x7f6f28008c20 2024-09-21T00:26:30.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:29 smithi027 ceph-mon[30745]: pgmap v530: 129 pgs: 129 active+clean; 595 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.4 MiB/s wr, 482 op/s 2024-09-21T00:26:30.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:29 smithi089 ceph-mon[34481]: pgmap v530: 129 pgs: 129 active+clean; 595 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.4 MiB/s wr, 482 op/s 2024-09-21T00:26:30.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:29 smithi098 ceph-mon[34725]: pgmap v530: 129 pgs: 129 active+clean; 595 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.4 MiB/s wr, 482 op/s 2024-09-21T00:26:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.901+0000 7f6f2fe00640 1 -- 172.21.15.27:0/3017677389 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f6f400ad950 con 0x5645b1a431c0 2024-09-21T00:26:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.901+0000 7f6f2fe00640 10 client.26003 handle_command_reply: tid=0 2024-09-21T00:26:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.901+0000 7f6f5c061740 1 client.26003 shutdown 2024-09-21T00:26:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.901+0000 7f6f5c061740 1 -- 172.21.15.27:0/3017677389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 msgr2=0x5645b1e65ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.901+0000 7f6f5c061740 1 --2- 172.21.15.27:0/3017677389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e65ad0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c014c90 tx=0x7f6f3c006690 comp rx=0 tx=0).stop 2024-09-21T00:26:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.901+0000 7f6f5c061740 1 -- 172.21.15.27:0/3017677389 shutdown_connections 2024-09-21T00:26:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.901+0000 7f6f5c061740 1 --2- 172.21.15.27:0/3017677389 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5645b1a431c0 0x5645b1eddee0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.901+0000 7f6f5c061740 1 --2- 172.21.15.27:0/3017677389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645b1aaf1b0 0x5645b1e65ad0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.901+0000 7f6f5c061740 1 --2- 172.21.15.27:0/3017677389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645b1a8e4d0 0x5645b1e653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.901+0000 7f6f5c061740 1 --2- 172.21.15.27:0/3017677389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645b1a862f0 0x5645b1e67ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.901+0000 7f6f5c061740 1 -- 172.21.15.27:0/3017677389 >> 172.21.15.27:0/3017677389 conn(0x5645b1aa3300 msgr2=0x5645b1e6a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.902+0000 7f6f2fe00640 0 client.26003 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:26:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.902+0000 7f6f5c061740 1 -- 172.21.15.27:0/3017677389 shutdown_connections 2024-09-21T00:26:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.902+0000 7f6f5c061740 1 -- 172.21.15.27:0/3017677389 wait complete. 2024-09-21T00:26:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.902+0000 7f6f5c061740 20 client.26003 trim_cache size 0 max 16384 2024-09-21T00:26:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:26:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:26:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:26:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:26:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.915+0000 7f6f4f200640 1 -- 172.21.15.27:0/3244398157 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6f340b6890 msgr2=0x7f6f340b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.915+0000 7f6f4f200640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6f340b6890 0x7f6f340b8da0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00b830 tx=0x7f6f3c008040 comp rx=0 tx=0).stop 2024-09-21T00:26:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.915+0000 7f6f4f200640 1 -- 172.21.15.27:0/3244398157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f28008c20 msgr2=0x7f6f2800b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.915+0000 7f6f4f200640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f28008c20 0x7f6f2800b100 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x5645b1e716e0 tx=0x7f6f4801a060 comp rx=0 tx=0).stop 2024-09-21T00:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.916+0000 7f6f4f200640 1 -- 172.21.15.27:0/3244398157 shutdown_connections 2024-09-21T00:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.916+0000 7f6f4f200640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f28008c20 0x7f6f2800b100 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.916+0000 7f6f4f200640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f50119370 0x7f6f280081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.916+0000 7f6f4f200640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f50111c20 0x7f6f280056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.916+0000 7f6f4f200640 1 --2- 172.21.15.27:0/3244398157 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6f340b6890 0x7f6f340b8da0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.916+0000 7f6f4f200640 1 -- 172.21.15.27:0/3244398157 >> 172.21.15.27:0/3244398157 conn(0x7f6f5006e8f0 msgr2=0x7f6f5010b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.917+0000 7f6f4f200640 1 -- 172.21.15.27:0/3244398157 shutdown_connections 2024-09-21T00:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:30.917+0000 7f6f4f200640 1 -- 172.21.15.27:0/3244398157 wait complete. 2024-09-21T00:26:32.071 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:31 smithi027 ceph-mon[30745]: pgmap v531: 129 pgs: 129 active+clean; 606 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 750 op/s 2024-09-21T00:26:32.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:31 smithi089 ceph-mon[34481]: pgmap v531: 129 pgs: 129 active+clean; 606 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 750 op/s 2024-09-21T00:26:32.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:31 smithi098 ceph-mon[34725]: pgmap v531: 129 pgs: 129 active+clean; 606 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 750 op/s 2024-09-21T00:26:34.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:33 smithi089 ceph-mon[34481]: pgmap v532: 129 pgs: 129 active+clean; 602 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.8 MiB/s wr, 942 op/s 2024-09-21T00:26:34.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:33 smithi098 ceph-mon[34725]: pgmap v532: 129 pgs: 129 active+clean; 602 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.8 MiB/s wr, 942 op/s 2024-09-21T00:26:34.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:33 smithi027 ceph-mon[30745]: pgmap v532: 129 pgs: 129 active+clean; 602 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.8 MiB/s wr, 942 op/s 2024-09-21T00:26:36.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:35 smithi089 ceph-mon[34481]: pgmap v533: 129 pgs: 129 active+clean; 603 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.8 MiB/s wr, 945 op/s 2024-09-21T00:26:36.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:35 smithi098 ceph-mon[34725]: pgmap v533: 129 pgs: 129 active+clean; 603 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.8 MiB/s wr, 945 op/s 2024-09-21T00:26:36.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:35 smithi027 ceph-mon[30745]: pgmap v533: 129 pgs: 129 active+clean; 603 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.8 MiB/s wr, 945 op/s 2024-09-21T00:26:38.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:37 smithi027 ceph-mon[30745]: pgmap v534: 129 pgs: 129 active+clean; 599 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 903 op/s 2024-09-21T00:26:38.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:37 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:26:38.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:37 smithi089 ceph-mon[34481]: pgmap v534: 129 pgs: 129 active+clean; 599 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 903 op/s 2024-09-21T00:26:38.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:37 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:26:38.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:37 smithi098 ceph-mon[34725]: pgmap v534: 129 pgs: 129 active+clean; 599 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 903 op/s 2024-09-21T00:26:38.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:37 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:26:38.343 INFO:tasks.workunit.client.0.smithi027.stdout: 206 504238 986 336541 1109 125624 585 2024-09-21T00:26:39.974 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:39 smithi027 ceph-mon[30745]: pgmap v535: 129 pgs: 129 active+clean; 605 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 774 op/s 2024-09-21T00:26:40.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:39 smithi089 ceph-mon[34481]: pgmap v535: 129 pgs: 129 active+clean; 605 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 774 op/s 2024-09-21T00:26:40.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:39 smithi098 ceph-mon[34725]: pgmap v535: 129 pgs: 129 active+clean; 605 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 774 op/s 2024-09-21T00:26:42.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:41 smithi089 ceph-mon[34481]: pgmap v536: 129 pgs: 129 active+clean; 623 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 776 op/s 2024-09-21T00:26:42.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:41 smithi098 ceph-mon[34725]: pgmap v536: 129 pgs: 129 active+clean; 623 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 776 op/s 2024-09-21T00:26:42.321 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:41 smithi027 ceph-mon[30745]: pgmap v536: 129 pgs: 129 active+clean; 623 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 776 op/s 2024-09-21T00:26:43.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:42 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:26:43.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:42 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:26:43.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:42 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:26:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:43 smithi027 ceph-mon[30745]: pgmap v537: 129 pgs: 129 active+clean; 647 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 598 op/s 2024-09-21T00:26:44.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:43 smithi089 ceph-mon[34481]: pgmap v537: 129 pgs: 129 active+clean; 647 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 598 op/s 2024-09-21T00:26:44.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:43 smithi098 ceph-mon[34725]: pgmap v537: 129 pgs: 129 active+clean; 647 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 598 op/s 2024-09-21T00:26:44.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.634+0000 7f64ffe00640 1 -- 172.21.15.27:0/373773206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f650010ff80 msgr2=0x7f65001123a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:44.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.634+0000 7f64ffe00640 1 --2- 172.21.15.27:0/373773206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f650010ff80 0x7f65001123a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009ca0 tx=0x7f64f802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.634+0000 7f64ffe00640 1 -- 172.21.15.27:0/373773206 shutdown_connections 2024-09-21T00:26:44.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.634+0000 7f64ffe00640 1 --2- 172.21.15.27:0/373773206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f650010ff80 0x7f65001123a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.634+0000 7f64ffe00640 1 --2- 172.21.15.27:0/373773206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6500102d20 0x7f650010fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.634+0000 7f64ffe00640 1 --2- 172.21.15.27:0/373773206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65001023d0 0x7f65001027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.634+0000 7f64ffe00640 1 -- 172.21.15.27:0/373773206 >> 172.21.15.27:0/373773206 conn(0x7f65000fe210 msgr2=0x7f6500100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.634+0000 7f64ffe00640 1 -- 172.21.15.27:0/373773206 shutdown_connections 2024-09-21T00:26:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.634+0000 7f64ffe00640 1 -- 172.21.15.27:0/373773206 wait complete. 2024-09-21T00:26:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.635+0000 7f64ffe00640 1 Processor -- start 2024-09-21T00:26:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.635+0000 7f64ffe00640 1 -- start start 2024-09-21T00:26:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65001023d0 0x7f650019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6500102d20 0x7f650019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f650010ff80 0x7f65001a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64ffe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6500115a70 con 0x7f65001023d0 2024-09-21T00:26:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64ffe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f65001158f0 con 0x7f650010ff80 2024-09-21T00:26:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64ffe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6500115bf0 con 0x7f6500102d20 2024-09-21T00:26:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64fe000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6500102d20 0x7f650019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64fea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65001023d0 0x7f650019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64ff400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f650010ff80 0x7f65001a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64fe000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6500102d20 0x7f650019c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49974/0 (socket says 172.21.15.27:49974) 2024-09-21T00:26:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64fe000640 1 -- 172.21.15.27:0/2510464259 learned_addr learned my addr 172.21.15.27:0/2510464259 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64fea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65001023d0 0x7f650019c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58758/0 (socket says 172.21.15.27:58758) 2024-09-21T00:26:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64ff400640 1 -- 172.21.15.27:0/2510464259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6500102d20 msgr2=0x7f650019c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64ff400640 1 --2- 172.21.15.27:0/2510464259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6500102d20 0x7f650019c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64ff400640 1 -- 172.21.15.27:0/2510464259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65001023d0 msgr2=0x7f650019c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64ff400640 1 --2- 172.21.15.27:0/2510464259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65001023d0 0x7f650019c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.636+0000 7f64ff400640 1 -- 172.21.15.27:0/2510464259 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65001a2340 con 0x7f650010ff80 2024-09-21T00:26:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.637+0000 7f64ff400640 1 --2- 172.21.15.27:0/2510464259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f650010ff80 0x7f65001a1c40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f64f8005e30 tx=0x7f64f80317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.637+0000 7f64f7400640 1 -- 172.21.15.27:0/2510464259 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8003b70 con 0x7f650010ff80 2024-09-21T00:26:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.637+0000 7f64ffe00640 1 -- 172.21.15.27:0/2510464259 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65001a25d0 con 0x7f650010ff80 2024-09-21T00:26:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.637+0000 7f64f7400640 1 -- 172.21.15.27:0/2510464259 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64f8003d10 con 0x7f650010ff80 2024-09-21T00:26:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.637+0000 7f64f7400640 1 -- 172.21.15.27:0/2510464259 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f802f970 con 0x7f650010ff80 2024-09-21T00:26:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.637+0000 7f64ffe00640 1 -- 172.21.15.27:0/2510464259 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65001a8dc0 con 0x7f650010ff80 2024-09-21T00:26:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.642+0000 7f650cc0d740 1 Processor -- start 2024-09-21T00:26:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.643+0000 7f64f7400640 1 -- 172.21.15.27:0/2510464259 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f64f802fb10 con 0x7f650010ff80 2024-09-21T00:26:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.643+0000 7f650cc0d740 1 -- start start 2024-09-21T00:26:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.643+0000 7f650cc0d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a6522e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.643+0000 7f650cc0d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a6523350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.643+0000 7f650cc0d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 0x5555a6523890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.643+0000 7f650cc0d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5555a64986b0 con 0x5555a61552f0 2024-09-21T00:26:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.643+0000 7f650cc0d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5555a6487ae0 con 0x5555a617e1b0 2024-09-21T00:26:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.643+0000 7f650cc0d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5555a648b8f0 con 0x5555a615d4d0 2024-09-21T00:26:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.643+0000 7f64fe000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a6522e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.643+0000 7f64fea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a6523350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.644+0000 7f64fe000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a6522e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:36554/0 (socket says 172.21.15.27:36554) 2024-09-21T00:26:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.644+0000 7f64fe000640 1 -- 172.21.15.27:0/2247879233 learned_addr learned my addr 172.21.15.27:0/2247879233 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.644+0000 7f64fe000640 1 --2- 172.21.15.27:0/2247879233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 0x5555a6523890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.644+0000 7f64fe000640 1 -- 172.21.15.27:0/2247879233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 msgr2=0x5555a6523890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.644+0000 7f64fe000640 1 --2- 172.21.15.27:0/2247879233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 0x5555a6523890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.644+0000 7f64fe000640 1 -- 172.21.15.27:0/2247879233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 msgr2=0x5555a6523350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.644+0000 7f64fe000640 1 --2- 172.21.15.27:0/2247879233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a6523350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.644+0000 7f64fe000640 1 -- 172.21.15.27:0/2247879233 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555a6454780 con 0x5555a617e1b0 2024-09-21T00:26:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.644+0000 7f64fea00640 1 --2- 172.21.15.27:0/2247879233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a6523350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:26:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.644+0000 7f64fe000640 1 --2- 172.21.15.27:0/2247879233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a6522e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f64ec011860 tx=0x7f64ec009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa6b155b0ee4794d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.644+0000 7f64f7400640 1 --2- 172.21.15.27:0/2510464259 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f64e40b66f0 0x7f64e40b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.645+0000 7f64f7400640 1 -- 172.21.15.27:0/2510464259 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f64f8108a80 con 0x7f650010ff80 2024-09-21T00:26:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.645+0000 7f64ebe00640 1 -- 172.21.15.27:0/2247879233 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec005020 con 0x5555a617e1b0 2024-09-21T00:26:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.645+0000 7f64fea00640 1 --2- 172.21.15.27:0/2510464259 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f64e40b66f0 0x7f64e40b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.645+0000 7f64ebe00640 1 -- 172.21.15.27:0/2247879233 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64ec0047a0 con 0x5555a617e1b0 2024-09-21T00:26:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.645+0000 7f64ebe00640 1 -- 172.21.15.27:0/2247879233 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec00fa00 con 0x5555a617e1b0 2024-09-21T00:26:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.646+0000 7f650cc0d740 1 -- 172.21.15.27:0/2247879233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 msgr2=0x5555a6522e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.646+0000 7f650cc0d740 1 --2- 172.21.15.27:0/2247879233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a6522e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f64ec011860 tx=0x7f64ec009550 comp rx=0 tx=0).stop 2024-09-21T00:26:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.646+0000 7f64fea00640 1 --2- 172.21.15.27:0/2510464259 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f64e40b66f0 0x7f64e40b8c00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x5555a6521a90 tx=0x7f64f0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.646+0000 7f650cc0d740 1 -- 172.21.15.27:0/2247879233 shutdown_connections 2024-09-21T00:26:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.646+0000 7f650cc0d740 1 --2- 172.21.15.27:0/2247879233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 0x5555a6523890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.646+0000 7f650cc0d740 1 --2- 172.21.15.27:0/2247879233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a6523350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.646+0000 7f650cc0d740 1 --2- 172.21.15.27:0/2247879233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a6522e10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.646+0000 7f650cc0d740 1 -- 172.21.15.27:0/2247879233 >> 172.21.15.27:0/2247879233 conn(0x5555a6172300 msgr2=0x5555a6520d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.646+0000 7f650cc0d740 1 -- 172.21.15.27:0/2247879233 shutdown_connections 2024-09-21T00:26:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.646+0000 7f650cc0d740 1 -- 172.21.15.27:0/2247879233 wait complete. 2024-09-21T00:26:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.647+0000 7f650cc0d740 1 -- start start 2024-09-21T00:26:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.647+0000 7f650cc0d740 1 Processor -- start 2024-09-21T00:26:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f650cc0d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a652e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f650cc0d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 0x5555a652e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f650cc0d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a653a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f650cc0d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5555a63bfab0 con 0x5555a61552f0 2024-09-21T00:26:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f650cc0d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5555a642e570 con 0x5555a617e1b0 2024-09-21T00:26:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f650cc0d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5555a6531590 con 0x5555a615d4d0 2024-09-21T00:26:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f650cc0d740 10 client.0 _ll_register_callbacks cb 0x5555a643d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:26:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f64fe000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 0x5555a652e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f64fea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a652e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f64fe000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 0x5555a652e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49994/0 (socket says 172.21.15.27:49994) 2024-09-21T00:26:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f64fe000640 1 -- 172.21.15.27:0/1728708081 learned_addr learned my addr 172.21.15.27:0/1728708081 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f64ff400640 1 --2- 172.21.15.27:0/1728708081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a653a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f64fe000640 1 -- 172.21.15.27:0/1728708081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 msgr2=0x5555a653a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f64fe000640 1 --2- 172.21.15.27:0/1728708081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a653a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f64fe000640 1 -- 172.21.15.27:0/1728708081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 msgr2=0x5555a652e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f64fe000640 1 --2- 172.21.15.27:0/1728708081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a652e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f64fe000640 1 -- 172.21.15.27:0/1728708081 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555a61e9790 con 0x5555a615d4d0 2024-09-21T00:26:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.648+0000 7f64fea00640 1 --2- 172.21.15.27:0/1728708081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a652e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:26:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.649+0000 7f64fe000640 1 --2- 172.21.15.27:0/1728708081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 0x5555a652e9d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f64ec006010 tx=0x7f64ec004e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d662531c48fb643 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.649+0000 7f64e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:26:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.649+0000 7f64e8c00640 1 -- 172.21.15.27:0/1728708081 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec02c060 con 0x5555a615d4d0 2024-09-21T00:26:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.649+0000 7f64e8c00640 1 -- 172.21.15.27:0/1728708081 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f64ec004310 con 0x5555a615d4d0 2024-09-21T00:26:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.649+0000 7f650cc0d740 1 -- 172.21.15.27:0/1728708081 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64ec0047a0 con 0x5555a615d4d0 2024-09-21T00:26:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.649+0000 7f64e8c00640 1 -- 172.21.15.27:0/1728708081 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f64ec033050 con 0x5555a615d4d0 2024-09-21T00:26:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.649+0000 7f650cc0d740 10 client.25993 fetch_fsmap learned FSMap version 30 2024-09-21T00:26:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.649+0000 7f650cc0d740 1 -- 172.21.15.27:0/1728708081 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5555a6262620 con 0x5555a615d4d0 2024-09-21T00:26:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.650+0000 7f64e8c00640 1 -- 172.21.15.27:0/1728708081 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64ec0047a0 con 0x5555a615d4d0 2024-09-21T00:26:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.650+0000 7f64e8c00640 1 -- 172.21.15.27:0/1728708081 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f64ec01d030 con 0x5555a615d4d0 2024-09-21T00:26:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.650+0000 7f650cc0d740 10 client.25993 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T00:26:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.650+0000 7f650cc0d740 10 client.25993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:26:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.650+0000 7f650cc0d740 20 client.25993 populate_metadata read hostname 'smithi027' 2024-09-21T00:26:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.650+0000 7f650cc0d740 1 --2- 172.21.15.27:0/1728708081 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5555a613cd80 0x5555a6540460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.650+0000 7f650cc0d740 4 client.25993 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:26:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.650+0000 7f650cc0d740 1 -- 172.21.15.27:0/1728708081 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555a649a600 con 0x5555a613cd80 2024-09-21T00:26:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.651+0000 7f64fea00640 1 --2- 172.21.15.27:0/1728708081 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5555a613cd80 0x5555a6540460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.651+0000 7f64fea00640 1 --2- 172.21.15.27:0/1728708081 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5555a613cd80 0x5555a6540460 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57b2fcc7f143df5 server_cookie=b89f3879d7ef8bd5 in_seq=0 out_seq=0 2024-09-21T00:26:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.651+0000 7f64e8c00640 10 client.25993 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:26:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.653+0000 7f64e8c00640 1 -- 172.21.15.27:0/1728708081 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5555a649a600 con 0x5555a613cd80 2024-09-21T00:26:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.653+0000 7f64e8c00640 10 client.25993 handle_command_reply: tid=0 2024-09-21T00:26:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.653+0000 7f650cc0d740 1 client.25993 shutdown 2024-09-21T00:26:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.653+0000 7f650cc0d740 1 -- 172.21.15.27:0/1728708081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 msgr2=0x5555a652e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.653+0000 7f650cc0d740 1 --2- 172.21.15.27:0/1728708081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 0x5555a652e9d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f64ec006010 tx=0x7f64ec004e20 comp rx=0 tx=0).stop 2024-09-21T00:26:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.653+0000 7f650cc0d740 1 -- 172.21.15.27:0/1728708081 shutdown_connections 2024-09-21T00:26:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.653+0000 7f650cc0d740 1 --2- 172.21.15.27:0/1728708081 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5555a613cd80 0x5555a6540460 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.653+0000 7f650cc0d740 1 --2- 172.21.15.27:0/1728708081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a653a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.653+0000 7f650cc0d740 1 --2- 172.21.15.27:0/1728708081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 0x5555a652e9d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.653+0000 7f64e8c00640 0 client.25993 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:26:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.653+0000 7f650cc0d740 1 --2- 172.21.15.27:0/1728708081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a652e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.653+0000 7f650cc0d740 1 -- 172.21.15.27:0/1728708081 >> 172.21.15.27:0/1728708081 conn(0x5555a6172300 msgr2=0x5555a6523e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.653+0000 7f650cc0d740 1 -- 172.21.15.27:0/1728708081 shutdown_connections 2024-09-21T00:26:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.653+0000 7f650cc0d740 1 -- 172.21.15.27:0/1728708081 wait complete. 2024-09-21T00:26:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.653+0000 7f650cc0d740 20 client.25993 trim_cache size 0 max 16384 2024-09-21T00:26:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.688+0000 7f650cc0d740 1 Processor -- start 2024-09-21T00:26:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.689+0000 7f650cc0d740 1 -- start start 2024-09-21T00:26:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.689+0000 7f650cc0d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a61552f0 0x5555a651fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.689+0000 7f650cc0d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a615d4d0 0x5555a65300c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.689+0000 7f650cc0d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a652b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.689+0000 7f650cc0d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f65001b5740 con 0x5555a615d4d0 2024-09-21T00:26:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.689+0000 7f650cc0d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f65001b2fb0 con 0x5555a617e1b0 2024-09-21T00:26:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.689+0000 7f650cc0d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65001b0e00 con 0x5555a61552f0 2024-09-21T00:26:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.689+0000 7f64ff400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a652b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.689+0000 7f64fea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a61552f0 0x5555a651fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.689+0000 7f64fe000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a615d4d0 0x5555a65300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.689+0000 7f64ff400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a652b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:36580/0 (socket says 172.21.15.27:36580) 2024-09-21T00:26:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.689+0000 7f64ff400640 1 -- 172.21.15.27:0/2129295895 learned_addr learned my addr 172.21.15.27:0/2129295895 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.689+0000 7f64fea00640 1 --2- 172.21.15.27:0/2129295895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a61552f0 0x5555a651fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:50006/0 (socket says 172.21.15.27:50006) 2024-09-21T00:26:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.690+0000 7f64fea00640 1 -- 172.21.15.27:0/2129295895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 msgr2=0x5555a652b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.690+0000 7f64fea00640 1 --2- 172.21.15.27:0/2129295895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a652b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.690+0000 7f64fea00640 1 -- 172.21.15.27:0/2129295895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a615d4d0 msgr2=0x5555a65300c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.690+0000 7f64fea00640 1 --2- 172.21.15.27:0/2129295895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a615d4d0 0x5555a65300c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.690+0000 7f64fea00640 1 -- 172.21.15.27:0/2129295895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555a652c970 con 0x5555a61552f0 2024-09-21T00:26:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.690+0000 7f64ff400640 1 --2- 172.21.15.27:0/2129295895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a652b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:26:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.690+0000 7f64fea00640 1 --2- 172.21.15.27:0/2129295895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a61552f0 0x5555a651fd20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x5555a65409d0 tx=0x7f64f001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8394924d0fd203c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.690+0000 7f64eb400640 1 -- 172.21.15.27:0/2129295895 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f001f690 con 0x5555a61552f0 2024-09-21T00:26:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.690+0000 7f64eb400640 1 -- 172.21.15.27:0/2129295895 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64f001fce0 con 0x5555a61552f0 2024-09-21T00:26:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.690+0000 7f64eb400640 1 -- 172.21.15.27:0/2129295895 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0032450 con 0x5555a61552f0 2024-09-21T00:26:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.691+0000 7f650cc0d740 1 -- 172.21.15.27:0/2129295895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a61552f0 msgr2=0x5555a651fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.691+0000 7f650cc0d740 1 --2- 172.21.15.27:0/2129295895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a61552f0 0x5555a651fd20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x5555a65409d0 tx=0x7f64f001b000 comp rx=0 tx=0).stop 2024-09-21T00:26:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.691+0000 7f650cc0d740 1 -- 172.21.15.27:0/2129295895 shutdown_connections 2024-09-21T00:26:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.691+0000 7f650cc0d740 1 --2- 172.21.15.27:0/2129295895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a652b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.691+0000 7f650cc0d740 1 --2- 172.21.15.27:0/2129295895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a615d4d0 0x5555a65300c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.691+0000 7f650cc0d740 1 --2- 172.21.15.27:0/2129295895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a61552f0 0x5555a651fd20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.691+0000 7f650cc0d740 1 -- 172.21.15.27:0/2129295895 >> 172.21.15.27:0/2129295895 conn(0x5555a6172300 msgr2=0x5555a652f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.692+0000 7f650cc0d740 1 -- 172.21.15.27:0/2129295895 shutdown_connections 2024-09-21T00:26:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.692+0000 7f650cc0d740 1 -- 172.21.15.27:0/2129295895 wait complete. 2024-09-21T00:26:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.692+0000 7f650cc0d740 1 -- start start 2024-09-21T00:26:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.692+0000 7f650cc0d740 1 Processor -- start 2024-09-21T00:26:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f650cc0d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a6536e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f650cc0d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 0x5555a65342e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f650cc0d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a6534820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f650cc0d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f65001b1470 con 0x5555a61552f0 2024-09-21T00:26:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f650cc0d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f65001b28e0 con 0x5555a617e1b0 2024-09-21T00:26:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f650cc0d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65001b1b40 con 0x5555a615d4d0 2024-09-21T00:26:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f650cc0d740 10 client.0 _ll_register_callbacks cb 0x7f64f00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:26:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f64fea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a6536e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f64ff400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a6534820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f64fea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a6536e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58788/0 (socket says 172.21.15.27:58788) 2024-09-21T00:26:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f64fea00640 1 -- 172.21.15.27:0/1075458499 learned_addr learned my addr 172.21.15.27:0/1075458499 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f64fe000640 1 --2- 172.21.15.27:0/1075458499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 0x5555a65342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f64fea00640 1 -- 172.21.15.27:0/1075458499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 msgr2=0x5555a65342e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f64fea00640 1 --2- 172.21.15.27:0/1075458499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 0x5555a65342e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f64fea00640 1 -- 172.21.15.27:0/1075458499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 msgr2=0x5555a6534820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f64fea00640 1 --2- 172.21.15.27:0/1075458499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a6534820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f64fea00640 1 -- 172.21.15.27:0/1075458499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555a6534e10 con 0x5555a61552f0 2024-09-21T00:26:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f64fe000640 1 --2- 172.21.15.27:0/1075458499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 0x5555a65342e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.693+0000 7f64ff400640 1 --2- 172.21.15.27:0/1075458499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a6534820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:26:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.694+0000 7f64fea00640 1 --2- 172.21.15.27:0/1075458499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a6536e20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009d10 tx=0x7f64f0021eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95d67bc4118f1da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.694+0000 7f64e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:26:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.694+0000 7f64e8c00640 1 -- 172.21.15.27:0/1075458499 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0040060 con 0x5555a61552f0 2024-09-21T00:26:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.694+0000 7f64e8c00640 1 -- 172.21.15.27:0/1075458499 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f64f002bb60 con 0x5555a61552f0 2024-09-21T00:26:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.694+0000 7f650cc0d740 1 -- 172.21.15.27:0/1075458499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64f001fce0 con 0x5555a61552f0 2024-09-21T00:26:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.694+0000 7f64e8c00640 1 -- 172.21.15.27:0/1075458499 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f64f0047050 con 0x5555a61552f0 2024-09-21T00:26:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.694+0000 7f650cc0d740 10 client.15978 fetch_fsmap learned FSMap version 30 2024-09-21T00:26:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.694+0000 7f650cc0d740 1 -- 172.21.15.27:0/1075458499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5555a65aaa80 con 0x5555a61552f0 2024-09-21T00:26:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.695+0000 7f64e8c00640 1 -- 172.21.15.27:0/1075458499 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64f001fce0 con 0x5555a61552f0 2024-09-21T00:26:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.695+0000 7f64e8c00640 1 -- 172.21.15.27:0/1075458499 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f64f0005040 con 0x5555a61552f0 2024-09-21T00:26:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.695+0000 7f650cc0d740 10 client.15978 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T00:26:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.695+0000 7f650cc0d740 10 client.15978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:26:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.695+0000 7f650cc0d740 20 client.15978 populate_metadata read hostname 'smithi027' 2024-09-21T00:26:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.695+0000 7f650cc0d740 1 --2- 172.21.15.27:0/1075458499 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5555a61121c0 0x5555a65ad270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.695+0000 7f650cc0d740 4 client.15978 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:26:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.695+0000 7f650cc0d740 1 -- 172.21.15.27:0/1075458499 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555a64960b0 con 0x5555a61121c0 2024-09-21T00:26:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.695+0000 7f64fe000640 1 --2- 172.21.15.27:0/1075458499 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5555a61121c0 0x5555a65ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.696+0000 7f64fe000640 1 --2- 172.21.15.27:0/1075458499 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5555a61121c0 0x5555a65ad270 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f18fb9fc5fbfdcd server_cookie=667de21c0b196178 in_seq=0 out_seq=0 2024-09-21T00:26:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:44.696+0000 7f64e8c00640 10 client.15978 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:26:45.918 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:45 smithi027 ceph-mon[30745]: pgmap v538: 129 pgs: 129 active+clean; 657 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 366 op/s 2024-09-21T00:26:46.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:45 smithi089 ceph-mon[34481]: pgmap v538: 129 pgs: 129 active+clean; 657 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 366 op/s 2024-09-21T00:26:46.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:45 smithi098 ceph-mon[34725]: pgmap v538: 129 pgs: 129 active+clean; 657 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 366 op/s 2024-09-21T00:26:47.158 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-21T00:26:47.158 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:26:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.006+0000 7f64e8c00640 1 -- 172.21.15.27:0/1075458499 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555a64960b0 con 0x5555a61121c0 2024-09-21T00:26:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.006+0000 7f64e8c00640 10 client.15978 handle_command_reply: tid=0 2024-09-21T00:26:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.007+0000 7f650cc0d740 1 client.15978 shutdown 2024-09-21T00:26:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.009+0000 7f650cc0d740 1 -- 172.21.15.27:0/1075458499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 msgr2=0x5555a6536e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.009+0000 7f650cc0d740 1 --2- 172.21.15.27:0/1075458499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a6536e20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009d10 tx=0x7f64f0021eb0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.010+0000 7f650cc0d740 1 -- 172.21.15.27:0/1075458499 shutdown_connections 2024-09-21T00:26:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.010+0000 7f650cc0d740 1 --2- 172.21.15.27:0/1075458499 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5555a61121c0 0x5555a65ad270 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.010+0000 7f650cc0d740 1 --2- 172.21.15.27:0/1075458499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555a617e1b0 0x5555a6534820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.010+0000 7f650cc0d740 1 --2- 172.21.15.27:0/1075458499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555a615d4d0 0x5555a65342e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.010+0000 7f650cc0d740 1 --2- 172.21.15.27:0/1075458499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5555a61552f0 0x5555a6536e20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.010+0000 7f650cc0d740 1 -- 172.21.15.27:0/1075458499 >> 172.21.15.27:0/1075458499 conn(0x5555a6172300 msgr2=0x5555a651faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.010+0000 7f64e8c00640 0 client.15978 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:26:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.012+0000 7f650cc0d740 1 -- 172.21.15.27:0/1075458499 shutdown_connections 2024-09-21T00:26:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.012+0000 7f650cc0d740 1 -- 172.21.15.27:0/1075458499 wait complete. 2024-09-21T00:26:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.012+0000 7f650cc0d740 20 client.15978 trim_cache size 0 max 16384 2024-09-21T00:26:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:26:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:26:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:26:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:26:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.019+0000 7f64ffe00640 1 -- 172.21.15.27:0/2510464259 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f64e40b66f0 msgr2=0x7f64e40b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.019+0000 7f64ffe00640 1 --2- 172.21.15.27:0/2510464259 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f64e40b66f0 0x7f64e40b8c00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x5555a6521a90 tx=0x7f64f0005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.019+0000 7f64ffe00640 1 -- 172.21.15.27:0/2510464259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f650010ff80 msgr2=0x7f65001a1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.019+0000 7f64ffe00640 1 --2- 172.21.15.27:0/2510464259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f650010ff80 0x7f65001a1c40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f64f8005e30 tx=0x7f64f80317c0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.019+0000 7f64ffe00640 1 -- 172.21.15.27:0/2510464259 shutdown_connections 2024-09-21T00:26:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.019+0000 7f64ffe00640 1 --2- 172.21.15.27:0/2510464259 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f64e40b66f0 0x7f64e40b8c00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.019+0000 7f64ffe00640 1 --2- 172.21.15.27:0/2510464259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f650010ff80 0x7f65001a1c40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.019+0000 7f64ffe00640 1 --2- 172.21.15.27:0/2510464259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6500102d20 0x7f650019c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.019+0000 7f64ffe00640 1 --2- 172.21.15.27:0/2510464259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65001023d0 0x7f650019c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.019+0000 7f64ffe00640 1 -- 172.21.15.27:0/2510464259 >> 172.21.15.27:0/2510464259 conn(0x7f65000fe210 msgr2=0x7f6500112070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.020+0000 7f64ffe00640 1 -- 172.21.15.27:0/2510464259 shutdown_connections 2024-09-21T00:26:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:26:48.020+0000 7f64ffe00640 1 -- 172.21.15.27:0/2510464259 wait complete. 2024-09-21T00:26:48.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:47 smithi027 ceph-mon[30745]: pgmap v539: 129 pgs: 129 active+clean; 693 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.0 MiB/s wr, 575 op/s 2024-09-21T00:26:48.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:47 smithi089 ceph-mon[34481]: pgmap v539: 129 pgs: 129 active+clean; 693 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.0 MiB/s wr, 575 op/s 2024-09-21T00:26:48.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:47 smithi098 ceph-mon[34725]: pgmap v539: 129 pgs: 129 active+clean; 693 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.0 MiB/s wr, 575 op/s 2024-09-21T00:26:48.350 INFO:tasks.workunit.client.0.smithi027.stdout: 209 227294 222 150366 231 57806 557 2024-09-21T00:26:48.883 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:26:50.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:49 smithi089 ceph-mon[34481]: pgmap v540: 129 pgs: 129 active+clean; 709 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.7 MiB/s wr, 691 op/s 2024-09-21T00:26:50.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:49 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:26:50.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:49 smithi027 ceph-mon[30745]: pgmap v540: 129 pgs: 129 active+clean; 709 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.7 MiB/s wr, 691 op/s 2024-09-21T00:26:50.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:50 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:26:50.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:49 smithi098 ceph-mon[34725]: pgmap v540: 129 pgs: 129 active+clean; 709 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.7 MiB/s wr, 691 op/s 2024-09-21T00:26:50.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:49 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:26:51.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.735+0000 7f60eb200640 1 -- 172.21.15.27:0/2726616009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60ec109c40 msgr2=0x7f60ec110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:51.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.735+0000 7f60eb200640 1 --2- 172.21.15.27:0/2726616009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60ec109c40 0x7f60ec110470 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009ca0 tx=0x7f60e402a8c0 comp rx=0 tx=0).stop 2024-09-21T00:26:51.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.735+0000 7f60eb200640 1 -- 172.21.15.27:0/2726616009 shutdown_connections 2024-09-21T00:26:51.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.735+0000 7f60eb200640 1 --2- 172.21.15.27:0/2726616009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60ec109c40 0x7f60ec110470 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:51.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.735+0000 7f60eb200640 1 --2- 172.21.15.27:0/2726616009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60ec1092f0 0x7f60ec109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:51.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.735+0000 7f60eb200640 1 --2- 172.21.15.27:0/2726616009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60ec1039b0 0x7f60ec103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:51.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.735+0000 7f60eb200640 1 -- 172.21.15.27:0/2726616009 >> 172.21.15.27:0/2726616009 conn(0x7f60ec0ff400 msgr2=0x7f60ec101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:51.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.735+0000 7f60eb200640 1 -- 172.21.15.27:0/2726616009 shutdown_connections 2024-09-21T00:26:51.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.735+0000 7f60eb200640 1 -- 172.21.15.27:0/2726616009 wait complete. 2024-09-21T00:26:51.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.737+0000 7f60eb200640 1 Processor -- start 2024-09-21T00:26:51.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.742+0000 7f60eb200640 1 -- start start 2024-09-21T00:26:51.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.743+0000 7f60eb200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60ec1039b0 0x7f60ec074dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:51.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.744+0000 7f60eb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60ec1092f0 0x7f60ec06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:51.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.744+0000 7f60e9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60ec1039b0 0x7f60ec074dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:51.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.744+0000 7f60e9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60ec1039b0 0x7f60ec074dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:50032/0 (socket says 172.21.15.27:50032) 2024-09-21T00:26:51.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.744+0000 7f60e9e00640 1 -- 172.21.15.27:0/3111593608 learned_addr learned my addr 172.21.15.27:0/3111593608 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:26:51.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.746+0000 7f60eb200640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60ec109c40 0x7f60ec06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:51.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.745+0000 7f60e9400640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60ec1092f0 0x7f60ec06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:51.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.746+0000 7f60ea800640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60ec109c40 0x7f60ec06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:51.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.747+0000 7f60eb200640 1 -- 172.21.15.27:0/3111593608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec113950 con 0x7f60ec1092f0 2024-09-21T00:26:51.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.747+0000 7f60eb200640 1 -- 172.21.15.27:0/3111593608 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec1137d0 con 0x7f60ec109c40 2024-09-21T00:26:51.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.747+0000 7f60eb200640 1 -- 172.21.15.27:0/3111593608 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec113ad0 con 0x7f60ec1039b0 2024-09-21T00:26:51.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.747+0000 7f60ea800640 1 -- 172.21.15.27:0/3111593608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60ec1039b0 msgr2=0x7f60ec074dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:51.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.747+0000 7f60ea800640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60ec1039b0 0x7f60ec074dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:51.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.747+0000 7f60ea800640 1 -- 172.21.15.27:0/3111593608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60ec1092f0 msgr2=0x7f60ec06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:51.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.748+0000 7f60ea800640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60ec1092f0 0x7f60ec06e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:51.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.748+0000 7f60ea800640 1 -- 172.21.15.27:0/3111593608 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ec06ed20 con 0x7f60ec109c40 2024-09-21T00:26:51.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.748+0000 7f60e9400640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60ec1092f0 0x7f60ec06e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:26:51.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.752+0000 7f60ea800640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60ec109c40 0x7f60ec06e5e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f60e4005e30 tx=0x7f60e40317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:51.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.752+0000 7f60e2a00640 1 -- 172.21.15.27:0/3111593608 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4003b70 con 0x7f60ec109c40 2024-09-21T00:26:51.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.752+0000 7f60e2a00640 1 -- 172.21.15.27:0/3111593608 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60e4003d10 con 0x7f60ec109c40 2024-09-21T00:26:51.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.752+0000 7f60eb200640 1 -- 172.21.15.27:0/3111593608 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ec073a40 con 0x7f60ec109c40 2024-09-21T00:26:51.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.752+0000 7f60e2a00640 1 -- 172.21.15.27:0/3111593608 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e402fa00 con 0x7f60ec109c40 2024-09-21T00:26:51.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.752+0000 7f60eb200640 1 -- 172.21.15.27:0/3111593608 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ec073fd0 con 0x7f60ec109c40 2024-09-21T00:26:51.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.755+0000 7f60eb200640 1 -- 172.21.15.27:0/3111593608 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ec103dc0 con 0x7f60ec109c40 2024-09-21T00:26:51.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.761+0000 7f60e2a00640 1 -- 172.21.15.27:0/3111593608 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f60e4004580 con 0x7f60ec109c40 2024-09-21T00:26:51.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.768+0000 7f60e2a00640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f60d00b6780 0x7f60d00b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:51.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.768+0000 7f60e2a00640 1 -- 172.21.15.27:0/3111593608 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f60e4108d90 con 0x7f60ec109c40 2024-09-21T00:26:51.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.768+0000 7f60e2a00640 1 -- 172.21.15.27:0/3111593608 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f60e410d3c0 con 0x7f60ec109c40 2024-09-21T00:26:51.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.769+0000 7f60e9e00640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f60d00b6780 0x7f60d00b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:51.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:51.771+0000 7f60e9e00640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f60d00b6780 0x7f60d00b8c90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f60d8010db0 tx=0x7f60d800a380 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:52.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:51 smithi027 ceph-mon[30745]: pgmap v541: 129 pgs: 129 active+clean; 710 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.9 MiB/s wr, 941 op/s 2024-09-21T00:26:52.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:51 smithi089 ceph-mon[34481]: pgmap v541: 129 pgs: 129 active+clean; 710 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.9 MiB/s wr, 941 op/s 2024-09-21T00:26:52.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:51 smithi098 ceph-mon[34725]: pgmap v541: 129 pgs: 129 active+clean; 710 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.9 MiB/s wr, 941 op/s 2024-09-21T00:26:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:52.300+0000 7f60eb200640 1 -- 172.21.15.27:0/3111593608 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60ec06f3a0 con 0x7f60ec109c40 2024-09-21T00:26:52.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:52.301+0000 7f60e2a00640 1 -- 172.21.15.27:0/3111593608 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11899 (secure 0 0 0) 0x7f60e40ceef0 con 0x7f60ec109c40 2024-09-21T00:26:52.308 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:26:52.308 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"btime":"2024-09-21T00:26:41:854080+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:26:41.645934+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24722":{"gid":24722,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/3044996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3044996985},{"type":"v1","addr":"172.21.15.89:6837","nonce":3044996985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:26:52.308 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 30 2024-09-21T00:26:52.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:52.306+0000 7f60eb200640 1 -- 172.21.15.27:0/3111593608 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f60d00b6780 msgr2=0x7f60d00b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:52.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:52.306+0000 7f60eb200640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f60d00b6780 0x7f60d00b8c90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f60d8010db0 tx=0x7f60d800a380 comp rx=0 tx=0).stop 2024-09-21T00:26:52.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:52.306+0000 7f60eb200640 1 -- 172.21.15.27:0/3111593608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60ec109c40 msgr2=0x7f60ec06e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:52.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:52.306+0000 7f60eb200640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60ec109c40 0x7f60ec06e5e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f60e4005e30 tx=0x7f60e40317c0 comp rx=0 tx=0).stop 2024-09-21T00:26:52.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:52.306+0000 7f60eb200640 1 -- 172.21.15.27:0/3111593608 shutdown_connections 2024-09-21T00:26:52.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:52.306+0000 7f60eb200640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f60d00b6780 0x7f60d00b8c90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:52.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:52.306+0000 7f60eb200640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60ec109c40 0x7f60ec06e5e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:52.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:52.306+0000 7f60eb200640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60ec1092f0 0x7f60ec06e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:52.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:52.306+0000 7f60eb200640 1 --2- 172.21.15.27:0/3111593608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60ec1039b0 0x7f60ec074dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:52.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:52.306+0000 7f60eb200640 1 -- 172.21.15.27:0/3111593608 >> 172.21.15.27:0/3111593608 conn(0x7f60ec0ff400 msgr2=0x7f60ec108940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:52.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:52.307+0000 7f60eb200640 1 -- 172.21.15.27:0/3111593608 shutdown_connections 2024-09-21T00:26:52.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:26:52.307+0000 7f60eb200640 1 -- 172.21.15.27:0/3111593608 wait complete. 2024-09-21T00:26:53.215 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:52 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3111593608' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:26:53.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:52 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3111593608' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:26:53.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:52 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3111593608' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:26:54.224 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-21T00:26:54.224 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-21T00:26:54.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:53 smithi089 ceph-mon[34481]: pgmap v542: 129 pgs: 129 active+clean; 702 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.5 MiB/s wr, 928 op/s 2024-09-21T00:26:54.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:53 smithi098 ceph-mon[34725]: pgmap v542: 129 pgs: 129 active+clean; 702 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.5 MiB/s wr, 928 op/s 2024-09-21T00:26:54.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:53 smithi027 ceph-mon[30745]: pgmap v542: 129 pgs: 129 active+clean; 702 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.5 MiB/s wr, 928 op/s 2024-09-21T00:26:56.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:55 smithi089 ceph-mon[34481]: pgmap v543: 129 pgs: 129 active+clean; 704 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.5 MiB/s wr, 823 op/s 2024-09-21T00:26:56.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:55 smithi098 ceph-mon[34725]: pgmap v543: 129 pgs: 129 active+clean; 704 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.5 MiB/s wr, 823 op/s 2024-09-21T00:26:56.247 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:55 smithi027 ceph-mon[30745]: pgmap v543: 129 pgs: 129 active+clean; 704 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.5 MiB/s wr, 823 op/s 2024-09-21T00:26:58.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:57 smithi089 ceph-mon[34481]: pgmap v544: 129 pgs: 129 active+clean; 713 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.4 MiB/s wr, 813 op/s 2024-09-21T00:26:58.236 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:57 smithi027 ceph-mon[30745]: pgmap v544: 129 pgs: 129 active+clean; 713 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.4 MiB/s wr, 813 op/s 2024-09-21T00:26:58.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:57 smithi098 ceph-mon[34725]: pgmap v544: 129 pgs: 129 active+clean; 713 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.4 MiB/s wr, 813 op/s 2024-09-21T00:26:58.353 INFO:tasks.workunit.client.0.smithi027.stdout: 223 491916 652 330609 799 136375 1910 2024-09-21T00:27:00.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:26:59 smithi089 ceph-mon[34481]: pgmap v545: 129 pgs: 129 active+clean; 713 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 558 op/s 2024-09-21T00:27:00.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:26:59 smithi098 ceph-mon[34725]: pgmap v545: 129 pgs: 129 active+clean; 713 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 558 op/s 2024-09-21T00:27:00.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:26:59 smithi027 ceph-mon[30745]: pgmap v545: 129 pgs: 129 active+clean; 713 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 558 op/s 2024-09-21T00:27:02.076 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:01 smithi027 ceph-mon[30745]: pgmap v546: 129 pgs: 129 active+clean; 720 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 431 op/s 2024-09-21T00:27:02.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:01 smithi089 ceph-mon[34481]: pgmap v546: 129 pgs: 129 active+clean; 720 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 431 op/s 2024-09-21T00:27:02.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:01 smithi098 ceph-mon[34725]: pgmap v546: 129 pgs: 129 active+clean; 720 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 431 op/s 2024-09-21T00:27:04.083 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:03 smithi027 ceph-mon[30745]: pgmap v547: 129 pgs: 129 active+clean; 746 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 292 op/s 2024-09-21T00:27:04.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:03 smithi089 ceph-mon[34481]: pgmap v547: 129 pgs: 129 active+clean; 746 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 292 op/s 2024-09-21T00:27:04.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:03 smithi098 ceph-mon[34725]: pgmap v547: 129 pgs: 129 active+clean; 746 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 292 op/s 2024-09-21T00:27:06.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:05 smithi027 ceph-mon[30745]: pgmap v548: 129 pgs: 129 active+clean; 753 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 274 op/s 2024-09-21T00:27:06.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:05 smithi089 ceph-mon[34481]: pgmap v548: 129 pgs: 129 active+clean; 753 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 274 op/s 2024-09-21T00:27:06.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:05 smithi098 ceph-mon[34725]: pgmap v548: 129 pgs: 129 active+clean; 753 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 274 op/s 2024-09-21T00:27:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.177+0000 7f6429200640 1 -- 172.21.15.27:0/1091883977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6424111ea0 msgr2=0x7f642410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.177+0000 7f6429200640 1 --2- 172.21.15.27:0/1091883977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6424111ea0 0x7f642410a3c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f641c01acb0 tx=0x7f641c03b990 comp rx=0 tx=0).stop 2024-09-21T00:27:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.183+0000 7f6429200640 1 -- 172.21.15.27:0/1091883977 shutdown_connections 2024-09-21T00:27:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.183+0000 7f6429200640 1 --2- 172.21.15.27:0/1091883977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f642411cba0 0x7f642411f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.183+0000 7f6429200640 1 --2- 172.21.15.27:0/1091883977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f642410a900 0x7f642410ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.183+0000 7f6429200640 1 --2- 172.21.15.27:0/1091883977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6424111ea0 0x7f642410a3c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.183+0000 7f6429200640 1 -- 172.21.15.27:0/1091883977 >> 172.21.15.27:0/1091883977 conn(0x7f6424070d90 msgr2=0x7f64240711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.186+0000 7f6429200640 1 -- 172.21.15.27:0/1091883977 shutdown_connections 2024-09-21T00:27:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.186+0000 7f6429200640 1 -- 172.21.15.27:0/1091883977 wait complete. 2024-09-21T00:27:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.188+0000 7f6429200640 1 Processor -- start 2024-09-21T00:27:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.193+0000 7f6429200640 1 -- start start 2024-09-21T00:27:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.195+0000 7f6429200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6424111ea0 0x7f642411c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.195+0000 7f6429200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f642411cba0 0x7f64241188a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.195+0000 7f6429200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6424118ed0 0x7f6424128010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.196+0000 7f6421400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f642411cba0 0x7f64241188a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.196+0000 7f6421400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f642411cba0 0x7f64241188a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:58280/0 (socket says 172.21.15.27:58280) 2024-09-21T00:27:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.196+0000 7f6422800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6424118ed0 0x7f6424128010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.196+0000 7f6422800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6424118ed0 0x7f6424128010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:43100/0 (socket says 172.21.15.27:43100) 2024-09-21T00:27:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.196+0000 7f6421400640 1 -- 172.21.15.27:0/3141022627 learned_addr learned my addr 172.21.15.27:0/3141022627 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.196+0000 7f6429200640 1 -- 172.21.15.27:0/3141022627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6424114cf0 con 0x7f6424111ea0 2024-09-21T00:27:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.196+0000 7f6429200640 1 -- 172.21.15.27:0/3141022627 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6424114b70 con 0x7f6424118ed0 2024-09-21T00:27:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.196+0000 7f6429200640 1 -- 172.21.15.27:0/3141022627 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6424114e70 con 0x7f642411cba0 2024-09-21T00:27:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.197+0000 7f6421400640 1 -- 172.21.15.27:0/3141022627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6424118ed0 msgr2=0x7f6424128010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.197+0000 7f6421400640 1 --2- 172.21.15.27:0/3141022627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6424118ed0 0x7f6424128010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.197+0000 7f6421400640 1 -- 172.21.15.27:0/3141022627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6424111ea0 msgr2=0x7f642411c5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:27:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.197+0000 7f6421400640 1 --2- 172.21.15.27:0/3141022627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6424111ea0 0x7f642411c5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.197+0000 7f6421400640 1 -- 172.21.15.27:0/3141022627 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64241196b0 con 0x7f642411cba0 2024-09-21T00:27:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.198+0000 7f6421400640 1 --2- 172.21.15.27:0/3141022627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f642411cba0 0x7f64241188a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f641000b410 tx=0x7f641000b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.229+0000 7f641aa00640 1 -- 172.21.15.27:0/3141022627 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410015030 con 0x7f642411cba0 2024-09-21T00:27:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.229+0000 7f641aa00640 1 -- 172.21.15.27:0/3141022627 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6410005e40 con 0x7f642411cba0 2024-09-21T00:27:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.229+0000 7f641aa00640 1 -- 172.21.15.27:0/3141022627 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410011890 con 0x7f642411cba0 2024-09-21T00:27:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.229+0000 7f6429200640 1 -- 172.21.15.27:0/3141022627 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6424128670 con 0x7f642411cba0 2024-09-21T00:27:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.229+0000 7f6429200640 1 -- 172.21.15.27:0/3141022627 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6424128b60 con 0x7f642411cba0 2024-09-21T00:27:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.251+0000 7f643028f740 1 Processor -- start 2024-09-21T00:27:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.253+0000 7f643028f740 1 -- start start 2024-09-21T00:27:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.253+0000 7f643028f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087d71b0 0x559808b7bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.254+0000 7f641aa00640 1 -- 172.21.15.27:0/3141022627 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6410018070 con 0x7f642411cba0 2024-09-21T00:27:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.257+0000 7f641aa00640 1 --2- 172.21.15.27:0/3141022627 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f64080b6800 0x7f64080b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.261+0000 7f643028f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087ae2f0 0x559808b7c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.261+0000 7f643028f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087b64d0 0x559808b7c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.261+0000 7f643028f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559808af16b0 con 0x5598087ae2f0 2024-09-21T00:27:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.261+0000 7f643028f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559808ae0ae0 con 0x5598087d71b0 2024-09-21T00:27:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.261+0000 7f643028f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559808ae48f0 con 0x5598087b64d0 2024-09-21T00:27:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.261+0000 7f6421e00640 1 --2- 172.21.15.27:0/3141022627 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f64080b6800 0x7f64080b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.261+0000 7f6422800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087d71b0 0x559808b7bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.261+0000 7f6422800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087d71b0 0x559808b7bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:43114/0 (socket says 172.21.15.27:43114) 2024-09-21T00:27:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.261+0000 7f6422800640 1 -- 172.21.15.27:0/894153223 learned_addr learned my addr 172.21.15.27:0/894153223 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.262+0000 7f6422800640 1 --2- 172.21.15.27:0/894153223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087b64d0 0x559808b7c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.264+0000 7f6422800640 1 -- 172.21.15.27:0/894153223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087b64d0 msgr2=0x559808b7c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.264+0000 7f6422800640 1 --2- 172.21.15.27:0/894153223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087b64d0 0x559808b7c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.264+0000 7f6422800640 1 -- 172.21.15.27:0/894153223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087ae2f0 msgr2=0x559808b7c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.264+0000 7f6422800640 1 --2- 172.21.15.27:0/894153223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087ae2f0 0x559808b7c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.264+0000 7f6422800640 1 -- 172.21.15.27:0/894153223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559808aad780 con 0x5598087d71b0 2024-09-21T00:27:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.268+0000 7f6422800640 1 --2- 172.21.15.27:0/894153223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087d71b0 0x559808b7bc30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6414017730 tx=0x7f64140094a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cd195e7f44fb7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.268+0000 7f641aa00640 1 -- 172.21.15.27:0/3141022627 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f64100062d0 con 0x7f642411cba0 2024-09-21T00:27:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.274+0000 7f640f400640 1 -- 172.21.15.27:0/894153223 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414015760 con 0x5598087d71b0 2024-09-21T00:27:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.274+0000 7f640f400640 1 -- 172.21.15.27:0/894153223 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f641401d070 con 0x5598087d71b0 2024-09-21T00:27:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.274+0000 7f640f400640 1 -- 172.21.15.27:0/894153223 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414018ab0 con 0x5598087d71b0 2024-09-21T00:27:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.278+0000 7f643028f740 1 -- 172.21.15.27:0/894153223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087d71b0 msgr2=0x559808b7bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.278+0000 7f643028f740 1 --2- 172.21.15.27:0/894153223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087d71b0 0x559808b7bc30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6414017730 tx=0x7f64140094a0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.282+0000 7f6421e00640 1 --2- 172.21.15.27:0/3141022627 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f64080b6800 0x7f64080b8d10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f641c03bf80 tx=0x7f641c000f40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.296+0000 7f643028f740 1 -- 172.21.15.27:0/894153223 shutdown_connections 2024-09-21T00:27:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.296+0000 7f643028f740 1 --2- 172.21.15.27:0/894153223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087b64d0 0x559808b7c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.296+0000 7f643028f740 1 --2- 172.21.15.27:0/894153223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087ae2f0 0x559808b7c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.296+0000 7f643028f740 1 --2- 172.21.15.27:0/894153223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087d71b0 0x559808b7bc30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.296+0000 7f643028f740 1 -- 172.21.15.27:0/894153223 >> 172.21.15.27:0/894153223 conn(0x5598087cb300 msgr2=0x559808b79b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.326+0000 7f643028f740 1 -- 172.21.15.27:0/894153223 shutdown_connections 2024-09-21T00:27:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.326+0000 7f643028f740 1 -- 172.21.15.27:0/894153223 wait complete. 2024-09-21T00:27:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.327+0000 7f643028f740 1 -- start start 2024-09-21T00:27:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.336+0000 7f643028f740 1 Processor -- start 2024-09-21T00:27:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.343+0000 7f643028f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 0x559808b87430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.343+0000 7f643028f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087b64d0 0x559808b879b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.343+0000 7f643028f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087d71b0 0x559808b93640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.343+0000 7f643028f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559808a18ab0 con 0x5598087d71b0 2024-09-21T00:27:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.343+0000 7f643028f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559808a87570 con 0x5598087b64d0 2024-09-21T00:27:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.343+0000 7f643028f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559808b8a350 con 0x5598087ae2f0 2024-09-21T00:27:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.343+0000 7f643028f740 10 client.0 _ll_register_callbacks cb 0x559808a96100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.349+0000 7f6421400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087b64d0 0x559808b879b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.349+0000 7f6422800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087d71b0 0x559808b93640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.350+0000 7f6421400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087b64d0 0x559808b879b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:43126/0 (socket says 172.21.15.27:43126) 2024-09-21T00:27:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.350+0000 7f6421400640 1 -- 172.21.15.27:0/4058371353 learned_addr learned my addr 172.21.15.27:0/4058371353 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.350+0000 7f6421e00640 1 --2- 172.21.15.27:0/4058371353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 0x559808b87430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.352+0000 7f6421400640 1 -- 172.21.15.27:0/4058371353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 msgr2=0x559808b87430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.352+0000 7f6421400640 1 --2- 172.21.15.27:0/4058371353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 0x559808b87430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.352+0000 7f6421400640 1 -- 172.21.15.27:0/4058371353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087d71b0 msgr2=0x559808b93640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.352+0000 7f6421400640 1 --2- 172.21.15.27:0/4058371353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087d71b0 0x559808b93640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.353+0000 7f6421400640 1 -- 172.21.15.27:0/4058371353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559808842790 con 0x5598087b64d0 2024-09-21T00:27:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.353+0000 7f6422800640 1 --2- 172.21.15.27:0/4058371353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087d71b0 0x559808b93640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.353+0000 7f6421400640 1 --2- 172.21.15.27:0/4058371353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087b64d0 0x559808b879b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f64100a9330 tx=0x7f64100a6e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=829c69e3cb68db2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.358+0000 7f6403e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:27:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.358+0000 7f6403e00640 1 -- 172.21.15.27:0/4058371353 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64100a8ff0 con 0x5598087b64d0 2024-09-21T00:27:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.358+0000 7f6403e00640 1 -- 172.21.15.27:0/4058371353 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f64100adb30 con 0x5598087b64d0 2024-09-21T00:27:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.358+0000 7f643028f740 1 -- 172.21.15.27:0/4058371353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f641401d070 con 0x5598087b64d0 2024-09-21T00:27:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.358+0000 7f6403e00640 1 -- 172.21.15.27:0/4058371353 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f64100a9190 con 0x5598087b64d0 2024-09-21T00:27:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.358+0000 7f643028f740 10 client.26036 fetch_fsmap learned FSMap version 30 2024-09-21T00:27:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.358+0000 7f643028f740 1 -- 172.21.15.27:0/4058371353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5598088bb620 con 0x5598087b64d0 2024-09-21T00:27:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.359+0000 7f6403e00640 1 -- 172.21.15.27:0/4058371353 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f641401d070 con 0x5598087b64d0 2024-09-21T00:27:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.362+0000 7f6403e00640 1 -- 172.21.15.27:0/4058371353 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6410022060 con 0x5598087b64d0 2024-09-21T00:27:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.362+0000 7f643028f740 10 client.26036 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T00:27:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.362+0000 7f643028f740 10 client.26036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:27:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.362+0000 7f643028f740 20 client.26036 populate_metadata read hostname 'smithi027' 2024-09-21T00:27:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.362+0000 7f643028f740 1 --2- 172.21.15.27:0/4058371353 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x559808795d80 0x559808b99850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.362+0000 7f643028f740 4 client.26036 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:27:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.362+0000 7f643028f740 1 -- 172.21.15.27:0/4058371353 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559808af3600 con 0x559808795d80 2024-09-21T00:27:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.363+0000 7f6421e00640 1 --2- 172.21.15.27:0/4058371353 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x559808795d80 0x559808b99850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.363+0000 7f6403e00640 10 client.26036 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.364+0000 7f6421e00640 1 --2- 172.21.15.27:0/4058371353 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x559808795d80 0x559808b99850 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2e9833095a471d1 server_cookie=e101d03778160dd5 in_seq=0 out_seq=0 2024-09-21T00:27:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.365+0000 7f6403e00640 1 -- 172.21.15.27:0/4058371353 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559808af3600 con 0x559808795d80 2024-09-21T00:27:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.365+0000 7f6403e00640 10 client.26036 handle_command_reply: tid=0 2024-09-21T00:27:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.365+0000 7f643028f740 1 client.26036 shutdown 2024-09-21T00:27:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.391+0000 7f643028f740 1 -- 172.21.15.27:0/4058371353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087b64d0 msgr2=0x559808b879b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.391+0000 7f643028f740 1 --2- 172.21.15.27:0/4058371353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087b64d0 0x559808b879b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f64100a9330 tx=0x7f64100a6e90 comp rx=0 tx=0).stop 2024-09-21T00:27:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.393+0000 7f643028f740 1 -- 172.21.15.27:0/4058371353 shutdown_connections 2024-09-21T00:27:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.393+0000 7f643028f740 1 --2- 172.21.15.27:0/4058371353 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x559808795d80 0x559808b99850 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.393+0000 7f643028f740 1 --2- 172.21.15.27:0/4058371353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087d71b0 0x559808b93640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.393+0000 7f643028f740 1 --2- 172.21.15.27:0/4058371353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087b64d0 0x559808b879b0 secure :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f64100a9330 tx=0x7f64100a6e90 comp rx=0 tx=0).stop 2024-09-21T00:27:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.393+0000 7f643028f740 1 --2- 172.21.15.27:0/4058371353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 0x559808b87430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.393+0000 7f643028f740 1 -- 172.21.15.27:0/4058371353 >> 172.21.15.27:0/4058371353 conn(0x5598087cb300 msgr2=0x559808b7cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.393+0000 7f6403e00640 0 client.26036 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.396+0000 7f643028f740 1 -- 172.21.15.27:0/4058371353 shutdown_connections 2024-09-21T00:27:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.396+0000 7f643028f740 1 -- 172.21.15.27:0/4058371353 wait complete. 2024-09-21T00:27:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.396+0000 7f643028f740 20 client.26036 trim_cache size 0 max 16384 2024-09-21T00:27:07.899 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:27:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.961+0000 7f643028f740 1 Processor -- start 2024-09-21T00:27:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.962+0000 7f643028f740 1 -- start start 2024-09-21T00:27:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.963+0000 7f643028f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 0x559808b82ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.965+0000 7f643028f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087d71b0 0x559808b89060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.966+0000 7f6421e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 0x559808b82ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.966+0000 7f6421e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 0x559808b82ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:58308/0 (socket says 172.21.15.27:58308) 2024-09-21T00:27:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.967+0000 7f6421400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087d71b0 0x559808b89060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.968+0000 7f6421400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087d71b0 0x559808b89060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45760/0 (socket says 172.21.15.27:45760) 2024-09-21T00:27:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.969+0000 7f6421e00640 1 -- 172.21.15.27:0/3229476034 learned_addr learned my addr 172.21.15.27:0/3229476034 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.981+0000 7f643028f740 1 --2- 172.21.15.27:0/3229476034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55980876b1c0 0x559808b93540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.982+0000 7f643028f740 1 -- 172.21.15.27:0/3229476034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6424081ec0 con 0x5598087d71b0 2024-09-21T00:27:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.982+0000 7f643028f740 1 -- 172.21.15.27:0/3229476034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f642407f770 con 0x55980876b1c0 2024-09-21T00:27:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.982+0000 7f643028f740 1 -- 172.21.15.27:0/3229476034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f642410b3b0 con 0x5598087ae2f0 2024-09-21T00:27:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.983+0000 7f6422800640 1 --2- 172.21.15.27:0/3229476034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55980876b1c0 0x559808b93540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.985+0000 7f6421e00640 1 -- 172.21.15.27:0/3229476034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55980876b1c0 msgr2=0x559808b93540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.985+0000 7f6421e00640 1 --2- 172.21.15.27:0/3229476034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55980876b1c0 0x559808b93540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.985+0000 7f6421e00640 1 -- 172.21.15.27:0/3229476034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087d71b0 msgr2=0x559808b89060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.985+0000 7f6421e00640 1 --2- 172.21.15.27:0/3229476034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087d71b0 0x559808b89060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.985+0000 7f6421e00640 1 -- 172.21.15.27:0/3229476034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559808b886f0 con 0x5598087ae2f0 2024-09-21T00:27:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.985+0000 7f6421400640 1 --2- 172.21.15.27:0/3229476034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087d71b0 0x559808b89060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:27:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.990+0000 7f6421e00640 1 --2- 172.21.15.27:0/3229476034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 0x559808b82ab0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f641c01acb0 tx=0x7f641c00c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5510041b48bca04d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.990+0000 7f640ea00640 1 -- 172.21.15.27:0/3229476034 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c056070 con 0x5598087ae2f0 2024-09-21T00:27:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.991+0000 7f640ea00640 1 -- 172.21.15.27:0/3229476034 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f641c016040 con 0x5598087ae2f0 2024-09-21T00:27:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.991+0000 7f640ea00640 1 -- 172.21.15.27:0/3229476034 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c056070 con 0x5598087ae2f0 2024-09-21T00:27:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.992+0000 7f643028f740 1 -- 172.21.15.27:0/3229476034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 msgr2=0x559808b82ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:07.992+0000 7f643028f740 1 --2- 172.21.15.27:0/3229476034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 0x559808b82ab0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f641c01acb0 tx=0x7f641c00c000 comp rx=0 tx=0).stop 2024-09-21T00:27:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.013+0000 7f643028f740 1 -- 172.21.15.27:0/3229476034 shutdown_connections 2024-09-21T00:27:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.013+0000 7f643028f740 1 --2- 172.21.15.27:0/3229476034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55980876b1c0 0x559808b93540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.013+0000 7f643028f740 1 --2- 172.21.15.27:0/3229476034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598087d71b0 0x559808b89060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.013+0000 7f643028f740 1 --2- 172.21.15.27:0/3229476034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 0x559808b82ab0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.013+0000 7f643028f740 1 -- 172.21.15.27:0/3229476034 >> 172.21.15.27:0/3229476034 conn(0x5598087cb300 msgr2=0x559808b88950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.015+0000 7f643028f740 1 -- 172.21.15.27:0/3229476034 shutdown_connections 2024-09-21T00:27:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.036+0000 7f643028f740 1 -- 172.21.15.27:0/3229476034 wait complete. 2024-09-21T00:27:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.037+0000 7f643028f740 1 -- start start 2024-09-21T00:27:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.037+0000 7f643028f740 1 Processor -- start 2024-09-21T00:27:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.047+0000 7f643028f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980876b1c0 0x559808b8b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.047+0000 7f643028f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 0x559808b8b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.047+0000 7f643028f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087d71b0 0x559808b8f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.047+0000 7f643028f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f642407dc30 con 0x55980876b1c0 2024-09-21T00:27:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.047+0000 7f643028f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f642407f0a0 con 0x5598087d71b0 2024-09-21T00:27:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.047+0000 7f643028f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f642407e300 con 0x5598087ae2f0 2024-09-21T00:27:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.047+0000 7f643028f740 10 client.0 _ll_register_callbacks cb 0x7f6414006ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.060+0000 7f6422800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087d71b0 0x559808b8f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.063+0000 7f6421e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980876b1c0 0x559808b8b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.063+0000 7f6421e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980876b1c0 0x559808b8b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45772/0 (socket says 172.21.15.27:45772) 2024-09-21T00:27:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.063+0000 7f6421e00640 1 -- 172.21.15.27:0/1316909066 learned_addr learned my addr 172.21.15.27:0/1316909066 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.067+0000 7f6421400640 1 --2- 172.21.15.27:0/1316909066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 0x559808b8b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.070+0000 7f6421e00640 1 -- 172.21.15.27:0/1316909066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 msgr2=0x559808b8b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.070+0000 7f6421e00640 1 --2- 172.21.15.27:0/1316909066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 0x559808b8b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.070+0000 7f6421e00640 1 -- 172.21.15.27:0/1316909066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087d71b0 msgr2=0x559808b8f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.070+0000 7f6421e00640 1 --2- 172.21.15.27:0/1316909066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087d71b0 0x559808b8f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.070+0000 7f6421e00640 1 -- 172.21.15.27:0/1316909066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559808b8fb80 con 0x55980876b1c0 2024-09-21T00:27:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.071+0000 7f6421e00640 1 --2- 172.21.15.27:0/1316909066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980876b1c0 0x559808b8b040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f641c03bf40 tx=0x7f641c01a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd55650f6d69faf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.079+0000 7f6403e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:27:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.079+0000 7f6403e00640 1 -- 172.21.15.27:0/1316909066 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c056040 con 0x55980876b1c0 2024-09-21T00:27:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.079+0000 7f6403e00640 1 -- 172.21.15.27:0/1316909066 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f641c041890 con 0x55980876b1c0 2024-09-21T00:27:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.118+0000 7f643028f740 1 -- 172.21.15.27:0/1316909066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f641c016040 con 0x55980876b1c0 2024-09-21T00:27:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.139+0000 7f6403e00640 1 -- 172.21.15.27:0/1316909066 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f641c015040 con 0x55980876b1c0 2024-09-21T00:27:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.153+0000 7f643028f740 10 client.15990 fetch_fsmap learned FSMap version 30 2024-09-21T00:27:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.153+0000 7f643028f740 1 -- 172.21.15.27:0/1316909066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559808c03d50 con 0x55980876b1c0 2024-09-21T00:27:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.156+0000 7f6403e00640 1 -- 172.21.15.27:0/1316909066 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f641c016040 con 0x55980876b1c0 2024-09-21T00:27:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.156+0000 7f6403e00640 1 -- 172.21.15.27:0/1316909066 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f641c059070 con 0x55980876b1c0 2024-09-21T00:27:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.160+0000 7f643028f740 10 client.15990 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T00:27:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.160+0000 7f643028f740 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:27:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.161+0000 7f643028f740 20 client.15990 populate_metadata read hostname 'smithi027' 2024-09-21T00:27:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.161+0000 7f643028f740 1 --2- 172.21.15.27:0/1316909066 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x559808c044e0 0x559808c06900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.161+0000 7f643028f740 4 client.15990 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:27:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.161+0000 7f643028f740 1 -- 172.21.15.27:0/1316909066 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559808aef0b0 con 0x559808c044e0 2024-09-21T00:27:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.161+0000 7f6421400640 1 --2- 172.21.15.27:0/1316909066 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x559808c044e0 0x559808c06900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.162+0000 7f6421400640 1 --2- 172.21.15.27:0/1316909066 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x559808c044e0 0x559808c06900 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b6528bfaf186541 server_cookie=743bda1d39ebe480 in_seq=0 out_seq=0 2024-09-21T00:27:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:08.162+0000 7f6403e00640 10 client.15990 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:08.263 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:07 smithi027 ceph-mon[30745]: pgmap v549: 129 pgs: 129 active+clean; 767 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 355 op/s 2024-09-21T00:27:08.263 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:07 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.263 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:07 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.264 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:07 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.264 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:07 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.264 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:07 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.264 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:07 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.264 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:07 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.264 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:07 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.264 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:07 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:27:08.264 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:07 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:27:08.264 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:07 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:27:08.372 INFO:tasks.workunit.client.0.smithi027.stdout: 227 427403 315 283823 270 120350 226 2024-09-21T00:27:08.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:07 smithi089 ceph-mon[34481]: pgmap v549: 129 pgs: 129 active+clean; 767 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 355 op/s 2024-09-21T00:27:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:07 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:07 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:07 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:07 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:07 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:07 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:07 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:07 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:07 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:27:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:07 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:27:08.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:07 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:27:08.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:07 smithi098 ceph-mon[34725]: pgmap v549: 129 pgs: 129 active+clean; 767 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 355 op/s 2024-09-21T00:27:08.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:07 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:07 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:07 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:07 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:07 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:07 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:07 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:07 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T00:27:08.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:07 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:27:08.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:07 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:27:08.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:07 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:27:09.025 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:09 smithi027 ceph-mon[30745]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-09-21T00:27:09.025 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:09 smithi027 ceph-mon[30745]: Unable to set osd_memory_target on smithi027 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-21T00:27:09.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:09 smithi089 ceph-mon[34481]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-09-21T00:27:09.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:09 smithi089 ceph-mon[34481]: Unable to set osd_memory_target on smithi027 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-21T00:27:09.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:09 smithi098 ceph-mon[34725]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-09-21T00:27:09.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:09 smithi098 ceph-mon[34725]: Unable to set osd_memory_target on smithi027 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-21T00:27:10.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.195+0000 7f8704a00640 1 -- 172.21.15.27:0/2123900782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8700108b30 msgr2=0x7f870010f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:10.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.195+0000 7f8704a00640 1 --2- 172.21.15.27:0/2123900782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8700108b30 0x7f870010f360 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009ca0 tx=0x7f86f802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.196+0000 7f8704a00640 1 -- 172.21.15.27:0/2123900782 shutdown_connections 2024-09-21T00:27:10.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.196+0000 7f8704a00640 1 --2- 172.21.15.27:0/2123900782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8700108b30 0x7f870010f360 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.196+0000 7f8704a00640 1 --2- 172.21.15.27:0/2123900782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87001081e0 0x7f87001085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.196+0000 7f8704a00640 1 --2- 172.21.15.27:0/2123900782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87001028a0 0x7f8700102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.196+0000 7f8704a00640 1 -- 172.21.15.27:0/2123900782 >> 172.21.15.27:0/2123900782 conn(0x7f87000fe330 msgr2=0x7f8700100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.198+0000 7f8704a00640 1 -- 172.21.15.27:0/2123900782 shutdown_connections 2024-09-21T00:27:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.198+0000 7f8704a00640 1 -- 172.21.15.27:0/2123900782 wait complete. 2024-09-21T00:27:10.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.199+0000 7f8704a00640 1 Processor -- start 2024-09-21T00:27:10.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:10 smithi027 ceph-mon[30745]: pgmap v550: 129 pgs: 129 active+clean; 780 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 405 op/s 2024-09-21T00:27:10.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.201+0000 7f8704a00640 1 -- start start 2024-09-21T00:27:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.217+0000 7f8704a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87001028a0 0x7f870010f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.217+0000 7f8704a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87001081e0 0x7f870010b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.217+0000 7f8704a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8700108b30 0x7f870010b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.217+0000 7f8704a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8700111720 con 0x7f8700108b30 2024-09-21T00:27:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.218+0000 7f8704a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f87001115a0 con 0x7f87001028a0 2024-09-21T00:27:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.218+0000 7f8704a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f87001118a0 con 0x7f87001081e0 2024-09-21T00:27:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.218+0000 7f86fea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87001081e0 0x7f870010b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.218+0000 7f86fea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87001081e0 0x7f870010b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:58348/0 (socket says 172.21.15.27:58348) 2024-09-21T00:27:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.218+0000 7f86fea00640 1 -- 172.21.15.27:0/867917926 learned_addr learned my addr 172.21.15.27:0/867917926 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.220+0000 7f86fea00640 1 -- 172.21.15.27:0/867917926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87001028a0 msgr2=0x7f870010f0e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:27:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.220+0000 7f86fea00640 1 --2- 172.21.15.27:0/867917926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87001028a0 0x7f870010f0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.220+0000 7f86fea00640 1 -- 172.21.15.27:0/867917926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8700108b30 msgr2=0x7f870010b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.220+0000 7f86fea00640 1 --2- 172.21.15.27:0/867917926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8700108b30 0x7f870010b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.220+0000 7f86fea00640 1 -- 172.21.15.27:0/867917926 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870010c090 con 0x7f87001081e0 2024-09-21T00:27:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.221+0000 7f86fea00640 1 --2- 172.21.15.27:0/867917926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87001081e0 0x7f870010b3c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00d950 tx=0x7f86ec00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.221+0000 7f86f7e00640 1 -- 172.21.15.27:0/867917926 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec017070 con 0x7f87001081e0 2024-09-21T00:27:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.221+0000 7f86f7e00640 1 -- 172.21.15.27:0/867917926 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ec009c30 con 0x7f87001081e0 2024-09-21T00:27:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.221+0000 7f86f7e00640 1 -- 172.21.15.27:0/867917926 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec00f620 con 0x7f87001081e0 2024-09-21T00:27:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.223+0000 7f8704a00640 1 -- 172.21.15.27:0/867917926 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87001acbf0 con 0x7f87001081e0 2024-09-21T00:27:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.223+0000 7f8704a00640 1 -- 172.21.15.27:0/867917926 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87001ad0b0 con 0x7f87001081e0 2024-09-21T00:27:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.240+0000 7f86f7e00640 1 -- 172.21.15.27:0/867917926 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f86ec01a070 con 0x7f87001081e0 2024-09-21T00:27:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.242+0000 7f870ce0c740 1 Processor -- start 2024-09-21T00:27:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.246+0000 7f870ce0c740 1 -- start start 2024-09-21T00:27:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.246+0000 7f86f7e00640 1 --2- 172.21.15.27:0/867917926 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f86e40b66f0 0x7f86e40b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.246+0000 7f86f7e00640 1 -- 172.21.15.27:0/867917926 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f86ec0e7460 con 0x7f87001081e0 2024-09-21T00:27:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.249+0000 7f870ce0c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3039840 0x561bd33e1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.249+0000 7f870ce0c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3010980 0x561bd33e1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.249+0000 7f86ffe00640 1 --2- 172.21.15.27:0/867917926 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f86e40b66f0 0x7f86e40b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.249+0000 7f86ff400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3039840 0x561bd33e1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.249+0000 7f870ce0c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3018b60 0x561bd33e1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.249+0000 7f870ce0c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561bd3356ca0 con 0x561bd3010980 2024-09-21T00:27:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.249+0000 7f870ce0c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561bd33460d0 con 0x561bd3018b60 2024-09-21T00:27:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.249+0000 7f870ce0c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561bd3349ee0 con 0x561bd3039840 2024-09-21T00:27:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.249+0000 7f86ff400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3039840 0x561bd33e1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:58364/0 (socket says 172.21.15.27:58364) 2024-09-21T00:27:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.249+0000 7f86ff400640 1 -- 172.21.15.27:0/1554177251 learned_addr learned my addr 172.21.15.27:0/1554177251 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.250+0000 7f86ff400640 1 -- 172.21.15.27:0/1554177251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3018b60 msgr2=0x561bd33e1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.250+0000 7f86ff400640 1 --2- 172.21.15.27:0/1554177251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3018b60 0x561bd33e1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.250+0000 7f86ff400640 1 -- 172.21.15.27:0/1554177251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3010980 msgr2=0x561bd33e1a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:27:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.250+0000 7f86ff400640 1 --2- 172.21.15.27:0/1554177251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3010980 0x561bd33e1a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.250+0000 7f86ff400640 1 -- 172.21.15.27:0/1554177251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd32f8400 con 0x561bd3039840 2024-09-21T00:27:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.250+0000 7f86ff400640 1 --2- 172.21.15.27:0/1554177251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3039840 0x561bd33e1510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f86f000a9f0 tx=0x7f86f000aeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d0e98e83e79474b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.253+0000 7f86ebe00640 1 -- 172.21.15.27:0/1554177251 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f000cde0 con 0x561bd3039840 2024-09-21T00:27:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.253+0000 7f86ebe00640 1 -- 172.21.15.27:0/1554177251 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f000f900 con 0x561bd3039840 2024-09-21T00:27:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.253+0000 7f86ebe00640 1 -- 172.21.15.27:0/1554177251 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0018a40 con 0x561bd3039840 2024-09-21T00:27:10.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.257+0000 7f86ffe00640 1 --2- 172.21.15.27:0/867917926 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f86e40b66f0 0x7f86e40b8c00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f86f802aeb0 tx=0x7f86f8031b90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:10.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.258+0000 7f870ce0c740 1 -- 172.21.15.27:0/1554177251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3039840 msgr2=0x561bd33e1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:10.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.258+0000 7f870ce0c740 1 --2- 172.21.15.27:0/1554177251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3039840 0x561bd33e1510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f86f000a9f0 tx=0x7f86f000aeb0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.258+0000 7f870ce0c740 1 -- 172.21.15.27:0/1554177251 shutdown_connections 2024-09-21T00:27:10.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.258+0000 7f870ce0c740 1 --2- 172.21.15.27:0/1554177251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3018b60 0x561bd33e1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.258+0000 7f870ce0c740 1 --2- 172.21.15.27:0/1554177251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3010980 0x561bd33e1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.258+0000 7f870ce0c740 1 --2- 172.21.15.27:0/1554177251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3039840 0x561bd33e1510 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.258+0000 7f870ce0c740 1 -- 172.21.15.27:0/1554177251 >> 172.21.15.27:0/1554177251 conn(0x561bd302d990 msgr2=0x561bd33df400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:10.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.260+0000 7f870ce0c740 1 -- 172.21.15.27:0/1554177251 shutdown_connections 2024-09-21T00:27:10.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.261+0000 7f870ce0c740 1 -- 172.21.15.27:0/1554177251 wait complete. 2024-09-21T00:27:10.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.261+0000 7f870ce0c740 1 -- start start 2024-09-21T00:27:10.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.262+0000 7f870ce0c740 1 Processor -- start 2024-09-21T00:27:10.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.263+0000 7f870ce0c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3010980 0x561bd33ecc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.263+0000 7f870ce0c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3018b60 0x561bd33ed180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.263+0000 7f870ce0c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 0x561bd33f9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.264+0000 7f86fea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3018b60 0x561bd33ed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.264+0000 7f86fea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3018b60 0x561bd33ed180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:58380/0 (socket says 172.21.15.27:58380) 2024-09-21T00:27:10.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.267+0000 7f86ffe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 0x561bd33f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.267+0000 7f86ffe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 0x561bd33f9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:43178/0 (socket says 172.21.15.27:43178) 2024-09-21T00:27:10.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.267+0000 7f86ff400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3010980 0x561bd33ecc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.267+0000 7f86ff400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3010980 0x561bd33ecc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45806/0 (socket says 172.21.15.27:45806) 2024-09-21T00:27:10.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.267+0000 7f870ce0c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561bd31a77a0 con 0x561bd3010980 2024-09-21T00:27:10.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.267+0000 7f870ce0c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561bd32eba10 con 0x561bd3039840 2024-09-21T00:27:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.267+0000 7f86fea00640 1 -- 172.21.15.27:0/3724683052 learned_addr learned my addr 172.21.15.27:0/3724683052 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.268+0000 7f870ce0c740 1 -- 172.21.15.27:0/3724683052 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561bd33efcf0 con 0x561bd3018b60 2024-09-21T00:27:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.271+0000 7f86ffe00640 1 -- 172.21.15.27:0/3724683052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3018b60 msgr2=0x561bd33ed180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.271+0000 7f86ffe00640 1 --2- 172.21.15.27:0/3724683052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3018b60 0x561bd33ed180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.271+0000 7f86ffe00640 1 -- 172.21.15.27:0/3724683052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3010980 msgr2=0x561bd33ecc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.271+0000 7f86ffe00640 1 --2- 172.21.15.27:0/3724683052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3010980 0x561bd33ecc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.271+0000 7f86ffe00640 1 -- 172.21.15.27:0/3724683052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd30b3b90 con 0x561bd3039840 2024-09-21T00:27:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.273+0000 7f86fea00640 1 --2- 172.21.15.27:0/3724683052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3018b60 0x561bd33ed180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:27:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.273+0000 7f86ffe00640 1 --2- 172.21.15.27:0/3724683052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 0x561bd33f9030 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f86f80094e0 tx=0x7f86f8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19cbd97b45376c50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.276+0000 7f86e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:27:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.276+0000 7f86e8c00640 1 -- 172.21.15.27:0/3724683052 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f80047a0 con 0x561bd3039840 2024-09-21T00:27:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.276+0000 7f86e8c00640 1 -- 172.21.15.27:0/3724683052 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f86f80305c0 con 0x561bd3039840 2024-09-21T00:27:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.276+0000 7f86e8c00640 10 client.0 inactive, discarding osd_map(90..90 src has 1..90) 2024-09-21T00:27:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.280+0000 7f870ce0c740 10 client.0 _ll_register_callbacks cb 0x561bd32fbbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.280+0000 7f870ce0c740 1 -- 172.21.15.27:0/3724683052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f000f900 con 0x561bd3039840 2024-09-21T00:27:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.280+0000 7f86e8c00640 1 -- 172.21.15.27:0/3724683052 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f86f8030800 con 0x561bd3039840 2024-09-21T00:27:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.280+0000 7f870ce0c740 10 client.26045 fetch_fsmap learned FSMap version 30 2024-09-21T00:27:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.280+0000 7f870ce0c740 1 -- 172.21.15.27:0/3724683052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({fsmap=30}) -- 0x561bd33fb880 con 0x561bd3039840 2024-09-21T00:27:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.281+0000 7f86e8c00640 1 -- 172.21.15.27:0/3724683052 <== mon.2 v2:172.21.15.89:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f86f80309a0 con 0x561bd3039840 2024-09-21T00:27:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.281+0000 7f870ce0c740 10 client.26045 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T00:27:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.281+0000 7f870ce0c740 10 client.26045 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:27:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.281+0000 7f870ce0c740 20 client.26045 populate_metadata read hostname 'smithi027' 2024-09-21T00:27:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.281+0000 7f870ce0c740 1 --2- 172.21.15.27:0/3724683052 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561bd2ff81c0 0x561bd33fee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.281+0000 7f870ce0c740 4 client.26045 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:27:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.281+0000 7f870ce0c740 1 -- 172.21.15.27:0/3724683052 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bd3358bf0 con 0x561bd2ff81c0 2024-09-21T00:27:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.282+0000 7f86ff400640 1 --2- 172.21.15.27:0/3724683052 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561bd2ff81c0 0x561bd33fee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.282+0000 7f86e8c00640 10 client.26045 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.282+0000 7f86ff400640 1 --2- 172.21.15.27:0/3724683052 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561bd2ff81c0 0x561bd33fee10 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf47b702fd644143 server_cookie=22465e146a7c143c in_seq=0 out_seq=0 2024-09-21T00:27:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.283+0000 7f86e8c00640 1 -- 172.21.15.27:0/3724683052 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561bd3358bf0 con 0x561bd2ff81c0 2024-09-21T00:27:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.283+0000 7f86e8c00640 10 client.26045 handle_command_reply: tid=0 2024-09-21T00:27:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.284+0000 7f870ce0c740 1 client.26045 shutdown 2024-09-21T00:27:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.284+0000 7f870ce0c740 1 -- 172.21.15.27:0/3724683052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 msgr2=0x561bd33f9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.284+0000 7f870ce0c740 1 --2- 172.21.15.27:0/3724683052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 0x561bd33f9030 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f86f80094e0 tx=0x7f86f8005fd0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.284+0000 7f870ce0c740 1 -- 172.21.15.27:0/3724683052 shutdown_connections 2024-09-21T00:27:10.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.284+0000 7f870ce0c740 1 --2- 172.21.15.27:0/3724683052 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561bd2ff81c0 0x561bd33fee10 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.284+0000 7f870ce0c740 1 --2- 172.21.15.27:0/3724683052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 0x561bd33f9030 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.284+0000 7f870ce0c740 1 --2- 172.21.15.27:0/3724683052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3018b60 0x561bd33ed180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.284+0000 7f86e8c00640 0 client.26045 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:10.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.284+0000 7f870ce0c740 1 --2- 172.21.15.27:0/3724683052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3010980 0x561bd33ecc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.284+0000 7f870ce0c740 1 -- 172.21.15.27:0/3724683052 >> 172.21.15.27:0/3724683052 conn(0x561bd302d990 msgr2=0x561bd33e46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:10.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.288+0000 7f870ce0c740 1 -- 172.21.15.27:0/3724683052 shutdown_connections 2024-09-21T00:27:10.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.288+0000 7f870ce0c740 1 -- 172.21.15.27:0/3724683052 wait complete. 2024-09-21T00:27:10.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.289+0000 7f870ce0c740 20 client.26045 trim_cache size 0 max 16384 2024-09-21T00:27:10.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.339+0000 7f870ce0c740 1 Processor -- start 2024-09-21T00:27:10.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.343+0000 7f870ce0c740 1 -- start start 2024-09-21T00:27:10.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.344+0000 7f870ce0c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3010980 0x561bd33df460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.344+0000 7f870ce0c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3018b60 0x561bd33e9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.344+0000 7f870ce0c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 0x561bd33ea510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.344+0000 7f870ce0c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f87001b9c70 con 0x561bd3018b60 2024-09-21T00:27:10.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.344+0000 7f870ce0c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f87001b74e0 con 0x561bd3039840 2024-09-21T00:27:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.344+0000 7f870ce0c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f87001b5330 con 0x561bd3010980 2024-09-21T00:27:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.344+0000 7f86ff400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3010980 0x561bd33df460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.345+0000 7f86ff400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3010980 0x561bd33df460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:58394/0 (socket says 172.21.15.27:58394) 2024-09-21T00:27:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.345+0000 7f86ff400640 1 -- 172.21.15.27:0/149527264 learned_addr learned my addr 172.21.15.27:0/149527264 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.345+0000 7f86ff400640 1 -- 172.21.15.27:0/149527264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 msgr2=0x561bd33ea510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.345+0000 7f86ff400640 1 --2- 172.21.15.27:0/149527264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 0x561bd33ea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.345+0000 7f86ff400640 1 -- 172.21.15.27:0/149527264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3018b60 msgr2=0x561bd33e9fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:27:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.345+0000 7f86ff400640 1 --2- 172.21.15.27:0/149527264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3018b60 0x561bd33e9fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.345+0000 7f86ff400640 1 -- 172.21.15.27:0/149527264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd33ef380 con 0x561bd3010980 2024-09-21T00:27:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.346+0000 7f86ff400640 1 --2- 172.21.15.27:0/149527264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3010980 0x561bd33df460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f86f000a9f0 tx=0x7f86f000a690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9eee1c7fc9a6e51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.351+0000 7f86eb400640 1 -- 172.21.15.27:0/149527264 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f000f650 con 0x561bd3010980 2024-09-21T00:27:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.352+0000 7f86eb400640 1 -- 172.21.15.27:0/149527264 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f0030070 con 0x561bd3010980 2024-09-21T00:27:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.352+0000 7f86eb400640 1 -- 172.21.15.27:0/149527264 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0004660 con 0x561bd3010980 2024-09-21T00:27:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.354+0000 7f870ce0c740 1 -- 172.21.15.27:0/149527264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3010980 msgr2=0x561bd33df460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:10.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.354+0000 7f870ce0c740 1 --2- 172.21.15.27:0/149527264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3010980 0x561bd33df460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f86f000a9f0 tx=0x7f86f000a690 comp rx=0 tx=0).stop 2024-09-21T00:27:10.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.355+0000 7f870ce0c740 1 -- 172.21.15.27:0/149527264 shutdown_connections 2024-09-21T00:27:10.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.355+0000 7f870ce0c740 1 --2- 172.21.15.27:0/149527264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 0x561bd33ea510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.355+0000 7f870ce0c740 1 --2- 172.21.15.27:0/149527264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3018b60 0x561bd33e9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.355+0000 7f870ce0c740 1 --2- 172.21.15.27:0/149527264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd3010980 0x561bd33df460 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f86f000a9f0 tx=0x7f86f000a690 comp rx=0 tx=0).stop 2024-09-21T00:27:10.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.355+0000 7f870ce0c740 1 -- 172.21.15.27:0/149527264 >> 172.21.15.27:0/149527264 conn(0x561bd302d990 msgr2=0x561bd33eeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:10.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.355+0000 7f870ce0c740 1 -- 172.21.15.27:0/149527264 shutdown_connections 2024-09-21T00:27:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.356+0000 7f870ce0c740 1 -- 172.21.15.27:0/149527264 wait complete. 2024-09-21T00:27:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.357+0000 7f870ce0c740 1 -- start start 2024-09-21T00:27:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.357+0000 7f870ce0c740 1 Processor -- start 2024-09-21T00:27:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.358+0000 7f870ce0c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3018b60 0x561bd33f5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.359+0000 7f870ce0c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 0x561bd33f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.359+0000 7f870ce0c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd2fcd570 0x561bd33f4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.359+0000 7f870ce0c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f87001b59a0 con 0x561bd3018b60 2024-09-21T00:27:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.359+0000 7f86fea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 0x561bd33f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.359+0000 7f870ce0c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f87001b6e10 con 0x561bd3039840 2024-09-21T00:27:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.359+0000 7f870ce0c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f87001b6070 con 0x561bd2fcd570 2024-09-21T00:27:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.360+0000 7f86fea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 0x561bd33f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:43198/0 (socket says 172.21.15.27:43198) 2024-09-21T00:27:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.360+0000 7f86fea00640 1 -- 172.21.15.27:0/2057353471 learned_addr learned my addr 172.21.15.27:0/2057353471 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.360+0000 7f86ffe00640 1 --2- 172.21.15.27:0/2057353471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd2fcd570 0x561bd33f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.360+0000 7f870ce0c740 10 client.0 _ll_register_callbacks cb 0x7f86f8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.360+0000 7f86ff400640 1 --2- 172.21.15.27:0/2057353471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3018b60 0x561bd33f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.360+0000 7f86ffe00640 1 -- 172.21.15.27:0/2057353471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 msgr2=0x561bd33f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.360+0000 7f86ffe00640 1 --2- 172.21.15.27:0/2057353471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 0x561bd33f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.360+0000 7f86ffe00640 1 -- 172.21.15.27:0/2057353471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3018b60 msgr2=0x561bd33f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.360+0000 7f86ffe00640 1 --2- 172.21.15.27:0/2057353471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3018b60 0x561bd33f5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.360+0000 7f86ffe00640 1 -- 172.21.15.27:0/2057353471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd33f4870 con 0x561bd2fcd570 2024-09-21T00:27:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.361+0000 7f86ffe00640 1 --2- 172.21.15.27:0/2057353471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd2fcd570 0x561bd33f4130 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f86f8006010 tx=0x7f86f80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed037a11ca7c02cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.361+0000 7f86e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:27:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.362+0000 7f86e8c00640 1 -- 172.21.15.27:0/2057353471 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f80047a0 con 0x561bd2fcd570 2024-09-21T00:27:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.362+0000 7f86e8c00640 1 -- 172.21.15.27:0/2057353471 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7f86f8064470 con 0x561bd2fcd570 2024-09-21T00:27:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.363+0000 7f870ce0c740 1 -- 172.21.15.27:0/2057353471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f0030070 con 0x561bd2fcd570 2024-09-21T00:27:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.363+0000 7f86e8c00640 1 -- 172.21.15.27:0/2057353471 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f86f8064670 con 0x561bd2fcd570 2024-09-21T00:27:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.363+0000 7f870ce0c740 10 client.26038 fetch_fsmap learned FSMap version 30 2024-09-21T00:27:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.363+0000 7f870ce0c740 1 -- 172.21.15.27:0/2057353471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561bd346a6d0 con 0x561bd2fcd570 2024-09-21T00:27:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.368+0000 7f86e8c00640 1 -- 172.21.15.27:0/2057353471 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f0030070 con 0x561bd2fcd570 2024-09-21T00:27:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.368+0000 7f86e8c00640 1 -- 172.21.15.27:0/2057353471 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f86f8048870 con 0x561bd2fcd570 2024-09-21T00:27:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.369+0000 7f870ce0c740 10 client.26038 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T00:27:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.369+0000 7f870ce0c740 10 client.26038 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:27:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.369+0000 7f870ce0c740 20 client.26038 populate_metadata read hostname 'smithi027' 2024-09-21T00:27:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.369+0000 7f870ce0c740 1 --2- 172.21.15.27:0/2057353471 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561bd346af10 0x561bd346d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.369+0000 7f870ce0c740 4 client.26038 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-21T00:27:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.369+0000 7f870ce0c740 1 -- 172.21.15.27:0/2057353471 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x561bd33546a0 con 0x561bd346af10 2024-09-21T00:27:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.369+0000 7f86ff400640 1 --2- 172.21.15.27:0/2057353471 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561bd346af10 0x561bd346d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.370+0000 7f86ff400640 1 --2- 172.21.15.27:0/2057353471 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561bd346af10 0x561bd346d330 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f3c78fef98c0e1 server_cookie=cfc4d13945f84cbb in_seq=0 out_seq=0 2024-09-21T00:27:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:10.370+0000 7f86e8c00640 10 client.26038 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:10.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:10 smithi089 ceph-mon[34481]: pgmap v550: 129 pgs: 129 active+clean; 780 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 405 op/s 2024-09-21T00:27:10.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:10 smithi098 ceph-mon[34725]: pgmap v550: 129 pgs: 129 active+clean; 780 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 405 op/s 2024-09-21T00:27:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.429+0000 7f6403e00640 1 -- 172.21.15.27:0/1316909066 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559808aef0b0 con 0x559808c044e0 2024-09-21T00:27:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.429+0000 7f6403e00640 10 client.15990 handle_command_reply: tid=0 2024-09-21T00:27:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.429+0000 7f643028f740 1 client.15990 shutdown 2024-09-21T00:27:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.430+0000 7f643028f740 1 -- 172.21.15.27:0/1316909066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980876b1c0 msgr2=0x559808b8b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.430+0000 7f643028f740 1 --2- 172.21.15.27:0/1316909066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980876b1c0 0x559808b8b040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f641c03bf40 tx=0x7f641c01a910 comp rx=0 tx=0).stop 2024-09-21T00:27:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.430+0000 7f643028f740 1 -- 172.21.15.27:0/1316909066 shutdown_connections 2024-09-21T00:27:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.430+0000 7f643028f740 1 --2- 172.21.15.27:0/1316909066 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x559808c044e0 0x559808c06900 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.430+0000 7f643028f740 1 --2- 172.21.15.27:0/1316909066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598087d71b0 0x559808b8f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.430+0000 7f643028f740 1 --2- 172.21.15.27:0/1316909066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598087ae2f0 0x559808b8b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.430+0000 7f643028f740 1 --2- 172.21.15.27:0/1316909066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980876b1c0 0x559808b8b040 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.430+0000 7f643028f740 1 -- 172.21.15.27:0/1316909066 >> 172.21.15.27:0/1316909066 conn(0x5598087cb300 msgr2=0x559808b7d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.430+0000 7f6403e00640 0 client.15990 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.432+0000 7f643028f740 1 -- 172.21.15.27:0/1316909066 shutdown_connections 2024-09-21T00:27:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.432+0000 7f643028f740 1 -- 172.21.15.27:0/1316909066 wait complete. 2024-09-21T00:27:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.433+0000 7f643028f740 20 client.15990 trim_cache size 0 max 16384 2024-09-21T00:27:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:27:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:27:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:27:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:27:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.435+0000 7f6429200640 1 -- 172.21.15.27:0/3141022627 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f64080b6800 msgr2=0x7f64080b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.435+0000 7f6429200640 1 --2- 172.21.15.27:0/3141022627 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f64080b6800 0x7f64080b8d10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f641c03bf80 tx=0x7f641c000f40 comp rx=0 tx=0).stop 2024-09-21T00:27:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.435+0000 7f6429200640 1 -- 172.21.15.27:0/3141022627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f642411cba0 msgr2=0x7f64241188a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.435+0000 7f6429200640 1 --2- 172.21.15.27:0/3141022627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f642411cba0 0x7f64241188a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f641000b410 tx=0x7f641000b8d0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.435+0000 7f6429200640 1 -- 172.21.15.27:0/3141022627 shutdown_connections 2024-09-21T00:27:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.435+0000 7f6429200640 1 --2- 172.21.15.27:0/3141022627 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f64080b6800 0x7f64080b8d10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.435+0000 7f6429200640 1 --2- 172.21.15.27:0/3141022627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6424118ed0 0x7f6424128010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.435+0000 7f6429200640 1 --2- 172.21.15.27:0/3141022627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f642411cba0 0x7f64241188a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.435+0000 7f6429200640 1 --2- 172.21.15.27:0/3141022627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6424111ea0 0x7f642411c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.435+0000 7f6429200640 1 -- 172.21.15.27:0/3141022627 >> 172.21.15.27:0/3141022627 conn(0x7f6424070d90 msgr2=0x7f642411e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.436+0000 7f6429200640 1 -- 172.21.15.27:0/3141022627 shutdown_connections 2024-09-21T00:27:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:11.436+0000 7f6429200640 1 -- 172.21.15.27:0/3141022627 wait complete. 2024-09-21T00:27:12.294 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:12 smithi027 ceph-mon[30745]: pgmap v551: 129 pgs: 129 active+clean; 797 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.6 MiB/s wr, 526 op/s 2024-09-21T00:27:12.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:12 smithi089 ceph-mon[34481]: pgmap v551: 129 pgs: 129 active+clean; 797 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.6 MiB/s wr, 526 op/s 2024-09-21T00:27:12.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:12 smithi098 ceph-mon[34725]: pgmap v551: 129 pgs: 129 active+clean; 797 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.6 MiB/s wr, 526 op/s 2024-09-21T00:27:14.291 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:14 smithi027 ceph-mon[30745]: pgmap v552: 129 pgs: 129 active+clean; 806 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.2 MiB/s wr, 739 op/s 2024-09-21T00:27:14.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:14 smithi089 ceph-mon[34481]: pgmap v552: 129 pgs: 129 active+clean; 806 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.2 MiB/s wr, 739 op/s 2024-09-21T00:27:14.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:14 smithi098 ceph-mon[34725]: pgmap v552: 129 pgs: 129 active+clean; 806 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.2 MiB/s wr, 739 op/s 2024-09-21T00:27:16.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:16 smithi027 ceph-mon[30745]: pgmap v553: 129 pgs: 129 active+clean; 812 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 730 op/s 2024-09-21T00:27:16.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:16 smithi089 ceph-mon[34481]: pgmap v553: 129 pgs: 129 active+clean; 812 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 730 op/s 2024-09-21T00:27:16.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:16 smithi098 ceph-mon[34725]: pgmap v553: 129 pgs: 129 active+clean; 812 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 730 op/s 2024-09-21T00:27:18.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:18 smithi027 ceph-mon[30745]: pgmap v554: 129 pgs: 129 active+clean; 804 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 701 op/s 2024-09-21T00:27:18.370 INFO:tasks.workunit.client.0.smithi027.stdout: 227 94395 38 62888 59 29395 108 2024-09-21T00:27:18.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:18 smithi089 ceph-mon[34481]: pgmap v554: 129 pgs: 129 active+clean; 804 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 701 op/s 2024-09-21T00:27:18.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:18 smithi098 ceph-mon[34725]: pgmap v554: 129 pgs: 129 active+clean; 804 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 701 op/s 2024-09-21T00:27:20.150 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:20 smithi027 ceph-mon[30745]: pgmap v555: 129 pgs: 129 active+clean; 807 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 677 op/s 2024-09-21T00:27:20.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:20 smithi089 ceph-mon[34481]: pgmap v555: 129 pgs: 129 active+clean; 807 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 677 op/s 2024-09-21T00:27:20.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:20 smithi098 ceph-mon[34725]: pgmap v555: 129 pgs: 129 active+clean; 807 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 677 op/s 2024-09-21T00:27:22.101 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:22 smithi027 ceph-mon[30745]: pgmap v556: 129 pgs: 129 active+clean; 809 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 617 op/s 2024-09-21T00:27:22.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:22 smithi089 ceph-mon[34481]: pgmap v556: 129 pgs: 129 active+clean; 809 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 617 op/s 2024-09-21T00:27:22.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:22 smithi098 ceph-mon[34725]: pgmap v556: 129 pgs: 129 active+clean; 809 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 617 op/s 2024-09-21T00:27:24.081 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:24 smithi027 ceph-mon[30745]: pgmap v557: 129 pgs: 129 active+clean; 819 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 527 op/s 2024-09-21T00:27:24.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:24 smithi089 ceph-mon[34481]: pgmap v557: 129 pgs: 129 active+clean; 819 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 527 op/s 2024-09-21T00:27:24.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:24 smithi098 ceph-mon[34725]: pgmap v557: 129 pgs: 129 active+clean; 819 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 527 op/s 2024-09-21T00:27:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.724+0000 7ff565600640 1 -- 172.21.15.27:0/1812674279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff560102350 msgr2=0x7ff560102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.724+0000 7ff565600640 1 --2- 172.21.15.27:0/1812674279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff560102350 0x7ff560102760 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff550009900 tx=0x7ff55002a440 comp rx=0 tx=0).stop 2024-09-21T00:27:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.724+0000 7ff565600640 1 -- 172.21.15.27:0/1812674279 shutdown_connections 2024-09-21T00:27:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.724+0000 7ff565600640 1 --2- 172.21.15.27:0/1812674279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff56010ff00 0x7ff560112320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.724+0000 7ff565600640 1 --2- 172.21.15.27:0/1812674279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff560102ca0 0x7ff56010f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.724+0000 7ff565600640 1 --2- 172.21.15.27:0/1812674279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff560102350 0x7ff560102760 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.724+0000 7ff565600640 1 -- 172.21.15.27:0/1812674279 >> 172.21.15.27:0/1812674279 conn(0x7ff5600fe210 msgr2=0x7ff560100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.724+0000 7ff565600640 1 -- 172.21.15.27:0/1812674279 shutdown_connections 2024-09-21T00:27:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.724+0000 7ff565600640 1 -- 172.21.15.27:0/1812674279 wait complete. 2024-09-21T00:27:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff565600640 1 Processor -- start 2024-09-21T00:27:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff565600640 1 -- start start 2024-09-21T00:27:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff565600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff560102350 0x7ff56019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff565600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff560102ca0 0x7ff56019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff565600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff56010ff00 0x7ff5601a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff565600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff560115b10 con 0x7ff560102350 2024-09-21T00:27:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff565600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff560115990 con 0x7ff56010ff00 2024-09-21T00:27:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff565600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff560115c90 con 0x7ff560102ca0 2024-09-21T00:27:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff560102350 0x7ff56019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff560102350 0x7ff56019c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38030/0 (socket says 172.21.15.27:38030) 2024-09-21T00:27:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff55fe00640 1 -- 172.21.15.27:0/2758152235 learned_addr learned my addr 172.21.15.27:0/2758152235 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff55fe00640 1 -- 172.21.15.27:0/2758152235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff560102ca0 msgr2=0x7ff56019cb70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:27:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff55fe00640 1 --2- 172.21.15.27:0/2758152235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff560102ca0 0x7ff56019cb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff55fe00640 1 -- 172.21.15.27:0/2758152235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff56010ff00 msgr2=0x7ff5601a1e00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:27:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff55fe00640 1 --2- 172.21.15.27:0/2758152235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff56010ff00 0x7ff5601a1e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff55fe00640 1 -- 172.21.15.27:0/2758152235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5601a2500 con 0x7ff560102350 2024-09-21T00:27:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff55fe00640 1 --2- 172.21.15.27:0/2758152235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff560102350 0x7ff56019c5d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff550005940 tx=0x7ff550004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff55cc00640 1 -- 172.21.15.27:0/2758152235 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55003f030 con 0x7ff560102350 2024-09-21T00:27:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff55cc00640 1 -- 172.21.15.27:0/2758152235 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff550003cb0 con 0x7ff560102350 2024-09-21T00:27:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff55cc00640 1 -- 172.21.15.27:0/2758152235 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550035070 con 0x7ff560102350 2024-09-21T00:27:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff565600640 1 -- 172.21.15.27:0/2758152235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5601a2790 con 0x7ff560102350 2024-09-21T00:27:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.730+0000 7ff565600640 1 -- 172.21.15.27:0/2758152235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5601a8f80 con 0x7ff560102350 2024-09-21T00:27:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.736+0000 7ff56dbfe740 1 Processor -- start 2024-09-21T00:27:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.736+0000 7ff56dbfe740 1 -- start start 2024-09-21T00:27:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.736+0000 7ff55cc00640 1 -- 172.21.15.27:0/2758152235 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff550042030 con 0x7ff560102350 2024-09-21T00:27:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.736+0000 7ff56dbfe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d6241b0 0x55ba5d9d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.736+0000 7ff56dbfe740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d5fb2f0 0x55ba5d9d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.736+0000 7ff56dbfe740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d6034d0 0x55ba5d9d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.736+0000 7ff56dbfe740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ba5d93e6b0 con 0x55ba5d5fb2f0 2024-09-21T00:27:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.736+0000 7ff56dbfe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ba5d92dae0 con 0x55ba5d6241b0 2024-09-21T00:27:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.736+0000 7ff56dbfe740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ba5d9318f0 con 0x55ba5d6034d0 2024-09-21T00:27:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.741+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d5fb2f0 0x55ba5d9d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.741+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d5fb2f0 0x55ba5d9d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38040/0 (socket says 172.21.15.27:38040) 2024-09-21T00:27:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.741+0000 7ff55fe00640 1 -- 172.21.15.27:0/2940301077 learned_addr learned my addr 172.21.15.27:0/2940301077 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.741+0000 7ff55fe00640 1 -- 172.21.15.27:0/2940301077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d6034d0 msgr2=0x55ba5d9d20a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:27:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.741+0000 7ff55fe00640 1 --2- 172.21.15.27:0/2940301077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d6034d0 0x55ba5d9d20a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.741+0000 7ff55fe00640 1 -- 172.21.15.27:0/2940301077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d6241b0 msgr2=0x55ba5d9d1620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:27:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.741+0000 7ff55fe00640 1 --2- 172.21.15.27:0/2940301077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d6241b0 0x55ba5d9d1620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.741+0000 7ff55fe00640 1 -- 172.21.15.27:0/2940301077 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba5d8fa780 con 0x55ba5d5fb2f0 2024-09-21T00:27:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff55fe00640 1 --2- 172.21.15.27:0/2940301077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d5fb2f0 0x55ba5d9d1b60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff550104d30 tx=0x7ff55010bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f232b95bfb041e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff55cc00640 1 --2- 172.21.15.27:0/2758152235 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff5440b6740 0x7ff5440b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff55cc00640 1 -- 172.21.15.27:0/2758152235 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7ff550107a50 con 0x7ff560102350 2024-09-21T00:27:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff555200640 1 -- 172.21.15.27:0/2940301077 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5500c9b40 con 0x55ba5d5fb2f0 2024-09-21T00:27:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff555200640 1 -- 172.21.15.27:0/2940301077 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff550111040 con 0x55ba5d5fb2f0 2024-09-21T00:27:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff555200640 1 -- 172.21.15.27:0/2940301077 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550003e60 con 0x55ba5d5fb2f0 2024-09-21T00:27:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff564c00640 1 --2- 172.21.15.27:0/2758152235 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff5440b6740 0x7ff5440b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2940301077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d5fb2f0 msgr2=0x55ba5d9d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2940301077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d5fb2f0 0x55ba5d9d1b60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff550104d30 tx=0x7ff55010bda0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff564c00640 1 --2- 172.21.15.27:0/2758152235 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff5440b6740 0x7ff5440b8c50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff55800aab0 tx=0x7ff558009250 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2940301077 shutdown_connections 2024-09-21T00:27:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2940301077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d6034d0 0x55ba5d9d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2940301077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d5fb2f0 0x55ba5d9d1b60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2940301077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d6241b0 0x55ba5d9d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2940301077 >> 172.21.15.27:0/2940301077 conn(0x55ba5d618300 msgr2=0x55ba5d9cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2940301077 shutdown_connections 2024-09-21T00:27:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2940301077 wait complete. 2024-09-21T00:27:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 -- start start 2024-09-21T00:27:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 Processor -- start 2024-09-21T00:27:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d5fb2f0 0x55ba5d9dccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d6034d0 0x55ba5d9dd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d6241b0 0x55ba5d9e8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ba5d865ab0 con 0x55ba5d6034d0 2024-09-21T00:27:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ba5d8d4570 con 0x55ba5d6241b0 2024-09-21T00:27:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ba5d9dfb60 con 0x55ba5d5fb2f0 2024-09-21T00:27:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff56dbfe740 10 client.0 _ll_register_callbacks cb 0x55ba5d8e3100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff55f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d6034d0 0x55ba5d9dd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff55f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d6034d0 0x55ba5d9dd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38052/0 (socket says 172.21.15.27:38052) 2024-09-21T00:27:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.742+0000 7ff55f400640 1 -- 172.21.15.27:0/2846123774 learned_addr learned my addr 172.21.15.27:0/2846123774 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff55f400640 1 -- 172.21.15.27:0/2846123774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d5fb2f0 msgr2=0x55ba5d9dccb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:27:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff55f400640 1 --2- 172.21.15.27:0/2846123774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d5fb2f0 0x55ba5d9dccb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff55f400640 1 -- 172.21.15.27:0/2846123774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d6241b0 msgr2=0x55ba5d9e8e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:27:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff55f400640 1 --2- 172.21.15.27:0/2846123774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d6241b0 0x55ba5d9e8e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff55f400640 1 -- 172.21.15.27:0/2846123774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba5d68f790 con 0x55ba5d6034d0 2024-09-21T00:27:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff55f400640 1 --2- 172.21.15.27:0/2846123774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d6034d0 0x55ba5d9dd1f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009d80 tx=0x7ff54c00b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c36330c83dc899 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff549600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:27:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff549600640 1 -- 172.21.15.27:0/2846123774 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c011420 con 0x55ba5d6034d0 2024-09-21T00:27:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2846123774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff550111040 con 0x55ba5d6034d0 2024-09-21T00:27:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff549600640 1 -- 172.21.15.27:0/2846123774 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7ff54c0115c0 con 0x55ba5d6034d0 2024-09-21T00:27:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff549600640 1 -- 172.21.15.27:0/2846123774 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff54c01f050 con 0x55ba5d6034d0 2024-09-21T00:27:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff56dbfe740 10 client.16005 fetch_fsmap learned FSMap version 30 2024-09-21T00:27:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2846123774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ba5d708620 con 0x55ba5d6034d0 2024-09-21T00:27:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff549600640 1 -- 172.21.15.27:0/2846123774 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff550111040 con 0x55ba5d6034d0 2024-09-21T00:27:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff549600640 1 -- 172.21.15.27:0/2846123774 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff54c01a9a0 con 0x55ba5d6034d0 2024-09-21T00:27:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff56dbfe740 10 client.16005 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T00:27:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff56dbfe740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:27:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff56dbfe740 20 client.16005 populate_metadata read hostname 'smithi027' 2024-09-21T00:27:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2846123774 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ba5d5e2d80 0x55ba5d9eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff56dbfe740 4 client.16005 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:27:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.748+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2846123774 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba5d940600 con 0x55ba5d5e2d80 2024-09-21T00:27:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.754+0000 7ff55fe00640 1 --2- 172.21.15.27:0/2846123774 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ba5d5e2d80 0x55ba5d9eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.765+0000 7ff55fe00640 1 --2- 172.21.15.27:0/2846123774 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ba5d5e2d80 0x55ba5d9eeda0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fbe0765eab9c4e2 server_cookie=81fd133b94015294 in_seq=0 out_seq=0 2024-09-21T00:27:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.765+0000 7ff549600640 10 client.16005 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.777+0000 7ff549600640 1 -- 172.21.15.27:0/2846123774 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ba5d940600 con 0x55ba5d5e2d80 2024-09-21T00:27:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.777+0000 7ff549600640 10 client.16005 handle_command_reply: tid=0 2024-09-21T00:27:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.777+0000 7ff56dbfe740 1 client.16005 shutdown 2024-09-21T00:27:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.777+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2846123774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d6034d0 msgr2=0x55ba5d9dd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.777+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2846123774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d6034d0 0x55ba5d9dd1f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009d80 tx=0x7ff54c00b4b0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.777+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2846123774 shutdown_connections 2024-09-21T00:27:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.777+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2846123774 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ba5d5e2d80 0x55ba5d9eeda0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.777+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2846123774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d6241b0 0x55ba5d9e8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.777+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2846123774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d6034d0 0x55ba5d9dd1f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.777+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2846123774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d5fb2f0 0x55ba5d9dccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.777+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2846123774 >> 172.21.15.27:0/2846123774 conn(0x55ba5d618300 msgr2=0x55ba5d9c5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.777+0000 7ff549600640 0 client.16005 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.777+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2846123774 shutdown_connections 2024-09-21T00:27:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.777+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2846123774 wait complete. 2024-09-21T00:27:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.777+0000 7ff56dbfe740 20 client.16005 trim_cache size 0 max 16384 2024-09-21T00:27:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.801+0000 7ff56dbfe740 1 Processor -- start 2024-09-21T00:27:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 -- start start 2024-09-21T00:27:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d5fb2f0 0x55ba5d9e8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d6241b0 0x55ba5d9d66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d5b81c0 0x55ba5d9d6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601b1430 con 0x55ba5d5fb2f0 2024-09-21T00:27:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601aeca0 con 0x55ba5d6241b0 2024-09-21T00:27:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601acaf0 con 0x55ba5d5b81c0 2024-09-21T00:27:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d5fb2f0 0x55ba5d9e8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d5fb2f0 0x55ba5d9e8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38054/0 (socket says 172.21.15.27:38054) 2024-09-21T00:27:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff55fe00640 1 -- 172.21.15.27:0/4232640191 learned_addr learned my addr 172.21.15.27:0/4232640191 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff55fe00640 1 -- 172.21.15.27:0/4232640191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d5b81c0 msgr2=0x55ba5d9d6c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:27:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff55fe00640 1 --2- 172.21.15.27:0/4232640191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d5b81c0 0x55ba5d9d6c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff55fe00640 1 -- 172.21.15.27:0/4232640191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d6241b0 msgr2=0x55ba5d9d66e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:27:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff55fe00640 1 --2- 172.21.15.27:0/4232640191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d6241b0 0x55ba5d9d66e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff55fe00640 1 -- 172.21.15.27:0/4232640191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba5d9cfe90 con 0x55ba5d5fb2f0 2024-09-21T00:27:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff55fe00640 1 --2- 172.21.15.27:0/4232640191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d5fb2f0 0x55ba5d9e8d50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff5500c9ac0 tx=0x7ff55002a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9f13970303a549 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff54be00640 1 -- 172.21.15.27:0/4232640191 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550049070 con 0x55ba5d5fb2f0 2024-09-21T00:27:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff54be00640 1 -- 172.21.15.27:0/4232640191 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff55010b470 con 0x55ba5d5fb2f0 2024-09-21T00:27:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff54be00640 1 -- 172.21.15.27:0/4232640191 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5500ccc60 con 0x55ba5d5fb2f0 2024-09-21T00:27:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 -- 172.21.15.27:0/4232640191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d5fb2f0 msgr2=0x55ba5d9e8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/4232640191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d5fb2f0 0x55ba5d9e8d50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff5500c9ac0 tx=0x7ff55002a440 comp rx=0 tx=0).stop 2024-09-21T00:27:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 -- 172.21.15.27:0/4232640191 shutdown_connections 2024-09-21T00:27:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/4232640191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d5b81c0 0x55ba5d9d6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/4232640191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d6241b0 0x55ba5d9d66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/4232640191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d5fb2f0 0x55ba5d9e8d50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 -- 172.21.15.27:0/4232640191 >> 172.21.15.27:0/4232640191 conn(0x55ba5d618300 msgr2=0x55ba5d9db510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 -- 172.21.15.27:0/4232640191 shutdown_connections 2024-09-21T00:27:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 -- 172.21.15.27:0/4232640191 wait complete. 2024-09-21T00:27:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.807+0000 7ff56dbfe740 1 -- start start 2024-09-21T00:27:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff56dbfe740 1 Processor -- start 2024-09-21T00:27:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff56dbfe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d5b81c0 0x55ba5d9e4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff56dbfe740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d6241b0 0x55ba5d9e52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff56dbfe740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d9e58d0 0x55ba5da52bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff56dbfe740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601ad160 con 0x55ba5d9e58d0 2024-09-21T00:27:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff56dbfe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601ae5d0 con 0x55ba5d5b81c0 2024-09-21T00:27:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff56dbfe740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601ad830 con 0x55ba5d6241b0 2024-09-21T00:27:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff56dbfe740 10 client.0 _ll_register_callbacks cb 0x7ff54c006e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d5b81c0 0x55ba5d9e4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d5b81c0 0x55ba5d9e4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:52924/0 (socket says 172.21.15.27:52924) 2024-09-21T00:27:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff55fe00640 1 -- 172.21.15.27:0/2601374514 learned_addr learned my addr 172.21.15.27:0/2601374514 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff564c00640 1 --2- 172.21.15.27:0/2601374514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d9e58d0 0x55ba5da52bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff564c00640 1 -- 172.21.15.27:0/2601374514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d6241b0 msgr2=0x55ba5d9e52f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:27:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff564c00640 1 --2- 172.21.15.27:0/2601374514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d6241b0 0x55ba5d9e52f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff564c00640 1 -- 172.21.15.27:0/2601374514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d5b81c0 msgr2=0x55ba5d9e4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff564c00640 1 --2- 172.21.15.27:0/2601374514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d5b81c0 0x55ba5d9e4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff564c00640 1 -- 172.21.15.27:0/2601374514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba5da53310 con 0x55ba5d9e58d0 2024-09-21T00:27:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff564c00640 1 --2- 172.21.15.27:0/2601374514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d9e58d0 0x55ba5da52bb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff55800afd0 tx=0x7ff558004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10784811cfa65dc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff549600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:27:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff549600640 1 -- 172.21.15.27:0/2601374514 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff558018ce0 con 0x55ba5d9e58d0 2024-09-21T00:27:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff549600640 1 -- 172.21.15.27:0/2601374514 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9431+0+0 (secure 0 0 0) 0x7ff558023480 con 0x55ba5d9e58d0 2024-09-21T00:27:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2601374514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff55010b470 con 0x55ba5d9e58d0 2024-09-21T00:27:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff549600640 1 -- 172.21.15.27:0/2601374514 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff558023680 con 0x55ba5d9e58d0 2024-09-21T00:27:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff56dbfe740 10 client.16011 fetch_fsmap learned FSMap version 30 2024-09-21T00:27:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.812+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2601374514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ba5d9cb090 con 0x55ba5d9e58d0 2024-09-21T00:27:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.817+0000 7ff549600640 1 -- 172.21.15.27:0/2601374514 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff55010b470 con 0x55ba5d9e58d0 2024-09-21T00:27:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.817+0000 7ff549600640 1 -- 172.21.15.27:0/2601374514 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff55802d8e0 con 0x55ba5d9e58d0 2024-09-21T00:27:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.817+0000 7ff56dbfe740 10 client.16011 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T00:27:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.817+0000 7ff56dbfe740 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:27:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.817+0000 7ff56dbfe740 20 client.16011 populate_metadata read hostname 'smithi027' 2024-09-21T00:27:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.817+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2601374514 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ba5d9cb850 0x55ba5d9cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.817+0000 7ff56dbfe740 4 client.16011 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:27:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.817+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2601374514 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba5d93c0b0 con 0x55ba5d9cb850 2024-09-21T00:27:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.817+0000 7ff55fe00640 1 --2- 172.21.15.27:0/2601374514 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ba5d9cb850 0x55ba5d9cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.872+0000 7ff549600640 10 client.16011 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:24.872+0000 7ff55fe00640 1 --2- 172.21.15.27:0/2601374514 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ba5d9cb850 0x55ba5d9cdc70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf0efc8de5947c37 server_cookie=7cb4fa2d285761d1 in_seq=0 out_seq=0 2024-09-21T00:27:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.582+0000 7ff549600640 1 -- 172.21.15.27:0/2601374514 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba5d93c0b0 con 0x55ba5d9cb850 2024-09-21T00:27:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.582+0000 7ff549600640 10 client.16011 handle_command_reply: tid=0 2024-09-21T00:27:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.582+0000 7ff56dbfe740 1 client.16011 shutdown 2024-09-21T00:27:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.585+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2601374514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d9e58d0 msgr2=0x55ba5da52bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.585+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2601374514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d9e58d0 0x55ba5da52bb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff55800afd0 tx=0x7ff558004ad0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2601374514 shutdown_connections 2024-09-21T00:27:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2601374514 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ba5d9cb850 0x55ba5d9cdc70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2601374514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ba5d9e58d0 0x55ba5da52bb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2601374514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba5d6241b0 0x55ba5d9e52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff56dbfe740 1 --2- 172.21.15.27:0/2601374514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ba5d5b81c0 0x55ba5d9e4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2601374514 >> 172.21.15.27:0/2601374514 conn(0x55ba5d618300 msgr2=0x55ba5d9d66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff549600640 0 client.16011 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2601374514 shutdown_connections 2024-09-21T00:27:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff56dbfe740 1 -- 172.21.15.27:0/2601374514 wait complete. 2024-09-21T00:27:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff56dbfe740 20 client.16011 trim_cache size 0 max 16384 2024-09-21T00:27:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:27:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:27:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:27:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:27:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff565600640 1 -- 172.21.15.27:0/2758152235 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff5440b6740 msgr2=0x7ff5440b8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff565600640 1 --2- 172.21.15.27:0/2758152235 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff5440b6740 0x7ff5440b8c50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff55800aab0 tx=0x7ff558009250 comp rx=0 tx=0).stop 2024-09-21T00:27:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff565600640 1 -- 172.21.15.27:0/2758152235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff560102350 msgr2=0x7ff56019c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff565600640 1 --2- 172.21.15.27:0/2758152235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff560102350 0x7ff56019c5d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff550005940 tx=0x7ff550004360 comp rx=0 tx=0).stop 2024-09-21T00:27:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff565600640 1 -- 172.21.15.27:0/2758152235 shutdown_connections 2024-09-21T00:27:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff565600640 1 --2- 172.21.15.27:0/2758152235 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7ff5440b6740 0x7ff5440b8c50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff565600640 1 --2- 172.21.15.27:0/2758152235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff56010ff00 0x7ff5601a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff565600640 1 --2- 172.21.15.27:0/2758152235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff560102ca0 0x7ff56019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff565600640 1 --2- 172.21.15.27:0/2758152235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff560102350 0x7ff56019c5d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff565600640 1 -- 172.21.15.27:0/2758152235 >> 172.21.15.27:0/2758152235 conn(0x7ff5600fe210 msgr2=0x7ff560100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff565600640 1 -- 172.21.15.27:0/2758152235 shutdown_connections 2024-09-21T00:27:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:25.586+0000 7ff565600640 1 -- 172.21.15.27:0/2758152235 wait complete. 2024-09-21T00:27:26.141 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:27:25 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h[62840]: ignoring --setuser ceph since I am not root 2024-09-21T00:27:26.142 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:27:25 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h[62840]: ignoring --setgroup ceph since I am not root 2024-09-21T00:27:26.142 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:27:25 smithi089 ceph-mds[62844]: -- 172.21.15.89:0/461361910 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562610139860 con 0x5626142e1180 2024-09-21T00:27:26.142 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:27:25 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h[62840]: starting mds.h at 2024-09-21T00:27:26.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:26 smithi089 ceph-mon[34481]: pgmap v558: 129 pgs: 129 active+clean; 829 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-09-21T00:27:26.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:26 smithi089 ceph-mon[34481]: daemon mds.h restarted 2024-09-21T00:27:26.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:26 smithi098 ceph-mon[34725]: pgmap v558: 129 pgs: 129 active+clean; 829 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-09-21T00:27:26.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:26 smithi098 ceph-mon[34725]: daemon mds.h restarted 2024-09-21T00:27:26.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:26 smithi027 ceph-mon[30745]: pgmap v558: 129 pgs: 129 active+clean; 829 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-09-21T00:27:26.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:26 smithi027 ceph-mon[30745]: daemon mds.h restarted 2024-09-21T00:27:27.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:27 smithi089 ceph-mon[34481]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T00:27:27.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:27 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.89:6836/4028763758,v1:172.21.15.89:6837/4028763758] up:boot 2024-09-21T00:27:27.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:27 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-21T00:27:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:27 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T00:27:27.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:27 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:27:27.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:27 smithi098 ceph-mon[34725]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T00:27:27.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:27 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.89:6836/4028763758,v1:172.21.15.89:6837/4028763758] up:boot 2024-09-21T00:27:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:27 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-21T00:27:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:27 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T00:27:27.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:27 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:27:27.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:27 smithi027 ceph-mon[30745]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T00:27:27.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:27 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.89:6836/4028763758,v1:172.21.15.89:6837/4028763758] up:boot 2024-09-21T00:27:27.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:27 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-21T00:27:27.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:27 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T00:27:27.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:27 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:27:28.373 INFO:tasks.workunit.client.0.smithi027.stdout: 227 77 0 70 0 27 1 2024-09-21T00:27:28.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:28 smithi089 ceph-mon[34481]: pgmap v560: 129 pgs: 129 active+clean; 859 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.0 MiB/s wr, 525 op/s 2024-09-21T00:27:28.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:28 smithi098 ceph-mon[34725]: pgmap v560: 129 pgs: 129 active+clean; 859 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.0 MiB/s wr, 525 op/s 2024-09-21T00:27:28.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:28 smithi027 ceph-mon[30745]: pgmap v560: 129 pgs: 129 active+clean; 859 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.0 MiB/s wr, 525 op/s 2024-09-21T00:27:30.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:30 smithi089 ceph-mon[34481]: pgmap v561: 129 pgs: 129 active+clean; 865 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.7 MiB/s wr, 667 op/s 2024-09-21T00:27:30.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:30 smithi098 ceph-mon[34725]: pgmap v561: 129 pgs: 129 active+clean; 865 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.7 MiB/s wr, 667 op/s 2024-09-21T00:27:30.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:30 smithi027 ceph-mon[30745]: pgmap v561: 129 pgs: 129 active+clean; 865 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.7 MiB/s wr, 667 op/s 2024-09-21T00:27:32.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:32 smithi098 ceph-mon[34725]: pgmap v562: 129 pgs: 129 active+clean; 866 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.5 MiB/s wr, 664 op/s 2024-09-21T00:27:32.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:32 smithi027 ceph-mon[30745]: pgmap v562: 129 pgs: 129 active+clean; 866 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.5 MiB/s wr, 664 op/s 2024-09-21T00:27:32.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:32 smithi089 ceph-mon[34481]: pgmap v562: 129 pgs: 129 active+clean; 866 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.5 MiB/s wr, 664 op/s 2024-09-21T00:27:34.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:34 smithi027 ceph-mon[30745]: pgmap v563: 129 pgs: 129 active+clean; 875 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.5 MiB/s wr, 807 op/s 2024-09-21T00:27:34.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:34 smithi089 ceph-mon[34481]: pgmap v563: 129 pgs: 129 active+clean; 875 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.5 MiB/s wr, 807 op/s 2024-09-21T00:27:34.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:34 smithi098 ceph-mon[34725]: pgmap v563: 129 pgs: 129 active+clean; 875 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.5 MiB/s wr, 807 op/s 2024-09-21T00:27:36.464 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:36 smithi027 ceph-mon[30745]: pgmap v564: 129 pgs: 129 active+clean; 875 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.5 MiB/s wr, 804 op/s 2024-09-21T00:27:36.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:36 smithi089 ceph-mon[34481]: pgmap v564: 129 pgs: 129 active+clean; 875 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.5 MiB/s wr, 804 op/s 2024-09-21T00:27:36.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:36 smithi098 ceph-mon[34725]: pgmap v564: 129 pgs: 129 active+clean; 875 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.5 MiB/s wr, 804 op/s 2024-09-21T00:27:38.373 INFO:tasks.workunit.client.0.smithi027.stdout: 227 2135 0 1967 0 1245 0 2024-09-21T00:27:38.395 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:38 smithi027 ceph-mon[30745]: pgmap v565: 129 pgs: 129 active+clean; 875 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 479 op/s 2024-09-21T00:27:38.547 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:38 smithi098 ceph-mon[34725]: pgmap v565: 129 pgs: 129 active+clean; 875 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 479 op/s 2024-09-21T00:27:38.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:38 smithi089 ceph-mon[34481]: pgmap v565: 129 pgs: 129 active+clean; 875 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 479 op/s 2024-09-21T00:27:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 -- 172.21.15.27:0/2562724397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dd8109240 msgr2=0x7f1dd8109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 --2- 172.21.15.27:0/2562724397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dd8109240 0x7f1dd8109650 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009900 tx=0x7f1dc002a3f0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 -- 172.21.15.27:0/2562724397 shutdown_connections 2024-09-21T00:27:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 --2- 172.21.15.27:0/2562724397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8109b90 0x7f1dd81103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 --2- 172.21.15.27:0/2562724397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dd8109240 0x7f1dd8109650 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 --2- 172.21.15.27:0/2562724397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dd8103910 0x7f1dd8103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 -- 172.21.15.27:0/2562724397 >> 172.21.15.27:0/2562724397 conn(0x7f1dd80ff390 msgr2=0x7f1dd81017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 -- 172.21.15.27:0/2562724397 shutdown_connections 2024-09-21T00:27:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 -- 172.21.15.27:0/2562724397 wait complete. 2024-09-21T00:27:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 Processor -- start 2024-09-21T00:27:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 -- start start 2024-09-21T00:27:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dd8103910 0x7f1dd819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8109240 0x7f1dd819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dd8109b90 0x7f1dd81a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd8113850 con 0x7f1dd8109240 2024-09-21T00:27:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81136d0 con 0x7f1dd8109b90 2024-09-21T00:27:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81139d0 con 0x7f1dd8103910 2024-09-21T00:27:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1dcfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8109240 0x7f1dd819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1dd7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dd8103910 0x7f1dd819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1dcfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8109240 0x7f1dd819ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60190/0 (socket says 172.21.15.27:60190) 2024-09-21T00:27:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1dcfe00640 1 -- 172.21.15.27:0/2959270266 learned_addr learned my addr 172.21.15.27:0/2959270266 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1dd7e00640 1 -- 172.21.15.27:0/2959270266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dd8109b90 msgr2=0x7f1dd81a1d00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:27:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1dd7e00640 1 --2- 172.21.15.27:0/2959270266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dd8109b90 0x7f1dd81a1d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1dd7e00640 1 -- 172.21.15.27:0/2959270266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8109240 msgr2=0x7f1dd819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1dd7e00640 1 --2- 172.21.15.27:0/2959270266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8109240 0x7f1dd819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1dd7e00640 1 -- 172.21.15.27:0/2959270266 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd81a2400 con 0x7f1dd8103910 2024-09-21T00:27:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1dcfe00640 1 --2- 172.21.15.27:0/2959270266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8109240 0x7f1dd819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:27:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1dd7e00640 1 --2- 172.21.15.27:0/2959270266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dd8103910 0x7f1dd819c4d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800e9f0 tx=0x7f1dc800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1dd5600640 1 -- 172.21.15.27:0/2959270266 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc800ccc0 con 0x7f1dd8103910 2024-09-21T00:27:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1dd5600640 1 -- 172.21.15.27:0/2959270266 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc8005020 con 0x7f1dd8103910 2024-09-21T00:27:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1dd5600640 1 -- 172.21.15.27:0/2959270266 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8004a00 con 0x7f1dd8103910 2024-09-21T00:27:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 -- 172.21.15.27:0/2959270266 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd81a26f0 con 0x7f1dd8103910 2024-09-21T00:27:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.798+0000 7f1ddd800640 1 -- 172.21.15.27:0/2959270266 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd81a8e80 con 0x7f1dd8103910 2024-09-21T00:27:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1de600c740 1 Processor -- start 2024-09-21T00:27:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1de600c740 1 -- start start 2024-09-21T00:27:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1de600c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5b21b0 0x562edb956e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1de600c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5892f0 0x562edb957350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1de600c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5914d0 0x562edb957890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1de600c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562edb8cc6b0 con 0x562edb5b21b0 2024-09-21T00:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1de600c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562edb8bbae0 con 0x562edb5914d0 2024-09-21T00:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1de600c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562edb8bf8f0 con 0x562edb5892f0 2024-09-21T00:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1ddce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5b21b0 0x562edb956e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1ddce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5b21b0 0x562edb956e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60192/0 (socket says 172.21.15.27:60192) 2024-09-21T00:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1dcfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5914d0 0x562edb957890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1ddce00640 1 -- 172.21.15.27:0/2857149652 learned_addr learned my addr 172.21.15.27:0/2857149652 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1dd5600640 1 -- 172.21.15.27:0/2959270266 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1dc8004ba0 con 0x7f1dd8103910 2024-09-21T00:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1dcfe00640 1 -- 172.21.15.27:0/2857149652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5892f0 msgr2=0x562edb957350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1dcfe00640 1 --2- 172.21.15.27:0/2857149652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5892f0 0x562edb957350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1dcfe00640 1 -- 172.21.15.27:0/2857149652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5b21b0 msgr2=0x562edb956e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1dcfe00640 1 --2- 172.21.15.27:0/2857149652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5b21b0 0x562edb956e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1dcfe00640 1 -- 172.21.15.27:0/2857149652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562edb888780 con 0x562edb5914d0 2024-09-21T00:27:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.804+0000 7f1ddce00640 1 --2- 172.21.15.27:0/2857149652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5b21b0 0x562edb956e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:27:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dcfe00640 1 --2- 172.21.15.27:0/2857149652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5914d0 0x562edb957890 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0000c00 tx=0x7f1dc0005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e5bfb9418558049 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dcd600640 1 -- 172.21.15.27:0/2857149652 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0035040 con 0x562edb5914d0 2024-09-21T00:27:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dcd600640 1 -- 172.21.15.27:0/2857149652 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc0031720 con 0x562edb5914d0 2024-09-21T00:27:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dcd600640 1 -- 172.21.15.27:0/2857149652 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc00047a0 con 0x562edb5914d0 2024-09-21T00:27:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 -- 172.21.15.27:0/2857149652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5914d0 msgr2=0x562edb957890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 --2- 172.21.15.27:0/2857149652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5914d0 0x562edb957890 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0000c00 tx=0x7f1dc0005e20 comp rx=0 tx=0).stop 2024-09-21T00:27:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 -- 172.21.15.27:0/2857149652 shutdown_connections 2024-09-21T00:27:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 --2- 172.21.15.27:0/2857149652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5914d0 0x562edb957890 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 --2- 172.21.15.27:0/2857149652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5892f0 0x562edb957350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 --2- 172.21.15.27:0/2857149652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5b21b0 0x562edb956e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 -- 172.21.15.27:0/2857149652 >> 172.21.15.27:0/2857149652 conn(0x562edb5a6300 msgr2=0x562edb954d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dd5600640 1 --2- 172.21.15.27:0/2959270266 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1dbc0b6670 0x7f1dbc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 -- 172.21.15.27:0/2857149652 shutdown_connections 2024-09-21T00:27:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 -- 172.21.15.27:0/2857149652 wait complete. 2024-09-21T00:27:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dd5600640 1 -- 172.21.15.27:0/2959270266 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9431+0+0 (secure 0 0 0) 0x7f1dc80e7b60 con 0x7f1dd8103910 2024-09-21T00:27:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1ddce00640 1 --2- 172.21.15.27:0/2959270266 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1dbc0b6670 0x7f1dbc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 -- start start 2024-09-21T00:27:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1ddce00640 1 --2- 172.21.15.27:0/2959270266 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1dbc0b6670 0x7f1dbc0b8b80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00049a0 tx=0x7f1dd0009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 Processor -- start 2024-09-21T00:27:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5892f0 0x562edb9624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5914d0 0x562edb962a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5b21b0 0x562edb96e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562edb7f3ab0 con 0x562edb5914d0 2024-09-21T00:27:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562edb862570 con 0x562edb5b21b0 2024-09-21T00:27:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562edb965530 con 0x562edb5892f0 2024-09-21T00:27:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 10 client.0 _ll_register_callbacks cb 0x562edb871100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dcfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5914d0 0x562edb962a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dd7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5892f0 0x562edb9624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dcfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5914d0 0x562edb962a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60198/0 (socket says 172.21.15.27:60198) 2024-09-21T00:27:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dcfe00640 1 -- 172.21.15.27:0/101422469 learned_addr learned my addr 172.21.15.27:0/101422469 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dcfe00640 1 -- 172.21.15.27:0/101422469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5892f0 msgr2=0x562edb9624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dcfe00640 1 --2- 172.21.15.27:0/101422469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5892f0 0x562edb9624f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dcfe00640 1 -- 172.21.15.27:0/101422469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5b21b0 msgr2=0x562edb96e820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:27:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dcfe00640 1 --2- 172.21.15.27:0/101422469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5b21b0 0x562edb96e820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dcfe00640 1 -- 172.21.15.27:0/101422469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562edb61d790 con 0x562edb5914d0 2024-09-21T00:27:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dd7e00640 1 --2- 172.21.15.27:0/101422469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5892f0 0x562edb9624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:27:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dcfe00640 1 --2- 172.21.15.27:0/101422469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5914d0 0x562edb962a30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0004620 tx=0x7f1dc00095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d150074474dd8e7e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dc6000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:27:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dc6000640 1 -- 172.21.15.27:0/101422469 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc004e070 con 0x562edb5914d0 2024-09-21T00:27:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 -- 172.21.15.27:0/101422469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dc0031720 con 0x562edb5914d0 2024-09-21T00:27:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dc6000640 1 -- 172.21.15.27:0/101422469 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9431+0+0 (secure 0 0 0) 0x7f1dc0046860 con 0x562edb5914d0 2024-09-21T00:27:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dc6000640 1 -- 172.21.15.27:0/101422469 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1dc005a050 con 0x562edb5914d0 2024-09-21T00:27:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 10 client.16023 fetch_fsmap learned FSMap version 32 2024-09-21T00:27:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1de600c740 1 -- 172.21.15.27:0/101422469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562edb696620 con 0x562edb5914d0 2024-09-21T00:27:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dc6000640 1 -- 172.21.15.27:0/101422469 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc0031720 con 0x562edb5914d0 2024-09-21T00:27:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.806+0000 7f1dc6000640 1 -- 172.21.15.27:0/101422469 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f1dc003ca80 con 0x562edb5914d0 2024-09-21T00:27:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 10 client.16023 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T00:27:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:27:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 20 client.16023 populate_metadata read hostname 'smithi027' 2024-09-21T00:27:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 1 --2- 172.21.15.27:0/101422469 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562edb570d80 0x562edb974390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 4 client.16023 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:27:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 1 -- 172.21.15.27:0/101422469 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562edb8ce600 con 0x562edb570d80 2024-09-21T00:27:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1dd7e00640 1 --2- 172.21.15.27:0/101422469 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562edb570d80 0x562edb974390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1dd7e00640 1 --2- 172.21.15.27:0/101422469 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562edb570d80 0x562edb974390 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f95545c8b576287 server_cookie=840b0174ccef5c29 in_seq=0 out_seq=0 2024-09-21T00:27:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1dc6000640 10 client.16023 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1dc6000640 1 -- 172.21.15.27:0/101422469 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562edb8ce600 con 0x562edb570d80 2024-09-21T00:27:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1dc6000640 10 client.16023 handle_command_reply: tid=0 2024-09-21T00:27:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 1 client.16023 shutdown 2024-09-21T00:27:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 1 -- 172.21.15.27:0/101422469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5914d0 msgr2=0x562edb962a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 1 --2- 172.21.15.27:0/101422469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5914d0 0x562edb962a30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0004620 tx=0x7f1dc00095a0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 1 -- 172.21.15.27:0/101422469 shutdown_connections 2024-09-21T00:27:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 1 --2- 172.21.15.27:0/101422469 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562edb570d80 0x562edb974390 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 1 --2- 172.21.15.27:0/101422469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5b21b0 0x562edb96e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 1 --2- 172.21.15.27:0/101422469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5914d0 0x562edb962a30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 1 --2- 172.21.15.27:0/101422469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5892f0 0x562edb9624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1dc6000640 0 client.16023 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 1 -- 172.21.15.27:0/101422469 >> 172.21.15.27:0/101422469 conn(0x562edb5a6300 msgr2=0x562edb957e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 1 -- 172.21.15.27:0/101422469 shutdown_connections 2024-09-21T00:27:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 1 -- 172.21.15.27:0/101422469 wait complete. 2024-09-21T00:27:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.812+0000 7f1de600c740 20 client.16023 trim_cache size 0 max 16384 2024-09-21T00:27:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 Processor -- start 2024-09-21T00:27:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 -- start start 2024-09-21T00:27:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5892f0 0x562edb9629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5914d0 0x562edb954cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5b21b0 0x562edb96e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81b5750 con 0x562edb5892f0 2024-09-21T00:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81b2fc0 con 0x562edb5914d0 2024-09-21T00:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81b0e10 con 0x562edb5b21b0 2024-09-21T00:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dd7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5892f0 0x562edb9629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dcfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5914d0 0x562edb954cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dcfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5914d0 0x562edb954cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:42640/0 (socket says 172.21.15.27:42640) 2024-09-21T00:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dcfe00640 1 -- 172.21.15.27:0/1312016846 learned_addr learned my addr 172.21.15.27:0/1312016846 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dcfe00640 1 -- 172.21.15.27:0/1312016846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5b21b0 msgr2=0x562edb96e720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dcfe00640 1 --2- 172.21.15.27:0/1312016846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5b21b0 0x562edb96e720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dcfe00640 1 -- 172.21.15.27:0/1312016846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5892f0 msgr2=0x562edb9629d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dcfe00640 1 --2- 172.21.15.27:0/1312016846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5892f0 0x562edb9629d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dcfe00640 1 -- 172.21.15.27:0/1312016846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562edb955200 con 0x562edb5914d0 2024-09-21T00:27:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dd7e00640 1 --2- 172.21.15.27:0/1312016846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5892f0 0x562edb9629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:27:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dcfe00640 1 --2- 172.21.15.27:0/1312016846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5914d0 0x562edb954cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0058040 tx=0x7f1dc0032ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1809ed140763263 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dccc00640 1 -- 172.21.15.27:0/1312016846 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc004e070 con 0x562edb5914d0 2024-09-21T00:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dccc00640 1 -- 172.21.15.27:0/1312016846 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc0052d00 con 0x562edb5914d0 2024-09-21T00:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dccc00640 1 -- 172.21.15.27:0/1312016846 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0035040 con 0x562edb5914d0 2024-09-21T00:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 -- 172.21.15.27:0/1312016846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5914d0 msgr2=0x562edb954cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 --2- 172.21.15.27:0/1312016846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5914d0 0x562edb954cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0058040 tx=0x7f1dc0032ed0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 -- 172.21.15.27:0/1312016846 shutdown_connections 2024-09-21T00:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 --2- 172.21.15.27:0/1312016846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5b21b0 0x562edb96e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 --2- 172.21.15.27:0/1312016846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5914d0 0x562edb954cc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 --2- 172.21.15.27:0/1312016846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5892f0 0x562edb9629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 -- 172.21.15.27:0/1312016846 >> 172.21.15.27:0/1312016846 conn(0x562edb5a6300 msgr2=0x562edb9545b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 -- 172.21.15.27:0/1312016846 shutdown_connections 2024-09-21T00:27:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 -- 172.21.15.27:0/1312016846 wait complete. 2024-09-21T00:27:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 -- start start 2024-09-21T00:27:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 Processor -- start 2024-09-21T00:27:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5892f0 0x562edb9675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5914d0 0x562edb967b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5b21b0 0x562edb9d6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81b1480 con 0x562edb5892f0 2024-09-21T00:27:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81b28f0 con 0x562edb5b21b0 2024-09-21T00:27:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81b1b50 con 0x562edb5914d0 2024-09-21T00:27:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1de600c740 10 client.0 _ll_register_callbacks cb 0x7f1dc0040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dd7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5892f0 0x562edb9675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dd7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5892f0 0x562edb9675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60206/0 (socket says 172.21.15.27:60206) 2024-09-21T00:27:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dd7e00640 1 -- 172.21.15.27:0/49190573 learned_addr learned my addr 172.21.15.27:0/49190573 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1dcfe00640 1 --2- 172.21.15.27:0/49190573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5914d0 0x562edb967b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.829+0000 7f1ddce00640 1 --2- 172.21.15.27:0/49190573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5b21b0 0x562edb9d6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dcfe00640 1 -- 172.21.15.27:0/49190573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5b21b0 msgr2=0x562edb9d6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dcfe00640 1 --2- 172.21.15.27:0/49190573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5b21b0 0x562edb9d6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dcfe00640 1 -- 172.21.15.27:0/49190573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5892f0 msgr2=0x562edb9675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dcfe00640 1 --2- 172.21.15.27:0/49190573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5892f0 0x562edb9675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dcfe00640 1 -- 172.21.15.27:0/49190573 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562edb9d7160 con 0x562edb5914d0 2024-09-21T00:27:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dd7e00640 1 --2- 172.21.15.27:0/49190573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5892f0 0x562edb9675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:27:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dcfe00640 1 --2- 172.21.15.27:0/49190573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5914d0 0x562edb967b70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x562edb957d30 tx=0x7f1dc00095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf20d2ae0b4f032f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dc6000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:27:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dc6000640 1 -- 172.21.15.27:0/49190573 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc004e070 con 0x562edb5914d0 2024-09-21T00:27:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1de600c740 1 -- 172.21.15.27:0/49190573 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dc0052d00 con 0x562edb5914d0 2024-09-21T00:27:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dc6000640 1 -- 172.21.15.27:0/49190573 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9431+0+0 (secure 0 0 0) 0x7f1dc0004030 con 0x562edb5914d0 2024-09-21T00:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dc6000640 1 -- 172.21.15.27:0/49190573 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1dc006f050 con 0x562edb5914d0 2024-09-21T00:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1de600c740 10 client.26056 fetch_fsmap learned FSMap version 32 2024-09-21T00:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1de600c740 1 -- 172.21.15.27:0/49190573 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562edb9de490 con 0x562edb5914d0 2024-09-21T00:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dc6000640 1 -- 172.21.15.27:0/49190573 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc0052d00 con 0x562edb5914d0 2024-09-21T00:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dc6000640 1 -- 172.21.15.27:0/49190573 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f1dc0065030 con 0x562edb5914d0 2024-09-21T00:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1de600c740 10 client.26056 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T00:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1de600c740 10 client.26056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1de600c740 20 client.26056 populate_metadata read hostname 'smithi027' 2024-09-21T00:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1de600c740 1 --2- 172.21.15.27:0/49190573 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562edb5461c0 0x562edb9e0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1de600c740 4 client.26056 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:27:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1de600c740 1 -- 172.21.15.27:0/49190573 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562edb8ca0b0 con 0x562edb5461c0 2024-09-21T00:27:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dd7e00640 1 --2- 172.21.15.27:0/49190573 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562edb5461c0 0x562edb9e0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dd7e00640 1 --2- 172.21.15.27:0/49190573 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562edb5461c0 0x562edb9e0bf0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be948a4c19b7f04b server_cookie=42de99db3d14d57e in_seq=0 out_seq=0 2024-09-21T00:27:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:38.834+0000 7f1dc6000640 10 client.26056 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1dc6000640 1 -- 172.21.15.27:0/49190573 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562edb8ca0b0 con 0x562edb5461c0 2024-09-21T00:27:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1dc6000640 10 client.26056 handle_command_reply: tid=0 2024-09-21T00:27:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1de600c740 1 client.26056 shutdown 2024-09-21T00:27:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1de600c740 1 -- 172.21.15.27:0/49190573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5914d0 msgr2=0x562edb967b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1de600c740 1 --2- 172.21.15.27:0/49190573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5914d0 0x562edb967b70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x562edb957d30 tx=0x7f1dc00095a0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1de600c740 1 -- 172.21.15.27:0/49190573 shutdown_connections 2024-09-21T00:27:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1de600c740 1 --2- 172.21.15.27:0/49190573 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562edb5461c0 0x562edb9e0bf0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1de600c740 1 --2- 172.21.15.27:0/49190573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562edb5b21b0 0x562edb9d6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1de600c740 1 --2- 172.21.15.27:0/49190573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562edb5914d0 0x562edb967b70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1dc6000640 0 client.26056 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1de600c740 1 --2- 172.21.15.27:0/49190573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562edb5892f0 0x562edb9675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1de600c740 1 -- 172.21.15.27:0/49190573 >> 172.21.15.27:0/49190573 conn(0x562edb5a6300 msgr2=0x562edb957e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1de600c740 1 -- 172.21.15.27:0/49190573 shutdown_connections 2024-09-21T00:27:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1de600c740 1 -- 172.21.15.27:0/49190573 wait complete. 2024-09-21T00:27:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1de600c740 20 client.26056 trim_cache size 0 max 16384 2024-09-21T00:27:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:27:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:27:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:27:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:27:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1ddd800640 1 -- 172.21.15.27:0/2959270266 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1dbc0b6670 msgr2=0x7f1dbc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1ddd800640 1 --2- 172.21.15.27:0/2959270266 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1dbc0b6670 0x7f1dbc0b8b80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00049a0 tx=0x7f1dd0009500 comp rx=0 tx=0).stop 2024-09-21T00:27:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1ddd800640 1 -- 172.21.15.27:0/2959270266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dd8103910 msgr2=0x7f1dd819c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1ddd800640 1 --2- 172.21.15.27:0/2959270266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dd8103910 0x7f1dd819c4d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800e9f0 tx=0x7f1dc800eeb0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1ddd800640 1 -- 172.21.15.27:0/2959270266 shutdown_connections 2024-09-21T00:27:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1ddd800640 1 --2- 172.21.15.27:0/2959270266 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1dbc0b6670 0x7f1dbc0b8b80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1ddd800640 1 --2- 172.21.15.27:0/2959270266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dd8109b90 0x7f1dd81a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1ddd800640 1 --2- 172.21.15.27:0/2959270266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8109240 0x7f1dd819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1ddd800640 1 --2- 172.21.15.27:0/2959270266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dd8103910 0x7f1dd819c4d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1ddd800640 1 -- 172.21.15.27:0/2959270266 >> 172.21.15.27:0/2959270266 conn(0x7f1dd80ff390 msgr2=0x7f1dd810e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1ddd800640 1 -- 172.21.15.27:0/2959270266 shutdown_connections 2024-09-21T00:27:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:39.029+0000 7f1ddd800640 1 -- 172.21.15.27:0/2959270266 wait complete. 2024-09-21T00:27:40.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:40 smithi027 ceph-mon[30745]: pgmap v566: 129 pgs: 129 active+clean; 875 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 379 op/s 2024-09-21T00:27:40.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:40 smithi089 ceph-mon[34481]: pgmap v566: 129 pgs: 129 active+clean; 875 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 379 op/s 2024-09-21T00:27:40.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:40 smithi098 ceph-mon[34725]: pgmap v566: 129 pgs: 129 active+clean; 875 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 379 op/s 2024-09-21T00:27:40.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.890+0000 7f86e8c00640 1 -- 172.21.15.27:0/2057353471 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x561bd33546a0 con 0x561bd346af10 2024-09-21T00:27:40.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.890+0000 7f86e8c00640 10 client.26038 handle_command_reply: tid=0 2024-09-21T00:27:40.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.890+0000 7f870ce0c740 1 client.26038 shutdown 2024-09-21T00:27:40.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.891+0000 7f870ce0c740 1 -- 172.21.15.27:0/2057353471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd2fcd570 msgr2=0x561bd33f4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:40.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.891+0000 7f870ce0c740 1 --2- 172.21.15.27:0/2057353471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd2fcd570 0x561bd33f4130 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f86f8006010 tx=0x7f86f80095c0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.891+0000 7f870ce0c740 1 -- 172.21.15.27:0/2057353471 shutdown_connections 2024-09-21T00:27:40.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.891+0000 7f870ce0c740 1 --2- 172.21.15.27:0/2057353471 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561bd346af10 0x561bd346d330 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.891+0000 7f870ce0c740 1 --2- 172.21.15.27:0/2057353471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561bd2fcd570 0x561bd33f4130 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.891+0000 7f870ce0c740 1 --2- 172.21.15.27:0/2057353471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561bd3039840 0x561bd33f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.891+0000 7f870ce0c740 1 --2- 172.21.15.27:0/2057353471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561bd3018b60 0x561bd33f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.891+0000 7f86e8c00640 0 client.26038 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:40.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.891+0000 7f870ce0c740 1 -- 172.21.15.27:0/2057353471 >> 172.21.15.27:0/2057353471 conn(0x561bd302d990 msgr2=0x561bd33e8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:40.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.891+0000 7f870ce0c740 1 -- 172.21.15.27:0/2057353471 shutdown_connections 2024-09-21T00:27:40.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.891+0000 7f870ce0c740 1 -- 172.21.15.27:0/2057353471 wait complete. 2024-09-21T00:27:40.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.891+0000 7f870ce0c740 20 client.26038 trim_cache size 0 max 16384 2024-09-21T00:27:40.894 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:27:40.894 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 32, 2024-09-21T00:27:40.894 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24761, 2024-09-21T00:27:40.895 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 18, 2024-09-21T00:27:40.895 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T00:27:40.895 INFO:teuthology.orchestra.run.smithi027.stdout: "3e1e43d": { 2024-09-21T00:27:40.895 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 18, 2024-09-21T00:27:40.895 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T00:27:40.895 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:27:40.895 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T00:27:40.895 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:27:40.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:27:40.895 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T00:27:40.895 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 130.0, 2024-09-21T00:27:40.895 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T00:27:40.895 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T00:27:40.895 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T00:27:40.895 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:27:40.896 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T00:27:40.896 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:27:40.896 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:27:40.896 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:27:40.896 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:27:40.896 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:27:40.896 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:27:40.896 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:27:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.892+0000 7f8704a00640 1 -- 172.21.15.27:0/867917926 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f86e40b66f0 msgr2=0x7f86e40b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.892+0000 7f8704a00640 1 --2- 172.21.15.27:0/867917926 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f86e40b66f0 0x7f86e40b8c00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f86f802aeb0 tx=0x7f86f8031b90 comp rx=0 tx=0).stop 2024-09-21T00:27:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.892+0000 7f8704a00640 1 -- 172.21.15.27:0/867917926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87001081e0 msgr2=0x7f870010b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.892+0000 7f8704a00640 1 --2- 172.21.15.27:0/867917926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87001081e0 0x7f870010b3c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00d950 tx=0x7f86ec00de10 comp rx=0 tx=0).stop 2024-09-21T00:27:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.893+0000 7f8704a00640 1 -- 172.21.15.27:0/867917926 shutdown_connections 2024-09-21T00:27:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.893+0000 7f8704a00640 1 --2- 172.21.15.27:0/867917926 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f86e40b66f0 0x7f86e40b8c00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.893+0000 7f8704a00640 1 --2- 172.21.15.27:0/867917926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8700108b30 0x7f870010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.893+0000 7f8704a00640 1 --2- 172.21.15.27:0/867917926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87001081e0 0x7f870010b3c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.893+0000 7f8704a00640 1 --2- 172.21.15.27:0/867917926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87001028a0 0x7f870010f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.893+0000 7f8704a00640 1 -- 172.21.15.27:0/867917926 >> 172.21.15.27:0/867917926 conn(0x7f87000fe330 msgr2=0x7f87001077f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.893+0000 7f8704a00640 1 -- 172.21.15.27:0/867917926 shutdown_connections 2024-09-21T00:27:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:40.893+0000 7f8704a00640 1 -- 172.21.15.27:0/867917926 wait complete. 2024-09-21T00:27:41.909 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3e1e43d', quiesce took 54.8 seconds. Will release after: -44.8 2024-09-21T00:27:41.909 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:27:42.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:42 smithi027 ceph-mon[30745]: pgmap v567: 129 pgs: 129 active+clean; 876 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 217 op/s 2024-09-21T00:27:42.572 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:27:42.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:42 smithi089 ceph-mon[34481]: pgmap v567: 129 pgs: 129 active+clean; 876 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 217 op/s 2024-09-21T00:27:42.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:42 smithi098 ceph-mon[34725]: pgmap v567: 129 pgs: 129 active+clean; 876 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 217 op/s 2024-09-21T00:27:43.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.435+0000 7f7fe8e00640 1 -- 172.21.15.27:0/2275882204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fe4111cd0 msgr2=0x7f7fe410d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:43.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.435+0000 7f7fe8e00640 1 --2- 172.21.15.27:0/2275882204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fe4111cd0 0x7f7fe410d6b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009900 tx=0x7f7fd402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.435+0000 7f7fe8e00640 1 -- 172.21.15.27:0/2275882204 shutdown_connections 2024-09-21T00:27:43.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.435+0000 7f7fe8e00640 1 --2- 172.21.15.27:0/2275882204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fe411cba0 0x7f7fe411efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.435+0000 7f7fe8e00640 1 --2- 172.21.15.27:0/2275882204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fe410dbf0 0x7f7fe410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.435+0000 7f7fe8e00640 1 --2- 172.21.15.27:0/2275882204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fe4111cd0 0x7f7fe410d6b0 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009900 tx=0x7f7fd402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.435+0000 7f7fe8e00640 1 -- 172.21.15.27:0/2275882204 >> 172.21.15.27:0/2275882204 conn(0x7f7fe406e9f0 msgr2=0x7f7fe406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:43.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.436+0000 7f7fe8e00640 1 -- 172.21.15.27:0/2275882204 shutdown_connections 2024-09-21T00:27:43.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.437+0000 7f7fe8e00640 1 -- 172.21.15.27:0/2275882204 wait complete. 2024-09-21T00:27:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.438+0000 7f7fe8e00640 1 Processor -- start 2024-09-21T00:27:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.438+0000 7f7fe8e00640 1 -- start start 2024-09-21T00:27:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.440+0000 7f7fe8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fe410dbf0 0x7f7fe41a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.440+0000 7f7fe8e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fe411cba0 0x7f7fe41a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.440+0000 7f7fe8e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fe41aea20 0x7f7fe41aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.440+0000 7f7fe8e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe41226d0 con 0x7f7fe410dbf0 2024-09-21T00:27:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.440+0000 7f7fe8e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe4122550 con 0x7f7fe411cba0 2024-09-21T00:27:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.440+0000 7f7fe8e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe4122850 con 0x7f7fe41aea20 2024-09-21T00:27:43.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.440+0000 7f7fe3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fe41aea20 0x7f7fe41aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:43.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.440+0000 7f7fe3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fe41aea20 0x7f7fe41aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:46414/0 (socket says 172.21.15.27:46414) 2024-09-21T00:27:43.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.440+0000 7f7fe3e00640 1 -- 172.21.15.27:0/972569215 learned_addr learned my addr 172.21.15.27:0/972569215 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:43.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.441+0000 7f7fe2a00640 1 --2- 172.21.15.27:0/972569215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fe411cba0 0x7f7fe41a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:43.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.441+0000 7f7fe3400640 1 --2- 172.21.15.27:0/972569215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fe410dbf0 0x7f7fe41a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:43.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.441+0000 7f7fe3e00640 1 -- 172.21.15.27:0/972569215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fe411cba0 msgr2=0x7f7fe41a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:43.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.441+0000 7f7fe3e00640 1 --2- 172.21.15.27:0/972569215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fe411cba0 0x7f7fe41a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.441+0000 7f7fe3e00640 1 -- 172.21.15.27:0/972569215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fe410dbf0 msgr2=0x7f7fe41a92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:43.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.442+0000 7f7fe3e00640 1 --2- 172.21.15.27:0/972569215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fe410dbf0 0x7f7fe41a92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.442+0000 7f7fe3e00640 1 -- 172.21.15.27:0/972569215 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe41af570 con 0x7f7fe41aea20 2024-09-21T00:27:43.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.442+0000 7f7fe3400640 1 --2- 172.21.15.27:0/972569215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fe410dbf0 0x7f7fe41a92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:27:43.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.443+0000 7f7fe3e00640 1 --2- 172.21.15.27:0/972569215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fe41aea20 0x7f7fe41aee30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00cad0 tx=0x7f7fdc00cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:43.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.444+0000 7f7fdbe00640 1 -- 172.21.15.27:0/972569215 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc00ae20 con 0x7f7fe41aea20 2024-09-21T00:27:43.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.444+0000 7f7fdbe00640 1 -- 172.21.15.27:0/972569215 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fdc005020 con 0x7f7fe41aea20 2024-09-21T00:27:43.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.445+0000 7f7fe8e00640 1 -- 172.21.15.27:0/972569215 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe41af860 con 0x7f7fe41aea20 2024-09-21T00:27:43.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.445+0000 7f7fdbe00640 1 -- 172.21.15.27:0/972569215 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc00f6b0 con 0x7f7fe41aea20 2024-09-21T00:27:43.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.446+0000 7f7fe8e00640 1 -- 172.21.15.27:0/972569215 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe41b5f10 con 0x7f7fe41aea20 2024-09-21T00:27:43.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.448+0000 7f7fe8e00640 1 -- 172.21.15.27:0/972569215 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe410d6b0 con 0x7f7fe41aea20 2024-09-21T00:27:43.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.451+0000 7f7fdbe00640 1 -- 172.21.15.27:0/972569215 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7fdc0051d0 con 0x7f7fe41aea20 2024-09-21T00:27:43.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.454+0000 7f7fdbe00640 1 --2- 172.21.15.27:0/972569215 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7fc80b67d0 0x7f7fc80b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:43.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.454+0000 7f7fdbe00640 1 -- 172.21.15.27:0/972569215 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9431+0+0 (secure 0 0 0) 0x7f7fdc0e7a00 con 0x7f7fe41aea20 2024-09-21T00:27:43.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.455+0000 7f7fe3400640 1 --2- 172.21.15.27:0/972569215 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7fc80b67d0 0x7f7fc80b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:43.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.459+0000 7f7fe3400640 1 --2- 172.21.15.27:0/972569215 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7fc80b67d0 0x7f7fc80b8ce0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4006010 tx=0x7f7fd40058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.460+0000 7f7fdbe00640 1 -- 172.21.15.27:0/972569215 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7fdc014030 con 0x7f7fe41aea20 2024-09-21T00:27:43.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.890+0000 7f7fe8e00640 1 -- 172.21.15.27:0/972569215 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fe41aa310 con 0x7f7fe41aea20 2024-09-21T00:27:43.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.890+0000 7f7fdbe00640 1 -- 172.21.15.27:0/972569215 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11899 (secure 0 0 0) 0x7f7fdc0adbc0 con 0x7f7fe41aea20 2024-09-21T00:27:43.892 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:27:43.892 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":32,"btime":"2024-09-21T00:27:26:126757+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:27:26.126719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26057":{"gid":26057,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/4028763758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":4028763758},{"type":"v1","addr":"172.21.15.89:6837","nonce":4028763758}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:27:43.893 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 32 2024-09-21T00:27:43.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.896+0000 7f7fe8e00640 1 -- 172.21.15.27:0/972569215 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7fc80b67d0 msgr2=0x7f7fc80b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:43.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.896+0000 7f7fe8e00640 1 --2- 172.21.15.27:0/972569215 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7fc80b67d0 0x7f7fc80b8ce0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4006010 tx=0x7f7fd40058d0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.896+0000 7f7fe8e00640 1 -- 172.21.15.27:0/972569215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fe41aea20 msgr2=0x7f7fe41aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:43.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.896+0000 7f7fe8e00640 1 --2- 172.21.15.27:0/972569215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fe41aea20 0x7f7fe41aee30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00cad0 tx=0x7f7fdc00cf90 comp rx=0 tx=0).stop 2024-09-21T00:27:43.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.896+0000 7f7fe8e00640 1 -- 172.21.15.27:0/972569215 shutdown_connections 2024-09-21T00:27:43.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.896+0000 7f7fe8e00640 1 --2- 172.21.15.27:0/972569215 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7fc80b67d0 0x7f7fc80b8ce0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.896+0000 7f7fe8e00640 1 --2- 172.21.15.27:0/972569215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fe41aea20 0x7f7fe41aee30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.896+0000 7f7fe8e00640 1 --2- 172.21.15.27:0/972569215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fe411cba0 0x7f7fe41a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.896+0000 7f7fe8e00640 1 --2- 172.21.15.27:0/972569215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fe410dbf0 0x7f7fe41a92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.896+0000 7f7fe8e00640 1 -- 172.21.15.27:0/972569215 >> 172.21.15.27:0/972569215 conn(0x7f7fe406e9f0 msgr2=0x7f7fe4110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:43.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.899+0000 7f7fe8e00640 1 -- 172.21.15.27:0/972569215 shutdown_connections 2024-09-21T00:27:43.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:43.899+0000 7f7fe8e00640 1 -- 172.21.15.27:0/972569215 wait complete. 2024-09-21T00:27:44.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:44 smithi027 ceph-mon[30745]: pgmap v568: 129 pgs: 129 active+clean; 869 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 284 op/s 2024-09-21T00:27:44.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:44 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/972569215' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:27:44.717 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id 3e1e43d --release --await' 2024-09-21T00:27:44.717 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id 3e1e43d --release --await 2024-09-21T00:27:44.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:44 smithi089 ceph-mon[34481]: pgmap v568: 129 pgs: 129 active+clean; 869 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 284 op/s 2024-09-21T00:27:44.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:44 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/972569215' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:27:44.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:44 smithi098 ceph-mon[34725]: pgmap v568: 129 pgs: 129 active+clean; 869 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 284 op/s 2024-09-21T00:27:44.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:44 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/972569215' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:27:45.368 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:27:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.243+0000 7f6ebb200640 1 -- 172.21.15.27:0/2688789974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ebc06fa20 msgr2=0x7f6ebc11c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.243+0000 7f6ebb200640 1 --2- 172.21.15.27:0/2688789974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ebc06fa20 0x7f6ebc11c820 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8009860 tx=0x7f6ea802a340 comp rx=0 tx=0).stop 2024-09-21T00:27:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.243+0000 7f6ebb200640 1 -- 172.21.15.27:0/2688789974 shutdown_connections 2024-09-21T00:27:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.243+0000 7f6ebb200640 1 --2- 172.21.15.27:0/2688789974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ebc11cd60 0x7f6ebc11f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.243+0000 7f6ebb200640 1 --2- 172.21.15.27:0/2688789974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ebc06fa20 0x7f6ebc11c820 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.243+0000 7f6ebb200640 1 --2- 172.21.15.27:0/2688789974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ebc06f0d0 0x7f6ebc06f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.243+0000 7f6ebb200640 1 -- 172.21.15.27:0/2688789974 >> 172.21.15.27:0/2688789974 conn(0x7f6ebc06ea90 msgr2=0x7f6ebc10c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:46.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.244+0000 7f6ebb200640 1 -- 172.21.15.27:0/2688789974 shutdown_connections 2024-09-21T00:27:46.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.245+0000 7f6ebb200640 1 -- 172.21.15.27:0/2688789974 wait complete. 2024-09-21T00:27:46.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.246+0000 7f6ebb200640 1 Processor -- start 2024-09-21T00:27:46.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.246+0000 7f6ebb200640 1 -- start start 2024-09-21T00:27:46.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.247+0000 7f6ebb200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ebc06f0d0 0x7f6ebc1a9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.247+0000 7f6ebb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ebc06fa20 0x7f6ebc1a9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.247+0000 7f6ebb200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ebc11cd60 0x7f6ebc1ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.247+0000 7f6ebb200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1226e0 con 0x7f6ebc06fa20 2024-09-21T00:27:46.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.247+0000 7f6ebb200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc122560 con 0x7f6ebc06f0d0 2024-09-21T00:27:46.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.247+0000 7f6ebb200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc122860 con 0x7f6ebc11cd60 2024-09-21T00:27:46.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.247+0000 7f6eb9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ebc06fa20 0x7f6ebc1a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.247+0000 7f6eb9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ebc06f0d0 0x7f6ebc1a9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.247+0000 7f6eb9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ebc06f0d0 0x7f6ebc1a9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54942/0 (socket says 172.21.15.27:54942) 2024-09-21T00:27:46.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.247+0000 7f6eb9e00640 1 -- 172.21.15.27:0/2807591070 learned_addr learned my addr 172.21.15.27:0/2807591070 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:46.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.247+0000 7f6eba800640 1 --2- 172.21.15.27:0/2807591070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ebc11cd60 0x7f6ebc1ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.248+0000 7f6eb9e00640 1 -- 172.21.15.27:0/2807591070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ebc11cd60 msgr2=0x7f6ebc1ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:46.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.248+0000 7f6eb9e00640 1 --2- 172.21.15.27:0/2807591070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ebc11cd60 0x7f6ebc1ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.248+0000 7f6eb9e00640 1 -- 172.21.15.27:0/2807591070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ebc06fa20 msgr2=0x7f6ebc1a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:46.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.248+0000 7f6eb9e00640 1 --2- 172.21.15.27:0/2807591070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ebc06fa20 0x7f6ebc1a9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.248+0000 7f6eb9e00640 1 -- 172.21.15.27:0/2807591070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ebc1af100 con 0x7f6ebc06f0d0 2024-09-21T00:27:46.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.248+0000 7f6eb9e00640 1 --2- 172.21.15.27:0/2807591070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ebc06f0d0 0x7f6ebc1a9190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00e9c0 tx=0x7f6eac00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:46.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.248+0000 7f6eb2a00640 1 -- 172.21.15.27:0/2807591070 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00ccc0 con 0x7f6ebc06f0d0 2024-09-21T00:27:46.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.248+0000 7f6eb2a00640 1 -- 172.21.15.27:0/2807591070 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eac0054d0 con 0x7f6ebc06f0d0 2024-09-21T00:27:46.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.249+0000 7f6ebb200640 1 -- 172.21.15.27:0/2807591070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ebc1af3f0 con 0x7f6ebc06f0d0 2024-09-21T00:27:46.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.249+0000 7f6eb2a00640 1 -- 172.21.15.27:0/2807591070 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac010430 con 0x7f6ebc06f0d0 2024-09-21T00:27:46.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.249+0000 7f6ebb200640 1 -- 172.21.15.27:0/2807591070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ebc1b5b50 con 0x7f6ebc06f0d0 2024-09-21T00:27:46.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.252+0000 7f6ec7695740 1 Processor -- start 2024-09-21T00:27:46.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.253+0000 7f6ec7695740 1 -- start start 2024-09-21T00:27:46.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.253+0000 7f6ec7695740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d55a520 0x55b22d901340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.253+0000 7f6ec7695740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d531660 0x55b22d901880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.253+0000 7f6ec7695740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d539840 0x55b22d901dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.253+0000 7f6ec7695740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b22d878290 con 0x55b22d539840 2024-09-21T00:27:46.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.253+0000 7f6ec7695740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b22d8676c0 con 0x55b22d55a520 2024-09-21T00:27:46.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.253+0000 7f6ec7695740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b22d86b4d0 con 0x55b22d531660 2024-09-21T00:27:46.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.253+0000 7f6eba800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d55a520 0x55b22d901340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.253+0000 7f6eba800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d55a520 0x55b22d901340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54944/0 (socket says 172.21.15.27:54944) 2024-09-21T00:27:46.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.253+0000 7f6eba800640 1 -- 172.21.15.27:0/3895939554 learned_addr learned my addr 172.21.15.27:0/3895939554 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:46.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.254+0000 7f6eb2a00640 1 -- 172.21.15.27:0/2807591070 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6eac0046c0 con 0x7f6ebc06f0d0 2024-09-21T00:27:46.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.254+0000 7f6eb9400640 1 --2- 172.21.15.27:0/3895939554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d539840 0x55b22d901dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.254+0000 7f6eba800640 1 -- 172.21.15.27:0/3895939554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d531660 msgr2=0x55b22d901880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:27:46.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.254+0000 7f6eba800640 1 --2- 172.21.15.27:0/3895939554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d531660 0x55b22d901880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.254+0000 7f6eba800640 1 -- 172.21.15.27:0/3895939554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d539840 msgr2=0x55b22d901dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:46.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.254+0000 7f6eba800640 1 --2- 172.21.15.27:0/3895939554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d539840 0x55b22d901dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.254+0000 7f6eba800640 1 -- 172.21.15.27:0/3895939554 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22d7f4460 con 0x55b22d55a520 2024-09-21T00:27:46.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.254+0000 7f6eb9400640 1 --2- 172.21.15.27:0/3895939554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d539840 0x55b22d901dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:46.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.255+0000 7f6eba800640 1 --2- 172.21.15.27:0/3895939554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d55a520 0x55b22d901340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009f50 tx=0x7f6eb40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20b9e020136bce1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:46.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.255+0000 7f6ea7000640 1 -- 172.21.15.27:0/3895939554 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4004d20 con 0x55b22d55a520 2024-09-21T00:27:46.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.255+0000 7f6ea7000640 1 -- 172.21.15.27:0/3895939554 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb4004050 con 0x55b22d55a520 2024-09-21T00:27:46.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.255+0000 7f6ea7000640 1 -- 172.21.15.27:0/3895939554 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4005280 con 0x55b22d55a520 2024-09-21T00:27:46.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.256+0000 7f6ec7695740 1 -- 172.21.15.27:0/3895939554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d55a520 msgr2=0x55b22d901340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:46.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.256+0000 7f6ec7695740 1 --2- 172.21.15.27:0/3895939554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d55a520 0x55b22d901340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009f50 tx=0x7f6eb40118d0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.256+0000 7f6ec7695740 1 -- 172.21.15.27:0/3895939554 shutdown_connections 2024-09-21T00:27:46.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.256+0000 7f6ec7695740 1 --2- 172.21.15.27:0/3895939554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d539840 0x55b22d901dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.256+0000 7f6ec7695740 1 --2- 172.21.15.27:0/3895939554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d531660 0x55b22d901880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.256+0000 7f6ec7695740 1 --2- 172.21.15.27:0/3895939554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d55a520 0x55b22d901340 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.256+0000 7f6ec7695740 1 -- 172.21.15.27:0/3895939554 >> 172.21.15.27:0/3895939554 conn(0x55b22d54e910 msgr2=0x55b22d8ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:46.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.256+0000 7f6ec7695740 1 -- 172.21.15.27:0/3895939554 shutdown_connections 2024-09-21T00:27:46.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.256+0000 7f6ec7695740 1 -- 172.21.15.27:0/3895939554 wait complete. 2024-09-21T00:27:46.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.257+0000 7f6eb2a00640 1 --2- 172.21.15.27:0/2807591070 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6ea00b6650 0x7f6ea00b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.257+0000 7f6ec7695740 1 -- start start 2024-09-21T00:27:46.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.257+0000 7f6eb2a00640 1 -- 172.21.15.27:0/2807591070 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9431+0+0 (secure 0 0 0) 0x7f6eac0e7e90 con 0x7f6ebc06f0d0 2024-09-21T00:27:46.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.258+0000 7f6ec7695740 1 Processor -- start 2024-09-21T00:27:46.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.258+0000 7f6eba800640 1 --2- 172.21.15.27:0/2807591070 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6ea00b6650 0x7f6ea00b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.259+0000 7f6ec7695740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d531660 0x55b22d90cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.259+0000 7f6ec7695740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d539840 0x55b22d90d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.259+0000 7f6ec7695740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d55a520 0x55b22d918e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.259+0000 7f6ec7695740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b22d6c8bb0 con 0x55b22d55a520 2024-09-21T00:27:46.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.259+0000 7f6ec7695740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b22d8026c0 con 0x55b22d539840 2024-09-21T00:27:46.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.259+0000 7f6ec7695740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b22d90fb20 con 0x55b22d531660 2024-09-21T00:27:46.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.259+0000 7f6ec7695740 10 client.0 _ll_register_callbacks cb 0x55b22d81d490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:46.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.259+0000 7f6eb9400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d539840 0x55b22d90d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.259+0000 7f6eb9400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d539840 0x55b22d90d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54950/0 (socket says 172.21.15.27:54950) 2024-09-21T00:27:46.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.259+0000 7f6eb9400640 1 -- 172.21.15.27:0/4096757724 learned_addr learned my addr 172.21.15.27:0/4096757724 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:46.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.259+0000 7f6eb9e00640 1 --2- 172.21.15.27:0/4096757724 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d531660 0x55b22d90cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.259+0000 7f6eb9400640 1 -- 172.21.15.27:0/4096757724 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d531660 msgr2=0x55b22d90cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:46.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.259+0000 7f6eb9400640 1 --2- 172.21.15.27:0/4096757724 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d531660 0x55b22d90cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.260+0000 7f6eb9400640 1 -- 172.21.15.27:0/4096757724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d55a520 msgr2=0x55b22d918e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:46.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.260+0000 7f6eb9400640 1 --2- 172.21.15.27:0/4096757724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d55a520 0x55b22d918e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.260+0000 7f6eb9400640 1 -- 172.21.15.27:0/4096757724 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22d5c5b00 con 0x55b22d539840 2024-09-21T00:27:46.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.260+0000 7f6eba800640 1 --2- 172.21.15.27:0/2807591070 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6ea00b6650 0x7f6ea00b8b60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009250 tx=0x7f6eb4004710 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:46.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.260+0000 7f6eb9e00640 1 --2- 172.21.15.27:0/4096757724 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d531660 0x55b22d90cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:27:46.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.260+0000 7f6eb9400640 1 --2- 172.21.15.27:0/4096757724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d539840 0x55b22d90d560 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80303f0 tx=0x7f6ea8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c09c574eade1be40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:46.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.260+0000 7f6e9b400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:27:46.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.261+0000 7f6e9b400640 1 -- 172.21.15.27:0/4096757724 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8037040 con 0x55b22d539840 2024-09-21T00:27:46.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.261+0000 7f6e9b400640 1 -- 172.21.15.27:0/4096757724 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9431+0+0 (secure 0 0 0) 0x7f6ea8030450 con 0x55b22d539840 2024-09-21T00:27:46.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.261+0000 7f6ec7695740 1 -- 172.21.15.27:0/4096757724 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb4004050 con 0x55b22d539840 2024-09-21T00:27:46.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.261+0000 7f6e9b400640 1 -- 172.21.15.27:0/4096757724 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6ea8005d30 con 0x55b22d539840 2024-09-21T00:27:46.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.262+0000 7f6ec7695740 10 client.26084 fetch_fsmap learned FSMap version 32 2024-09-21T00:27:46.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.262+0000 7f6ec7695740 1 -- 172.21.15.27:0/4096757724 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b22d63c960 con 0x55b22d539840 2024-09-21T00:27:46.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.262+0000 7f6e9b400640 1 -- 172.21.15.27:0/4096757724 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb4004050 con 0x55b22d539840 2024-09-21T00:27:46.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.262+0000 7f6e9b400640 1 -- 172.21.15.27:0/4096757724 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f6ea80048f0 con 0x55b22d539840 2024-09-21T00:27:46.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.262+0000 7f6ec7695740 10 client.26084 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T00:27:46.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.262+0000 7f6ec7695740 10 client.26084 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:27:46.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.262+0000 7f6ec7695740 20 client.26084 populate_metadata read hostname 'smithi027' 2024-09-21T00:27:46.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.263+0000 7f6ec7695740 1 --2- 172.21.15.27:0/4096757724 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b22d518ea0 0x55b22d91ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.263+0000 7f6ec7695740 4 client.26084 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:27:46.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.263+0000 7f6ec7695740 1 -- 172.21.15.27:0/4096757724 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b22d87a1e0 con 0x55b22d518ea0 2024-09-21T00:27:46.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.263+0000 7f6eb9e00640 1 --2- 172.21.15.27:0/4096757724 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b22d518ea0 0x55b22d91ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.264+0000 7f6eb9e00640 1 --2- 172.21.15.27:0/4096757724 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b22d518ea0 0x55b22d91ee90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c013bb489e8d91f8 server_cookie=8d92d0430d04d921 in_seq=0 out_seq=0 2024-09-21T00:27:46.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.264+0000 7f6e9b400640 10 client.26084 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:46.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.265+0000 7f6e9b400640 1 -- 172.21.15.27:0/4096757724 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b22d87a1e0 con 0x55b22d518ea0 2024-09-21T00:27:46.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.265+0000 7f6e9b400640 10 client.26084 handle_command_reply: tid=0 2024-09-21T00:27:46.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.265+0000 7f6ec7695740 1 client.26084 shutdown 2024-09-21T00:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.265+0000 7f6ec7695740 1 -- 172.21.15.27:0/4096757724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d539840 msgr2=0x55b22d90d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.265+0000 7f6ec7695740 1 --2- 172.21.15.27:0/4096757724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d539840 0x55b22d90d560 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80303f0 tx=0x7f6ea8004270 comp rx=0 tx=0).stop 2024-09-21T00:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.266+0000 7f6ec7695740 1 -- 172.21.15.27:0/4096757724 shutdown_connections 2024-09-21T00:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.266+0000 7f6ec7695740 1 --2- 172.21.15.27:0/4096757724 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b22d518ea0 0x55b22d91ee90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.266+0000 7f6ec7695740 1 --2- 172.21.15.27:0/4096757724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d55a520 0x55b22d918e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.266+0000 7f6ec7695740 1 --2- 172.21.15.27:0/4096757724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d539840 0x55b22d90d560 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.266+0000 7f6ec7695740 1 --2- 172.21.15.27:0/4096757724 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d531660 0x55b22d90cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.266+0000 7f6e9b400640 0 client.26084 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.266+0000 7f6ec7695740 1 -- 172.21.15.27:0/4096757724 >> 172.21.15.27:0/4096757724 conn(0x55b22d54e910 msgr2=0x55b22d9044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.266+0000 7f6ec7695740 1 -- 172.21.15.27:0/4096757724 shutdown_connections 2024-09-21T00:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.266+0000 7f6ec7695740 1 -- 172.21.15.27:0/4096757724 wait complete. 2024-09-21T00:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.266+0000 7f6ec7695740 20 client.26084 trim_cache size 0 max 16384 2024-09-21T00:27:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.298+0000 7f6ec7695740 1 Processor -- start 2024-09-21T00:27:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.298+0000 7f6ec7695740 1 -- start start 2024-09-21T00:27:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.298+0000 7f6ec7695740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d531660 0x55b22d8ff260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.298+0000 7f6ec7695740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d539840 0x55b22d918d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.298+0000 7f6ec7695740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d55a520 0x55b22d9063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.298+0000 7f6ec7695740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1c2530 con 0x55b22d539840 2024-09-21T00:27:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.298+0000 7f6ec7695740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1bfda0 con 0x55b22d531660 2024-09-21T00:27:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.298+0000 7f6ec7695740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1bdbf0 con 0x55b22d55a520 2024-09-21T00:27:46.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.299+0000 7f6eb9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d531660 0x55b22d8ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.299+0000 7f6eb9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d531660 0x55b22d8ff260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54952/0 (socket says 172.21.15.27:54952) 2024-09-21T00:27:46.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.299+0000 7f6eb9e00640 1 -- 172.21.15.27:0/2095062982 learned_addr learned my addr 172.21.15.27:0/2095062982 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:46.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.299+0000 7f6eba800640 1 --2- 172.21.15.27:0/2095062982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d55a520 0x55b22d9063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.299+0000 7f6eb9400640 1 --2- 172.21.15.27:0/2095062982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d539840 0x55b22d918d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.299+0000 7f6eb9e00640 1 -- 172.21.15.27:0/2095062982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d55a520 msgr2=0x55b22d9063d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.299+0000 7f6eb9e00640 1 --2- 172.21.15.27:0/2095062982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d55a520 0x55b22d9063d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.299+0000 7f6eb9e00640 1 -- 172.21.15.27:0/2095062982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d539840 msgr2=0x55b22d918d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.299+0000 7f6eb9e00640 1 --2- 172.21.15.27:0/2095062982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d539840 0x55b22d918d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.299+0000 7f6eb9e00640 1 -- 172.21.15.27:0/2095062982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22d8fe960 con 0x55b22d531660 2024-09-21T00:27:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.300+0000 7f6eb9e00640 1 --2- 172.21.15.27:0/2095062982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d531660 0x55b22d8ff260 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6eac004a30 tx=0x7f6eac0ad7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ce2048901a7c9e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.300+0000 7f6ea6600640 1 -- 172.21.15.27:0/2095062982 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac005680 con 0x55b22d531660 2024-09-21T00:27:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.300+0000 7f6ea6600640 1 -- 172.21.15.27:0/2095062982 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eac0ad400 con 0x55b22d531660 2024-09-21T00:27:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.300+0000 7f6ea6600640 1 -- 172.21.15.27:0/2095062982 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0e5470 con 0x55b22d531660 2024-09-21T00:27:46.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.301+0000 7f6ec7695740 1 -- 172.21.15.27:0/2095062982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d531660 msgr2=0x55b22d8ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:46.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.301+0000 7f6ec7695740 1 --2- 172.21.15.27:0/2095062982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d531660 0x55b22d8ff260 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6eac004a30 tx=0x7f6eac0ad7e0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.301+0000 7f6ec7695740 1 -- 172.21.15.27:0/2095062982 shutdown_connections 2024-09-21T00:27:46.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.301+0000 7f6ec7695740 1 --2- 172.21.15.27:0/2095062982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d55a520 0x55b22d9063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.301+0000 7f6ec7695740 1 --2- 172.21.15.27:0/2095062982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d539840 0x55b22d918d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.301+0000 7f6ec7695740 1 --2- 172.21.15.27:0/2095062982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d531660 0x55b22d8ff260 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.301+0000 7f6ec7695740 1 -- 172.21.15.27:0/2095062982 >> 172.21.15.27:0/2095062982 conn(0x55b22d54e910 msgr2=0x55b22d8feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:46.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.302+0000 7f6ec7695740 1 -- 172.21.15.27:0/2095062982 shutdown_connections 2024-09-21T00:27:46.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.302+0000 7f6ec7695740 1 -- 172.21.15.27:0/2095062982 wait complete. 2024-09-21T00:27:46.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.302+0000 7f6ec7695740 1 -- start start 2024-09-21T00:27:46.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.303+0000 7f6ec7695740 1 Processor -- start 2024-09-21T00:27:46.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.303+0000 7f6ec7695740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d531660 0x55b22d9157b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.303+0000 7f6ec7695740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d539840 0x55b22d90fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.303+0000 7f6ec7695740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d55a520 0x55b22d9104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.303+0000 7f6ec7695740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1be260 con 0x55b22d531660 2024-09-21T00:27:46.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.303+0000 7f6ec7695740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1bf6d0 con 0x55b22d55a520 2024-09-21T00:27:46.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.303+0000 7f6ec7695740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1be930 con 0x55b22d539840 2024-09-21T00:27:46.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.303+0000 7f6ec7695740 10 client.0 _ll_register_callbacks cb 0x7f6eb40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:46.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.304+0000 7f6eba800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d55a520 0x55b22d9104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.304+0000 7f6eba800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d55a520 0x55b22d9104d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54956/0 (socket says 172.21.15.27:54956) 2024-09-21T00:27:46.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.304+0000 7f6eba800640 1 -- 172.21.15.27:0/351726101 learned_addr learned my addr 172.21.15.27:0/351726101 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:46.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.304+0000 7f6eb9e00640 1 --2- 172.21.15.27:0/351726101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d531660 0x55b22d9157b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.304+0000 7f6eb9400640 1 --2- 172.21.15.27:0/351726101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d539840 0x55b22d90fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.305+0000 7f6eba800640 1 -- 172.21.15.27:0/351726101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d539840 msgr2=0x55b22d90fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:46.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.305+0000 7f6eba800640 1 --2- 172.21.15.27:0/351726101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d539840 0x55b22d90fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.305+0000 7f6eba800640 1 -- 172.21.15.27:0/351726101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d531660 msgr2=0x55b22d9157b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:46.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.305+0000 7f6eba800640 1 --2- 172.21.15.27:0/351726101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d531660 0x55b22d9157b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.305+0000 7f6eba800640 1 -- 172.21.15.27:0/351726101 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22d989890 con 0x55b22d55a520 2024-09-21T00:27:46.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.305+0000 7f6eb9e00640 1 --2- 172.21.15.27:0/351726101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d531660 0x55b22d9157b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:27:46.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.305+0000 7f6eba800640 1 --2- 172.21.15.27:0/351726101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d55a520 0x55b22d9104d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4011dd0 tx=0x7f6eb4004950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd39a5973271aa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:46.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.305+0000 7f6e9b400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:27:46.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.306+0000 7f6e9b400640 1 -- 172.21.15.27:0/351726101 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4032030 con 0x55b22d55a520 2024-09-21T00:27:46.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.306+0000 7f6ec7695740 1 -- 172.21.15.27:0/351726101 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eac0ad400 con 0x55b22d55a520 2024-09-21T00:27:46.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.306+0000 7f6e9b400640 1 -- 172.21.15.27:0/351726101 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9431+0+0 (secure 0 0 0) 0x7f6eb4005510 con 0x55b22d55a520 2024-09-21T00:27:46.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.307+0000 7f6e9b400640 1 -- 172.21.15.27:0/351726101 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6eb4005710 con 0x55b22d55a520 2024-09-21T00:27:46.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.307+0000 7f6ec7695740 10 client.26090 fetch_fsmap learned FSMap version 32 2024-09-21T00:27:46.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.307+0000 7f6ec7695740 1 -- 172.21.15.27:0/351726101 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b22d98aa50 con 0x55b22d55a520 2024-09-21T00:27:46.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.307+0000 7f6e9b400640 1 -- 172.21.15.27:0/351726101 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eac0ad400 con 0x55b22d55a520 2024-09-21T00:27:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.308+0000 7f6e9b400640 1 -- 172.21.15.27:0/351726101 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f6eb402d040 con 0x55b22d55a520 2024-09-21T00:27:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.308+0000 7f6ec7695740 10 client.26090 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T00:27:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.308+0000 7f6ec7695740 10 client.26090 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:27:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.308+0000 7f6ec7695740 20 client.26090 populate_metadata read hostname 'smithi027' 2024-09-21T00:27:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.308+0000 7f6ec7695740 1 --2- 172.21.15.27:0/351726101 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b22d4ee250 0x55b22d98d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.308+0000 7f6ec7695740 4 client.26090 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3e1e43d", "release": true, "await": true}] 2024-09-21T00:27:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.308+0000 7f6ec7695740 1 -- 172.21.15.27:0/351726101 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3e1e43d", "release": true, "await": true}) -- 0x55b22d875c90 con 0x55b22d4ee250 2024-09-21T00:27:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.308+0000 7f6eb9e00640 1 --2- 172.21.15.27:0/351726101 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b22d4ee250 0x55b22d98d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.309+0000 7f6eb9e00640 1 --2- 172.21.15.27:0/351726101 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b22d4ee250 0x55b22d98d090 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3fc9f58cc946722 server_cookie=a2c0ad06243ec9de in_seq=0 out_seq=0 2024-09-21T00:27:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.309+0000 7f6e9b400640 10 client.26090 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:46.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:46 smithi027 ceph-mon[30745]: pgmap v569: 129 pgs: 129 active+clean; 877 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.0 MiB/s wr, 250 op/s 2024-09-21T00:27:46.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:46 smithi089 ceph-mon[34481]: pgmap v569: 129 pgs: 129 active+clean; 877 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.0 MiB/s wr, 250 op/s 2024-09-21T00:27:46.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:46 smithi098 ceph-mon[34725]: pgmap v569: 129 pgs: 129 active+clean; 877 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.0 MiB/s wr, 250 op/s 2024-09-21T00:27:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.995+0000 7f6e9b400640 1 -- 172.21.15.27:0/351726101 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55b22d875c90 con 0x55b22d4ee250 2024-09-21T00:27:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.995+0000 7f6e9b400640 10 client.26090 handle_command_reply: tid=0 2024-09-21T00:27:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.996+0000 7f6ec7695740 1 client.26090 shutdown 2024-09-21T00:27:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.996+0000 7f6ec7695740 1 -- 172.21.15.27:0/351726101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d55a520 msgr2=0x55b22d9104d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.996+0000 7f6ec7695740 1 --2- 172.21.15.27:0/351726101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d55a520 0x55b22d9104d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4011dd0 tx=0x7f6eb4004950 comp rx=0 tx=0).stop 2024-09-21T00:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.996+0000 7f6ec7695740 1 -- 172.21.15.27:0/351726101 shutdown_connections 2024-09-21T00:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.996+0000 7f6ec7695740 1 --2- 172.21.15.27:0/351726101 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55b22d4ee250 0x55b22d98d090 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.996+0000 7f6ec7695740 1 --2- 172.21.15.27:0/351726101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b22d55a520 0x55b22d9104d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.996+0000 7f6ec7695740 1 --2- 172.21.15.27:0/351726101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b22d539840 0x55b22d90fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.996+0000 7f6e9b400640 0 client.26090 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.996+0000 7f6ec7695740 1 --2- 172.21.15.27:0/351726101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b22d531660 0x55b22d9157b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.996+0000 7f6ec7695740 1 -- 172.21.15.27:0/351726101 >> 172.21.15.27:0/351726101 conn(0x55b22d54e910 msgr2=0x55b22d90e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:46.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.996+0000 7f6ec7695740 1 -- 172.21.15.27:0/351726101 shutdown_connections 2024-09-21T00:27:46.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.996+0000 7f6ec7695740 1 -- 172.21.15.27:0/351726101 wait complete. 2024-09-21T00:27:46.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.997+0000 7f6ec7695740 20 client.26090 trim_cache size 0 max 16384 2024-09-21T00:27:46.999 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:27:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 32, 2024-09-21T00:27:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24761, 2024-09-21T00:27:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 20, 2024-09-21T00:27:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T00:27:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "3e1e43d": { 2024-09-21T00:27:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 20, 2024-09-21T00:27:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T00:27:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:27:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T00:27:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:27:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:27:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T00:27:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 130.0, 2024-09-21T00:27:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T00:27:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T00:27:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T00:27:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:27:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T00:27:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:27:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:27:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:27:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:27:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:27:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:27:47.001 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:27:47.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.998+0000 7f6ebb200640 1 -- 172.21.15.27:0/2807591070 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6ea00b6650 msgr2=0x7f6ea00b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.998+0000 7f6ebb200640 1 --2- 172.21.15.27:0/2807591070 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6ea00b6650 0x7f6ea00b8b60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009250 tx=0x7f6eb4004710 comp rx=0 tx=0).stop 2024-09-21T00:27:47.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.998+0000 7f6ebb200640 1 -- 172.21.15.27:0/2807591070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ebc06f0d0 msgr2=0x7f6ebc1a9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.998+0000 7f6ebb200640 1 --2- 172.21.15.27:0/2807591070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ebc06f0d0 0x7f6ebc1a9190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00e9c0 tx=0x7f6eac00ee80 comp rx=0 tx=0).stop 2024-09-21T00:27:47.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.998+0000 7f6ebb200640 1 -- 172.21.15.27:0/2807591070 shutdown_connections 2024-09-21T00:27:47.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.998+0000 7f6ebb200640 1 --2- 172.21.15.27:0/2807591070 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6ea00b6650 0x7f6ea00b8b60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.998+0000 7f6ebb200640 1 --2- 172.21.15.27:0/2807591070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ebc11cd60 0x7f6ebc1ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.998+0000 7f6ebb200640 1 --2- 172.21.15.27:0/2807591070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ebc06fa20 0x7f6ebc1a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.998+0000 7f6ebb200640 1 --2- 172.21.15.27:0/2807591070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ebc06f0d0 0x7f6ebc1a9190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.998+0000 7f6ebb200640 1 -- 172.21.15.27:0/2807591070 >> 172.21.15.27:0/2807591070 conn(0x7f6ebc06ea90 msgr2=0x7f6ebc11ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:47.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.998+0000 7f6ebb200640 1 -- 172.21.15.27:0/2807591070 shutdown_connections 2024-09-21T00:27:47.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:27:46.998+0000 7f6ebb200640 1 -- 172.21.15.27:0/2807591070 wait complete. 2024-09-21T00:27:47.886 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3e1e43d', total seconds elapsed: 60.7 2024-09-21T00:27:48.300 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:48 smithi027 ceph-mon[30745]: pgmap v570: 129 pgs: 129 active+clean; 877 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.1 MiB/s wr, 428 op/s 2024-09-21T00:27:48.373 INFO:tasks.workunit.client.0.smithi027.stdout: 227 28908 18 20112 16 8229 152 2024-09-21T00:27:48.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:48 smithi089 ceph-mon[34481]: pgmap v570: 129 pgs: 129 active+clean; 877 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.1 MiB/s wr, 428 op/s 2024-09-21T00:27:48.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:48 smithi098 ceph-mon[34725]: pgmap v570: 129 pgs: 129 active+clean; 877 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.1 MiB/s wr, 428 op/s 2024-09-21T00:27:50.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:50 smithi027 ceph-mon[30745]: pgmap v571: 129 pgs: 129 active+clean; 879 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.2 MiB/s wr, 610 op/s 2024-09-21T00:27:50.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:50 smithi089 ceph-mon[34481]: pgmap v571: 129 pgs: 129 active+clean; 879 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.2 MiB/s wr, 610 op/s 2024-09-21T00:27:50.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:50 smithi098 ceph-mon[34725]: pgmap v571: 129 pgs: 129 active+clean; 879 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.2 MiB/s wr, 610 op/s 2024-09-21T00:27:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.259+0000 7f4b87e00640 1 -- 172.21.15.27:0/3147652646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b881047a0 msgr2=0x7f4b88104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.259+0000 7f4b87e00640 1 --2- 172.21.15.27:0/3147652646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b881047a0 0x7f4b88104bb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009900 tx=0x7f4b7802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.260+0000 7f4b87e00640 1 -- 172.21.15.27:0/3147652646 shutdown_connections 2024-09-21T00:27:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.260+0000 7f4b87e00640 1 --2- 172.21.15.27:0/3147652646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b8810dbc0 0x7f4b881100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.260+0000 7f4b87e00640 1 --2- 172.21.15.27:0/3147652646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b881050f0 0x7f4b8810d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.260+0000 7f4b87e00640 1 --2- 172.21.15.27:0/3147652646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b881047a0 0x7f4b88104bb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.260+0000 7f4b87e00640 1 -- 172.21.15.27:0/3147652646 >> 172.21.15.27:0/3147652646 conn(0x7f4b880fe210 msgr2=0x7f4b88100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.260+0000 7f4b87e00640 1 -- 172.21.15.27:0/3147652646 shutdown_connections 2024-09-21T00:27:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.260+0000 7f4b87e00640 1 -- 172.21.15.27:0/3147652646 wait complete. 2024-09-21T00:27:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.261+0000 7f4b87e00640 1 Processor -- start 2024-09-21T00:27:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.261+0000 7f4b87e00640 1 -- start start 2024-09-21T00:27:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.261+0000 7f4b87e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b881047a0 0x7f4b881a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.261+0000 7f4b87e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b881050f0 0x7f4b881a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.261+0000 7f4b87e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b8810dbc0 0x7f4b881a6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.261+0000 7f4b87e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4b88119e60 con 0x7f4b8810dbc0 2024-09-21T00:27:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.261+0000 7f4b87e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4b88119ce0 con 0x7f4b881050f0 2024-09-21T00:27:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.261+0000 7f4b87e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4b88119fe0 con 0x7f4b881047a0 2024-09-21T00:27:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.262+0000 7f4b86000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b881050f0 0x7f4b881a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.262+0000 7f4b86000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b881050f0 0x7f4b881a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54968/0 (socket says 172.21.15.27:54968) 2024-09-21T00:27:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.262+0000 7f4b86000640 1 -- 172.21.15.27:0/1013747366 learned_addr learned my addr 172.21.15.27:0/1013747366 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.262+0000 7f4b86a00640 1 --2- 172.21.15.27:0/1013747366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b881047a0 0x7f4b881a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.262+0000 7f4b87400640 1 --2- 172.21.15.27:0/1013747366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b8810dbc0 0x7f4b881a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.262+0000 7f4b86000640 1 -- 172.21.15.27:0/1013747366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b881047a0 msgr2=0x7f4b881a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.262+0000 7f4b86000640 1 --2- 172.21.15.27:0/1013747366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b881047a0 0x7f4b881a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.262+0000 7f4b86000640 1 -- 172.21.15.27:0/1013747366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b8810dbc0 msgr2=0x7f4b881a6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.262+0000 7f4b86000640 1 --2- 172.21.15.27:0/1013747366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b8810dbc0 0x7f4b881a6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.262+0000 7f4b86000640 1 -- 172.21.15.27:0/1013747366 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b881a6790 con 0x7f4b881050f0 2024-09-21T00:27:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.262+0000 7f4b87400640 1 --2- 172.21.15.27:0/1013747366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b8810dbc0 0x7f4b881a6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.263+0000 7f4b86000640 1 --2- 172.21.15.27:0/1013747366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b881050f0 0x7f4b881a0e00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400e9f0 tx=0x7f4b7400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.263+0000 7f4b7f400640 1 -- 172.21.15.27:0/1013747366 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7400ccc0 con 0x7f4b881050f0 2024-09-21T00:27:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.263+0000 7f4b7f400640 1 -- 172.21.15.27:0/1013747366 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b740108e0 con 0x7f4b881050f0 2024-09-21T00:27:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.263+0000 7f4b7f400640 1 -- 172.21.15.27:0/1013747366 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74018860 con 0x7f4b881050f0 2024-09-21T00:27:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.263+0000 7f4b87e00640 1 -- 172.21.15.27:0/1013747366 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b881a6a80 con 0x7f4b881050f0 2024-09-21T00:27:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.263+0000 7f4b87e00640 1 -- 172.21.15.27:0/1013747366 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b881ad210 con 0x7f4b881050f0 2024-09-21T00:27:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.267+0000 7f4b947d4740 1 Processor -- start 2024-09-21T00:27:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.268+0000 7f4b947d4740 1 -- start start 2024-09-21T00:27:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.268+0000 7f4b947d4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd7e1b0 0x55996d122e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.268+0000 7f4b947d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd552f0 0x55996d123350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.268+0000 7f4b947d4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd5d4d0 0x55996d123890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.268+0000 7f4b947d4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55996d0986b0 con 0x55996cd5d4d0 2024-09-21T00:27:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.268+0000 7f4b947d4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55996d087ae0 con 0x55996cd7e1b0 2024-09-21T00:27:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.268+0000 7f4b947d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55996d08b8f0 con 0x55996cd552f0 2024-09-21T00:27:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.268+0000 7f4b87400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd7e1b0 0x55996d122e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.268+0000 7f4b86a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd552f0 0x55996d123350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.268+0000 7f4b86a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd552f0 0x55996d123350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:46488/0 (socket says 172.21.15.27:46488) 2024-09-21T00:27:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.268+0000 7f4b87400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd7e1b0 0x55996d122e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54980/0 (socket says 172.21.15.27:54980) 2024-09-21T00:27:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.268+0000 7f4b86a00640 1 -- 172.21.15.27:0/1962167165 learned_addr learned my addr 172.21.15.27:0/1962167165 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.269+0000 7f4b86a00640 1 -- 172.21.15.27:0/1962167165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd7e1b0 msgr2=0x55996d122e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.269+0000 7f4b86a00640 1 --2- 172.21.15.27:0/1962167165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd7e1b0 0x55996d122e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.269+0000 7f4b86a00640 1 -- 172.21.15.27:0/1962167165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd5d4d0 msgr2=0x55996d123890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:27:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.269+0000 7f4b86a00640 1 --2- 172.21.15.27:0/1962167165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd5d4d0 0x55996d123890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.269+0000 7f4b86a00640 1 -- 172.21.15.27:0/1962167165 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55996d054780 con 0x55996cd552f0 2024-09-21T00:27:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.269+0000 7f4b86a00640 1 --2- 172.21.15.27:0/1962167165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd552f0 0x55996d123350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009900 tx=0x7f4b78004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8cf0196b9a93359 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.269+0000 7f4b73400640 1 -- 172.21.15.27:0/1962167165 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78040070 con 0x55996cd552f0 2024-09-21T00:27:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.269+0000 7f4b73400640 1 -- 172.21.15.27:0/1962167165 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b78005500 con 0x55996cd552f0 2024-09-21T00:27:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.269+0000 7f4b73400640 1 -- 172.21.15.27:0/1962167165 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78035040 con 0x55996cd552f0 2024-09-21T00:27:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.269+0000 7f4b7f400640 1 -- 172.21.15.27:0/1013747366 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4b74010430 con 0x7f4b881050f0 2024-09-21T00:27:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.270+0000 7f4b947d4740 1 -- 172.21.15.27:0/1962167165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd552f0 msgr2=0x55996d123350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.270+0000 7f4b947d4740 1 --2- 172.21.15.27:0/1962167165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd552f0 0x55996d123350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009900 tx=0x7f4b78004040 comp rx=0 tx=0).stop 2024-09-21T00:27:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.271+0000 7f4b947d4740 1 -- 172.21.15.27:0/1962167165 shutdown_connections 2024-09-21T00:27:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.271+0000 7f4b947d4740 1 --2- 172.21.15.27:0/1962167165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd5d4d0 0x55996d123890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.271+0000 7f4b947d4740 1 --2- 172.21.15.27:0/1962167165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd552f0 0x55996d123350 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.271+0000 7f4b947d4740 1 --2- 172.21.15.27:0/1962167165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd7e1b0 0x55996d122e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.271+0000 7f4b947d4740 1 -- 172.21.15.27:0/1962167165 >> 172.21.15.27:0/1962167165 conn(0x55996cd72300 msgr2=0x55996d120d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.271+0000 7f4b947d4740 1 -- 172.21.15.27:0/1962167165 shutdown_connections 2024-09-21T00:27:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.271+0000 7f4b947d4740 1 -- 172.21.15.27:0/1962167165 wait complete. 2024-09-21T00:27:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.271+0000 7f4b947d4740 1 -- start start 2024-09-21T00:27:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.271+0000 7f4b947d4740 1 Processor -- start 2024-09-21T00:27:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.272+0000 7f4b7f400640 1 --2- 172.21.15.27:0/1013747366 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4b6c0b65f0 0x7f4b6c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.272+0000 7f4b7f400640 1 -- 172.21.15.27:0/1013747366 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9431+0+0 (secure 0 0 0) 0x7f4b740e7d70 con 0x7f4b881050f0 2024-09-21T00:27:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.272+0000 7f4b947d4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd552f0 0x55996d12e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.272+0000 7f4b947d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd5d4d0 0x55996d12e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.272+0000 7f4b947d4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd7e1b0 0x55996d13a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.272+0000 7f4b947d4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55996cfbfab0 con 0x55996cd7e1b0 2024-09-21T00:27:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.272+0000 7f4b947d4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55996d02e570 con 0x55996cd552f0 2024-09-21T00:27:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.272+0000 7f4b947d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55996d131530 con 0x55996cd5d4d0 2024-09-21T00:27:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.272+0000 7f4b947d4740 10 client.0 _ll_register_callbacks cb 0x55996d03d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.272+0000 7f4b86a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd552f0 0x55996d12e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.273+0000 7f4b86a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd552f0 0x55996d12e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54992/0 (socket says 172.21.15.27:54992) 2024-09-21T00:27:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.273+0000 7f4b86a00640 1 -- 172.21.15.27:0/283472239 learned_addr learned my addr 172.21.15.27:0/283472239 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.273+0000 7f4b87400640 1 --2- 172.21.15.27:0/1013747366 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4b6c0b65f0 0x7f4b6c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.273+0000 7f4b87400640 1 --2- 172.21.15.27:0/283472239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd7e1b0 0x55996d13a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.273+0000 7f4b86a00640 1 -- 172.21.15.27:0/283472239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd5d4d0 msgr2=0x55996d12e9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:27:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.273+0000 7f4b86a00640 1 --2- 172.21.15.27:0/283472239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd5d4d0 0x55996d12e9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.273+0000 7f4b86a00640 1 -- 172.21.15.27:0/283472239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd7e1b0 msgr2=0x55996d13a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.273+0000 7f4b86a00640 1 --2- 172.21.15.27:0/283472239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd7e1b0 0x55996d13a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.273+0000 7f4b86a00640 1 -- 172.21.15.27:0/283472239 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55996cde9790 con 0x55996cd552f0 2024-09-21T00:27:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.273+0000 7f4b87400640 1 --2- 172.21.15.27:0/283472239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd7e1b0 0x55996d13a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:27:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.273+0000 7f4b86a00640 1 --2- 172.21.15.27:0/283472239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd552f0 0x55996d12e4b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4b78031440 tx=0x7f4b7802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdac2444ea549ab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.274+0000 7f4b67e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:27:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.274+0000 7f4b67e00640 1 -- 172.21.15.27:0/283472239 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7804d020 con 0x55996cd552f0 2024-09-21T00:27:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.274+0000 7f4b67e00640 1 -- 172.21.15.27:0/283472239 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9431+0+0 (secure 0 0 0) 0x7f4b78005050 con 0x55996cd552f0 2024-09-21T00:27:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.274+0000 7f4b947d4740 1 -- 172.21.15.27:0/283472239 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b78005500 con 0x55996cd552f0 2024-09-21T00:27:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.274+0000 7f4b67e00640 1 -- 172.21.15.27:0/283472239 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4b78058050 con 0x55996cd552f0 2024-09-21T00:27:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.274+0000 7f4b947d4740 10 client.26102 fetch_fsmap learned FSMap version 32 2024-09-21T00:27:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.274+0000 7f4b947d4740 1 -- 172.21.15.27:0/283472239 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55996ce62620 con 0x55996cd552f0 2024-09-21T00:27:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.274+0000 7f4b87400640 1 --2- 172.21.15.27:0/1013747366 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4b6c0b65f0 0x7f4b6c0b8b00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009fd0 tx=0x7f4b80015040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.275+0000 7f4b67e00640 1 -- 172.21.15.27:0/283472239 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b78005500 con 0x55996cd552f0 2024-09-21T00:27:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.275+0000 7f4b67e00640 1 -- 172.21.15.27:0/283472239 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f4b78003c10 con 0x55996cd552f0 2024-09-21T00:27:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.275+0000 7f4b947d4740 10 client.26102 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T00:27:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.275+0000 7f4b947d4740 10 client.26102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:27:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.275+0000 7f4b947d4740 20 client.26102 populate_metadata read hostname 'smithi027' 2024-09-21T00:27:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.275+0000 7f4b947d4740 1 --2- 172.21.15.27:0/283472239 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55996cd3cd80 0x55996d140400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.275+0000 7f4b947d4740 4 client.26102 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:27:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.275+0000 7f4b947d4740 1 -- 172.21.15.27:0/283472239 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55996d09a600 con 0x55996cd3cd80 2024-09-21T00:27:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.276+0000 7f4b86000640 1 --2- 172.21.15.27:0/283472239 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55996cd3cd80 0x55996d140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.276+0000 7f4b86000640 1 --2- 172.21.15.27:0/283472239 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55996cd3cd80 0x55996d140400 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a7aa7186af63365 server_cookie=bf2998130adac16a in_seq=0 out_seq=0 2024-09-21T00:27:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.276+0000 7f4b67e00640 10 client.26102 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.277+0000 7f4b67e00640 1 -- 172.21.15.27:0/283472239 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55996d09a600 con 0x55996cd3cd80 2024-09-21T00:27:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.277+0000 7f4b67e00640 10 client.26102 handle_command_reply: tid=0 2024-09-21T00:27:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.277+0000 7f4b947d4740 1 client.26102 shutdown 2024-09-21T00:27:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.277+0000 7f4b947d4740 1 -- 172.21.15.27:0/283472239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd552f0 msgr2=0x55996d12e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.277+0000 7f4b947d4740 1 --2- 172.21.15.27:0/283472239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd552f0 0x55996d12e4b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4b78031440 tx=0x7f4b7802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.278+0000 7f4b947d4740 1 -- 172.21.15.27:0/283472239 shutdown_connections 2024-09-21T00:27:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.278+0000 7f4b947d4740 1 --2- 172.21.15.27:0/283472239 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55996cd3cd80 0x55996d140400 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.278+0000 7f4b947d4740 1 --2- 172.21.15.27:0/283472239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd7e1b0 0x55996d13a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.278+0000 7f4b947d4740 1 --2- 172.21.15.27:0/283472239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd5d4d0 0x55996d12e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.278+0000 7f4b947d4740 1 --2- 172.21.15.27:0/283472239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd552f0 0x55996d12e4b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.278+0000 7f4b947d4740 1 -- 172.21.15.27:0/283472239 >> 172.21.15.27:0/283472239 conn(0x55996cd72300 msgr2=0x55996d123e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.278+0000 7f4b67e00640 0 client.26102 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.278+0000 7f4b947d4740 1 -- 172.21.15.27:0/283472239 shutdown_connections 2024-09-21T00:27:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.278+0000 7f4b947d4740 1 -- 172.21.15.27:0/283472239 wait complete. 2024-09-21T00:27:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.278+0000 7f4b947d4740 20 client.26102 trim_cache size 0 max 16384 2024-09-21T00:27:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.302+0000 7f4b947d4740 1 Processor -- start 2024-09-21T00:27:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.302+0000 7f4b947d4740 1 -- start start 2024-09-21T00:27:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.302+0000 7f4b947d4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd552f0 0x55996d12e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.302+0000 7f4b947d4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd5d4d0 0x55996d124550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.302+0000 7f4b947d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd7e1b0 0x55996d12b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.302+0000 7f4b947d4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881b9c60 con 0x55996cd552f0 2024-09-21T00:27:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.302+0000 7f4b947d4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881b74d0 con 0x55996cd5d4d0 2024-09-21T00:27:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.302+0000 7f4b947d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881b5320 con 0x55996cd7e1b0 2024-09-21T00:27:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.303+0000 7f4b86000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd5d4d0 0x55996d124550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.303+0000 7f4b86000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd5d4d0 0x55996d124550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54994/0 (socket says 172.21.15.27:54994) 2024-09-21T00:27:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.303+0000 7f4b86000640 1 -- 172.21.15.27:0/4089813792 learned_addr learned my addr 172.21.15.27:0/4089813792 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.303+0000 7f4b86a00640 1 --2- 172.21.15.27:0/4089813792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd552f0 0x55996d12e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.303+0000 7f4b87400640 1 --2- 172.21.15.27:0/4089813792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd7e1b0 0x55996d12b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.303+0000 7f4b86000640 1 -- 172.21.15.27:0/4089813792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd7e1b0 msgr2=0x55996d12b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.303+0000 7f4b86000640 1 --2- 172.21.15.27:0/4089813792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd7e1b0 0x55996d12b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.303+0000 7f4b86000640 1 -- 172.21.15.27:0/4089813792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd552f0 msgr2=0x55996d12e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.303+0000 7f4b86000640 1 --2- 172.21.15.27:0/4089813792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd552f0 0x55996d12e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.303+0000 7f4b86000640 1 -- 172.21.15.27:0/4089813792 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55996d131470 con 0x55996cd5d4d0 2024-09-21T00:27:52.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.304+0000 7f4b86000640 1 --2- 172.21.15.27:0/4089813792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd5d4d0 0x55996d124550 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4b740ad7c0 tx=0x7f4b74009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25449be7619cef4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:52.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.304+0000 7f4b72a00640 1 -- 172.21.15.27:0/4089813792 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b740b2070 con 0x55996cd5d4d0 2024-09-21T00:27:52.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.304+0000 7f4b72a00640 1 -- 172.21.15.27:0/4089813792 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b740adf30 con 0x55996cd5d4d0 2024-09-21T00:27:52.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.304+0000 7f4b72a00640 1 -- 172.21.15.27:0/4089813792 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b740aeb60 con 0x55996cd5d4d0 2024-09-21T00:27:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.305+0000 7f4b947d4740 1 -- 172.21.15.27:0/4089813792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd5d4d0 msgr2=0x55996d124550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.305+0000 7f4b947d4740 1 --2- 172.21.15.27:0/4089813792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd5d4d0 0x55996d124550 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4b740ad7c0 tx=0x7f4b74009740 comp rx=0 tx=0).stop 2024-09-21T00:27:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.305+0000 7f4b947d4740 1 -- 172.21.15.27:0/4089813792 shutdown_connections 2024-09-21T00:27:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.305+0000 7f4b947d4740 1 --2- 172.21.15.27:0/4089813792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd7e1b0 0x55996d12b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.305+0000 7f4b947d4740 1 --2- 172.21.15.27:0/4089813792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd5d4d0 0x55996d124550 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.305+0000 7f4b947d4740 1 --2- 172.21.15.27:0/4089813792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd552f0 0x55996d12e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.305+0000 7f4b947d4740 1 -- 172.21.15.27:0/4089813792 >> 172.21.15.27:0/4089813792 conn(0x55996cd72300 msgr2=0x55996d123e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.305+0000 7f4b947d4740 1 -- 172.21.15.27:0/4089813792 shutdown_connections 2024-09-21T00:27:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.305+0000 7f4b947d4740 1 -- 172.21.15.27:0/4089813792 wait complete. 2024-09-21T00:27:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.306+0000 7f4b947d4740 1 -- start start 2024-09-21T00:27:52.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.306+0000 7f4b947d4740 1 Processor -- start 2024-09-21T00:27:52.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.307+0000 7f4b947d4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd552f0 0x55996d140b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.307+0000 7f4b947d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd5d4d0 0x55996d141050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.307+0000 7f4b947d4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd7e1b0 0x55996d1a8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.307+0000 7f4b947d4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881b5990 con 0x55996cd552f0 2024-09-21T00:27:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.307+0000 7f4b947d4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881b6e00 con 0x55996cd7e1b0 2024-09-21T00:27:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.307+0000 7f4b947d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881b6060 con 0x55996cd5d4d0 2024-09-21T00:27:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.307+0000 7f4b947d4740 10 client.0 _ll_register_callbacks cb 0x7f4b800099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.307+0000 7f4b86000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd5d4d0 0x55996d141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.307+0000 7f4b87400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd7e1b0 0x55996d1a8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.307+0000 7f4b86000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd5d4d0 0x55996d141050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:46514/0 (socket says 172.21.15.27:46514) 2024-09-21T00:27:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.307+0000 7f4b86000640 1 -- 172.21.15.27:0/1573275193 learned_addr learned my addr 172.21.15.27:0/1573275193 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:27:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.307+0000 7f4b86a00640 1 --2- 172.21.15.27:0/1573275193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd552f0 0x55996d140b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.307+0000 7f4b86000640 1 -- 172.21.15.27:0/1573275193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd7e1b0 msgr2=0x55996d1a8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.307+0000 7f4b86000640 1 --2- 172.21.15.27:0/1573275193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd7e1b0 0x55996d1a8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.307+0000 7f4b86000640 1 -- 172.21.15.27:0/1573275193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd552f0 msgr2=0x55996d140b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.308+0000 7f4b86000640 1 --2- 172.21.15.27:0/1573275193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd552f0 0x55996d140b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.308+0000 7f4b86000640 1 -- 172.21.15.27:0/1573275193 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55996d1a90b0 con 0x55996cd5d4d0 2024-09-21T00:27:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.308+0000 7f4b86a00640 1 --2- 172.21.15.27:0/1573275193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd552f0 0x55996d140b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:27:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.308+0000 7f4b86000640 1 --2- 172.21.15.27:0/1573275193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd5d4d0 0x55996d141050 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4b740a9e00 tx=0x7f4b740096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3880edd64f217d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.308+0000 7f4b67e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:27:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.308+0000 7f4b67e00640 1 -- 172.21.15.27:0/1573275193 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b740b2040 con 0x55996cd5d4d0 2024-09-21T00:27:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.308+0000 7f4b947d4740 1 -- 172.21.15.27:0/1573275193 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b740adf30 con 0x55996cd5d4d0 2024-09-21T00:27:52.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.312+0000 7f4b67e00640 1 -- 172.21.15.27:0/1573275193 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9431+0+0 (secure 0 0 0) 0x7f4b740aeb60 con 0x55996cd5d4d0 2024-09-21T00:27:52.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.312+0000 7f4b67e00640 1 -- 172.21.15.27:0/1573275193 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4b740aed80 con 0x55996cd5d4d0 2024-09-21T00:27:52.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.312+0000 7f4b947d4740 10 client.26089 fetch_fsmap learned FSMap version 32 2024-09-21T00:27:52.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.312+0000 7f4b947d4740 1 -- 172.21.15.27:0/1573275193 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55996d1aa360 con 0x55996cd5d4d0 2024-09-21T00:27:52.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.313+0000 7f4b67e00640 1 -- 172.21.15.27:0/1573275193 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b740adf30 con 0x55996cd5d4d0 2024-09-21T00:27:52.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.313+0000 7f4b67e00640 1 -- 172.21.15.27:0/1573275193 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f4b740ac500 con 0x55996cd5d4d0 2024-09-21T00:27:52.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.313+0000 7f4b947d4740 10 client.26089 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T00:27:52.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.313+0000 7f4b947d4740 10 client.26089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:27:52.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.313+0000 7f4b947d4740 20 client.26089 populate_metadata read hostname 'smithi027' 2024-09-21T00:27:52.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.313+0000 7f4b947d4740 1 --2- 172.21.15.27:0/1573275193 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55996cd121c0 0x55996d1acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:52.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.313+0000 7f4b947d4740 4 client.26089 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:27:52.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.313+0000 7f4b947d4740 1 -- 172.21.15.27:0/1573275193 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55996d0960b0 con 0x55996cd121c0 2024-09-21T00:27:52.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.313+0000 7f4b86a00640 1 --2- 172.21.15.27:0/1573275193 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55996cd121c0 0x55996d1acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:52.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.314+0000 7f4b86a00640 1 --2- 172.21.15.27:0/1573275193 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55996cd121c0 0x55996d1acb80 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84a0533760ef867b server_cookie=55afb796afbeb524 in_seq=0 out_seq=0 2024-09-21T00:27:52.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.314+0000 7f4b67e00640 10 client.26089 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:52.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:52 smithi027 ceph-mon[30745]: pgmap v572: 129 pgs: 129 active+clean; 881 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 642 op/s 2024-09-21T00:27:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.572+0000 7f4b67e00640 1 -- 172.21.15.27:0/1573275193 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55996d0960b0 con 0x55996cd121c0 2024-09-21T00:27:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.572+0000 7f4b67e00640 10 client.26089 handle_command_reply: tid=0 2024-09-21T00:27:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.572+0000 7f4b947d4740 1 client.26089 shutdown 2024-09-21T00:27:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.572+0000 7f4b947d4740 1 -- 172.21.15.27:0/1573275193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd5d4d0 msgr2=0x55996d141050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.572+0000 7f4b947d4740 1 --2- 172.21.15.27:0/1573275193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd5d4d0 0x55996d141050 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4b740a9e00 tx=0x7f4b740096a0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.573+0000 7f4b947d4740 1 -- 172.21.15.27:0/1573275193 shutdown_connections 2024-09-21T00:27:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.573+0000 7f4b947d4740 1 --2- 172.21.15.27:0/1573275193 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55996cd121c0 0x55996d1acb80 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.573+0000 7f4b947d4740 1 --2- 172.21.15.27:0/1573275193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55996cd7e1b0 0x55996d1a8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.573+0000 7f4b947d4740 1 --2- 172.21.15.27:0/1573275193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55996cd5d4d0 0x55996d141050 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.573+0000 7f4b947d4740 1 --2- 172.21.15.27:0/1573275193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55996cd552f0 0x55996d140b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.573+0000 7f4b947d4740 1 -- 172.21.15.27:0/1573275193 >> 172.21.15.27:0/1573275193 conn(0x55996cd72300 msgr2=0x55996d123e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.573+0000 7f4b67e00640 0 client.26089 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:27:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.573+0000 7f4b947d4740 1 -- 172.21.15.27:0/1573275193 shutdown_connections 2024-09-21T00:27:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.573+0000 7f4b947d4740 1 -- 172.21.15.27:0/1573275193 wait complete. 2024-09-21T00:27:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.573+0000 7f4b947d4740 20 client.26089 trim_cache size 0 max 16384 2024-09-21T00:27:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:27:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:27:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:27:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:27:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.574+0000 7f4b87e00640 1 -- 172.21.15.27:0/1013747366 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4b6c0b65f0 msgr2=0x7f4b6c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.574+0000 7f4b87e00640 1 --2- 172.21.15.27:0/1013747366 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4b6c0b65f0 0x7f4b6c0b8b00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009fd0 tx=0x7f4b80015040 comp rx=0 tx=0).stop 2024-09-21T00:27:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.574+0000 7f4b87e00640 1 -- 172.21.15.27:0/1013747366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b881050f0 msgr2=0x7f4b881a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.574+0000 7f4b87e00640 1 --2- 172.21.15.27:0/1013747366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b881050f0 0x7f4b881a0e00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400e9f0 tx=0x7f4b7400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.574+0000 7f4b87e00640 1 -- 172.21.15.27:0/1013747366 shutdown_connections 2024-09-21T00:27:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.574+0000 7f4b87e00640 1 --2- 172.21.15.27:0/1013747366 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f4b6c0b65f0 0x7f4b6c0b8b00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.574+0000 7f4b87e00640 1 --2- 172.21.15.27:0/1013747366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b8810dbc0 0x7f4b881a6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.574+0000 7f4b87e00640 1 --2- 172.21.15.27:0/1013747366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b881050f0 0x7f4b881a0e00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.574+0000 7f4b87e00640 1 --2- 172.21.15.27:0/1013747366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b881047a0 0x7f4b881a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.574+0000 7f4b87e00640 1 -- 172.21.15.27:0/1013747366 >> 172.21.15.27:0/1013747366 conn(0x7f4b880fe210 msgr2=0x7f4b88107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.575+0000 7f4b87e00640 1 -- 172.21.15.27:0/1013747366 shutdown_connections 2024-09-21T00:27:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:27:52.575+0000 7f4b87e00640 1 -- 172.21.15.27:0/1013747366 wait complete. 2024-09-21T00:27:52.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:52 smithi089 ceph-mon[34481]: pgmap v572: 129 pgs: 129 active+clean; 881 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 642 op/s 2024-09-21T00:27:52.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:52 smithi098 ceph-mon[34725]: pgmap v572: 129 pgs: 129 active+clean; 881 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 642 op/s 2024-09-21T00:27:54.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:54 smithi027 ceph-mon[30745]: pgmap v573: 129 pgs: 129 active+clean; 887 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.0 MiB/s wr, 717 op/s 2024-09-21T00:27:54.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:54 smithi089 ceph-mon[34481]: pgmap v573: 129 pgs: 129 active+clean; 887 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.0 MiB/s wr, 717 op/s 2024-09-21T00:27:54.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:54 smithi098 ceph-mon[34725]: pgmap v573: 129 pgs: 129 active+clean; 887 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.0 MiB/s wr, 717 op/s 2024-09-21T00:27:56.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:56 smithi089 ceph-mon[34481]: pgmap v574: 129 pgs: 129 active+clean; 890 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.0 MiB/s wr, 706 op/s 2024-09-21T00:27:56.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:56 smithi098 ceph-mon[34725]: pgmap v574: 129 pgs: 129 active+clean; 890 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.0 MiB/s wr, 706 op/s 2024-09-21T00:27:56.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:56 smithi027 ceph-mon[30745]: pgmap v574: 129 pgs: 129 active+clean; 890 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.0 MiB/s wr, 706 op/s 2024-09-21T00:27:58.374 INFO:tasks.workunit.client.0.smithi027.stdout: 234 85607 231 57604 356 29531 715 2024-09-21T00:27:58.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:58 smithi089 ceph-mon[34481]: pgmap v575: 129 pgs: 129 active+clean; 887 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 653 op/s 2024-09-21T00:27:58.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:58 smithi027 ceph-mon[30745]: pgmap v575: 129 pgs: 129 active+clean; 887 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 653 op/s 2024-09-21T00:27:58.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:58 smithi098 ceph-mon[34725]: pgmap v575: 129 pgs: 129 active+clean; 887 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 653 op/s 2024-09-21T00:27:59.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:27:59 smithi027 ceph-mon[30745]: pgmap v576: 129 pgs: 129 active+clean; 899 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.4 MiB/s wr, 564 op/s 2024-09-21T00:27:59.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:27:59 smithi089 ceph-mon[34481]: pgmap v576: 129 pgs: 129 active+clean; 899 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.4 MiB/s wr, 564 op/s 2024-09-21T00:27:59.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:27:59 smithi098 ceph-mon[34725]: pgmap v576: 129 pgs: 129 active+clean; 899 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.4 MiB/s wr, 564 op/s 2024-09-21T00:28:01.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:01 smithi089 ceph-mon[34481]: pgmap v577: 129 pgs: 129 active+clean; 904 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.8 MiB/s wr, 409 op/s 2024-09-21T00:28:01.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:01 smithi098 ceph-mon[34725]: pgmap v577: 129 pgs: 129 active+clean; 904 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.8 MiB/s wr, 409 op/s 2024-09-21T00:28:02.073 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:01 smithi027 ceph-mon[30745]: pgmap v577: 129 pgs: 129 active+clean; 904 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.8 MiB/s wr, 409 op/s 2024-09-21T00:28:03.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:03 smithi089 ceph-mon[34481]: pgmap v578: 129 pgs: 129 active+clean; 914 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 434 op/s 2024-09-21T00:28:03.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:03 smithi089 ceph-mon[34481]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T00:28:03.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:03 smithi098 ceph-mon[34725]: pgmap v578: 129 pgs: 129 active+clean; 914 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 434 op/s 2024-09-21T00:28:03.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:03 smithi098 ceph-mon[34725]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T00:28:04.072 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:03 smithi027 ceph-mon[30745]: pgmap v578: 129 pgs: 129 active+clean; 914 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 434 op/s 2024-09-21T00:28:04.072 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:03 smithi027 ceph-mon[30745]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T00:28:05.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:05 smithi089 ceph-mon[34481]: pgmap v580: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 918 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 376 op/s 2024-09-21T00:28:05.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:05 smithi098 ceph-mon[34725]: pgmap v580: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 918 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 376 op/s 2024-09-21T00:28:06.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:05 smithi027 ceph-mon[30745]: pgmap v580: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 918 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 376 op/s 2024-09-21T00:28:06.458 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T00:28:06.459 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.25 2024-09-21T00:28:06.459 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:28:06.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:06 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:28:06.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:06 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:28:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.011+0000 7fe18ce00640 1 -- 172.21.15.27:0/952423302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe18811cba0 msgr2=0x7fe18811f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.014+0000 7fe18ce00640 1 --2- 172.21.15.27:0/952423302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe18811cba0 0x7fe18811f090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe180009ca0 tx=0x7fe18002a960 comp rx=0 tx=0).stop 2024-09-21T00:28:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.015+0000 7fe18ce00640 1 -- 172.21.15.27:0/952423302 shutdown_connections 2024-09-21T00:28:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.015+0000 7fe18ce00640 1 --2- 172.21.15.27:0/952423302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe18811cba0 0x7fe18811f090 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.015+0000 7fe18ce00640 1 --2- 172.21.15.27:0/952423302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe18810a900 0x7fe18810ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.015+0000 7fe18ce00640 1 --2- 172.21.15.27:0/952423302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe188111ea0 0x7fe18810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.015+0000 7fe18ce00640 1 -- 172.21.15.27:0/952423302 >> 172.21.15.27:0/952423302 conn(0x7fe188070d90 msgr2=0x7fe1880711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.015+0000 7fe18ce00640 1 -- 172.21.15.27:0/952423302 shutdown_connections 2024-09-21T00:28:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.015+0000 7fe18ce00640 1 -- 172.21.15.27:0/952423302 wait complete. 2024-09-21T00:28:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.021+0000 7fe18ce00640 1 Processor -- start 2024-09-21T00:28:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.022+0000 7fe18ce00640 1 -- start start 2024-09-21T00:28:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.022+0000 7fe18ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe18810a900 0x7fe1881a9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.026+0000 7fe18ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe188111ea0 0x7fe1881a9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.026+0000 7fe18ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe18811cba0 0x7fe1881ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.026+0000 7fe185e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe18810a900 0x7fe1881a9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.026+0000 7fe185e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe18810a900 0x7fe1881a9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:34982/0 (socket says 172.21.15.27:34982) 2024-09-21T00:28:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.026+0000 7fe185e00640 1 -- 172.21.15.27:0/945183804 learned_addr learned my addr 172.21.15.27:0/945183804 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.026+0000 7fe18ce00640 1 -- 172.21.15.27:0/945183804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881227f0 con 0x7fe18811cba0 2024-09-21T00:28:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.026+0000 7fe18ce00640 1 -- 172.21.15.27:0/945183804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe188122670 con 0x7fe18810a900 2024-09-21T00:28:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.026+0000 7fe18ce00640 1 -- 172.21.15.27:0/945183804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe188122970 con 0x7fe188111ea0 2024-09-21T00:28:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.027+0000 7fe185e00640 1 -- 172.21.15.27:0/945183804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe188111ea0 msgr2=0x7fe1881a9730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:28:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.027+0000 7fe185e00640 1 --2- 172.21.15.27:0/945183804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe188111ea0 0x7fe1881a9730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.027+0000 7fe185e00640 1 -- 172.21.15.27:0/945183804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe18811cba0 msgr2=0x7fe1881ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.027+0000 7fe185e00640 1 --2- 172.21.15.27:0/945183804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe18811cba0 0x7fe1881ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.027+0000 7fe185e00640 1 -- 172.21.15.27:0/945183804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1881af030 con 0x7fe18810a900 2024-09-21T00:28:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.028+0000 7fe185e00640 1 --2- 172.21.15.27:0/945183804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe18810a900 0x7fe1881a9190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe17000d950 tx=0x7fe17000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.028+0000 7fe17ea00640 1 -- 172.21.15.27:0/945183804 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170017070 con 0x7fe18810a900 2024-09-21T00:28:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.028+0000 7fe17ea00640 1 -- 172.21.15.27:0/945183804 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe170009c30 con 0x7fe18810a900 2024-09-21T00:28:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.030+0000 7fe18ce00640 1 -- 172.21.15.27:0/945183804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1881af320 con 0x7fe18810a900 2024-09-21T00:28:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.030+0000 7fe18ce00640 1 -- 172.21.15.27:0/945183804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1881b5b40 con 0x7fe18810a900 2024-09-21T00:28:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.030+0000 7fe17ea00640 1 -- 172.21.15.27:0/945183804 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170017070 con 0x7fe18810a900 2024-09-21T00:28:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.045+0000 7fe193e85740 1 Processor -- start 2024-09-21T00:28:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.046+0000 7fe193e85740 1 -- start start 2024-09-21T00:28:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.047+0000 7fe193e85740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4471b0 0x5566de7ebc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.050+0000 7fe17ea00640 1 -- 172.21.15.27:0/945183804 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe1700050d0 con 0x7fe18810a900 2024-09-21T00:28:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.051+0000 7fe186800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4471b0 0x5566de7ebc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.051+0000 7fe186800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4471b0 0x5566de7ebc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54666/0 (socket says 172.21.15.27:54666) 2024-09-21T00:28:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.051+0000 7fe186800640 1 -- 172.21.15.27:0/3574175743 learned_addr learned my addr 172.21.15.27:0/3574175743 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.051+0000 7fe193e85740 1 --2- 172.21.15.27:0/3574175743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de41e2f0 0x5566de7ec170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.051+0000 7fe193e85740 1 --2- 172.21.15.27:0/3574175743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de4264d0 0x5566de7ec6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.051+0000 7fe193e85740 1 -- 172.21.15.27:0/3574175743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5566de7616b0 con 0x5566de4471b0 2024-09-21T00:28:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.051+0000 7fe193e85740 1 -- 172.21.15.27:0/3574175743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5566de750ae0 con 0x5566de41e2f0 2024-09-21T00:28:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.051+0000 7fe193e85740 1 -- 172.21.15.27:0/3574175743 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5566de7548f0 con 0x5566de4264d0 2024-09-21T00:28:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.053+0000 7fe17ea00640 1 --2- 172.21.15.27:0/945183804 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe16c0b6ad0 0x7fe16c0b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.053+0000 7fe17ea00640 1 -- 172.21.15.27:0/945183804 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9607+0+0 (secure 0 0 0) 0x7fe170009a30 con 0x7fe18810a900 2024-09-21T00:28:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.056+0000 7fe186800640 1 --2- 172.21.15.27:0/945183804 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe16c0b6ad0 0x7fe16c0b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.056+0000 7fe186800640 1 -- 172.21.15.27:0/3574175743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de4264d0 msgr2=0x5566de7ec6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:28:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.056+0000 7fe186800640 1 --2- 172.21.15.27:0/3574175743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de4264d0 0x5566de7ec6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.056+0000 7fe186800640 1 -- 172.21.15.27:0/3574175743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de41e2f0 msgr2=0x5566de7ec170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:28:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.056+0000 7fe186800640 1 --2- 172.21.15.27:0/3574175743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de41e2f0 0x5566de7ec170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.056+0000 7fe186800640 1 -- 172.21.15.27:0/3574175743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566de71d780 con 0x5566de4471b0 2024-09-21T00:28:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.061+0000 7fe186800640 1 --2- 172.21.15.27:0/3574175743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4471b0 0x5566de7ebc30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe1800094e0 tx=0x7fe180003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8945bc605dc47e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.062+0000 7fe177400640 1 -- 172.21.15.27:0/3574175743 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180034040 con 0x5566de4471b0 2024-09-21T00:28:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.063+0000 7fe177400640 1 -- 172.21.15.27:0/3574175743 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe18005d070 con 0x5566de4471b0 2024-09-21T00:28:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.064+0000 7fe177400640 1 -- 172.21.15.27:0/3574175743 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1800044d0 con 0x5566de4471b0 2024-09-21T00:28:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.064+0000 7fe193e85740 1 -- 172.21.15.27:0/3574175743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4471b0 msgr2=0x5566de7ebc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.065+0000 7fe193e85740 1 --2- 172.21.15.27:0/3574175743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4471b0 0x5566de7ebc30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe1800094e0 tx=0x7fe180003a70 comp rx=0 tx=0).stop 2024-09-21T00:28:07.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:06 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:28:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.067+0000 7fe186800640 1 --2- 172.21.15.27:0/945183804 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe16c0b6ad0 0x7fe16c0b8fe0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe18005c000 tx=0x7fe18005a000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.069+0000 7fe193e85740 1 -- 172.21.15.27:0/3574175743 shutdown_connections 2024-09-21T00:28:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.069+0000 7fe193e85740 1 --2- 172.21.15.27:0/3574175743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de4264d0 0x5566de7ec6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.069+0000 7fe193e85740 1 --2- 172.21.15.27:0/3574175743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de41e2f0 0x5566de7ec170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.069+0000 7fe193e85740 1 --2- 172.21.15.27:0/3574175743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4471b0 0x5566de7ebc30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.069+0000 7fe193e85740 1 -- 172.21.15.27:0/3574175743 >> 172.21.15.27:0/3574175743 conn(0x5566de43b300 msgr2=0x5566de7e9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.071+0000 7fe193e85740 1 -- 172.21.15.27:0/3574175743 shutdown_connections 2024-09-21T00:28:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.071+0000 7fe193e85740 1 -- 172.21.15.27:0/3574175743 wait complete. 2024-09-21T00:28:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.072+0000 7fe193e85740 1 -- start start 2024-09-21T00:28:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.072+0000 7fe193e85740 1 Processor -- start 2024-09-21T00:28:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.073+0000 7fe193e85740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de41e2f0 0x5566de7f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.074+0000 7fe193e85740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4264d0 0x5566de7f7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.074+0000 7fe185e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de41e2f0 0x5566de7f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.074+0000 7fe185e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de41e2f0 0x5566de7f71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49592/0 (socket says 172.21.15.27:49592) 2024-09-21T00:28:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.075+0000 7fe193e85740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 0x5566de7f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.075+0000 7fe193e85740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5566de688ab0 con 0x5566de4264d0 2024-09-21T00:28:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.075+0000 7fe193e85740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5566de6f7570 con 0x5566de4471b0 2024-09-21T00:28:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.075+0000 7fe193e85740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5566de7fa450 con 0x5566de41e2f0 2024-09-21T00:28:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.075+0000 7fe193e85740 10 client.0 _ll_register_callbacks cb 0x5566de706100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.075+0000 7fe185e00640 1 -- 172.21.15.27:0/3397211810 learned_addr learned my addr 172.21.15.27:0/3397211810 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.075+0000 7fe185400640 1 --2- 172.21.15.27:0/3397211810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4264d0 0x5566de7f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.075+0000 7fe185e00640 1 -- 172.21.15.27:0/3397211810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 msgr2=0x5566de7f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.075+0000 7fe185e00640 1 --2- 172.21.15.27:0/3397211810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 0x5566de7f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.075+0000 7fe185e00640 1 -- 172.21.15.27:0/3397211810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4264d0 msgr2=0x5566de7f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.075+0000 7fe185e00640 1 --2- 172.21.15.27:0/3397211810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4264d0 0x5566de7f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.075+0000 7fe185e00640 1 -- 172.21.15.27:0/3397211810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566de4b2790 con 0x5566de41e2f0 2024-09-21T00:28:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.078+0000 7fe185e00640 1 --2- 172.21.15.27:0/3397211810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de41e2f0 0x5566de7f71d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe170009610 tx=0x7fe1700e3d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=751c236bc0444197 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.078+0000 7fe16be00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:28:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.079+0000 7fe16be00640 1 -- 172.21.15.27:0/3397211810 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170017040 con 0x5566de41e2f0 2024-09-21T00:28:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.080+0000 7fe16be00640 1 -- 172.21.15.27:0/3397211810 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9607+0+0 (secure 0 0 0) 0x7fe1700e6170 con 0x5566de41e2f0 2024-09-21T00:28:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.080+0000 7fe193e85740 1 -- 172.21.15.27:0/3397211810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe18005d070 con 0x5566de41e2f0 2024-09-21T00:28:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.081+0000 7fe16be00640 1 -- 172.21.15.27:0/3397211810 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe1700ac6c0 con 0x5566de41e2f0 2024-09-21T00:28:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.081+0000 7fe193e85740 10 client.26095 fetch_fsmap learned FSMap version 33 2024-09-21T00:28:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.081+0000 7fe193e85740 1 -- 172.21.15.27:0/3397211810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5566de52b620 con 0x5566de41e2f0 2024-09-21T00:28:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.082+0000 7fe16be00640 1 -- 172.21.15.27:0/3397211810 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe18005d070 con 0x5566de41e2f0 2024-09-21T00:28:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.083+0000 7fe16be00640 1 -- 172.21.15.27:0/3397211810 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fe1700ae060 con 0x5566de41e2f0 2024-09-21T00:28:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.084+0000 7fe193e85740 10 client.26095 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T00:28:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.084+0000 7fe193e85740 10 client.26095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:28:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.084+0000 7fe193e85740 20 client.26095 populate_metadata read hostname 'smithi027' 2024-09-21T00:28:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.084+0000 7fe193e85740 1 --2- 172.21.15.27:0/3397211810 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5566de405d80 0x5566de8094f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.084+0000 7fe193e85740 4 client.26095 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:28:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.084+0000 7fe193e85740 1 -- 172.21.15.27:0/3397211810 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566de763600 con 0x5566de405d80 2024-09-21T00:28:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.084+0000 7fe185400640 1 --2- 172.21.15.27:0/3397211810 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5566de405d80 0x5566de8094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.085+0000 7fe185400640 1 --2- 172.21.15.27:0/3397211810 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5566de405d80 0x5566de8094f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b47cd9478f492b7e server_cookie=506b2b4f96258c91 in_seq=0 out_seq=0 2024-09-21T00:28:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.085+0000 7fe16be00640 10 client.26095 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.087+0000 7fe16be00640 1 -- 172.21.15.27:0/3397211810 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5566de763600 con 0x5566de405d80 2024-09-21T00:28:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.087+0000 7fe16be00640 10 client.26095 handle_command_reply: tid=0 2024-09-21T00:28:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.087+0000 7fe193e85740 1 client.26095 shutdown 2024-09-21T00:28:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.096+0000 7fe193e85740 1 -- 172.21.15.27:0/3397211810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de41e2f0 msgr2=0x5566de7f71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.096+0000 7fe193e85740 1 --2- 172.21.15.27:0/3397211810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de41e2f0 0x5566de7f71d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe170009610 tx=0x7fe1700e3d30 comp rx=0 tx=0).stop 2024-09-21T00:28:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.096+0000 7fe193e85740 1 -- 172.21.15.27:0/3397211810 shutdown_connections 2024-09-21T00:28:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.096+0000 7fe193e85740 1 --2- 172.21.15.27:0/3397211810 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5566de405d80 0x5566de8094f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.096+0000 7fe193e85740 1 --2- 172.21.15.27:0/3397211810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 0x5566de7f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.096+0000 7fe193e85740 1 --2- 172.21.15.27:0/3397211810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4264d0 0x5566de7f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.096+0000 7fe193e85740 1 --2- 172.21.15.27:0/3397211810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de41e2f0 0x5566de7f71d0 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe170009610 tx=0x7fe1700e3d30 comp rx=0 tx=0).stop 2024-09-21T00:28:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.096+0000 7fe193e85740 1 -- 172.21.15.27:0/3397211810 >> 172.21.15.27:0/3397211810 conn(0x5566de43b300 msgr2=0x5566de7ecc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.096+0000 7fe16be00640 0 client.26095 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.101+0000 7fe193e85740 1 -- 172.21.15.27:0/3397211810 shutdown_connections 2024-09-21T00:28:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.101+0000 7fe193e85740 1 -- 172.21.15.27:0/3397211810 wait complete. 2024-09-21T00:28:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.103+0000 7fe193e85740 20 client.26095 trim_cache size 0 max 16384 2024-09-21T00:28:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.145+0000 7fe193e85740 1 Processor -- start 2024-09-21T00:28:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.146+0000 7fe193e85740 1 -- start start 2024-09-21T00:28:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.146+0000 7fe193e85740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de4264d0 0x5566de7e87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.146+0000 7fe193e85740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 0x5566de7f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.146+0000 7fe193e85740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de3db1c0 0x5566de7f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.146+0000 7fe193e85740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881b9890 con 0x5566de3db1c0 2024-09-21T00:28:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.146+0000 7fe193e85740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881b9fc0 con 0x5566de4471b0 2024-09-21T00:28:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.146+0000 7fe193e85740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881b7800 con 0x5566de4264d0 2024-09-21T00:28:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.147+0000 7fe185e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de4264d0 0x5566de7e87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.147+0000 7fe185e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de4264d0 0x5566de7e87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49600/0 (socket says 172.21.15.27:49600) 2024-09-21T00:28:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.147+0000 7fe185e00640 1 -- 172.21.15.27:0/4183649176 learned_addr learned my addr 172.21.15.27:0/4183649176 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.147+0000 7fe185400640 1 --2- 172.21.15.27:0/4183649176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 0x5566de7f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.147+0000 7fe185e00640 1 -- 172.21.15.27:0/4183649176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 msgr2=0x5566de7f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.147+0000 7fe185e00640 1 --2- 172.21.15.27:0/4183649176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 0x5566de7f43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.147+0000 7fe185e00640 1 -- 172.21.15.27:0/4183649176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de3db1c0 msgr2=0x5566de7f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.147+0000 7fe185e00640 1 --2- 172.21.15.27:0/4183649176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de3db1c0 0x5566de7f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.147+0000 7fe185e00640 1 -- 172.21.15.27:0/4183649176 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566de7f8490 con 0x5566de4264d0 2024-09-21T00:28:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.147+0000 7fe185e00640 1 --2- 172.21.15.27:0/4183649176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de4264d0 0x5566de7e87f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe1700abcc0 tx=0x7fe170000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7c49b259a3bc29e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.148+0000 7fe176a00640 1 -- 172.21.15.27:0/4183649176 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170017070 con 0x5566de4264d0 2024-09-21T00:28:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.148+0000 7fe176a00640 1 -- 172.21.15.27:0/4183649176 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe170009c30 con 0x5566de4264d0 2024-09-21T00:28:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.149+0000 7fe193e85740 1 -- 172.21.15.27:0/4183649176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de4264d0 msgr2=0x5566de7e87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.149+0000 7fe193e85740 1 --2- 172.21.15.27:0/4183649176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de4264d0 0x5566de7e87f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe1700abcc0 tx=0x7fe170000bc0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.150+0000 7fe193e85740 1 -- 172.21.15.27:0/4183649176 shutdown_connections 2024-09-21T00:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.151+0000 7fe193e85740 1 --2- 172.21.15.27:0/4183649176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de3db1c0 0x5566de7f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.151+0000 7fe193e85740 1 --2- 172.21.15.27:0/4183649176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 0x5566de7f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.151+0000 7fe193e85740 1 --2- 172.21.15.27:0/4183649176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de4264d0 0x5566de7e87f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.151+0000 7fe193e85740 1 -- 172.21.15.27:0/4183649176 >> 172.21.15.27:0/4183649176 conn(0x5566de43b300 msgr2=0x5566de7e80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.156+0000 7fe193e85740 1 -- 172.21.15.27:0/4183649176 shutdown_connections 2024-09-21T00:28:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.156+0000 7fe193e85740 1 -- 172.21.15.27:0/4183649176 wait complete. 2024-09-21T00:28:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.157+0000 7fe193e85740 1 -- start start 2024-09-21T00:28:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.157+0000 7fe193e85740 1 Processor -- start 2024-09-21T00:28:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.158+0000 7fe193e85740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de3db1c0 0x5566de8000f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.159+0000 7fe193e85740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4264d0 0x5566de7fd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.159+0000 7fe193e85740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 0x5566de7fdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.160+0000 7fe193e85740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881b5d10 con 0x5566de4264d0 2024-09-21T00:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.160+0000 7fe193e85740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881b7130 con 0x5566de4471b0 2024-09-21T00:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.160+0000 7fe193e85740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881b6390 con 0x5566de3db1c0 2024-09-21T00:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.160+0000 7fe193e85740 10 client.0 _ll_register_callbacks cb 0x7fe170005020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.164+0000 7fe186800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 0x5566de7fdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.164+0000 7fe186800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 0x5566de7fdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:35006/0 (socket says 172.21.15.27:35006) 2024-09-21T00:28:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.164+0000 7fe186800640 1 -- 172.21.15.27:0/2777186028 learned_addr learned my addr 172.21.15.27:0/2777186028 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.164+0000 7fe185400640 1 --2- 172.21.15.27:0/2777186028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4264d0 0x5566de7fd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.164+0000 7fe185e00640 1 --2- 172.21.15.27:0/2777186028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de3db1c0 0x5566de8000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.166+0000 7fe186800640 1 -- 172.21.15.27:0/2777186028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de3db1c0 msgr2=0x5566de8000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.166+0000 7fe186800640 1 --2- 172.21.15.27:0/2777186028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de3db1c0 0x5566de8000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.166+0000 7fe186800640 1 -- 172.21.15.27:0/2777186028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4264d0 msgr2=0x5566de7fd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.166+0000 7fe186800640 1 --2- 172.21.15.27:0/2777186028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4264d0 0x5566de7fd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.166+0000 7fe186800640 1 -- 172.21.15.27:0/2777186028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566de872ec0 con 0x5566de4471b0 2024-09-21T00:28:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.167+0000 7fe186800640 1 --2- 172.21.15.27:0/2777186028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 0x5566de7fdd00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe180030e80 tx=0x7fe180030eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc0b6e6634cf2b35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.168+0000 7fe16be00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:28:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.170+0000 7fe16be00640 1 -- 172.21.15.27:0/2777186028 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180069020 con 0x5566de4471b0 2024-09-21T00:28:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.170+0000 7fe16be00640 1 -- 172.21.15.27:0/2777186028 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9607+0+0 (secure 0 0 0) 0x7fe180004200 con 0x5566de4471b0 2024-09-21T00:28:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.174+0000 7fe193e85740 1 -- 172.21.15.27:0/2777186028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe170009c30 con 0x5566de4471b0 2024-09-21T00:28:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.174+0000 7fe16be00640 1 -- 172.21.15.27:0/2777186028 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe180034050 con 0x5566de4471b0 2024-09-21T00:28:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.175+0000 7fe193e85740 10 client.26117 fetch_fsmap learned FSMap version 33 2024-09-21T00:28:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.175+0000 7fe193e85740 1 -- 172.21.15.27:0/2777186028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5566de874020 con 0x5566de4471b0 2024-09-21T00:28:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.177+0000 7fe16be00640 1 -- 172.21.15.27:0/2777186028 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe170009c30 con 0x5566de4471b0 2024-09-21T00:28:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.177+0000 7fe16be00640 1 -- 172.21.15.27:0/2777186028 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fe180057070 con 0x5566de4471b0 2024-09-21T00:28:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.177+0000 7fe193e85740 10 client.26117 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T00:28:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.177+0000 7fe193e85740 10 client.26117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:28:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.177+0000 7fe193e85740 20 client.26117 populate_metadata read hostname 'smithi027' 2024-09-21T00:28:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.177+0000 7fe193e85740 1 --2- 172.21.15.27:0/2777186028 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5566de874350 0x5566de876770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.177+0000 7fe193e85740 4 client.26117 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:28:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.177+0000 7fe193e85740 1 -- 172.21.15.27:0/2777186028 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566de75f0b0 con 0x5566de874350 2024-09-21T00:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.178+0000 7fe185e00640 1 --2- 172.21.15.27:0/2777186028 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5566de874350 0x5566de876770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.182+0000 7fe185e00640 1 --2- 172.21.15.27:0/2777186028 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5566de874350 0x5566de876770 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87561a2bae4a2970 server_cookie=6ff8c060285a76d9 in_seq=0 out_seq=0 2024-09-21T00:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:07.182+0000 7fe16be00640 10 client.26117 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:07.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:07 smithi089 ceph-mon[34481]: pgmap v581: 129 pgs: 15 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 73 active+clean; 920 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 308 op/s 2024-09-21T00:28:07.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:07 smithi098 ceph-mon[34725]: pgmap v581: 129 pgs: 15 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 73 active+clean; 920 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 308 op/s 2024-09-21T00:28:08.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:07 smithi027 ceph-mon[30745]: pgmap v581: 129 pgs: 15 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 73 active+clean; 920 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 308 op/s 2024-09-21T00:28:08.376 INFO:tasks.workunit.client.0.smithi027.stdout: 234 329634 141 219437 53 103155 186 2024-09-21T00:28:08.419 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:28:09.076 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:08 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:28:09.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:08 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:28:09.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:08 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:28:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.388+0000 7fe16be00640 1 -- 172.21.15.27:0/2777186028 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566de75f0b0 con 0x5566de874350 2024-09-21T00:28:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.388+0000 7fe16be00640 10 client.26117 handle_command_reply: tid=0 2024-09-21T00:28:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.389+0000 7fe193e85740 1 client.26117 shutdown 2024-09-21T00:28:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.401+0000 7fe193e85740 1 -- 172.21.15.27:0/2777186028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 msgr2=0x5566de7fdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.401+0000 7fe193e85740 1 --2- 172.21.15.27:0/2777186028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 0x5566de7fdd00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe180030e80 tx=0x7fe180030eb0 comp rx=0 tx=0).stop 2024-09-21T00:28:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.407+0000 7fe193e85740 1 -- 172.21.15.27:0/2777186028 shutdown_connections 2024-09-21T00:28:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.407+0000 7fe193e85740 1 --2- 172.21.15.27:0/2777186028 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5566de874350 0x5566de876770 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.407+0000 7fe193e85740 1 --2- 172.21.15.27:0/2777186028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566de4471b0 0x5566de7fdd00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.407+0000 7fe193e85740 1 --2- 172.21.15.27:0/2777186028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566de4264d0 0x5566de7fd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.407+0000 7fe193e85740 1 --2- 172.21.15.27:0/2777186028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566de3db1c0 0x5566de8000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.407+0000 7fe193e85740 1 -- 172.21.15.27:0/2777186028 >> 172.21.15.27:0/2777186028 conn(0x5566de43b300 msgr2=0x5566de802690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.407+0000 7fe16be00640 0 client.26117 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.407+0000 7fe193e85740 1 -- 172.21.15.27:0/2777186028 shutdown_connections 2024-09-21T00:28:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.407+0000 7fe193e85740 1 -- 172.21.15.27:0/2777186028 wait complete. 2024-09-21T00:28:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.408+0000 7fe193e85740 20 client.26117 trim_cache size 0 max 16384 2024-09-21T00:28:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:28:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:28:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:28:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:28:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.434+0000 7fe18ce00640 1 -- 172.21.15.27:0/945183804 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe16c0b6ad0 msgr2=0x7fe16c0b8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.434+0000 7fe18ce00640 1 --2- 172.21.15.27:0/945183804 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe16c0b6ad0 0x7fe16c0b8fe0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe18005c000 tx=0x7fe18005a000 comp rx=0 tx=0).stop 2024-09-21T00:28:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.434+0000 7fe18ce00640 1 -- 172.21.15.27:0/945183804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe18810a900 msgr2=0x7fe1881a9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.434+0000 7fe18ce00640 1 --2- 172.21.15.27:0/945183804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe18810a900 0x7fe1881a9190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe17000d950 tx=0x7fe17000de10 comp rx=0 tx=0).stop 2024-09-21T00:28:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.451+0000 7fe18ce00640 1 -- 172.21.15.27:0/945183804 shutdown_connections 2024-09-21T00:28:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.451+0000 7fe18ce00640 1 --2- 172.21.15.27:0/945183804 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe16c0b6ad0 0x7fe16c0b8fe0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.451+0000 7fe18ce00640 1 --2- 172.21.15.27:0/945183804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe18811cba0 0x7fe1881ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.451+0000 7fe18ce00640 1 --2- 172.21.15.27:0/945183804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe188111ea0 0x7fe1881a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.451+0000 7fe18ce00640 1 --2- 172.21.15.27:0/945183804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe18810a900 0x7fe1881a9190 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.451+0000 7fe18ce00640 1 -- 172.21.15.27:0/945183804 >> 172.21.15.27:0/945183804 conn(0x7fe188070d90 msgr2=0x7fe18811ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.461+0000 7fe18ce00640 1 -- 172.21.15.27:0/945183804 shutdown_connections 2024-09-21T00:28:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:09.461+0000 7fe18ce00640 1 -- 172.21.15.27:0/945183804 wait complete. 2024-09-21T00:28:10.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: pgmap v582: 129 pgs: 19 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 56 active+clean; 921 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 249 op/s 2024-09-21T00:28:10.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: Adjusting osd_memory_target on smithi089 to 175.9M 2024-09-21T00:28:10.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: Adjusting osd_memory_target on smithi098 to 176.0M 2024-09-21T00:28:10.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: Unable to set osd_memory_target on smithi089 to 184531558: error parsing value: Value '184531558' is below minimum 939524096 2024-09-21T00:28:10.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:09 smithi027 ceph-mon[30745]: Unable to set osd_memory_target on smithi098 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-21T00:28:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: pgmap v582: 129 pgs: 19 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 56 active+clean; 921 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 249 op/s 2024-09-21T00:28:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: Adjusting osd_memory_target on smithi089 to 175.9M 2024-09-21T00:28:10.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: Adjusting osd_memory_target on smithi098 to 176.0M 2024-09-21T00:28:10.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: Unable to set osd_memory_target on smithi089 to 184531558: error parsing value: Value '184531558' is below minimum 939524096 2024-09-21T00:28:10.230 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:09 smithi089 ceph-mon[34481]: Unable to set osd_memory_target on smithi098 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-21T00:28:10.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: pgmap v582: 129 pgs: 19 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 56 active+clean; 921 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 249 op/s 2024-09-21T00:28:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: Adjusting osd_memory_target on smithi089 to 175.9M 2024-09-21T00:28:10.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T00:28:10.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: Adjusting osd_memory_target on smithi098 to 176.0M 2024-09-21T00:28:10.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: Unable to set osd_memory_target on smithi089 to 184531558: error parsing value: Value '184531558' is below minimum 939524096 2024-09-21T00:28:10.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:09 smithi098 ceph-mon[34725]: Unable to set osd_memory_target on smithi098 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-21T00:28:11.119 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:10 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:28:11.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:10 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:28:11.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:10 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:28:12.084 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:11 smithi027 ceph-mon[30745]: pgmap v583: 129 pgs: 19 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 56 active+clean; 922 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 287 op/s 2024-09-21T00:28:12.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:11 smithi089 ceph-mon[34481]: pgmap v583: 129 pgs: 19 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 56 active+clean; 922 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 287 op/s 2024-09-21T00:28:12.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:11 smithi098 ceph-mon[34725]: pgmap v583: 129 pgs: 19 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 56 active+clean; 922 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 287 op/s 2024-09-21T00:28:14.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:13 smithi089 ceph-mon[34481]: pgmap v584: 129 pgs: 19 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 56 active+clean; 919 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 308 op/s 2024-09-21T00:28:14.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:13 smithi098 ceph-mon[34725]: pgmap v584: 129 pgs: 19 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 56 active+clean; 919 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 308 op/s 2024-09-21T00:28:14.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:13 smithi027 ceph-mon[30745]: pgmap v584: 129 pgs: 19 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 56 active+clean; 919 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 308 op/s 2024-09-21T00:28:16.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:15 smithi027 ceph-mon[30745]: pgmap v585: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 919 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 1.0 MiB/s wr, 280 op/s 2024-09-21T00:28:16.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:15 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:28:16.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:15 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:28:16.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:15 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:28:16.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:15 smithi089 ceph-mon[34481]: pgmap v585: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 919 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 1.0 MiB/s wr, 280 op/s 2024-09-21T00:28:16.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:15 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:28:16.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:15 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:28:16.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:15 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:28:16.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:15 smithi098 ceph-mon[34725]: pgmap v585: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 919 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 1.0 MiB/s wr, 280 op/s 2024-09-21T00:28:16.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:15 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:28:16.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:15 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:28:16.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:15 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:28:16.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.891+0000 7f99d7200640 1 -- 172.21.15.27:0/3851466259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d81039b0 msgr2=0x7f99d8103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:16.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.891+0000 7f99d7200640 1 --2- 172.21.15.27:0/3851466259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d81039b0 0x7f99d8103dc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f99c8009900 tx=0x7f99c802a400 comp rx=0 tx=0).stop 2024-09-21T00:28:16.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.891+0000 7f99d7200640 1 -- 172.21.15.27:0/3851466259 shutdown_connections 2024-09-21T00:28:16.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.891+0000 7f99d7200640 1 --2- 172.21.15.27:0/3851466259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99d8109c40 0x7f99d8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.891+0000 7f99d7200640 1 --2- 172.21.15.27:0/3851466259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d81092f0 0x7f99d8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.891+0000 7f99d7200640 1 --2- 172.21.15.27:0/3851466259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d81039b0 0x7f99d8103dc0 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f99c8009900 tx=0x7f99c802a400 comp rx=0 tx=0).stop 2024-09-21T00:28:16.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.891+0000 7f99d7200640 1 -- 172.21.15.27:0/3851466259 >> 172.21.15.27:0/3851466259 conn(0x7f99d80ff440 msgr2=0x7f99d8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:16.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.895+0000 7f99d7200640 1 -- 172.21.15.27:0/3851466259 shutdown_connections 2024-09-21T00:28:16.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.895+0000 7f99d7200640 1 -- 172.21.15.27:0/3851466259 wait complete. 2024-09-21T00:28:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.912+0000 7f99d7200640 1 Processor -- start 2024-09-21T00:28:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.913+0000 7f99d7200640 1 -- start start 2024-09-21T00:28:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.914+0000 7f99d7200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d81092f0 0x7f99d81a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.914+0000 7f99d7200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99d8109c40 0x7f99d81a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.914+0000 7f99d7200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d81aea20 0x7f99d81aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:16.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.914+0000 7f99d7200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f99d806d6a0 con 0x7f99d8109c40 2024-09-21T00:28:16.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.914+0000 7f99d7200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f99d806d520 con 0x7f99d81aea20 2024-09-21T00:28:16.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.914+0000 7f99d7200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f99d806d820 con 0x7f99d81092f0 2024-09-21T00:28:16.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.915+0000 7f99d5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d81092f0 0x7f99d81a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:16.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.915+0000 7f99d5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99d8109c40 0x7f99d81a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:16.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.917+0000 7f99d5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99d8109c40 0x7f99d81a9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47668/0 (socket says 172.21.15.27:47668) 2024-09-21T00:28:16.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.917+0000 7f99d5400640 1 -- 172.21.15.27:0/2621893849 learned_addr learned my addr 172.21.15.27:0/2621893849 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:16.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.915+0000 7f99d6800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d81aea20 0x7f99d81aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:16.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.920+0000 7f99d5e00640 1 -- 172.21.15.27:0/2621893849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d81aea20 msgr2=0x7f99d81aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:16.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.920+0000 7f99d5e00640 1 --2- 172.21.15.27:0/2621893849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d81aea20 0x7f99d81aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.920+0000 7f99d5e00640 1 -- 172.21.15.27:0/2621893849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99d8109c40 msgr2=0x7f99d81a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:16.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.920+0000 7f99d5e00640 1 --2- 172.21.15.27:0/2621893849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99d8109c40 0x7f99d81a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.920+0000 7f99d5e00640 1 -- 172.21.15.27:0/2621893849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d81af600 con 0x7f99d81092f0 2024-09-21T00:28:16.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.921+0000 7f99d5400640 1 --2- 172.21.15.27:0/2621893849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99d8109c40 0x7f99d81a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:28:16.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.926+0000 7f99d5e00640 1 --2- 172.21.15.27:0/2621893849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d81092f0 0x7f99d81a92a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f99c8008000 tx=0x7f99c8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:16.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.927+0000 7f99cea00640 1 -- 172.21.15.27:0/2621893849 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c803f020 con 0x7f99d81092f0 2024-09-21T00:28:16.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.927+0000 7f99cea00640 1 -- 172.21.15.27:0/2621893849 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99c8031b60 con 0x7f99d81092f0 2024-09-21T00:28:16.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.927+0000 7f99cea00640 1 -- 172.21.15.27:0/2621893849 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c803f020 con 0x7f99d81092f0 2024-09-21T00:28:16.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.929+0000 7f99d7200640 1 -- 172.21.15.27:0/2621893849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d81b5ba0 con 0x7f99d81092f0 2024-09-21T00:28:16.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.929+0000 7f99d7200640 1 -- 172.21.15.27:0/2621893849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d81b5f40 con 0x7f99d81092f0 2024-09-21T00:28:16.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.936+0000 7f99d7200640 1 -- 172.21.15.27:0/2621893849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d8103dc0 con 0x7f99d81092f0 2024-09-21T00:28:16.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.942+0000 7f99cea00640 1 -- 172.21.15.27:0/2621893849 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f99c8042020 con 0x7f99d81092f0 2024-09-21T00:28:16.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.944+0000 7f99cea00640 1 --2- 172.21.15.27:0/2621893849 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f99bc0b6980 0x7f99bc0b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:16.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.945+0000 7f99cea00640 1 -- 172.21.15.27:0/2621893849 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9607+0+0 (secure 0 0 0) 0x7f99c81076d0 con 0x7f99d81092f0 2024-09-21T00:28:16.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.948+0000 7f99cea00640 1 -- 172.21.15.27:0/2621893849 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f99c80cd780 con 0x7f99d81092f0 2024-09-21T00:28:16.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.955+0000 7f99d5400640 1 --2- 172.21.15.27:0/2621893849 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f99bc0b6980 0x7f99bc0b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:16.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:16.957+0000 7f99d5400640 1 --2- 172.21.15.27:0/2621893849 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f99bc0b6980 0x7f99bc0b8e90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f99d8108d70 tx=0x7f99c4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:17.475+0000 7f99d7200640 1 -- 172.21.15.27:0/2621893849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99d81b6110 con 0x7f99d81092f0 2024-09-21T00:28:17.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:17.477+0000 7f99cea00640 1 -- 172.21.15.27:0/2621893849 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11902 (secure 0 0 0) 0x7f99c80d1b60 con 0x7f99d81092f0 2024-09-21T00:28:17.479 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:28:17.488 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":34,"btime":"2024-09-21T00:28:09:754006+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:28:09.743913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26057":{"gid":26057,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/4028763758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":4028763758},{"type":"v1","addr":"172.21.15.89:6837","nonce":4028763758}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:28:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 34 2024-09-21T00:28:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:17.483+0000 7f99d7200640 1 -- 172.21.15.27:0/2621893849 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f99bc0b6980 msgr2=0x7f99bc0b8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:17.483+0000 7f99d7200640 1 --2- 172.21.15.27:0/2621893849 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f99bc0b6980 0x7f99bc0b8e90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f99d8108d70 tx=0x7f99c4005f90 comp rx=0 tx=0).stop 2024-09-21T00:28:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:17.483+0000 7f99d7200640 1 -- 172.21.15.27:0/2621893849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d81092f0 msgr2=0x7f99d81a92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:17.483+0000 7f99d7200640 1 --2- 172.21.15.27:0/2621893849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d81092f0 0x7f99d81a92a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f99c8008000 tx=0x7f99c8003c10 comp rx=0 tx=0).stop 2024-09-21T00:28:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:17.484+0000 7f99d7200640 1 -- 172.21.15.27:0/2621893849 shutdown_connections 2024-09-21T00:28:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:17.484+0000 7f99d7200640 1 --2- 172.21.15.27:0/2621893849 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f99bc0b6980 0x7f99bc0b8e90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:17.484+0000 7f99d7200640 1 --2- 172.21.15.27:0/2621893849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d81aea20 0x7f99d81aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:17.484+0000 7f99d7200640 1 --2- 172.21.15.27:0/2621893849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99d8109c40 0x7f99d81a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:17.484+0000 7f99d7200640 1 --2- 172.21.15.27:0/2621893849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d81092f0 0x7f99d81a92a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:17.484+0000 7f99d7200640 1 -- 172.21.15.27:0/2621893849 >> 172.21.15.27:0/2621893849 conn(0x7f99d80ff440 msgr2=0x7f99d8107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:17.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:17.487+0000 7f99d7200640 1 -- 172.21.15.27:0/2621893849 shutdown_connections 2024-09-21T00:28:17.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:17.487+0000 7f99d7200640 1 -- 172.21.15.27:0/2621893849 wait complete. 2024-09-21T00:28:18.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:17 smithi089 ceph-mon[34481]: pgmap v586: 129 pgs: 17 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 57 active+clean; 925 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.4 MiB/s wr, 284 op/s 2024-09-21T00:28:18.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:17 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2621893849' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:28:18.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:17 smithi098 ceph-mon[34725]: pgmap v586: 129 pgs: 17 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 57 active+clean; 925 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.4 MiB/s wr, 284 op/s 2024-09-21T00:28:18.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:17 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2621893849' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:28:18.320 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:17 smithi027 ceph-mon[30745]: pgmap v586: 129 pgs: 17 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 57 active+clean; 925 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.4 MiB/s wr, 284 op/s 2024-09-21T00:28:18.320 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:17 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2621893849' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:28:18.375 INFO:tasks.workunit.client.0.smithi027.stdout: 234 350573 23 238119 39 113402 249 2024-09-21T00:28:19.184 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0' 2024-09-21T00:28:19.184 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0 2024-09-21T00:28:20.016 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:28:20.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:19 smithi027 ceph-mon[30745]: pgmap v587: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 930 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 1.7 MiB/s wr, 284 op/s 2024-09-21T00:28:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:19 smithi089 ceph-mon[34481]: pgmap v587: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 930 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 1.7 MiB/s wr, 284 op/s 2024-09-21T00:28:20.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:19 smithi098 ceph-mon[34725]: pgmap v587: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 930 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 1.7 MiB/s wr, 284 op/s 2024-09-21T00:28:21.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.508+0000 7f8bcca00640 1 -- 172.21.15.27:0/799234302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8bc8111d70 msgr2=0x7f8bc810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:21.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.508+0000 7f8bcca00640 1 --2- 172.21.15.27:0/799234302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8bc8111d70 0x7f8bc810a3c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009900 tx=0x7f8bb802a400 comp rx=0 tx=0).stop 2024-09-21T00:28:21.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.512+0000 7f8bcca00640 1 -- 172.21.15.27:0/799234302 shutdown_connections 2024-09-21T00:28:21.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.512+0000 7f8bcca00640 1 --2- 172.21.15.27:0/799234302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc811cba0 0x7f8bc811efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.512+0000 7f8bcca00640 1 --2- 172.21.15.27:0/799234302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bc810a900 0x7f8bc810ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.512+0000 7f8bcca00640 1 --2- 172.21.15.27:0/799234302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8bc8111d70 0x7f8bc810a3c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.512+0000 7f8bcca00640 1 -- 172.21.15.27:0/799234302 >> 172.21.15.27:0/799234302 conn(0x7f8bc806ea10 msgr2=0x7f8bc806ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:21.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.514+0000 7f8bcca00640 1 -- 172.21.15.27:0/799234302 shutdown_connections 2024-09-21T00:28:21.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.514+0000 7f8bcca00640 1 -- 172.21.15.27:0/799234302 wait complete. 2024-09-21T00:28:21.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.519+0000 7f8bcca00640 1 Processor -- start 2024-09-21T00:28:21.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.519+0000 7f8bcca00640 1 -- start start 2024-09-21T00:28:21.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.526+0000 7f8bcca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bc810a900 0x7f8bc811b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.526+0000 7f8bcca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8bc8111d70 0x7f8bc81146a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.526+0000 7f8bc7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bc810a900 0x7f8bc811b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.526+0000 7f8bcca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc811cba0 0x7f8bc8114be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.526+0000 7f8bcca00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81227b0 con 0x7f8bc811cba0 2024-09-21T00:28:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.526+0000 7f8bcca00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8122630 con 0x7f8bc8111d70 2024-09-21T00:28:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.526+0000 7f8bcca00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8122930 con 0x7f8bc810a900 2024-09-21T00:28:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.527+0000 7f8bc7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bc810a900 0x7f8bc811b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41868/0 (socket says 172.21.15.27:41868) 2024-09-21T00:28:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.527+0000 7f8bc7400640 1 -- 172.21.15.27:0/3208353206 learned_addr learned my addr 172.21.15.27:0/3208353206 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.527+0000 7f8bc6a00640 1 --2- 172.21.15.27:0/3208353206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8bc8111d70 0x7f8bc81146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.531+0000 7f8bc7400640 1 -- 172.21.15.27:0/3208353206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8bc8111d70 msgr2=0x7f8bc81146a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.531+0000 7f8bc7400640 1 --2- 172.21.15.27:0/3208353206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8bc8111d70 0x7f8bc81146a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.531+0000 7f8bc7400640 1 -- 172.21.15.27:0/3208353206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc811cba0 msgr2=0x7f8bc8114be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.531+0000 7f8bc7400640 1 --2- 172.21.15.27:0/3208353206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc811cba0 0x7f8bc8114be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.532+0000 7f8bc7400640 1 -- 172.21.15.27:0/3208353206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc8115370 con 0x7f8bc810a900 2024-09-21T00:28:21.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.532+0000 7f8bc7400640 1 --2- 172.21.15.27:0/3208353206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bc810a900 0x7f8bc811b3d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005ab0 tx=0x7f8bb8003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:21.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.532+0000 7f8bbfe00640 1 -- 172.21.15.27:0/3208353206 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb803f070 con 0x7f8bc810a900 2024-09-21T00:28:21.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.533+0000 7f8bcca00640 1 -- 172.21.15.27:0/3208353206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc811a040 con 0x7f8bc810a900 2024-09-21T00:28:21.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.533+0000 7f8bcca00640 1 -- 172.21.15.27:0/3208353206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc811a570 con 0x7f8bc810a900 2024-09-21T00:28:21.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.533+0000 7f8bbfe00640 1 -- 172.21.15.27:0/3208353206 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb8031d60 con 0x7f8bc810a900 2024-09-21T00:28:21.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.533+0000 7f8bbfe00640 1 -- 172.21.15.27:0/3208353206 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8035030 con 0x7f8bc810a900 2024-09-21T00:28:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.547+0000 7f8bd4c47740 1 Processor -- start 2024-09-21T00:28:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.548+0000 7f8bd4c47740 1 -- start start 2024-09-21T00:28:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.550+0000 7f8bd4c47740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f9720850 0x5617f9ac7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.551+0000 7f8bd4c47740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f96f7990 0x5617f9ac80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.551+0000 7f8bd4c47740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f96ffb70 0x5617f9ac8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.551+0000 7f8bd4c47740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5617f9a3da90 con 0x5617f96f7990 2024-09-21T00:28:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.551+0000 7f8bd4c47740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5617f9a2cec0 con 0x5617f96ffb70 2024-09-21T00:28:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.551+0000 7f8bd4c47740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5617f9a30cd0 con 0x5617f9720850 2024-09-21T00:28:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.551+0000 7f8bc7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f9720850 0x5617f9ac7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.552+0000 7f8bc7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f9720850 0x5617f9ac7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41880/0 (socket says 172.21.15.27:41880) 2024-09-21T00:28:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.552+0000 7f8bc7e00640 1 -- 172.21.15.27:0/3045766190 learned_addr learned my addr 172.21.15.27:0/3045766190 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.552+0000 7f8bbfe00640 1 -- 172.21.15.27:0/3208353206 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8bb8042070 con 0x7f8bc810a900 2024-09-21T00:28:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.552+0000 7f8bc7400640 1 --2- 172.21.15.27:0/3045766190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f96f7990 0x5617f9ac80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.553+0000 7f8bc6a00640 1 --2- 172.21.15.27:0/3045766190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f96ffb70 0x5617f9ac8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.554+0000 7f8bc7e00640 1 -- 172.21.15.27:0/3045766190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f96ffb70 msgr2=0x5617f9ac8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.554+0000 7f8bc7e00640 1 --2- 172.21.15.27:0/3045766190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f96ffb70 0x5617f9ac8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.554+0000 7f8bc7e00640 1 -- 172.21.15.27:0/3045766190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f96f7990 msgr2=0x5617f9ac80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.554+0000 7f8bc7e00640 1 --2- 172.21.15.27:0/3045766190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f96f7990 0x5617f9ac80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.554+0000 7f8bc7e00640 1 -- 172.21.15.27:0/3045766190 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617f99d29d0 con 0x5617f9720850 2024-09-21T00:28:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.554+0000 7f8bc7e00640 1 --2- 172.21.15.27:0/3045766190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f9720850 0x5617f9ac7b90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009de0 tx=0x7f8bc000d6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d37039080ba2fdaf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.555+0000 7f8bb3e00640 1 -- 172.21.15.27:0/3045766190 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0003970 con 0x5617f9720850 2024-09-21T00:28:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.555+0000 7f8bb3e00640 1 -- 172.21.15.27:0/3045766190 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bc0004020 con 0x5617f9720850 2024-09-21T00:28:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.555+0000 7f8bb3e00640 1 -- 172.21.15.27:0/3045766190 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0010820 con 0x5617f9720850 2024-09-21T00:28:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.557+0000 7f8bbfe00640 1 --2- 172.21.15.27:0/3208353206 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8bac0b6770 0x7f8bac0b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.557+0000 7f8bbfe00640 1 -- 172.21.15.27:0/3208353206 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9607+0+0 (secure 0 0 0) 0x7f8bb8108470 con 0x7f8bc810a900 2024-09-21T00:28:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.557+0000 7f8bc7400640 1 --2- 172.21.15.27:0/3208353206 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8bac0b6770 0x7f8bac0b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.558+0000 7f8bd4c47740 1 -- 172.21.15.27:0/3045766190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f9720850 msgr2=0x5617f9ac7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:21.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.558+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/3045766190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f9720850 0x5617f9ac7b90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009de0 tx=0x7f8bc000d6b0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.559+0000 7f8bc7400640 1 --2- 172.21.15.27:0/3208353206 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8bac0b6770 0x7f8bac0b8c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8bb810bfd0 tx=0x7f8bb810bea0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:21.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.559+0000 7f8bd4c47740 1 -- 172.21.15.27:0/3045766190 shutdown_connections 2024-09-21T00:28:21.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.559+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/3045766190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f96ffb70 0x5617f9ac8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.559+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/3045766190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f96f7990 0x5617f9ac80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.559+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/3045766190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f9720850 0x5617f9ac7b90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.559+0000 7f8bd4c47740 1 -- 172.21.15.27:0/3045766190 >> 172.21.15.27:0/3045766190 conn(0x5617f9714c40 msgr2=0x5617f9ac5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:21.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.560+0000 7f8bd4c47740 1 -- 172.21.15.27:0/3045766190 shutdown_connections 2024-09-21T00:28:21.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.560+0000 7f8bd4c47740 1 -- 172.21.15.27:0/3045766190 wait complete. 2024-09-21T00:28:21.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.561+0000 7f8bd4c47740 1 -- start start 2024-09-21T00:28:21.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.563+0000 7f8bd4c47740 1 Processor -- start 2024-09-21T00:28:21.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.564+0000 7f8bd4c47740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f96f7990 0x5617f9ad3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.566+0000 7f8bd4c47740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96ffb70 0x5617f9ad3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.566+0000 7f8bd4c47740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f9720850 0x5617f9adf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.566+0000 7f8bd4c47740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5617f988e7b0 con 0x5617f96f7990 2024-09-21T00:28:21.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.566+0000 7f8bd4c47740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5617f99d1650 con 0x5617f9720850 2024-09-21T00:28:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.566+0000 7f8bd4c47740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5617f9ad6370 con 0x5617f96ffb70 2024-09-21T00:28:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.566+0000 7f8bd4c47740 10 client.0 _ll_register_callbacks cb 0x5617f97380a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.566+0000 7f8bc6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96ffb70 0x5617f9ad3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.566+0000 7f8bc6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96ffb70 0x5617f9ad3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41882/0 (socket says 172.21.15.27:41882) 2024-09-21T00:28:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.566+0000 7f8bc6a00640 1 -- 172.21.15.27:0/539848185 learned_addr learned my addr 172.21.15.27:0/539848185 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.567+0000 7f8bc6a00640 1 -- 172.21.15.27:0/539848185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f9720850 msgr2=0x5617f9adf6c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:28:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.567+0000 7f8bc6a00640 1 --2- 172.21.15.27:0/539848185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f9720850 0x5617f9adf6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.567+0000 7f8bc6a00640 1 -- 172.21.15.27:0/539848185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f96f7990 msgr2=0x5617f9ad3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.567+0000 7f8bc6a00640 1 --2- 172.21.15.27:0/539848185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f96f7990 0x5617f9ad3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.567+0000 7f8bc6a00640 1 -- 172.21.15.27:0/539848185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617f979aba0 con 0x5617f96ffb70 2024-09-21T00:28:21.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.567+0000 7f8bc6a00640 1 --2- 172.21.15.27:0/539848185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96ffb70 0x5617f9ad3a30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400cd80 tx=0x7f8bb40039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3405be4bfa0a9ea1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:21.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.567+0000 7f8bb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:28:21.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.568+0000 7f8bb0c00640 1 -- 172.21.15.27:0/539848185 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4011910 con 0x5617f96ffb70 2024-09-21T00:28:21.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.568+0000 7f8bb0c00640 1 -- 172.21.15.27:0/539848185 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9607+0+0 (secure 0 0 0) 0x7f8bb4011ab0 con 0x5617f96ffb70 2024-09-21T00:28:21.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.568+0000 7f8bd4c47740 1 -- 172.21.15.27:0/539848185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc0004020 con 0x5617f96ffb70 2024-09-21T00:28:21.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.568+0000 7f8bb0c00640 1 -- 172.21.15.27:0/539848185 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8bb4011cb0 con 0x5617f96ffb70 2024-09-21T00:28:21.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.568+0000 7f8bd4c47740 10 client.26116 fetch_fsmap learned FSMap version 34 2024-09-21T00:28:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.568+0000 7f8bd4c47740 1 -- 172.21.15.27:0/539848185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5617f9ae2320 con 0x5617f96ffb70 2024-09-21T00:28:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.569+0000 7f8bb0c00640 1 -- 172.21.15.27:0/539848185 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bc0004020 con 0x5617f96ffb70 2024-09-21T00:28:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.570+0000 7f8bb0c00640 1 -- 172.21.15.27:0/539848185 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f8bb4018070 con 0x5617f96ffb70 2024-09-21T00:28:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.571+0000 7f8bd4c47740 10 client.26116 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T00:28:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.571+0000 7f8bd4c47740 10 client.26116 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:28:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.571+0000 7f8bd4c47740 20 client.26116 populate_metadata read hostname 'smithi027' 2024-09-21T00:28:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.571+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/539848185 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5617f96df1d0 0x5617f9ae58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.571+0000 7f8bd4c47740 4 client.26116 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:28:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.571+0000 7f8bd4c47740 1 -- 172.21.15.27:0/539848185 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617f9a3f9e0 con 0x5617f96df1d0 2024-09-21T00:28:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.571+0000 7f8bc7e00640 1 --2- 172.21.15.27:0/539848185 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5617f96df1d0 0x5617f9ae58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.572+0000 7f8bb0c00640 10 client.26116 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.572+0000 7f8bc7e00640 1 --2- 172.21.15.27:0/539848185 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5617f96df1d0 0x5617f9ae58a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6883aac396731a76 server_cookie=5621fe9d4b1f90fa in_seq=0 out_seq=0 2024-09-21T00:28:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.573+0000 7f8bb0c00640 1 -- 172.21.15.27:0/539848185 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5617f9a3f9e0 con 0x5617f96df1d0 2024-09-21T00:28:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.573+0000 7f8bb0c00640 10 client.26116 handle_command_reply: tid=0 2024-09-21T00:28:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.574+0000 7f8bd4c47740 1 client.26116 shutdown 2024-09-21T00:28:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.574+0000 7f8bd4c47740 1 -- 172.21.15.27:0/539848185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96ffb70 msgr2=0x5617f9ad3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.574+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/539848185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96ffb70 0x5617f9ad3a30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400cd80 tx=0x7f8bb40039c0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.574+0000 7f8bd4c47740 1 -- 172.21.15.27:0/539848185 shutdown_connections 2024-09-21T00:28:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.574+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/539848185 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5617f96df1d0 0x5617f9ae58a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.574+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/539848185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f9720850 0x5617f9adf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.574+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/539848185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96ffb70 0x5617f9ad3a30 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400cd80 tx=0x7f8bb40039c0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.574+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/539848185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f96f7990 0x5617f9ad3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.574+0000 7f8bd4c47740 1 -- 172.21.15.27:0/539848185 >> 172.21.15.27:0/539848185 conn(0x5617f9714c40 msgr2=0x5617f9acad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:21.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.577+0000 7f8bb0c00640 0 client.26116 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:21.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.578+0000 7f8bd4c47740 1 -- 172.21.15.27:0/539848185 shutdown_connections 2024-09-21T00:28:21.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.578+0000 7f8bd4c47740 1 -- 172.21.15.27:0/539848185 wait complete. 2024-09-21T00:28:21.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.578+0000 7f8bd4c47740 20 client.26116 trim_cache size 0 max 16384 2024-09-21T00:28:21.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.631+0000 7f8bd4c47740 1 Processor -- start 2024-09-21T00:28:21.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.633+0000 7f8bd4c47740 1 -- start start 2024-09-21T00:28:21.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.633+0000 7f8bd4c47740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f96f7990 0x5617f9ad2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.633+0000 7f8bd4c47740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f9720850 0x5617f9acdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.633+0000 7f8bd4c47740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96b4580 0x5617f9ac92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.633+0000 7f8bd4c47740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba8012bd0 con 0x5617f96f7990 2024-09-21T00:28:21.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.633+0000 7f8bd4c47740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba80103e0 con 0x5617f9720850 2024-09-21T00:28:21.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.633+0000 7f8bd4c47740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba800e250 con 0x5617f96b4580 2024-09-21T00:28:21.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.634+0000 7f8bc6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f9720850 0x5617f9acdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.634+0000 7f8bc6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f9720850 0x5617f9acdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58988/0 (socket says 172.21.15.27:58988) 2024-09-21T00:28:21.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.634+0000 7f8bc6a00640 1 -- 172.21.15.27:0/396971796 learned_addr learned my addr 172.21.15.27:0/396971796 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:21.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.634+0000 7f8bc7e00640 1 --2- 172.21.15.27:0/396971796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96b4580 0x5617f9ac92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.634+0000 7f8bc7e00640 1 --2- 172.21.15.27:0/396971796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f96f7990 0x5617f9ad2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.634+0000 7f8bc6a00640 1 -- 172.21.15.27:0/396971796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96b4580 msgr2=0x5617f9ac92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:21.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.635+0000 7f8bc6a00640 1 --2- 172.21.15.27:0/396971796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96b4580 0x5617f9ac92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.635+0000 7f8bc6a00640 1 -- 172.21.15.27:0/396971796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f96f7990 msgr2=0x5617f9ad2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:21.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.635+0000 7f8bc6a00640 1 --2- 172.21.15.27:0/396971796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f96f7990 0x5617f9ad2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.635+0000 7f8bc6a00640 1 -- 172.21.15.27:0/396971796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617f9ac9e80 con 0x5617f9720850 2024-09-21T00:28:21.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.635+0000 7f8bc6a00640 1 --2- 172.21.15.27:0/396971796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f9720850 0x5617f9acdcb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400cd50 tx=0x7f8bb4005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35bc48f761c47876 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:21.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.635+0000 7f8bb3400640 1 -- 172.21.15.27:0/396971796 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb401a950 con 0x5617f9720850 2024-09-21T00:28:21.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.636+0000 7f8bb3400640 1 -- 172.21.15.27:0/396971796 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb4029070 con 0x5617f9720850 2024-09-21T00:28:21.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.636+0000 7f8bb3400640 1 -- 172.21.15.27:0/396971796 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4024870 con 0x5617f9720850 2024-09-21T00:28:21.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.637+0000 7f8bd4c47740 1 -- 172.21.15.27:0/396971796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f9720850 msgr2=0x5617f9acdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:21.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.637+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/396971796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f9720850 0x5617f9acdcb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400cd50 tx=0x7f8bb4005ce0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.638+0000 7f8bd4c47740 1 -- 172.21.15.27:0/396971796 shutdown_connections 2024-09-21T00:28:21.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.638+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/396971796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96b4580 0x5617f9ac92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.638+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/396971796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f9720850 0x5617f9acdcb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.638+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/396971796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f96f7990 0x5617f9ad2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.638+0000 7f8bd4c47740 1 -- 172.21.15.27:0/396971796 >> 172.21.15.27:0/396971796 conn(0x5617f9714c40 msgr2=0x5617f9ac8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:21.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.638+0000 7f8bd4c47740 1 -- 172.21.15.27:0/396971796 shutdown_connections 2024-09-21T00:28:21.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.639+0000 7f8bd4c47740 1 -- 172.21.15.27:0/396971796 wait complete. 2024-09-21T00:28:21.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.639+0000 7f8bd4c47740 1 -- start start 2024-09-21T00:28:21.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.640+0000 7f8bd4c47740 1 Processor -- start 2024-09-21T00:28:21.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.642+0000 7f8bd4c47740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96b4580 0x5617f9adb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.642+0000 7f8bd4c47740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f96f7990 0x5617f9adbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bd4c47740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f9720850 0x5617f9ad65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bd4c47740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba800e8a0 con 0x5617f9720850 2024-09-21T00:28:21.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bd4c47740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba800fd10 con 0x5617f96f7990 2024-09-21T00:28:21.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bd4c47740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba800ef70 con 0x5617f96b4580 2024-09-21T00:28:21.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bc7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96b4580 0x5617f9adb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bc7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96b4580 0x5617f9adb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41896/0 (socket says 172.21.15.27:41896) 2024-09-21T00:28:21.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bc7e00640 1 -- 172.21.15.27:0/564504261 learned_addr learned my addr 172.21.15.27:0/564504261 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:21.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bc6a00640 1 --2- 172.21.15.27:0/564504261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f96f7990 0x5617f9adbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bc7e00640 1 -- 172.21.15.27:0/564504261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f96f7990 msgr2=0x5617f9adbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:21.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bc7e00640 1 --2- 172.21.15.27:0/564504261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f96f7990 0x5617f9adbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bc7e00640 1 -- 172.21.15.27:0/564504261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f9720850 msgr2=0x5617f9ad65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:21.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bc7e00640 1 --2- 172.21.15.27:0/564504261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f9720850 0x5617f9ad65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bc7e00640 1 -- 172.21.15.27:0/564504261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617f9adc390 con 0x5617f96b4580 2024-09-21T00:28:21.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bd4c47740 10 client.0 _ll_register_callbacks cb 0x7f8bb4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:21.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bc6a00640 1 --2- 172.21.15.27:0/564504261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f96f7990 0x5617f9adbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:28:21.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.643+0000 7f8bd4c47740 1 -- 172.21.15.27:0/564504261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb4029070 con 0x5617f96b4580 2024-09-21T00:28:21.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.644+0000 7f8bc7e00640 1 --2- 172.21.15.27:0/564504261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96b4580 0x5617f9adb630 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0006010 tx=0x7f8bc000df10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abc11317729748a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:21.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.644+0000 7f8bb0c00640 10 client.26122 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:28:21.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.644+0000 7f8bb0c00640 1 -- 172.21.15.27:0/564504261 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0004d20 con 0x5617f96b4580 2024-09-21T00:28:21.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.644+0000 7f8bb0c00640 1 -- 172.21.15.27:0/564504261 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9607+0+0 (secure 0 0 0) 0x7f8bc0005020 con 0x5617f96b4580 2024-09-21T00:28:21.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.645+0000 7f8bb0c00640 1 -- 172.21.15.27:0/564504261 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8bc00052a0 con 0x5617f96b4580 2024-09-21T00:28:21.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.645+0000 7f8bd4c47740 10 client.26122 fetch_fsmap learned FSMap version 34 2024-09-21T00:28:21.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.645+0000 7f8bd4c47740 1 -- 172.21.15.27:0/564504261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=34}) -- 0x5617f9b507b0 con 0x5617f96b4580 2024-09-21T00:28:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.649+0000 7f8bb0c00640 1 -- 172.21.15.27:0/564504261 <== mon.1 v2:172.21.15.98:3300/0 4 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f8bc0005440 con 0x5617f96b4580 2024-09-21T00:28:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.649+0000 7f8bd4c47740 10 client.26122 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T00:28:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.649+0000 7f8bd4c47740 10 client.26122 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:28:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.649+0000 7f8bd4c47740 20 client.26122 populate_metadata read hostname 'smithi027' 2024-09-21T00:28:21.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.649+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/564504261 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5617f9b50bb0 0x5617f9b52fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.649+0000 7f8bd4c47740 4 client.26122 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}] 2024-09-21T00:28:21.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.649+0000 7f8bd4c47740 1 -- 172.21.15.27:0/564504261 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}) -- 0x5617f9a3b490 con 0x5617f9b50bb0 2024-09-21T00:28:21.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.650+0000 7f8bc6a00640 1 --2- 172.21.15.27:0/564504261 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5617f9b50bb0 0x5617f9b52fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.653+0000 7f8bb0c00640 10 client.26122 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:21.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:21.653+0000 7f8bc6a00640 1 --2- 172.21.15.27:0/564504261 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5617f9b50bb0 0x5617f9b52fd0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15264b21b584752d server_cookie=aefd970fd99d9b50 in_seq=0 out_seq=0 2024-09-21T00:28:22.006 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:21 smithi027 ceph-mon[30745]: pgmap v588: 129 pgs: 11 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 87 active+clean; 934 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.9 MiB/s wr, 267 op/s 2024-09-21T00:28:22.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:21 smithi089 ceph-mon[34481]: pgmap v588: 129 pgs: 11 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 87 active+clean; 934 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.9 MiB/s wr, 267 op/s 2024-09-21T00:28:22.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:21 smithi098 ceph-mon[34725]: pgmap v588: 129 pgs: 11 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 87 active+clean; 934 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.9 MiB/s wr, 267 op/s 2024-09-21T00:28:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.936+0000 7fb2c0a00640 1 -- 172.21.15.27:0/2927206946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2bc103910 msgr2=0x7fb2bc103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.936+0000 7fb2c0a00640 1 --2- 172.21.15.27:0/2927206946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2bc103910 0x7fb2bc103d20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009900 tx=0x7fb2ac02a420 comp rx=0 tx=0).stop 2024-09-21T00:28:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.937+0000 7fb2c0a00640 1 -- 172.21.15.27:0/2927206946 shutdown_connections 2024-09-21T00:28:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.937+0000 7fb2c0a00640 1 --2- 172.21.15.27:0/2927206946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2bc109b90 0x7fb2bc1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.937+0000 7fb2c0a00640 1 --2- 172.21.15.27:0/2927206946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2bc109240 0x7fb2bc109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.937+0000 7fb2c0a00640 1 --2- 172.21.15.27:0/2927206946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2bc103910 0x7fb2bc103d20 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009900 tx=0x7fb2ac02a420 comp rx=0 tx=0).stop 2024-09-21T00:28:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.937+0000 7fb2c0a00640 1 -- 172.21.15.27:0/2927206946 >> 172.21.15.27:0/2927206946 conn(0x7fb2bc0ff390 msgr2=0x7fb2bc1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.938+0000 7fb2c0a00640 1 -- 172.21.15.27:0/2927206946 shutdown_connections 2024-09-21T00:28:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.938+0000 7fb2c0a00640 1 -- 172.21.15.27:0/2927206946 wait complete. 2024-09-21T00:28:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.940+0000 7fb2c0a00640 1 Processor -- start 2024-09-21T00:28:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.940+0000 7fb2c0a00640 1 -- start start 2024-09-21T00:28:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.941+0000 7fb2c0a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2bc109240 0x7fb2bc10a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.941+0000 7fb2c0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2bc109b90 0x7fb2bc10a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.941+0000 7fb2c0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2bc10ad20 0x7fb2bc1acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.941+0000 7fb2c0a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc113910 con 0x7fb2bc10ad20 2024-09-21T00:28:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.941+0000 7fb2c0a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc113790 con 0x7fb2bc109b90 2024-09-21T00:28:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.941+0000 7fb2c0a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc113a90 con 0x7fb2bc109240 2024-09-21T00:28:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.941+0000 7fb2bb400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2bc109240 0x7fb2bc10a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.941+0000 7fb2baa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2bc109b90 0x7fb2bc10a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.941+0000 7fb2bb400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2bc109240 0x7fb2bc10a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41902/0 (socket says 172.21.15.27:41902) 2024-09-21T00:28:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.941+0000 7fb2bb400640 1 -- 172.21.15.27:0/2689101980 learned_addr learned my addr 172.21.15.27:0/2689101980 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.942+0000 7fb2bbe00640 1 --2- 172.21.15.27:0/2689101980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2bc10ad20 0x7fb2bc1acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.942+0000 7fb2baa00640 1 -- 172.21.15.27:0/2689101980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2bc109240 msgr2=0x7fb2bc10a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.942+0000 7fb2baa00640 1 --2- 172.21.15.27:0/2689101980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2bc109240 0x7fb2bc10a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.942+0000 7fb2baa00640 1 -- 172.21.15.27:0/2689101980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2bc10ad20 msgr2=0x7fb2bc1acc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.942+0000 7fb2baa00640 1 --2- 172.21.15.27:0/2689101980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2bc10ad20 0x7fb2bc1acc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.942+0000 7fb2baa00640 1 -- 172.21.15.27:0/2689101980 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc19c3d0 con 0x7fb2bc109b90 2024-09-21T00:28:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.942+0000 7fb2baa00640 1 --2- 172.21.15.27:0/2689101980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2bc109b90 0x7fb2bc10a7e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800b3c0 tx=0x7fb2a800b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.943+0000 7fb2b3e00640 1 -- 172.21.15.27:0/2689101980 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8011830 con 0x7fb2bc109b90 2024-09-21T00:28:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.943+0000 7fb2b3e00640 1 -- 172.21.15.27:0/2689101980 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2a8016070 con 0x7fb2bc109b90 2024-09-21T00:28:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.943+0000 7fb2b3e00640 1 -- 172.21.15.27:0/2689101980 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8010e50 con 0x7fb2bc109b90 2024-09-21T00:28:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.945+0000 7fb2c0a00640 1 -- 172.21.15.27:0/2689101980 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2bc19c6c0 con 0x7fb2bc109b90 2024-09-21T00:28:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.945+0000 7fb2c0a00640 1 -- 172.21.15.27:0/2689101980 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2bc19cc10 con 0x7fb2bc109b90 2024-09-21T00:28:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.952+0000 7fb2c9066740 1 Processor -- start 2024-09-21T00:28:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.952+0000 7fb2c9066740 1 -- start start 2024-09-21T00:28:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.953+0000 7fb2c9066740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526791b0 0x562652a1de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.953+0000 7fb2c9066740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526502f0 0x562652a1e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.953+0000 7fb2c9066740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a1e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.953+0000 7fb2c9066740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5626529936b0 con 0x5626526502f0 2024-09-21T00:28:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.953+0000 7fb2c9066740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562652982ae0 con 0x5626526584d0 2024-09-21T00:28:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.953+0000 7fb2c9066740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5626529868f0 con 0x5626526791b0 2024-09-21T00:28:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.953+0000 7fb2bbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526791b0 0x562652a1de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.953+0000 7fb2bbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526791b0 0x562652a1de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41914/0 (socket says 172.21.15.27:41914) 2024-09-21T00:28:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.953+0000 7fb2bbe00640 1 -- 172.21.15.27:0/2086954952 learned_addr learned my addr 172.21.15.27:0/2086954952 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.954+0000 7fb2b3e00640 1 -- 172.21.15.27:0/2689101980 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb2a801a410 con 0x7fb2bc109b90 2024-09-21T00:28:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.954+0000 7fb2baa00640 1 --2- 172.21.15.27:0/2086954952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a1e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.954+0000 7fb2bb400640 1 --2- 172.21.15.27:0/2086954952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526502f0 0x562652a1e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.955+0000 7fb2bbe00640 1 -- 172.21.15.27:0/2086954952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 msgr2=0x562652a1e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.955+0000 7fb2bbe00640 1 --2- 172.21.15.27:0/2086954952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a1e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.955+0000 7fb2bbe00640 1 -- 172.21.15.27:0/2086954952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526502f0 msgr2=0x562652a1e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.955+0000 7fb2bbe00640 1 --2- 172.21.15.27:0/2086954952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526502f0 0x562652a1e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.955+0000 7fb2bbe00640 1 -- 172.21.15.27:0/2086954952 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56265294f780 con 0x5626526791b0 2024-09-21T00:28:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.955+0000 7fb2baa00640 1 --2- 172.21.15.27:0/2086954952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a1e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.956+0000 7fb2bbe00640 1 --2- 172.21.15.27:0/2086954952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526791b0 0x562652a1de10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4007dc0 tx=0x7fb2b400c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c038e64b2e4e4857 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.956+0000 7fb2b3e00640 1 --2- 172.21.15.27:0/2689101980 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb2a00b6890 0x7fb2a00b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.957+0000 7fb2a7e00640 1 -- 172.21.15.27:0/2086954952 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4015020 con 0x5626526791b0 2024-09-21T00:28:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.957+0000 7fb2b3e00640 1 -- 172.21.15.27:0/2689101980 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9607+0+0 (secure 0 0 0) 0x7fb2a80e6810 con 0x7fb2bc109b90 2024-09-21T00:28:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.957+0000 7fb2a7e00640 1 -- 172.21.15.27:0/2086954952 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2b4011ae0 con 0x5626526791b0 2024-09-21T00:28:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.957+0000 7fb2a7e00640 1 -- 172.21.15.27:0/2086954952 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4010900 con 0x5626526791b0 2024-09-21T00:28:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.957+0000 7fb2bb400640 1 --2- 172.21.15.27:0/2689101980 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb2a00b6890 0x7fb2a00b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.959+0000 7fb2bb400640 1 --2- 172.21.15.27:0/2689101980 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb2a00b6890 0x7fb2a00b8da0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac031440 tx=0x7fb2ac0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.959+0000 7fb2c9066740 1 -- 172.21.15.27:0/2086954952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526791b0 msgr2=0x562652a1de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.959+0000 7fb2c9066740 1 --2- 172.21.15.27:0/2086954952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526791b0 0x562652a1de10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4007dc0 tx=0x7fb2b400c560 comp rx=0 tx=0).stop 2024-09-21T00:28:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.960+0000 7fb2c9066740 1 -- 172.21.15.27:0/2086954952 shutdown_connections 2024-09-21T00:28:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.960+0000 7fb2c9066740 1 --2- 172.21.15.27:0/2086954952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a1e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.960+0000 7fb2c9066740 1 --2- 172.21.15.27:0/2086954952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526502f0 0x562652a1e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.960+0000 7fb2c9066740 1 --2- 172.21.15.27:0/2086954952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526791b0 0x562652a1de10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.960+0000 7fb2c9066740 1 -- 172.21.15.27:0/2086954952 >> 172.21.15.27:0/2086954952 conn(0x56265266d300 msgr2=0x562652a1bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.960+0000 7fb2c9066740 1 -- 172.21.15.27:0/2086954952 shutdown_connections 2024-09-21T00:28:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.960+0000 7fb2c9066740 1 -- 172.21.15.27:0/2086954952 wait complete. 2024-09-21T00:28:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.961+0000 7fb2c9066740 1 -- start start 2024-09-21T00:28:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.961+0000 7fb2c9066740 1 Processor -- start 2024-09-21T00:28:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.964+0000 7fb2c9066740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526502f0 0x562652a293c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.964+0000 7fb2c9066740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a29900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.964+0000 7fb2c9066740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526791b0 0x562652a29e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.965+0000 7fb2baa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a29900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.965+0000 7fb2baa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a29900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:59042/0 (socket says 172.21.15.27:59042) 2024-09-21T00:28:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.965+0000 7fb2c9066740 1 -- 172.21.15.27:0/861719610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5626528baab0 con 0x5626526502f0 2024-09-21T00:28:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.965+0000 7fb2c9066740 1 -- 172.21.15.27:0/861719610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562652929570 con 0x5626526584d0 2024-09-21T00:28:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.965+0000 7fb2c9066740 1 -- 172.21.15.27:0/861719610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562652a2c590 con 0x5626526791b0 2024-09-21T00:28:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.965+0000 7fb2c9066740 10 client.0 _ll_register_callbacks cb 0x562652938100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.965+0000 7fb2bb400640 1 --2- 172.21.15.27:0/861719610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526502f0 0x562652a293c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.965+0000 7fb2bbe00640 1 --2- 172.21.15.27:0/861719610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526791b0 0x562652a29e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.965+0000 7fb2baa00640 1 -- 172.21.15.27:0/861719610 learned_addr learned my addr 172.21.15.27:0/861719610 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.966+0000 7fb2baa00640 1 -- 172.21.15.27:0/861719610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526791b0 msgr2=0x562652a29e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.966+0000 7fb2baa00640 1 --2- 172.21.15.27:0/861719610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526791b0 0x562652a29e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.966+0000 7fb2baa00640 1 -- 172.21.15.27:0/861719610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526502f0 msgr2=0x562652a293c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.966+0000 7fb2baa00640 1 --2- 172.21.15.27:0/861719610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526502f0 0x562652a293c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.966+0000 7fb2baa00640 1 -- 172.21.15.27:0/861719610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626526e4790 con 0x5626526584d0 2024-09-21T00:28:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.967+0000 7fb2baa00640 1 --2- 172.21.15.27:0/861719610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a29900 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80abfb0 tx=0x7fb2a80106d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d2b123db8cac88f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.967+0000 7fb2a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:28:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.967+0000 7fb2a4c00640 1 -- 172.21.15.27:0/861719610 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80abc00 con 0x5626526584d0 2024-09-21T00:28:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.967+0000 7fb2a4c00640 1 -- 172.21.15.27:0/861719610 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9607+0+0 (secure 0 0 0) 0x7fb2a80b30b0 con 0x5626526584d0 2024-09-21T00:28:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.967+0000 7fb2c9066740 1 -- 172.21.15.27:0/861719610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2b4011ae0 con 0x5626526584d0 2024-09-21T00:28:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.967+0000 7fb2a4c00640 1 -- 172.21.15.27:0/861719610 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb2a80aba20 con 0x5626526584d0 2024-09-21T00:28:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.968+0000 7fb2c9066740 10 client.26147 fetch_fsmap learned FSMap version 34 2024-09-21T00:28:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.968+0000 7fb2c9066740 1 -- 172.21.15.27:0/861719610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56265275d620 con 0x5626526584d0 2024-09-21T00:28:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.968+0000 7fb2a4c00640 1 -- 172.21.15.27:0/861719610 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2b4011ae0 con 0x5626526584d0 2024-09-21T00:28:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.968+0000 7fb2a4c00640 1 -- 172.21.15.27:0/861719610 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fb2a80cfba0 con 0x5626526584d0 2024-09-21T00:28:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.970+0000 7fb2c9066740 10 client.26147 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T00:28:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.970+0000 7fb2c9066740 10 client.26147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:28:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.970+0000 7fb2c9066740 20 client.26147 populate_metadata read hostname 'smithi027' 2024-09-21T00:28:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.970+0000 7fb2c9066740 1 --2- 172.21.15.27:0/861719610 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562652637d80 0x562652a3b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.970+0000 7fb2c9066740 4 client.26147 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:28:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.970+0000 7fb2c9066740 1 -- 172.21.15.27:0/861719610 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562652995600 con 0x562652637d80 2024-09-21T00:28:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.970+0000 7fb2bb400640 1 --2- 172.21.15.27:0/861719610 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562652637d80 0x562652a3b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.971+0000 7fb2bb400640 1 --2- 172.21.15.27:0/861719610 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562652637d80 0x562652a3b5a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53166215f8aaf82f server_cookie=bb169cd8a6cf2ed3 in_seq=0 out_seq=0 2024-09-21T00:28:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.972+0000 7fb2a4c00640 10 client.26147 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.972+0000 7fb2a4c00640 1 -- 172.21.15.27:0/861719610 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562652995600 con 0x562652637d80 2024-09-21T00:28:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.972+0000 7fb2a4c00640 10 client.26147 handle_command_reply: tid=0 2024-09-21T00:28:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.973+0000 7fb2c9066740 1 client.26147 shutdown 2024-09-21T00:28:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.973+0000 7fb2c9066740 1 -- 172.21.15.27:0/861719610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 msgr2=0x562652a29900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.973+0000 7fb2c9066740 1 --2- 172.21.15.27:0/861719610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a29900 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80abfb0 tx=0x7fb2a80106d0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.973+0000 7fb2c9066740 1 -- 172.21.15.27:0/861719610 shutdown_connections 2024-09-21T00:28:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.973+0000 7fb2c9066740 1 --2- 172.21.15.27:0/861719610 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562652637d80 0x562652a3b5a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.973+0000 7fb2c9066740 1 --2- 172.21.15.27:0/861719610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526791b0 0x562652a29e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.973+0000 7fb2c9066740 1 --2- 172.21.15.27:0/861719610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a29900 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.973+0000 7fb2c9066740 1 --2- 172.21.15.27:0/861719610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526502f0 0x562652a293c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.973+0000 7fb2c9066740 1 -- 172.21.15.27:0/861719610 >> 172.21.15.27:0/861719610 conn(0x56265266d300 msgr2=0x562652a1ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.973+0000 7fb2a4c00640 0 client.26147 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.974+0000 7fb2c9066740 1 -- 172.21.15.27:0/861719610 shutdown_connections 2024-09-21T00:28:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.974+0000 7fb2c9066740 1 -- 172.21.15.27:0/861719610 wait complete. 2024-09-21T00:28:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:22.974+0000 7fb2c9066740 20 client.26147 trim_cache size 0 max 16384 2024-09-21T00:28:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.016+0000 7fb2c9066740 1 Processor -- start 2024-09-21T00:28:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.016+0000 7fb2c9066740 1 -- start start 2024-09-21T00:28:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.017+0000 7fb2c9066740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526502f0 0x562652a1bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.017+0000 7fb2c9066740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a26590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.017+0000 7fb2c9066740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526791b0 0x562652a26ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.017+0000 7fb2c9066740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc1c2e20 con 0x5626526791b0 2024-09-21T00:28:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.017+0000 7fb2c9066740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc1c0630 con 0x5626526584d0 2024-09-21T00:28:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.017+0000 7fb2c9066740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc104bc0 con 0x5626526502f0 2024-09-21T00:28:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.017+0000 7fb2bb400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526502f0 0x562652a1bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.018+0000 7fb2bb400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526502f0 0x562652a1bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:53932/0 (socket says 172.21.15.27:53932) 2024-09-21T00:28:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.018+0000 7fb2bb400640 1 -- 172.21.15.27:0/1714249268 learned_addr learned my addr 172.21.15.27:0/1714249268 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.017+0000 7fb2bbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526791b0 0x562652a26ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.017+0000 7fb2baa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a26590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.019+0000 7fb2bb400640 1 -- 172.21.15.27:0/1714249268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 msgr2=0x562652a26590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.019+0000 7fb2bb400640 1 --2- 172.21.15.27:0/1714249268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a26590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.019+0000 7fb2bb400640 1 -- 172.21.15.27:0/1714249268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526791b0 msgr2=0x562652a26ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.019+0000 7fb2bb400640 1 --2- 172.21.15.27:0/1714249268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526791b0 0x562652a26ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.019+0000 7fb2bb400640 1 -- 172.21.15.27:0/1714249268 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562652a24ee0 con 0x5626526502f0 2024-09-21T00:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.019+0000 7fb2baa00640 1 --2- 172.21.15.27:0/1714249268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a26590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.020+0000 7fb2bb400640 1 --2- 172.21.15.27:0/1714249268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526502f0 0x562652a1bd30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac006010 tx=0x7fb2ac041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37c66b44283fcbd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.020+0000 7fb2a7400640 1 -- 172.21.15.27:0/1714249268 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac035040 con 0x5626526502f0 2024-09-21T00:28:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.020+0000 7fb2a7400640 1 -- 172.21.15.27:0/1714249268 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2ac056070 con 0x5626526502f0 2024-09-21T00:28:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.020+0000 7fb2a7400640 1 -- 172.21.15.27:0/1714249268 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac03c8c0 con 0x5626526502f0 2024-09-21T00:28:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.022+0000 7fb2c9066740 1 -- 172.21.15.27:0/1714249268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526502f0 msgr2=0x562652a1bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.022+0000 7fb2c9066740 1 --2- 172.21.15.27:0/1714249268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526502f0 0x562652a1bd30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac006010 tx=0x7fb2ac041000 comp rx=0 tx=0).stop 2024-09-21T00:28:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.022+0000 7fb2c9066740 1 -- 172.21.15.27:0/1714249268 shutdown_connections 2024-09-21T00:28:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.022+0000 7fb2c9066740 1 --2- 172.21.15.27:0/1714249268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526791b0 0x562652a26ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.022+0000 7fb2c9066740 1 --2- 172.21.15.27:0/1714249268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a26590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.022+0000 7fb2c9066740 1 --2- 172.21.15.27:0/1714249268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526502f0 0x562652a1bd30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.022+0000 7fb2c9066740 1 -- 172.21.15.27:0/1714249268 >> 172.21.15.27:0/1714249268 conn(0x56265266d300 msgr2=0x562652a1b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.023+0000 7fb2c9066740 1 -- 172.21.15.27:0/1714249268 shutdown_connections 2024-09-21T00:28:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.023+0000 7fb2c9066740 1 -- 172.21.15.27:0/1714249268 wait complete. 2024-09-21T00:28:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.024+0000 7fb2c9066740 1 -- start start 2024-09-21T00:28:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.024+0000 7fb2c9066740 1 Processor -- start 2024-09-21T00:28:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.026+0000 7fb2c9066740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526502f0 0x562652a2ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.026+0000 7fb2c9066740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a2f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.026+0000 7fb2c9066740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526791b0 0x562652aa3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.026+0000 7fb2bb400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526502f0 0x562652a2ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.027+0000 7fb2c9066740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc1beb10 con 0x5626526791b0 2024-09-21T00:28:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.027+0000 7fb2baa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a2f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.027+0000 7fb2baa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a2f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:56786/0 (socket says 172.21.15.27:56786) 2024-09-21T00:28:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.027+0000 7fb2baa00640 1 -- 172.21.15.27:0/2769077048 learned_addr learned my addr 172.21.15.27:0/2769077048 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.028+0000 7fb2bbe00640 1 --2- 172.21.15.27:0/2769077048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526791b0 0x562652aa3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.028+0000 7fb2c9066740 1 -- 172.21.15.27:0/2769077048 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc1bff60 con 0x5626526584d0 2024-09-21T00:28:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.028+0000 7fb2c9066740 1 -- 172.21.15.27:0/2769077048 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc1bf1c0 con 0x5626526502f0 2024-09-21T00:28:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.028+0000 7fb2c9066740 10 client.0 _ll_register_callbacks cb 0x7fb2a80a8900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.029+0000 7fb2bbe00640 1 -- 172.21.15.27:0/2769077048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526502f0 msgr2=0x562652a2ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.030+0000 7fb2bbe00640 1 --2- 172.21.15.27:0/2769077048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526502f0 0x562652a2ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.030+0000 7fb2bbe00640 1 -- 172.21.15.27:0/2769077048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 msgr2=0x562652a2f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.030+0000 7fb2bbe00640 1 --2- 172.21.15.27:0/2769077048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a2f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.030+0000 7fb2bbe00640 1 -- 172.21.15.27:0/2769077048 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562652aa44e0 con 0x5626526791b0 2024-09-21T00:28:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.030+0000 7fb2baa00640 1 --2- 172.21.15.27:0/2769077048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a2f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:28:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.030+0000 7fb2bb400640 1 --2- 172.21.15.27:0/2769077048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526502f0 0x562652a2ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:28:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.032+0000 7fb2bbe00640 1 --2- 172.21.15.27:0/2769077048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526791b0 0x562652aa3fa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4004b90 tx=0x7fb2b4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09b82e500f2bdb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.032+0000 7fb2a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:28:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.033+0000 7fb2a4c00640 1 -- 172.21.15.27:0/2769077048 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b402a060 con 0x5626526791b0 2024-09-21T00:28:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.033+0000 7fb2a4c00640 1 -- 172.21.15.27:0/2769077048 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9607+0+0 (secure 0 0 0) 0x7fb2b401a420 con 0x5626526791b0 2024-09-21T00:28:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.033+0000 7fb2c9066740 1 -- 172.21.15.27:0/2769077048 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2ac056070 con 0x5626526791b0 2024-09-21T00:28:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.035+0000 7fb2a4c00640 1 -- 172.21.15.27:0/2769077048 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb2b4031050 con 0x5626526791b0 2024-09-21T00:28:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.035+0000 7fb2c9066740 10 client.16092 fetch_fsmap learned FSMap version 34 2024-09-21T00:28:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.035+0000 7fb2c9066740 1 -- 172.21.15.27:0/2769077048 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562652aa55c0 con 0x5626526791b0 2024-09-21T00:28:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.036+0000 7fb2a4c00640 1 -- 172.21.15.27:0/2769077048 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2ac056070 con 0x5626526791b0 2024-09-21T00:28:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.036+0000 7fb2a4c00640 1 -- 172.21.15.27:0/2769077048 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fb2b4019540 con 0x5626526791b0 2024-09-21T00:28:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.036+0000 7fb2c9066740 10 client.16092 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T00:28:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.036+0000 7fb2c9066740 10 client.16092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:28:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.036+0000 7fb2c9066740 20 client.16092 populate_metadata read hostname 'smithi027' 2024-09-21T00:28:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.037+0000 7fb2c9066740 1 --2- 172.21.15.27:0/2769077048 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56265260d1c0 0x562652aa7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.037+0000 7fb2c9066740 4 client.16092 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:28:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.037+0000 7fb2c9066740 1 -- 172.21.15.27:0/2769077048 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562652a2a280 con 0x56265260d1c0 2024-09-21T00:28:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.037+0000 7fb2bb400640 1 --2- 172.21.15.27:0/2769077048 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56265260d1c0 0x562652aa7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.040+0000 7fb2bb400640 1 --2- 172.21.15.27:0/2769077048 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56265260d1c0 0x562652aa7bb0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34346163e4fb0bc1 server_cookie=16e7dff2c6a06e64 in_seq=0 out_seq=0 2024-09-21T00:28:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:23.040+0000 7fb2a4c00640 10 client.16092 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:24.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:23 smithi089 ceph-mon[34481]: pgmap v589: 129 pgs: 11 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 87 active+clean; 938 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 210 op/s 2024-09-21T00:28:24.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:23 smithi098 ceph-mon[34725]: pgmap v589: 129 pgs: 11 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 87 active+clean; 938 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 210 op/s 2024-09-21T00:28:24.298 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:23 smithi027 ceph-mon[30745]: pgmap v589: 129 pgs: 11 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 87 active+clean; 938 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 210 op/s 2024-09-21T00:28:26.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:25 smithi027 ceph-mon[30745]: pgmap v590: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 942 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 248 op/s 2024-09-21T00:28:26.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:25 smithi089 ceph-mon[34481]: pgmap v590: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 942 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 248 op/s 2024-09-21T00:28:26.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:25 smithi098 ceph-mon[34725]: pgmap v590: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 942 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 248 op/s 2024-09-21T00:28:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.764+0000 7fb2a4c00640 1 -- 172.21.15.27:0/2769077048 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562652a2a280 con 0x56265260d1c0 2024-09-21T00:28:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.764+0000 7fb2a4c00640 10 client.16092 handle_command_reply: tid=0 2024-09-21T00:28:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.764+0000 7fb2c9066740 1 client.16092 shutdown 2024-09-21T00:28:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.765+0000 7fb2c9066740 1 -- 172.21.15.27:0/2769077048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526791b0 msgr2=0x562652aa3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.765+0000 7fb2c9066740 1 --2- 172.21.15.27:0/2769077048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526791b0 0x562652aa3fa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4004b90 tx=0x7fb2b4011bf0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.766+0000 7fb2c9066740 1 -- 172.21.15.27:0/2769077048 shutdown_connections 2024-09-21T00:28:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.766+0000 7fb2c9066740 1 --2- 172.21.15.27:0/2769077048 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56265260d1c0 0x562652aa7bb0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.766+0000 7fb2c9066740 1 --2- 172.21.15.27:0/2769077048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626526791b0 0x562652aa3fa0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.766+0000 7fb2a4c00640 0 client.16092 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.766+0000 7fb2c9066740 1 --2- 172.21.15.27:0/2769077048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5626526584d0 0x562652a2f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.766+0000 7fb2c9066740 1 --2- 172.21.15.27:0/2769077048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626526502f0 0x562652a2ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.766+0000 7fb2c9066740 1 -- 172.21.15.27:0/2769077048 >> 172.21.15.27:0/2769077048 conn(0x56265266d300 msgr2=0x562652a1a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.769+0000 7fb2c9066740 1 -- 172.21.15.27:0/2769077048 shutdown_connections 2024-09-21T00:28:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.769+0000 7fb2c9066740 1 -- 172.21.15.27:0/2769077048 wait complete. 2024-09-21T00:28:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.769+0000 7fb2c9066740 20 client.16092 trim_cache size 0 max 16384 2024-09-21T00:28:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:28:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:28:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:28:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:28:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.772+0000 7fb2c0a00640 1 -- 172.21.15.27:0/2689101980 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb2a00b6890 msgr2=0x7fb2a00b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.772+0000 7fb2c0a00640 1 --2- 172.21.15.27:0/2689101980 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb2a00b6890 0x7fb2a00b8da0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac031440 tx=0x7fb2ac0058d0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.772+0000 7fb2c0a00640 1 -- 172.21.15.27:0/2689101980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2bc109b90 msgr2=0x7fb2bc10a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.772+0000 7fb2c0a00640 1 --2- 172.21.15.27:0/2689101980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2bc109b90 0x7fb2bc10a7e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800b3c0 tx=0x7fb2a800b880 comp rx=0 tx=0).stop 2024-09-21T00:28:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.772+0000 7fb2c0a00640 1 -- 172.21.15.27:0/2689101980 shutdown_connections 2024-09-21T00:28:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.772+0000 7fb2c0a00640 1 --2- 172.21.15.27:0/2689101980 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb2a00b6890 0x7fb2a00b8da0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.772+0000 7fb2c0a00640 1 --2- 172.21.15.27:0/2689101980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2bc10ad20 0x7fb2bc1acc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.772+0000 7fb2c0a00640 1 --2- 172.21.15.27:0/2689101980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2bc109b90 0x7fb2bc10a7e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.772+0000 7fb2c0a00640 1 --2- 172.21.15.27:0/2689101980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2bc109240 0x7fb2bc10a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.772+0000 7fb2c0a00640 1 -- 172.21.15.27:0/2689101980 >> 172.21.15.27:0/2689101980 conn(0x7fb2bc0ff390 msgr2=0x7fb2bc10e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.772+0000 7fb2c0a00640 1 -- 172.21.15.27:0/2689101980 shutdown_connections 2024-09-21T00:28:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:26.773+0000 7fb2c0a00640 1 -- 172.21.15.27:0/2689101980 wait complete. 2024-09-21T00:28:28.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:27 smithi089 ceph-mon[34481]: pgmap v591: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 952 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 460 op/s 2024-09-21T00:28:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:27 smithi098 ceph-mon[34725]: pgmap v591: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 952 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 460 op/s 2024-09-21T00:28:28.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:27 smithi027 ceph-mon[30745]: pgmap v591: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 952 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 460 op/s 2024-09-21T00:28:28.375 INFO:tasks.workunit.client.0.smithi027.stdout: 241 96859 344 66822 203 27240 120 2024-09-21T00:28:30.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:29 smithi089 ceph-mon[34481]: pgmap v592: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 957 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 595 op/s 2024-09-21T00:28:30.231 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:29 smithi027 ceph-mon[30745]: pgmap v592: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 957 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 595 op/s 2024-09-21T00:28:30.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:29 smithi098 ceph-mon[34725]: pgmap v592: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 957 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 595 op/s 2024-09-21T00:28:32.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:32 smithi098 ceph-mon[34725]: pgmap v593: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 963 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 678 op/s 2024-09-21T00:28:32.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:32 smithi098 ceph-mon[34725]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T00:28:32.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:32 smithi027 ceph-mon[30745]: pgmap v593: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 963 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 678 op/s 2024-09-21T00:28:32.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:32 smithi027 ceph-mon[30745]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T00:28:32.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:32 smithi089 ceph-mon[34481]: pgmap v593: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 963 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 678 op/s 2024-09-21T00:28:32.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:32 smithi089 ceph-mon[34481]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T00:28:33.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.083+0000 7f8bb0c00640 1 -- 172.21.15.27:0/564504261 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5617f9a3b490 con 0x5617f9b50bb0 2024-09-21T00:28:33.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.083+0000 7f8bb0c00640 10 client.26122 handle_command_reply: tid=0 2024-09-21T00:28:33.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.084+0000 7f8bd4c47740 1 client.26122 shutdown 2024-09-21T00:28:33.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.085+0000 7f8bd4c47740 1 -- 172.21.15.27:0/564504261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96b4580 msgr2=0x5617f9adb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:33.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.085+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/564504261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96b4580 0x5617f9adb630 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0006010 tx=0x7f8bc000df10 comp rx=0 tx=0).stop 2024-09-21T00:28:33.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.086+0000 7f8bd4c47740 1 -- 172.21.15.27:0/564504261 shutdown_connections 2024-09-21T00:28:33.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.086+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/564504261 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5617f9b50bb0 0x5617f9b52fd0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.086+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/564504261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617f9720850 0x5617f9ad65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.086+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/564504261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5617f96f7990 0x5617f9adbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.086+0000 7f8bd4c47740 1 --2- 172.21.15.27:0/564504261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5617f96b4580 0x5617f9adb630 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.086+0000 7f8bd4c47740 1 -- 172.21.15.27:0/564504261 >> 172.21.15.27:0/564504261 conn(0x5617f9714c40 msgr2=0x5617f9ad2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:33.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.086+0000 7f8bb0c00640 0 client.26122 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:33.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.086+0000 7f8bd4c47740 1 -- 172.21.15.27:0/564504261 shutdown_connections 2024-09-21T00:28:33.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.087+0000 7f8bd4c47740 1 -- 172.21.15.27:0/564504261 wait complete. 2024-09-21T00:28:33.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.087+0000 7f8bd4c47740 20 client.26122 trim_cache size 0 max 16384 2024-09-21T00:28:33.090 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:28:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 34, 2024-09-21T00:28:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24761, 2024-09-21T00:28:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 22, 2024-09-21T00:28:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T00:28:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: "3b281b3b": { 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 22, 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 139.2, 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.088+0000 7f8bcca00640 1 -- 172.21.15.27:0/3208353206 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8bac0b6770 msgr2=0x7f8bac0b8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.088+0000 7f8bcca00640 1 --2- 172.21.15.27:0/3208353206 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8bac0b6770 0x7f8bac0b8c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8bb810bfd0 tx=0x7f8bb810bea0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.088+0000 7f8bcca00640 1 -- 172.21.15.27:0/3208353206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bc810a900 msgr2=0x7f8bc811b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.088+0000 7f8bcca00640 1 --2- 172.21.15.27:0/3208353206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bc810a900 0x7f8bc811b3d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005ab0 tx=0x7f8bb8003cd0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.088+0000 7f8bcca00640 1 -- 172.21.15.27:0/3208353206 shutdown_connections 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.088+0000 7f8bcca00640 1 --2- 172.21.15.27:0/3208353206 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8bac0b6770 0x7f8bac0b8c80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.088+0000 7f8bcca00640 1 --2- 172.21.15.27:0/3208353206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc811cba0 0x7f8bc8114be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.088+0000 7f8bcca00640 1 --2- 172.21.15.27:0/3208353206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8bc8111d70 0x7f8bc81146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.088+0000 7f8bcca00640 1 --2- 172.21.15.27:0/3208353206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bc810a900 0x7f8bc811b3d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.088+0000 7f8bcca00640 1 -- 172.21.15.27:0/3208353206 >> 172.21.15.27:0/3208353206 conn(0x7f8bc806ea10 msgr2=0x7f8bc8071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.089+0000 7f8bcca00640 1 -- 172.21.15.27:0/3208353206 shutdown_connections 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:33.089+0000 7f8bcca00640 1 -- 172.21.15.27:0/3208353206 wait complete. 2024-09-21T00:28:34.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:34 smithi098 ceph-mon[34725]: pgmap v595: 129 pgs: 129 active+clean; 967 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 824 op/s 2024-09-21T00:28:34.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:34 smithi027 ceph-mon[30745]: pgmap v595: 129 pgs: 129 active+clean; 967 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 824 op/s 2024-09-21T00:28:34.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:34 smithi089 ceph-mon[34481]: pgmap v595: 129 pgs: 129 active+clean; 967 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 824 op/s 2024-09-21T00:28:36.259 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3b281b3b', quiesce took 29.8 seconds. Will release after: -10.55 2024-09-21T00:28:36.259 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:28:36.351 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:36 smithi027 ceph-mon[30745]: pgmap v596: 129 pgs: 129 active+clean; 968 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 685 op/s 2024-09-21T00:28:36.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:36 smithi089 ceph-mon[34481]: pgmap v596: 129 pgs: 129 active+clean; 968 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 685 op/s 2024-09-21T00:28:36.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:36 smithi098 ceph-mon[34725]: pgmap v596: 129 pgs: 129 active+clean; 968 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 685 op/s 2024-09-21T00:28:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:28:37.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:37 smithi027 ceph-mon[30745]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T00:28:37.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:37 smithi089 ceph-mon[34481]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T00:28:37.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:37 smithi098 ceph-mon[34725]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T00:28:38.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.149+0000 7f8ec4600640 1 -- 172.21.15.27:0/3175987000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ebc101b20 msgr2=0x7f8ebc101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:38.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.149+0000 7f8ec4600640 1 --2- 172.21.15.27:0/3175987000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ebc101b20 0x7f8ebc101f30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009900 tx=0x7f8eb002a490 comp rx=0 tx=0).stop 2024-09-21T00:28:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.149+0000 7f8ec4600640 1 -- 172.21.15.27:0/3175987000 shutdown_connections 2024-09-21T00:28:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.149+0000 7f8ec4600640 1 --2- 172.21.15.27:0/3175987000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ebc110070 0x7f8ebc112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.149+0000 7f8ec4600640 1 --2- 172.21.15.27:0/3175987000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ebc102470 0x7f8ebc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.149+0000 7f8ec4600640 1 --2- 172.21.15.27:0/3175987000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ebc101b20 0x7f8ebc101f30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.149+0000 7f8ec4600640 1 -- 172.21.15.27:0/3175987000 >> 172.21.15.27:0/3175987000 conn(0x7f8ebc077800 msgr2=0x7f8ebc077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:38.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.149+0000 7f8ec4600640 1 -- 172.21.15.27:0/3175987000 shutdown_connections 2024-09-21T00:28:38.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.149+0000 7f8ec4600640 1 -- 172.21.15.27:0/3175987000 wait complete. 2024-09-21T00:28:38.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.150+0000 7f8ec4600640 1 Processor -- start 2024-09-21T00:28:38.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.151+0000 7f8ec4600640 1 -- start start 2024-09-21T00:28:38.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.151+0000 7f8ec4600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ebc101b20 0x7f8ebc19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:38.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.151+0000 7f8ec4600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ebc102470 0x7f8ebc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:38.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.151+0000 7f8ec4600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ebc110070 0x7f8ebc1a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:38.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.151+0000 7f8ec4600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc115cc0 con 0x7f8ebc110070 2024-09-21T00:28:38.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.151+0000 7f8ec4600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc115b40 con 0x7f8ebc101b20 2024-09-21T00:28:38.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.151+0000 7f8ec4600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc115e40 con 0x7f8ebc102470 2024-09-21T00:28:38.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.152+0000 7f8ec1c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ebc101b20 0x7f8ebc19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:38.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.152+0000 7f8ec1200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ebc102470 0x7f8ebc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:38.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.152+0000 7f8ec1c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ebc101b20 0x7f8ebc19c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57920/0 (socket says 172.21.15.27:57920) 2024-09-21T00:28:38.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.152+0000 7f8ec1200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ebc102470 0x7f8ebc19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38838/0 (socket says 172.21.15.27:38838) 2024-09-21T00:28:38.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.152+0000 7f8ec1c00640 1 -- 172.21.15.27:0/2909782640 learned_addr learned my addr 172.21.15.27:0/2909782640 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:38.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.152+0000 7f8ec2600640 1 --2- 172.21.15.27:0/2909782640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ebc110070 0x7f8ebc1a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:38.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.152+0000 7f8ec1c00640 1 -- 172.21.15.27:0/2909782640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ebc102470 msgr2=0x7f8ebc19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:38.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.152+0000 7f8ec1c00640 1 --2- 172.21.15.27:0/2909782640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ebc102470 0x7f8ebc19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:38.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.152+0000 7f8ec1c00640 1 -- 172.21.15.27:0/2909782640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ebc110070 msgr2=0x7f8ebc1a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:38.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.152+0000 7f8ec1c00640 1 --2- 172.21.15.27:0/2909782640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ebc110070 0x7f8ebc1a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:38.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.152+0000 7f8ec1c00640 1 -- 172.21.15.27:0/2909782640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ebc1a2680 con 0x7f8ebc101b20 2024-09-21T00:28:38.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.152+0000 7f8ec2600640 1 --2- 172.21.15.27:0/2909782640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ebc110070 0x7f8ebc1a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:38.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.152+0000 7f8ec1c00640 1 --2- 172.21.15.27:0/2909782640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ebc101b20 0x7f8ebc19c730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0005ab0 tx=0x7f8eb0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:38.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.153+0000 7f8eba000640 1 -- 172.21.15.27:0/2909782640 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb003f070 con 0x7f8ebc101b20 2024-09-21T00:28:38.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.153+0000 7f8eba000640 1 -- 172.21.15.27:0/2909782640 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eb0031b60 con 0x7f8ebc101b20 2024-09-21T00:28:38.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.153+0000 7f8eba000640 1 -- 172.21.15.27:0/2909782640 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0035030 con 0x7f8ebc101b20 2024-09-21T00:28:38.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.153+0000 7f8ec4600640 1 -- 172.21.15.27:0/2909782640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ebc1a2910 con 0x7f8ebc101b20 2024-09-21T00:28:38.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.153+0000 7f8ec4600640 1 -- 172.21.15.27:0/2909782640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ebc1a9030 con 0x7f8ebc101b20 2024-09-21T00:28:38.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.155+0000 7f8ec4600640 1 -- 172.21.15.27:0/2909782640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ebc101f30 con 0x7f8ebc101b20 2024-09-21T00:28:38.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.158+0000 7f8eba000640 1 -- 172.21.15.27:0/2909782640 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8eb0042070 con 0x7f8ebc101b20 2024-09-21T00:28:38.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.161+0000 7f8eba000640 1 --2- 172.21.15.27:0/2909782640 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8ea40b6680 0x7f8ea40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:38.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.161+0000 7f8eba000640 1 -- 172.21.15.27:0/2909782640 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7f8eb0108700 con 0x7f8ebc101b20 2024-09-21T00:28:38.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.161+0000 7f8ec1200640 1 --2- 172.21.15.27:0/2909782640 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8ea40b6680 0x7f8ea40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:38.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.162+0000 7f8ec1200640 1 --2- 172.21.15.27:0/2909782640 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8ea40b6680 0x7f8ea40b8b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc104450 tx=0x7f8ea8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:38.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.164+0000 7f8eba000640 1 -- 172.21.15.27:0/2909782640 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8eb0049030 con 0x7f8ebc101b20 2024-09-21T00:28:38.375 INFO:tasks.workunit.client.0.smithi027.stdout: 241 28117 0 19672 0 8576 1 2024-09-21T00:28:38.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.437+0000 7f8ec4600640 1 -- 172.21.15.27:0/2909782640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ebc10b020 con 0x7f8ebc101b20 2024-09-21T00:28:38.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.437+0000 7f8eba000640 1 -- 172.21.15.27:0/2909782640 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11902 (secure 0 0 0) 0x7f8eb00ce820 con 0x7f8ebc101b20 2024-09-21T00:28:38.440 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:28:38.440 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":34,"btime":"2024-09-21T00:28:09:754006+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:28:09.743913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26057":{"gid":26057,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/4028763758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":4028763758},{"type":"v1","addr":"172.21.15.89:6837","nonce":4028763758}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:28:38.441 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 34 2024-09-21T00:28:38.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.440+0000 7f8ec4600640 1 -- 172.21.15.27:0/2909782640 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8ea40b6680 msgr2=0x7f8ea40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:38.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.440+0000 7f8ec4600640 1 --2- 172.21.15.27:0/2909782640 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8ea40b6680 0x7f8ea40b8b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc104450 tx=0x7f8ea8005f90 comp rx=0 tx=0).stop 2024-09-21T00:28:38.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.440+0000 7f8ec4600640 1 -- 172.21.15.27:0/2909782640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ebc101b20 msgr2=0x7f8ebc19c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:38.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.440+0000 7f8ec4600640 1 --2- 172.21.15.27:0/2909782640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ebc101b20 0x7f8ebc19c730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0005ab0 tx=0x7f8eb0003c10 comp rx=0 tx=0).stop 2024-09-21T00:28:38.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.440+0000 7f8ec4600640 1 -- 172.21.15.27:0/2909782640 shutdown_connections 2024-09-21T00:28:38.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.440+0000 7f8ec4600640 1 --2- 172.21.15.27:0/2909782640 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8ea40b6680 0x7f8ea40b8b90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:38.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.440+0000 7f8ec4600640 1 --2- 172.21.15.27:0/2909782640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ebc110070 0x7f8ebc1a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:38.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.440+0000 7f8ec4600640 1 --2- 172.21.15.27:0/2909782640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ebc102470 0x7f8ebc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:38.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.440+0000 7f8ec4600640 1 --2- 172.21.15.27:0/2909782640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ebc101b20 0x7f8ebc19c730 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:38.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.440+0000 7f8ec4600640 1 -- 172.21.15.27:0/2909782640 >> 172.21.15.27:0/2909782640 conn(0x7f8ebc077800 msgr2=0x7f8ebc110790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:38.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.440+0000 7f8ec4600640 1 -- 172.21.15.27:0/2909782640 shutdown_connections 2024-09-21T00:28:38.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:38.440+0000 7f8ec4600640 1 -- 172.21.15.27:0/2909782640 wait complete. 2024-09-21T00:28:38.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:38 smithi027 ceph-mon[30745]: pgmap v598: 129 pgs: 129 active+clean; 972 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 247 op/s 2024-09-21T00:28:38.646 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:38 smithi098 ceph-mon[34725]: pgmap v598: 129 pgs: 129 active+clean; 972 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 247 op/s 2024-09-21T00:28:38.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:38 smithi089 ceph-mon[34481]: pgmap v598: 129 pgs: 129 active+clean; 972 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 247 op/s 2024-09-21T00:28:39.253 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id 3b281b3b --release --await' 2024-09-21T00:28:39.254 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id 3b281b3b --release --await 2024-09-21T00:28:39.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:39 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2909782640' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:28:39.686 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:28:39.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:39 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2909782640' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:28:39.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:39 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2909782640' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:28:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.002+0000 7f6d09600640 1 -- 172.21.15.27:0/104567047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d04103ab0 msgr2=0x7f6d04103ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.002+0000 7f6d09600640 1 --2- 172.21.15.27:0/104567047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d04103ab0 0x7f6d04103ec0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009900 tx=0x7f6cf402a410 comp rx=0 tx=0).stop 2024-09-21T00:28:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.002+0000 7f6d09600640 1 -- 172.21.15.27:0/104567047 shutdown_connections 2024-09-21T00:28:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.002+0000 7f6d09600640 1 --2- 172.21.15.27:0/104567047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d04109d30 0x7f6d04110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.002+0000 7f6d09600640 1 --2- 172.21.15.27:0/104567047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d041093e0 0x7f6d041097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.002+0000 7f6d09600640 1 --2- 172.21.15.27:0/104567047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d04103ab0 0x7f6d04103ec0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.002+0000 7f6d09600640 1 -- 172.21.15.27:0/104567047 >> 172.21.15.27:0/104567047 conn(0x7f6d040ff4f0 msgr2=0x7f6d04101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.002+0000 7f6d09600640 1 -- 172.21.15.27:0/104567047 shutdown_connections 2024-09-21T00:28:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.002+0000 7f6d09600640 1 -- 172.21.15.27:0/104567047 wait complete. 2024-09-21T00:28:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.003+0000 7f6d09600640 1 Processor -- start 2024-09-21T00:28:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.003+0000 7f6d09600640 1 -- start start 2024-09-21T00:28:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d09600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d04103ab0 0x7f6d04075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d09600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d041093e0 0x7f6d04075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d09600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d04109d30 0x7f6d04076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d09600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041139f0 con 0x7f6d041093e0 2024-09-21T00:28:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d09600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04113870 con 0x7f6d04103ab0 2024-09-21T00:28:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d09600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04113b70 con 0x7f6d04109d30 2024-09-21T00:28:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d02a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d04103ab0 0x7f6d04075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d02000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d041093e0 0x7f6d04075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d02a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d04103ab0 0x7f6d04075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57940/0 (socket says 172.21.15.27:57940) 2024-09-21T00:28:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d02a00640 1 -- 172.21.15.27:0/1419365734 learned_addr learned my addr 172.21.15.27:0/1419365734 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d03400640 1 --2- 172.21.15.27:0/1419365734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d04109d30 0x7f6d04076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d02a00640 1 -- 172.21.15.27:0/1419365734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d04109d30 msgr2=0x7f6d04076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d02a00640 1 --2- 172.21.15.27:0/1419365734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d04109d30 0x7f6d04076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d02a00640 1 -- 172.21.15.27:0/1419365734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d041093e0 msgr2=0x7f6d04075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d02a00640 1 --2- 172.21.15.27:0/1419365734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d041093e0 0x7f6d04075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.004+0000 7f6d02a00640 1 -- 172.21.15.27:0/1419365734 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d041a86e0 con 0x7f6d04103ab0 2024-09-21T00:28:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.005+0000 7f6d02a00640 1 --2- 172.21.15.27:0/1419365734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d04103ab0 0x7f6d04075710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4005940 tx=0x7f6cf40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.005+0000 7f6cfb400640 1 -- 172.21.15.27:0/1419365734 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf403f030 con 0x7f6d04103ab0 2024-09-21T00:28:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.005+0000 7f6d09600640 1 -- 172.21.15.27:0/1419365734 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d041a8970 con 0x7f6d04103ab0 2024-09-21T00:28:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.005+0000 7f6cfb400640 1 -- 172.21.15.27:0/1419365734 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf40044c0 con 0x7f6d04103ab0 2024-09-21T00:28:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.005+0000 7f6cfb400640 1 -- 172.21.15.27:0/1419365734 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4035070 con 0x7f6d04103ab0 2024-09-21T00:28:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.005+0000 7f6d09600640 1 -- 172.21.15.27:0/1419365734 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d041a8e60 con 0x7f6d04103ab0 2024-09-21T00:28:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.008+0000 7f6d107d3740 1 Processor -- start 2024-09-21T00:28:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.008+0000 7f6d107d3740 1 -- start start 2024-09-21T00:28:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.009+0000 7f6d107d3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5e51b0 0x562d8c989c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.009+0000 7f6d107d3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 0x562d8c98a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.009+0000 7f6d107d3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 0x562d8c98a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.009+0000 7f6d107d3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562d8c8ff6b0 con 0x562d8c5e51b0 2024-09-21T00:28:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.009+0000 7f6d107d3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562d8c8eeae0 con 0x562d8c5bc2f0 2024-09-21T00:28:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.009+0000 7f6d107d3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562d8c8f28f0 con 0x562d8c5c44d0 2024-09-21T00:28:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.009+0000 7f6d03400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5e51b0 0x562d8c989c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.009+0000 7f6d02000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 0x562d8c98a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.009+0000 7f6d03400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5e51b0 0x562d8c989c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57448/0 (socket says 172.21.15.27:57448) 2024-09-21T00:28:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.009+0000 7f6d03400640 1 -- 172.21.15.27:0/2845230595 learned_addr learned my addr 172.21.15.27:0/2845230595 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.009+0000 7f6cfb400640 1 -- 172.21.15.27:0/1419365734 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6cf4042030 con 0x7f6d04103ab0 2024-09-21T00:28:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.010+0000 7f6d02000640 1 -- 172.21.15.27:0/2845230595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 msgr2=0x562d8c98a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.010+0000 7f6d02000640 1 --2- 172.21.15.27:0/2845230595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 0x562d8c98a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.010+0000 7f6d02000640 1 -- 172.21.15.27:0/2845230595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5e51b0 msgr2=0x562d8c989c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.010+0000 7f6d02000640 1 --2- 172.21.15.27:0/2845230595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5e51b0 0x562d8c989c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.010+0000 7f6d02000640 1 -- 172.21.15.27:0/2845230595 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d8c8bb780 con 0x562d8c5c44d0 2024-09-21T00:28:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.010+0000 7f6d02000640 1 --2- 172.21.15.27:0/2845230595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 0x562d8c98a6b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6d0406ca30 tx=0x7f6cf0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ee4e1c3adb20701 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.010+0000 7f6cef400640 1 -- 172.21.15.27:0/2845230595 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0004050 con 0x562d8c5c44d0 2024-09-21T00:28:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.011+0000 7f6cef400640 1 -- 172.21.15.27:0/2845230595 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf00054d0 con 0x562d8c5c44d0 2024-09-21T00:28:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.011+0000 7f6cef400640 1 -- 172.21.15.27:0/2845230595 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0004ba0 con 0x562d8c5c44d0 2024-09-21T00:28:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.011+0000 7f6cfb400640 1 --2- 172.21.15.27:0/1419365734 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6ce80b66b0 0x7f6ce80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.012+0000 7f6d107d3740 1 -- 172.21.15.27:0/2845230595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 msgr2=0x562d8c98a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.012+0000 7f6d107d3740 1 --2- 172.21.15.27:0/2845230595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 0x562d8c98a6b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6d0406ca30 tx=0x7f6cf0011810 comp rx=0 tx=0).stop 2024-09-21T00:28:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.012+0000 7f6cfb400640 1 -- 172.21.15.27:0/1419365734 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7f6cf4107e70 con 0x7f6d04103ab0 2024-09-21T00:28:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.012+0000 7f6d03400640 1 --2- 172.21.15.27:0/1419365734 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6ce80b66b0 0x7f6ce80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.012+0000 7f6d107d3740 1 -- 172.21.15.27:0/2845230595 shutdown_connections 2024-09-21T00:28:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.012+0000 7f6d107d3740 1 --2- 172.21.15.27:0/2845230595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 0x562d8c98a6b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.012+0000 7f6d107d3740 1 --2- 172.21.15.27:0/2845230595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 0x562d8c98a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.012+0000 7f6d107d3740 1 --2- 172.21.15.27:0/2845230595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5e51b0 0x562d8c989c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.012+0000 7f6d107d3740 1 -- 172.21.15.27:0/2845230595 >> 172.21.15.27:0/2845230595 conn(0x562d8c5d9300 msgr2=0x562d8c987b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.012+0000 7f6d107d3740 1 -- 172.21.15.27:0/2845230595 shutdown_connections 2024-09-21T00:28:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.012+0000 7f6d107d3740 1 -- 172.21.15.27:0/2845230595 wait complete. 2024-09-21T00:28:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.013+0000 7f6d03400640 1 --2- 172.21.15.27:0/1419365734 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6ce80b66b0 0x7f6ce80b8bc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc005fd0 tx=0x7f6cfc009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.013+0000 7f6d107d3740 1 -- start start 2024-09-21T00:28:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.013+0000 7f6d107d3740 1 Processor -- start 2024-09-21T00:28:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.014+0000 7f6d107d3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5bc2f0 0x562d8c995230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.014+0000 7f6d107d3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 0x562d8c995770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.014+0000 7f6d107d3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5e51b0 0x562d8c995cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.014+0000 7f6d107d3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562d8c826ab0 con 0x562d8c5bc2f0 2024-09-21T00:28:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.014+0000 7f6d107d3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562d8c895570 con 0x562d8c5e51b0 2024-09-21T00:28:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.014+0000 7f6d107d3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562d8c9983b0 con 0x562d8c5c44d0 2024-09-21T00:28:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.014+0000 7f6d107d3740 10 client.0 _ll_register_callbacks cb 0x562d8c8a4100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.014+0000 7f6d02a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5bc2f0 0x562d8c995230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.014+0000 7f6d03400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5e51b0 0x562d8c995cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.014+0000 7f6d02a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5bc2f0 0x562d8c995230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57460/0 (socket says 172.21.15.27:57460) 2024-09-21T00:28:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.014+0000 7f6d02a00640 1 -- 172.21.15.27:0/4027963328 learned_addr learned my addr 172.21.15.27:0/4027963328 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.014+0000 7f6d02000640 1 --2- 172.21.15.27:0/4027963328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 0x562d8c995770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.014+0000 7f6d03400640 1 -- 172.21.15.27:0/4027963328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 msgr2=0x562d8c995770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.014+0000 7f6d03400640 1 --2- 172.21.15.27:0/4027963328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 0x562d8c995770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.015+0000 7f6d03400640 1 -- 172.21.15.27:0/4027963328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5bc2f0 msgr2=0x562d8c995230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.015+0000 7f6d03400640 1 --2- 172.21.15.27:0/4027963328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5bc2f0 0x562d8c995230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.015+0000 7f6d03400640 1 -- 172.21.15.27:0/4027963328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d8c650790 con 0x562d8c5e51b0 2024-09-21T00:28:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.015+0000 7f6d02a00640 1 --2- 172.21.15.27:0/4027963328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5bc2f0 0x562d8c995230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:28:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.015+0000 7f6d02000640 1 --2- 172.21.15.27:0/4027963328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 0x562d8c995770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:28:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.015+0000 7f6d03400640 1 --2- 172.21.15.27:0/4027963328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5e51b0 0x562d8c995cd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc003f20 tx=0x7f6cfc004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cb2080d8596f424 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.015+0000 7f6ce3e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:28:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.015+0000 7f6ce3e00640 1 -- 172.21.15.27:0/4027963328 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc00f8f0 con 0x562d8c5e51b0 2024-09-21T00:28:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.015+0000 7f6d107d3740 1 -- 172.21.15.27:0/4027963328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cf00054d0 con 0x562d8c5e51b0 2024-09-21T00:28:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.015+0000 7f6ce3e00640 1 -- 172.21.15.27:0/4027963328 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7f6cfc00fa90 con 0x562d8c5e51b0 2024-09-21T00:28:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.015+0000 7f6ce3e00640 1 -- 172.21.15.27:0/4027963328 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6cfc00fc90 con 0x562d8c5e51b0 2024-09-21T00:28:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.015+0000 7f6d107d3740 10 client.26168 fetch_fsmap learned FSMap version 34 2024-09-21T00:28:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.015+0000 7f6d107d3740 1 -- 172.21.15.27:0/4027963328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562d8c6c9620 con 0x562d8c5e51b0 2024-09-21T00:28:40.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.016+0000 7f6ce3e00640 1 -- 172.21.15.27:0/4027963328 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf00054d0 con 0x562d8c5e51b0 2024-09-21T00:28:40.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.016+0000 7f6ce3e00640 1 -- 172.21.15.27:0/4027963328 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f6cfc026070 con 0x562d8c5e51b0 2024-09-21T00:28:40.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.016+0000 7f6d107d3740 10 client.26168 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T00:28:40.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.016+0000 7f6d107d3740 10 client.26168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:28:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.016+0000 7f6d107d3740 20 client.26168 populate_metadata read hostname 'smithi027' 2024-09-21T00:28:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.016+0000 7f6d107d3740 1 --2- 172.21.15.27:0/4027963328 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562d8c5a3d80 0x562d8c9a7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.016+0000 7f6d107d3740 4 client.26168 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:28:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.016+0000 7f6d107d3740 1 -- 172.21.15.27:0/4027963328 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d8c901600 con 0x562d8c5a3d80 2024-09-21T00:28:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.016+0000 7f6d02a00640 1 --2- 172.21.15.27:0/4027963328 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562d8c5a3d80 0x562d8c9a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.017+0000 7f6d02a00640 1 --2- 172.21.15.27:0/4027963328 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562d8c5a3d80 0x562d8c9a7590 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42785d38ce80269c server_cookie=79fbcc1b9c3d1977 in_seq=0 out_seq=0 2024-09-21T00:28:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.017+0000 7f6ce3e00640 10 client.26168 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.019+0000 7f6ce3e00640 1 -- 172.21.15.27:0/4027963328 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562d8c901600 con 0x562d8c5a3d80 2024-09-21T00:28:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.019+0000 7f6ce3e00640 10 client.26168 handle_command_reply: tid=0 2024-09-21T00:28:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.019+0000 7f6d107d3740 1 client.26168 shutdown 2024-09-21T00:28:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.019+0000 7f6d107d3740 1 -- 172.21.15.27:0/4027963328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5e51b0 msgr2=0x562d8c995cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.019+0000 7f6d107d3740 1 --2- 172.21.15.27:0/4027963328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5e51b0 0x562d8c995cd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc003f20 tx=0x7f6cfc004ab0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.019+0000 7f6d107d3740 1 -- 172.21.15.27:0/4027963328 shutdown_connections 2024-09-21T00:28:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.019+0000 7f6d107d3740 1 --2- 172.21.15.27:0/4027963328 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562d8c5a3d80 0x562d8c9a7590 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.019+0000 7f6d107d3740 1 --2- 172.21.15.27:0/4027963328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5e51b0 0x562d8c995cd0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.019+0000 7f6d107d3740 1 --2- 172.21.15.27:0/4027963328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 0x562d8c995770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.019+0000 7f6d107d3740 1 --2- 172.21.15.27:0/4027963328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5bc2f0 0x562d8c995230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.019+0000 7f6d107d3740 1 -- 172.21.15.27:0/4027963328 >> 172.21.15.27:0/4027963328 conn(0x562d8c5d9300 msgr2=0x562d8c98ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.019+0000 7f6ce3e00640 0 client.26168 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.019+0000 7f6d107d3740 1 -- 172.21.15.27:0/4027963328 shutdown_connections 2024-09-21T00:28:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.019+0000 7f6d107d3740 1 -- 172.21.15.27:0/4027963328 wait complete. 2024-09-21T00:28:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.019+0000 7f6d107d3740 20 client.26168 trim_cache size 0 max 16384 2024-09-21T00:28:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.036+0000 7f6d107d3740 1 Processor -- start 2024-09-21T00:28:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.036+0000 7f6d107d3740 1 -- start start 2024-09-21T00:28:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.036+0000 7f6d107d3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 0x562d8c986aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.036+0000 7f6d107d3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5c44d0 0x562d8c996e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.036+0000 7f6d107d3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5e51b0 0x562d8c9acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.036+0000 7f6d107d3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041b5a10 con 0x562d8c5c44d0 2024-09-21T00:28:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.036+0000 7f6d107d3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041b3220 con 0x562d8c5bc2f0 2024-09-21T00:28:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.036+0000 7f6d107d3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041b1090 con 0x562d8c5e51b0 2024-09-21T00:28:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.036+0000 7f6d02a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 0x562d8c986aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.036+0000 7f6d02a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 0x562d8c986aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57960/0 (socket says 172.21.15.27:57960) 2024-09-21T00:28:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.036+0000 7f6d02a00640 1 -- 172.21.15.27:0/4147993006 learned_addr learned my addr 172.21.15.27:0/4147993006 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.037+0000 7f6d03400640 1 --2- 172.21.15.27:0/4147993006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5e51b0 0x562d8c9acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.037+0000 7f6d02000640 1 --2- 172.21.15.27:0/4147993006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5c44d0 0x562d8c996e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.037+0000 7f6d02a00640 1 -- 172.21.15.27:0/4147993006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5e51b0 msgr2=0x562d8c9acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.037+0000 7f6d02a00640 1 --2- 172.21.15.27:0/4147993006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5e51b0 0x562d8c9acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.037+0000 7f6d02a00640 1 -- 172.21.15.27:0/4147993006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5c44d0 msgr2=0x562d8c996e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.037+0000 7f6d02a00640 1 --2- 172.21.15.27:0/4147993006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5c44d0 0x562d8c996e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.037+0000 7f6d02a00640 1 -- 172.21.15.27:0/4147993006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d8c9888b0 con 0x562d8c5bc2f0 2024-09-21T00:28:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.037+0000 7f6d02000640 1 --2- 172.21.15.27:0/4147993006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5c44d0 0x562d8c996e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.037+0000 7f6d02a00640 1 --2- 172.21.15.27:0/4147993006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 0x562d8c986aa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40cd640 tx=0x7f6cf403b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6827ddb281fa202e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.037+0000 7f6ceea00640 1 -- 172.21.15.27:0/4147993006 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4049040 con 0x562d8c5bc2f0 2024-09-21T00:28:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.037+0000 7f6ceea00640 1 -- 172.21.15.27:0/4147993006 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf40d72b0 con 0x562d8c5bc2f0 2024-09-21T00:28:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.037+0000 7f6ceea00640 1 -- 172.21.15.27:0/4147993006 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40d6e20 con 0x562d8c5bc2f0 2024-09-21T00:28:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.038+0000 7f6d107d3740 1 -- 172.21.15.27:0/4147993006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 msgr2=0x562d8c986aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.038+0000 7f6d107d3740 1 --2- 172.21.15.27:0/4147993006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 0x562d8c986aa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40cd640 tx=0x7f6cf403b460 comp rx=0 tx=0).stop 2024-09-21T00:28:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.038+0000 7f6d107d3740 1 -- 172.21.15.27:0/4147993006 shutdown_connections 2024-09-21T00:28:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.038+0000 7f6d107d3740 1 --2- 172.21.15.27:0/4147993006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5e51b0 0x562d8c9acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.038+0000 7f6d107d3740 1 --2- 172.21.15.27:0/4147993006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5c44d0 0x562d8c996e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.038+0000 7f6d107d3740 1 --2- 172.21.15.27:0/4147993006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 0x562d8c986aa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.038+0000 7f6d107d3740 1 -- 172.21.15.27:0/4147993006 >> 172.21.15.27:0/4147993006 conn(0x562d8c5d9300 msgr2=0x562d8c996750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.038+0000 7f6d107d3740 1 -- 172.21.15.27:0/4147993006 shutdown_connections 2024-09-21T00:28:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.039+0000 7f6d107d3740 1 -- 172.21.15.27:0/4147993006 wait complete. 2024-09-21T00:28:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.039+0000 7f6d107d3740 1 -- start start 2024-09-21T00:28:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.039+0000 7f6d107d3740 1 Processor -- start 2024-09-21T00:28:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d107d3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 0x562d8c99b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d107d3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 0x562d8c99bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d107d3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5e51b0 0x562d8c99c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d107d3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041b16e0 con 0x562d8c5e51b0 2024-09-21T00:28:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d107d3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041b2b50 con 0x562d8c5bc2f0 2024-09-21T00:28:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d107d3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041b1db0 con 0x562d8c5c44d0 2024-09-21T00:28:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d107d3740 10 client.0 _ll_register_callbacks cb 0x7f6cfc009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d02a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 0x562d8c99b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d02000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 0x562d8c99bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d02a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 0x562d8c99b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57970/0 (socket says 172.21.15.27:57970) 2024-09-21T00:28:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d02a00640 1 -- 172.21.15.27:0/3113241095 learned_addr learned my addr 172.21.15.27:0/3113241095 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d03400640 1 --2- 172.21.15.27:0/3113241095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5e51b0 0x562d8c99c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d02a00640 1 -- 172.21.15.27:0/3113241095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 msgr2=0x562d8c99bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d02a00640 1 --2- 172.21.15.27:0/3113241095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 0x562d8c99bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d02a00640 1 -- 172.21.15.27:0/3113241095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5e51b0 msgr2=0x562d8c99c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d02a00640 1 --2- 172.21.15.27:0/3113241095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5e51b0 0x562d8c99c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.040+0000 7f6d02a00640 1 -- 172.21.15.27:0/3113241095 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d8ca10f60 con 0x562d8c5bc2f0 2024-09-21T00:28:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.041+0000 7f6d02a00640 1 --2- 172.21.15.27:0/3113241095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 0x562d8c99b610 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40cd300 tx=0x7f6cf40c9de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6fda3c7902326a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.041+0000 7f6ce3e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:28:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.041+0000 7f6ce3e00640 1 -- 172.21.15.27:0/3113241095 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4049070 con 0x562d8c5bc2f0 2024-09-21T00:28:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.041+0000 7f6ce3e00640 1 -- 172.21.15.27:0/3113241095 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7f6cf40d6e20 con 0x562d8c5bc2f0 2024-09-21T00:28:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.041+0000 7f6d107d3740 1 -- 172.21.15.27:0/3113241095 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cf40d72b0 con 0x562d8c5bc2f0 2024-09-21T00:28:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.042+0000 7f6ce3e00640 1 -- 172.21.15.27:0/3113241095 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6cf4105610 con 0x562d8c5bc2f0 2024-09-21T00:28:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.042+0000 7f6d107d3740 10 client.26174 fetch_fsmap learned FSMap version 34 2024-09-21T00:28:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.042+0000 7f6d107d3740 1 -- 172.21.15.27:0/3113241095 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562d8ca12130 con 0x562d8c5bc2f0 2024-09-21T00:28:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.042+0000 7f6ce3e00640 1 -- 172.21.15.27:0/3113241095 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf40d72b0 con 0x562d8c5bc2f0 2024-09-21T00:28:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.042+0000 7f6ce3e00640 1 -- 172.21.15.27:0/3113241095 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f6cf40e5b10 con 0x562d8c5bc2f0 2024-09-21T00:28:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.042+0000 7f6d107d3740 10 client.26174 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T00:28:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.042+0000 7f6d107d3740 10 client.26174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:28:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.042+0000 7f6d107d3740 20 client.26174 populate_metadata read hostname 'smithi027' 2024-09-21T00:28:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.042+0000 7f6d107d3740 1 --2- 172.21.15.27:0/3113241095 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562d8c5791c0 0x562d8ca14890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.042+0000 7f6d107d3740 4 client.26174 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:28:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.042+0000 7f6d107d3740 1 -- 172.21.15.27:0/3113241095 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d8c8fd0b0 con 0x562d8c5791c0 2024-09-21T00:28:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.042+0000 7f6d02000640 1 --2- 172.21.15.27:0/3113241095 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562d8c5791c0 0x562d8ca14890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.043+0000 7f6ce3e00640 10 client.26174 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.043+0000 7f6d02000640 1 --2- 172.21.15.27:0/3113241095 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562d8c5791c0 0x562d8ca14890 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=357ecf86dadacc32 server_cookie=1d9b1ab2ed7fb0c3 in_seq=0 out_seq=0 2024-09-21T00:28:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.391+0000 7f6ce3e00640 1 -- 172.21.15.27:0/3113241095 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d8c8fd0b0 con 0x562d8c5791c0 2024-09-21T00:28:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.391+0000 7f6ce3e00640 10 client.26174 handle_command_reply: tid=0 2024-09-21T00:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.391+0000 7f6d107d3740 1 client.26174 shutdown 2024-09-21T00:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.392+0000 7f6d107d3740 1 -- 172.21.15.27:0/3113241095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 msgr2=0x562d8c99b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.392+0000 7f6d107d3740 1 --2- 172.21.15.27:0/3113241095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 0x562d8c99b610 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40cd300 tx=0x7f6cf40c9de0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.392+0000 7f6d107d3740 1 -- 172.21.15.27:0/3113241095 shutdown_connections 2024-09-21T00:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.392+0000 7f6d107d3740 1 --2- 172.21.15.27:0/3113241095 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562d8c5791c0 0x562d8ca14890 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.392+0000 7f6d107d3740 1 --2- 172.21.15.27:0/3113241095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d8c5e51b0 0x562d8c99c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.392+0000 7f6d107d3740 1 --2- 172.21.15.27:0/3113241095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562d8c5c44d0 0x562d8c99bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.392+0000 7f6d107d3740 1 --2- 172.21.15.27:0/3113241095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d8c5bc2f0 0x562d8c99b610 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.392+0000 7f6ce3e00640 0 client.26174 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.392+0000 7f6d107d3740 1 -- 172.21.15.27:0/3113241095 >> 172.21.15.27:0/3113241095 conn(0x562d8c5d9300 msgr2=0x562d8c986870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.392+0000 7f6d107d3740 1 -- 172.21.15.27:0/3113241095 shutdown_connections 2024-09-21T00:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.392+0000 7f6d107d3740 1 -- 172.21.15.27:0/3113241095 wait complete. 2024-09-21T00:28:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.392+0000 7f6d107d3740 20 client.26174 trim_cache size 0 max 16384 2024-09-21T00:28:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:28:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:28:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:28:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.393+0000 7f6d09600640 1 -- 172.21.15.27:0/1419365734 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6ce80b66b0 msgr2=0x7f6ce80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.393+0000 7f6d09600640 1 --2- 172.21.15.27:0/1419365734 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6ce80b66b0 0x7f6ce80b8bc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc005fd0 tx=0x7f6cfc009500 comp rx=0 tx=0).stop 2024-09-21T00:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.393+0000 7f6d09600640 1 -- 172.21.15.27:0/1419365734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d04103ab0 msgr2=0x7f6d04075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.393+0000 7f6d09600640 1 --2- 172.21.15.27:0/1419365734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d04103ab0 0x7f6d04075710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4005940 tx=0x7f6cf40040c0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.393+0000 7f6d09600640 1 -- 172.21.15.27:0/1419365734 shutdown_connections 2024-09-21T00:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.393+0000 7f6d09600640 1 --2- 172.21.15.27:0/1419365734 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f6ce80b66b0 0x7f6ce80b8bc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.393+0000 7f6d09600640 1 --2- 172.21.15.27:0/1419365734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d04109d30 0x7f6d04076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.393+0000 7f6d09600640 1 --2- 172.21.15.27:0/1419365734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d041093e0 0x7f6d04075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.393+0000 7f6d09600640 1 --2- 172.21.15.27:0/1419365734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d04103ab0 0x7f6d04075710 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.393+0000 7f6d09600640 1 -- 172.21.15.27:0/1419365734 >> 172.21.15.27:0/1419365734 conn(0x7f6d040ff4f0 msgr2=0x7f6d04101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.393+0000 7f6d09600640 1 -- 172.21.15.27:0/1419365734 shutdown_connections 2024-09-21T00:28:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:40.394+0000 7f6d09600640 1 -- 172.21.15.27:0/1419365734 wait complete. 2024-09-21T00:28:40.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:40 smithi027 ceph-mon[30745]: pgmap v599: 129 pgs: 129 active+clean; 972 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 110 op/s 2024-09-21T00:28:40.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:40 smithi089 ceph-mon[34481]: pgmap v599: 129 pgs: 129 active+clean; 972 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 110 op/s 2024-09-21T00:28:40.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:40 smithi098 ceph-mon[34725]: pgmap v599: 129 pgs: 129 active+clean; 972 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 110 op/s 2024-09-21T00:28:41.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.465+0000 7f5eee000640 1 -- 172.21.15.27:0/1170227932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee81039b0 msgr2=0x7f5ee8103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.465+0000 7f5eee000640 1 --2- 172.21.15.27:0/1170227932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee81039b0 0x7f5ee8103dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009900 tx=0x7f5edc02a410 comp rx=0 tx=0).stop 2024-09-21T00:28:41.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.465+0000 7f5eee000640 1 -- 172.21.15.27:0/1170227932 shutdown_connections 2024-09-21T00:28:41.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.465+0000 7f5eee000640 1 --2- 172.21.15.27:0/1170227932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee8109c40 0x7f5ee8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.465+0000 7f5eee000640 1 --2- 172.21.15.27:0/1170227932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ee81092f0 0x7f5ee8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.465+0000 7f5eee000640 1 --2- 172.21.15.27:0/1170227932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee81039b0 0x7f5ee8103dc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.465+0000 7f5eee000640 1 -- 172.21.15.27:0/1170227932 >> 172.21.15.27:0/1170227932 conn(0x7f5ee80ff400 msgr2=0x7f5ee8101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:41.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.465+0000 7f5eee000640 1 -- 172.21.15.27:0/1170227932 shutdown_connections 2024-09-21T00:28:41.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.465+0000 7f5eee000640 1 -- 172.21.15.27:0/1170227932 wait complete. 2024-09-21T00:28:41.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.466+0000 7f5eee000640 1 Processor -- start 2024-09-21T00:28:41.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.466+0000 7f5eee000640 1 -- start start 2024-09-21T00:28:41.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.467+0000 7f5eee000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ee81039b0 0x7f5ee819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.467+0000 7f5eee000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee81092f0 0x7f5ee819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.467+0000 7f5eee000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee8109c40 0x7f5ee819fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.467+0000 7f5eee000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee8113900 con 0x7f5ee8109c40 2024-09-21T00:28:41.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.467+0000 7f5eee000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee8113780 con 0x7f5ee81092f0 2024-09-21T00:28:41.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.467+0000 7f5eee000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee8113a80 con 0x7f5ee81039b0 2024-09-21T00:28:41.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.467+0000 7f5eecc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ee81039b0 0x7f5ee819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.467+0000 7f5eed600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee8109c40 0x7f5ee819fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.467+0000 7f5eecc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ee81039b0 0x7f5ee819a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38912/0 (socket says 172.21.15.27:38912) 2024-09-21T00:28:41.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.467+0000 7f5eecc00640 1 -- 172.21.15.27:0/3865555476 learned_addr learned my addr 172.21.15.27:0/3865555476 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:41.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.467+0000 7f5ee7e00640 1 --2- 172.21.15.27:0/3865555476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee81092f0 0x7f5ee819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.468+0000 7f5eecc00640 1 -- 172.21.15.27:0/3865555476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee81092f0 msgr2=0x7f5ee819a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.468+0000 7f5eecc00640 1 --2- 172.21.15.27:0/3865555476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee81092f0 0x7f5ee819a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.468+0000 7f5eecc00640 1 -- 172.21.15.27:0/3865555476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee8109c40 msgr2=0x7f5ee819fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.468+0000 7f5eecc00640 1 --2- 172.21.15.27:0/3865555476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee8109c40 0x7f5ee819fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.468+0000 7f5eecc00640 1 -- 172.21.15.27:0/3865555476 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee81a0240 con 0x7f5ee81039b0 2024-09-21T00:28:41.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.468+0000 7f5eed600640 1 --2- 172.21.15.27:0/3865555476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee8109c40 0x7f5ee819fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:28:41.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.468+0000 7f5eecc00640 1 --2- 172.21.15.27:0/3865555476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ee81039b0 0x7f5ee819a2d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5edc005940 tx=0x7f5edc0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:41.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.468+0000 7f5ee5600640 1 -- 172.21.15.27:0/3865555476 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc03f030 con 0x7f5ee81039b0 2024-09-21T00:28:41.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.468+0000 7f5ee5600640 1 -- 172.21.15.27:0/3865555476 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5edc0044c0 con 0x7f5ee81039b0 2024-09-21T00:28:41.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.468+0000 7f5ee5600640 1 -- 172.21.15.27:0/3865555476 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc035070 con 0x7f5ee81039b0 2024-09-21T00:28:41.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.469+0000 7f5eee000640 1 -- 172.21.15.27:0/3865555476 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee81a04d0 con 0x7f5ee81039b0 2024-09-21T00:28:41.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.469+0000 7f5eee000640 1 -- 172.21.15.27:0/3865555476 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee81a6c90 con 0x7f5ee81039b0 2024-09-21T00:28:41.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.472+0000 7f5ef61fb740 1 Processor -- start 2024-09-21T00:28:41.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.472+0000 7f5ef61fb740 1 -- start start 2024-09-21T00:28:41.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.472+0000 7f5ef61fb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9398f520 0x563a93d36340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.472+0000 7f5ef61fb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 0x563a93d36880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.472+0000 7f5ef61fb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9396e840 0x563a93d36dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.472+0000 7f5ef61fb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563a93cad290 con 0x563a9398f520 2024-09-21T00:28:41.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.472+0000 7f5ef61fb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563a93c9c6c0 con 0x563a9396e840 2024-09-21T00:28:41.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.472+0000 7f5ef61fb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563a93ca04d0 con 0x563a93966660 2024-09-21T00:28:41.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.472+0000 7f5eed600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9398f520 0x563a93d36340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.472+0000 7f5eed600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9398f520 0x563a93d36340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57508/0 (socket says 172.21.15.27:57508) 2024-09-21T00:28:41.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.472+0000 7f5eed600640 1 -- 172.21.15.27:0/3652034790 learned_addr learned my addr 172.21.15.27:0/3652034790 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:41.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.473+0000 7f5ee5600640 1 -- 172.21.15.27:0/3865555476 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5edc042030 con 0x7f5ee81039b0 2024-09-21T00:28:41.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.473+0000 7f5eed600640 1 -- 172.21.15.27:0/3652034790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 msgr2=0x563a93d36880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:41.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.473+0000 7f5eecc00640 1 --2- 172.21.15.27:0/3652034790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 0x563a93d36880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.473+0000 7f5ee7e00640 1 --2- 172.21.15.27:0/3652034790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9396e840 0x563a93d36dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.473+0000 7f5eed600640 1 --2- 172.21.15.27:0/3652034790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 0x563a93d36880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.473+0000 7f5eed600640 1 -- 172.21.15.27:0/3652034790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9396e840 msgr2=0x563a93d36dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:41.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.473+0000 7f5eed600640 1 --2- 172.21.15.27:0/3652034790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9396e840 0x563a93d36dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.473+0000 7f5eed600640 1 -- 172.21.15.27:0/3652034790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a93c29460 con 0x563a9398f520 2024-09-21T00:28:41.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.473+0000 7f5eecc00640 1 --2- 172.21.15.27:0/3652034790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 0x563a93d36880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:41.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.474+0000 7f5eed600640 1 --2- 172.21.15.27:0/3652034790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9398f520 0x563a93d36340 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00117e0 tx=0x7f5ee0011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa349c159f16da7e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:41.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.474+0000 7f5ed9c00640 1 -- 172.21.15.27:0/3652034790 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0011310 con 0x563a9398f520 2024-09-21T00:28:41.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.474+0000 7f5ed9c00640 1 -- 172.21.15.27:0/3652034790 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ee0003e20 con 0x563a9398f520 2024-09-21T00:28:41.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.475+0000 7f5ed9c00640 1 -- 172.21.15.27:0/3652034790 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0004460 con 0x563a9398f520 2024-09-21T00:28:41.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.475+0000 7f5ef61fb740 1 -- 172.21.15.27:0/3652034790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9398f520 msgr2=0x563a93d36340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.475+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/3652034790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9398f520 0x563a93d36340 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00117e0 tx=0x7f5ee0011ca0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.475+0000 7f5ef61fb740 1 -- 172.21.15.27:0/3652034790 shutdown_connections 2024-09-21T00:28:41.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.475+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/3652034790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9396e840 0x563a93d36dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.475+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/3652034790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 0x563a93d36880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.475+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/3652034790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9398f520 0x563a93d36340 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.475+0000 7f5ef61fb740 1 -- 172.21.15.27:0/3652034790 >> 172.21.15.27:0/3652034790 conn(0x563a93983910 msgr2=0x563a93d34230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:41.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.475+0000 7f5ef61fb740 1 -- 172.21.15.27:0/3652034790 shutdown_connections 2024-09-21T00:28:41.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.475+0000 7f5ee5600640 1 --2- 172.21.15.27:0/3865555476 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5ed00b65b0 0x7f5ed00b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.475+0000 7f5ee5600640 1 -- 172.21.15.27:0/3865555476 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7f5edc107e80 con 0x7f5ee81039b0 2024-09-21T00:28:41.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.475+0000 7f5ef61fb740 1 -- 172.21.15.27:0/3652034790 wait complete. 2024-09-21T00:28:41.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.476+0000 7f5ee7e00640 1 --2- 172.21.15.27:0/3865555476 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5ed00b65b0 0x7f5ed00b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.476+0000 7f5ef61fb740 1 -- start start 2024-09-21T00:28:41.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.476+0000 7f5ef61fb740 1 Processor -- start 2024-09-21T00:28:41.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.476+0000 7f5ee7e00640 1 --2- 172.21.15.27:0/3865555476 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5ed00b65b0 0x7f5ed00b8ac0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8108eb0 tx=0x7f5ed4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:41.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5ef61fb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 0x563a93d41ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5ef61fb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9396e840 0x563a93d42430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5ef61fb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9398f520 0x563a93d4dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5ef61fb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563a93afdbb0 con 0x563a9396e840 2024-09-21T00:28:41.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5ef61fb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563a93c376c0 con 0x563a9398f520 2024-09-21T00:28:41.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5ef61fb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563a93d44b70 con 0x563a93966660 2024-09-21T00:28:41.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5ef61fb740 10 client.0 _ll_register_callbacks cb 0x563a93c52490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:41.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5eecc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 0x563a93d41ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5ee7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9396e840 0x563a93d42430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5eecc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 0x563a93d41ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38922/0 (socket says 172.21.15.27:38922) 2024-09-21T00:28:41.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5eecc00640 1 -- 172.21.15.27:0/1833439868 learned_addr learned my addr 172.21.15.27:0/1833439868 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:41.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5eed600640 1 --2- 172.21.15.27:0/1833439868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9398f520 0x563a93d4dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5eecc00640 1 -- 172.21.15.27:0/1833439868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9398f520 msgr2=0x563a93d4dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5eecc00640 1 --2- 172.21.15.27:0/1833439868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9398f520 0x563a93d4dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5eecc00640 1 -- 172.21.15.27:0/1833439868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9396e840 msgr2=0x563a93d42430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.477+0000 7f5eecc00640 1 --2- 172.21.15.27:0/1833439868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9396e840 0x563a93d42430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.478+0000 7f5eecc00640 1 -- 172.21.15.27:0/1833439868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a939fab00 con 0x563a93966660 2024-09-21T00:28:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.478+0000 7f5eecc00640 1 --2- 172.21.15.27:0/1833439868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 0x563a93d41ef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5edc105200 tx=0x7f5edc0c98b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6390fd4379846e63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.478+0000 7f5ece000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:28:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.478+0000 7f5ece000640 1 -- 172.21.15.27:0/1833439868 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc049070 con 0x563a93966660 2024-09-21T00:28:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.478+0000 7f5ef61fb740 1 -- 172.21.15.27:0/1833439868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ee0003e20 con 0x563a93966660 2024-09-21T00:28:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.478+0000 7f5ece000640 1 -- 172.21.15.27:0/1833439868 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7f5edc0d6e30 con 0x563a93966660 2024-09-21T00:28:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.478+0000 7f5ece000640 1 -- 172.21.15.27:0/1833439868 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5edc105620 con 0x563a93966660 2024-09-21T00:28:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.478+0000 7f5ef61fb740 10 client.26173 fetch_fsmap learned FSMap version 34 2024-09-21T00:28:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.478+0000 7f5ef61fb740 1 -- 172.21.15.27:0/1833439868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563a93a71960 con 0x563a93966660 2024-09-21T00:28:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.479+0000 7f5ece000640 1 -- 172.21.15.27:0/1833439868 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ee0003e20 con 0x563a93966660 2024-09-21T00:28:41.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.479+0000 7f5ece000640 1 -- 172.21.15.27:0/1833439868 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f5edc0c92a0 con 0x563a93966660 2024-09-21T00:28:41.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.479+0000 7f5ef61fb740 10 client.26173 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T00:28:41.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.479+0000 7f5ef61fb740 10 client.26173 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:28:41.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.479+0000 7f5ef61fb740 20 client.26173 populate_metadata read hostname 'smithi027' 2024-09-21T00:28:41.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.479+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/1833439868 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563a9394dea0 0x563a93d53ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.479+0000 7f5ef61fb740 4 client.26173 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:28:41.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.479+0000 7f5ef61fb740 1 -- 172.21.15.27:0/1833439868 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a93caf1e0 con 0x563a9394dea0 2024-09-21T00:28:41.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.480+0000 7f5ee7e00640 1 --2- 172.21.15.27:0/1833439868 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563a9394dea0 0x563a93d53ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.480+0000 7f5ee7e00640 1 --2- 172.21.15.27:0/1833439868 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563a9394dea0 0x563a93d53ca0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc4f25f76f5274fc server_cookie=4b198be6c7fab7c2 in_seq=0 out_seq=0 2024-09-21T00:28:41.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.480+0000 7f5ece000640 10 client.26173 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:41.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.481+0000 7f5ece000640 1 -- 172.21.15.27:0/1833439868 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563a93caf1e0 con 0x563a9394dea0 2024-09-21T00:28:41.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.481+0000 7f5ece000640 10 client.26173 handle_command_reply: tid=0 2024-09-21T00:28:41.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.481+0000 7f5ef61fb740 1 client.26173 shutdown 2024-09-21T00:28:41.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.481+0000 7f5ef61fb740 1 -- 172.21.15.27:0/1833439868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 msgr2=0x563a93d41ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.481+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/1833439868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 0x563a93d41ef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5edc105200 tx=0x7f5edc0c98b0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.481+0000 7f5ef61fb740 1 -- 172.21.15.27:0/1833439868 shutdown_connections 2024-09-21T00:28:41.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.481+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/1833439868 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563a9394dea0 0x563a93d53ca0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.481+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/1833439868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9398f520 0x563a93d4dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.481+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/1833439868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9396e840 0x563a93d42430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.481+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/1833439868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 0x563a93d41ef0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.481+0000 7f5ece000640 0 client.26173 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:41.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.481+0000 7f5ef61fb740 1 -- 172.21.15.27:0/1833439868 >> 172.21.15.27:0/1833439868 conn(0x563a93983910 msgr2=0x563a93d39520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:41.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.482+0000 7f5ef61fb740 1 -- 172.21.15.27:0/1833439868 shutdown_connections 2024-09-21T00:28:41.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.482+0000 7f5ef61fb740 1 -- 172.21.15.27:0/1833439868 wait complete. 2024-09-21T00:28:41.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.482+0000 7f5ef61fb740 20 client.26173 trim_cache size 0 max 16384 2024-09-21T00:28:41.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.500+0000 7f5ef61fb740 1 Processor -- start 2024-09-21T00:28:41.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.500+0000 7f5ef61fb740 1 -- start start 2024-09-21T00:28:41.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.500+0000 7f5ef61fb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 0x563a93d4dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.500+0000 7f5ef61fb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9396e840 0x563a93d4e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.500+0000 7f5ef61fb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9398f520 0x563a93d3b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.500+0000 7f5ef61fb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee81b14e0 con 0x563a9398f520 2024-09-21T00:28:41.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.500+0000 7f5ef61fb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee81aecf0 con 0x563a9396e840 2024-09-21T00:28:41.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.500+0000 7f5ef61fb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee81acb60 con 0x563a93966660 2024-09-21T00:28:41.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.501+0000 7f5eed600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9398f520 0x563a93d3b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.501+0000 7f5eecc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 0x563a93d4dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.501+0000 7f5ee7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9396e840 0x563a93d4e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.501+0000 7f5eed600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9398f520 0x563a93d3b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57528/0 (socket says 172.21.15.27:57528) 2024-09-21T00:28:41.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.501+0000 7f5eed600640 1 -- 172.21.15.27:0/1399573629 learned_addr learned my addr 172.21.15.27:0/1399573629 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:41.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.501+0000 7f5ee7e00640 1 -- 172.21.15.27:0/1399573629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 msgr2=0x563a93d4dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:41.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.501+0000 7f5ee7e00640 1 --2- 172.21.15.27:0/1399573629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 0x563a93d4dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.501+0000 7f5ee7e00640 1 -- 172.21.15.27:0/1399573629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9398f520 msgr2=0x563a93d3b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:41.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.501+0000 7f5ee7e00640 1 --2- 172.21.15.27:0/1399573629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9398f520 0x563a93d3b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.501+0000 7f5ee7e00640 1 -- 172.21.15.27:0/1399573629 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a93d427e0 con 0x563a9396e840 2024-09-21T00:28:41.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.502+0000 7f5ee7e00640 1 --2- 172.21.15.27:0/1399573629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9396e840 0x563a93d4e320 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x563a93d543a0 tx=0x7f5ed4016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3773cd9049ae6ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:41.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.502+0000 7f5ed9200640 1 -- 172.21.15.27:0/1399573629 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4003770 con 0x563a9396e840 2024-09-21T00:28:41.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.502+0000 7f5ed9200640 1 -- 172.21.15.27:0/1399573629 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ed4003dc0 con 0x563a9396e840 2024-09-21T00:28:41.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.502+0000 7f5ed9200640 1 -- 172.21.15.27:0/1399573629 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4031da0 con 0x563a9396e840 2024-09-21T00:28:41.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.503+0000 7f5ef61fb740 1 -- 172.21.15.27:0/1399573629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9396e840 msgr2=0x563a93d4e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.503+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/1399573629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9396e840 0x563a93d4e320 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x563a93d543a0 tx=0x7f5ed4016000 comp rx=0 tx=0).stop 2024-09-21T00:28:41.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.503+0000 7f5ef61fb740 1 -- 172.21.15.27:0/1399573629 shutdown_connections 2024-09-21T00:28:41.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.503+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/1399573629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9398f520 0x563a93d3b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.503+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/1399573629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a9396e840 0x563a93d4e320 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.503+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/1399573629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a93966660 0x563a93d4dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.503+0000 7f5ef61fb740 1 -- 172.21.15.27:0/1399573629 >> 172.21.15.27:0/1399573629 conn(0x563a93983910 msgr2=0x563a93d341e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:41.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.503+0000 7f5ef61fb740 1 -- 172.21.15.27:0/1399573629 shutdown_connections 2024-09-21T00:28:41.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.503+0000 7f5ef61fb740 1 -- 172.21.15.27:0/1399573629 wait complete. 2024-09-21T00:28:41.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.503+0000 7f5ef61fb740 1 -- start start 2024-09-21T00:28:41.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.504+0000 7f5ef61fb740 1 Processor -- start 2024-09-21T00:28:41.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.504+0000 7f5ef61fb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a93966660 0x563a93d45670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.504+0000 7f5ef61fb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9396e840 0x563a93d45c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.504+0000 7f5ef61fb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a9398f520 0x563a93db8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.504+0000 7f5ef61fb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee81ad1b0 con 0x563a9396e840 2024-09-21T00:28:41.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.504+0000 7f5ef61fb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee81ae620 con 0x563a93966660 2024-09-21T00:28:41.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.504+0000 7f5ef61fb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee81ad880 con 0x563a9398f520 2024-09-21T00:28:41.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.504+0000 7f5ef61fb740 10 client.0 _ll_register_callbacks cb 0x7f5edc005ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:41.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.504+0000 7f5ee7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9396e840 0x563a93d45c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.504+0000 7f5ee7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9396e840 0x563a93d45c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57534/0 (socket says 172.21.15.27:57534) 2024-09-21T00:28:41.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.504+0000 7f5ee7e00640 1 -- 172.21.15.27:0/2931328626 learned_addr learned my addr 172.21.15.27:0/2931328626 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:41.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.504+0000 7f5eecc00640 1 --2- 172.21.15.27:0/2931328626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a93966660 0x563a93d45670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.505+0000 7f5eed600640 1 --2- 172.21.15.27:0/2931328626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a9398f520 0x563a93db8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.505+0000 7f5eecc00640 1 -- 172.21.15.27:0/2931328626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a9398f520 msgr2=0x563a93db8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:41.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.505+0000 7f5eecc00640 1 --2- 172.21.15.27:0/2931328626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a9398f520 0x563a93db8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.505+0000 7f5eecc00640 1 -- 172.21.15.27:0/2931328626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9396e840 msgr2=0x563a93d45c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:41.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.505+0000 7f5eecc00640 1 --2- 172.21.15.27:0/2931328626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9396e840 0x563a93d45c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.505+0000 7f5eecc00640 1 -- 172.21.15.27:0/2931328626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a93db8ca0 con 0x563a93966660 2024-09-21T00:28:41.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.505+0000 7f5ee7e00640 1 --2- 172.21.15.27:0/2931328626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9396e840 0x563a93d45c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:28:41.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.505+0000 7f5eecc00640 1 --2- 172.21.15.27:0/2931328626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a93966660 0x563a93d45670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0c9f20 tx=0x7f5edc105650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f3bbec979e8a136 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:41.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.505+0000 7f5ece000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:28:41.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.505+0000 7f5ece000640 1 -- 172.21.15.27:0/2931328626 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc0cd2a0 con 0x563a93966660 2024-09-21T00:28:41.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.505+0000 7f5ece000640 1 -- 172.21.15.27:0/2931328626 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7f5edc0cede0 con 0x563a93966660 2024-09-21T00:28:41.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.505+0000 7f5ef61fb740 1 -- 172.21.15.27:0/2931328626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ed4003dc0 con 0x563a93966660 2024-09-21T00:28:41.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.506+0000 7f5ece000640 1 -- 172.21.15.27:0/2931328626 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5edc0d2150 con 0x563a93966660 2024-09-21T00:28:41.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.506+0000 7f5ef61fb740 10 client.26189 fetch_fsmap learned FSMap version 34 2024-09-21T00:28:41.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.506+0000 7f5ef61fb740 1 -- 172.21.15.27:0/2931328626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563a93dc0000 con 0x563a93966660 2024-09-21T00:28:41.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.506+0000 7f5ece000640 1 -- 172.21.15.27:0/2931328626 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ed4003dc0 con 0x563a93966660 2024-09-21T00:28:41.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.506+0000 7f5ece000640 1 -- 172.21.15.27:0/2931328626 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f5edc031cc0 con 0x563a93966660 2024-09-21T00:28:41.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.506+0000 7f5ef61fb740 10 client.26189 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T00:28:41.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.506+0000 7f5ef61fb740 10 client.26189 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:28:41.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.506+0000 7f5ef61fb740 20 client.26189 populate_metadata read hostname 'smithi027' 2024-09-21T00:28:41.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.506+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/2931328626 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563a93923250 0x563a93dc26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.506+0000 7f5ef61fb740 4 client.26189 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3b281b3b", "release": true, "await": true}] 2024-09-21T00:28:41.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.506+0000 7f5ef61fb740 1 -- 172.21.15.27:0/2931328626 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3b281b3b", "release": true, "await": true}) -- 0x563a93caac90 con 0x563a93923250 2024-09-21T00:28:41.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.507+0000 7f5ee7e00640 1 --2- 172.21.15.27:0/2931328626 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563a93923250 0x563a93dc26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.507+0000 7f5ee7e00640 1 --2- 172.21.15.27:0/2931328626 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563a93923250 0x563a93dc26b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bca6131d83cc6db9 server_cookie=3e9a3af184bf61d2 in_seq=0 out_seq=0 2024-09-21T00:28:41.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:41.507+0000 7f5ece000640 10 client.26189 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:42.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.288+0000 7f5ece000640 1 -- 172.21.15.27:0/2931328626 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563a93caac90 con 0x563a93923250 2024-09-21T00:28:42.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.288+0000 7f5ece000640 10 client.26189 handle_command_reply: tid=0 2024-09-21T00:28:42.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.288+0000 7f5ef61fb740 1 client.26189 shutdown 2024-09-21T00:28:42.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.288+0000 7f5ef61fb740 1 -- 172.21.15.27:0/2931328626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a93966660 msgr2=0x563a93d45670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:42.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.288+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/2931328626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a93966660 0x563a93d45670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0c9f20 tx=0x7f5edc105650 comp rx=0 tx=0).stop 2024-09-21T00:28:42.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.288+0000 7f5ef61fb740 1 -- 172.21.15.27:0/2931328626 shutdown_connections 2024-09-21T00:28:42.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.289+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/2931328626 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563a93923250 0x563a93dc26b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.289+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/2931328626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563a9398f520 0x563a93db8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.289+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/2931328626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a9396e840 0x563a93d45c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.289+0000 7f5ef61fb740 1 --2- 172.21.15.27:0/2931328626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563a93966660 0x563a93d45670 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.289+0000 7f5ef61fb740 1 -- 172.21.15.27:0/2931328626 >> 172.21.15.27:0/2931328626 conn(0x563a93983910 msgr2=0x563a93d43b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.289+0000 7f5ece000640 0 client.26189 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.289+0000 7f5ef61fb740 1 -- 172.21.15.27:0/2931328626 shutdown_connections 2024-09-21T00:28:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.289+0000 7f5ef61fb740 1 -- 172.21.15.27:0/2931328626 wait complete. 2024-09-21T00:28:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.289+0000 7f5ef61fb740 20 client.26189 trim_cache size 0 max 16384 2024-09-21T00:28:42.292 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:28:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 34, 2024-09-21T00:28:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24761, 2024-09-21T00:28:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 24, 2024-09-21T00:28:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T00:28:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "3b281b3b": { 2024-09-21T00:28:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 24, 2024-09-21T00:28:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T00:28:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:28:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T00:28:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:28:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:28:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T00:28:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 139.2, 2024-09-21T00:28:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T00:28:42.293 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T00:28:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T00:28:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:28:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T00:28:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:28:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:28:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:28:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:28:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:28:42.294 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:28:42.294 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:28:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.290+0000 7f5eee000640 1 -- 172.21.15.27:0/3865555476 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5ed00b65b0 msgr2=0x7f5ed00b8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.290+0000 7f5eee000640 1 --2- 172.21.15.27:0/3865555476 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5ed00b65b0 0x7f5ed00b8ac0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8108eb0 tx=0x7f5ed4005f90 comp rx=0 tx=0).stop 2024-09-21T00:28:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.290+0000 7f5eee000640 1 -- 172.21.15.27:0/3865555476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ee81039b0 msgr2=0x7f5ee819a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.290+0000 7f5eee000640 1 --2- 172.21.15.27:0/3865555476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ee81039b0 0x7f5ee819a2d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5edc005940 tx=0x7f5edc0040c0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.291+0000 7f5eee000640 1 -- 172.21.15.27:0/3865555476 shutdown_connections 2024-09-21T00:28:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.291+0000 7f5eee000640 1 --2- 172.21.15.27:0/3865555476 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5ed00b65b0 0x7f5ed00b8ac0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.291+0000 7f5eee000640 1 --2- 172.21.15.27:0/3865555476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee8109c40 0x7f5ee819fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.291+0000 7f5eee000640 1 --2- 172.21.15.27:0/3865555476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee81092f0 0x7f5ee819a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.291+0000 7f5eee000640 1 --2- 172.21.15.27:0/3865555476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ee81039b0 0x7f5ee819a2d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.291+0000 7f5eee000640 1 -- 172.21.15.27:0/3865555476 >> 172.21.15.27:0/3865555476 conn(0x7f5ee80ff400 msgr2=0x7f5ee8101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:42.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.292+0000 7f5eee000640 1 -- 172.21.15.27:0/3865555476 shutdown_connections 2024-09-21T00:28:42.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:42.292+0000 7f5eee000640 1 -- 172.21.15.27:0/3865555476 wait complete. 2024-09-21T00:28:42.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:42 smithi027 ceph-mon[30745]: pgmap v600: 129 pgs: 129 active+clean; 972 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 111 op/s 2024-09-21T00:28:42.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:42 smithi089 ceph-mon[34481]: pgmap v600: 129 pgs: 129 active+clean; 972 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 111 op/s 2024-09-21T00:28:42.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:42 smithi098 ceph-mon[34725]: pgmap v600: 129 pgs: 129 active+clean; 972 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 111 op/s 2024-09-21T00:28:43.126 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3b281b3b', total seconds elapsed: 36.7 2024-09-21T00:28:43.127 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.25 2024-09-21T00:28:43.127 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:28:43.239 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T00:28:43.239 INFO:tasks.workunit.client.0.smithi027.stdout:Final score for writes: 241 2024-09-21T00:28:43.239 INFO:tasks.workunit.client.0.smithi027.stdout:Final score for reads : 141104 2024-09-21T00:28:43.239 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T00:28:43.240 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T00:28:43.240 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T00:28:43.550 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:28:44.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.213+0000 7f9436000640 1 -- 172.21.15.27:0/2449807130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94301039b0 msgr2=0x7f9430103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:44.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.213+0000 7f9436000640 1 --2- 172.21.15.27:0/2449807130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94301039b0 0x7f9430103dc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9424009900 tx=0x7f942402a410 comp rx=0 tx=0).stop 2024-09-21T00:28:44.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.214+0000 7f9436000640 1 -- 172.21.15.27:0/2449807130 shutdown_connections 2024-09-21T00:28:44.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.214+0000 7f9436000640 1 --2- 172.21.15.27:0/2449807130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9430109c40 0x7f9430110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.214+0000 7f9436000640 1 --2- 172.21.15.27:0/2449807130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94301092f0 0x7f9430109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.214+0000 7f9436000640 1 --2- 172.21.15.27:0/2449807130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94301039b0 0x7f9430103dc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.214+0000 7f9436000640 1 -- 172.21.15.27:0/2449807130 >> 172.21.15.27:0/2449807130 conn(0x7f94300ff440 msgr2=0x7f9430101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:44.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.214+0000 7f9436000640 1 -- 172.21.15.27:0/2449807130 shutdown_connections 2024-09-21T00:28:44.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.214+0000 7f9436000640 1 -- 172.21.15.27:0/2449807130 wait complete. 2024-09-21T00:28:44.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.215+0000 7f9436000640 1 Processor -- start 2024-09-21T00:28:44.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.215+0000 7f9436000640 1 -- start start 2024-09-21T00:28:44.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.216+0000 7f9436000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94301039b0 0x7f943019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:44.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.216+0000 7f9436000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94301092f0 0x7f943019a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:44.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.216+0000 7f9436000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9430109c40 0x7f943019fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:44.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.216+0000 7f9436000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9430113950 con 0x7f94301092f0 2024-09-21T00:28:44.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.216+0000 7f9436000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f94301137d0 con 0x7f9430109c40 2024-09-21T00:28:44.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.216+0000 7f9436000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9430113ad0 con 0x7f94301039b0 2024-09-21T00:28:44.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.216+0000 7f9434c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94301039b0 0x7f943019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:44.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.216+0000 7f9434c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94301039b0 0x7f943019a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41760/0 (socket says 172.21.15.27:41760) 2024-09-21T00:28:44.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.216+0000 7f9434c00640 1 -- 172.21.15.27:0/2626987616 learned_addr learned my addr 172.21.15.27:0/2626987616 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:44.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.216+0000 7f9435600640 1 --2- 172.21.15.27:0/2626987616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9430109c40 0x7f943019fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:44.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.216+0000 7f942fe00640 1 --2- 172.21.15.27:0/2626987616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94301092f0 0x7f943019a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:44.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.217+0000 7f9434c00640 1 -- 172.21.15.27:0/2626987616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9430109c40 msgr2=0x7f943019fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:44.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.217+0000 7f9434c00640 1 --2- 172.21.15.27:0/2626987616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9430109c40 0x7f943019fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.217+0000 7f9434c00640 1 -- 172.21.15.27:0/2626987616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94301092f0 msgr2=0x7f943019a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:44.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.217+0000 7f9434c00640 1 --2- 172.21.15.27:0/2626987616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94301092f0 0x7f943019a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.217+0000 7f9434c00640 1 -- 172.21.15.27:0/2626987616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94301a0250 con 0x7f94301039b0 2024-09-21T00:28:44.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.217+0000 7f9435600640 1 --2- 172.21.15.27:0/2626987616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9430109c40 0x7f943019fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:28:44.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.217+0000 7f9434c00640 1 --2- 172.21.15.27:0/2626987616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94301039b0 0x7f943019a2e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9424005ab0 tx=0x7f9424003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:44.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.217+0000 7f942d600640 1 -- 172.21.15.27:0/2626987616 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942403f070 con 0x7f94301039b0 2024-09-21T00:28:44.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.217+0000 7f9436000640 1 -- 172.21.15.27:0/2626987616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94301a04e0 con 0x7f94301039b0 2024-09-21T00:28:44.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.217+0000 7f942d600640 1 -- 172.21.15.27:0/2626987616 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9424031b60 con 0x7f94301039b0 2024-09-21T00:28:44.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.217+0000 7f942d600640 1 -- 172.21.15.27:0/2626987616 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424035030 con 0x7f94301039b0 2024-09-21T00:28:44.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.217+0000 7f9436000640 1 -- 172.21.15.27:0/2626987616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94301a6ca0 con 0x7f94301039b0 2024-09-21T00:28:44.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.219+0000 7f9436000640 1 -- 172.21.15.27:0/2626987616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9430103dc0 con 0x7f94301039b0 2024-09-21T00:28:44.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.221+0000 7f942d600640 1 -- 172.21.15.27:0/2626987616 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9424042070 con 0x7f94301039b0 2024-09-21T00:28:44.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.223+0000 7f942d600640 1 --2- 172.21.15.27:0/2626987616 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f94180b6680 0x7f94180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:44.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.223+0000 7f942d600640 1 -- 172.21.15.27:0/2626987616 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7f94241088f0 con 0x7f94301039b0 2024-09-21T00:28:44.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.223+0000 7f942fe00640 1 --2- 172.21.15.27:0/2626987616 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f94180b6680 0x7f94180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:44.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.224+0000 7f942fe00640 1 --2- 172.21.15.27:0/2626987616 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f94180b6680 0x7f94180b8b90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f941c0095f0 tx=0x7f941c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:44.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.227+0000 7f942d600640 1 -- 172.21.15.27:0/2626987616 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94240d2df0 con 0x7f94301039b0 2024-09-21T00:28:44.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.491+0000 7f9436000640 1 -- 172.21.15.27:0/2626987616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f943019b410 con 0x7f94301039b0 2024-09-21T00:28:44.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.492+0000 7f942d600640 1 -- 172.21.15.27:0/2626987616 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11902 (secure 0 0 0) 0x7f943019b410 con 0x7f94301039b0 2024-09-21T00:28:44.494 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:28:44.494 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":34,"btime":"2024-09-21T00:28:09:754006+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:28:09.743913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26057":{"gid":26057,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/4028763758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":4028763758},{"type":"v1","addr":"172.21.15.89:6837","nonce":4028763758}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:28:44.494 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 34 2024-09-21T00:28:44.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.495+0000 7f9436000640 1 -- 172.21.15.27:0/2626987616 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f94180b6680 msgr2=0x7f94180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:44.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.495+0000 7f9436000640 1 --2- 172.21.15.27:0/2626987616 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f94180b6680 0x7f94180b8b90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f941c0095f0 tx=0x7f941c005f90 comp rx=0 tx=0).stop 2024-09-21T00:28:44.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.495+0000 7f9436000640 1 -- 172.21.15.27:0/2626987616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94301039b0 msgr2=0x7f943019a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:44.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.495+0000 7f9436000640 1 --2- 172.21.15.27:0/2626987616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94301039b0 0x7f943019a2e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9424005ab0 tx=0x7f9424003c10 comp rx=0 tx=0).stop 2024-09-21T00:28:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.495+0000 7f9436000640 1 -- 172.21.15.27:0/2626987616 shutdown_connections 2024-09-21T00:28:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.495+0000 7f9436000640 1 --2- 172.21.15.27:0/2626987616 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f94180b6680 0x7f94180b8b90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.495+0000 7f9436000640 1 --2- 172.21.15.27:0/2626987616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9430109c40 0x7f943019fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.495+0000 7f9436000640 1 --2- 172.21.15.27:0/2626987616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94301092f0 0x7f943019a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.495+0000 7f9436000640 1 --2- 172.21.15.27:0/2626987616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94301039b0 0x7f943019a2e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.495+0000 7f9436000640 1 -- 172.21.15.27:0/2626987616 >> 172.21.15.27:0/2626987616 conn(0x7f94300ff440 msgr2=0x7f9430101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.496+0000 7f9436000640 1 -- 172.21.15.27:0/2626987616 shutdown_connections 2024-09-21T00:28:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:44.496+0000 7f9436000640 1 -- 172.21.15.27:0/2626987616 wait complete. 2024-09-21T00:28:44.530 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:44 smithi089 ceph-mon[34481]: pgmap v601: 129 pgs: 129 active+clean; 969 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 700 KiB/s wr, 86 op/s 2024-09-21T00:28:44.562 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:44 smithi027 ceph-mon[30745]: pgmap v601: 129 pgs: 129 active+clean; 969 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 700 KiB/s wr, 86 op/s 2024-09-21T00:28:44.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:44 smithi098 ceph-mon[34725]: pgmap v601: 129 pgs: 129 active+clean; 969 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 700 KiB/s wr, 86 op/s 2024-09-21T00:28:45.405 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0' 2024-09-21T00:28:45.405 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0 2024-09-21T00:28:45.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:45 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2626987616' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:28:45.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:45 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2626987616' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:28:45.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:45 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2626987616' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:28:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:28:46.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.528+0000 7f8204e00640 1 -- 172.21.15.27:0/2082922128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f820010c010 msgr2=0x7f8200112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.528+0000 7f8204e00640 1 --2- 172.21.15.27:0/2082922128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f820010c010 0x7f8200112840 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f81f8009be0 tx=0x7f81f802a6d0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.528+0000 7f8204e00640 1 -- 172.21.15.27:0/2082922128 shutdown_connections 2024-09-21T00:28:46.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.528+0000 7f8204e00640 1 --2- 172.21.15.27:0/2082922128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f820010c010 0x7f8200112840 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.528+0000 7f8204e00640 1 --2- 172.21.15.27:0/2082922128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f820010b6c0 0x7f820010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.528+0000 7f8204e00640 1 --2- 172.21.15.27:0/2082922128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8200105d80 0x7f8200106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.528+0000 7f8204e00640 1 -- 172.21.15.27:0/2082922128 >> 172.21.15.27:0/2082922128 conn(0x7f82001017d0 msgr2=0x7f8200103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:46.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.528+0000 7f8204e00640 1 -- 172.21.15.27:0/2082922128 shutdown_connections 2024-09-21T00:28:46.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.528+0000 7f8204e00640 1 -- 172.21.15.27:0/2082922128 wait complete. 2024-09-21T00:28:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.529+0000 7f8204e00640 1 Processor -- start 2024-09-21T00:28:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.529+0000 7f8204e00640 1 -- start start 2024-09-21T00:28:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.529+0000 7f8204e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8200105d80 0x7f820019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f8204e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f820010b6c0 0x7f820019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f8204e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f820010c010 0x7f82001a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f8204e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8200115cd0 con 0x7f820010b6c0 2024-09-21T00:28:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f8204e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8200115b50 con 0x7f820010c010 2024-09-21T00:28:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f8204e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8200115e50 con 0x7f8200105d80 2024-09-21T00:28:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f81fde00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8200105d80 0x7f820019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f81fd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f820010b6c0 0x7f820019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f81fde00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8200105d80 0x7f820019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41778/0 (socket says 172.21.15.27:41778) 2024-09-21T00:28:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f81fde00640 1 -- 172.21.15.27:0/2792854909 learned_addr learned my addr 172.21.15.27:0/2792854909 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f81fe800640 1 --2- 172.21.15.27:0/2792854909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f820010c010 0x7f82001a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f81fde00640 1 -- 172.21.15.27:0/2792854909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f820010c010 msgr2=0x7f82001a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f81fde00640 1 --2- 172.21.15.27:0/2792854909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f820010c010 0x7f82001a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f81fde00640 1 -- 172.21.15.27:0/2792854909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f820010b6c0 msgr2=0x7f820019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f81fde00640 1 --2- 172.21.15.27:0/2792854909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f820010b6c0 0x7f820019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f81fde00640 1 -- 172.21.15.27:0/2792854909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82001a2590 con 0x7f8200105d80 2024-09-21T00:28:46.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.530+0000 7f81fd400640 1 --2- 172.21.15.27:0/2792854909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f820010b6c0 0x7f820019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:28:46.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.531+0000 7f81fde00640 1 --2- 172.21.15.27:0/2792854909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8200105d80 0x7f820019c620 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f81f000ea60 tx=0x7f81f000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:46.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.531+0000 7f81f6a00640 1 -- 172.21.15.27:0/2792854909 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f000cde0 con 0x7f8200105d80 2024-09-21T00:28:46.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.531+0000 7f8204e00640 1 -- 172.21.15.27:0/2792854909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82001a2880 con 0x7f8200105d80 2024-09-21T00:28:46.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.531+0000 7f81f6a00640 1 -- 172.21.15.27:0/2792854909 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f00035f0 con 0x7f8200105d80 2024-09-21T00:28:46.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.531+0000 7f81f6a00640 1 -- 172.21.15.27:0/2792854909 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0010690 con 0x7f8200105d80 2024-09-21T00:28:46.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.531+0000 7f8204e00640 1 -- 172.21.15.27:0/2792854909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82001a8fe0 con 0x7f8200105d80 2024-09-21T00:28:46.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.534+0000 7f820b9bf740 1 Processor -- start 2024-09-21T00:28:46.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.535+0000 7f820b9bf740 1 -- start start 2024-09-21T00:28:46.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.535+0000 7f820b9bf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c599850 0x55895c9409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.535+0000 7f820b9bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c570990 0x55895c940ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.535+0000 7f820b9bf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c941430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.535+0000 7f820b9bf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55895c8b6a90 con 0x55895c599850 2024-09-21T00:28:46.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.535+0000 7f820b9bf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55895c8a5ec0 con 0x55895c570990 2024-09-21T00:28:46.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.535+0000 7f820b9bf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55895c8a9cd0 con 0x55895c578b70 2024-09-21T00:28:46.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.535+0000 7f81f6a00640 1 -- 172.21.15.27:0/2792854909 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f81f00037a0 con 0x7f8200105d80 2024-09-21T00:28:46.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.535+0000 7f81fe800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c599850 0x55895c9409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.535+0000 7f81fd400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c941430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.535+0000 7f81fe800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c599850 0x55895c9409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57098/0 (socket says 172.21.15.27:57098) 2024-09-21T00:28:46.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.535+0000 7f81fd400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c941430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41782/0 (socket says 172.21.15.27:41782) 2024-09-21T00:28:46.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.535+0000 7f81fe800640 1 -- 172.21.15.27:0/3025799843 learned_addr learned my addr 172.21.15.27:0/3025799843 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:46.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.536+0000 7f81fde00640 1 --2- 172.21.15.27:0/3025799843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c570990 0x55895c940ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.536+0000 7f81fd400640 1 -- 172.21.15.27:0/3025799843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c570990 msgr2=0x55895c940ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:46.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.536+0000 7f81fd400640 1 --2- 172.21.15.27:0/3025799843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c570990 0x55895c940ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.536+0000 7f81fd400640 1 -- 172.21.15.27:0/3025799843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c599850 msgr2=0x55895c9409b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:46.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.536+0000 7f81fd400640 1 --2- 172.21.15.27:0/3025799843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c599850 0x55895c9409b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.536+0000 7f81fd400640 1 -- 172.21.15.27:0/3025799843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895c84b9d0 con 0x55895c578b70 2024-09-21T00:28:46.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.536+0000 7f81fe800640 1 --2- 172.21.15.27:0/3025799843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c599850 0x55895c9409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:28:46.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.537+0000 7f81fd400640 1 --2- 172.21.15.27:0/3025799843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c941430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009530 tx=0x7f81ec0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=589b2b8384203fff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:46.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.537+0000 7f81eb400640 1 -- 172.21.15.27:0/3025799843 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec004020 con 0x55895c578b70 2024-09-21T00:28:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.537+0000 7f81eb400640 1 -- 172.21.15.27:0/3025799843 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ec0041c0 con 0x55895c578b70 2024-09-21T00:28:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.537+0000 7f81eb400640 1 -- 172.21.15.27:0/3025799843 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec00f850 con 0x55895c578b70 2024-09-21T00:28:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.537+0000 7f81f6a00640 1 --2- 172.21.15.27:0/2792854909 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f81e40b6680 0x7f81e40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.538+0000 7f81f6a00640 1 -- 172.21.15.27:0/2792854909 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7f81f00e7e30 con 0x7f8200105d80 2024-09-21T00:28:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.538+0000 7f81fe800640 1 --2- 172.21.15.27:0/2792854909 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f81e40b6680 0x7f81e40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.538+0000 7f820b9bf740 1 -- 172.21.15.27:0/3025799843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 msgr2=0x55895c941430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.538+0000 7f820b9bf740 1 --2- 172.21.15.27:0/3025799843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c941430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009530 tx=0x7f81ec0118d0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.538+0000 7f820b9bf740 1 -- 172.21.15.27:0/3025799843 shutdown_connections 2024-09-21T00:28:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.538+0000 7f820b9bf740 1 --2- 172.21.15.27:0/3025799843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c941430 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.538+0000 7f820b9bf740 1 --2- 172.21.15.27:0/3025799843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c570990 0x55895c940ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.538+0000 7f820b9bf740 1 --2- 172.21.15.27:0/3025799843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c599850 0x55895c9409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.538+0000 7f820b9bf740 1 -- 172.21.15.27:0/3025799843 >> 172.21.15.27:0/3025799843 conn(0x55895c58dc40 msgr2=0x55895c93e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.538+0000 7f820b9bf740 1 -- 172.21.15.27:0/3025799843 shutdown_connections 2024-09-21T00:28:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.538+0000 7f820b9bf740 1 -- 172.21.15.27:0/3025799843 wait complete. 2024-09-21T00:28:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.539+0000 7f81fe800640 1 --2- 172.21.15.27:0/2792854909 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f81e40b6680 0x7f81e40b8b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f81f8006010 tx=0x7f81f8004a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.539+0000 7f820b9bf740 1 -- start start 2024-09-21T00:28:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.539+0000 7f820b9bf740 1 Processor -- start 2024-09-21T00:28:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.540+0000 7f820b9bf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c570990 0x55895c94c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.540+0000 7f820b9bf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c94ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.540+0000 7f820b9bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c599850 0x55895c958480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.540+0000 7f820b9bf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55895c7077b0 con 0x55895c570990 2024-09-21T00:28:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.540+0000 7f820b9bf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55895c84a650 con 0x55895c599850 2024-09-21T00:28:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.540+0000 7f820b9bf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55895c94f190 con 0x55895c578b70 2024-09-21T00:28:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.540+0000 7f820b9bf740 10 client.0 _ll_register_callbacks cb 0x55895c5b10a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.540+0000 7f81fd400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c94ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.540+0000 7f81fe800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c599850 0x55895c958480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.540+0000 7f81fd400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c94ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41786/0 (socket says 172.21.15.27:41786) 2024-09-21T00:28:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.540+0000 7f81fe800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c599850 0x55895c958480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:34042/0 (socket says 172.21.15.27:34042) 2024-09-21T00:28:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.540+0000 7f81fd400640 1 -- 172.21.15.27:0/2101224061 learned_addr learned my addr 172.21.15.27:0/2101224061 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.540+0000 7f81fde00640 1 --2- 172.21.15.27:0/2101224061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c570990 0x55895c94c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.541+0000 7f81fd400640 1 -- 172.21.15.27:0/2101224061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c599850 msgr2=0x55895c958480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.541+0000 7f81fd400640 1 --2- 172.21.15.27:0/2101224061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c599850 0x55895c958480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.541+0000 7f81fd400640 1 -- 172.21.15.27:0/2101224061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c570990 msgr2=0x55895c94c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.541+0000 7f81fd400640 1 --2- 172.21.15.27:0/2101224061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c570990 0x55895c94c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.541+0000 7f81fd400640 1 -- 172.21.15.27:0/2101224061 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895c613ba0 con 0x55895c578b70 2024-09-21T00:28:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.541+0000 7f81fd400640 1 --2- 172.21.15.27:0/2101224061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c94ca20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f81ec005c80 tx=0x7f81ec005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ed15f856c694b21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.541+0000 7f81dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:28:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.541+0000 7f81dfe00640 1 -- 172.21.15.27:0/2101224061 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec01d050 con 0x55895c578b70 2024-09-21T00:28:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.541+0000 7f820b9bf740 1 -- 172.21.15.27:0/2101224061 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81ec0041c0 con 0x55895c578b70 2024-09-21T00:28:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.541+0000 7f81dfe00640 1 -- 172.21.15.27:0/2101224061 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7f81ec016040 con 0x55895c578b70 2024-09-21T00:28:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.542+0000 7f81dfe00640 1 -- 172.21.15.27:0/2101224061 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f81ec031050 con 0x55895c578b70 2024-09-21T00:28:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.542+0000 7f820b9bf740 10 client.26197 fetch_fsmap learned FSMap version 34 2024-09-21T00:28:46.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.542+0000 7f820b9bf740 1 -- 172.21.15.27:0/2101224061 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55895c95b0b0 con 0x55895c578b70 2024-09-21T00:28:46.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.542+0000 7f81dfe00640 1 -- 172.21.15.27:0/2101224061 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ec0041c0 con 0x55895c578b70 2024-09-21T00:28:46.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.542+0000 7f81dfe00640 1 -- 172.21.15.27:0/2101224061 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f81ec004a60 con 0x55895c578b70 2024-09-21T00:28:46.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.542+0000 7f820b9bf740 10 client.26197 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T00:28:46.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.542+0000 7f820b9bf740 10 client.26197 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:28:46.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.542+0000 7f820b9bf740 20 client.26197 populate_metadata read hostname 'smithi027' 2024-09-21T00:28:46.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.542+0000 7f820b9bf740 1 --2- 172.21.15.27:0/2101224061 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55895c5581d0 0x55895c95e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.542+0000 7f820b9bf740 4 client.26197 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:28:46.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.542+0000 7f820b9bf740 1 -- 172.21.15.27:0/2101224061 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55895c8b89e0 con 0x55895c5581d0 2024-09-21T00:28:46.546 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:46 smithi027 ceph-mon[30745]: pgmap v602: 129 pgs: 129 active+clean; 969 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 615 KiB/s wr, 94 op/s 2024-09-21T00:28:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.543+0000 7f81fde00640 1 --2- 172.21.15.27:0/2101224061 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55895c5581d0 0x55895c95e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.543+0000 7f81fde00640 1 --2- 172.21.15.27:0/2101224061 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55895c5581d0 0x55895c95e520 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dee0840b5096f35 server_cookie=72fde6cb2a8055e0 in_seq=0 out_seq=0 2024-09-21T00:28:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.543+0000 7f81dfe00640 10 client.26197 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.544+0000 7f81dfe00640 1 -- 172.21.15.27:0/2101224061 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55895c8b89e0 con 0x55895c5581d0 2024-09-21T00:28:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.544+0000 7f81dfe00640 10 client.26197 handle_command_reply: tid=0 2024-09-21T00:28:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.544+0000 7f820b9bf740 1 client.26197 shutdown 2024-09-21T00:28:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.544+0000 7f820b9bf740 1 -- 172.21.15.27:0/2101224061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 msgr2=0x55895c94ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.544+0000 7f820b9bf740 1 --2- 172.21.15.27:0/2101224061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c94ca20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f81ec005c80 tx=0x7f81ec005720 comp rx=0 tx=0).stop 2024-09-21T00:28:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.545+0000 7f820b9bf740 1 -- 172.21.15.27:0/2101224061 shutdown_connections 2024-09-21T00:28:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.545+0000 7f820b9bf740 1 --2- 172.21.15.27:0/2101224061 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55895c5581d0 0x55895c95e520 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.545+0000 7f820b9bf740 1 --2- 172.21.15.27:0/2101224061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c599850 0x55895c958480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.545+0000 7f820b9bf740 1 --2- 172.21.15.27:0/2101224061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c94ca20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.545+0000 7f820b9bf740 1 --2- 172.21.15.27:0/2101224061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c570990 0x55895c94c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.545+0000 7f81dfe00640 0 client.26197 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.545+0000 7f820b9bf740 1 -- 172.21.15.27:0/2101224061 >> 172.21.15.27:0/2101224061 conn(0x55895c58dc40 msgr2=0x55895c943b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.545+0000 7f820b9bf740 1 -- 172.21.15.27:0/2101224061 shutdown_connections 2024-09-21T00:28:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.545+0000 7f820b9bf740 1 -- 172.21.15.27:0/2101224061 wait complete. 2024-09-21T00:28:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.545+0000 7f820b9bf740 20 client.26197 trim_cache size 0 max 16384 2024-09-21T00:28:46.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.563+0000 7f820b9bf740 1 Processor -- start 2024-09-21T00:28:46.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.563+0000 7f820b9bf740 1 -- start start 2024-09-21T00:28:46.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.564+0000 7f820b9bf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c570990 0x55895c93e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.564+0000 7f820b9bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c578b70 0x55895c9423a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.564+0000 7f820b9bf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c599850 0x55895c958380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.564+0000 7f820b9bf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f82001b5b00 con 0x55895c570990 2024-09-21T00:28:46.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.564+0000 7f820b9bf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f82001b3310 con 0x55895c578b70 2024-09-21T00:28:46.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.564+0000 7f820b9bf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f82001b1180 con 0x55895c599850 2024-09-21T00:28:46.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.564+0000 7f81fde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c570990 0x55895c93e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.564+0000 7f81fe800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c599850 0x55895c958380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.564+0000 7f81fe800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c599850 0x55895c958380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41796/0 (socket says 172.21.15.27:41796) 2024-09-21T00:28:46.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.564+0000 7f81fe800640 1 -- 172.21.15.27:0/2324069552 learned_addr learned my addr 172.21.15.27:0/2324069552 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:46.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.564+0000 7f81fde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c570990 0x55895c93e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57108/0 (socket says 172.21.15.27:57108) 2024-09-21T00:28:46.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.564+0000 7f81fd400640 1 --2- 172.21.15.27:0/2324069552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c578b70 0x55895c9423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.565+0000 7f81fe800640 1 -- 172.21.15.27:0/2324069552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c578b70 msgr2=0x55895c9423a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:46.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.565+0000 7f81fe800640 1 --2- 172.21.15.27:0/2324069552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c578b70 0x55895c9423a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.565+0000 7f81fe800640 1 -- 172.21.15.27:0/2324069552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c570990 msgr2=0x55895c93e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:46.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.565+0000 7f81fe800640 1 --2- 172.21.15.27:0/2324069552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c570990 0x55895c93e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.565+0000 7f81fe800640 1 -- 172.21.15.27:0/2324069552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895c94d380 con 0x55895c599850 2024-09-21T00:28:46.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.565+0000 7f81fde00640 1 --2- 172.21.15.27:0/2324069552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c570990 0x55895c93e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:28:46.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.565+0000 7f81fe800640 1 --2- 172.21.15.27:0/2324069552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c599850 0x55895c958380 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f81f8009be0 tx=0x7f81f8040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=506bfa2c003c2dcf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:46.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.565+0000 7f81eaa00640 1 -- 172.21.15.27:0/2324069552 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f8036040 con 0x55895c599850 2024-09-21T00:28:46.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.566+0000 7f81eaa00640 1 -- 172.21.15.27:0/2324069552 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f8050070 con 0x55895c599850 2024-09-21T00:28:46.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.566+0000 7f81eaa00640 1 -- 172.21.15.27:0/2324069552 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f8055870 con 0x55895c599850 2024-09-21T00:28:46.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.566+0000 7f820b9bf740 1 -- 172.21.15.27:0/2324069552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c599850 msgr2=0x55895c958380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.566+0000 7f820b9bf740 1 --2- 172.21.15.27:0/2324069552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c599850 0x55895c958380 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f81f8009be0 tx=0x7f81f8040000 comp rx=0 tx=0).stop 2024-09-21T00:28:46.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.566+0000 7f820b9bf740 1 -- 172.21.15.27:0/2324069552 shutdown_connections 2024-09-21T00:28:46.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.566+0000 7f820b9bf740 1 --2- 172.21.15.27:0/2324069552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c599850 0x55895c958380 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.566+0000 7f820b9bf740 1 --2- 172.21.15.27:0/2324069552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c578b70 0x55895c9423a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.566+0000 7f820b9bf740 1 --2- 172.21.15.27:0/2324069552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c570990 0x55895c93e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.566+0000 7f820b9bf740 1 -- 172.21.15.27:0/2324069552 >> 172.21.15.27:0/2324069552 conn(0x55895c58dc40 msgr2=0x55895c93d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:46.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.567+0000 7f820b9bf740 1 -- 172.21.15.27:0/2324069552 shutdown_connections 2024-09-21T00:28:46.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.567+0000 7f820b9bf740 1 -- 172.21.15.27:0/2324069552 wait complete. 2024-09-21T00:28:46.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.567+0000 7f820b9bf740 1 -- start start 2024-09-21T00:28:46.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.567+0000 7f820b9bf740 1 Processor -- start 2024-09-21T00:28:46.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.568+0000 7f820b9bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c570990 0x55895c9506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.568+0000 7f820b9bf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c950c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.568+0000 7f820b9bf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c599850 0x55895c9c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.568+0000 7f820b9bf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f82001b17d0 con 0x55895c599850 2024-09-21T00:28:46.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.568+0000 7f820b9bf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f82001b2c40 con 0x55895c570990 2024-09-21T00:28:46.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.568+0000 7f820b9bf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f82001b1ea0 con 0x55895c578b70 2024-09-21T00:28:46.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.568+0000 7f820b9bf740 10 client.0 _ll_register_callbacks cb 0x7f81f0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:46.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.568+0000 7f81fd400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c950c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.568+0000 7f81fd400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c950c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41798/0 (socket says 172.21.15.27:41798) 2024-09-21T00:28:46.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.568+0000 7f81fd400640 1 -- 172.21.15.27:0/807047038 learned_addr learned my addr 172.21.15.27:0/807047038 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:46.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.568+0000 7f81fe800640 1 --2- 172.21.15.27:0/807047038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c599850 0x55895c9c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.568+0000 7f81fde00640 1 --2- 172.21.15.27:0/807047038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c570990 0x55895c9506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.569+0000 7f81fd400640 1 -- 172.21.15.27:0/807047038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c570990 msgr2=0x55895c9506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:46.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.569+0000 7f81fd400640 1 --2- 172.21.15.27:0/807047038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c570990 0x55895c9506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.569+0000 7f81fd400640 1 -- 172.21.15.27:0/807047038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c599850 msgr2=0x55895c9c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:46.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.569+0000 7f81fd400640 1 --2- 172.21.15.27:0/807047038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c599850 0x55895c9c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.569+0000 7f81fd400640 1 -- 172.21.15.27:0/807047038 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895c9514a0 con 0x55895c578b70 2024-09-21T00:28:46.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.569+0000 7f81fd400640 1 --2- 172.21.15.27:0/807047038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c950c00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f81ec005a90 tx=0x7f81ec00fba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7652a3ba7de6730 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:46.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.569+0000 7f81dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:28:46.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.569+0000 7f81dfe00640 1 -- 172.21.15.27:0/807047038 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec01d040 con 0x55895c578b70 2024-09-21T00:28:46.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.569+0000 7f820b9bf740 1 -- 172.21.15.27:0/807047038 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81f8050070 con 0x55895c578b70 2024-09-21T00:28:46.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.569+0000 7f81dfe00640 1 -- 172.21.15.27:0/807047038 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7f81ec004660 con 0x55895c578b70 2024-09-21T00:28:46.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.570+0000 7f81dfe00640 1 -- 172.21.15.27:0/807047038 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f81ec042050 con 0x55895c578b70 2024-09-21T00:28:46.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.570+0000 7f820b9bf740 10 client.26203 fetch_fsmap learned FSMap version 34 2024-09-21T00:28:46.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.570+0000 7f820b9bf740 1 -- 172.21.15.27:0/807047038 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55895c9c9320 con 0x55895c578b70 2024-09-21T00:28:46.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.570+0000 7f81dfe00640 1 -- 172.21.15.27:0/807047038 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f8050070 con 0x55895c578b70 2024-09-21T00:28:46.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.570+0000 7f81dfe00640 1 -- 172.21.15.27:0/807047038 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f81ec03d030 con 0x55895c578b70 2024-09-21T00:28:46.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.570+0000 7f820b9bf740 10 client.26203 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T00:28:46.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.570+0000 7f820b9bf740 10 client.26203 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:28:46.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.570+0000 7f820b9bf740 20 client.26203 populate_metadata read hostname 'smithi027' 2024-09-21T00:28:46.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.570+0000 7f820b9bf740 1 --2- 172.21.15.27:0/807047038 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55895c52d580 0x55895c9cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.570+0000 7f820b9bf740 4 client.26203 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}] 2024-09-21T00:28:46.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.570+0000 7f820b9bf740 1 -- 172.21.15.27:0/807047038 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}) -- 0x55895c8b4490 con 0x55895c52d580 2024-09-21T00:28:46.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.571+0000 7f81fde00640 1 --2- 172.21.15.27:0/807047038 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55895c52d580 0x55895c9cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.571+0000 7f81fde00640 1 --2- 172.21.15.27:0/807047038 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55895c52d580 0x55895c9cbba0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36eb73f25c65573b server_cookie=b414a58f1eb2ce3e in_seq=0 out_seq=0 2024-09-21T00:28:46.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:28:46.571+0000 7f81dfe00640 10 client.26203 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:46.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:46 smithi089 ceph-mon[34481]: pgmap v602: 129 pgs: 129 active+clean; 969 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 615 KiB/s wr, 94 op/s 2024-09-21T00:28:46.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:46 smithi098 ceph-mon[34725]: pgmap v602: 129 pgs: 129 active+clean; 969 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 615 KiB/s wr, 94 op/s 2024-09-21T00:28:48.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:48 smithi027 ceph-mon[30745]: pgmap v603: 129 pgs: 129 active+clean; 971 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 921 KiB/s wr, 132 op/s 2024-09-21T00:28:48.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:48 smithi089 ceph-mon[34481]: pgmap v603: 129 pgs: 129 active+clean; 971 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 921 KiB/s wr, 132 op/s 2024-09-21T00:28:48.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:48 smithi098 ceph-mon[34725]: pgmap v603: 129 pgs: 129 active+clean; 971 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 921 KiB/s wr, 132 op/s 2024-09-21T00:28:50.377 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:50 smithi027 ceph-mon[30745]: pgmap v604: 129 pgs: 129 active+clean; 980 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-09-21T00:28:50.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:50 smithi089 ceph-mon[34481]: pgmap v604: 129 pgs: 129 active+clean; 980 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-09-21T00:28:50.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:50 smithi098 ceph-mon[34725]: pgmap v604: 129 pgs: 129 active+clean; 980 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-09-21T00:28:52.328 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:52 smithi027 ceph-mon[30745]: pgmap v605: 129 pgs: 129 active+clean; 981 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-09-21T00:28:52.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:52 smithi089 ceph-mon[34481]: pgmap v605: 129 pgs: 129 active+clean; 981 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-09-21T00:28:52.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:52 smithi098 ceph-mon[34725]: pgmap v605: 129 pgs: 129 active+clean; 981 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-09-21T00:28:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 -- 172.21.15.27:0/2614256582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc55010bf80 msgr2=0x7fc5501127b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 --2- 172.21.15.27:0/2614256582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc55010bf80 0x7fc5501127b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc548009be0 tx=0x7fc54802a6c0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 -- 172.21.15.27:0/2614256582 shutdown_connections 2024-09-21T00:28:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 --2- 172.21.15.27:0/2614256582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc55010bf80 0x7fc5501127b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 --2- 172.21.15.27:0/2614256582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc55010b630 0x7fc55010ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 --2- 172.21.15.27:0/2614256582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc550105d00 0x7fc550106110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 -- 172.21.15.27:0/2614256582 >> 172.21.15.27:0/2614256582 conn(0x7fc550101720 msgr2=0x7fc550103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 -- 172.21.15.27:0/2614256582 shutdown_connections 2024-09-21T00:28:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 -- 172.21.15.27:0/2614256582 wait complete. 2024-09-21T00:28:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 Processor -- start 2024-09-21T00:28:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 -- start start 2024-09-21T00:28:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550105d00 0x7fc55019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc55010b630 0x7fc55019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc55010bf80 0x7fc5501a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc550115c40 con 0x7fc55010b630 2024-09-21T00:28:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc550115ac0 con 0x7fc550105d00 2024-09-21T00:28:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc550115dc0 con 0x7fc55010bf80 2024-09-21T00:28:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc54d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc55010b630 0x7fc55019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc54d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc55010b630 0x7fc55019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46176/0 (socket says 172.21.15.27:46176) 2024-09-21T00:28:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc54de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550105d00 0x7fc55019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc54d400640 1 -- 172.21.15.27:0/2929429253 learned_addr learned my addr 172.21.15.27:0/2929429253 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc54e800640 1 --2- 172.21.15.27:0/2929429253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc55010bf80 0x7fc5501a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc54de00640 1 -- 172.21.15.27:0/2929429253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc55010bf80 msgr2=0x7fc5501a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc54de00640 1 --2- 172.21.15.27:0/2929429253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc55010bf80 0x7fc5501a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc54de00640 1 -- 172.21.15.27:0/2929429253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc55010b630 msgr2=0x7fc55019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc54de00640 1 --2- 172.21.15.27:0/2929429253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc55010b630 0x7fc55019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc54de00640 1 -- 172.21.15.27:0/2929429253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5501a2550 con 0x7fc550105d00 2024-09-21T00:28:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc54d400640 1 --2- 172.21.15.27:0/2929429253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc55010b630 0x7fc55019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:28:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc54de00640 1 --2- 172.21.15.27:0/2929429253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550105d00 0x7fc55019c620 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc540005fd0 tx=0x7fc54000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc546a00640 1 -- 172.21.15.27:0/2929429253 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54000cd00 con 0x7fc550105d00 2024-09-21T00:28:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 -- 172.21.15.27:0/2929429253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5501a2840 con 0x7fc550105d00 2024-09-21T00:28:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc546a00640 1 -- 172.21.15.27:0/2929429253 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5400035f0 con 0x7fc550105d00 2024-09-21T00:28:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc546a00640 1 -- 172.21.15.27:0/2929429253 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540010670 con 0x7fc550105d00 2024-09-21T00:28:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.592+0000 7fc555200640 1 -- 172.21.15.27:0/2929429253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5501a8fd0 con 0x7fc550105d00 2024-09-21T00:28:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc55c3d2740 1 Processor -- start 2024-09-21T00:28:53.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc55c3d2740 1 -- start start 2024-09-21T00:28:53.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc546a00640 1 -- 172.21.15.27:0/2929429253 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc5400037a0 con 0x7fc550105d00 2024-09-21T00:28:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc55c3d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00301b0 0x5618f03d4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc55c3d2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f00072f0 0x5618f03d5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc55c3d2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f03d56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc55c3d2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5618f034a6b0 con 0x5618f000f4d0 2024-09-21T00:28:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc55c3d2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5618f0339ae0 con 0x5618f00072f0 2024-09-21T00:28:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc55c3d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5618f033d8f0 con 0x5618f00301b0 2024-09-21T00:28:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc54e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00301b0 0x5618f03d4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc54d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f03d56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc54e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00301b0 0x5618f03d4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41242/0 (socket says 172.21.15.27:41242) 2024-09-21T00:28:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc54e800640 1 -- 172.21.15.27:0/1028174570 learned_addr learned my addr 172.21.15.27:0/1028174570 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc54de00640 1 --2- 172.21.15.27:0/1028174570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f00072f0 0x5618f03d5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc54e800640 1 -- 172.21.15.27:0/1028174570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f00072f0 msgr2=0x5618f03d5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc54e800640 1 --2- 172.21.15.27:0/1028174570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f00072f0 0x5618f03d5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc54e800640 1 -- 172.21.15.27:0/1028174570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 msgr2=0x5618f03d56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc54e800640 1 --2- 172.21.15.27:0/1028174570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f03d56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc54e800640 1 -- 172.21.15.27:0/1028174570 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f0306780 con 0x5618f00301b0 2024-09-21T00:28:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc54e800640 1 --2- 172.21.15.27:0/1028174570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00301b0 0x5618f03d4c30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc548009be0 tx=0x7fc548030420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a20823fa80c6494e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc53b400640 1 -- 172.21.15.27:0/1028174570 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc548041070 con 0x5618f00301b0 2024-09-21T00:28:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc53b400640 1 -- 172.21.15.27:0/1028174570 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc548003e20 con 0x5618f00301b0 2024-09-21T00:28:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc53b400640 1 -- 172.21.15.27:0/1028174570 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc548036040 con 0x5618f00301b0 2024-09-21T00:28:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.597+0000 7fc546a00640 1 --2- 172.21.15.27:0/2929429253 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc5340b6700 0x7fc5340b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc546a00640 1 -- 172.21.15.27:0/2929429253 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7fc5400e7cd0 con 0x7fc550105d00 2024-09-21T00:28:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc54de00640 1 --2- 172.21.15.27:0/2929429253 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc5340b6700 0x7fc5340b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 -- 172.21.15.27:0/1028174570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00301b0 msgr2=0x5618f03d4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/1028174570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00301b0 0x5618f03d4c30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc548009be0 tx=0x7fc548030420 comp rx=0 tx=0).stop 2024-09-21T00:28:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 -- 172.21.15.27:0/1028174570 shutdown_connections 2024-09-21T00:28:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/1028174570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f03d56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/1028174570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f00072f0 0x5618f03d5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/1028174570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00301b0 0x5618f03d4c30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 -- 172.21.15.27:0/1028174570 >> 172.21.15.27:0/1028174570 conn(0x5618f0024300 msgr2=0x5618f03d2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 -- 172.21.15.27:0/1028174570 shutdown_connections 2024-09-21T00:28:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 -- 172.21.15.27:0/1028174570 wait complete. 2024-09-21T00:28:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc54de00640 1 --2- 172.21.15.27:0/2929429253 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc5340b6700 0x7fc5340b8c10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5618f03d38b0 tx=0x7fc540000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 -- start start 2024-09-21T00:28:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 Processor -- start 2024-09-21T00:28:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f00072f0 0x5618f03e0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f03e07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00301b0 0x5618f03e0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5618f0271ab0 con 0x5618f000f4d0 2024-09-21T00:28:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5618f02e0570 con 0x5618f00072f0 2024-09-21T00:28:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5618f03e3350 con 0x5618f00301b0 2024-09-21T00:28:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 10 client.0 _ll_register_callbacks cb 0x5618f02ef100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc54d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f03e07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc54d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f03e07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46202/0 (socket says 172.21.15.27:46202) 2024-09-21T00:28:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc54d400640 1 -- 172.21.15.27:0/938754978 learned_addr learned my addr 172.21.15.27:0/938754978 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc54d400640 1 -- 172.21.15.27:0/938754978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00301b0 msgr2=0x5618f03e0d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:28:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc54d400640 1 --2- 172.21.15.27:0/938754978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00301b0 0x5618f03e0d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc54d400640 1 -- 172.21.15.27:0/938754978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f00072f0 msgr2=0x5618f03e0260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:28:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc54d400640 1 --2- 172.21.15.27:0/938754978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f00072f0 0x5618f03e0260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc54d400640 1 -- 172.21.15.27:0/938754978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f009b790 con 0x5618f000f4d0 2024-09-21T00:28:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc54d400640 1 --2- 172.21.15.27:0/938754978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f03e07a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc53c012d20 tx=0x7fc53c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92d62dfc0bc0d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc52fe00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:28:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc52fe00640 1 -- 172.21.15.27:0/938754978 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c00fe20 con 0x5618f000f4d0 2024-09-21T00:28:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc52fe00640 1 -- 172.21.15.27:0/938754978 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7fc53c0047a0 con 0x5618f000f4d0 2024-09-21T00:28:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 -- 172.21.15.27:0/938754978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc548003e20 con 0x5618f000f4d0 2024-09-21T00:28:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc52fe00640 1 -- 172.21.15.27:0/938754978 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc53c0049a0 con 0x5618f000f4d0 2024-09-21T00:28:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 10 client.16167 fetch_fsmap learned FSMap version 34 2024-09-21T00:28:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 -- 172.21.15.27:0/938754978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5618f0114620 con 0x5618f000f4d0 2024-09-21T00:28:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc52fe00640 1 -- 172.21.15.27:0/938754978 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc548003e20 con 0x5618f000f4d0 2024-09-21T00:28:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc52fe00640 1 -- 172.21.15.27:0/938754978 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fc53c010550 con 0x5618f000f4d0 2024-09-21T00:28:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 10 client.16167 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T00:28:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:28:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 20 client.16167 populate_metadata read hostname 'smithi027' 2024-09-21T00:28:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/938754978 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5618effeed80 0x5618f03f2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 4 client.16167 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:28:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.603+0000 7fc55c3d2740 1 -- 172.21.15.27:0/938754978 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618f034c600 con 0x5618effeed80 2024-09-21T00:28:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.608+0000 7fc54e800640 1 --2- 172.21.15.27:0/938754978 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5618effeed80 0x5618f03f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc54e800640 1 --2- 172.21.15.27:0/938754978 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5618effeed80 0x5618f03f2570 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade6f741754cbd01 server_cookie=2d63b5cda4db67c2 in_seq=0 out_seq=0 2024-09-21T00:28:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc52fe00640 10 client.16167 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc52fe00640 1 -- 172.21.15.27:0/938754978 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5618f034c600 con 0x5618effeed80 2024-09-21T00:28:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc52fe00640 10 client.16167 handle_command_reply: tid=0 2024-09-21T00:28:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc55c3d2740 1 client.16167 shutdown 2024-09-21T00:28:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc55c3d2740 1 -- 172.21.15.27:0/938754978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 msgr2=0x5618f03e07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/938754978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f03e07a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc53c012d20 tx=0x7fc53c00f790 comp rx=0 tx=0).stop 2024-09-21T00:28:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc55c3d2740 1 -- 172.21.15.27:0/938754978 shutdown_connections 2024-09-21T00:28:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/938754978 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5618effeed80 0x5618f03f2570 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/938754978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00301b0 0x5618f03e0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/938754978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f03e07a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/938754978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f00072f0 0x5618f03e0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc52fe00640 0 client.16167 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc55c3d2740 1 -- 172.21.15.27:0/938754978 >> 172.21.15.27:0/938754978 conn(0x5618f0024300 msgr2=0x5618f03d5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc55c3d2740 1 -- 172.21.15.27:0/938754978 shutdown_connections 2024-09-21T00:28:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc55c3d2740 1 -- 172.21.15.27:0/938754978 wait complete. 2024-09-21T00:28:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.632+0000 7fc55c3d2740 20 client.16167 trim_cache size 0 max 16384 2024-09-21T00:28:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 Processor -- start 2024-09-21T00:28:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 -- start start 2024-09-21T00:28:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00072f0 0x5618f03d2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f000f4d0 0x5618f03d6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f00301b0 0x5618f03e1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc5501b5b10 con 0x5618f00301b0 2024-09-21T00:28:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc5501b3320 con 0x5618f000f4d0 2024-09-21T00:28:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc5501b1190 con 0x5618f00072f0 2024-09-21T00:28:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc54e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f00301b0 0x5618f03e1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc54e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f00301b0 0x5618f03e1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46204/0 (socket says 172.21.15.27:46204) 2024-09-21T00:28:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc54e800640 1 -- 172.21.15.27:0/2748352799 learned_addr learned my addr 172.21.15.27:0/2748352799 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc54d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f000f4d0 0x5618f03d6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc54d400640 1 -- 172.21.15.27:0/2748352799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00072f0 msgr2=0x5618f03d2970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:28:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc54d400640 1 --2- 172.21.15.27:0/2748352799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00072f0 0x5618f03d2970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc54d400640 1 -- 172.21.15.27:0/2748352799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f00301b0 msgr2=0x5618f03e1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc54d400640 1 --2- 172.21.15.27:0/2748352799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f00301b0 0x5618f03e1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc54d400640 1 -- 172.21.15.27:0/2748352799 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f03e0ad0 con 0x5618f000f4d0 2024-09-21T00:28:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc54e800640 1 --2- 172.21.15.27:0/2748352799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f00301b0 0x5618f03e1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:28:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc54d400640 1 --2- 172.21.15.27:0/2748352799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f000f4d0 0x5618f03d6620 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc53c001eb0 tx=0x7fc53c0129b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e49e7d9ab01a1acd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc53aa00640 1 -- 172.21.15.27:0/2748352799 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c00fea0 con 0x5618f000f4d0 2024-09-21T00:28:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc53aa00640 1 -- 172.21.15.27:0/2748352799 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc53c010be0 con 0x5618f000f4d0 2024-09-21T00:28:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc53aa00640 1 -- 172.21.15.27:0/2748352799 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c0035f0 con 0x5618f000f4d0 2024-09-21T00:28:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 -- 172.21.15.27:0/2748352799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f000f4d0 msgr2=0x5618f03d6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/2748352799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f000f4d0 0x5618f03d6620 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc53c001eb0 tx=0x7fc53c0129b0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 -- 172.21.15.27:0/2748352799 shutdown_connections 2024-09-21T00:28:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/2748352799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f00301b0 0x5618f03e1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/2748352799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f000f4d0 0x5618f03d6620 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/2748352799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00072f0 0x5618f03d2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 -- 172.21.15.27:0/2748352799 >> 172.21.15.27:0/2748352799 conn(0x5618f0024300 msgr2=0x5618f03e1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 -- 172.21.15.27:0/2748352799 shutdown_connections 2024-09-21T00:28:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 -- 172.21.15.27:0/2748352799 wait complete. 2024-09-21T00:28:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 -- start start 2024-09-21T00:28:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 Processor -- start 2024-09-21T00:28:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f00072f0 0x5618f0454210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f04547d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00301b0 0x5618f03e7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc5501b17e0 con 0x5618f000f4d0 2024-09-21T00:28:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc5501b2c50 con 0x5618f00072f0 2024-09-21T00:28:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc5501b1eb0 con 0x5618f00301b0 2024-09-21T00:28:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc55c3d2740 10 client.0 _ll_register_callbacks cb 0x7fc53c009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc54d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f04547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc54d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f04547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46212/0 (socket says 172.21.15.27:46212) 2024-09-21T00:28:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.650+0000 7fc54d400640 1 -- 172.21.15.27:0/3994183360 learned_addr learned my addr 172.21.15.27:0/3994183360 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:28:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc54d400640 1 -- 172.21.15.27:0/3994183360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00301b0 msgr2=0x5618f03e7e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:28:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc54d400640 1 --2- 172.21.15.27:0/3994183360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00301b0 0x5618f03e7e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc54d400640 1 -- 172.21.15.27:0/3994183360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f00072f0 msgr2=0x5618f0454210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:28:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc54d400640 1 --2- 172.21.15.27:0/3994183360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f00072f0 0x5618f0454210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc54d400640 1 -- 172.21.15.27:0/3994183360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f03e85b0 con 0x5618f000f4d0 2024-09-21T00:28:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc54d400640 1 --2- 172.21.15.27:0/3994183360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f04547d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00fd90 tx=0x7fc53c010a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69b4ea4c226f9ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc52fe00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:28:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc52fe00640 1 -- 172.21.15.27:0/3994183360 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c037060 con 0x5618f000f4d0 2024-09-21T00:28:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc52fe00640 1 -- 172.21.15.27:0/3994183360 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7fc53c004220 con 0x5618f000f4d0 2024-09-21T00:28:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc55c3d2740 1 -- 172.21.15.27:0/3994183360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc53c010be0 con 0x5618f000f4d0 2024-09-21T00:28:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc52fe00640 1 -- 172.21.15.27:0/3994183360 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc53c02e040 con 0x5618f000f4d0 2024-09-21T00:28:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc55c3d2740 10 client.16173 fetch_fsmap learned FSMap version 34 2024-09-21T00:28:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc55c3d2740 1 -- 172.21.15.27:0/3994183360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5618f045caa0 con 0x5618f000f4d0 2024-09-21T00:28:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc52fe00640 1 -- 172.21.15.27:0/3994183360 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc53c010be0 con 0x5618f000f4d0 2024-09-21T00:28:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc52fe00640 1 -- 172.21.15.27:0/3994183360 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fc53c031030 con 0x5618f000f4d0 2024-09-21T00:28:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc55c3d2740 10 client.16173 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T00:28:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc55c3d2740 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:28:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc55c3d2740 20 client.16173 populate_metadata read hostname 'smithi027' 2024-09-21T00:28:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/3994183360 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5618effc41c0 0x5618f045f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc55c3d2740 4 client.16173 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:28:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.655+0000 7fc55c3d2740 1 -- 172.21.15.27:0/3994183360 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618f03480b0 con 0x5618effc41c0 2024-09-21T00:28:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.691+0000 7fc54e800640 1 --2- 172.21.15.27:0/3994183360 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5618effc41c0 0x5618f045f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.761+0000 7fc54e800640 1 --2- 172.21.15.27:0/3994183360 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5618effc41c0 0x5618f045f170 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56b73371504952db server_cookie=124e0a3039c617f3 in_seq=0 out_seq=0 2024-09-21T00:28:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.761+0000 7fc52fe00640 10 client.16173 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc52fe00640 1 -- 172.21.15.27:0/3994183360 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5618f03480b0 con 0x5618effc41c0 2024-09-21T00:28:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc52fe00640 10 client.16173 handle_command_reply: tid=0 2024-09-21T00:28:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc55c3d2740 1 client.16173 shutdown 2024-09-21T00:28:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc55c3d2740 1 -- 172.21.15.27:0/3994183360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 msgr2=0x5618f04547d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/3994183360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f04547d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00fd90 tx=0x7fc53c010a70 comp rx=0 tx=0).stop 2024-09-21T00:28:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc55c3d2740 1 -- 172.21.15.27:0/3994183360 shutdown_connections 2024-09-21T00:28:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/3994183360 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5618effc41c0 0x5618f045f170 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/3994183360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618f00301b0 0x5618f03e7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/3994183360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5618f000f4d0 0x5618f04547d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc55c3d2740 1 --2- 172.21.15.27:0/3994183360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5618f00072f0 0x5618f0454210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc55c3d2740 1 -- 172.21.15.27:0/3994183360 >> 172.21.15.27:0/3994183360 conn(0x5618f0024300 msgr2=0x5618f03e2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc52fe00640 0 client.16173 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:28:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc55c3d2740 1 -- 172.21.15.27:0/3994183360 shutdown_connections 2024-09-21T00:28:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc55c3d2740 1 -- 172.21.15.27:0/3994183360 wait complete. 2024-09-21T00:28:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc55c3d2740 20 client.16173 trim_cache size 0 max 16384 2024-09-21T00:28:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:28:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:28:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:28:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:28:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc555200640 1 -- 172.21.15.27:0/2929429253 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc5340b6700 msgr2=0x7fc5340b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc555200640 1 --2- 172.21.15.27:0/2929429253 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc5340b6700 0x7fc5340b8c10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5618f03d38b0 tx=0x7fc540000bc0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc555200640 1 -- 172.21.15.27:0/2929429253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550105d00 msgr2=0x7fc55019c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc555200640 1 --2- 172.21.15.27:0/2929429253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550105d00 0x7fc55019c620 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc540005fd0 tx=0x7fc54000ee50 comp rx=0 tx=0).stop 2024-09-21T00:28:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc555200640 1 -- 172.21.15.27:0/2929429253 shutdown_connections 2024-09-21T00:28:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc555200640 1 --2- 172.21.15.27:0/2929429253 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc5340b6700 0x7fc5340b8c10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc555200640 1 --2- 172.21.15.27:0/2929429253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc55010bf80 0x7fc5501a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc555200640 1 --2- 172.21.15.27:0/2929429253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc55010b630 0x7fc55019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc555200640 1 --2- 172.21.15.27:0/2929429253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550105d00 0x7fc55019c620 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc555200640 1 -- 172.21.15.27:0/2929429253 >> 172.21.15.27:0/2929429253 conn(0x7fc550101720 msgr2=0x7fc550103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc555200640 1 -- 172.21.15.27:0/2929429253 shutdown_connections 2024-09-21T00:28:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:28:53.975+0000 7fc555200640 1 -- 172.21.15.27:0/2929429253 wait complete. 2024-09-21T00:28:54.329 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:54 smithi027 ceph-mon[30745]: pgmap v606: 129 pgs: 129 active+clean; 983 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-09-21T00:28:54.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:54 smithi089 ceph-mon[34481]: pgmap v606: 129 pgs: 129 active+clean; 983 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-09-21T00:28:54.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:54 smithi098 ceph-mon[34725]: pgmap v606: 129 pgs: 129 active+clean; 983 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-09-21T00:28:56.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:56 smithi027 ceph-mon[30745]: pgmap v607: 129 pgs: 129 active+clean; 983 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 197 op/s 2024-09-21T00:28:56.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:56 smithi089 ceph-mon[34481]: pgmap v607: 129 pgs: 129 active+clean; 983 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 197 op/s 2024-09-21T00:28:56.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:56 smithi098 ceph-mon[34725]: pgmap v607: 129 pgs: 129 active+clean; 983 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 197 op/s 2024-09-21T00:28:58.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:28:58 smithi027 ceph-mon[30745]: pgmap v608: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-09-21T00:28:58.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:28:58 smithi089 ceph-mon[34481]: pgmap v608: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-09-21T00:28:58.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:28:58 smithi098 ceph-mon[34725]: pgmap v608: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-09-21T00:29:00.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:00 smithi027 ceph-mon[30745]: pgmap v609: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-09-21T00:29:00.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:00 smithi089 ceph-mon[34481]: pgmap v609: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-09-21T00:29:00.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:00 smithi098 ceph-mon[34725]: pgmap v609: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-09-21T00:29:01.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:01 smithi027 ceph-mon[30745]: pgmap v610: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 339 KiB/s wr, 129 op/s 2024-09-21T00:29:01.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:01 smithi089 ceph-mon[34481]: pgmap v610: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 339 KiB/s wr, 129 op/s 2024-09-21T00:29:01.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:01 smithi098 ceph-mon[34725]: pgmap v610: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 339 KiB/s wr, 129 op/s 2024-09-21T00:29:03.703 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:03 smithi027 ceph-mon[30745]: pgmap v611: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 230 KiB/s wr, 77 op/s 2024-09-21T00:29:03.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:03 smithi089 ceph-mon[34481]: pgmap v611: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 230 KiB/s wr, 77 op/s 2024-09-21T00:29:03.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:03 smithi098 ceph-mon[34725]: pgmap v611: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 230 KiB/s wr, 77 op/s 2024-09-21T00:29:05.712 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:05 smithi027 ceph-mon[30745]: pgmap v612: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 71 KiB/s wr, 47 op/s 2024-09-21T00:29:05.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:05 smithi089 ceph-mon[34481]: pgmap v612: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 71 KiB/s wr, 47 op/s 2024-09-21T00:29:05.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:05 smithi098 ceph-mon[34725]: pgmap v612: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 71 KiB/s wr, 47 op/s 2024-09-21T00:29:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 -- 172.21.15.27:0/3629810108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe4103920 msgr2=0x7fbfe4103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 --2- 172.21.15.27:0/3629810108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe4103920 0x7fbfe4103d30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009900 tx=0x7fbfd802a410 comp rx=0 tx=0).stop 2024-09-21T00:29:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 -- 172.21.15.27:0/3629810108 shutdown_connections 2024-09-21T00:29:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 --2- 172.21.15.27:0/3629810108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfe4109ba0 0x7fbfe41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 --2- 172.21.15.27:0/3629810108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfe4109250 0x7fbfe4109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 --2- 172.21.15.27:0/3629810108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe4103920 0x7fbfe4103d30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 -- 172.21.15.27:0/3629810108 >> 172.21.15.27:0/3629810108 conn(0x7fbfe40ff340 msgr2=0x7fbfe4101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 -- 172.21.15.27:0/3629810108 shutdown_connections 2024-09-21T00:29:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 -- 172.21.15.27:0/3629810108 wait complete. 2024-09-21T00:29:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 Processor -- start 2024-09-21T00:29:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 -- start start 2024-09-21T00:29:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfe4103920 0x7fbfe419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe4109250 0x7fbfe419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfe4109ba0 0x7fbfe41a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4113910 con 0x7fbfe4109250 2024-09-21T00:29:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4113790 con 0x7fbfe4109ba0 2024-09-21T00:29:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.185+0000 7fbfea600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4113a90 con 0x7fbfe4103920 2024-09-21T00:29:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe4109250 0x7fbfe419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe9200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfe4103920 0x7fbfe419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe4109250 0x7fbfe419cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55536/0 (socket says 172.21.15.27:55536) 2024-09-21T00:29:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe9200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfe4103920 0x7fbfe419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:53352/0 (socket says 172.21.15.27:53352) 2024-09-21T00:29:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe3e00640 1 -- 172.21.15.27:0/300018643 learned_addr learned my addr 172.21.15.27:0/300018643 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe9c00640 1 --2- 172.21.15.27:0/300018643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfe4109ba0 0x7fbfe41a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe3e00640 1 -- 172.21.15.27:0/300018643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfe4103920 msgr2=0x7fbfe419c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe3e00640 1 --2- 172.21.15.27:0/300018643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfe4103920 0x7fbfe419c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe3e00640 1 -- 172.21.15.27:0/300018643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfe4109ba0 msgr2=0x7fbfe41a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe3e00640 1 --2- 172.21.15.27:0/300018643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfe4109ba0 0x7fbfe41a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe3e00640 1 -- 172.21.15.27:0/300018643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe41a2440 con 0x7fbfe4109250 2024-09-21T00:29:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe9c00640 1 --2- 172.21.15.27:0/300018643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfe4109ba0 0x7fbfe41a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe3e00640 1 --2- 172.21.15.27:0/300018643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe4109250 0x7fbfe419cab0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000b450 tx=0x7fbfd000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe1600640 1 -- 172.21.15.27:0/300018643 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0015020 con 0x7fbfe4109250 2024-09-21T00:29:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfea600640 1 -- 172.21.15.27:0/300018643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe41a2730 con 0x7fbfe4109250 2024-09-21T00:29:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe1600640 1 -- 172.21.15.27:0/300018643 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfd0011d40 con 0x7fbfe4109250 2024-09-21T00:29:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfe1600640 1 -- 172.21.15.27:0/300018643 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd000f340 con 0x7fbfe4109250 2024-09-21T00:29:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.191+0000 7fbfea600640 1 -- 172.21.15.27:0/300018643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe41a8ec0 con 0x7fbfe4109250 2024-09-21T00:29:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.196+0000 7fbff2c7d740 1 Processor -- start 2024-09-21T00:29:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.196+0000 7fbff2c7d740 1 -- start start 2024-09-21T00:29:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.197+0000 7fbff2c7d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b1bde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.197+0000 7fbff2c7d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6adf02f0 0x557b6b1be350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.197+0000 7fbff2c7d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf84d0 0x557b6b1be890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.197+0000 7fbff2c7d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557b6b1336b0 con 0x557b6ae191b0 2024-09-21T00:29:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.197+0000 7fbff2c7d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557b6b122ae0 con 0x557b6adf02f0 2024-09-21T00:29:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.197+0000 7fbff2c7d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557b6b1268f0 con 0x557b6adf84d0 2024-09-21T00:29:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.197+0000 7fbfe9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b1bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.197+0000 7fbfe1600640 1 -- 172.21.15.27:0/300018643 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fbfd0018020 con 0x7fbfe4109250 2024-09-21T00:29:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.197+0000 7fbfe9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b1bde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55548/0 (socket says 172.21.15.27:55548) 2024-09-21T00:29:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.197+0000 7fbfe9c00640 1 -- 172.21.15.27:0/1953215868 learned_addr learned my addr 172.21.15.27:0/1953215868 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.197+0000 7fbfe9c00640 1 -- 172.21.15.27:0/1953215868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf84d0 msgr2=0x557b6b1be890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.197+0000 7fbfe9c00640 1 --2- 172.21.15.27:0/1953215868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf84d0 0x557b6b1be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.197+0000 7fbfe9c00640 1 -- 172.21.15.27:0/1953215868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6adf02f0 msgr2=0x557b6b1be350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.197+0000 7fbfe9c00640 1 --2- 172.21.15.27:0/1953215868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6adf02f0 0x557b6b1be350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.198+0000 7fbfe9c00640 1 -- 172.21.15.27:0/1953215868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b6b0ef780 con 0x557b6ae191b0 2024-09-21T00:29:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.198+0000 7fbfe9c00640 1 --2- 172.21.15.27:0/1953215868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b1bde10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009fd0 tx=0x7fbfdc0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2801c367f7816066 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.198+0000 7fbfd6000640 1 -- 172.21.15.27:0/1953215868 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc011760 con 0x557b6ae191b0 2024-09-21T00:29:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.198+0000 7fbfd6000640 1 -- 172.21.15.27:0/1953215868 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfdc011db0 con 0x557b6ae191b0 2024-09-21T00:29:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.198+0000 7fbfd6000640 1 -- 172.21.15.27:0/1953215868 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc00e3c0 con 0x557b6ae191b0 2024-09-21T00:29:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.198+0000 7fbfe1600640 1 --2- 172.21.15.27:0/300018643 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbfcc0b66f0 0x7fbfcc0b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.200+0000 7fbfe1600640 1 -- 172.21.15.27:0/300018643 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7fbfd00e7420 con 0x7fbfe4109250 2024-09-21T00:29:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.200+0000 7fbfe9200640 1 --2- 172.21.15.27:0/300018643 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbfcc0b66f0 0x7fbfcc0b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.200+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1953215868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 msgr2=0x557b6b1bde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.200+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/1953215868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b1bde10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009fd0 tx=0x7fbfdc0108e0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.200+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1953215868 shutdown_connections 2024-09-21T00:29:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.200+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/1953215868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf84d0 0x557b6b1be890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.200+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/1953215868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6adf02f0 0x557b6b1be350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.200+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/1953215868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b1bde10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.200+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1953215868 >> 172.21.15.27:0/1953215868 conn(0x557b6ae0d300 msgr2=0x557b6b1bbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.201+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1953215868 shutdown_connections 2024-09-21T00:29:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.201+0000 7fbfe9200640 1 --2- 172.21.15.27:0/300018643 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbfcc0b66f0 0x7fbfcc0b8c00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8005ab0 tx=0x7fbfd8005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.201+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1953215868 wait complete. 2024-09-21T00:29:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.201+0000 7fbff2c7d740 1 -- start start 2024-09-21T00:29:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 1 Processor -- start 2024-09-21T00:29:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf02f0 0x557b6b1c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6adf84d0 0x557b6b1c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b1c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557b6b05aab0 con 0x557b6ae191b0 2024-09-21T00:29:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557b6b0c9570 con 0x557b6adf84d0 2024-09-21T00:29:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557b6b1cc590 con 0x557b6adf02f0 2024-09-21T00:29:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 10 client.0 _ll_register_callbacks cb 0x557b6b0d8100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfe9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b1c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfe9200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf02f0 0x557b6b1c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfe9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b1c9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55558/0 (socket says 172.21.15.27:55558) 2024-09-21T00:29:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfe9c00640 1 -- 172.21.15.27:0/3700027481 learned_addr learned my addr 172.21.15.27:0/3700027481 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfe9200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf02f0 0x557b6b1c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:53376/0 (socket says 172.21.15.27:53376) 2024-09-21T00:29:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfe9c00640 1 -- 172.21.15.27:0/3700027481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf02f0 msgr2=0x557b6b1c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfe9c00640 1 --2- 172.21.15.27:0/3700027481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf02f0 0x557b6b1c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfe9c00640 1 -- 172.21.15.27:0/3700027481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6adf84d0 msgr2=0x557b6b1c9960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfe9c00640 1 --2- 172.21.15.27:0/3700027481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6adf84d0 0x557b6b1c9960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfe9c00640 1 -- 172.21.15.27:0/3700027481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b6ae84790 con 0x557b6ae191b0 2024-09-21T00:29:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfe9c00640 1 --2- 172.21.15.27:0/3700027481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b1c9ec0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc01ba30 tx=0x7fbfdc004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb458be938bd6cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfcaa00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:29:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfcaa00640 1 -- 172.21.15.27:0/3700027481 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc02b060 con 0x557b6ae191b0 2024-09-21T00:29:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 1 -- 172.21.15.27:0/3700027481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfdc011db0 con 0x557b6ae191b0 2024-09-21T00:29:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfcaa00640 1 -- 172.21.15.27:0/3700027481 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7fbfdc011770 con 0x557b6ae191b0 2024-09-21T00:29:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfcaa00640 1 -- 172.21.15.27:0/3700027481 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbfdc032050 con 0x557b6ae191b0 2024-09-21T00:29:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 10 client.16185 fetch_fsmap learned FSMap version 34 2024-09-21T00:29:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 1 -- 172.21.15.27:0/3700027481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557b6aefd620 con 0x557b6ae191b0 2024-09-21T00:29:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfcaa00640 1 -- 172.21.15.27:0/3700027481 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfdc011db0 con 0x557b6ae191b0 2024-09-21T00:29:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfcaa00640 1 -- 172.21.15.27:0/3700027481 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fbfdc005720 con 0x557b6ae191b0 2024-09-21T00:29:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 10 client.16185 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T00:29:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 10 client.16185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:29:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 20 client.16185 populate_metadata read hostname 'smithi027' 2024-09-21T00:29:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/3700027481 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557b6add7d80 0x557b6b1db4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 4 client.16185 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:29:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbff2c7d740 1 -- 172.21.15.27:0/3700027481 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b6b135600 con 0x557b6add7d80 2024-09-21T00:29:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfe9200640 1 --2- 172.21.15.27:0/3700027481 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557b6add7d80 0x557b6b1db4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfe9200640 1 --2- 172.21.15.27:0/3700027481 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557b6add7d80 0x557b6b1db4b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a65226959d805f82 server_cookie=29db5b76d93703a2 in_seq=0 out_seq=0 2024-09-21T00:29:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.202+0000 7fbfcaa00640 10 client.16185 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.207+0000 7fbfcaa00640 1 -- 172.21.15.27:0/3700027481 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557b6b135600 con 0x557b6add7d80 2024-09-21T00:29:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.207+0000 7fbfcaa00640 10 client.16185 handle_command_reply: tid=0 2024-09-21T00:29:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.207+0000 7fbff2c7d740 1 client.16185 shutdown 2024-09-21T00:29:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.207+0000 7fbff2c7d740 1 -- 172.21.15.27:0/3700027481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 msgr2=0x557b6b1c9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.207+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/3700027481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b1c9ec0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc01ba30 tx=0x7fbfdc004cf0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.207+0000 7fbff2c7d740 1 -- 172.21.15.27:0/3700027481 shutdown_connections 2024-09-21T00:29:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.207+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/3700027481 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557b6add7d80 0x557b6b1db4b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.207+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/3700027481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b1c9ec0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.207+0000 7fbfcaa00640 0 client.16185 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.207+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/3700027481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6adf84d0 0x557b6b1c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.207+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/3700027481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf02f0 0x557b6b1c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.207+0000 7fbff2c7d740 1 -- 172.21.15.27:0/3700027481 >> 172.21.15.27:0/3700027481 conn(0x557b6ae0d300 msgr2=0x557b6b1bee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.207+0000 7fbff2c7d740 1 -- 172.21.15.27:0/3700027481 shutdown_connections 2024-09-21T00:29:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.207+0000 7fbff2c7d740 1 -- 172.21.15.27:0/3700027481 wait complete. 2024-09-21T00:29:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.207+0000 7fbff2c7d740 20 client.16185 trim_cache size 0 max 16384 2024-09-21T00:29:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 Processor -- start 2024-09-21T00:29:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 -- start start 2024-09-21T00:29:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6adf02f0 0x557b6b1baa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf84d0 0x557b6b1c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6ae191b0 0x557b6b1c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41b5a80 con 0x557b6adf02f0 2024-09-21T00:29:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41b3290 con 0x557b6ae191b0 2024-09-21T00:29:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41b1100 con 0x557b6adf84d0 2024-09-21T00:29:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6adf02f0 0x557b6b1baa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6adf02f0 0x557b6b1baa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55574/0 (socket says 172.21.15.27:55574) 2024-09-21T00:29:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9200640 1 -- 172.21.15.27:0/1861094380 learned_addr learned my addr 172.21.15.27:0/1861094380 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9c00640 1 --2- 172.21.15.27:0/1861094380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6ae191b0 0x557b6b1c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9200640 1 -- 172.21.15.27:0/1861094380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf84d0 msgr2=0x557b6b1c9900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9200640 1 --2- 172.21.15.27:0/1861094380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf84d0 0x557b6b1c9900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9200640 1 -- 172.21.15.27:0/1861094380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6ae191b0 msgr2=0x557b6b1c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9200640 1 --2- 172.21.15.27:0/1861094380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6ae191b0 0x557b6b1c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9200640 1 -- 172.21.15.27:0/1861094380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b6b1cc470 con 0x557b6adf02f0 2024-09-21T00:29:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9200640 1 --2- 172.21.15.27:0/1861094380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6adf02f0 0x557b6b1baa70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009230 tx=0x7fbfd8040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb10bf77a4d5945b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfd5600640 1 -- 172.21.15.27:0/1861094380 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8057070 con 0x557b6adf02f0 2024-09-21T00:29:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfd5600640 1 -- 172.21.15.27:0/1861094380 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfd8005590 con 0x557b6adf02f0 2024-09-21T00:29:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfd5600640 1 -- 172.21.15.27:0/1861094380 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8035040 con 0x557b6adf02f0 2024-09-21T00:29:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1861094380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6adf02f0 msgr2=0x557b6b1baa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/1861094380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6adf02f0 0x557b6b1baa70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009230 tx=0x7fbfd8040000 comp rx=0 tx=0).stop 2024-09-21T00:29:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1861094380 shutdown_connections 2024-09-21T00:29:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/1861094380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6ae191b0 0x557b6b1c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/1861094380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf84d0 0x557b6b1c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/1861094380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6adf02f0 0x557b6b1baa70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1861094380 >> 172.21.15.27:0/1861094380 conn(0x557b6ae0d300 msgr2=0x557b6b1ba360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1861094380 shutdown_connections 2024-09-21T00:29:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1861094380 wait complete. 2024-09-21T00:29:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 -- start start 2024-09-21T00:29:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 Processor -- start 2024-09-21T00:29:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6adf02f0 0x557b6b1c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf84d0 0x557b6b1c6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b23dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41b1750 con 0x557b6ae191b0 2024-09-21T00:29:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41b2bc0 con 0x557b6adf02f0 2024-09-21T00:29:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41b1e20 con 0x557b6adf84d0 2024-09-21T00:29:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbff2c7d740 10 client.0 _ll_register_callbacks cb 0x7fbfdc009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b23dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b23dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55590/0 (socket says 172.21.15.27:55590) 2024-09-21T00:29:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9c00640 1 -- 172.21.15.27:0/1889056075 learned_addr learned my addr 172.21.15.27:0/1889056075 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe3e00640 1 --2- 172.21.15.27:0/1889056075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf84d0 0x557b6b1c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9c00640 1 -- 172.21.15.27:0/1889056075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf84d0 msgr2=0x557b6b1c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9c00640 1 --2- 172.21.15.27:0/1889056075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf84d0 0x557b6b1c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9c00640 1 -- 172.21.15.27:0/1889056075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6adf02f0 msgr2=0x557b6b1c60f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9c00640 1 --2- 172.21.15.27:0/1889056075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6adf02f0 0x557b6b1c60f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.223+0000 7fbfe9c00640 1 -- 172.21.15.27:0/1889056075 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b6b23e2e0 con 0x557b6ae191b0 2024-09-21T00:29:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbfe9c00640 1 --2- 172.21.15.27:0/1889056075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b23dc70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc01ba30 tx=0x7fbfdc009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7513892756d69c41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbfcaa00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:29:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbfcaa00640 1 -- 172.21.15.27:0/1889056075 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc03a020 con 0x557b6ae191b0 2024-09-21T00:29:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbfcaa00640 1 -- 172.21.15.27:0/1889056075 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9503+0+0 (secure 0 0 0) 0x7fbfdc011980 con 0x557b6ae191b0 2024-09-21T00:29:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1889056075 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfd8005590 con 0x557b6ae191b0 2024-09-21T00:29:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbfcaa00640 1 -- 172.21.15.27:0/1889056075 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbfdc045050 con 0x557b6ae191b0 2024-09-21T00:29:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbff2c7d740 10 client.16191 fetch_fsmap learned FSMap version 34 2024-09-21T00:29:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1889056075 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557b6b245700 con 0x557b6ae191b0 2024-09-21T00:29:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbfcaa00640 1 -- 172.21.15.27:0/1889056075 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfd8005590 con 0x557b6ae191b0 2024-09-21T00:29:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbfcaa00640 1 -- 172.21.15.27:0/1889056075 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fbfdc004fa0 con 0x557b6ae191b0 2024-09-21T00:29:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbff2c7d740 10 client.16191 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T00:29:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbff2c7d740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:29:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbff2c7d740 20 client.16191 populate_metadata read hostname 'smithi027' 2024-09-21T00:29:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/1889056075 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557b6adad1c0 0x557b6b247eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbff2c7d740 4 client.16191 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:29:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.228+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1889056075 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b6b1310b0 con 0x557b6adad1c0 2024-09-21T00:29:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.241+0000 7fbfe9200640 1 --2- 172.21.15.27:0/1889056075 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557b6adad1c0 0x557b6b247eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.300+0000 7fbfe9200640 1 --2- 172.21.15.27:0/1889056075 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557b6adad1c0 0x557b6b247eb0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c245f7f4f61f21f0 server_cookie=5ecdbd71af6b80cc in_seq=0 out_seq=0 2024-09-21T00:29:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.300+0000 7fbfcaa00640 10 client.16191 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbfcaa00640 1 -- 172.21.15.27:0/1889056075 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b6b1310b0 con 0x557b6adad1c0 2024-09-21T00:29:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbfcaa00640 10 client.16191 handle_command_reply: tid=0 2024-09-21T00:29:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbff2c7d740 1 client.16191 shutdown 2024-09-21T00:29:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1889056075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 msgr2=0x557b6b23dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/1889056075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b23dc70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc01ba30 tx=0x7fbfdc009cd0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1889056075 shutdown_connections 2024-09-21T00:29:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/1889056075 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557b6adad1c0 0x557b6b247eb0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/1889056075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557b6ae191b0 0x557b6b23dc70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/1889056075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b6adf84d0 0x557b6b1c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbff2c7d740 1 --2- 172.21.15.27:0/1889056075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b6adf02f0 0x557b6b1c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1889056075 >> 172.21.15.27:0/1889056075 conn(0x557b6ae0d300 msgr2=0x557b6b1bedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbfcaa00640 0 client.16191 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1889056075 shutdown_connections 2024-09-21T00:29:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbff2c7d740 1 -- 172.21.15.27:0/1889056075 wait complete. 2024-09-21T00:29:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbff2c7d740 20 client.16191 trim_cache size 0 max 16384 2024-09-21T00:29:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:29:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:29:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:29:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:29:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbfea600640 1 -- 172.21.15.27:0/300018643 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbfcc0b66f0 msgr2=0x7fbfcc0b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbfea600640 1 --2- 172.21.15.27:0/300018643 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbfcc0b66f0 0x7fbfcc0b8c00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8005ab0 tx=0x7fbfd8005a20 comp rx=0 tx=0).stop 2024-09-21T00:29:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbfea600640 1 -- 172.21.15.27:0/300018643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe4109250 msgr2=0x7fbfe419cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbfea600640 1 --2- 172.21.15.27:0/300018643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe4109250 0x7fbfe419cab0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000b450 tx=0x7fbfd000b910 comp rx=0 tx=0).stop 2024-09-21T00:29:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbfea600640 1 -- 172.21.15.27:0/300018643 shutdown_connections 2024-09-21T00:29:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbfea600640 1 --2- 172.21.15.27:0/300018643 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbfcc0b66f0 0x7fbfcc0b8c00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbfea600640 1 --2- 172.21.15.27:0/300018643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfe4109ba0 0x7fbfe41a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbfea600640 1 --2- 172.21.15.27:0/300018643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe4109250 0x7fbfe419cab0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbfea600640 1 --2- 172.21.15.27:0/300018643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfe4103920 0x7fbfe419c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbfea600640 1 -- 172.21.15.27:0/300018643 >> 172.21.15.27:0/300018643 conn(0x7fbfe40ff340 msgr2=0x7fbfe41088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:07.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbfea600640 1 -- 172.21.15.27:0/300018643 shutdown_connections 2024-09-21T00:29:07.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:07.396+0000 7fbfea600640 1 -- 172.21.15.27:0/300018643 wait complete. 2024-09-21T00:29:07.920 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:07 smithi027 ceph-mon[30745]: pgmap v613: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 95 KiB/s wr, 52 op/s 2024-09-21T00:29:07.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:07 smithi089 ceph-mon[34481]: pgmap v613: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 95 KiB/s wr, 52 op/s 2024-09-21T00:29:07.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:07 smithi098 ceph-mon[34725]: pgmap v613: 129 pgs: 129 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 95 KiB/s wr, 52 op/s 2024-09-21T00:29:08.717 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:08 smithi027 ceph-mon[30745]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T00:29:08.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:08 smithi089 ceph-mon[34481]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T00:29:08.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:08 smithi098 ceph-mon[34725]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T00:29:09.800 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:09 smithi027 ceph-mon[30745]: pgmap v615: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-09-21T00:29:09.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:09 smithi089 ceph-mon[34481]: pgmap v615: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-09-21T00:29:09.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:09 smithi098 ceph-mon[34725]: pgmap v615: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-09-21T00:29:11.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:11 smithi027 ceph-mon[30745]: pgmap v616: 129 pgs: 12 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 82 active+clean; 979 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 61 KiB/s wr, 28 op/s 2024-09-21T00:29:11.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:11 smithi089 ceph-mon[34481]: pgmap v616: 129 pgs: 12 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 82 active+clean; 979 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 61 KiB/s wr, 28 op/s 2024-09-21T00:29:11.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:11 smithi098 ceph-mon[34725]: pgmap v616: 129 pgs: 12 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 82 active+clean; 979 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 61 KiB/s wr, 28 op/s 2024-09-21T00:29:13.995 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:13 smithi027 ceph-mon[30745]: pgmap v617: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-09-21T00:29:13.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:13 smithi098 ceph-mon[34725]: pgmap v617: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-09-21T00:29:14.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:13 smithi089 ceph-mon[34481]: pgmap v617: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 979 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-09-21T00:29:16.003 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:15 smithi027 ceph-mon[30745]: pgmap v618: 129 pgs: 17 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 65 active+clean; 979 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-09-21T00:29:16.003 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:15 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:29:16.030 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:15 smithi089 ceph-mon[34481]: pgmap v618: 129 pgs: 17 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 65 active+clean; 979 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-09-21T00:29:16.030 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:15 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:29:16.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:15 smithi098 ceph-mon[34725]: pgmap v618: 129 pgs: 17 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 65 active+clean; 979 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-09-21T00:29:16.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:15 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:29:17.994 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:17 smithi027 ceph-mon[30745]: pgmap v619: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 979 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-09-21T00:29:17.994 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:17 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:29:17.994 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:17 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:29:17.994 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:17 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:29:18.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:17 smithi089 ceph-mon[34481]: pgmap v619: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 979 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-09-21T00:29:18.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:17 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:29:18.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:17 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:29:18.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:17 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:29:18.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:17 smithi098 ceph-mon[34725]: pgmap v619: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 979 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-09-21T00:29:18.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:17 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:29:18.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:17 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:29:18.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:17 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:29:18.861 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:18 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:29:19.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:18 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:29:19.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:18 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:29:19.993 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:19 smithi027 ceph-mon[30745]: pgmap v620: 129 pgs: 12 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 85 active+clean; 979 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 10 op/s 2024-09-21T00:29:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:19 smithi089 ceph-mon[34481]: pgmap v620: 129 pgs: 12 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 85 active+clean; 979 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 10 op/s 2024-09-21T00:29:20.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:19 smithi098 ceph-mon[34725]: pgmap v620: 129 pgs: 12 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 85 active+clean; 979 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 10 op/s 2024-09-21T00:29:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.623+0000 7f389d800640 1 -- 172.21.15.27:0/2632938301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3898075dd0 msgr2=0x7f38980761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.623+0000 7f389d800640 1 --2- 172.21.15.27:0/2632938301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3898075dd0 0x7f38980761e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3884009900 tx=0x7f388402a400 comp rx=0 tx=0).stop 2024-09-21T00:29:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.623+0000 7f389d800640 1 -- 172.21.15.27:0/2632938301 shutdown_connections 2024-09-21T00:29:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.623+0000 7f389d800640 1 --2- 172.21.15.27:0/2632938301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38981141d0 0x7f38981166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.623+0000 7f389d800640 1 --2- 172.21.15.27:0/2632938301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3898075dd0 0x7f38980761e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.623+0000 7f389d800640 1 --2- 172.21.15.27:0/2632938301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3898077390 0x7f38980757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.623+0000 7f389d800640 1 -- 172.21.15.27:0/2632938301 >> 172.21.15.27:0/2632938301 conn(0x7f38980ff390 msgr2=0x7f38981017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.623+0000 7f389d800640 1 -- 172.21.15.27:0/2632938301 shutdown_connections 2024-09-21T00:29:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.623+0000 7f389d800640 1 -- 172.21.15.27:0/2632938301 wait complete. 2024-09-21T00:29:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.629+0000 7f389d800640 1 Processor -- start 2024-09-21T00:29:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.629+0000 7f389d800640 1 -- start start 2024-09-21T00:29:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.629+0000 7f389d800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3898075dd0 0x7f38981a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.629+0000 7f389d800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3898077390 0x7f38981a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.629+0000 7f389d800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38981141d0 0x7f38981a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.629+0000 7f389d800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3898119e60 con 0x7f3898077390 2024-09-21T00:29:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.629+0000 7f389d800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3898119ce0 con 0x7f3898075dd0 2024-09-21T00:29:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.629+0000 7f389d800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3898119fe0 con 0x7f38981141d0 2024-09-21T00:29:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.631+0000 7f3897e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3898075dd0 0x7f38981a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.631+0000 7f3897e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3898075dd0 0x7f38981a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:50192/0 (socket says 172.21.15.27:50192) 2024-09-21T00:29:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.631+0000 7f3897e00640 1 -- 172.21.15.27:0/337290712 learned_addr learned my addr 172.21.15.27:0/337290712 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.631+0000 7f389ce00640 1 --2- 172.21.15.27:0/337290712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38981141d0 0x7f38981a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.631+0000 7f3897400640 1 --2- 172.21.15.27:0/337290712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3898077390 0x7f38981a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.631+0000 7f389ce00640 1 -- 172.21.15.27:0/337290712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3898075dd0 msgr2=0x7f38981a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.631+0000 7f389ce00640 1 --2- 172.21.15.27:0/337290712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3898075dd0 0x7f38981a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.631+0000 7f389ce00640 1 -- 172.21.15.27:0/337290712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3898077390 msgr2=0x7f38981a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.631+0000 7f389ce00640 1 --2- 172.21.15.27:0/337290712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3898077390 0x7f38981a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.631+0000 7f389ce00640 1 -- 172.21.15.27:0/337290712 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38981a67b0 con 0x7f38981141d0 2024-09-21T00:29:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.631+0000 7f3897400640 1 --2- 172.21.15.27:0/337290712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3898077390 0x7f38981a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.633+0000 7f389ce00640 1 --2- 172.21.15.27:0/337290712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38981141d0 0x7f38981a60b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3890009fd0 tx=0x7f389000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.633+0000 7f3894c00640 1 -- 172.21.15.27:0/337290712 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389000ce20 con 0x7f38981141d0 2024-09-21T00:29:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.633+0000 7f3894c00640 1 -- 172.21.15.27:0/337290712 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3890005020 con 0x7f38981141d0 2024-09-21T00:29:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.633+0000 7f389d800640 1 -- 172.21.15.27:0/337290712 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38981a6aa0 con 0x7f38981141d0 2024-09-21T00:29:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.633+0000 7f3894c00640 1 -- 172.21.15.27:0/337290712 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890010690 con 0x7f38981141d0 2024-09-21T00:29:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.633+0000 7f389d800640 1 -- 172.21.15.27:0/337290712 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38981ad230 con 0x7f38981141d0 2024-09-21T00:29:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.633+0000 7f38a5e85740 1 Processor -- start 2024-09-21T00:29:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.633+0000 7f3894c00640 1 -- 172.21.15.27:0/337290712 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3890010830 con 0x7f38981141d0 2024-09-21T00:29:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.633+0000 7f38a5e85740 1 -- start start 2024-09-21T00:29:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f38a5e85740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e4f1b0 0x557a831f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f38a5e85740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a831f4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f38a5e85740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e2e4d0 0x557a831f4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f38a5e85740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557a831696b0 con 0x557a82e262f0 2024-09-21T00:29:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f38a5e85740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557a83158ae0 con 0x557a82e4f1b0 2024-09-21T00:29:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f38a5e85740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557a8315c8f0 con 0x557a82e2e4d0 2024-09-21T00:29:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f3897e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a831f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f3897e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a831f4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45316/0 (socket says 172.21.15.27:45316) 2024-09-21T00:29:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f3897e00640 1 -- 172.21.15.27:0/299006483 learned_addr learned my addr 172.21.15.27:0/299006483 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f3897e00640 1 -- 172.21.15.27:0/299006483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e2e4d0 msgr2=0x557a831f4890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f3897e00640 1 --2- 172.21.15.27:0/299006483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e2e4d0 0x557a831f4890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f3897e00640 1 -- 172.21.15.27:0/299006483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e4f1b0 msgr2=0x557a831f3e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f3897e00640 1 --2- 172.21.15.27:0/299006483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e4f1b0 0x557a831f3e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f3897e00640 1 -- 172.21.15.27:0/299006483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a83125780 con 0x557a82e262f0 2024-09-21T00:29:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f3897e00640 1 --2- 172.21.15.27:0/299006483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a831f4350 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f388800bd90 tx=0x7f388800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b59053c356b3fc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.639+0000 7f3894c00640 1 --2- 172.21.15.27:0/337290712 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f387c0b66f0 0x7f387c0b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f3894c00640 1 -- 172.21.15.27:0/337290712 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9607+0+0 (secure 0 0 0) 0x7f38900e8270 con 0x7f38981141d0 2024-09-21T00:29:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f3897400640 1 --2- 172.21.15.27:0/337290712 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f387c0b66f0 0x7f387c0b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f388d600640 1 -- 172.21.15.27:0/299006483 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888015070 con 0x557a82e262f0 2024-09-21T00:29:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f388d600640 1 -- 172.21.15.27:0/299006483 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38880035f0 con 0x557a82e262f0 2024-09-21T00:29:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f388d600640 1 -- 172.21.15.27:0/299006483 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888011680 con 0x557a82e262f0 2024-09-21T00:29:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f3897400640 1 --2- 172.21.15.27:0/337290712 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f387c0b66f0 0x7f387c0b8c00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3884005ab0 tx=0x7f3884005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 -- 172.21.15.27:0/299006483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 msgr2=0x557a831f4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 --2- 172.21.15.27:0/299006483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a831f4350 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f388800bd90 tx=0x7f388800f730 comp rx=0 tx=0).stop 2024-09-21T00:29:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 -- 172.21.15.27:0/299006483 shutdown_connections 2024-09-21T00:29:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 --2- 172.21.15.27:0/299006483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e2e4d0 0x557a831f4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 --2- 172.21.15.27:0/299006483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a831f4350 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 --2- 172.21.15.27:0/299006483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e4f1b0 0x557a831f3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 -- 172.21.15.27:0/299006483 >> 172.21.15.27:0/299006483 conn(0x557a82e43300 msgr2=0x557a831f1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 -- 172.21.15.27:0/299006483 shutdown_connections 2024-09-21T00:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 -- 172.21.15.27:0/299006483 wait complete. 2024-09-21T00:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 -- start start 2024-09-21T00:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 Processor -- start 2024-09-21T00:29:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a831ff4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 0x557a831ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e4f1b0 0x557a8320b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557a83090ab0 con 0x557a82e262f0 2024-09-21T00:29:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557a830ff570 con 0x557a82e2e4d0 2024-09-21T00:29:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557a83202590 con 0x557a82e4f1b0 2024-09-21T00:29:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f38a5e85740 10 client.0 _ll_register_callbacks cb 0x557a8310e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f3897400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 0x557a831ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f389ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e4f1b0 0x557a8320b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f389ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e4f1b0 0x557a8320b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:34170/0 (socket says 172.21.15.27:34170) 2024-09-21T00:29:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f389ce00640 1 -- 172.21.15.27:0/1270320105 learned_addr learned my addr 172.21.15.27:0/1270320105 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f3897400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 0x557a831ffa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:50216/0 (socket says 172.21.15.27:50216) 2024-09-21T00:29:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f3897e00640 1 --2- 172.21.15.27:0/1270320105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a831ff4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f389ce00640 1 -- 172.21.15.27:0/1270320105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 msgr2=0x557a831ffa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.641+0000 7f389ce00640 1 --2- 172.21.15.27:0/1270320105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 0x557a831ffa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f389ce00640 1 -- 172.21.15.27:0/1270320105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 msgr2=0x557a831ff4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f389ce00640 1 --2- 172.21.15.27:0/1270320105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a831ff4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f389ce00640 1 -- 172.21.15.27:0/1270320105 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a82eba790 con 0x557a82e4f1b0 2024-09-21T00:29:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f389ce00640 1 --2- 172.21.15.27:0/1270320105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e4f1b0 0x557a8320b8b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f38900aa250 tx=0x7f38900054d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e1e57e87bc32c51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f3882000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:29:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f3882000640 1 -- 172.21.15.27:0/1270320105 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38900ad0e0 con 0x557a82e4f1b0 2024-09-21T00:29:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f3882000640 1 -- 172.21.15.27:0/1270320105 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9607+0+0 (secure 0 0 0) 0x7f3890010b20 con 0x557a82e4f1b0 2024-09-21T00:29:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f38a5e85740 1 -- 172.21.15.27:0/1270320105 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38880035f0 con 0x557a82e4f1b0 2024-09-21T00:29:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f3882000640 1 -- 172.21.15.27:0/1270320105 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f38900ae380 con 0x557a82e4f1b0 2024-09-21T00:29:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f38a5e85740 10 client.26233 fetch_fsmap learned FSMap version 35 2024-09-21T00:29:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f38a5e85740 1 -- 172.21.15.27:0/1270320105 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557a82f33620 con 0x557a82e4f1b0 2024-09-21T00:29:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f3882000640 1 -- 172.21.15.27:0/1270320105 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38880035f0 con 0x557a82e4f1b0 2024-09-21T00:29:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f3882000640 1 -- 172.21.15.27:0/1270320105 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f38900a95d0 con 0x557a82e4f1b0 2024-09-21T00:29:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f38a5e85740 10 client.26233 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T00:29:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f38a5e85740 10 client.26233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:29:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f38a5e85740 20 client.26233 populate_metadata read hostname 'smithi027' 2024-09-21T00:29:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f38a5e85740 1 --2- 172.21.15.27:0/1270320105 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557a82e0dd80 0x557a83211740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f38a5e85740 4 client.26233 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:29:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f38a5e85740 1 -- 172.21.15.27:0/1270320105 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a8316b600 con 0x557a82e0dd80 2024-09-21T00:29:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f3897e00640 1 --2- 172.21.15.27:0/1270320105 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557a82e0dd80 0x557a83211740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f3897e00640 1 --2- 172.21.15.27:0/1270320105 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557a82e0dd80 0x557a83211740 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7fd35df21c7f98 server_cookie=d3ddf94fd4c2f4e in_seq=0 out_seq=0 2024-09-21T00:29:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f3882000640 10 client.26233 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f3882000640 1 -- 172.21.15.27:0/1270320105 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a8316b600 con 0x557a82e0dd80 2024-09-21T00:29:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f3882000640 10 client.26233 handle_command_reply: tid=0 2024-09-21T00:29:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f38a5e85740 1 client.26233 shutdown 2024-09-21T00:29:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f38a5e85740 1 -- 172.21.15.27:0/1270320105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e4f1b0 msgr2=0x557a8320b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f38a5e85740 1 --2- 172.21.15.27:0/1270320105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e4f1b0 0x557a8320b8b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f38900aa250 tx=0x7f38900054d0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.646+0000 7f38a5e85740 1 -- 172.21.15.27:0/1270320105 shutdown_connections 2024-09-21T00:29:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.651+0000 7f38a5e85740 1 --2- 172.21.15.27:0/1270320105 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557a82e0dd80 0x557a83211740 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.651+0000 7f38a5e85740 1 --2- 172.21.15.27:0/1270320105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e4f1b0 0x557a8320b8b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.651+0000 7f38a5e85740 1 --2- 172.21.15.27:0/1270320105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 0x557a831ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.651+0000 7f38a5e85740 1 --2- 172.21.15.27:0/1270320105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a831ff4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.651+0000 7f38a5e85740 1 -- 172.21.15.27:0/1270320105 >> 172.21.15.27:0/1270320105 conn(0x557a82e43300 msgr2=0x557a831f4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.651+0000 7f3882000640 0 client.26233 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.651+0000 7f38a5e85740 1 -- 172.21.15.27:0/1270320105 shutdown_connections 2024-09-21T00:29:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.651+0000 7f38a5e85740 1 -- 172.21.15.27:0/1270320105 wait complete. 2024-09-21T00:29:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.651+0000 7f38a5e85740 20 client.26233 trim_cache size 0 max 16384 2024-09-21T00:29:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 Processor -- start 2024-09-21T00:29:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 -- start start 2024-09-21T00:29:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e262f0 0x557a831f0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 0x557a831f5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e4f1b0 0x557a83216f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f38981b9e90 con 0x557a82e4f1b0 2024-09-21T00:29:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f38981b76a0 con 0x557a82e2e4d0 2024-09-21T00:29:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f38981b5510 con 0x557a82e262f0 2024-09-21T00:29:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f389ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e4f1b0 0x557a83216f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f3897e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e262f0 0x557a831f0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f3897400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 0x557a831f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f3897e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e262f0 0x557a831f0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:34174/0 (socket says 172.21.15.27:34174) 2024-09-21T00:29:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f3897400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 0x557a831f5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:50226/0 (socket says 172.21.15.27:50226) 2024-09-21T00:29:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f3897e00640 1 -- 172.21.15.27:0/2664202735 learned_addr learned my addr 172.21.15.27:0/2664202735 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f3897400640 1 -- 172.21.15.27:0/2664202735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e262f0 msgr2=0x557a831f0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f3897400640 1 --2- 172.21.15.27:0/2664202735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e262f0 0x557a831f0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f3897400640 1 -- 172.21.15.27:0/2664202735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e4f1b0 msgr2=0x557a83216f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f3897400640 1 --2- 172.21.15.27:0/2664202735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e4f1b0 0x557a83216f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f3897400640 1 -- 172.21.15.27:0/2664202735 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a832007a0 con 0x557a82e2e4d0 2024-09-21T00:29:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f389ce00640 1 --2- 172.21.15.27:0/2664202735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e4f1b0 0x557a83216f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f3897400640 1 --2- 172.21.15.27:0/2664202735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 0x557a831f5550 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f388402a9f0 tx=0x7f388403d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3fa8951baeb134 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f388cc00640 1 -- 172.21.15.27:0/2664202735 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388404e030 con 0x557a82e2e4d0 2024-09-21T00:29:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f388cc00640 1 -- 172.21.15.27:0/2664202735 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38840035f0 con 0x557a82e2e4d0 2024-09-21T00:29:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f388cc00640 1 -- 172.21.15.27:0/2664202735 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884035040 con 0x557a82e2e4d0 2024-09-21T00:29:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 -- 172.21.15.27:0/2664202735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 msgr2=0x557a831f5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 --2- 172.21.15.27:0/2664202735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 0x557a831f5550 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f388402a9f0 tx=0x7f388403d000 comp rx=0 tx=0).stop 2024-09-21T00:29:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 -- 172.21.15.27:0/2664202735 shutdown_connections 2024-09-21T00:29:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 --2- 172.21.15.27:0/2664202735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e4f1b0 0x557a83216f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 --2- 172.21.15.27:0/2664202735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 0x557a831f5550 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 --2- 172.21.15.27:0/2664202735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e262f0 0x557a831f0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 -- 172.21.15.27:0/2664202735 >> 172.21.15.27:0/2664202735 conn(0x557a82e43300 msgr2=0x557a831f4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 -- 172.21.15.27:0/2664202735 shutdown_connections 2024-09-21T00:29:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.678+0000 7f38a5e85740 1 -- 172.21.15.27:0/2664202735 wait complete. 2024-09-21T00:29:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 1 -- start start 2024-09-21T00:29:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 1 Processor -- start 2024-09-21T00:29:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a83206ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 0x557a83207410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e4f1b0 0x557a8327a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f38981b5b60 con 0x557a82e262f0 2024-09-21T00:29:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f38981b6fd0 con 0x557a82e2e4d0 2024-09-21T00:29:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f38981b6230 con 0x557a82e4f1b0 2024-09-21T00:29:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 10 client.0 _ll_register_callbacks cb 0x7f38900aa2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3897e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a83206ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3897e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a83206ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45358/0 (socket says 172.21.15.27:45358) 2024-09-21T00:29:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3897e00640 1 -- 172.21.15.27:0/3566766726 learned_addr learned my addr 172.21.15.27:0/3566766726 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f389ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e4f1b0 0x557a8327a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3897400640 1 --2- 172.21.15.27:0/3566766726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 0x557a83207410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3897e00640 1 -- 172.21.15.27:0/3566766726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e4f1b0 msgr2=0x557a8327a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3897e00640 1 --2- 172.21.15.27:0/3566766726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e4f1b0 0x557a8327a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3897e00640 1 -- 172.21.15.27:0/3566766726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 msgr2=0x557a83207410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3897e00640 1 --2- 172.21.15.27:0/3566766726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 0x557a83207410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3897e00640 1 -- 172.21.15.27:0/3566766726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a8327a690 con 0x557a82e262f0 2024-09-21T00:29:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3897e00640 1 --2- 172.21.15.27:0/3566766726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a83206ed0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3888000c00 tx=0x7f388800b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2017ffc98ed4b0f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3882000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:29:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3882000640 1 -- 172.21.15.27:0/3566766726 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888030070 con 0x557a82e262f0 2024-09-21T00:29:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 1 -- 172.21.15.27:0/3566766726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38840035f0 con 0x557a82e262f0 2024-09-21T00:29:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3882000640 1 -- 172.21.15.27:0/3566766726 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9607+0+0 (secure 0 0 0) 0x7f38880035f0 con 0x557a82e262f0 2024-09-21T00:29:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3882000640 1 -- 172.21.15.27:0/3566766726 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f388803c050 con 0x557a82e262f0 2024-09-21T00:29:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 10 client.16209 fetch_fsmap learned FSMap version 35 2024-09-21T00:29:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 1 -- 172.21.15.27:0/3566766726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557a8327b890 con 0x557a82e262f0 2024-09-21T00:29:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3882000640 1 -- 172.21.15.27:0/3566766726 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38840035f0 con 0x557a82e262f0 2024-09-21T00:29:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f3882000640 1 -- 172.21.15.27:0/3566766726 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f3888004680 con 0x557a82e262f0 2024-09-21T00:29:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 10 client.16209 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T00:29:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:29:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 20 client.16209 populate_metadata read hostname 'smithi027' 2024-09-21T00:29:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 1 --2- 172.21.15.27:0/3566766726 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557a82de31c0 0x557a8327dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 4 client.16209 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:29:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.682+0000 7f38a5e85740 1 -- 172.21.15.27:0/3566766726 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a831670b0 con 0x557a82de31c0 2024-09-21T00:29:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.687+0000 7f3897400640 1 --2- 172.21.15.27:0/3566766726 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557a82de31c0 0x557a8327dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.687+0000 7f3897400640 1 --2- 172.21.15.27:0/3566766726 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557a82de31c0 0x557a8327dda0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9d02b221c419934 server_cookie=3ceaebdc529ac506 in_seq=0 out_seq=0 2024-09-21T00:29:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.687+0000 7f3882000640 10 client.16209 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f3882000640 1 -- 172.21.15.27:0/3566766726 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a831670b0 con 0x557a82de31c0 2024-09-21T00:29:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f3882000640 10 client.16209 handle_command_reply: tid=0 2024-09-21T00:29:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f38a5e85740 1 client.16209 shutdown 2024-09-21T00:29:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f38a5e85740 1 -- 172.21.15.27:0/3566766726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 msgr2=0x557a83206ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f38a5e85740 1 --2- 172.21.15.27:0/3566766726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a83206ed0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3888000c00 tx=0x7f388800b930 comp rx=0 tx=0).stop 2024-09-21T00:29:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f38a5e85740 1 -- 172.21.15.27:0/3566766726 shutdown_connections 2024-09-21T00:29:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f38a5e85740 1 --2- 172.21.15.27:0/3566766726 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557a82de31c0 0x557a8327dda0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f38a5e85740 1 --2- 172.21.15.27:0/3566766726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557a82e4f1b0 0x557a8327a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f38a5e85740 1 --2- 172.21.15.27:0/3566766726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557a82e2e4d0 0x557a83207410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f38a5e85740 1 --2- 172.21.15.27:0/3566766726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a82e262f0 0x557a83206ed0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f38a5e85740 1 -- 172.21.15.27:0/3566766726 >> 172.21.15.27:0/3566766726 conn(0x557a82e43300 msgr2=0x557a831f0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f3882000640 0 client.16209 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f38a5e85740 1 -- 172.21.15.27:0/3566766726 shutdown_connections 2024-09-21T00:29:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f38a5e85740 1 -- 172.21.15.27:0/3566766726 wait complete. 2024-09-21T00:29:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f38a5e85740 20 client.16209 trim_cache size 0 max 16384 2024-09-21T00:29:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:29:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:29:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:29:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:29:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f389d800640 1 -- 172.21.15.27:0/337290712 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f387c0b66f0 msgr2=0x7f387c0b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f389d800640 1 --2- 172.21.15.27:0/337290712 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f387c0b66f0 0x7f387c0b8c00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3884005ab0 tx=0x7f3884005a20 comp rx=0 tx=0).stop 2024-09-21T00:29:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f389d800640 1 -- 172.21.15.27:0/337290712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38981141d0 msgr2=0x7f38981a60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f389d800640 1 --2- 172.21.15.27:0/337290712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38981141d0 0x7f38981a60b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3890009fd0 tx=0x7f389000ef10 comp rx=0 tx=0).stop 2024-09-21T00:29:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f389d800640 1 -- 172.21.15.27:0/337290712 shutdown_connections 2024-09-21T00:29:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f389d800640 1 --2- 172.21.15.27:0/337290712 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f387c0b66f0 0x7f387c0b8c00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f389d800640 1 --2- 172.21.15.27:0/337290712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38981141d0 0x7f38981a60b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f389d800640 1 --2- 172.21.15.27:0/337290712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3898077390 0x7f38981a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f389d800640 1 --2- 172.21.15.27:0/337290712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3898075dd0 0x7f38981a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f389d800640 1 -- 172.21.15.27:0/337290712 >> 172.21.15.27:0/337290712 conn(0x7f38980ff390 msgr2=0x7f38981100d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f389d800640 1 -- 172.21.15.27:0/337290712 shutdown_connections 2024-09-21T00:29:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:20.704+0000 7f389d800640 1 -- 172.21.15.27:0/337290712 wait complete. 2024-09-21T00:29:22.269 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:22 smithi027 ceph-mon[30745]: pgmap v621: 129 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 113 active+clean; 979 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-09-21T00:29:22.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:22 smithi089 ceph-mon[34481]: pgmap v621: 129 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 113 active+clean; 979 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-09-21T00:29:22.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:22 smithi098 ceph-mon[34725]: pgmap v621: 129 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 113 active+clean; 979 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-09-21T00:29:24.501 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:24 smithi027 ceph-mon[30745]: pgmap v622: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 980 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 49 KiB/s wr, 10 op/s 2024-09-21T00:29:24.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:24 smithi089 ceph-mon[34481]: pgmap v622: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 980 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 49 KiB/s wr, 10 op/s 2024-09-21T00:29:24.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:24 smithi098 ceph-mon[34725]: pgmap v622: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 980 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 49 KiB/s wr, 10 op/s 2024-09-21T00:29:26.502 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:26 smithi027 ceph-mon[30745]: pgmap v623: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 980 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 25 KiB/s wr, 7 op/s 2024-09-21T00:29:26.502 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:26 smithi027 ceph-mon[30745]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T00:29:26.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:26 smithi089 ceph-mon[34481]: pgmap v623: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 980 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 25 KiB/s wr, 7 op/s 2024-09-21T00:29:26.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:26 smithi089 ceph-mon[34481]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T00:29:26.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:26 smithi098 ceph-mon[34725]: pgmap v623: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 980 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 25 KiB/s wr, 7 op/s 2024-09-21T00:29:26.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:26 smithi098 ceph-mon[34725]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T00:29:28.443 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:28 smithi027 ceph-mon[30745]: pgmap v625: 129 pgs: 129 active+clean; 980 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 8 op/s 2024-09-21T00:29:28.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:28 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:29:28.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:28 smithi089 ceph-mon[34481]: pgmap v625: 129 pgs: 129 active+clean; 980 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 8 op/s 2024-09-21T00:29:28.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:28 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:29:28.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:28 smithi098 ceph-mon[34725]: pgmap v625: 129 pgs: 129 active+clean; 980 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 8 op/s 2024-09-21T00:29:28.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:28 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:29:30.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:30 smithi027 ceph-mon[30745]: pgmap v626: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 8 op/s 2024-09-21T00:29:30.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:30 smithi089 ceph-mon[34481]: pgmap v626: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 8 op/s 2024-09-21T00:29:30.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:30 smithi098 ceph-mon[34725]: pgmap v626: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 8 op/s 2024-09-21T00:29:31.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:31.572+0000 7f81dfe00640 1 -- 172.21.15.27:0/807047038 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: -115 ) ==== 8+0+595 (crc 0 0 0) 0x55895c8b4490 con 0x55895c52d580 2024-09-21T00:29:31.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:31.572+0000 7f81dfe00640 10 client.26203 handle_command_reply: tid=0 2024-09-21T00:29:31.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:31.572+0000 7f820b9bf740 1 client.26203 shutdown 2024-09-21T00:29:31.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:31.572+0000 7f820b9bf740 1 -- 172.21.15.27:0/807047038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 msgr2=0x55895c950c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:31.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:31.572+0000 7f820b9bf740 1 --2- 172.21.15.27:0/807047038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c950c00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f81ec005a90 tx=0x7f81ec00fba0 comp rx=0 tx=0).stop 2024-09-21T00:29:31.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:31.572+0000 7f820b9bf740 1 -- 172.21.15.27:0/807047038 shutdown_connections 2024-09-21T00:29:31.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:31.572+0000 7f820b9bf740 1 --2- 172.21.15.27:0/807047038 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55895c52d580 0x55895c9cbba0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:31.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:31.572+0000 7f820b9bf740 1 --2- 172.21.15.27:0/807047038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55895c599850 0x55895c9c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:31.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:31.572+0000 7f820b9bf740 1 --2- 172.21.15.27:0/807047038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55895c578b70 0x55895c950c00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:31.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:31.572+0000 7f81dfe00640 0 client.26203 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:31.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:31.572+0000 7f820b9bf740 1 --2- 172.21.15.27:0/807047038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55895c570990 0x55895c9506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:31.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:31.572+0000 7f820b9bf740 1 -- 172.21.15.27:0/807047038 >> 172.21.15.27:0/807047038 conn(0x55895c58dc40 msgr2=0x55895c93d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:31.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:31.572+0000 7f820b9bf740 1 -- 172.21.15.27:0/807047038 shutdown_connections 2024-09-21T00:29:31.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:31.572+0000 7f820b9bf740 1 -- 172.21.15.27:0/807047038 wait complete. 2024-09-21T00:29:31.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:31.572+0000 7f820b9bf740 20 client.26203 trim_cache size 0 max 16384 2024-09-21T00:29:31.576 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:29:31.576 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 36, 2024-09-21T00:29:31.576 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24761, 2024-09-21T00:29:31.576 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 25, 2024-09-21T00:29:31.576 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T00:29:31.576 INFO:teuthology.orchestra.run.smithi027.stdout: "47099ce0": { 2024-09-21T00:29:31.576 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 25, 2024-09-21T00:29:31.576 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T00:29:31.576 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:29:31.576 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCING", 2024-09-21T00:29:31.576 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.0 2024-09-21T00:29:31.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:29:31.577 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T00:29:31.577 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 139.2, 2024-09-21T00:29:31.577 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T00:29:31.577 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T00:29:31.577 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T00:29:31.577 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:29:31.577 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCING", 2024-09-21T00:29:31.577 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.0 2024-09-21T00:29:31.577 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:29:31.577 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:29:31.577 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:29:31.577 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:29:31.577 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:29:31.577 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:29:31.578 INFO:teuthology.orchestra.run.smithi027.stderr:Error EINPROGRESS: 2024-09-21T00:29:32.398 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:32 smithi027 ceph-mon[30745]: pgmap v627: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 8 op/s 2024-09-21T00:29:32.398 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:32 smithi027 ceph-mon[30745]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T00:29:32.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:32 smithi089 ceph-mon[34481]: pgmap v627: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 8 op/s 2024-09-21T00:29:32.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:32 smithi089 ceph-mon[34481]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T00:29:32.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:32 smithi098 ceph-mon[34725]: pgmap v627: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 8 op/s 2024-09-21T00:29:32.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:32 smithi098 ceph-mon[34725]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T00:29:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 -- 172.21.15.27:0/1631055407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3444077390 msgr2=0x7f34440757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 --2- 172.21.15.27:0/1631055407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3444077390 0x7f34440757b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3434009900 tx=0x7f343402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 -- 172.21.15.27:0/1631055407 shutdown_connections 2024-09-21T00:29:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 --2- 172.21.15.27:0/1631055407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444114390 0x7f3444116860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 --2- 172.21.15.27:0/1631055407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3444075dd0 0x7f34440761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 --2- 172.21.15.27:0/1631055407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3444077390 0x7f34440757b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 -- 172.21.15.27:0/1631055407 >> 172.21.15.27:0/1631055407 conn(0x7f34440ff4d0 msgr2=0x7f3444101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 -- 172.21.15.27:0/1631055407 shutdown_connections 2024-09-21T00:29:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 -- 172.21.15.27:0/1631055407 wait complete. 2024-09-21T00:29:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 Processor -- start 2024-09-21T00:29:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 -- start start 2024-09-21T00:29:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3444075dd0 0x7f34441a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444077390 0x7f34441a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3444114390 0x7f34441a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3444119fa0 con 0x7f3444114390 2024-09-21T00:29:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3444119e20 con 0x7f3444077390 2024-09-21T00:29:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f344411a120 con 0x7f3444075dd0 2024-09-21T00:29:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f3443e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3444114390 0x7f34441a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f3443400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3444075dd0 0x7f34441a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f3443e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3444114390 0x7f34441a61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43958/0 (socket says 172.21.15.27:43958) 2024-09-21T00:29:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f3443e00640 1 -- 172.21.15.27:0/2551289657 learned_addr learned my addr 172.21.15.27:0/2551289657 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f3442a00640 1 --2- 172.21.15.27:0/2551289657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444077390 0x7f34441a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f3443e00640 1 -- 172.21.15.27:0/2551289657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3444075dd0 msgr2=0x7f34441a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f3443e00640 1 --2- 172.21.15.27:0/2551289657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3444075dd0 0x7f34441a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f3443e00640 1 -- 172.21.15.27:0/2551289657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444077390 msgr2=0x7f34441a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f3443e00640 1 --2- 172.21.15.27:0/2551289657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444077390 0x7f34441a0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f3443e00640 1 -- 172.21.15.27:0/2551289657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34441a68f0 con 0x7f3444114390 2024-09-21T00:29:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f3442a00640 1 --2- 172.21.15.27:0/2551289657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444077390 0x7f34441a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f3443e00640 1 --2- 172.21.15.27:0/2551289657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3444114390 0x7f34441a61f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f343c00eb10 tx=0x7f343c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f343be00640 1 -- 172.21.15.27:0/2551289657 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c004b20 con 0x7f3444114390 2024-09-21T00:29:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 -- 172.21.15.27:0/2551289657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34441a6be0 con 0x7f3444114390 2024-09-21T00:29:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f343be00640 1 -- 172.21.15.27:0/2551289657 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f343c004cc0 con 0x7f3444114390 2024-09-21T00:29:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f343be00640 1 -- 172.21.15.27:0/2551289657 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c010670 con 0x7f3444114390 2024-09-21T00:29:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.904+0000 7f344a800640 1 -- 172.21.15.27:0/2551289657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34441ad370 con 0x7f3444114390 2024-09-21T00:29:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3451873740 1 Processor -- start 2024-09-21T00:29:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f343be00640 1 -- 172.21.15.27:0/2551289657 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f343c005020 con 0x7f3444114390 2024-09-21T00:29:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3451873740 1 -- start start 2024-09-21T00:29:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3451873740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd9101b0 0x5578cdcb4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3451873740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd8e72f0 0x5578cdcb5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3451873740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8ef4d0 0x5578cdcb56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3451873740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5578cdc2a6b0 con 0x5578cd9101b0 2024-09-21T00:29:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3451873740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5578cdc19ae0 con 0x5578cd8e72f0 2024-09-21T00:29:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3451873740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5578cdc1d8f0 con 0x5578cd8ef4d0 2024-09-21T00:29:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3442a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd9101b0 0x5578cdcb4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3442a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd9101b0 0x5578cdcb4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43968/0 (socket says 172.21.15.27:43968) 2024-09-21T00:29:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3442a00640 1 -- 172.21.15.27:0/3339498426 learned_addr learned my addr 172.21.15.27:0/3339498426 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3442a00640 1 --2- 172.21.15.27:0/3339498426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8ef4d0 0x5578cdcb56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3442a00640 1 -- 172.21.15.27:0/3339498426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8ef4d0 msgr2=0x5578cdcb56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3442a00640 1 --2- 172.21.15.27:0/3339498426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8ef4d0 0x5578cdcb56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3442a00640 1 -- 172.21.15.27:0/3339498426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd8e72f0 msgr2=0x5578cdcb5170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3442a00640 1 --2- 172.21.15.27:0/3339498426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd8e72f0 0x5578cdcb5170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3442a00640 1 -- 172.21.15.27:0/3339498426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578cdbe6780 con 0x5578cd9101b0 2024-09-21T00:29:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3442a00640 1 --2- 172.21.15.27:0/3339498426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd9101b0 0x5578cdcb4c30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f34300139a0 tx=0x7f3430009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6a54de362c7cb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3438c00640 1 -- 172.21.15.27:0/3339498426 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430004030 con 0x5578cd9101b0 2024-09-21T00:29:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3438c00640 1 -- 172.21.15.27:0/3339498426 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f343000f940 con 0x5578cd9101b0 2024-09-21T00:29:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f343be00640 1 --2- 172.21.15.27:0/2551289657 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f34280b6680 0x7f34280b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3438c00640 1 -- 172.21.15.27:0/3339498426 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430003ab0 con 0x5578cd9101b0 2024-09-21T00:29:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f343be00640 1 -- 172.21.15.27:0/2551289657 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f343c0e81f0 con 0x7f3444114390 2024-09-21T00:29:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.910+0000 7f3443400640 1 --2- 172.21.15.27:0/2551289657 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f34280b6680 0x7f34280b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 -- 172.21.15.27:0/3339498426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd9101b0 msgr2=0x5578cdcb4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 --2- 172.21.15.27:0/3339498426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd9101b0 0x5578cdcb4c30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f34300139a0 tx=0x7f3430009c00 comp rx=0 tx=0).stop 2024-09-21T00:29:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3443400640 1 --2- 172.21.15.27:0/2551289657 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f34280b6680 0x7f34280b8b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3434031440 tx=0x7f3434005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 -- 172.21.15.27:0/3339498426 shutdown_connections 2024-09-21T00:29:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 --2- 172.21.15.27:0/3339498426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8ef4d0 0x5578cdcb56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 --2- 172.21.15.27:0/3339498426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd8e72f0 0x5578cdcb5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 --2- 172.21.15.27:0/3339498426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd9101b0 0x5578cdcb4c30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 -- 172.21.15.27:0/3339498426 >> 172.21.15.27:0/3339498426 conn(0x5578cd904300 msgr2=0x5578cdcb2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 -- 172.21.15.27:0/3339498426 shutdown_connections 2024-09-21T00:29:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 -- 172.21.15.27:0/3339498426 wait complete. 2024-09-21T00:29:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 -- start start 2024-09-21T00:29:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 Processor -- start 2024-09-21T00:29:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd8e72f0 0x5578cdcc02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8ef4d0 0x5578cdcc07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd9101b0 0x5578cdccc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5578cdb51ab0 con 0x5578cd8e72f0 2024-09-21T00:29:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5578cdbc0570 con 0x5578cd9101b0 2024-09-21T00:29:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5578cdcc33b0 con 0x5578cd8ef4d0 2024-09-21T00:29:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 10 client.0 _ll_register_callbacks cb 0x5578cdbcf100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3443400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd8e72f0 0x5578cdcc02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3443400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd8e72f0 0x5578cdcc02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43976/0 (socket says 172.21.15.27:43976) 2024-09-21T00:29:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3443400640 1 -- 172.21.15.27:0/268682508 learned_addr learned my addr 172.21.15.27:0/268682508 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3443e00640 1 --2- 172.21.15.27:0/268682508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd9101b0 0x5578cdccc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3443400640 1 -- 172.21.15.27:0/268682508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8ef4d0 msgr2=0x5578cdcc07e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3443400640 1 --2- 172.21.15.27:0/268682508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8ef4d0 0x5578cdcc07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3443400640 1 -- 172.21.15.27:0/268682508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd9101b0 msgr2=0x5578cdccc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3443400640 1 --2- 172.21.15.27:0/268682508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd9101b0 0x5578cdccc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3443400640 1 -- 172.21.15.27:0/268682508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578cd97b790 con 0x5578cd8e72f0 2024-09-21T00:29:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3443400640 1 --2- 172.21.15.27:0/268682508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd8e72f0 0x5578cdcc02a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3434033000 tx=0x7f3434009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6eaf9c6ddc07577 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f342d600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:29:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f342d600640 1 -- 172.21.15.27:0/268682508 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343404c020 con 0x5578cd8e72f0 2024-09-21T00:29:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f342d600640 1 -- 172.21.15.27:0/268682508 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f3434005240 con 0x5578cd8e72f0 2024-09-21T00:29:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 -- 172.21.15.27:0/268682508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f343000f940 con 0x5578cd8e72f0 2024-09-21T00:29:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f342d600640 1 -- 172.21.15.27:0/268682508 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3434057050 con 0x5578cd8e72f0 2024-09-21T00:29:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 10 client.16221 fetch_fsmap learned FSMap version 36 2024-09-21T00:29:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 -- 172.21.15.27:0/268682508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5578cd9f4620 con 0x5578cd8e72f0 2024-09-21T00:29:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f342d600640 1 -- 172.21.15.27:0/268682508 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f343000f940 con 0x5578cd8e72f0 2024-09-21T00:29:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f342d600640 1 -- 172.21.15.27:0/268682508 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f3434004c70 con 0x5578cd8e72f0 2024-09-21T00:29:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 10 client.16221 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T00:29:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:29:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 20 client.16221 populate_metadata read hostname 'smithi027' 2024-09-21T00:29:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 --2- 172.21.15.27:0/268682508 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5578cd8ced80 0x5578cdcd2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 4 client.16221 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:29:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3451873740 1 -- 172.21.15.27:0/268682508 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578cdc2c600 con 0x5578cd8ced80 2024-09-21T00:29:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.916+0000 7f3442a00640 1 --2- 172.21.15.27:0/268682508 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5578cd8ced80 0x5578cdcd2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.921+0000 7f3442a00640 1 --2- 172.21.15.27:0/268682508 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5578cd8ced80 0x5578cdcd2520 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b42a6dde4a7f63d0 server_cookie=88e60277c5c23502 in_seq=0 out_seq=0 2024-09-21T00:29:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.921+0000 7f342d600640 10 client.16221 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.921+0000 7f342d600640 1 -- 172.21.15.27:0/268682508 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5578cdc2c600 con 0x5578cd8ced80 2024-09-21T00:29:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.921+0000 7f342d600640 10 client.16221 handle_command_reply: tid=0 2024-09-21T00:29:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.922+0000 7f3451873740 1 client.16221 shutdown 2024-09-21T00:29:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.922+0000 7f3451873740 1 -- 172.21.15.27:0/268682508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd8e72f0 msgr2=0x5578cdcc02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.922+0000 7f3451873740 1 --2- 172.21.15.27:0/268682508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd8e72f0 0x5578cdcc02a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3434033000 tx=0x7f3434009260 comp rx=0 tx=0).stop 2024-09-21T00:29:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.922+0000 7f3451873740 1 -- 172.21.15.27:0/268682508 shutdown_connections 2024-09-21T00:29:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.922+0000 7f3451873740 1 --2- 172.21.15.27:0/268682508 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5578cd8ced80 0x5578cdcd2520 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.922+0000 7f3451873740 1 --2- 172.21.15.27:0/268682508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd9101b0 0x5578cdccc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.922+0000 7f3451873740 1 --2- 172.21.15.27:0/268682508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8ef4d0 0x5578cdcc07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.922+0000 7f3451873740 1 --2- 172.21.15.27:0/268682508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd8e72f0 0x5578cdcc02a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.922+0000 7f342d600640 0 client.16221 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.922+0000 7f3451873740 1 -- 172.21.15.27:0/268682508 >> 172.21.15.27:0/268682508 conn(0x5578cd904300 msgr2=0x5578cdcb5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.922+0000 7f3451873740 1 -- 172.21.15.27:0/268682508 shutdown_connections 2024-09-21T00:29:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.922+0000 7f3451873740 1 -- 172.21.15.27:0/268682508 wait complete. 2024-09-21T00:29:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.922+0000 7f3451873740 20 client.16221 trim_cache size 0 max 16384 2024-09-21T00:29:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3451873740 1 Processor -- start 2024-09-21T00:29:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3451873740 1 -- start start 2024-09-21T00:29:33.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3451873740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8e72f0 0x5578cdcc0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3451873740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd8ef4d0 0x5578cdcb2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3451873740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd9101b0 0x5578cdccc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3451873740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f34441b9e40 con 0x5578cd9101b0 2024-09-21T00:29:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3451873740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f34441b7680 con 0x5578cd8ef4d0 2024-09-21T00:29:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3451873740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f34441b54f0 con 0x5578cd8e72f0 2024-09-21T00:29:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3443e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd9101b0 0x5578cdccc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3443e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd9101b0 0x5578cdccc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43992/0 (socket says 172.21.15.27:43992) 2024-09-21T00:29:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3443e00640 1 -- 172.21.15.27:0/483316310 learned_addr learned my addr 172.21.15.27:0/483316310 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3443400640 1 --2- 172.21.15.27:0/483316310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8e72f0 0x5578cdcc0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3443400640 1 -- 172.21.15.27:0/483316310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd8ef4d0 msgr2=0x5578cdcb2b50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3443400640 1 --2- 172.21.15.27:0/483316310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd8ef4d0 0x5578cdcb2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3443400640 1 -- 172.21.15.27:0/483316310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd9101b0 msgr2=0x5578cdccc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3443400640 1 --2- 172.21.15.27:0/483316310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd9101b0 0x5578cdccc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3443400640 1 -- 172.21.15.27:0/483316310 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578cdcbe780 con 0x5578cd8e72f0 2024-09-21T00:29:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3443400640 1 --2- 172.21.15.27:0/483316310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8e72f0 0x5578cdcc0780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f34340036f0 tx=0x7f343402a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4262adc9af9ff37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f342fe00640 1 -- 172.21.15.27:0/483316310 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343405f020 con 0x5578cd8e72f0 2024-09-21T00:29:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f342fe00640 1 -- 172.21.15.27:0/483316310 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3434003d90 con 0x5578cd8e72f0 2024-09-21T00:29:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f342fe00640 1 -- 172.21.15.27:0/483316310 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343404c070 con 0x5578cd8e72f0 2024-09-21T00:29:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3451873740 1 -- 172.21.15.27:0/483316310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8e72f0 msgr2=0x5578cdcc0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3451873740 1 --2- 172.21.15.27:0/483316310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8e72f0 0x5578cdcc0780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f34340036f0 tx=0x7f343402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3451873740 1 -- 172.21.15.27:0/483316310 shutdown_connections 2024-09-21T00:29:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.946+0000 7f3451873740 1 --2- 172.21.15.27:0/483316310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd9101b0 0x5578cdccc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 --2- 172.21.15.27:0/483316310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd8ef4d0 0x5578cdcb2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 --2- 172.21.15.27:0/483316310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8e72f0 0x5578cdcc0780 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 -- 172.21.15.27:0/483316310 >> 172.21.15.27:0/483316310 conn(0x5578cd904300 msgr2=0x5578cdcb2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 -- 172.21.15.27:0/483316310 shutdown_connections 2024-09-21T00:29:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 -- 172.21.15.27:0/483316310 wait complete. 2024-09-21T00:29:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 -- start start 2024-09-21T00:29:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 Processor -- start 2024-09-21T00:29:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8e72f0 0x5578cdcc8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd8ef4d0 0x5578cdcc9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd9101b0 0x5578cdcc4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f34441b5b40 con 0x5578cd8ef4d0 2024-09-21T00:29:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f34441b6fb0 con 0x5578cd9101b0 2024-09-21T00:29:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f34441b6210 con 0x5578cd8e72f0 2024-09-21T00:29:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 10 client.0 _ll_register_callbacks cb 0x7f343403f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3443400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8e72f0 0x5578cdcc8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3442a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd8ef4d0 0x5578cdcc9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3443400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8e72f0 0x5578cdcc8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38420/0 (socket says 172.21.15.27:38420) 2024-09-21T00:29:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3443400640 1 -- 172.21.15.27:0/776611596 learned_addr learned my addr 172.21.15.27:0/776611596 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3443400640 1 -- 172.21.15.27:0/776611596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd9101b0 msgr2=0x5578cdcc4110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3443400640 1 --2- 172.21.15.27:0/776611596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd9101b0 0x5578cdcc4110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3443400640 1 -- 172.21.15.27:0/776611596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd8ef4d0 msgr2=0x5578cdcc9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3443400640 1 --2- 172.21.15.27:0/776611596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd8ef4d0 0x5578cdcc9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3443400640 1 -- 172.21.15.27:0/776611596 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578cdcc48a0 con 0x5578cd8e72f0 2024-09-21T00:29:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3442a00640 1 --2- 172.21.15.27:0/776611596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd8ef4d0 0x5578cdcc9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3443400640 1 --2- 172.21.15.27:0/776611596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8e72f0 0x5578cdcc8aa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3434003670 tx=0x7f34340318f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=557eee64aaaff67e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f342d600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:29:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f342d600640 1 -- 172.21.15.27:0/776611596 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343405f070 con 0x5578cd8e72f0 2024-09-21T00:29:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 -- 172.21.15.27:0/776611596 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3434003d90 con 0x5578cd8e72f0 2024-09-21T00:29:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f342d600640 1 -- 172.21.15.27:0/776611596 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f3434005230 con 0x5578cd8e72f0 2024-09-21T00:29:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f342d600640 1 -- 172.21.15.27:0/776611596 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3434070050 con 0x5578cd8e72f0 2024-09-21T00:29:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 10 client.26254 fetch_fsmap learned FSMap version 36 2024-09-21T00:29:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 -- 172.21.15.27:0/776611596 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5578cdd3cb60 con 0x5578cd8e72f0 2024-09-21T00:29:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f342d600640 1 -- 172.21.15.27:0/776611596 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3434003d90 con 0x5578cd8e72f0 2024-09-21T00:29:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f342d600640 1 -- 172.21.15.27:0/776611596 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f343406b030 con 0x5578cd8e72f0 2024-09-21T00:29:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 10 client.26254 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T00:29:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 10 client.26254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:29:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 20 client.26254 populate_metadata read hostname 'smithi027' 2024-09-21T00:29:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 --2- 172.21.15.27:0/776611596 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5578cd8a41c0 0x5578cdd3f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 4 client.26254 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:29:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3451873740 1 -- 172.21.15.27:0/776611596 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578cdc280b0 con 0x5578cd8a41c0 2024-09-21T00:29:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3442a00640 1 --2- 172.21.15.27:0/776611596 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5578cd8a41c0 0x5578cdd3f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f3442a00640 1 --2- 172.21.15.27:0/776611596 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5578cd8a41c0 0x5578cdd3f2e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a37aa7e78562b44c server_cookie=6200b1d1af3347d4 in_seq=0 out_seq=0 2024-09-21T00:29:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:33.951+0000 7f342d600640 10 client.26254 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.043+0000 7f342d600640 1 -- 172.21.15.27:0/776611596 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5578cdc280b0 con 0x5578cd8a41c0 2024-09-21T00:29:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.043+0000 7f342d600640 10 client.26254 handle_command_reply: tid=0 2024-09-21T00:29:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.043+0000 7f3451873740 1 client.26254 shutdown 2024-09-21T00:29:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.043+0000 7f3451873740 1 -- 172.21.15.27:0/776611596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8e72f0 msgr2=0x5578cdcc8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.043+0000 7f3451873740 1 --2- 172.21.15.27:0/776611596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8e72f0 0x5578cdcc8aa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3434003670 tx=0x7f34340318f0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.043+0000 7f3451873740 1 -- 172.21.15.27:0/776611596 shutdown_connections 2024-09-21T00:29:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.043+0000 7f3451873740 1 --2- 172.21.15.27:0/776611596 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5578cd8a41c0 0x5578cdd3f2e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.043+0000 7f3451873740 1 --2- 172.21.15.27:0/776611596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578cd9101b0 0x5578cdcc4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.043+0000 7f342d600640 0 client.26254 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.043+0000 7f3451873740 1 --2- 172.21.15.27:0/776611596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578cd8ef4d0 0x5578cdcc9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.043+0000 7f3451873740 1 --2- 172.21.15.27:0/776611596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578cd8e72f0 0x5578cdcc8aa0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.043+0000 7f3451873740 1 -- 172.21.15.27:0/776611596 >> 172.21.15.27:0/776611596 conn(0x5578cd904300 msgr2=0x5578cdcb10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.043+0000 7f3451873740 1 -- 172.21.15.27:0/776611596 shutdown_connections 2024-09-21T00:29:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.043+0000 7f3451873740 1 -- 172.21.15.27:0/776611596 wait complete. 2024-09-21T00:29:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.043+0000 7f3451873740 20 client.26254 trim_cache size 0 max 16384 2024-09-21T00:29:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:29:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:29:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:29:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:29:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.049+0000 7f344a800640 1 -- 172.21.15.27:0/2551289657 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f34280b6680 msgr2=0x7f34280b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.049+0000 7f344a800640 1 --2- 172.21.15.27:0/2551289657 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f34280b6680 0x7f34280b8b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3434031440 tx=0x7f3434005d50 comp rx=0 tx=0).stop 2024-09-21T00:29:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.049+0000 7f344a800640 1 -- 172.21.15.27:0/2551289657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3444114390 msgr2=0x7f34441a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.049+0000 7f344a800640 1 --2- 172.21.15.27:0/2551289657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3444114390 0x7f34441a61f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f343c00eb10 tx=0x7f343c00c2d0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.049+0000 7f344a800640 1 -- 172.21.15.27:0/2551289657 shutdown_connections 2024-09-21T00:29:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.049+0000 7f344a800640 1 --2- 172.21.15.27:0/2551289657 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f34280b6680 0x7f34280b8b90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.049+0000 7f344a800640 1 --2- 172.21.15.27:0/2551289657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3444114390 0x7f34441a61f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.049+0000 7f344a800640 1 --2- 172.21.15.27:0/2551289657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444077390 0x7f34441a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.049+0000 7f344a800640 1 --2- 172.21.15.27:0/2551289657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3444075dd0 0x7f34441a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.049+0000 7f344a800640 1 -- 172.21.15.27:0/2551289657 >> 172.21.15.27:0/2551289657 conn(0x7f34440ff4d0 msgr2=0x7f3444110290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.049+0000 7f344a800640 1 -- 172.21.15.27:0/2551289657 shutdown_connections 2024-09-21T00:29:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:34.049+0000 7f344a800640 1 -- 172.21.15.27:0/2551289657 wait complete. 2024-09-21T00:29:34.354 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:34 smithi027 ceph-mon[30745]: pgmap v629: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-09-21T00:29:34.531 DEBUG:teuthology.orchestra.run:got remote process result: 115 2024-09-21T00:29:34.532 WARNING:tasks.quiescer.fs.[cephfs]:Set '47099ce0' hasn't quiesced after 51.4 seconds (timeout: 90). Dumping ops with locks from all ranks. 2024-09-21T00:29:34.532 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:29:34.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:34 smithi089 ceph-mon[34481]: pgmap v629: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-09-21T00:29:34.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:34 smithi098 ceph-mon[34725]: pgmap v629: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-09-21T00:29:34.971 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:29:36.328 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:36 smithi027 ceph-mon[30745]: pgmap v630: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-09-21T00:29:36.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:36 smithi089 ceph-mon[34481]: pgmap v630: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-09-21T00:29:36.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:36 smithi098 ceph-mon[34725]: pgmap v630: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-09-21T00:29:36.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.684+0000 7f930f200640 1 -- 172.21.15.27:0/1051763192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93100757b0 msgr2=0x7f9310075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:36.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.684+0000 7f930f200640 1 --2- 172.21.15.27:0/1051763192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93100757b0 0x7f9310075bc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9300009900 tx=0x7f930002a410 comp rx=0 tx=0).stop 2024-09-21T00:29:36.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.684+0000 7f930f200640 1 -- 172.21.15.27:0/1051763192 shutdown_connections 2024-09-21T00:29:36.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.684+0000 7f930f200640 1 --2- 172.21.15.27:0/1051763192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f931010ff10 0x7f9310112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.684+0000 7f930f200640 1 --2- 172.21.15.27:0/1051763192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9310076100 0x7f931010f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.684+0000 7f930f200640 1 --2- 172.21.15.27:0/1051763192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93100757b0 0x7f9310075bc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.684+0000 7f930f200640 1 -- 172.21.15.27:0/1051763192 >> 172.21.15.27:0/1051763192 conn(0x7f93100ff440 msgr2=0x7f9310101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:36.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.684+0000 7f930f200640 1 -- 172.21.15.27:0/1051763192 shutdown_connections 2024-09-21T00:29:36.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.684+0000 7f930f200640 1 -- 172.21.15.27:0/1051763192 wait complete. 2024-09-21T00:29:36.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.684+0000 7f930f200640 1 Processor -- start 2024-09-21T00:29:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.684+0000 7f930f200640 1 -- start start 2024-09-21T00:29:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f930f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93100757b0 0x7f931019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f930f200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9310076100 0x7f931019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f930f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f931010ff10 0x7f93101a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f930f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9310115b10 con 0x7f931010ff10 2024-09-21T00:29:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f930f200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9310115990 con 0x7f93100757b0 2024-09-21T00:29:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f930f200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9310115c90 con 0x7f9310076100 2024-09-21T00:29:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f9307e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9310076100 0x7f931019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f930de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93100757b0 0x7f931019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f9307e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9310076100 0x7f931019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38432/0 (socket says 172.21.15.27:38432) 2024-09-21T00:29:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f930de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93100757b0 0x7f931019c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58592/0 (socket says 172.21.15.27:58592) 2024-09-21T00:29:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f9307e00640 1 -- 172.21.15.27:0/2689461397 learned_addr learned my addr 172.21.15.27:0/2689461397 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f930e800640 1 --2- 172.21.15.27:0/2689461397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f931010ff10 0x7f93101a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f9307e00640 1 -- 172.21.15.27:0/2689461397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93100757b0 msgr2=0x7f931019c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:36.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f9307e00640 1 --2- 172.21.15.27:0/2689461397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93100757b0 0x7f931019c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f9307e00640 1 -- 172.21.15.27:0/2689461397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f931010ff10 msgr2=0x7f93101a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f9307e00640 1 --2- 172.21.15.27:0/2689461397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f931010ff10 0x7f93101a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f9307e00640 1 -- 172.21.15.27:0/2689461397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93101a2410 con 0x7f9310076100 2024-09-21T00:29:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.689+0000 7f930e800640 1 --2- 172.21.15.27:0/2689461397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f931010ff10 0x7f93101a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.694+0000 7f9307e00640 1 --2- 172.21.15.27:0/2689461397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9310076100 0x7f931019ca40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f92f800e9c0 tx=0x7f92f800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.694+0000 7f9306a00640 1 -- 172.21.15.27:0/2689461397 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f800cd80 con 0x7f9310076100 2024-09-21T00:29:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.694+0000 7f9306a00640 1 -- 172.21.15.27:0/2689461397 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f80035f0 con 0x7f9310076100 2024-09-21T00:29:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.694+0000 7f930f200640 1 -- 172.21.15.27:0/2689461397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93101a2700 con 0x7f9310076100 2024-09-21T00:29:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.694+0000 7f9306a00640 1 -- 172.21.15.27:0/2689461397 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f8010690 con 0x7f9310076100 2024-09-21T00:29:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.694+0000 7f930f200640 1 -- 172.21.15.27:0/2689461397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93101a8e60 con 0x7f9310076100 2024-09-21T00:29:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.694+0000 7f930f200640 1 -- 172.21.15.27:0/2689461397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9310075bc0 con 0x7f9310076100 2024-09-21T00:29:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.694+0000 7f9306a00640 1 -- 172.21.15.27:0/2689461397 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f92f80037a0 con 0x7f9310076100 2024-09-21T00:29:36.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.699+0000 7f9306a00640 1 --2- 172.21.15.27:0/2689461397 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f92f40b6680 0x7f92f40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:36.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.699+0000 7f9306a00640 1 -- 172.21.15.27:0/2689461397 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f92f80e7ff0 con 0x7f9310076100 2024-09-21T00:29:36.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.699+0000 7f930de00640 1 --2- 172.21.15.27:0/2689461397 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f92f40b6680 0x7f92f40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:36.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.699+0000 7f930de00640 1 --2- 172.21.15.27:0/2689461397 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f92f40b6680 0x7f92f40b8b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9300000c00 tx=0x7f9300031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:36.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:36.721+0000 7f9306a00640 1 -- 172.21.15.27:0/2689461397 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f92f80ae170 con 0x7f9310076100 2024-09-21T00:29:37.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:37.128+0000 7f930f200640 1 -- 172.21.15.27:0/2689461397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f931019d5d0 con 0x7f9310076100 2024-09-21T00:29:37.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:37.142+0000 7f9306a00640 1 -- 172.21.15.27:0/2689461397 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11898 (secure 0 0 0) 0x7f92f80b2550 con 0x7f9310076100 2024-09-21T00:29:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:29:37.143 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":36,"btime":"2024-09-21T00:29:27:270994+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:29:27.235184+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26057":{"gid":26057,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/4028763758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":4028763758},{"type":"v1","addr":"172.21.15.89:6837","nonce":4028763758}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:29:37.143 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 36 2024-09-21T00:29:37.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:37.145+0000 7f930f200640 1 -- 172.21.15.27:0/2689461397 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f92f40b6680 msgr2=0x7f92f40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:37.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:37.145+0000 7f930f200640 1 --2- 172.21.15.27:0/2689461397 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f92f40b6680 0x7f92f40b8b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9300000c00 tx=0x7f9300031400 comp rx=0 tx=0).stop 2024-09-21T00:29:37.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:37.145+0000 7f930f200640 1 -- 172.21.15.27:0/2689461397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9310076100 msgr2=0x7f931019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:37.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:37.145+0000 7f930f200640 1 --2- 172.21.15.27:0/2689461397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9310076100 0x7f931019ca40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f92f800e9c0 tx=0x7f92f800ee80 comp rx=0 tx=0).stop 2024-09-21T00:29:37.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:37.145+0000 7f930f200640 1 -- 172.21.15.27:0/2689461397 shutdown_connections 2024-09-21T00:29:37.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:37.145+0000 7f930f200640 1 --2- 172.21.15.27:0/2689461397 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f92f40b6680 0x7f92f40b8b90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:37.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:37.145+0000 7f930f200640 1 --2- 172.21.15.27:0/2689461397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f931010ff10 0x7f93101a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:37.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:37.145+0000 7f930f200640 1 --2- 172.21.15.27:0/2689461397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9310076100 0x7f931019ca40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:37.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:37.145+0000 7f930f200640 1 --2- 172.21.15.27:0/2689461397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93100757b0 0x7f931019c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:37.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:37.145+0000 7f930f200640 1 -- 172.21.15.27:0/2689461397 >> 172.21.15.27:0/2689461397 conn(0x7f93100ff440 msgr2=0x7f9310100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:37.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:37.145+0000 7f930f200640 1 -- 172.21.15.27:0/2689461397 shutdown_connections 2024-09-21T00:29:37.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:37.145+0000 7f930f200640 1 -- 172.21.15.27:0/2689461397 wait complete. 2024-09-21T00:29:37.464 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:37 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2689461397' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:29:37.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:37 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2689461397' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:29:37.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:37 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2689461397' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:29:38.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:38 smithi027 ceph-mon[30745]: pgmap v631: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 7 op/s 2024-09-21T00:29:38.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:38 smithi098 ceph-mon[34725]: pgmap v631: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 7 op/s 2024-09-21T00:29:38.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:38 smithi089 ceph-mon[34481]: pgmap v631: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 7 op/s 2024-09-21T00:29:40.061 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 2 (i) to a remote file /var/run/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ops-47099ce0-1-mds.i.json 2024-09-21T00:29:40.061 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 ops --flags=locks --path=/var/run/ceph/ops-47099ce0-1-mds.i.json 2024-09-21T00:29:40.473 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:29:40.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:40 smithi027 ceph-mon[30745]: pgmap v632: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 8 op/s 2024-09-21T00:29:40.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:40 smithi089 ceph-mon[34481]: pgmap v632: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 8 op/s 2024-09-21T00:29:40.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:40 smithi098 ceph-mon[34725]: pgmap v632: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 8 op/s 2024-09-21T00:29:42.531 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:42 smithi027 ceph-mon[30745]: pgmap v633: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 8 op/s 2024-09-21T00:29:42.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 -- 172.21.15.27:0/1896819938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68301039b0 msgr2=0x7f6830103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 --2- 172.21.15.27:0/1896819938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68301039b0 0x7f6830103dc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6820009900 tx=0x7f682002a410 comp rx=0 tx=0).stop 2024-09-21T00:29:42.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 -- 172.21.15.27:0/1896819938 shutdown_connections 2024-09-21T00:29:42.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 --2- 172.21.15.27:0/1896819938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6830109c40 0x7f6830110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 --2- 172.21.15.27:0/1896819938 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68301092f0 0x7f6830109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 --2- 172.21.15.27:0/1896819938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68301039b0 0x7f6830103dc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 -- 172.21.15.27:0/1896819938 >> 172.21.15.27:0/1896819938 conn(0x7f68300ff440 msgr2=0x7f6830101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:42.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 -- 172.21.15.27:0/1896819938 shutdown_connections 2024-09-21T00:29:42.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 -- 172.21.15.27:0/1896819938 wait complete. 2024-09-21T00:29:42.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 Processor -- start 2024-09-21T00:29:42.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 -- start start 2024-09-21T00:29:42.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68301039b0 0x7f683019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68301092f0 0x7f683019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6830109c40 0x7f683019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6830113940 con 0x7f68301039b0 2024-09-21T00:29:42.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f68301137c0 con 0x7f68301092f0 2024-09-21T00:29:42.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.377+0000 7f682f200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6830113ac0 con 0x7f6830109c40 2024-09-21T00:29:42.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f682de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68301039b0 0x7f683019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f682de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68301039b0 0x7f683019a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44036/0 (socket says 172.21.15.27:44036) 2024-09-21T00:29:42.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f682de00640 1 -- 172.21.15.27:0/4052821996 learned_addr learned my addr 172.21.15.27:0/4052821996 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:42.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f682d400640 1 --2- 172.21.15.27:0/4052821996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68301092f0 0x7f683019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f682e800640 1 --2- 172.21.15.27:0/4052821996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6830109c40 0x7f683019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f682de00640 1 -- 172.21.15.27:0/4052821996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6830109c40 msgr2=0x7f683019fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f682de00640 1 --2- 172.21.15.27:0/4052821996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6830109c40 0x7f683019fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f682de00640 1 -- 172.21.15.27:0/4052821996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68301092f0 msgr2=0x7f683019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f682de00640 1 --2- 172.21.15.27:0/4052821996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68301092f0 0x7f683019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f682de00640 1 -- 172.21.15.27:0/4052821996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68301a02e0 con 0x7f68301039b0 2024-09-21T00:29:42.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f682d400640 1 --2- 172.21.15.27:0/4052821996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68301092f0 0x7f683019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:42.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f682de00640 1 --2- 172.21.15.27:0/4052821996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68301039b0 0x7f683019a370 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6820005940 tx=0x7f6820004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:42.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f6826a00640 1 -- 172.21.15.27:0/4052821996 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682003f030 con 0x7f68301039b0 2024-09-21T00:29:42.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f6826a00640 1 -- 172.21.15.27:0/4052821996 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6820003aa0 con 0x7f68301039b0 2024-09-21T00:29:42.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f6826a00640 1 -- 172.21.15.27:0/4052821996 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820035070 con 0x7f68301039b0 2024-09-21T00:29:42.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f682f200640 1 -- 172.21.15.27:0/4052821996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68301a0570 con 0x7f68301039b0 2024-09-21T00:29:42.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.383+0000 7f682f200640 1 -- 172.21.15.27:0/4052821996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68301a6d30 con 0x7f68301039b0 2024-09-21T00:29:42.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f683bc5b740 1 Processor -- start 2024-09-21T00:29:42.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f683bc5b740 1 -- start start 2024-09-21T00:29:42.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f683bc5b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745c8760 0x555c7496f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f683bc5b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c7459f790 0x555c7496f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f683bc5b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745a7970 0x555c7496fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f683bc5b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555c748e6450 con 0x555c7459f790 2024-09-21T00:29:42.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f683bc5b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555c748d5880 con 0x555c745c8760 2024-09-21T00:29:42.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f683bc5b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555c748d9690 con 0x555c745a7970 2024-09-21T00:29:42.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f682e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745c8760 0x555c7496f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f682e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745c8760 0x555c7496f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58624/0 (socket says 172.21.15.27:58624) 2024-09-21T00:29:42.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f682e800640 1 -- 172.21.15.27:0/668512098 learned_addr learned my addr 172.21.15.27:0/668512098 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:42.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f682d400640 1 --2- 172.21.15.27:0/668512098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745a7970 0x555c7496fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f682e800640 1 -- 172.21.15.27:0/668512098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745a7970 msgr2=0x555c7496fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:42.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f682e800640 1 --2- 172.21.15.27:0/668512098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745a7970 0x555c7496fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f682e800640 1 -- 172.21.15.27:0/668512098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c7459f790 msgr2=0x555c7496f570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:29:42.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f682e800640 1 --2- 172.21.15.27:0/668512098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c7459f790 0x555c7496f570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f682e800640 1 -- 172.21.15.27:0/668512098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c74887b00 con 0x555c745c8760 2024-09-21T00:29:42.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.385+0000 7f6826a00640 1 -- 172.21.15.27:0/4052821996 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6820042030 con 0x7f68301039b0 2024-09-21T00:29:42.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f682e800640 1 --2- 172.21.15.27:0/668512098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745c8760 0x555c7496f030 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6828009fd0 tx=0x7f68280118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8832eaa16cfd94e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:42.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f681b400640 1 -- 172.21.15.27:0/668512098 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828004d20 con 0x555c745c8760 2024-09-21T00:29:42.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f681b400640 1 -- 172.21.15.27:0/668512098 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6828004050 con 0x555c745c8760 2024-09-21T00:29:42.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f681b400640 1 -- 172.21.15.27:0/668512098 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828005260 con 0x555c745c8760 2024-09-21T00:29:42.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 -- 172.21.15.27:0/668512098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745c8760 msgr2=0x555c7496f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 --2- 172.21.15.27:0/668512098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745c8760 0x555c7496f030 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6828009fd0 tx=0x7f68280118d0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 -- 172.21.15.27:0/668512098 shutdown_connections 2024-09-21T00:29:42.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 --2- 172.21.15.27:0/668512098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745a7970 0x555c7496fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 --2- 172.21.15.27:0/668512098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c7459f790 0x555c7496f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 --2- 172.21.15.27:0/668512098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745c8760 0x555c7496f030 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 -- 172.21.15.27:0/668512098 >> 172.21.15.27:0/668512098 conn(0x555c745bc560 msgr2=0x555c7496cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:42.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f6826a00640 1 --2- 172.21.15.27:0/4052821996 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f68140b6700 0x7f68140b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 -- 172.21.15.27:0/668512098 shutdown_connections 2024-09-21T00:29:42.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f6826a00640 1 -- 172.21.15.27:0/4052821996 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f6820107f60 con 0x7f68301039b0 2024-09-21T00:29:42.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 -- 172.21.15.27:0/668512098 wait complete. 2024-09-21T00:29:42.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 -- start start 2024-09-21T00:29:42.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f682d400640 1 --2- 172.21.15.27:0/4052821996 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f68140b6700 0x7f68140b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 Processor -- start 2024-09-21T00:29:42.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c7459f790 0x555c7497a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745a7970 0x555c7497a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745c8760 0x555c7497aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f682d400640 1 --2- 172.21.15.27:0/4052821996 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f68140b6700 0x7f68140b8c10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6830108ed0 tx=0x7f681c009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:42.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555c74737380 con 0x555c7459f790 2024-09-21T00:29:42.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555c7487b260 con 0x555c745a7970 2024-09-21T00:29:42.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555c74981bc0 con 0x555c745c8760 2024-09-21T00:29:42.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f683bc5b740 10 client.0 _ll_register_callbacks cb 0x555c748cf7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:42.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f682d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745a7970 0x555c7497a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f682de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c7459f790 0x555c7497a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f682d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745a7970 0x555c7497a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58628/0 (socket says 172.21.15.27:58628) 2024-09-21T00:29:42.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f682d400640 1 -- 172.21.15.27:0/3627866500 learned_addr learned my addr 172.21.15.27:0/3627866500 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:42.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f682de00640 1 -- 172.21.15.27:0/3627866500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745c8760 msgr2=0x555c7497aeb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:42.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f682de00640 1 --2- 172.21.15.27:0/3627866500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745c8760 0x555c7497aeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f682de00640 1 -- 172.21.15.27:0/3627866500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745a7970 msgr2=0x555c7497a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:42.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f682de00640 1 --2- 172.21.15.27:0/3627866500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745a7970 0x555c7497a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.391+0000 7f682de00640 1 -- 172.21.15.27:0/3627866500 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c746a6490 con 0x555c7459f790 2024-09-21T00:29:42.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.396+0000 7f682de00640 1 --2- 172.21.15.27:0/3627866500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c7459f790 0x555c7497a430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6820105280 tx=0x7f68201057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41eef64e571e66c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:42.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.396+0000 7f680fe00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:29:42.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.396+0000 7f680fe00640 1 -- 172.21.15.27:0/3627866500 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68200d2070 con 0x555c7459f790 2024-09-21T00:29:42.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.396+0000 7f680fe00640 1 -- 172.21.15.27:0/3627866500 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f68200d7110 con 0x555c7459f790 2024-09-21T00:29:42.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.396+0000 7f683bc5b740 1 -- 172.21.15.27:0/3627866500 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6828004050 con 0x555c7459f790 2024-09-21T00:29:42.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.396+0000 7f680fe00640 1 -- 172.21.15.27:0/3627866500 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f68200cd2d0 con 0x555c7459f790 2024-09-21T00:29:42.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.396+0000 7f683bc5b740 10 client.16242 fetch_fsmap learned FSMap version 37 2024-09-21T00:29:42.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.396+0000 7f683bc5b740 1 -- 172.21.15.27:0/3627866500 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555c7498d560 con 0x555c7459f790 2024-09-21T00:29:42.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.397+0000 7f680fe00640 1 -- 172.21.15.27:0/3627866500 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6828004050 con 0x555c7459f790 2024-09-21T00:29:42.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.397+0000 7f680fe00640 1 -- 172.21.15.27:0/3627866500 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f68200e3b30 con 0x555c7459f790 2024-09-21T00:29:42.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.397+0000 7f683bc5b740 10 client.16242 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T00:29:42.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.397+0000 7f683bc5b740 10 client.16242 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T00:29:42.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.397+0000 7f683bc5b740 20 client.16242 populate_metadata read hostname 'smithi027' 2024-09-21T00:29:42.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.397+0000 7f683bc5b740 1 --2- 172.21.15.27:0/3627866500 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x555c74586fd0 0x555c749909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.397+0000 7f683bc5b740 4 client.16242 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:29:42.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.397+0000 7f683bc5b740 1 -- 172.21.15.27:0/3627866500 --> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c748e83a0 con 0x555c74586fd0 2024-09-21T00:29:42.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.421+0000 7f682d400640 1 --2- 172.21.15.27:0/3627866500 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x555c74586fd0 0x555c749909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.480+0000 7f682d400640 1 --2- 172.21.15.27:0/3627866500 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x555c74586fd0 0x555c749909c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b60effaa4d2ceeb9 server_cookie=517c784c3f8b45cc in_seq=0 out_seq=0 2024-09-21T00:29:42.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.480+0000 7f680fe00640 10 client.16242 ms_handle_connect on v2:172.21.15.98:6838/3074699335 2024-09-21T00:29:42.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.517+0000 7f680fe00640 1 -- 172.21.15.27:0/3627866500 <== mds.2 v2:172.21.15.98:6838/3074699335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555c748e83a0 con 0x555c74586fd0 2024-09-21T00:29:42.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.517+0000 7f680fe00640 10 client.16242 handle_command_reply: tid=0 2024-09-21T00:29:42.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.517+0000 7f683bc5b740 1 client.16242 shutdown 2024-09-21T00:29:42.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.517+0000 7f683bc5b740 1 -- 172.21.15.27:0/3627866500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c7459f790 msgr2=0x555c7497a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.517+0000 7f683bc5b740 1 --2- 172.21.15.27:0/3627866500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c7459f790 0x555c7497a430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6820105280 tx=0x7f68201057b0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.517+0000 7f683bc5b740 1 -- 172.21.15.27:0/3627866500 shutdown_connections 2024-09-21T00:29:42.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.517+0000 7f683bc5b740 1 --2- 172.21.15.27:0/3627866500 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x555c74586fd0 0x555c749909c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.517+0000 7f683bc5b740 1 --2- 172.21.15.27:0/3627866500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745c8760 0x555c7497aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.517+0000 7f683bc5b740 1 --2- 172.21.15.27:0/3627866500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745a7970 0x555c7497a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.517+0000 7f683bc5b740 1 --2- 172.21.15.27:0/3627866500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c7459f790 0x555c7497a430 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.517+0000 7f683bc5b740 1 -- 172.21.15.27:0/3627866500 >> 172.21.15.27:0/3627866500 conn(0x555c745bc560 msgr2=0x555c74970060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:42.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.517+0000 7f680fe00640 0 client.16242 ms_handle_reset on v2:172.21.15.98:6838/3074699335 2024-09-21T00:29:42.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.517+0000 7f683bc5b740 1 -- 172.21.15.27:0/3627866500 shutdown_connections 2024-09-21T00:29:42.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.517+0000 7f683bc5b740 1 -- 172.21.15.27:0/3627866500 wait complete. 2024-09-21T00:29:42.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.517+0000 7f683bc5b740 20 client.16242 trim_cache size 0 max 16384 2024-09-21T00:29:42.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 Processor -- start 2024-09-21T00:29:42.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 -- start start 2024-09-21T00:29:42.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7459f790 0x555c7498ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745a7970 0x555c7496cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c745c8760 0x555c74977760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b1620 con 0x555c745c8760 2024-09-21T00:29:42.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f68301aee90 con 0x555c7459f790 2024-09-21T00:29:42.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f68301acd00 con 0x555c745a7970 2024-09-21T00:29:42.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f682e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c745c8760 0x555c74977760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f682d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745a7970 0x555c7496cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f682d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745a7970 0x555c7496cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38476/0 (socket says 172.21.15.27:38476) 2024-09-21T00:29:42.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f682d400640 1 -- 172.21.15.27:0/3415886758 learned_addr learned my addr 172.21.15.27:0/3415886758 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:42.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f682de00640 1 --2- 172.21.15.27:0/3415886758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7459f790 0x555c7498ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f682d400640 1 -- 172.21.15.27:0/3415886758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7459f790 msgr2=0x555c7498ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:42.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f682d400640 1 --2- 172.21.15.27:0/3415886758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7459f790 0x555c7498ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f682d400640 1 -- 172.21.15.27:0/3415886758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c745c8760 msgr2=0x555c74977760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:42.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f682d400640 1 --2- 172.21.15.27:0/3415886758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c745c8760 0x555c74977760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f682d400640 1 -- 172.21.15.27:0/3415886758 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c7497b6f0 con 0x555c745a7970 2024-09-21T00:29:42.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f682de00640 1 --2- 172.21.15.27:0/3415886758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7459f790 0x555c7498ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:42.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f682d400640 1 --2- 172.21.15.27:0/3415886758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745a7970 0x555c7496cc60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x555c749910f0 tx=0x7f681c018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d666120a34dbf6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:42.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f681aa00640 1 -- 172.21.15.27:0/3415886758 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c030070 con 0x555c745a7970 2024-09-21T00:29:42.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f681aa00640 1 -- 172.21.15.27:0/3415886758 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f681c01b040 con 0x555c745a7970 2024-09-21T00:29:42.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f681aa00640 1 -- 172.21.15.27:0/3415886758 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c0224d0 con 0x555c745a7970 2024-09-21T00:29:42.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 -- 172.21.15.27:0/3415886758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745a7970 msgr2=0x555c7496cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 --2- 172.21.15.27:0/3415886758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745a7970 0x555c7496cc60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x555c749910f0 tx=0x7f681c018000 comp rx=0 tx=0).stop 2024-09-21T00:29:42.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 -- 172.21.15.27:0/3415886758 shutdown_connections 2024-09-21T00:29:42.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 --2- 172.21.15.27:0/3415886758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c745c8760 0x555c74977760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 --2- 172.21.15.27:0/3415886758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c745a7970 0x555c7496cc60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 --2- 172.21.15.27:0/3415886758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7459f790 0x555c7498ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 -- 172.21.15.27:0/3415886758 >> 172.21.15.27:0/3415886758 conn(0x555c745bc560 msgr2=0x555c7496c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:42.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.547+0000 7f683bc5b740 1 -- 172.21.15.27:0/3415886758 shutdown_connections 2024-09-21T00:29:42.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.552+0000 7f683bc5b740 1 -- 172.21.15.27:0/3415886758 wait complete. 2024-09-21T00:29:42.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.552+0000 7f683bc5b740 1 -- start start 2024-09-21T00:29:42.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.552+0000 7f683bc5b740 1 Processor -- start 2024-09-21T00:29:42.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7459f790 0x555c74987040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745a7970 0x555c74987580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c745c8760 0x555c74983e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f68301ad350 con 0x555c745c8760 2024-09-21T00:29:42.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f68301ae7c0 con 0x555c745a7970 2024-09-21T00:29:42.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f68301ada20 con 0x555c7459f790 2024-09-21T00:29:42.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 10 client.0 _ll_register_callbacks cb 0x7f6820009230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:42.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f682de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7459f790 0x555c74987040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f682de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7459f790 0x555c74987040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38484/0 (socket says 172.21.15.27:38484) 2024-09-21T00:29:42.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f682de00640 1 -- 172.21.15.27:0/2300170258 learned_addr learned my addr 172.21.15.27:0/2300170258 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:42.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f682e800640 1 --2- 172.21.15.27:0/2300170258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c745c8760 0x555c74983e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f682d400640 1 --2- 172.21.15.27:0/2300170258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745a7970 0x555c74987580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f682de00640 1 -- 172.21.15.27:0/2300170258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745a7970 msgr2=0x555c74987580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:42.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f682de00640 1 --2- 172.21.15.27:0/2300170258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745a7970 0x555c74987580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f682de00640 1 -- 172.21.15.27:0/2300170258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c745c8760 msgr2=0x555c74983e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:42.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f682de00640 1 --2- 172.21.15.27:0/2300170258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c745c8760 0x555c74983e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f682de00640 1 -- 172.21.15.27:0/2300170258 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c74984480 con 0x555c7459f790 2024-09-21T00:29:42.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f682de00640 1 --2- 172.21.15.27:0/2300170258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7459f790 0x555c74987040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f68200319b0 tx=0x7f68201081e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c4add767cf8066d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:42.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f680fe00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:29:42.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f680fe00640 1 -- 172.21.15.27:0/2300170258 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68200d2040 con 0x555c7459f790 2024-09-21T00:29:42.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f680fe00640 1 -- 172.21.15.27:0/2300170258 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f68200d6f10 con 0x555c7459f790 2024-09-21T00:29:42.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 1 -- 172.21.15.27:0/2300170258 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f681c01b040 con 0x555c7459f790 2024-09-21T00:29:42.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f680fe00640 1 -- 172.21.15.27:0/2300170258 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f68200cd0f0 con 0x555c7459f790 2024-09-21T00:29:42.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 10 client.26272 fetch_fsmap learned FSMap version 37 2024-09-21T00:29:42.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 1 -- 172.21.15.27:0/2300170258 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555c74984480 con 0x555c7459f790 2024-09-21T00:29:42.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f680fe00640 1 -- 172.21.15.27:0/2300170258 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f681c01b040 con 0x555c7459f790 2024-09-21T00:29:42.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f680fe00640 1 -- 172.21.15.27:0/2300170258 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f68200c92d0 con 0x555c7459f790 2024-09-21T00:29:42.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 10 client.26272 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T00:29:42.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 10 client.26272 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T00:29:42.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 20 client.26272 populate_metadata read hostname 'smithi027' 2024-09-21T00:29:42.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 1 --2- 172.21.15.27:0/2300170258 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x555c7455c380 0x555c749fe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 4 client.26272 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-47099ce0-1-mds.i.json"}] 2024-09-21T00:29:42.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.553+0000 7f683bc5b740 1 -- 172.21.15.27:0/2300170258 --> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-47099ce0-1-mds.i.json"}) -- 0x555c748e3e50 con 0x555c7455c380 2024-09-21T00:29:42.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.565+0000 7f682d400640 1 --2- 172.21.15.27:0/2300170258 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x555c7455c380 0x555c749fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.615+0000 7f680fe00640 10 client.26272 ms_handle_connect on v2:172.21.15.98:6838/3074699335 2024-09-21T00:29:42.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:42.615+0000 7f682d400640 1 --2- 172.21.15.27:0/2300170258 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x555c7455c380 0x555c749fe140 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6be14cedf97fbdb3 server_cookie=e10680bc5fc5234c in_seq=0 out_seq=0 2024-09-21T00:29:42.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:42 smithi089 ceph-mon[34481]: pgmap v633: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 8 op/s 2024-09-21T00:29:42.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:42 smithi098 ceph-mon[34725]: pgmap v633: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 8 op/s 2024-09-21T00:29:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f680fe00640 1 -- 172.21.15.27:0/2300170258 <== mds.2 v2:172.21.15.98:6838/3074699335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x555c748e3e50 con 0x555c7455c380 2024-09-21T00:29:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f680fe00640 10 client.26272 handle_command_reply: tid=0 2024-09-21T00:29:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f683bc5b740 1 client.26272 shutdown 2024-09-21T00:29:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f683bc5b740 1 -- 172.21.15.27:0/2300170258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7459f790 msgr2=0x555c74987040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f683bc5b740 1 --2- 172.21.15.27:0/2300170258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7459f790 0x555c74987040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f68200319b0 tx=0x7f68201081e0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f683bc5b740 1 -- 172.21.15.27:0/2300170258 shutdown_connections 2024-09-21T00:29:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f683bc5b740 1 --2- 172.21.15.27:0/2300170258 >> [v2:172.21.15.98:6838/3074699335,v1:172.21.15.98:6839/3074699335] conn(0x555c7455c380 0x555c749fe140 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f683bc5b740 1 --2- 172.21.15.27:0/2300170258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555c745c8760 0x555c74983e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f683bc5b740 1 --2- 172.21.15.27:0/2300170258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c745a7970 0x555c74987580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f683bc5b740 1 --2- 172.21.15.27:0/2300170258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7459f790 0x555c74987040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f683bc5b740 1 -- 172.21.15.27:0/2300170258 >> 172.21.15.27:0/2300170258 conn(0x555c745bc560 msgr2=0x555c7497b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:43.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f680fe00640 0 client.26272 ms_handle_reset on v2:172.21.15.98:6838/3074699335 2024-09-21T00:29:43.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f683bc5b740 1 -- 172.21.15.27:0/2300170258 shutdown_connections 2024-09-21T00:29:43.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f683bc5b740 1 -- 172.21.15.27:0/2300170258 wait complete. 2024-09-21T00:29:43.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f683bc5b740 20 client.26272 trim_cache size 0 max 16384 2024-09-21T00:29:43.351 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:29:43.351 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/var/run/ceph/ops-47099ce0-1-mds.i.json" 2024-09-21T00:29:43.351 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:29:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f682f200640 1 -- 172.21.15.27:0/4052821996 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f68140b6700 msgr2=0x7f68140b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f682f200640 1 --2- 172.21.15.27:0/4052821996 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f68140b6700 0x7f68140b8c10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6830108ed0 tx=0x7f681c009110 comp rx=0 tx=0).stop 2024-09-21T00:29:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f682f200640 1 -- 172.21.15.27:0/4052821996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68301039b0 msgr2=0x7f683019a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f682f200640 1 --2- 172.21.15.27:0/4052821996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68301039b0 0x7f683019a370 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6820005940 tx=0x7f6820004270 comp rx=0 tx=0).stop 2024-09-21T00:29:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f682f200640 1 -- 172.21.15.27:0/4052821996 shutdown_connections 2024-09-21T00:29:43.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f682f200640 1 --2- 172.21.15.27:0/4052821996 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f68140b6700 0x7f68140b8c10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f682f200640 1 --2- 172.21.15.27:0/4052821996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6830109c40 0x7f683019fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f682f200640 1 --2- 172.21.15.27:0/4052821996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68301092f0 0x7f683019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f682f200640 1 --2- 172.21.15.27:0/4052821996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68301039b0 0x7f683019a370 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f682f200640 1 -- 172.21.15.27:0/4052821996 >> 172.21.15.27:0/4052821996 conn(0x7f68300ff440 msgr2=0x7f6830101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:43.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f682f200640 1 -- 172.21.15.27:0/4052821996 shutdown_connections 2024-09-21T00:29:43.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:43.348+0000 7f682f200640 1 -- 172.21.15.27:0/4052821996 wait complete. 2024-09-21T00:29:43.638 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:43 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:29:43.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:43 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:29:43.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:43 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:29:44.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:44 smithi027 ceph-mon[30745]: pgmap v634: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-09-21T00:29:44.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:44 smithi089 ceph-mon[34481]: pgmap v634: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-09-21T00:29:44.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:44 smithi098 ceph-mon[34725]: pgmap v634: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-09-21T00:29:45.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:45 smithi027 ceph-mon[30745]: pgmap v635: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 6 op/s 2024-09-21T00:29:45.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:45 smithi089 ceph-mon[34481]: pgmap v635: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 6 op/s 2024-09-21T00:29:45.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:45 smithi098 ceph-mon[34725]: pgmap v635: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 6 op/s 2024-09-21T00:29:46.142 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 1 (a) to a remote file /var/run/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ops-47099ce0-1-mds.a.json 2024-09-21T00:29:46.142 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 ops --flags=locks --path=/var/run/ceph/ops-47099ce0-1-mds.a.json 2024-09-21T00:29:46.551 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:29:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.312+0000 7fb7fb200640 1 -- 172.21.15.27:0/810087148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7fc109b90 msgr2=0x7fb7fc1103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.312+0000 7fb7fb200640 1 --2- 172.21.15.27:0/810087148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7fc109b90 0x7fb7fc1103c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009ca0 tx=0x7fb7f402a8e0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.312+0000 7fb7fb200640 1 -- 172.21.15.27:0/810087148 shutdown_connections 2024-09-21T00:29:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.312+0000 7fb7fb200640 1 --2- 172.21.15.27:0/810087148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7fc109b90 0x7fb7fc1103c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.312+0000 7fb7fb200640 1 --2- 172.21.15.27:0/810087148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7fc109240 0x7fb7fc109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.312+0000 7fb7fb200640 1 --2- 172.21.15.27:0/810087148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7fc103910 0x7fb7fc103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.312+0000 7fb7fb200640 1 -- 172.21.15.27:0/810087148 >> 172.21.15.27:0/810087148 conn(0x7fb7fc0ff390 msgr2=0x7fb7fc1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.312+0000 7fb7fb200640 1 -- 172.21.15.27:0/810087148 shutdown_connections 2024-09-21T00:29:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.312+0000 7fb7fb200640 1 -- 172.21.15.27:0/810087148 wait complete. 2024-09-21T00:29:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.312+0000 7fb7fb200640 1 Processor -- start 2024-09-21T00:29:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7fb200640 1 -- start start 2024-09-21T00:29:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7fb200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7fc103910 0x7fb7fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7fb200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7fc109240 0x7fb7fc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7fb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7fc109b90 0x7fb7fc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7fb200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc113890 con 0x7fb7fc109b90 2024-09-21T00:29:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7fb200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc113710 con 0x7fb7fc109240 2024-09-21T00:29:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7fb200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc113a10 con 0x7fb7fc103910 2024-09-21T00:29:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7f9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7fc103910 0x7fb7fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7f9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7fc103910 0x7fb7fc075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:54674/0 (socket says 172.21.15.27:54674) 2024-09-21T00:29:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7f9e00640 1 -- 172.21.15.27:0/389464021 learned_addr learned my addr 172.21.15.27:0/389464021 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7f9400640 1 --2- 172.21.15.27:0/389464021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7fc109240 0x7fb7fc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7fa800640 1 --2- 172.21.15.27:0/389464021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7fc109b90 0x7fb7fc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7f9e00640 1 -- 172.21.15.27:0/389464021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7fc109240 msgr2=0x7fb7fc075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7f9e00640 1 --2- 172.21.15.27:0/389464021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7fc109240 0x7fb7fc075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7f9e00640 1 -- 172.21.15.27:0/389464021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7fc109b90 msgr2=0x7fb7fc076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7f9e00640 1 --2- 172.21.15.27:0/389464021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7fc109b90 0x7fb7fc076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7f9e00640 1 -- 172.21.15.27:0/389464021 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7fc1a8560 con 0x7fb7fc103910 2024-09-21T00:29:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7f9e00640 1 --2- 172.21.15.27:0/389464021 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7fc103910 0x7fb7fc075710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00e9f0 tx=0x7fb7ec00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7f2a00640 1 -- 172.21.15.27:0/389464021 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00ccc0 con 0x7fb7fc103910 2024-09-21T00:29:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7f2a00640 1 -- 172.21.15.27:0/389464021 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ec0108e0 con 0x7fb7fc103910 2024-09-21T00:29:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7f2a00640 1 -- 172.21.15.27:0/389464021 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec018880 con 0x7fb7fc103910 2024-09-21T00:29:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7fb200640 1 -- 172.21.15.27:0/389464021 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7fc1a8850 con 0x7fb7fc103910 2024-09-21T00:29:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.317+0000 7fb7fb200640 1 -- 172.21.15.27:0/389464021 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7fc1a8dd0 con 0x7fb7fc103910 2024-09-21T00:29:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.322+0000 7fb807aa3740 1 Processor -- start 2024-09-21T00:29:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb807aa3740 1 -- start start 2024-09-21T00:29:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb7f2a00640 1 -- 172.21.15.27:0/389464021 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb7ec010430 con 0x7fb7fc103910 2024-09-21T00:29:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb807aa3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 0x56321123ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb807aa3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e6f2f0 0x56321123d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb807aa3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321123d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb807aa3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5632111b26b0 con 0x563210e774d0 2024-09-21T00:29:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb807aa3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5632111a1ae0 con 0x563210e6f2f0 2024-09-21T00:29:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb807aa3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5632111a58f0 con 0x563210e981b0 2024-09-21T00:29:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb7f9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e6f2f0 0x56321123d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb7f9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e6f2f0 0x56321123d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40742/0 (socket says 172.21.15.27:40742) 2024-09-21T00:29:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb7f9e00640 1 -- 172.21.15.27:0/3297686040 learned_addr learned my addr 172.21.15.27:0/3297686040 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb7f9400640 1 --2- 172.21.15.27:0/3297686040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321123d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb7f9e00640 1 -- 172.21.15.27:0/3297686040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 msgr2=0x56321123ce10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb7f9e00640 1 --2- 172.21.15.27:0/3297686040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 0x56321123ce10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb7f9e00640 1 -- 172.21.15.27:0/3297686040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 msgr2=0x56321123d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb7f9e00640 1 --2- 172.21.15.27:0/3297686040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321123d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb7f9e00640 1 -- 172.21.15.27:0/3297686040 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321116e780 con 0x563210e6f2f0 2024-09-21T00:29:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb7f2a00640 1 --2- 172.21.15.27:0/389464021 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb7e00b6720 0x7fb7e00b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb7f2a00640 1 -- 172.21.15.27:0/389464021 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7fb7ec0e7e10 con 0x7fb7fc103910 2024-09-21T00:29:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb7fa800640 1 --2- 172.21.15.27:0/389464021 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb7e00b6720 0x7fb7e00b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.324+0000 7fb7fa800640 1 --2- 172.21.15.27:0/389464021 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb7e00b6720 0x7fb7e00b8c30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4006010 tx=0x7fb7f4009400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7f9e00640 1 --2- 172.21.15.27:0/3297686040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e6f2f0 0x56321123d350 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0e50d0 tx=0x7fb7ec0ebf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9091b63a5a9e87e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7e7000640 1 -- 172.21.15.27:0/3297686040 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec0a9ee0 con 0x563210e6f2f0 2024-09-21T00:29:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7e7000640 1 -- 172.21.15.27:0/3297686040 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ec010720 con 0x563210e6f2f0 2024-09-21T00:29:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7e7000640 1 -- 172.21.15.27:0/3297686040 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec0ad000 con 0x563210e6f2f0 2024-09-21T00:29:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 -- 172.21.15.27:0/3297686040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e6f2f0 msgr2=0x56321123d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 --2- 172.21.15.27:0/3297686040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e6f2f0 0x56321123d350 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0e50d0 tx=0x7fb7ec0ebf90 comp rx=0 tx=0).stop 2024-09-21T00:29:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 -- 172.21.15.27:0/3297686040 shutdown_connections 2024-09-21T00:29:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 --2- 172.21.15.27:0/3297686040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321123d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 --2- 172.21.15.27:0/3297686040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e6f2f0 0x56321123d350 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 --2- 172.21.15.27:0/3297686040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 0x56321123ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 -- 172.21.15.27:0/3297686040 >> 172.21.15.27:0/3297686040 conn(0x563210e8c300 msgr2=0x56321123ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 -- 172.21.15.27:0/3297686040 shutdown_connections 2024-09-21T00:29:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 -- 172.21.15.27:0/3297686040 wait complete. 2024-09-21T00:29:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 -- start start 2024-09-21T00:29:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 Processor -- start 2024-09-21T00:29:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e6f2f0 0x563211248440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e774d0 0x563211248980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 0x563211248f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5632110d9ab0 con 0x563210e6f2f0 2024-09-21T00:29:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563211148570 con 0x563210e774d0 2024-09-21T00:29:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56321124b590 con 0x563210e981b0 2024-09-21T00:29:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 10 client.0 _ll_register_callbacks cb 0x563211157100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7f9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e6f2f0 0x563211248440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7f9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e6f2f0 0x563211248440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51226/0 (socket says 172.21.15.27:51226) 2024-09-21T00:29:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7f9e00640 1 -- 172.21.15.27:0/119837733 learned_addr learned my addr 172.21.15.27:0/119837733 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7f9400640 1 --2- 172.21.15.27:0/119837733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e774d0 0x563211248980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7f9e00640 1 -- 172.21.15.27:0/119837733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 msgr2=0x563211248f00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7f9e00640 1 --2- 172.21.15.27:0/119837733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 0x563211248f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7f9e00640 1 -- 172.21.15.27:0/119837733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e774d0 msgr2=0x563211248980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7f9e00640 1 --2- 172.21.15.27:0/119837733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e774d0 0x563211248980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7f9e00640 1 -- 172.21.15.27:0/119837733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563210f03790 con 0x563210e6f2f0 2024-09-21T00:29:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7f9e00640 1 --2- 172.21.15.27:0/119837733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e6f2f0 0x563211248440 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0e5240 tx=0x7fb7ec010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357595801f7e7122 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7db400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:29:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7db400640 1 -- 172.21.15.27:0/119837733 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec0ad1e0 con 0x563210e6f2f0 2024-09-21T00:29:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7db400640 1 -- 172.21.15.27:0/119837733 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7fb7ec0a9b00 con 0x563210e6f2f0 2024-09-21T00:29:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 -- 172.21.15.27:0/119837733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5632111b4600 con 0x563210e6f2f0 2024-09-21T00:29:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7db400640 1 -- 172.21.15.27:0/119837733 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb7ec0ad660 con 0x563210e6f2f0 2024-09-21T00:29:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 10 client.16260 fetch_fsmap learned FSMap version 37 2024-09-21T00:29:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 1 -- 172.21.15.27:0/119837733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563210f7c620 con 0x563210e6f2f0 2024-09-21T00:29:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7db400640 1 -- 172.21.15.27:0/119837733 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5632111b4600 con 0x563210e6f2f0 2024-09-21T00:29:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb7db400640 1 -- 172.21.15.27:0/119837733 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fb7ec0b6e20 con 0x563210e6f2f0 2024-09-21T00:29:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.347+0000 7fb807aa3740 10 client.16260 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T00:29:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 10 client.16260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:29:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 20 client.16260 populate_metadata read hostname 'smithi027' 2024-09-21T00:29:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 1 --2- 172.21.15.27:0/119837733 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563210e56d80 0x56321125a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 4 client.16260 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:29:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 1 -- 172.21.15.27:0/119837733 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632111b00b0 con 0x563210e56d80 2024-09-21T00:29:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb7f9400640 1 --2- 172.21.15.27:0/119837733 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563210e56d80 0x56321125a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb7f9400640 1 --2- 172.21.15.27:0/119837733 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563210e56d80 0x56321125a3c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf4509f93e12b212 server_cookie=b2aa53634deac9b8 in_seq=0 out_seq=0 2024-09-21T00:29:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb7db400640 10 client.16260 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb7db400640 1 -- 172.21.15.27:0/119837733 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5632111b00b0 con 0x563210e56d80 2024-09-21T00:29:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb7db400640 10 client.16260 handle_command_reply: tid=0 2024-09-21T00:29:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 1 client.16260 shutdown 2024-09-21T00:29:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 1 -- 172.21.15.27:0/119837733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e6f2f0 msgr2=0x563211248440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 1 --2- 172.21.15.27:0/119837733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e6f2f0 0x563211248440 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0e5240 tx=0x7fb7ec010cc0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 1 -- 172.21.15.27:0/119837733 shutdown_connections 2024-09-21T00:29:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 1 --2- 172.21.15.27:0/119837733 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563210e56d80 0x56321125a3c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 1 --2- 172.21.15.27:0/119837733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 0x563211248f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 1 --2- 172.21.15.27:0/119837733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e774d0 0x563211248980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 1 --2- 172.21.15.27:0/119837733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e6f2f0 0x563211248440 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb7db400640 0 client.16260 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 1 -- 172.21.15.27:0/119837733 >> 172.21.15.27:0/119837733 conn(0x563210e8c300 msgr2=0x56321123de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 1 -- 172.21.15.27:0/119837733 shutdown_connections 2024-09-21T00:29:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 1 -- 172.21.15.27:0/119837733 wait complete. 2024-09-21T00:29:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.353+0000 7fb807aa3740 20 client.16260 trim_cache size 0 max 16384 2024-09-21T00:29:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.376+0000 7fb807aa3740 1 Processor -- start 2024-09-21T00:29:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb807aa3740 1 -- start start 2024-09-21T00:29:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb807aa3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e6f2f0 0x563211253ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb807aa3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321125d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb807aa3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 0x56321125d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb807aa3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1b59c0 con 0x563210e774d0 2024-09-21T00:29:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb807aa3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1b3230 con 0x563210e6f2f0 2024-09-21T00:29:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb807aa3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1b10a0 con 0x563210e981b0 2024-09-21T00:29:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb7f9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321125d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb7f9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321125d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51230/0 (socket says 172.21.15.27:51230) 2024-09-21T00:29:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb7f9400640 1 -- 172.21.15.27:0/164412979 learned_addr learned my addr 172.21.15.27:0/164412979 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb7fa800640 1 --2- 172.21.15.27:0/164412979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 0x56321125d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb7f9400640 1 -- 172.21.15.27:0/164412979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 msgr2=0x56321125d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb7f9400640 1 --2- 172.21.15.27:0/164412979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 0x56321125d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb7f9400640 1 -- 172.21.15.27:0/164412979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e6f2f0 msgr2=0x563211253ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb7f9400640 1 --2- 172.21.15.27:0/164412979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e6f2f0 0x563211253ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.381+0000 7fb7f9400640 1 -- 172.21.15.27:0/164412979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321123b600 con 0x563210e774d0 2024-09-21T00:29:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7f9400640 1 --2- 172.21.15.27:0/164412979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321125d3f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc06c3d0 tx=0x7fb7e8009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c84e1a93e1e519 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7e6600640 1 -- 172.21.15.27:0/164412979 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8025380 con 0x563210e774d0 2024-09-21T00:29:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7e6600640 1 -- 172.21.15.27:0/164412979 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e80259d0 con 0x563210e774d0 2024-09-21T00:29:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7e6600640 1 -- 172.21.15.27:0/164412979 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8015800 con 0x563210e774d0 2024-09-21T00:29:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 -- 172.21.15.27:0/164412979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 msgr2=0x56321125d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 --2- 172.21.15.27:0/164412979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321125d3f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc06c3d0 tx=0x7fb7e8009110 comp rx=0 tx=0).stop 2024-09-21T00:29:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 -- 172.21.15.27:0/164412979 shutdown_connections 2024-09-21T00:29:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 --2- 172.21.15.27:0/164412979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 0x56321125d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 --2- 172.21.15.27:0/164412979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321125d3f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 --2- 172.21.15.27:0/164412979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e6f2f0 0x563211253ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 -- 172.21.15.27:0/164412979 >> 172.21.15.27:0/164412979 conn(0x563210e8c300 msgr2=0x5632112537e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 -- 172.21.15.27:0/164412979 shutdown_connections 2024-09-21T00:29:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 -- 172.21.15.27:0/164412979 wait complete. 2024-09-21T00:29:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 -- start start 2024-09-21T00:29:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 Processor -- start 2024-09-21T00:29:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e6f2f0 0x56321125d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321125d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 0x5632112bacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1b16f0 con 0x563210e774d0 2024-09-21T00:29:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1b2b60 con 0x563210e6f2f0 2024-09-21T00:29:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1b1dc0 con 0x563210e981b0 2024-09-21T00:29:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 10 client.0 _ll_register_callbacks cb 0x7fb7e80093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7f9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321125d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7f9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321125d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51244/0 (socket says 172.21.15.27:51244) 2024-09-21T00:29:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7f9400640 1 -- 172.21.15.27:0/3037158229 learned_addr learned my addr 172.21.15.27:0/3037158229 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7fa800640 1 --2- 172.21.15.27:0/3037158229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 0x5632112bacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7f9400640 1 -- 172.21.15.27:0/3037158229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 msgr2=0x5632112bacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7f9400640 1 --2- 172.21.15.27:0/3037158229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 0x5632112bacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7f9400640 1 -- 172.21.15.27:0/3037158229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e6f2f0 msgr2=0x56321125d1f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7f9400640 1 --2- 172.21.15.27:0/3037158229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e6f2f0 0x56321125d1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7f9400640 1 -- 172.21.15.27:0/3037158229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632112bb350 con 0x563210e774d0 2024-09-21T00:29:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7f9400640 1 --2- 172.21.15.27:0/3037158229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321125d7d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8008010 tx=0x7fb7e8015660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef49f0de01bee84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7db400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:29:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7db400640 1 -- 172.21.15.27:0/3037158229 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8025e90 con 0x563210e774d0 2024-09-21T00:29:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb7db400640 1 -- 172.21.15.27:0/3037158229 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7fb7e8025380 con 0x563210e774d0 2024-09-21T00:29:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.382+0000 7fb807aa3740 1 -- 172.21.15.27:0/3037158229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7e80259d0 con 0x563210e774d0 2024-09-21T00:29:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.388+0000 7fb7db400640 1 -- 172.21.15.27:0/3037158229 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb7e8037050 con 0x563210e774d0 2024-09-21T00:29:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.388+0000 7fb807aa3740 10 client.16266 fetch_fsmap learned FSMap version 37 2024-09-21T00:29:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.388+0000 7fb807aa3740 1 -- 172.21.15.27:0/3037158229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5632112c4980 con 0x563210e774d0 2024-09-21T00:29:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.388+0000 7fb7db400640 1 -- 172.21.15.27:0/3037158229 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e80259d0 con 0x563210e774d0 2024-09-21T00:29:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.388+0000 7fb7db400640 1 -- 172.21.15.27:0/3037158229 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fb7e8026540 con 0x563210e774d0 2024-09-21T00:29:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.388+0000 7fb807aa3740 10 client.16266 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T00:29:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.388+0000 7fb807aa3740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:29:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.388+0000 7fb807aa3740 20 client.16266 populate_metadata read hostname 'smithi027' 2024-09-21T00:29:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.388+0000 7fb807aa3740 1 --2- 172.21.15.27:0/3037158229 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563210e2c1c0 0x5632112c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.388+0000 7fb807aa3740 4 client.16266 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:29:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.388+0000 7fb807aa3740 1 -- 172.21.15.27:0/3037158229 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5632111ad4e0 con 0x563210e2c1c0 2024-09-21T00:29:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.388+0000 7fb7f9e00640 1 --2- 172.21.15.27:0/3037158229 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563210e2c1c0 0x5632112c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.388+0000 7fb7f9e00640 1 --2- 172.21.15.27:0/3037158229 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563210e2c1c0 0x5632112c70b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ba4cc4bb23bd6f server_cookie=5866dc0e305868f8 in_seq=0 out_seq=0 2024-09-21T00:29:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.388+0000 7fb7db400640 10 client.16266 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.405+0000 7fb7db400640 1 -- 172.21.15.27:0/3037158229 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5632111ad4e0 con 0x563210e2c1c0 2024-09-21T00:29:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.405+0000 7fb7db400640 10 client.16266 handle_command_reply: tid=0 2024-09-21T00:29:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.405+0000 7fb807aa3740 1 client.16266 shutdown 2024-09-21T00:29:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.405+0000 7fb807aa3740 1 -- 172.21.15.27:0/3037158229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 msgr2=0x56321125d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.405+0000 7fb807aa3740 1 --2- 172.21.15.27:0/3037158229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321125d7d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8008010 tx=0x7fb7e8015660 comp rx=0 tx=0).stop 2024-09-21T00:29:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.405+0000 7fb807aa3740 1 -- 172.21.15.27:0/3037158229 shutdown_connections 2024-09-21T00:29:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.405+0000 7fb807aa3740 1 --2- 172.21.15.27:0/3037158229 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x563210e2c1c0 0x5632112c70b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.405+0000 7fb807aa3740 1 --2- 172.21.15.27:0/3037158229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563210e981b0 0x5632112bacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.405+0000 7fb807aa3740 1 --2- 172.21.15.27:0/3037158229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563210e774d0 0x56321125d7d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.405+0000 7fb807aa3740 1 --2- 172.21.15.27:0/3037158229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563210e6f2f0 0x56321125d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.405+0000 7fb7db400640 0 client.16266 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.405+0000 7fb807aa3740 1 -- 172.21.15.27:0/3037158229 >> 172.21.15.27:0/3037158229 conn(0x563210e8c300 msgr2=0x5632112537e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.405+0000 7fb807aa3740 1 -- 172.21.15.27:0/3037158229 shutdown_connections 2024-09-21T00:29:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.410+0000 7fb807aa3740 1 -- 172.21.15.27:0/3037158229 wait complete. 2024-09-21T00:29:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.410+0000 7fb807aa3740 20 client.16266 trim_cache size 0 max 16384 2024-09-21T00:29:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:29:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:29:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:29:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:29:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.410+0000 7fb7fb200640 1 -- 172.21.15.27:0/389464021 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb7e00b6720 msgr2=0x7fb7e00b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.410+0000 7fb7fb200640 1 --2- 172.21.15.27:0/389464021 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb7e00b6720 0x7fb7e00b8c30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4006010 tx=0x7fb7f4009400 comp rx=0 tx=0).stop 2024-09-21T00:29:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.410+0000 7fb7fb200640 1 -- 172.21.15.27:0/389464021 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7fc103910 msgr2=0x7fb7fc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.410+0000 7fb7fb200640 1 --2- 172.21.15.27:0/389464021 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7fc103910 0x7fb7fc075710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00e9f0 tx=0x7fb7ec00edf0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.410+0000 7fb7fb200640 1 -- 172.21.15.27:0/389464021 shutdown_connections 2024-09-21T00:29:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.410+0000 7fb7fb200640 1 --2- 172.21.15.27:0/389464021 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb7e00b6720 0x7fb7e00b8c30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.410+0000 7fb7fb200640 1 --2- 172.21.15.27:0/389464021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7fc109b90 0x7fb7fc076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.410+0000 7fb7fb200640 1 --2- 172.21.15.27:0/389464021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7fc109240 0x7fb7fc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.410+0000 7fb7fb200640 1 --2- 172.21.15.27:0/389464021 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7fc103910 0x7fb7fc075710 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.410+0000 7fb7fb200640 1 -- 172.21.15.27:0/389464021 >> 172.21.15.27:0/389464021 conn(0x7fb7fc0ff390 msgr2=0x7fb7fc108900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.410+0000 7fb7fb200640 1 -- 172.21.15.27:0/389464021 shutdown_connections 2024-09-21T00:29:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:29:47.410+0000 7fb7fb200640 1 -- 172.21.15.27:0/389464021 wait complete. 2024-09-21T00:29:47.847 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:47 smithi027 ceph-mon[30745]: pgmap v636: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 8 op/s 2024-09-21T00:29:47.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:47 smithi089 ceph-mon[34481]: pgmap v636: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 8 op/s 2024-09-21T00:29:47.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:47 smithi098 ceph-mon[34725]: pgmap v636: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 8 op/s 2024-09-21T00:29:48.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 -- 172.21.15.27:0/3936015402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e54076100 msgr2=0x7f2e5410f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:48.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 --2- 172.21.15.27:0/3936015402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e54076100 0x7f2e5410f9d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2e40009860 tx=0x7f2e4002a340 comp rx=0 tx=0).stop 2024-09-21T00:29:48.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 -- 172.21.15.27:0/3936015402 shutdown_connections 2024-09-21T00:29:48.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 --2- 172.21.15.27:0/3936015402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e5410ff10 0x7f2e54112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 --2- 172.21.15.27:0/3936015402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e54076100 0x7f2e5410f9d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 --2- 172.21.15.27:0/3936015402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e540757b0 0x7f2e54075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 -- 172.21.15.27:0/3936015402 >> 172.21.15.27:0/3936015402 conn(0x7f2e540ff440 msgr2=0x7f2e54101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:48.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 -- 172.21.15.27:0/3936015402 shutdown_connections 2024-09-21T00:29:48.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 -- 172.21.15.27:0/3936015402 wait complete. 2024-09-21T00:29:48.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 Processor -- start 2024-09-21T00:29:48.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 -- start start 2024-09-21T00:29:48.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540757b0 0x7f2e5419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e54076100 0x7f2e5419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e5410ff10 0x7f2e541a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e54115a80 con 0x7f2e540757b0 2024-09-21T00:29:48.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2e54115900 con 0x7f2e5410ff10 2024-09-21T00:29:48.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e54115c00 con 0x7f2e54076100 2024-09-21T00:29:48.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e51e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540757b0 0x7f2e5419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e51e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540757b0 0x7f2e5419c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51264/0 (socket says 172.21.15.27:51264) 2024-09-21T00:29:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e51e00640 1 -- 172.21.15.27:0/871702300 learned_addr learned my addr 172.21.15.27:0/871702300 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e51400640 1 --2- 172.21.15.27:0/871702300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e54076100 0x7f2e5419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e51400640 1 -- 172.21.15.27:0/871702300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e5410ff10 msgr2=0x7f2e541a1dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:29:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e51400640 1 --2- 172.21.15.27:0/871702300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e5410ff10 0x7f2e541a1dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e51400640 1 -- 172.21.15.27:0/871702300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540757b0 msgr2=0x7f2e5419c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e51400640 1 --2- 172.21.15.27:0/871702300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540757b0 0x7f2e5419c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e51400640 1 -- 172.21.15.27:0/871702300 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e541a2500 con 0x7f2e54076100 2024-09-21T00:29:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e51e00640 1 --2- 172.21.15.27:0/871702300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540757b0 0x7f2e5419c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e51400640 1 --2- 172.21.15.27:0/871702300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e54076100 0x7f2e5419cb30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2e40035040 tx=0x7f2e40032dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e4aa00640 1 -- 172.21.15.27:0/871702300 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e40040020 con 0x7f2e54076100 2024-09-21T00:29:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 -- 172.21.15.27:0/871702300 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e541a2790 con 0x7f2e54076100 2024-09-21T00:29:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e4aa00640 1 -- 172.21.15.27:0/871702300 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e40030450 con 0x7f2e54076100 2024-09-21T00:29:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e4aa00640 1 -- 172.21.15.27:0/871702300 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e40037040 con 0x7f2e54076100 2024-09-21T00:29:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.385+0000 7f2e53200640 1 -- 172.21.15.27:0/871702300 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e541a8f50 con 0x7f2e54076100 2024-09-21T00:29:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 Processor -- start 2024-09-21T00:29:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 -- start start 2024-09-21T00:29:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e425b760 0x5563e4602100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e4232790 0x5563e4602640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e423a970 0x5563e4602b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5563e4579450 con 0x5563e423a970 2024-09-21T00:29:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5563e4568880 con 0x5563e425b760 2024-09-21T00:29:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5563e456c690 con 0x5563e4232790 2024-09-21T00:29:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e4aa00640 1 -- 172.21.15.27:0/871702300 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2e40043020 con 0x7f2e54076100 2024-09-21T00:29:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e52800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e425b760 0x5563e4602100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e52800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e425b760 0x5563e4602100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40774/0 (socket says 172.21.15.27:40774) 2024-09-21T00:29:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e52800640 1 -- 172.21.15.27:0/2889657413 learned_addr learned my addr 172.21.15.27:0/2889657413 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e51400640 1 --2- 172.21.15.27:0/2889657413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e423a970 0x5563e4602b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e51e00640 1 --2- 172.21.15.27:0/2889657413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e4232790 0x5563e4602640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e52800640 1 -- 172.21.15.27:0/2889657413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e4232790 msgr2=0x5563e4602640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e52800640 1 --2- 172.21.15.27:0/2889657413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e4232790 0x5563e4602640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e52800640 1 -- 172.21.15.27:0/2889657413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e423a970 msgr2=0x5563e4602b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e52800640 1 --2- 172.21.15.27:0/2889657413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e423a970 0x5563e4602b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e52800640 1 -- 172.21.15.27:0/2889657413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563e451ab00 con 0x5563e425b760 2024-09-21T00:29:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e52800640 1 --2- 172.21.15.27:0/2889657413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e425b760 0x5563e4602100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009c80 tx=0x7f2e4c00e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae9cf4a14a901b7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e3f400640 1 -- 172.21.15.27:0/2889657413 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c016070 con 0x5563e425b760 2024-09-21T00:29:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e3f400640 1 -- 172.21.15.27:0/2889657413 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e4c005020 con 0x5563e425b760 2024-09-21T00:29:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e3f400640 1 -- 172.21.15.27:0/2889657413 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0116b0 con 0x5563e425b760 2024-09-21T00:29:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e4aa00640 1 --2- 172.21.15.27:0/871702300 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2e380b6700 0x7f2e380b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e4aa00640 1 -- 172.21.15.27:0/871702300 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f2e40107650 con 0x7f2e54076100 2024-09-21T00:29:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e51e00640 1 --2- 172.21.15.27:0/871702300 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2e380b6700 0x7f2e380b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/2889657413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e425b760 msgr2=0x5563e4602100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/2889657413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e425b760 0x5563e4602100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009c80 tx=0x7f2e4c00e6c0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/2889657413 shutdown_connections 2024-09-21T00:29:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/2889657413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e423a970 0x5563e4602b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/2889657413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e4232790 0x5563e4602640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/2889657413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e425b760 0x5563e4602100 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/2889657413 >> 172.21.15.27:0/2889657413 conn(0x5563e424f560 msgr2=0x5563e45ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/2889657413 shutdown_connections 2024-09-21T00:29:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.394+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/2889657413 wait complete. 2024-09-21T00:29:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e51e00640 1 --2- 172.21.15.27:0/871702300 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2e380b6700 0x7f2e380b8c10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2e440096c0 tx=0x7f2e44009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 1 -- start start 2024-09-21T00:29:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 1 Processor -- start 2024-09-21T00:29:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e4232790 0x5563e460fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e423a970 0x5563e460d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e425b760 0x5563e460db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5563e43ca380 con 0x5563e423a970 2024-09-21T00:29:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5563e450e260 con 0x5563e425b760 2024-09-21T00:29:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5563e46108e0 con 0x5563e4232790 2024-09-21T00:29:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 10 client.0 _ll_register_callbacks cb 0x5563e45627a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e51e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e4232790 0x5563e460fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e51e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e4232790 0x5563e460fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:54720/0 (socket says 172.21.15.27:54720) 2024-09-21T00:29:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e51e00640 1 -- 172.21.15.27:0/281677085 learned_addr learned my addr 172.21.15.27:0/281677085 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e51400640 1 --2- 172.21.15.27:0/281677085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e423a970 0x5563e460d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e52800640 1 --2- 172.21.15.27:0/281677085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e425b760 0x5563e460db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e51e00640 1 -- 172.21.15.27:0/281677085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e425b760 msgr2=0x5563e460db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e51e00640 1 --2- 172.21.15.27:0/281677085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e425b760 0x5563e460db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e51e00640 1 -- 172.21.15.27:0/281677085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e423a970 msgr2=0x5563e460d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e51e00640 1 --2- 172.21.15.27:0/281677085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e423a970 0x5563e460d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e51e00640 1 -- 172.21.15.27:0/281677085 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563e4339490 con 0x5563e4232790 2024-09-21T00:29:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e52800640 1 --2- 172.21.15.27:0/281677085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e425b760 0x5563e460db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e51e00640 1 --2- 172.21.15.27:0/281677085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e4232790 0x5563e460fef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2e4400fa60 tx=0x7f2e44004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b5480c78454a409 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e33e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:29:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e33e00640 1 -- 172.21.15.27:0/281677085 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e44005020 con 0x5563e4232790 2024-09-21T00:29:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/281677085 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e4c005020 con 0x5563e4232790 2024-09-21T00:29:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e33e00640 1 -- 172.21.15.27:0/281677085 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f2e44018030 con 0x5563e4232790 2024-09-21T00:29:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e33e00640 1 -- 172.21.15.27:0/281677085 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2e440051c0 con 0x5563e4232790 2024-09-21T00:29:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 10 client.26296 fetch_fsmap learned FSMap version 37 2024-09-21T00:29:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/281677085 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5563e461c290 con 0x5563e4232790 2024-09-21T00:29:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e33e00640 1 -- 172.21.15.27:0/281677085 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e4c005020 con 0x5563e4232790 2024-09-21T00:29:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e33e00640 1 -- 172.21.15.27:0/281677085 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f2e4401e5a0 con 0x5563e4232790 2024-09-21T00:29:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 10 client.26296 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T00:29:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 10 client.26296 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T00:29:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 20 client.26296 populate_metadata read hostname 'smithi027' 2024-09-21T00:29:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/281677085 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5563e4219fd0 0x5563e461f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 4 client.26296 mds_command: new command op to 24751 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:29:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/281677085 --> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563e457b3a0 con 0x5563e4219fd0 2024-09-21T00:29:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.399+0000 7f2e51400640 1 --2- 172.21.15.27:0/281677085 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5563e4219fd0 0x5563e461f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.404+0000 7f2e51400640 1 --2- 172.21.15.27:0/281677085 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5563e4219fd0 0x5563e461f8d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9868fe1cc864b343 server_cookie=bbe1932ebd88667f in_seq=0 out_seq=0 2024-09-21T00:29:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.404+0000 7f2e33e00640 10 client.26296 ms_handle_connect on v2:172.21.15.27:6840/378707351 2024-09-21T00:29:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.404+0000 7f2e33e00640 1 -- 172.21.15.27:0/281677085 <== mds.1 v2:172.21.15.27:6840/378707351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5563e457b3a0 con 0x5563e4219fd0 2024-09-21T00:29:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.404+0000 7f2e33e00640 10 client.26296 handle_command_reply: tid=0 2024-09-21T00:29:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.404+0000 7f2e5fc9f740 1 client.26296 shutdown 2024-09-21T00:29:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.404+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/281677085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e4232790 msgr2=0x5563e460fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.404+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/281677085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e4232790 0x5563e460fef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2e4400fa60 tx=0x7f2e44004730 comp rx=0 tx=0).stop 2024-09-21T00:29:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.404+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/281677085 shutdown_connections 2024-09-21T00:29:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.404+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/281677085 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5563e4219fd0 0x5563e461f8d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.404+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/281677085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e425b760 0x5563e460db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.404+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/281677085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e423a970 0x5563e460d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.404+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/281677085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e4232790 0x5563e460fef0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.404+0000 7f2e33e00640 0 client.26296 ms_handle_reset on v2:172.21.15.27:6840/378707351 2024-09-21T00:29:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.404+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/281677085 >> 172.21.15.27:0/281677085 conn(0x5563e424f560 msgr2=0x5563e4605290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.406+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/281677085 shutdown_connections 2024-09-21T00:29:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.406+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/281677085 wait complete. 2024-09-21T00:29:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.406+0000 7f2e5fc9f740 20 client.26296 trim_cache size 0 max 16384 2024-09-21T00:29:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e5fc9f740 1 Processor -- start 2024-09-21T00:29:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e5fc9f740 1 -- start start 2024-09-21T00:29:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e5fc9f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e4232790 0x5563e45ff680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e5fc9f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e423a970 0x5563e45ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e5fc9f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e425b760 0x5563e460a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e5fc9f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e541b59c0 con 0x5563e4232790 2024-09-21T00:29:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e5fc9f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2e541b3230 con 0x5563e423a970 2024-09-21T00:29:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e5fc9f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e541b10a0 con 0x5563e425b760 2024-09-21T00:29:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e51e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e4232790 0x5563e45ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e51e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e4232790 0x5563e45ff680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51302/0 (socket says 172.21.15.27:51302) 2024-09-21T00:29:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e51e00640 1 -- 172.21.15.27:0/2265964868 learned_addr learned my addr 172.21.15.27:0/2265964868 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e52800640 1 --2- 172.21.15.27:0/2265964868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e425b760 0x5563e460a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e51400640 1 --2- 172.21.15.27:0/2265964868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e423a970 0x5563e45ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e51e00640 1 -- 172.21.15.27:0/2265964868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e425b760 msgr2=0x5563e460a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:48.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e51e00640 1 --2- 172.21.15.27:0/2265964868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e425b760 0x5563e460a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e51e00640 1 -- 172.21.15.27:0/2265964868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e423a970 msgr2=0x5563e45ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e51e00640 1 --2- 172.21.15.27:0/2265964868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e423a970 0x5563e45ffbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e51e00640 1 -- 172.21.15.27:0/2265964868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563e4619f40 con 0x5563e4232790 2024-09-21T00:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e52800640 1 --2- 172.21.15.27:0/2265964868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e425b760 0x5563e460a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e51e00640 1 --2- 172.21.15.27:0/2265964868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e4232790 0x5563e45ff680 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2e44000c00 tx=0x7f2e4401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec1eefe015e73fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e3ea00640 1 -- 172.21.15.27:0/2265964868 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4401e5a0 con 0x5563e4232790 2024-09-21T00:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e3ea00640 1 -- 172.21.15.27:0/2265964868 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e4401e740 con 0x5563e4232790 2024-09-21T00:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.420+0000 7f2e3ea00640 1 -- 172.21.15.27:0/2265964868 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e440036e0 con 0x5563e4232790 2024-09-21T00:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/2265964868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e4232790 msgr2=0x5563e45ff680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/2265964868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e4232790 0x5563e45ff680 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2e44000c00 tx=0x7f2e4401ec50 comp rx=0 tx=0).stop 2024-09-21T00:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/2265964868 shutdown_connections 2024-09-21T00:29:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/2265964868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e425b760 0x5563e460a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/2265964868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e423a970 0x5563e45ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/2265964868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e4232790 0x5563e45ff680 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/2265964868 >> 172.21.15.27:0/2265964868 conn(0x5563e424f560 msgr2=0x5563e460bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/2265964868 shutdown_connections 2024-09-21T00:29:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/2265964868 wait complete. 2024-09-21T00:29:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 -- start start 2024-09-21T00:29:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 Processor -- start 2024-09-21T00:29:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e4232790 0x5563e46160b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e423a970 0x5563e4612570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e425b760 0x5563e4612ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e541b16f0 con 0x5563e425b760 2024-09-21T00:29:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2e541b2b60 con 0x5563e4232790 2024-09-21T00:29:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e541b1dc0 con 0x5563e423a970 2024-09-21T00:29:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 10 client.0 _ll_register_callbacks cb 0x7f2e44009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e51400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e423a970 0x5563e4612570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e51400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e423a970 0x5563e4612570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:54746/0 (socket says 172.21.15.27:54746) 2024-09-21T00:29:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e51400640 1 -- 172.21.15.27:0/4082258991 learned_addr learned my addr 172.21.15.27:0/4082258991 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e52800640 1 --2- 172.21.15.27:0/4082258991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e425b760 0x5563e4612ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e51e00640 1 --2- 172.21.15.27:0/4082258991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e4232790 0x5563e46160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e51400640 1 -- 172.21.15.27:0/4082258991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e4232790 msgr2=0x5563e46160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e51400640 1 --2- 172.21.15.27:0/4082258991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e4232790 0x5563e46160b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e51400640 1 -- 172.21.15.27:0/4082258991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e425b760 msgr2=0x5563e4612ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e51400640 1 --2- 172.21.15.27:0/4082258991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e425b760 0x5563e4612ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e51400640 1 -- 172.21.15.27:0/4082258991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563e4612ff0 con 0x5563e423a970 2024-09-21T00:29:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e51400640 1 --2- 172.21.15.27:0/4082258991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e423a970 0x5563e4612570 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e4002a860 tx=0x7f2e40005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=758414558e976200 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e33e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:29:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e33e00640 1 -- 172.21.15.27:0/4082258991 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e40104df0 con 0x5563e423a970 2024-09-21T00:29:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e33e00640 1 -- 172.21.15.27:0/4082258991 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f2e401071e0 con 0x5563e423a970 2024-09-21T00:29:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/4082258991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e4401e740 con 0x5563e423a970 2024-09-21T00:29:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e33e00640 1 -- 172.21.15.27:0/4082258991 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2e400ccb80 con 0x5563e423a970 2024-09-21T00:29:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 10 client.26302 fetch_fsmap learned FSMap version 37 2024-09-21T00:29:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/4082258991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5563e468afc0 con 0x5563e423a970 2024-09-21T00:29:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e33e00640 1 -- 172.21.15.27:0/4082258991 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e4401e740 con 0x5563e423a970 2024-09-21T00:29:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e33e00640 1 -- 172.21.15.27:0/4082258991 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f2e4004d060 con 0x5563e423a970 2024-09-21T00:29:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 10 client.26302 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T00:29:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 10 client.26302 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T00:29:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 20 client.26302 populate_metadata read hostname 'smithi027' 2024-09-21T00:29:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/4082258991 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5563e41ef380 0x5563e468d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 4 client.26302 mds_command: new command op to 24751 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-47099ce0-1-mds.a.json"}] 2024-09-21T00:29:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/4082258991 --> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-47099ce0-1-mds.a.json"}) -- 0x5563e4576e50 con 0x5563e41ef380 2024-09-21T00:29:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.426+0000 7f2e51e00640 1 --2- 172.21.15.27:0/4082258991 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5563e41ef380 0x5563e468d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.431+0000 7f2e51e00640 1 --2- 172.21.15.27:0/4082258991 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5563e41ef380 0x5563e468d850 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8862c718f871afab server_cookie=7887d2d4d827b473 in_seq=0 out_seq=0 2024-09-21T00:29:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:48.431+0000 7f2e33e00640 10 client.26302 ms_handle_connect on v2:172.21.15.27:6840/378707351 2024-09-21T00:29:49.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.262+0000 7f2e33e00640 1 -- 172.21.15.27:0/4082258991 <== mds.1 v2:172.21.15.27:6840/378707351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x5563e4576e50 con 0x5563e41ef380 2024-09-21T00:29:49.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.262+0000 7f2e33e00640 10 client.26302 handle_command_reply: tid=0 2024-09-21T00:29:49.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.262+0000 7f2e5fc9f740 1 client.26302 shutdown 2024-09-21T00:29:49.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.262+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/4082258991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e423a970 msgr2=0x5563e4612570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.262+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/4082258991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e423a970 0x5563e4612570 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e4002a860 tx=0x7f2e40005c90 comp rx=0 tx=0).stop 2024-09-21T00:29:49.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.262+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/4082258991 shutdown_connections 2024-09-21T00:29:49.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.262+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/4082258991 >> [v2:172.21.15.27:6840/378707351,v1:172.21.15.27:6841/378707351] conn(0x5563e41ef380 0x5563e468d850 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.262+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/4082258991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5563e425b760 0x5563e4612ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.262+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/4082258991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563e423a970 0x5563e4612570 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.262+0000 7f2e5fc9f740 1 --2- 172.21.15.27:0/4082258991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5563e4232790 0x5563e46160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.262+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/4082258991 >> 172.21.15.27:0/4082258991 conn(0x5563e424f560 msgr2=0x5563e460c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:49.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.262+0000 7f2e33e00640 0 client.26302 ms_handle_reset on v2:172.21.15.27:6840/378707351 2024-09-21T00:29:49.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.263+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/4082258991 shutdown_connections 2024-09-21T00:29:49.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.263+0000 7f2e5fc9f740 1 -- 172.21.15.27:0/4082258991 wait complete. 2024-09-21T00:29:49.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.263+0000 7f2e5fc9f740 20 client.26302 trim_cache size 0 max 16384 2024-09-21T00:29:49.266 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:29:49.266 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/var/run/ceph/ops-47099ce0-1-mds.a.json" 2024-09-21T00:29:49.266 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:29:49.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.263+0000 7f2e53200640 1 -- 172.21.15.27:0/871702300 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2e380b6700 msgr2=0x7f2e380b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.263+0000 7f2e53200640 1 --2- 172.21.15.27:0/871702300 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2e380b6700 0x7f2e380b8c10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2e440096c0 tx=0x7f2e44009260 comp rx=0 tx=0).stop 2024-09-21T00:29:49.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.263+0000 7f2e53200640 1 -- 172.21.15.27:0/871702300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e54076100 msgr2=0x7f2e5419cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.263+0000 7f2e53200640 1 --2- 172.21.15.27:0/871702300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e54076100 0x7f2e5419cb30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2e40035040 tx=0x7f2e40032dc0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.263+0000 7f2e53200640 1 -- 172.21.15.27:0/871702300 shutdown_connections 2024-09-21T00:29:49.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.263+0000 7f2e53200640 1 --2- 172.21.15.27:0/871702300 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2e380b6700 0x7f2e380b8c10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.263+0000 7f2e53200640 1 --2- 172.21.15.27:0/871702300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e5410ff10 0x7f2e541a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.263+0000 7f2e53200640 1 --2- 172.21.15.27:0/871702300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e54076100 0x7f2e5419cb30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.263+0000 7f2e53200640 1 --2- 172.21.15.27:0/871702300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540757b0 0x7f2e5419c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.263+0000 7f2e53200640 1 -- 172.21.15.27:0/871702300 >> 172.21.15.27:0/871702300 conn(0x7f2e540ff440 msgr2=0x7f2e541089e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:49.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.263+0000 7f2e53200640 1 -- 172.21.15.27:0/871702300 shutdown_connections 2024-09-21T00:29:49.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:49.263+0000 7f2e53200640 1 -- 172.21.15.27:0/871702300 wait complete. 2024-09-21T00:29:49.779 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:49 smithi027 ceph-mon[30745]: pgmap v637: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:29:49.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:49 smithi089 ceph-mon[34481]: pgmap v637: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:29:49.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:49 smithi098 ceph-mon[34725]: pgmap v637: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:29:51.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:51 smithi027 ceph-mon[30745]: pgmap v638: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-21T00:29:51.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:51 smithi089 ceph-mon[34481]: pgmap v638: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-21T00:29:51.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:51 smithi098 ceph-mon[34725]: pgmap v638: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-21T00:29:52.506 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 0 (b) to a remote file /var/run/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ops-47099ce0-1-mds.b.json 2024-09-21T00:29:52.507 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 ops --flags=locks --path=/var/run/ceph/ops-47099ce0-1-mds.b.json 2024-09-21T00:29:52.952 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:29:53.796 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:53 smithi027 ceph-mon[30745]: pgmap v639: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-09-21T00:29:53.797 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:53 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:29:53.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:53 smithi089 ceph-mon[34481]: pgmap v639: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-09-21T00:29:53.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:53 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:29:53.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:53 smithi098 ceph-mon[34725]: pgmap v639: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-09-21T00:29:53.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:53 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:29:55.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.814+0000 7f968f200640 1 -- 172.21.15.27:0/1399748379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f969010ff90 msgr2=0x7f96901123b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:55.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.814+0000 7f968f200640 1 --2- 172.21.15.27:0/1399748379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f969010ff90 0x7f96901123b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9688009ca0 tx=0x7f968802a950 comp rx=0 tx=0).stop 2024-09-21T00:29:55.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.814+0000 7f968f200640 1 -- 172.21.15.27:0/1399748379 shutdown_connections 2024-09-21T00:29:55.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.814+0000 7f968f200640 1 --2- 172.21.15.27:0/1399748379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f969010ff90 0x7f96901123b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.814+0000 7f968f200640 1 --2- 172.21.15.27:0/1399748379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9690102d30 0x7f969010fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.814+0000 7f968f200640 1 --2- 172.21.15.27:0/1399748379 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96901023e0 0x7f96901027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.814+0000 7f968f200640 1 -- 172.21.15.27:0/1399748379 >> 172.21.15.27:0/1399748379 conn(0x7f96900fe2c0 msgr2=0x7f9690100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:55.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.814+0000 7f968f200640 1 -- 172.21.15.27:0/1399748379 shutdown_connections 2024-09-21T00:29:55.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.814+0000 7f968f200640 1 -- 172.21.15.27:0/1399748379 wait complete. 2024-09-21T00:29:55.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968f200640 1 Processor -- start 2024-09-21T00:29:55.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968f200640 1 -- start start 2024-09-21T00:29:55.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968f200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96901023e0 0x7f969019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9690102d30 0x7f969019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f969010ff90 0x7f96901a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9690115b10 con 0x7f9690102d30 2024-09-21T00:29:55.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968f200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9690115990 con 0x7f969010ff90 2024-09-21T00:29:55.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968f200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9690115c90 con 0x7f96901023e0 2024-09-21T00:29:55.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f969010ff90 0x7f96901a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96901023e0 0x7f969019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f969010ff90 0x7f96901a1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:46150/0 (socket says 172.21.15.27:46150) 2024-09-21T00:29:55.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968e800640 1 -- 172.21.15.27:0/3424403074 learned_addr learned my addr 172.21.15.27:0/3424403074 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:55.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968d400640 1 --2- 172.21.15.27:0/3424403074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9690102d30 0x7f969019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968e800640 1 -- 172.21.15.27:0/3424403074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96901023e0 msgr2=0x7f969019c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:55.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968e800640 1 --2- 172.21.15.27:0/3424403074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96901023e0 0x7f969019c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968e800640 1 -- 172.21.15.27:0/3424403074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9690102d30 msgr2=0x7f969019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:55.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968e800640 1 --2- 172.21.15.27:0/3424403074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9690102d30 0x7f969019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.819+0000 7f968e800640 1 -- 172.21.15.27:0/3424403074 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96901a2430 con 0x7f969010ff90 2024-09-21T00:29:55.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.821+0000 7f968de00640 1 --2- 172.21.15.27:0/3424403074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96901023e0 0x7f969019c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:55.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.821+0000 7f968e800640 1 --2- 172.21.15.27:0/3424403074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f969010ff90 0x7f96901a1d00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9688009dc0 tx=0x7f9688003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:55.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.821+0000 7f9686a00640 1 -- 172.21.15.27:0/3424403074 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688034040 con 0x7f969010ff90 2024-09-21T00:29:55.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.821+0000 7f968f200640 1 -- 172.21.15.27:0/3424403074 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96901a26c0 con 0x7f969010ff90 2024-09-21T00:29:55.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.821+0000 7f968f200640 1 -- 172.21.15.27:0/3424403074 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96901a8e80 con 0x7f969010ff90 2024-09-21T00:29:55.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.821+0000 7f9686a00640 1 -- 172.21.15.27:0/3424403074 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9688030850 con 0x7f969010ff90 2024-09-21T00:29:55.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.821+0000 7f9686a00640 1 -- 172.21.15.27:0/3424403074 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688004600 con 0x7f969010ff90 2024-09-21T00:29:55.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.821+0000 7f969bc38740 1 Processor -- start 2024-09-21T00:29:55.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.821+0000 7f9686a00640 1 -- 172.21.15.27:0/3424403074 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9688037020 con 0x7f969010ff90 2024-09-21T00:29:55.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.821+0000 7f969bc38740 1 -- start start 2024-09-21T00:29:55.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.821+0000 7f969bc38740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336069760 0x562336410100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.821+0000 7f969bc38740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336040790 0x562336410640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.821+0000 7f969bc38740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336048970 0x562336410b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562336387450 con 0x562336048970 2024-09-21T00:29:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562336376880 con 0x562336040790 2024-09-21T00:29:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56233637a690 con 0x562336069760 2024-09-21T00:29:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f968d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336048970 0x562336410b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f968d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336048970 0x562336410b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48358/0 (socket says 172.21.15.27:48358) 2024-09-21T00:29:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f968d400640 1 -- 172.21.15.27:0/2087242248 learned_addr learned my addr 172.21.15.27:0/2087242248 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f968d400640 1 --2- 172.21.15.27:0/2087242248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336069760 0x562336410100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f968d400640 1 -- 172.21.15.27:0/2087242248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336069760 msgr2=0x562336410100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f968d400640 1 --2- 172.21.15.27:0/2087242248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336069760 0x562336410100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f968d400640 1 -- 172.21.15.27:0/2087242248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336040790 msgr2=0x562336410640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:29:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f968d400640 1 --2- 172.21.15.27:0/2087242248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336040790 0x562336410640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f968d400640 1 -- 172.21.15.27:0/2087242248 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562336328b00 con 0x562336048970 2024-09-21T00:29:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f968d400640 1 --2- 172.21.15.27:0/2087242248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336048970 0x562336410b80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f967c006aa0 tx=0x7f967c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbce5b76e2914b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f967b400640 1 -- 172.21.15.27:0/2087242248 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c015760 con 0x562336048970 2024-09-21T00:29:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f967b400640 1 -- 172.21.15.27:0/2087242248 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f967c01d070 con 0x562336048970 2024-09-21T00:29:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f967b400640 1 -- 172.21.15.27:0/2087242248 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c018a30 con 0x562336048970 2024-09-21T00:29:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f9686a00640 1 --2- 172.21.15.27:0/3424403074 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96740b6680 0x7f96740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f9686a00640 1 -- 172.21.15.27:0/3424403074 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f9688108840 con 0x7f969010ff90 2024-09-21T00:29:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f968de00640 1 --2- 172.21.15.27:0/3424403074 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96740b6680 0x7f96740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 -- 172.21.15.27:0/2087242248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336048970 msgr2=0x562336410b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 --2- 172.21.15.27:0/2087242248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336048970 0x562336410b80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f967c006aa0 tx=0x7f967c009510 comp rx=0 tx=0).stop 2024-09-21T00:29:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 -- 172.21.15.27:0/2087242248 shutdown_connections 2024-09-21T00:29:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 --2- 172.21.15.27:0/2087242248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336048970 0x562336410b80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 --2- 172.21.15.27:0/2087242248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336040790 0x562336410640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 --2- 172.21.15.27:0/2087242248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336069760 0x562336410100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 -- 172.21.15.27:0/2087242248 >> 172.21.15.27:0/2087242248 conn(0x56233605d560 msgr2=0x56233640dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 -- 172.21.15.27:0/2087242248 shutdown_connections 2024-09-21T00:29:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f968de00640 1 --2- 172.21.15.27:0/3424403074 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96740b6680 0x7f96740b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x56233640ed80 tx=0x7f9680005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 -- 172.21.15.27:0/2087242248 wait complete. 2024-09-21T00:29:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 -- start start 2024-09-21T00:29:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 Processor -- start 2024-09-21T00:29:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336040790 0x56233641b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336048970 0x56233641b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336069760 0x56233641bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5623361d8380 con 0x562336048970 2024-09-21T00:29:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56233631c260 con 0x562336069760 2024-09-21T00:29:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56233641e880 con 0x562336040790 2024-09-21T00:29:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f969bc38740 10 client.0 _ll_register_callbacks cb 0x5623363707a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.826+0000 7f968d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336048970 0x56233641b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.831+0000 7f968de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336040790 0x56233641b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.831+0000 7f968d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336048970 0x56233641b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48374/0 (socket says 172.21.15.27:48374) 2024-09-21T00:29:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.831+0000 7f968d400640 1 -- 172.21.15.27:0/3743219116 learned_addr learned my addr 172.21.15.27:0/3743219116 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.831+0000 7f968e800640 1 --2- 172.21.15.27:0/3743219116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336069760 0x56233641bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.831+0000 7f968d400640 1 -- 172.21.15.27:0/3743219116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336040790 msgr2=0x56233641b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.831+0000 7f968d400640 1 --2- 172.21.15.27:0/3743219116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336040790 0x56233641b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.831+0000 7f968d400640 1 -- 172.21.15.27:0/3743219116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336069760 msgr2=0x56233641bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:55.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.831+0000 7f968d400640 1 --2- 172.21.15.27:0/3743219116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336069760 0x56233641bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.831+0000 7f968d400640 1 -- 172.21.15.27:0/3743219116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562336147490 con 0x562336048970 2024-09-21T00:29:55.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.831+0000 7f968d400640 1 --2- 172.21.15.27:0/3743219116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336048970 0x56233641b9c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f967c018610 tx=0x7f967c019a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca7cfe9a7e736b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:55.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.831+0000 7f966fe00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:29:55.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.831+0000 7f966fe00640 1 -- 172.21.15.27:0/3743219116 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c015760 con 0x562336048970 2024-09-21T00:29:55.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.831+0000 7f966fe00640 1 -- 172.21.15.27:0/3743219116 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f967c015900 con 0x562336048970 2024-09-21T00:29:55.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.831+0000 7f969bc38740 1 -- 172.21.15.27:0/3743219116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f967c01d070 con 0x562336048970 2024-09-21T00:29:55.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f966fe00640 1 -- 172.21.15.27:0/3743219116 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f967c032050 con 0x562336048970 2024-09-21T00:29:55.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 10 client.16296 fetch_fsmap learned FSMap version 38 2024-09-21T00:29:55.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 1 -- 172.21.15.27:0/3743219116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56233642a290 con 0x562336048970 2024-09-21T00:29:55.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f966fe00640 1 -- 172.21.15.27:0/3743219116 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f967c01d070 con 0x562336048970 2024-09-21T00:29:55.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f966fe00640 1 -- 172.21.15.27:0/3743219116 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f967c029670 con 0x562336048970 2024-09-21T00:29:55.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 10 client.16296 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:29:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T00:29:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 20 client.16296 populate_metadata read hostname 'smithi027' 2024-09-21T00:29:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 1 --2- 172.21.15.27:0/3743219116 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562336027fd0 0x56233642d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 4 client.16296 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:29:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 1 -- 172.21.15.27:0/3743219116 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623363893a0 con 0x562336027fd0 2024-09-21T00:29:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f968de00640 1 --2- 172.21.15.27:0/3743219116 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562336027fd0 0x56233642d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f968de00640 1 --2- 172.21.15.27:0/3743219116 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562336027fd0 0x56233642d700 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cf2cf905e68b9ae server_cookie=4da31f756a418eb3 in_seq=0 out_seq=0 2024-09-21T00:29:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f966fe00640 10 client.16296 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f966fe00640 1 -- 172.21.15.27:0/3743219116 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5623363893a0 con 0x562336027fd0 2024-09-21T00:29:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f966fe00640 10 client.16296 handle_command_reply: tid=0 2024-09-21T00:29:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 1 client.16296 shutdown 2024-09-21T00:29:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 1 -- 172.21.15.27:0/3743219116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336048970 msgr2=0x56233641b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 1 --2- 172.21.15.27:0/3743219116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336048970 0x56233641b9c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f967c018610 tx=0x7f967c019a50 comp rx=0 tx=0).stop 2024-09-21T00:29:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 1 -- 172.21.15.27:0/3743219116 shutdown_connections 2024-09-21T00:29:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 1 --2- 172.21.15.27:0/3743219116 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562336027fd0 0x56233642d700 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 1 --2- 172.21.15.27:0/3743219116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336069760 0x56233641bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 1 --2- 172.21.15.27:0/3743219116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336048970 0x56233641b9c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 1 --2- 172.21.15.27:0/3743219116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336040790 0x56233641b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f966fe00640 0 client.16296 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 1 -- 172.21.15.27:0/3743219116 >> 172.21.15.27:0/3743219116 conn(0x56233605d560 msgr2=0x562336413290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 1 -- 172.21.15.27:0/3743219116 shutdown_connections 2024-09-21T00:29:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 1 -- 172.21.15.27:0/3743219116 wait complete. 2024-09-21T00:29:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.833+0000 7f969bc38740 20 client.16296 trim_cache size 0 max 16384 2024-09-21T00:29:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f969bc38740 1 Processor -- start 2024-09-21T00:29:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f969bc38740 1 -- start start 2024-09-21T00:29:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f969bc38740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336040790 0x562336411900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f969bc38740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336048970 0x56233641b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f969bc38740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336069760 0x56233641bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f969bc38740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f96901b5860 con 0x562336069760 2024-09-21T00:29:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f969bc38740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f96901b30d0 con 0x562336048970 2024-09-21T00:29:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f969bc38740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f96901b0f40 con 0x562336040790 2024-09-21T00:29:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f968d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336048970 0x56233641b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f968d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336048970 0x56233641b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:46174/0 (socket says 172.21.15.27:46174) 2024-09-21T00:29:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f968d400640 1 -- 172.21.15.27:0/2366746676 learned_addr learned my addr 172.21.15.27:0/2366746676 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f968e800640 1 --2- 172.21.15.27:0/2366746676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336069760 0x56233641bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f968de00640 1 --2- 172.21.15.27:0/2366746676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336040790 0x562336411900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f968d400640 1 -- 172.21.15.27:0/2366746676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336040790 msgr2=0x562336411900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f968d400640 1 --2- 172.21.15.27:0/2366746676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336040790 0x562336411900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f968d400640 1 -- 172.21.15.27:0/2366746676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336069760 msgr2=0x56233641bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f968d400640 1 --2- 172.21.15.27:0/2366746676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336069760 0x56233641bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f968d400640 1 -- 172.21.15.27:0/2366746676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562336412d40 con 0x562336048970 2024-09-21T00:29:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.850+0000 7f968e800640 1 --2- 172.21.15.27:0/2366746676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336069760 0x56233641bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.855+0000 7f968d400640 1 --2- 172.21.15.27:0/2366746676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336048970 0x56233641b980 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f967c018640 tx=0x7f967c009d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c080f4fda2c142f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.855+0000 7f967aa00640 1 -- 172.21.15.27:0/2366746676 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c03b070 con 0x562336048970 2024-09-21T00:29:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.855+0000 7f967aa00640 1 -- 172.21.15.27:0/2366746676 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f967c01d070 con 0x562336048970 2024-09-21T00:29:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.855+0000 7f967aa00640 1 -- 172.21.15.27:0/2366746676 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c019040 con 0x562336048970 2024-09-21T00:29:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 -- 172.21.15.27:0/2366746676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336048970 msgr2=0x56233641b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 --2- 172.21.15.27:0/2366746676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336048970 0x56233641b980 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f967c018640 tx=0x7f967c009d80 comp rx=0 tx=0).stop 2024-09-21T00:29:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 -- 172.21.15.27:0/2366746676 shutdown_connections 2024-09-21T00:29:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 --2- 172.21.15.27:0/2366746676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336069760 0x56233641bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 --2- 172.21.15.27:0/2366746676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336048970 0x56233641b980 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 --2- 172.21.15.27:0/2366746676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336040790 0x562336411900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 -- 172.21.15.27:0/2366746676 >> 172.21.15.27:0/2366746676 conn(0x56233605d560 msgr2=0x562336426b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 -- 172.21.15.27:0/2366746676 shutdown_connections 2024-09-21T00:29:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 -- 172.21.15.27:0/2366746676 wait complete. 2024-09-21T00:29:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 -- start start 2024-09-21T00:29:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 Processor -- start 2024-09-21T00:29:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336040790 0x562336491150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336048970 0x562336491710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336069760 0x56233642ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f96901b1590 con 0x562336069760 2024-09-21T00:29:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f96901b2a00 con 0x562336040790 2024-09-21T00:29:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f96901b1c60 con 0x562336048970 2024-09-21T00:29:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 10 client.0 _ll_register_callbacks cb 0x7f967c009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f968de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336040790 0x562336491150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f968de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336040790 0x562336491150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:46188/0 (socket says 172.21.15.27:46188) 2024-09-21T00:29:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f968de00640 1 -- 172.21.15.27:0/824132638 learned_addr learned my addr 172.21.15.27:0/824132638 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:29:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f968e800640 1 --2- 172.21.15.27:0/824132638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336069760 0x56233642ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f968d400640 1 --2- 172.21.15.27:0/824132638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336048970 0x562336491710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f968de00640 1 -- 172.21.15.27:0/824132638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336048970 msgr2=0x562336491710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f968de00640 1 --2- 172.21.15.27:0/824132638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336048970 0x562336491710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f968de00640 1 -- 172.21.15.27:0/824132638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336069760 msgr2=0x56233642ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f968de00640 1 --2- 172.21.15.27:0/824132638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336069760 0x56233642ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f968de00640 1 -- 172.21.15.27:0/824132638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56233642f590 con 0x562336040790 2024-09-21T00:29:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f968e800640 1 --2- 172.21.15.27:0/824132638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336069760 0x56233642ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f968de00640 1 --2- 172.21.15.27:0/824132638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336040790 0x562336491150 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5623361d8410 tx=0x7f9680004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c711d702ac31cae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f966fe00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:29:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f966fe00640 1 -- 172.21.15.27:0/824132638 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968001c790 con 0x562336040790 2024-09-21T00:29:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 -- 172.21.15.27:0/824132638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f967c01d070 con 0x562336040790 2024-09-21T00:29:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f966fe00640 1 -- 172.21.15.27:0/824132638 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f968001c930 con 0x562336040790 2024-09-21T00:29:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f966fe00640 1 -- 172.21.15.27:0/824132638 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f968003b050 con 0x562336040790 2024-09-21T00:29:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 10 client.26312 fetch_fsmap learned FSMap version 38 2024-09-21T00:29:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 -- 172.21.15.27:0/824132638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5623364988e0 con 0x562336040790 2024-09-21T00:29:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f966fe00640 1 -- 172.21.15.27:0/824132638 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f967c01d070 con 0x562336040790 2024-09-21T00:29:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f966fe00640 1 -- 172.21.15.27:0/824132638 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f9680030c80 con 0x562336040790 2024-09-21T00:29:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 10 client.26312 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:29:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 10 client.26312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T00:29:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 20 client.26312 populate_metadata read hostname 'smithi027' 2024-09-21T00:29:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 --2- 172.21.15.27:0/824132638 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562335ffd380 0x56233649b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 4 client.26312 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-47099ce0-1-mds.b.json"}] 2024-09-21T00:29:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f969bc38740 1 -- 172.21.15.27:0/824132638 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-47099ce0-1-mds.b.json"}) -- 0x562336384e50 con 0x562335ffd380 2024-09-21T00:29:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f968d400640 1 --2- 172.21.15.27:0/824132638 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562335ffd380 0x56233649b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f968d400640 1 --2- 172.21.15.27:0/824132638 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562335ffd380 0x56233649b170 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fceec5cdcc416d21 server_cookie=b18035edeaab240f in_seq=0 out_seq=0 2024-09-21T00:29:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:54.856+0000 7f966fe00640 10 client.26312 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:55.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f966fe00640 1 -- 172.21.15.27:0/824132638 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x562336384e50 con 0x562335ffd380 2024-09-21T00:29:55.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f966fe00640 10 client.26312 handle_command_reply: tid=0 2024-09-21T00:29:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f969bc38740 1 client.26312 shutdown 2024-09-21T00:29:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f969bc38740 1 -- 172.21.15.27:0/824132638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336040790 msgr2=0x562336491150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f969bc38740 1 --2- 172.21.15.27:0/824132638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336040790 0x562336491150 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5623361d8410 tx=0x7f9680004730 comp rx=0 tx=0).stop 2024-09-21T00:29:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f969bc38740 1 -- 172.21.15.27:0/824132638 shutdown_connections 2024-09-21T00:29:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f969bc38740 1 --2- 172.21.15.27:0/824132638 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x562335ffd380 0x56233649b170 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f969bc38740 1 --2- 172.21.15.27:0/824132638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336069760 0x56233642ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f969bc38740 1 --2- 172.21.15.27:0/824132638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562336048970 0x562336491710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f969bc38740 1 --2- 172.21.15.27:0/824132638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562336040790 0x562336491150 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f969bc38740 1 -- 172.21.15.27:0/824132638 >> 172.21.15.27:0/824132638 conn(0x56233605d560 msgr2=0x5623364274c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:55.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f966fe00640 0 client.26312 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:29:55.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f969bc38740 1 -- 172.21.15.27:0/824132638 shutdown_connections 2024-09-21T00:29:55.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f969bc38740 1 -- 172.21.15.27:0/824132638 wait complete. 2024-09-21T00:29:55.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f969bc38740 20 client.26312 trim_cache size 0 max 16384 2024-09-21T00:29:55.484 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:29:55.484 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/var/run/ceph/ops-47099ce0-1-mds.b.json" 2024-09-21T00:29:55.484 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:29:55.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f968f200640 1 -- 172.21.15.27:0/3424403074 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96740b6680 msgr2=0x7f96740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f968f200640 1 --2- 172.21.15.27:0/3424403074 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96740b6680 0x7f96740b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x56233640ed80 tx=0x7f9680005f90 comp rx=0 tx=0).stop 2024-09-21T00:29:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f968f200640 1 -- 172.21.15.27:0/3424403074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f969010ff90 msgr2=0x7f96901a1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.477+0000 7f968f200640 1 --2- 172.21.15.27:0/3424403074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f969010ff90 0x7f96901a1d00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9688009dc0 tx=0x7f9688003f10 comp rx=0 tx=0).stop 2024-09-21T00:29:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.479+0000 7f968f200640 1 -- 172.21.15.27:0/3424403074 shutdown_connections 2024-09-21T00:29:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.479+0000 7f968f200640 1 --2- 172.21.15.27:0/3424403074 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f96740b6680 0x7f96740b8b90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.479+0000 7f968f200640 1 --2- 172.21.15.27:0/3424403074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f969010ff90 0x7f96901a1d00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.479+0000 7f968f200640 1 --2- 172.21.15.27:0/3424403074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9690102d30 0x7f969019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.479+0000 7f968f200640 1 --2- 172.21.15.27:0/3424403074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96901023e0 0x7f969019c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.479+0000 7f968f200640 1 -- 172.21.15.27:0/3424403074 >> 172.21.15.27:0/3424403074 conn(0x7f96900fe2c0 msgr2=0x7f9690109aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.479+0000 7f968f200640 1 -- 172.21.15.27:0/3424403074 shutdown_connections 2024-09-21T00:29:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:29:55.479+0000 7f968f200640 1 -- 172.21.15.27:0/3424403074 wait complete. 2024-09-21T00:29:55.748 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:55 smithi027 ceph-mon[30745]: pgmap v640: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 6 op/s 2024-09-21T00:29:55.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:55 smithi089 ceph-mon[34481]: pgmap v640: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 6 op/s 2024-09-21T00:29:55.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:55 smithi098 ceph-mon[34725]: pgmap v640: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 6 op/s 2024-09-21T00:29:57.718 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:57 smithi027 ceph-mon[30745]: pgmap v641: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-09-21T00:29:57.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:57 smithi089 ceph-mon[34481]: pgmap v641: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-09-21T00:29:57.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:57 smithi098 ceph-mon[34725]: pgmap v641: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-09-21T00:29:58.107 DEBUG:teuthology.orchestra.run.smithi098:> mktemp 2024-09-21T00:29:58.127 INFO:teuthology.orchestra.run.smithi098.stdout:/tmp/tmp.vC1KSF1huV 2024-09-21T00:29:58.127 DEBUG:teuthology.orchestra.run.smithi098:> sudo cp /var/run/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ops-47099ce0-1-mds.i.json /tmp/tmp.vC1KSF1huV 2024-09-21T00:29:58.328 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0666 /tmp/tmp.vC1KSF1huV 2024-09-21T00:29:58.419 DEBUG:teuthology.orchestra.remote:smithi098:/tmp/tmp.vC1KSF1huV is 32MB 2024-09-21T00:29:59.091 DEBUG:teuthology.orchestra.run.smithi098:> rm -fr /tmp/tmp.vC1KSF1huV 2024-09-21T00:29:59.183 DEBUG:tasks.quiescer.fs.[cephfs]:read 33136695B of ops from '/var/run/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ops-47099ce0-1-mds.i.json' on mds.2 (i) 2024-09-21T00:30:00.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31750 fp=#0x10000003278' 2024-09-21T00:30:00.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31814 fp=#0x100000020db' 2024-09-21T00:30:00.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31942 fp=#0x10000001857' 2024-09-21T00:30:00.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34022 fp=#0x200000007e3' 2024-09-21T00:30:00.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34694 fp=#0x10000000c8d' 2024-09-21T00:30:00.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34726 fp=#0x10000000b62' 2024-09-21T00:30:00.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34758 fp=#0x300000022ad' 2024-09-21T00:30:00.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34854 fp=#0x1000000280f' 2024-09-21T00:30:00.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34886 fp=#0x10000002712' 2024-09-21T00:30:00.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34982 fp=#0x10000003863' 2024-09-21T00:30:00.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35014 fp=#0x3000000205e' 2024-09-21T00:30:00.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35046 fp=#0x10000003860' 2024-09-21T00:30:00.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35078 fp=#0x300000020ad' 2024-09-21T00:30:00.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35110 fp=#0x10000003906' 2024-09-21T00:30:00.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35142 fp=#0x10000003b0e' 2024-09-21T00:30:00.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35174 fp=#0x100000039bc' 2024-09-21T00:30:00.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35206 fp=#0x10000004216' 2024-09-21T00:30:00.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35238 fp=#0x10000003989' 2024-09-21T00:30:00.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35270 fp=#0x10000003a84' 2024-09-21T00:30:00.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35302 fp=#0x100000041b7' 2024-09-21T00:30:00.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35334 fp=#0x10000004244' 2024-09-21T00:30:00.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35366 fp=#0x100000038af' 2024-09-21T00:30:00.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35398 fp=#0x10000003887' 2024-09-21T00:30:00.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35430 fp=#0x100000038a6' 2024-09-21T00:30:00.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35462 fp=#0x30000001141' 2024-09-21T00:30:00.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35494 fp=#0x30000001148' 2024-09-21T00:30:00.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35526 fp=#0x3000000114b' 2024-09-21T00:30:00.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35558 fp=#0x30000002050' 2024-09-21T00:30:00.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35590 fp=#0x30000002024' 2024-09-21T00:30:00.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35622 fp=#0x30000001fef' 2024-09-21T00:30:00.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35654 fp=#0x30000001fd2' 2024-09-21T00:30:00.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35686 fp=#0x30000002029' 2024-09-21T00:30:00.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35814 fp=#0x1000000346a' 2024-09-21T00:30:00.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35846 fp=#0x10000003390' 2024-09-21T00:30:00.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35878 fp=#0x10000003453' 2024-09-21T00:30:00.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35910 fp=#0x1000000345f' 2024-09-21T00:30:00.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35942 fp=#0x300000011d6' 2024-09-21T00:30:00.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35974 fp=#0x300000011dd' 2024-09-21T00:30:00.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36070 fp=#0x10000002d7e' 2024-09-21T00:30:00.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36102 fp=#0x10000002d7b' 2024-09-21T00:30:00.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36134 fp=#0x10000002ca5' 2024-09-21T00:30:00.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36166 fp=#0x10000001ead' 2024-09-21T00:30:00.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36198 fp=#0x10000001f2e' 2024-09-21T00:30:00.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36230 fp=#0x10000001e90' 2024-09-21T00:30:00.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36262 fp=#0x300000015a9' 2024-09-21T00:30:00.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36294 fp=#0x300000015bf' 2024-09-21T00:30:00.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36326 fp=#0x3000000160b' 2024-09-21T00:30:00.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36358 fp=#0x300000015a5' 2024-09-21T00:30:00.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36422 fp=#0x300000015a4' 2024-09-21T00:30:00.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36454 fp=#0x10000001dc7' 2024-09-21T00:30:00.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36486 fp=#0x10000001e81' 2024-09-21T00:30:00.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36518 fp=#0x10000001d62' 2024-09-21T00:30:00.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36550 fp=#0x10000001e23' 2024-09-21T00:30:00.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36582 fp=#0x10000004428' 2024-09-21T00:30:00.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36614 fp=#0x10000004438' 2024-09-21T00:30:00.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36646 fp=#0x1000000440e' 2024-09-21T00:30:00.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36678 fp=#0x10000003774' 2024-09-21T00:30:00.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36710 fp=#0x10000004466' 2024-09-21T00:30:00.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36742 fp=#0x1000000471c' 2024-09-21T00:30:00.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36774 fp=#0x1000000477e' 2024-09-21T00:30:00.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36806 fp=#0x10000004750' 2024-09-21T00:30:00.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36838 fp=#0x1000000476d' 2024-09-21T00:30:00.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36902 fp=#0x10000003624' 2024-09-21T00:30:00.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36998 fp=#0x1000000253c' 2024-09-21T00:30:00.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37030 fp=#0x10000002605' 2024-09-21T00:30:00.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37062 fp=#0x300000017fb' 2024-09-21T00:30:00.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37126 fp=#0x300000017f7' 2024-09-21T00:30:00.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37158 fp=#0x10000001470' 2024-09-21T00:30:00.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37222 fp=#0x10000002ff0' 2024-09-21T00:30:00.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37254 fp=#0x10000003029' 2024-09-21T00:30:00.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37286 fp=#0x10000003109' 2024-09-21T00:30:00.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37318 fp=#0x30000001b32' 2024-09-21T00:30:00.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37350 fp=#0x30000000beb' 2024-09-21T00:30:00.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37382 fp=#0x30000001b09' 2024-09-21T00:30:00.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37414 fp=#0x30000000bf1' 2024-09-21T00:30:00.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37446 fp=#0x30000001af5' 2024-09-21T00:30:00.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37478 fp=#0x300000010be' 2024-09-21T00:30:00.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37510 fp=#0x30000001b67' 2024-09-21T00:30:00.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37574 fp=#0x30000000fd3' 2024-09-21T00:30:00.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37606 fp=#0x30000000fcf' 2024-09-21T00:30:00.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37638 fp=#0x30000000c39' 2024-09-21T00:30:00.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37670 fp=#0x10000002846' 2024-09-21T00:30:00.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37734 fp=#0x30000000c98' 2024-09-21T00:30:00.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37798 fp=#0x30000000c42' 2024-09-21T00:30:00.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37926 fp=#0x3000000224d' 2024-09-21T00:30:00.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37990 fp=#0x30000000cec' 2024-09-21T00:30:00.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38022 fp=#0x30000000cc3' 2024-09-21T00:30:00.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38086 fp=#0x10000000736' 2024-09-21T00:30:00.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38118 fp=#0x10000000759' 2024-09-21T00:30:00.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38182 fp=#0x1000000072d' 2024-09-21T00:30:00.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38214 fp=#0x10000000765' 2024-09-21T00:30:00.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38310 fp=#0x30000001a3e' 2024-09-21T00:30:00.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38342 fp=#0x30000001a61' 2024-09-21T00:30:00.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38406 fp=#0x10000003334' 2024-09-21T00:30:00.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38470 fp=#0x100000032dd' 2024-09-21T00:30:00.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38502 fp=#0x100000020e7' 2024-09-21T00:30:00.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38534 fp=#0x10000002122' 2024-09-21T00:30:00.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38566 fp=#0x10000002161' 2024-09-21T00:30:00.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38630 fp=#0x100000005b4' 2024-09-21T00:30:00.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38662 fp=#0x30000001bf3' 2024-09-21T00:30:00.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38694 fp=#0x30000001c40' 2024-09-21T00:30:00.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38726 fp=#0x30000001bdc' 2024-09-21T00:30:00.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38822 fp=#0x300000019be' 2024-09-21T00:30:00.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38854 fp=#0x300000019d9' 2024-09-21T00:30:00.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38886 fp=#0x1000000294d' 2024-09-21T00:30:00.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38918 fp=#0x10000002929' 2024-09-21T00:30:00.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38950 fp=#0x30000001d48' 2024-09-21T00:30:00.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39014 fp=#0x30000001c30' 2024-09-21T00:30:00.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39046 fp=#0x30000001c3c' 2024-09-21T00:30:00.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39238 fp=#0x10000001279' 2024-09-21T00:30:00.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39270 fp=#0x100000034f2' 2024-09-21T00:30:00.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39366 fp=#0x10000003f3d' 2024-09-21T00:30:00.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39398 fp=#0x10000003f9e' 2024-09-21T00:30:00.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39430 fp=#0x10000003f0b' 2024-09-21T00:30:00.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39462 fp=#0x30000001f5d' 2024-09-21T00:30:00.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39494 fp=#0x30000001f36' 2024-09-21T00:30:00.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39526 fp=#0x30000001f83' 2024-09-21T00:30:00.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39558 fp=#0x30000001f1f' 2024-09-21T00:30:00.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39590 fp=#0x10000002b21' 2024-09-21T00:30:00.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39622 fp=#0x10000002b2d' 2024-09-21T00:30:00.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39718 fp=#0x30000000e96' 2024-09-21T00:30:00.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39750 fp=#0x30000001991' 2024-09-21T00:30:00.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39814 fp=#0x30000000e5a' 2024-09-21T00:30:00.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39878 fp=#0x300000019a3' 2024-09-21T00:30:00.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39910 fp=#0x30000001010' 2024-09-21T00:30:00.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39974 fp=#0x30000000ffc' 2024-09-21T00:30:00.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40038 fp=#0x30000001817' 2024-09-21T00:30:00.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40070 fp=#0x30000001800' 2024-09-21T00:30:00.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40102 fp=#0x30000001677' 2024-09-21T00:30:00.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40134 fp=#0x3000000152d' 2024-09-21T00:30:00.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40166 fp=#0x30000001837' 2024-09-21T00:30:00.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40198 fp=#0x10000001d6e' 2024-09-21T00:30:00.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40230 fp=#0x10000001d25' 2024-09-21T00:30:00.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40262 fp=#0x30000000d9c' 2024-09-21T00:30:00.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40294 fp=#0x10000001d38' 2024-09-21T00:30:00.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40326 fp=#0x30000001d10' 2024-09-21T00:30:00.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40358 fp=#0x10000004c78' 2024-09-21T00:30:00.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40390 fp=#0x10000004e1b' 2024-09-21T00:30:00.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40422 fp=#0x10000004e25' 2024-09-21T00:30:00.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40454 fp=#0x10000004e78' 2024-09-21T00:30:00.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40486 fp=#0x10000004e0b' 2024-09-21T00:30:00.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40550 fp=#0x1000000101d' 2024-09-21T00:30:00.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40614 fp=#0x10000000f60' 2024-09-21T00:30:00.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40646 fp=#0x200000005ec' 2024-09-21T00:30:00.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40742 fp=#0x20000000835' 2024-09-21T00:30:00.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40774 fp=#0x30000001423' 2024-09-21T00:30:00.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40806 fp=#0x3000000142f' 2024-09-21T00:30:00.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40838 fp=#0x10000004a7b' 2024-09-21T00:30:00.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40870 fp=#0x10000004cab' 2024-09-21T00:30:00.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40902 fp=#0x10000004a7e' 2024-09-21T00:30:00.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41062 fp=#0x100000036df' 2024-09-21T00:30:00.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41094 fp=#0x10000003503' 2024-09-21T00:30:00.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41126 fp=#0x20000000211' 2024-09-21T00:30:00.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41158 fp=#0x10000000d25' 2024-09-21T00:30:00.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41190 fp=#0x20000000c79' 2024-09-21T00:30:00.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41222 fp=#0x20000000c51' 2024-09-21T00:30:00.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41254 fp=#0x10000000d4b' 2024-09-21T00:30:00.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41286 fp=#0x20000000c80' 2024-09-21T00:30:00.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41318 fp=#0x20000000be0' 2024-09-21T00:30:00.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41382 fp=#0x10000000ab7' 2024-09-21T00:30:00.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41414 fp=#0x10000000c11' 2024-09-21T00:30:00.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41446 fp=#0x10000000dfa' 2024-09-21T00:30:00.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41478 fp=#0x100000009ab' 2024-09-21T00:30:00.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41510 fp=#0x1000000099c' 2024-09-21T00:30:00.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41542 fp=#0x1000000096c' 2024-09-21T00:30:00.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41574 fp=#0x3000000190e' 2024-09-21T00:30:00.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41670 fp=#0x3000000183f' 2024-09-21T00:30:00.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41702 fp=#0x300000007f1' 2024-09-21T00:30:00.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41766 fp=#0x300000008fa' 2024-09-21T00:30:00.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41798 fp=#0x300000009b3' 2024-09-21T00:30:00.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41830 fp=#0x300000009c4' 2024-09-21T00:30:00.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41862 fp=#0x3000000086a' 2024-09-21T00:30:00.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41894 fp=#0x30000000856' 2024-09-21T00:30:00.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41926 fp=#0x30000000885' 2024-09-21T00:30:00.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41958 fp=#0x300000009ba' 2024-09-21T00:30:00.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41990 fp=#0x30000001d81' 2024-09-21T00:30:00.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42022 fp=#0x30000001315' 2024-09-21T00:30:00.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42054 fp=#0x30000001d95' 2024-09-21T00:30:00.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42086 fp=#0x3000000094a' 2024-09-21T00:30:00.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42118 fp=#0x30000001d52' 2024-09-21T00:30:00.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42150 fp=#0x30000000835' 2024-09-21T00:30:00.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42182 fp=#0x30000001ba7' 2024-09-21T00:30:00.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42214 fp=#0x30000001b9e' 2024-09-21T00:30:00.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42246 fp=#0x30000000bc4' 2024-09-21T00:30:00.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42278 fp=#0x30000002081' 2024-09-21T00:30:00.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42310 fp=#0x30000002082' 2024-09-21T00:30:00.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42342 fp=#0x30000000b53' 2024-09-21T00:30:00.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42374 fp=#0x30000000aed' 2024-09-21T00:30:00.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42406 fp=#0x30000000af4' 2024-09-21T00:30:00.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42566 fp=#0x30000000a54' 2024-09-21T00:30:00.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42598 fp=#0x30000000d33' 2024-09-21T00:30:00.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42662 fp=#0x10000001a29' 2024-09-21T00:30:00.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42694 fp=#0x10000001961' 2024-09-21T00:30:00.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42726 fp=#0x10000001a00' 2024-09-21T00:30:00.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42758 fp=#0x10000001917' 2024-09-21T00:30:00.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42822 fp=#0x10000001a18' 2024-09-21T00:30:00.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42854 fp=#0x1000000194e' 2024-09-21T00:30:00.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42886 fp=#0x100000019a8' 2024-09-21T00:30:00.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42918 fp=#0x30000000e4d' 2024-09-21T00:30:00.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42982 fp=#0x30000000dfd' 2024-09-21T00:30:00.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43014 fp=#0x30000001757' 2024-09-21T00:30:00.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43046 fp=#0x300000016e8' 2024-09-21T00:30:00.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43078 fp=#0x3000000175b' 2024-09-21T00:30:00.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43110 fp=#0x300000016af' 2024-09-21T00:30:00.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43142 fp=#0x100000018b9' 2024-09-21T00:30:00.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43270 fp=#0x30000000f78' 2024-09-21T00:30:00.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43462 fp=#0x30000002378' 2024-09-21T00:30:00.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43494 fp=#0x30000002381' 2024-09-21T00:30:00.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43526 fp=#0x300000023bb' 2024-09-21T00:30:00.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43558 fp=#0x30000002394' 2024-09-21T00:30:00.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43590 fp=#0x300000010e6' 2024-09-21T00:30:00.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43622 fp=#0x300000010ba' 2024-09-21T00:30:00.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43654 fp=#0x30000002100' 2024-09-21T00:30:00.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43686 fp=#0x30000002110' 2024-09-21T00:30:00.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43718 fp=#0x300000010e4' 2024-09-21T00:30:00.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43750 fp=#0x3000000210d' 2024-09-21T00:30:00.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43782 fp=#0x10000003990' 2024-09-21T00:30:00.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43846 fp=#0x10000003972' 2024-09-21T00:30:00.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43910 fp=#0x10000003a14' 2024-09-21T00:30:00.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43942 fp=#0x300000013a5' 2024-09-21T00:30:00.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31751 fp=#0x100000038b1' 2024-09-21T00:30:00.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31815 fp=#0x100000004fd' 2024-09-21T00:30:00.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31943 fp=#0x10000003d47' 2024-09-21T00:30:00.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34695 fp=#0x30000002245' 2024-09-21T00:30:00.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34727 fp=#0x10000000c6f' 2024-09-21T00:30:00.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34759 fp=#0x30000002290' 2024-09-21T00:30:00.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34983 fp=#0x1000000386c' 2024-09-21T00:30:00.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35015 fp=#0x1000000328f' 2024-09-21T00:30:00.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35047 fp=#0x100000032c4' 2024-09-21T00:30:00.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35079 fp=#0x300000020b4' 2024-09-21T00:30:00.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35111 fp=#0x10000003a5b' 2024-09-21T00:30:00.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35143 fp=#0x100000039df' 2024-09-21T00:30:00.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35175 fp=#0x1000000416d' 2024-09-21T00:30:00.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35207 fp=#0x1000000392e' 2024-09-21T00:30:00.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35239 fp=#0x10000003926' 2024-09-21T00:30:00.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35271 fp=#0x10000003aba' 2024-09-21T00:30:00.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35303 fp=#0x10000004137' 2024-09-21T00:30:00.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35335 fp=#0x1000000420d' 2024-09-21T00:30:00.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35367 fp=#0x10000003a22' 2024-09-21T00:30:00.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35399 fp=#0x100000038ba' 2024-09-21T00:30:00.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35431 fp=#0x1000000394f' 2024-09-21T00:30:00.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35463 fp=#0x30000001189' 2024-09-21T00:30:00.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35495 fp=#0x3000000114f' 2024-09-21T00:30:00.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35527 fp=#0x30000001192' 2024-09-21T00:30:00.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35559 fp=#0x3000000204b' 2024-09-21T00:30:00.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35591 fp=#0x30000002004' 2024-09-21T00:30:00.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35623 fp=#0x30000001fd6' 2024-09-21T00:30:00.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35655 fp=#0x30000001ff1' 2024-09-21T00:30:00.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35687 fp=#0x3000000200a' 2024-09-21T00:30:00.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35879 fp=#0x10000003412' 2024-09-21T00:30:00.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35943 fp=#0x300000011e4' 2024-09-21T00:30:00.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35975 fp=#0x300000011e1' 2024-09-21T00:30:00.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36071 fp=#0x10000002caa' 2024-09-21T00:30:00.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36103 fp=#0x10000002d63' 2024-09-21T00:30:00.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36135 fp=#0x10000002cb3' 2024-09-21T00:30:00.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36199 fp=#0x10000001edd' 2024-09-21T00:30:00.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36231 fp=#0x10000002075' 2024-09-21T00:30:00.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36295 fp=#0x300000021d2' 2024-09-21T00:30:00.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36391 fp=#0x300000015ac' 2024-09-21T00:30:00.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36423 fp=#0x300000015aa' 2024-09-21T00:30:00.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36455 fp=#0x10000001e53' 2024-09-21T00:30:00.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36487 fp=#0x10000001e27' 2024-09-21T00:30:00.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36519 fp=#0x10000001dc0' 2024-09-21T00:30:00.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36551 fp=#0x10000001e0d' 2024-09-21T00:30:00.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36583 fp=#0x10000003815' 2024-09-21T00:30:00.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36615 fp=#0x10000004426' 2024-09-21T00:30:00.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36647 fp=#0x10000003811' 2024-09-21T00:30:00.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36679 fp=#0x10000004424' 2024-09-21T00:30:00.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36711 fp=#0x10000004437' 2024-09-21T00:30:00.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36743 fp=#0x10000004724' 2024-09-21T00:30:00.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36775 fp=#0x100000047d0' 2024-09-21T00:30:00.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36807 fp=#0x1000000474a' 2024-09-21T00:30:00.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36839 fp=#0x10000004709' 2024-09-21T00:30:00.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36871 fp=#0x10000002583' 2024-09-21T00:30:00.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36903 fp=#0x1000000361d' 2024-09-21T00:30:00.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36967 fp=#0x10000002556' 2024-09-21T00:30:00.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37031 fp=#0x100000035f3' 2024-09-21T00:30:00.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37063 fp=#0x3000000180f' 2024-09-21T00:30:00.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37159 fp=#0x100000030b1' 2024-09-21T00:30:00.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37287 fp=#0x10000003128' 2024-09-21T00:30:00.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37319 fp=#0x30000000bef' 2024-09-21T00:30:00.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37351 fp=#0x30000000c1c' 2024-09-21T00:30:00.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37383 fp=#0x30000001abf' 2024-09-21T00:30:00.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37415 fp=#0x30000001b22' 2024-09-21T00:30:00.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37447 fp=#0x30000001ab5' 2024-09-21T00:30:00.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37479 fp=#0x30000000fce' 2024-09-21T00:30:00.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37511 fp=#0x30000001b70' 2024-09-21T00:30:00.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37703 fp=#0x30000000c8a' 2024-09-21T00:30:00.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37735 fp=#0x10000002948' 2024-09-21T00:30:00.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37767 fp=#0x1000000283b' 2024-09-21T00:30:00.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37863 fp=#0x30000001269' 2024-09-21T00:30:00.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37895 fp=#0x3000000126b' 2024-09-21T00:30:00.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37959 fp=#0x30000000dd4' 2024-09-21T00:30:00.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38087 fp=#0x1000000076c' 2024-09-21T00:30:00.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38119 fp=#0x10000000743' 2024-09-21T00:30:00.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38183 fp=#0x10000000752' 2024-09-21T00:30:00.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38215 fp=#0x10000000727' 2024-09-21T00:30:00.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38279 fp=#0x30000001a4c' 2024-09-21T00:30:00.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38311 fp=#0x10000002480' 2024-09-21T00:30:00.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38343 fp=#0x30000001a68' 2024-09-21T00:30:00.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38375 fp=#0x30000001a79' 2024-09-21T00:30:00.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38471 fp=#0x10000003289' 2024-09-21T00:30:00.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38503 fp=#0x10000002194' 2024-09-21T00:30:00.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38535 fp=#0x10000002130' 2024-09-21T00:30:00.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38663 fp=#0x300000019b5' 2024-09-21T00:30:00.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38695 fp=#0x100000005ed' 2024-09-21T00:30:00.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38727 fp=#0x10000000519' 2024-09-21T00:30:00.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38823 fp=#0x1000000295c' 2024-09-21T00:30:00.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38855 fp=#0x300000019c8' 2024-09-21T00:30:00.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38887 fp=#0x10000002964' 2024-09-21T00:30:00.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38919 fp=#0x10000002938' 2024-09-21T00:30:00.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38983 fp=#0x30000001d87' 2024-09-21T00:30:00.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39047 fp=#0x30000001d2b' 2024-09-21T00:30:00.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39207 fp=#0x10000001119' 2024-09-21T00:30:00.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39239 fp=#0x10000003527' 2024-09-21T00:30:00.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39335 fp=#0x100000034b9' 2024-09-21T00:30:00.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39367 fp=#0x10000003fbe' 2024-09-21T00:30:00.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39399 fp=#0x10000003f50' 2024-09-21T00:30:00.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39431 fp=#0x10000003f71' 2024-09-21T00:30:00.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39463 fp=#0x30000001fb2' 2024-09-21T00:30:00.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39495 fp=#0x30000001f10' 2024-09-21T00:30:00.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39527 fp=#0x30000001f2c' 2024-09-21T00:30:00.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39559 fp=#0x30000001f4c' 2024-09-21T00:30:00.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39591 fp=#0x10000002b0b' 2024-09-21T00:30:00.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39687 fp=#0x30000001998' 2024-09-21T00:30:00.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39719 fp=#0x30000001993' 2024-09-21T00:30:00.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39751 fp=#0x30000000ea1' 2024-09-21T00:30:00.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39783 fp=#0x30000000eb2' 2024-09-21T00:30:00.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39815 fp=#0x30000000ebb' 2024-09-21T00:30:00.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39847 fp=#0x30000000e5c' 2024-09-21T00:30:00.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39879 fp=#0x30000000e6c' 2024-09-21T00:30:00.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39943 fp=#0x30000001128' 2024-09-21T00:30:00.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39975 fp=#0x3000000110e' 2024-09-21T00:30:00.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40039 fp=#0x300000014bd' 2024-09-21T00:30:00.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40071 fp=#0x30000001860' 2024-09-21T00:30:00.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40103 fp=#0x300000017ed' 2024-09-21T00:30:00.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40135 fp=#0x3000000181f' 2024-09-21T00:30:00.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40199 fp=#0x30000001d15' 2024-09-21T00:30:00.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40231 fp=#0x10000001d0a' 2024-09-21T00:30:00.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40263 fp=#0x30000000dd1' 2024-09-21T00:30:00.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40295 fp=#0x30000000d57' 2024-09-21T00:30:00.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40327 fp=#0x30000000dc9' 2024-09-21T00:30:00.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40359 fp=#0x10000004c6d' 2024-09-21T00:30:00.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40391 fp=#0x10000004e7c' 2024-09-21T00:30:00.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40423 fp=#0x200000012d9' 2024-09-21T00:30:00.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40455 fp=#0x30000001371' 2024-09-21T00:30:00.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40487 fp=#0x3000000232f' 2024-09-21T00:30:00.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40519 fp=#0x10000004645' 2024-09-21T00:30:00.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40551 fp=#0x200000005e9' 2024-09-21T00:30:00.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40583 fp=#0x10000000f17' 2024-09-21T00:30:00.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40615 fp=#0x200000005cf' 2024-09-21T00:30:00.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40647 fp=#0x200000001f6' 2024-09-21T00:30:00.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40679 fp=#0x10000000c9d' 2024-09-21T00:30:00.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40711 fp=#0x20000000455' 2024-09-21T00:30:00.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40775 fp=#0x30000001459' 2024-09-21T00:30:00.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40807 fp=#0x30000001434' 2024-09-21T00:30:00.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40839 fp=#0x10000004a76' 2024-09-21T00:30:00.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40871 fp=#0x10000004ca8' 2024-09-21T00:30:00.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40903 fp=#0x10000004c9c' 2024-09-21T00:30:00.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41031 fp=#0x1000000355b' 2024-09-21T00:30:00.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41063 fp=#0x2000000070b' 2024-09-21T00:30:00.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41095 fp=#0x20000000700' 2024-09-21T00:30:00.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41127 fp=#0x20000000ba3' 2024-09-21T00:30:00.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41159 fp=#0x10000000fd4' 2024-09-21T00:30:00.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41191 fp=#0x20000000b55' 2024-09-21T00:30:00.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41223 fp=#0x10000000eaa' 2024-09-21T00:30:00.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41255 fp=#0x10000000cd7' 2024-09-21T00:30:00.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41287 fp=#0x10000000d6e' 2024-09-21T00:30:00.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41319 fp=#0x10000000f51' 2024-09-21T00:30:00.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41383 fp=#0x10000000df7' 2024-09-21T00:30:00.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41415 fp=#0x10000000bf8' 2024-09-21T00:30:00.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41447 fp=#0x10000000b42' 2024-09-21T00:30:00.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41479 fp=#0x10000000c4c' 2024-09-21T00:30:00.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41543 fp=#0x100000009e4' 2024-09-21T00:30:00.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41735 fp=#0x30000000916' 2024-09-21T00:30:00.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41799 fp=#0x300000009db' 2024-09-21T00:30:00.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41831 fp=#0x300000009b8' 2024-09-21T00:30:00.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41863 fp=#0x30000000889' 2024-09-21T00:30:00.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41895 fp=#0x300000009c5' 2024-09-21T00:30:00.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41959 fp=#0x30000000825' 2024-09-21T00:30:00.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41991 fp=#0x30000000860' 2024-09-21T00:30:00.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42023 fp=#0x30000000815' 2024-09-21T00:30:00.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42055 fp=#0x300000007e6' 2024-09-21T00:30:00.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42087 fp=#0x300000007f4' 2024-09-21T00:30:00.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42119 fp=#0x30000001d40' 2024-09-21T00:30:00.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42151 fp=#0x3000000084a' 2024-09-21T00:30:00.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42183 fp=#0x30000000b65' 2024-09-21T00:30:00.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42215 fp=#0x30000001b96' 2024-09-21T00:30:00.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42247 fp=#0x30000000b78' 2024-09-21T00:30:00.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42279 fp=#0x30000000bb0' 2024-09-21T00:30:00.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42311 fp=#0x30000000b90' 2024-09-21T00:30:00.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42343 fp=#0x30000000b05' 2024-09-21T00:30:00.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42375 fp=#0x30000000b4f' 2024-09-21T00:30:00.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42407 fp=#0x30000000b45' 2024-09-21T00:30:00.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42439 fp=#0x30000000b44' 2024-09-21T00:30:00.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42471 fp=#0x30000000a61' 2024-09-21T00:30:00.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42535 fp=#0x100000040ef' 2024-09-21T00:30:00.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42727 fp=#0x100000018c3' 2024-09-21T00:30:00.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42791 fp=#0x100000019c5' 2024-09-21T00:30:00.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42823 fp=#0x1000000195b' 2024-09-21T00:30:00.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42983 fp=#0x30000000e4a' 2024-09-21T00:30:00.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43015 fp=#0x3000000174e' 2024-09-21T00:30:00.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43047 fp=#0x3000000171e' 2024-09-21T00:30:00.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43079 fp=#0x300000016ec' 2024-09-21T00:30:00.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43111 fp=#0x300000016a3' 2024-09-21T00:30:00.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43143 fp=#0x100000018bd' 2024-09-21T00:30:00.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43239 fp=#0x10000001871' 2024-09-21T00:30:00.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43303 fp=#0x30000000f4f' 2024-09-21T00:30:00.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43335 fp=#0x30000000f2b' 2024-09-21T00:30:00.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43367 fp=#0x30000000f3d' 2024-09-21T00:30:00.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43463 fp=#0x3000000237b' 2024-09-21T00:30:00.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43495 fp=#0x3000000238e' 2024-09-21T00:30:00.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43527 fp=#0x3000000238a' 2024-09-21T00:30:00.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43559 fp=#0x300000023e2' 2024-09-21T00:30:00.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43591 fp=#0x30000000fa7' 2024-09-21T00:30:00.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43687 fp=#0x300000021bc' 2024-09-21T00:30:00.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43847 fp=#0x10000003a3f' 2024-09-21T00:30:00.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43943 fp=#0x30000001386' 2024-09-21T00:30:00.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31752 fp=#0x10000003872' 2024-09-21T00:30:00.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31816 fp=#0x1000000290e' 2024-09-21T00:30:00.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31944 fp=#0x10000003495' 2024-09-21T00:30:00.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34696 fp=#0x300000022b5' 2024-09-21T00:30:00.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34760 fp=#0x30000002283' 2024-09-21T00:30:00.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34856 fp=#0x10000002808' 2024-09-21T00:30:00.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34984 fp=#0x100000032ae' 2024-09-21T00:30:00.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35016 fp=#0x30000002092' 2024-09-21T00:30:00.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35048 fp=#0x100000032a8' 2024-09-21T00:30:00.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35080 fp=#0x10000003884' 2024-09-21T00:30:00.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35112 fp=#0x10000003b27' 2024-09-21T00:30:00.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35144 fp=#0x10000003aef' 2024-09-21T00:30:00.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35176 fp=#0x1000000415c' 2024-09-21T00:30:00.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35208 fp=#0x10000003942' 2024-09-21T00:30:00.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35240 fp=#0x10000003a4c' 2024-09-21T00:30:00.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35272 fp=#0x100000039c6' 2024-09-21T00:30:00.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35304 fp=#0x10000003ad4' 2024-09-21T00:30:00.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35336 fp=#0x1000000390f' 2024-09-21T00:30:00.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35368 fp=#0x30000001647' 2024-09-21T00:30:00.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35400 fp=#0x100000038e3' 2024-09-21T00:30:00.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35432 fp=#0x10000003905' 2024-09-21T00:30:00.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35464 fp=#0x3000000111c' 2024-09-21T00:30:00.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35496 fp=#0x30000001153' 2024-09-21T00:30:00.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35528 fp=#0x30000001163' 2024-09-21T00:30:00.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35560 fp=#0x30000001ff9' 2024-09-21T00:30:00.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35592 fp=#0x30000002033' 2024-09-21T00:30:00.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35656 fp=#0x3000000204e' 2024-09-21T00:30:00.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35688 fp=#0x3000000203b' 2024-09-21T00:30:00.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35880 fp=#0x10000003401' 2024-09-21T00:30:00.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35912 fp=#0x100000033e9' 2024-09-21T00:30:00.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35944 fp=#0x300000011c7' 2024-09-21T00:30:00.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35976 fp=#0x300000011fa' 2024-09-21T00:30:00.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36072 fp=#0x10000002d5b' 2024-09-21T00:30:00.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36104 fp=#0x10000002cd6' 2024-09-21T00:30:00.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36168 fp=#0x10000002086' 2024-09-21T00:30:00.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36232 fp=#0x10000001f13' 2024-09-21T00:30:00.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36264 fp=#0x300000015fd' 2024-09-21T00:30:00.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36456 fp=#0x10000001d39' 2024-09-21T00:30:00.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36488 fp=#0x10000001e17' 2024-09-21T00:30:00.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36520 fp=#0x3000000105b' 2024-09-21T00:30:00.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36552 fp=#0x10000001e47' 2024-09-21T00:30:00.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36584 fp=#0x1000000386b' 2024-09-21T00:30:00.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36616 fp=#0x100000044b1' 2024-09-21T00:30:00.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36648 fp=#0x100000037e4' 2024-09-21T00:30:00.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36680 fp=#0x10000004442' 2024-09-21T00:30:00.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36712 fp=#0x1000000379a' 2024-09-21T00:30:00.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36744 fp=#0x10000004797' 2024-09-21T00:30:00.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36776 fp=#0x30000001a96' 2024-09-21T00:30:00.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36808 fp=#0x1000000470c' 2024-09-21T00:30:00.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36872 fp=#0x10000002542' 2024-09-21T00:30:00.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36904 fp=#0x10000002565' 2024-09-21T00:30:00.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36936 fp=#0x10000002695' 2024-09-21T00:30:00.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36968 fp=#0x10000003580' 2024-09-21T00:30:00.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37032 fp=#0x100000036b2' 2024-09-21T00:30:00.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37064 fp=#0x3000000178c' 2024-09-21T00:30:00.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37128 fp=#0x30000001792' 2024-09-21T00:30:00.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37160 fp=#0x100000031a3' 2024-09-21T00:30:00.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37224 fp=#0x100000030d1' 2024-09-21T00:30:00.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37256 fp=#0x1000000319a' 2024-09-21T00:30:00.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37288 fp=#0x1000000310e' 2024-09-21T00:30:00.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37320 fp=#0x30000001b07' 2024-09-21T00:30:00.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37352 fp=#0x30000001aec' 2024-09-21T00:30:00.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37384 fp=#0x30000000be9' 2024-09-21T00:30:00.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37416 fp=#0x30000001abb' 2024-09-21T00:30:00.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37448 fp=#0x30000000c11' 2024-09-21T00:30:00.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37480 fp=#0x30000001068' 2024-09-21T00:30:00.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37576 fp=#0x3000000107a' 2024-09-21T00:30:00.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37640 fp=#0x1000000293c' 2024-09-21T00:30:00.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37704 fp=#0x30000000c85' 2024-09-21T00:30:00.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37736 fp=#0x30000000c66' 2024-09-21T00:30:00.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37768 fp=#0x30000000c60' 2024-09-21T00:30:00.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37832 fp=#0x30000000c68' 2024-09-21T00:30:00.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37864 fp=#0x30000001271' 2024-09-21T00:30:00.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37896 fp=#0x300000012c1' 2024-09-21T00:30:00.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37928 fp=#0x3000000224b' 2024-09-21T00:30:00.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38056 fp=#0x30000000ccb' 2024-09-21T00:30:00.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38120 fp=#0x10000000750' 2024-09-21T00:30:00.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38184 fp=#0x10000000e4c' 2024-09-21T00:30:00.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38216 fp=#0x10000000e06' 2024-09-21T00:30:00.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38248 fp=#0x1000000076a' 2024-09-21T00:30:00.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38312 fp=#0x30000001a3f' 2024-09-21T00:30:00.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38344 fp=#0x30000001a4b' 2024-09-21T00:30:00.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38376 fp=#0x30000001a53' 2024-09-21T00:30:00.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38440 fp=#0x30000001cd3' 2024-09-21T00:30:00.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38472 fp=#0x100000032e4' 2024-09-21T00:30:00.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38504 fp=#0x10000002199' 2024-09-21T00:30:00.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38536 fp=#0x10000002148' 2024-09-21T00:30:00.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38568 fp=#0x10000002146' 2024-09-21T00:30:00.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38696 fp=#0x30000001c08' 2024-09-21T00:30:00.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38760 fp=#0x10000000595' 2024-09-21T00:30:00.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38824 fp=#0x30000001eed' 2024-09-21T00:30:00.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38888 fp=#0x300000019db' 2024-09-21T00:30:00.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38920 fp=#0x300000019da' 2024-09-21T00:30:00.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38952 fp=#0x30000001d2e' 2024-09-21T00:30:00.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38984 fp=#0x30000001d06' 2024-09-21T00:30:00.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39048 fp=#0x30000001c46' 2024-09-21T00:30:00.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39304 fp=#0x100000034e4' 2024-09-21T00:30:00.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39336 fp=#0x10000003f2f' 2024-09-21T00:30:00.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39368 fp=#0x10000003f19' 2024-09-21T00:30:00.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39400 fp=#0x10000003faf' 2024-09-21T00:30:00.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39432 fp=#0x10000003f79' 2024-09-21T00:30:00.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39464 fp=#0x30000001fbd' 2024-09-21T00:30:00.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39496 fp=#0x30000001f40' 2024-09-21T00:30:00.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39528 fp=#0x30000001f11' 2024-09-21T00:30:00.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39560 fp=#0x30000001f1a' 2024-09-21T00:30:00.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39592 fp=#0x10000002a91' 2024-09-21T00:30:00.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39688 fp=#0x30000000e93' 2024-09-21T00:30:00.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39720 fp=#0x3000000199c' 2024-09-21T00:30:00.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39752 fp=#0x300000019b1' 2024-09-21T00:30:00.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39848 fp=#0x30000000e9f' 2024-09-21T00:30:00.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39880 fp=#0x30000000ee7' 2024-09-21T00:30:00.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39912 fp=#0x30000001126' 2024-09-21T00:30:00.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40008 fp=#0x300000014c3' 2024-09-21T00:30:00.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40040 fp=#0x3000000181a' 2024-09-21T00:30:00.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40072 fp=#0x300000014d2' 2024-09-21T00:30:00.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40104 fp=#0x30000001854' 2024-09-21T00:30:00.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40136 fp=#0x300000014d5' 2024-09-21T00:30:00.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40200 fp=#0x30000000dcb' 2024-09-21T00:30:00.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40232 fp=#0x300000009e7' 2024-09-21T00:30:00.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40264 fp=#0x10000001d11' 2024-09-21T00:30:00.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40328 fp=#0x30000000dbd' 2024-09-21T00:30:00.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40392 fp=#0x10000004e60' 2024-09-21T00:30:00.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40424 fp=#0x10000004e81' 2024-09-21T00:30:00.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40456 fp=#0x10000004c67' 2024-09-21T00:30:00.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40488 fp=#0x300000022ec' 2024-09-21T00:30:00.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40552 fp=#0x200000005cc' 2024-09-21T00:30:00.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40584 fp=#0x200000005c5' 2024-09-21T00:30:00.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40616 fp=#0x200000005c6' 2024-09-21T00:30:00.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40648 fp=#0x20000000575' 2024-09-21T00:30:00.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40712 fp=#0x2000000061c' 2024-09-21T00:30:00.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40744 fp=#0x10000000d75' 2024-09-21T00:30:00.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40776 fp=#0x30000001432' 2024-09-21T00:30:00.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40808 fp=#0x3000000143b' 2024-09-21T00:30:00.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40840 fp=#0x10000004cdf' 2024-09-21T00:30:00.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40968 fp=#0x10000004c8b' 2024-09-21T00:30:00.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41000 fp=#0x10000004d17' 2024-09-21T00:30:00.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41032 fp=#0x1000000364b' 2024-09-21T00:30:00.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41064 fp=#0x1000000361c' 2024-09-21T00:30:00.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41096 fp=#0x10000003485' 2024-09-21T00:30:00.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41128 fp=#0x2000000047f' 2024-09-21T00:30:00.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41160 fp=#0x200000005f1' 2024-09-21T00:30:00.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41192 fp=#0x10000000d5d' 2024-09-21T00:30:00.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41224 fp=#0x200000001fb' 2024-09-21T00:30:00.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41256 fp=#0x10000000fe8' 2024-09-21T00:30:00.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41288 fp=#0x10000000f8b' 2024-09-21T00:30:00.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41320 fp=#0x10000000eb4' 2024-09-21T00:30:00.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41384 fp=#0x10000000edd' 2024-09-21T00:30:00.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41448 fp=#0x10000000a46' 2024-09-21T00:30:00.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41480 fp=#0x10000000b0e' 2024-09-21T00:30:00.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41512 fp=#0x10000000964' 2024-09-21T00:30:00.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41544 fp=#0x10000000afb' 2024-09-21T00:30:00.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41576 fp=#0x3000000190c' 2024-09-21T00:30:00.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41640 fp=#0x3000000184b' 2024-09-21T00:30:00.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41832 fp=#0x30000000972' 2024-09-21T00:30:00.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41960 fp=#0x3000000099b' 2024-09-21T00:30:00.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42024 fp=#0x3000000083b' 2024-09-21T00:30:00.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42088 fp=#0x3000000093a' 2024-09-21T00:30:00.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42120 fp=#0x30000000984' 2024-09-21T00:30:00.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42184 fp=#0x30000001bc7' 2024-09-21T00:30:00.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42216 fp=#0x3000000209f' 2024-09-21T00:30:00.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42248 fp=#0x30000000b5b' 2024-09-21T00:30:00.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42280 fp=#0x30000000b5a' 2024-09-21T00:30:00.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42312 fp=#0x300000020a1' 2024-09-21T00:30:00.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42344 fp=#0x30000000ae2' 2024-09-21T00:30:00.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42376 fp=#0x30000000b21' 2024-09-21T00:30:00.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42440 fp=#0x30000000af7' 2024-09-21T00:30:00.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42504 fp=#0x30000000a77' 2024-09-21T00:30:00.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42568 fp=#0x30000001b3d' 2024-09-21T00:30:00.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42664 fp=#0x10000001946' 2024-09-21T00:30:00.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42696 fp=#0x100000018f8' 2024-09-21T00:30:00.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42920 fp=#0x30000000e4c' 2024-09-21T00:30:00.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42952 fp=#0x30000000deb' 2024-09-21T00:30:00.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43016 fp=#0x300000016d2' 2024-09-21T00:30:00.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43048 fp=#0x30000001724' 2024-09-21T00:30:00.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43080 fp=#0x300000016b1' 2024-09-21T00:30:00.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43112 fp=#0x30000001735' 2024-09-21T00:30:00.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43144 fp=#0x10000001866' 2024-09-21T00:30:00.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43176 fp=#0x100000018b1' 2024-09-21T00:30:00.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43208 fp=#0x100000018ab' 2024-09-21T00:30:00.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43272 fp=#0x30000000f2c' 2024-09-21T00:30:00.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43304 fp=#0x30000000f40' 2024-09-21T00:30:00.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43368 fp=#0x30000000f7f' 2024-09-21T00:30:00.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43400 fp=#0x30000000f05' 2024-09-21T00:30:00.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43432 fp=#0x10000004169' 2024-09-21T00:30:00.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43464 fp=#0x300000023a1' 2024-09-21T00:30:00.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43496 fp=#0x30000002398' 2024-09-21T00:30:00.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43528 fp=#0x30000002372' 2024-09-21T00:30:00.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43560 fp=#0x30000002392' 2024-09-21T00:30:00.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43624 fp=#0x30000001009' 2024-09-21T00:30:00.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43656 fp=#0x300000010c3' 2024-09-21T00:30:00.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43784 fp=#0x10000003a47' 2024-09-21T00:30:00.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43816 fp=#0x10000003a01' 2024-09-21T00:30:00.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43848 fp=#0x10000003a21' 2024-09-21T00:30:00.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43912 fp=#0x10000003a2b' 2024-09-21T00:30:00.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43944 fp=#0x3000000134d' 2024-09-21T00:30:00.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31753 fp=#0x10000004644' 2024-09-21T00:30:00.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31817 fp=#0x30000001c21' 2024-09-21T00:30:00.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31945 fp=#0x10000001b2f' 2024-09-21T00:30:00.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34729 fp=#0x30000002248' 2024-09-21T00:30:00.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34889 fp=#0x10000004863' 2024-09-21T00:30:00.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34921 fp=#0x100000027e0' 2024-09-21T00:30:00.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34953 fp=#0x100000026f4' 2024-09-21T00:30:00.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34985 fp=#0x3000000206b' 2024-09-21T00:30:00.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35017 fp=#0x10000003877' 2024-09-21T00:30:00.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35049 fp=#0x100000032de' 2024-09-21T00:30:00.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35081 fp=#0x30000002061' 2024-09-21T00:30:00.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35113 fp=#0x10000003983' 2024-09-21T00:30:00.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35145 fp=#0x10000003adb' 2024-09-21T00:30:00.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35177 fp=#0x10000004240' 2024-09-21T00:30:00.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35209 fp=#0x100000041aa' 2024-09-21T00:30:00.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35241 fp=#0x10000003aca' 2024-09-21T00:30:00.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35273 fp=#0x10000003b1b' 2024-09-21T00:30:00.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35305 fp=#0x10000003ac3' 2024-09-21T00:30:00.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35337 fp=#0x10000003b11' 2024-09-21T00:30:00.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35369 fp=#0x10000003a1b' 2024-09-21T00:30:00.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35401 fp=#0x10000003a28' 2024-09-21T00:30:00.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35433 fp=#0x300000020f5' 2024-09-21T00:30:00.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35465 fp=#0x30000001112' 2024-09-21T00:30:00.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35529 fp=#0x30000001151' 2024-09-21T00:30:00.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35561 fp=#0x3000000204a' 2024-09-21T00:30:00.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35657 fp=#0x30000002025' 2024-09-21T00:30:00.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35689 fp=#0x30000002028' 2024-09-21T00:30:00.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35817 fp=#0x10000003467' 2024-09-21T00:30:00.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35849 fp=#0x100000033b1' 2024-09-21T00:30:00.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35881 fp=#0x1000000370e' 2024-09-21T00:30:00.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35913 fp=#0x100000033ff' 2024-09-21T00:30:00.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35945 fp=#0x300000011c6' 2024-09-21T00:30:00.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35977 fp=#0x30000001205' 2024-09-21T00:30:00.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36041 fp=#0x10000003b50' 2024-09-21T00:30:00.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36073 fp=#0x10000002d70' 2024-09-21T00:30:00.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36169 fp=#0x10000001f98' 2024-09-21T00:30:00.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36201 fp=#0x100000020b1' 2024-09-21T00:30:00.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36233 fp=#0x10000003823' 2024-09-21T00:30:00.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36361 fp=#0x300000015f9' 2024-09-21T00:30:00.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36457 fp=#0x10000001e41' 2024-09-21T00:30:00.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36489 fp=#0x10000001d58' 2024-09-21T00:30:00.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36521 fp=#0x10000001e1b' 2024-09-21T00:30:00.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36553 fp=#0x10000001e21' 2024-09-21T00:30:00.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36585 fp=#0x10000004497' 2024-09-21T00:30:00.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36617 fp=#0x100000037e0' 2024-09-21T00:30:00.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36649 fp=#0x1000000377b' 2024-09-21T00:30:00.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36681 fp=#0x1000000441a' 2024-09-21T00:30:00.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36713 fp=#0x100000037b2' 2024-09-21T00:30:00.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36745 fp=#0x30000001a8f' 2024-09-21T00:30:00.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36777 fp=#0x10000004734' 2024-09-21T00:30:00.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36809 fp=#0x100000047b5' 2024-09-21T00:30:00.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36841 fp=#0x1000000476f' 2024-09-21T00:30:00.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36873 fp=#0x10000002687' 2024-09-21T00:30:00.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36937 fp=#0x1000000360e' 2024-09-21T00:30:00.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37001 fp=#0x1000000354b' 2024-09-21T00:30:00.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37033 fp=#0x10000003647' 2024-09-21T00:30:00.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37065 fp=#0x30000001804' 2024-09-21T00:30:00.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37161 fp=#0x1000000313f' 2024-09-21T00:30:00.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37193 fp=#0x1000000134f' 2024-09-21T00:30:00.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37225 fp=#0x10000003034' 2024-09-21T00:30:00.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37257 fp=#0x10000002fd9' 2024-09-21T00:30:00.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37289 fp=#0x10000002fd5' 2024-09-21T00:30:00.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37321 fp=#0x30000000c07' 2024-09-21T00:30:00.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37353 fp=#0x30000001aed' 2024-09-21T00:30:00.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37385 fp=#0x30000001ae1' 2024-09-21T00:30:00.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37417 fp=#0x30000000c16' 2024-09-21T00:30:00.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37449 fp=#0x30000001ab9' 2024-09-21T00:30:00.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37481 fp=#0x30000001b84' 2024-09-21T00:30:00.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37513 fp=#0x30000000fda' 2024-09-21T00:30:00.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37545 fp=#0x3000000112e' 2024-09-21T00:30:00.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37577 fp=#0x3000000106c' 2024-09-21T00:30:00.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37609 fp=#0x30000001077' 2024-09-21T00:30:00.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37641 fp=#0x10000002934' 2024-09-21T00:30:00.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37705 fp=#0x30000000ca7' 2024-09-21T00:30:00.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37737 fp=#0x30000000ca0' 2024-09-21T00:30:00.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37833 fp=#0x300000012cc' 2024-09-21T00:30:00.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37865 fp=#0x3000000127a' 2024-09-21T00:30:00.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37993 fp=#0x30000000dc7' 2024-09-21T00:30:00.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38025 fp=#0x30000000da9' 2024-09-21T00:30:00.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38057 fp=#0x30000000dae' 2024-09-21T00:30:00.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38089 fp=#0x10000002cdd' 2024-09-21T00:30:00.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38185 fp=#0x10000000734' 2024-09-21T00:30:00.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38217 fp=#0x10000000e51' 2024-09-21T00:30:00.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38281 fp=#0x30000001a83' 2024-09-21T00:30:00.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38313 fp=#0x30000001a60' 2024-09-21T00:30:00.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38345 fp=#0x30000001a47' 2024-09-21T00:30:00.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38377 fp=#0x30000001a5a' 2024-09-21T00:30:00.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38409 fp=#0x100000032ff' 2024-09-21T00:30:00.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38441 fp=#0x100000032f9' 2024-09-21T00:30:00.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38473 fp=#0x30000001d05' 2024-09-21T00:30:00.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38505 fp=#0x100000020f9' 2024-09-21T00:30:00.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38569 fp=#0x10000002105' 2024-09-21T00:30:00.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38601 fp=#0x30000001baa' 2024-09-21T00:30:00.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38633 fp=#0x30000001c2c' 2024-09-21T00:30:00.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38665 fp=#0x30000001c32' 2024-09-21T00:30:00.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38697 fp=#0x30000001c49' 2024-09-21T00:30:00.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38729 fp=#0x100000005e1' 2024-09-21T00:30:00.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38761 fp=#0x100000005dc' 2024-09-21T00:30:00.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38793 fp=#0x30000001c0a' 2024-09-21T00:30:00.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38825 fp=#0x30000001a09' 2024-09-21T00:30:00.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38889 fp=#0x300000019de' 2024-09-21T00:30:00.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38921 fp=#0x30000001d72' 2024-09-21T00:30:00.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38953 fp=#0x30000001d69' 2024-09-21T00:30:00.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39017 fp=#0x30000001c2d' 2024-09-21T00:30:00.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39049 fp=#0x30000001d4b' 2024-09-21T00:30:00.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39081 fp=#0x30000001d34' 2024-09-21T00:30:00.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39273 fp=#0x10000003583' 2024-09-21T00:30:00.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39305 fp=#0x1000000350b' 2024-09-21T00:30:00.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39337 fp=#0x10000003ed4' 2024-09-21T00:30:00.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39369 fp=#0x10000003f04' 2024-09-21T00:30:00.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39401 fp=#0x10000003ee6' 2024-09-21T00:30:00.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39433 fp=#0x10000003fb4' 2024-09-21T00:30:00.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39465 fp=#0x30000001f3a' 2024-09-21T00:30:00.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39497 fp=#0x30000001f47' 2024-09-21T00:30:00.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39529 fp=#0x30000001fa4' 2024-09-21T00:30:00.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39561 fp=#0x30000001f9c' 2024-09-21T00:30:00.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39593 fp=#0x10000002b06' 2024-09-21T00:30:00.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39625 fp=#0x10000002a88' 2024-09-21T00:30:00.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39657 fp=#0x10000002b01' 2024-09-21T00:30:00.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39689 fp=#0x30000000ef4' 2024-09-21T00:30:00.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39721 fp=#0x30000000edb' 2024-09-21T00:30:00.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39785 fp=#0x30000001988' 2024-09-21T00:30:00.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39817 fp=#0x30000000eb5' 2024-09-21T00:30:00.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39849 fp=#0x3000000198d' 2024-09-21T00:30:00.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39881 fp=#0x30000000e61' 2024-09-21T00:30:00.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40009 fp=#0x30000001819' 2024-09-21T00:30:00.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40073 fp=#0x30000001534' 2024-09-21T00:30:00.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40105 fp=#0x30000001557' 2024-09-21T00:30:00.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40137 fp=#0x30000001821' 2024-09-21T00:30:00.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40201 fp=#0x30000000dac' 2024-09-21T00:30:00.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40233 fp=#0x30000000d5e' 2024-09-21T00:30:00.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40265 fp=#0x30000001d1a' 2024-09-21T00:30:00.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40297 fp=#0x10000003187' 2024-09-21T00:30:00.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40329 fp=#0x10000001cf5' 2024-09-21T00:30:00.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40361 fp=#0x10000004e7d' 2024-09-21T00:30:00.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40393 fp=#0x10000004c97' 2024-09-21T00:30:00.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40425 fp=#0x20000001307' 2024-09-21T00:30:00.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40457 fp=#0x10000004d2c' 2024-09-21T00:30:00.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40489 fp=#0x10000004e74' 2024-09-21T00:30:00.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40585 fp=#0x200000005c2' 2024-09-21T00:30:00.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40617 fp=#0x20000000616' 2024-09-21T00:30:00.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40649 fp=#0x200000005fd' 2024-09-21T00:30:00.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40745 fp=#0x200000005b4' 2024-09-21T00:30:00.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40777 fp=#0x30000001457' 2024-09-21T00:30:00.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40809 fp=#0x3000000141f' 2024-09-21T00:30:00.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40841 fp=#0x10000004cba' 2024-09-21T00:30:00.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40873 fp=#0x10000004945' 2024-09-21T00:30:00.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40905 fp=#0x10000004a9b' 2024-09-21T00:30:00.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40937 fp=#0x10000004ab9' 2024-09-21T00:30:00.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41001 fp=#0x10000004cc0' 2024-09-21T00:30:00.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41097 fp=#0x2000000071e' 2024-09-21T00:30:00.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41129 fp=#0x200000001ff' 2024-09-21T00:30:00.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41161 fp=#0x20000000b43' 2024-09-21T00:30:00.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41193 fp=#0x10000000d93' 2024-09-21T00:30:00.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41225 fp=#0x10000000e57' 2024-09-21T00:30:00.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41257 fp=#0x10000000e04' 2024-09-21T00:30:00.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41289 fp=#0x10000000ca9' 2024-09-21T00:30:00.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41321 fp=#0x10000000e67' 2024-09-21T00:30:00.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41385 fp=#0x30000002064' 2024-09-21T00:30:00.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41417 fp=#0x10000000ad7' 2024-09-21T00:30:00.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41481 fp=#0x10000000936' 2024-09-21T00:30:00.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41513 fp=#0x10000000dc0' 2024-09-21T00:30:00.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41545 fp=#0x10000000a2e' 2024-09-21T00:30:00.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41577 fp=#0x3000000191c' 2024-09-21T00:30:00.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41641 fp=#0x30000001841' 2024-09-21T00:30:00.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41737 fp=#0x300000007ed' 2024-09-21T00:30:00.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41769 fp=#0x300000007c9' 2024-09-21T00:30:00.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41833 fp=#0x300000008cd' 2024-09-21T00:30:00.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41865 fp=#0x300000009d8' 2024-09-21T00:30:00.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41897 fp=#0x300000008ba' 2024-09-21T00:30:00.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41929 fp=#0x30000000874' 2024-09-21T00:30:00.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41961 fp=#0x300000007ca' 2024-09-21T00:30:00.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41993 fp=#0x300000007f8' 2024-09-21T00:30:00.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42025 fp=#0x3000000081c' 2024-09-21T00:30:00.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42121 fp=#0x3000000097b' 2024-09-21T00:30:00.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42153 fp=#0x3000000098a' 2024-09-21T00:30:00.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42185 fp=#0x30000002087' 2024-09-21T00:30:00.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42217 fp=#0x30000000c3a' 2024-09-21T00:30:00.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42249 fp=#0x30000001bc6' 2024-09-21T00:30:00.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42281 fp=#0x30000001b95' 2024-09-21T00:30:00.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42313 fp=#0x3000000208e' 2024-09-21T00:30:00.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42345 fp=#0x30000000b4e' 2024-09-21T00:30:00.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42409 fp=#0x30000000b24' 2024-09-21T00:30:00.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42441 fp=#0x30000000afd' 2024-09-21T00:30:00.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42473 fp=#0x10000004125' 2024-09-21T00:30:00.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42505 fp=#0x30000000a74' 2024-09-21T00:30:00.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42569 fp=#0x30000000a75' 2024-09-21T00:30:00.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42697 fp=#0x10000001a49' 2024-09-21T00:30:00.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42729 fp=#0x100000019d3' 2024-09-21T00:30:00.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42761 fp=#0x10000001a4b' 2024-09-21T00:30:00.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42793 fp=#0x10000001a3f' 2024-09-21T00:30:00.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42825 fp=#0x1000000194b' 2024-09-21T00:30:00.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42889 fp=#0x100000018c7' 2024-09-21T00:30:00.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42921 fp=#0x30000000e55' 2024-09-21T00:30:00.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42985 fp=#0x30000000e03' 2024-09-21T00:30:00.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43017 fp=#0x3000000174c' 2024-09-21T00:30:00.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43049 fp=#0x30000001681' 2024-09-21T00:30:00.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43081 fp=#0x300000016bb' 2024-09-21T00:30:00.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43113 fp=#0x300000016da' 2024-09-21T00:30:00.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43145 fp=#0x300000020eb' 2024-09-21T00:30:00.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43209 fp=#0x10000001887' 2024-09-21T00:30:00.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43273 fp=#0x30000000f87' 2024-09-21T00:30:00.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43401 fp=#0x30000000f7b' 2024-09-21T00:30:00.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43433 fp=#0x10000004170' 2024-09-21T00:30:00.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43465 fp=#0x300000023e1' 2024-09-21T00:30:00.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43497 fp=#0x300000023f8' 2024-09-21T00:30:00.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43529 fp=#0x300000023e7' 2024-09-21T00:30:00.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43561 fp=#0x30000002395' 2024-09-21T00:30:00.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43657 fp=#0x30000002178' 2024-09-21T00:30:00.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43689 fp=#0x300000010ed' 2024-09-21T00:30:00.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43721 fp=#0x30000000ffe' 2024-09-21T00:30:00.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43753 fp=#0x30000000fa4' 2024-09-21T00:30:00.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43817 fp=#0x100000039fc' 2024-09-21T00:30:00.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43849 fp=#0x100000039bb' 2024-09-21T00:30:00.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43881 fp=#0x100000039a2' 2024-09-21T00:30:00.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43913 fp=#0x100000039a6' 2024-09-21T00:30:00.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43945 fp=#0x3000000135b' 2024-09-21T00:30:00.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31754 fp=#0x3000000110f' 2024-09-21T00:30:00.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31818 fp=#0x30000001c5c' 2024-09-21T00:30:00.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31946 fp=#0x10000002044' 2024-09-21T00:30:00.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34698 fp=#0x30000002225' 2024-09-21T00:30:00.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34762 fp=#0x10000000b59' 2024-09-21T00:30:00.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34858 fp=#0x10000002804' 2024-09-21T00:30:00.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34922 fp=#0x100000026f7' 2024-09-21T00:30:00.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34986 fp=#0x300000020a0' 2024-09-21T00:30:00.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35018 fp=#0x10000003849' 2024-09-21T00:30:00.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35050 fp=#0x10000003880' 2024-09-21T00:30:00.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35082 fp=#0x1000000384c' 2024-09-21T00:30:00.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35114 fp=#0x10000003b22' 2024-09-21T00:30:00.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35146 fp=#0x10000003a7e' 2024-09-21T00:30:00.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35178 fp=#0x1000000394d' 2024-09-21T00:30:00.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35210 fp=#0x10000004210' 2024-09-21T00:30:00.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35242 fp=#0x10000004173' 2024-09-21T00:30:00.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35274 fp=#0x10000003ac1' 2024-09-21T00:30:00.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35306 fp=#0x100000039a7' 2024-09-21T00:30:00.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35338 fp=#0x100000038e5' 2024-09-21T00:30:00.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35402 fp=#0x10000003883' 2024-09-21T00:30:00.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35434 fp=#0x100000039c1' 2024-09-21T00:30:00.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35466 fp=#0x3000000119b' 2024-09-21T00:30:00.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35498 fp=#0x30000001172' 2024-09-21T00:30:00.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35530 fp=#0x3000000113b' 2024-09-21T00:30:00.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35562 fp=#0x30000001fd7' 2024-09-21T00:30:00.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35594 fp=#0x30000002036' 2024-09-21T00:30:00.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35626 fp=#0x3000000202a' 2024-09-21T00:30:00.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35658 fp=#0x3000000202e' 2024-09-21T00:30:00.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35786 fp=#0x30000001d97' 2024-09-21T00:30:00.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35818 fp=#0x10000003477' 2024-09-21T00:30:00.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35850 fp=#0x300000019b7' 2024-09-21T00:30:00.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35882 fp=#0x100000033e2' 2024-09-21T00:30:00.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35914 fp=#0x1000000337f' 2024-09-21T00:30:00.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35946 fp=#0x300000011cb' 2024-09-21T00:30:00.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35978 fp=#0x300000011bd' 2024-09-21T00:30:00.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36074 fp=#0x10000002d6a' 2024-09-21T00:30:00.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36138 fp=#0x10000002091' 2024-09-21T00:30:00.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36170 fp=#0x1000000206d' 2024-09-21T00:30:00.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36202 fp=#0x10000002097' 2024-09-21T00:30:00.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36234 fp=#0x10000001f01' 2024-09-21T00:30:00.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36266 fp=#0x3000000160c' 2024-09-21T00:30:00.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36298 fp=#0x300000015b8' 2024-09-21T00:30:00.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36362 fp=#0x300000015ad' 2024-09-21T00:30:00.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36458 fp=#0x10000001e69' 2024-09-21T00:30:00.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36490 fp=#0x10000001d55' 2024-09-21T00:30:00.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36522 fp=#0x10000001e2d' 2024-09-21T00:30:00.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36554 fp=#0x10000004d4b' 2024-09-21T00:30:00.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36586 fp=#0x100000037d2' 2024-09-21T00:30:00.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36618 fp=#0x100000037f0' 2024-09-21T00:30:00.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36650 fp=#0x10000004474' 2024-09-21T00:30:00.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36682 fp=#0x100000044b6' 2024-09-21T00:30:00.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36714 fp=#0x10000004453' 2024-09-21T00:30:00.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36746 fp=#0x30000001a98' 2024-09-21T00:30:00.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36778 fp=#0x30000001aaa' 2024-09-21T00:30:00.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36810 fp=#0x100000047a8' 2024-09-21T00:30:00.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36842 fp=#0x100000047b3' 2024-09-21T00:30:00.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36906 fp=#0x100000025f3' 2024-09-21T00:30:00.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36938 fp=#0x1000000353c' 2024-09-21T00:30:00.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36970 fp=#0x100000035cd' 2024-09-21T00:30:00.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37002 fp=#0x10000003585' 2024-09-21T00:30:00.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37034 fp=#0x10000002547' 2024-09-21T00:30:00.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37162 fp=#0x10000001498' 2024-09-21T00:30:00.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37290 fp=#0x10000003130' 2024-09-21T00:30:00.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37322 fp=#0x30000000c28' 2024-09-21T00:30:00.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37354 fp=#0x30000001ac5' 2024-09-21T00:30:00.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37386 fp=#0x30000001ac9' 2024-09-21T00:30:00.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37418 fp=#0x30000001acf' 2024-09-21T00:30:00.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37450 fp=#0x30000001b00' 2024-09-21T00:30:00.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37482 fp=#0x300000010b5' 2024-09-21T00:30:00.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37514 fp=#0x300000010c4' 2024-09-21T00:30:00.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37546 fp=#0x30000001b6e' 2024-09-21T00:30:00.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37578 fp=#0x30000001000' 2024-09-21T00:30:00.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37610 fp=#0x30000001093' 2024-09-21T00:30:00.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37642 fp=#0x30000000c96' 2024-09-21T00:30:00.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37674 fp=#0x30000000c84' 2024-09-21T00:30:00.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37770 fp=#0x30000000c4d' 2024-09-21T00:30:00.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37802 fp=#0x30000000c47' 2024-09-21T00:30:00.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37834 fp=#0x300000012c5' 2024-09-21T00:30:00.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37866 fp=#0x30000001280' 2024-09-21T00:30:00.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37930 fp=#0x30000002285' 2024-09-21T00:30:00.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37962 fp=#0x30000000cc9' 2024-09-21T00:30:00.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38090 fp=#0x1000000071d' 2024-09-21T00:30:00.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38122 fp=#0x10000000753' 2024-09-21T00:30:00.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38154 fp=#0x10000000eba' 2024-09-21T00:30:00.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38186 fp=#0x10000000ee2' 2024-09-21T00:30:00.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38250 fp=#0x10000000e9a' 2024-09-21T00:30:00.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38282 fp=#0x30000001a67' 2024-09-21T00:30:00.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38314 fp=#0x30000001a44' 2024-09-21T00:30:00.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38346 fp=#0x30000001a5e' 2024-09-21T00:30:00.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38378 fp=#0x30000001a43' 2024-09-21T00:30:00.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38442 fp=#0x30000001cea' 2024-09-21T00:30:00.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38474 fp=#0x100000032d7' 2024-09-21T00:30:00.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38506 fp=#0x10000002175' 2024-09-21T00:30:00.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38538 fp=#0x100000020dc' 2024-09-21T00:30:00.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38570 fp=#0x100000020fc' 2024-09-21T00:30:00.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38602 fp=#0x100000005e9' 2024-09-21T00:30:00.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38634 fp=#0x30000001c2b' 2024-09-21T00:30:00.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38666 fp=#0x10000000511' 2024-09-21T00:30:00.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38698 fp=#0x30000001bc5' 2024-09-21T00:30:00.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38730 fp=#0x10000000ab2' 2024-09-21T00:30:00.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38762 fp=#0x30000001bce' 2024-09-21T00:30:00.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38794 fp=#0x10000002962' 2024-09-21T00:30:00.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38826 fp=#0x30000001a07' 2024-09-21T00:30:00.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38858 fp=#0x300000019e6' 2024-09-21T00:30:00.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38890 fp=#0x1000000295a' 2024-09-21T00:30:00.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38922 fp=#0x30000001d66' 2024-09-21T00:30:00.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39050 fp=#0x30000001c42' 2024-09-21T00:30:00.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39242 fp=#0x100000034b6' 2024-09-21T00:30:00.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39306 fp=#0x100000034ea' 2024-09-21T00:30:00.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39338 fp=#0x10000003f09' 2024-09-21T00:30:00.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39370 fp=#0x10000003fd1' 2024-09-21T00:30:00.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39402 fp=#0x10000003f5a' 2024-09-21T00:30:00.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39434 fp=#0x10000003fce' 2024-09-21T00:30:00.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39466 fp=#0x30000001f16' 2024-09-21T00:30:00.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39498 fp=#0x30000001f6c' 2024-09-21T00:30:00.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39530 fp=#0x30000001f35' 2024-09-21T00:30:00.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39562 fp=#0x30000001f34' 2024-09-21T00:30:00.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39722 fp=#0x3000000198c' 2024-09-21T00:30:00.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39754 fp=#0x30000000ea8' 2024-09-21T00:30:00.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39786 fp=#0x30000000eb9' 2024-09-21T00:30:00.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39818 fp=#0x300000022ae' 2024-09-21T00:30:00.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39850 fp=#0x30000001994' 2024-09-21T00:30:00.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39882 fp=#0x300000019a9' 2024-09-21T00:30:00.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39978 fp=#0x3000000100d' 2024-09-21T00:30:00.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40010 fp=#0x30000001517' 2024-09-21T00:30:00.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40042 fp=#0x3000000182f' 2024-09-21T00:30:00.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40074 fp=#0x3000000153c' 2024-09-21T00:30:00.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40106 fp=#0x3000000181c' 2024-09-21T00:30:00.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40138 fp=#0x30000001538' 2024-09-21T00:30:00.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40170 fp=#0x30000001527' 2024-09-21T00:30:00.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40202 fp=#0x30000000d64' 2024-09-21T00:30:00.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40234 fp=#0x300000009e6' 2024-09-21T00:30:00.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40298 fp=#0x10000001d4a' 2024-09-21T00:30:00.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40362 fp=#0x10000004c9b' 2024-09-21T00:30:00.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40394 fp=#0x10000004cc5' 2024-09-21T00:30:00.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40426 fp=#0x10000004de3' 2024-09-21T00:30:00.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40458 fp=#0x10000004cb7' 2024-09-21T00:30:00.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40490 fp=#0x3000000231d' 2024-09-21T00:30:00.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40554 fp=#0x200000005a6' 2024-09-21T00:30:00.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40586 fp=#0x10000000f7d' 2024-09-21T00:30:00.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40682 fp=#0x10000000c9b' 2024-09-21T00:30:00.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40714 fp=#0x10000000cb8' 2024-09-21T00:30:00.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40746 fp=#0x10000000f35' 2024-09-21T00:30:00.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40778 fp=#0x30000001456' 2024-09-21T00:30:00.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40810 fp=#0x30000001443' 2024-09-21T00:30:00.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40874 fp=#0x10000004aeb' 2024-09-21T00:30:00.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40938 fp=#0x10000004b0c' 2024-09-21T00:30:00.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41034 fp=#0x10000003658' 2024-09-21T00:30:00.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41098 fp=#0x10000000d14' 2024-09-21T00:30:00.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41130 fp=#0x20000000218' 2024-09-21T00:30:00.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41162 fp=#0x20000000b48' 2024-09-21T00:30:00.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41194 fp=#0x2000000020b' 2024-09-21T00:30:00.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41226 fp=#0x10000000e25' 2024-09-21T00:30:00.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41258 fp=#0x20000000ad8' 2024-09-21T00:30:00.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41290 fp=#0x10000000def' 2024-09-21T00:30:00.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41322 fp=#0x20000000c71' 2024-09-21T00:30:00.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41386 fp=#0x100000009f4' 2024-09-21T00:30:00.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41418 fp=#0x10000000b4a' 2024-09-21T00:30:00.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41450 fp=#0x10000000be4' 2024-09-21T00:30:00.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41482 fp=#0x30000002074' 2024-09-21T00:30:00.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41546 fp=#0x10000000dd0' 2024-09-21T00:30:00.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41578 fp=#0x30000001850' 2024-09-21T00:30:00.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41642 fp=#0x30000001904' 2024-09-21T00:30:00.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41674 fp=#0x3000000090d' 2024-09-21T00:30:00.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41706 fp=#0x300000007e7' 2024-09-21T00:30:00.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41802 fp=#0x300000008ab' 2024-09-21T00:30:00.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41834 fp=#0x300000009a5' 2024-09-21T00:30:00.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41866 fp=#0x30000000995' 2024-09-21T00:30:00.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41898 fp=#0x3000000099f' 2024-09-21T00:30:00.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41930 fp=#0x300000009be' 2024-09-21T00:30:00.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41962 fp=#0x300000009a8' 2024-09-21T00:30:00.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41994 fp=#0x300000009b6' 2024-09-21T00:30:00.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42026 fp=#0x30000001d49' 2024-09-21T00:30:00.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42058 fp=#0x30000000966' 2024-09-21T00:30:00.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42090 fp=#0x300000007f2' 2024-09-21T00:30:00.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42122 fp=#0x30000000842' 2024-09-21T00:30:00.630 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42154 fp=#0x300000007bc' 2024-09-21T00:30:00.630 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42186 fp=#0x30000000ba0' 2024-09-21T00:30:00.630 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42218 fp=#0x30000000b7d' 2024-09-21T00:30:00.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42250 fp=#0x30000002084' 2024-09-21T00:30:00.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42282 fp=#0x30000001ba2' 2024-09-21T00:30:00.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42314 fp=#0x30000000bba' 2024-09-21T00:30:00.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42346 fp=#0x30000000af6' 2024-09-21T00:30:00.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42410 fp=#0x30000000b04' 2024-09-21T00:30:00.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42442 fp=#0x30000000b34' 2024-09-21T00:30:00.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42474 fp=#0x300000020e9' 2024-09-21T00:30:00.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42506 fp=#0x30000001b52' 2024-09-21T00:30:00.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42538 fp=#0x30000000a4c' 2024-09-21T00:30:00.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42570 fp=#0x30000000a70' 2024-09-21T00:30:00.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42666 fp=#0x10000001a24' 2024-09-21T00:30:00.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42698 fp=#0x100000018c9' 2024-09-21T00:30:00.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42730 fp=#0x10000001a45' 2024-09-21T00:30:00.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42890 fp=#0x300000006a4' 2024-09-21T00:30:00.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42922 fp=#0x30000000dfe' 2024-09-21T00:30:00.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42986 fp=#0x30000000dee' 2024-09-21T00:30:00.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43018 fp=#0x30000001689' 2024-09-21T00:30:00.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43050 fp=#0x30000001740' 2024-09-21T00:30:00.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43114 fp=#0x3000000172b' 2024-09-21T00:30:00.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43178 fp=#0x10000001864' 2024-09-21T00:30:00.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43274 fp=#0x3000000197b' 2024-09-21T00:30:00.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43338 fp=#0x30000000f4a' 2024-09-21T00:30:00.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43402 fp=#0x30000000f30' 2024-09-21T00:30:00.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43466 fp=#0x30000002376' 2024-09-21T00:30:00.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43498 fp=#0x300000023cb' 2024-09-21T00:30:00.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43530 fp=#0x300000023ae' 2024-09-21T00:30:00.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43562 fp=#0x300000023ad' 2024-09-21T00:30:00.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43594 fp=#0x30000000f9e' 2024-09-21T00:30:00.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43690 fp=#0x3000000100e' 2024-09-21T00:30:00.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43722 fp=#0x300000010eb' 2024-09-21T00:30:00.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43754 fp=#0x3000000219a' 2024-09-21T00:30:00.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43850 fp=#0x10000003a18' 2024-09-21T00:30:00.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43882 fp=#0x100000039a9' 2024-09-21T00:30:00.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43914 fp=#0x100000039be' 2024-09-21T00:30:00.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43946 fp=#0x30000001363' 2024-09-21T00:30:00.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31755 fp=#0x30000001fcc' 2024-09-21T00:30:00.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31819 fp=#0x1000000039b' 2024-09-21T00:30:00.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31947 fp=#0x100000016e7' 2024-09-21T00:30:00.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34699 fp=#0x10000000c73' 2024-09-21T00:30:00.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34731 fp=#0x3000000224c' 2024-09-21T00:30:00.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34859 fp=#0x10000002820' 2024-09-21T00:30:00.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34891 fp=#0x10000002815' 2024-09-21T00:30:00.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34923 fp=#0x100000026e8' 2024-09-21T00:30:00.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34955 fp=#0x10000002715' 2024-09-21T00:30:00.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34987 fp=#0x30000002062' 2024-09-21T00:30:00.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35019 fp=#0x300000020b9' 2024-09-21T00:30:00.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35051 fp=#0x10000003888' 2024-09-21T00:30:00.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35083 fp=#0x100000032b1' 2024-09-21T00:30:00.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35115 fp=#0x10000004226' 2024-09-21T00:30:00.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35147 fp=#0x100000039d1' 2024-09-21T00:30:00.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35179 fp=#0x100000039d4' 2024-09-21T00:30:00.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35211 fp=#0x1000000398c' 2024-09-21T00:30:00.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35243 fp=#0x1000000399d' 2024-09-21T00:30:00.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35275 fp=#0x10000004190' 2024-09-21T00:30:00.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35307 fp=#0x10000003b06' 2024-09-21T00:30:00.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35339 fp=#0x10000004222' 2024-09-21T00:30:00.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35371 fp=#0x10000003a1a' 2024-09-21T00:30:00.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35403 fp=#0x1000000387f' 2024-09-21T00:30:00.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35435 fp=#0x100000038db' 2024-09-21T00:30:00.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35467 fp=#0x3000000112f' 2024-09-21T00:30:00.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35499 fp=#0x30000001118' 2024-09-21T00:30:00.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35531 fp=#0x30000001123' 2024-09-21T00:30:00.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35595 fp=#0x30000001fd1' 2024-09-21T00:30:00.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35659 fp=#0x30000002001' 2024-09-21T00:30:00.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35691 fp=#0x3000000204d' 2024-09-21T00:30:00.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35851 fp=#0x10000003470' 2024-09-21T00:30:00.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35883 fp=#0x10000003460' 2024-09-21T00:30:00.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35915 fp=#0x300000011c5' 2024-09-21T00:30:00.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35947 fp=#0x300000011e9' 2024-09-21T00:30:00.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35979 fp=#0x300000011ee' 2024-09-21T00:30:00.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36011 fp=#0x10000003b54' 2024-09-21T00:30:00.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36075 fp=#0x10000002d67' 2024-09-21T00:30:00.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36139 fp=#0x10000002082' 2024-09-21T00:30:00.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36171 fp=#0x10000002028' 2024-09-21T00:30:00.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36203 fp=#0x10000001f8f' 2024-09-21T00:30:00.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36235 fp=#0x10000002041' 2024-09-21T00:30:00.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36459 fp=#0x10000001e09' 2024-09-21T00:30:00.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36491 fp=#0x10000001d64' 2024-09-21T00:30:00.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36523 fp=#0x10000001df5' 2024-09-21T00:30:00.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36555 fp=#0x10000001d3d' 2024-09-21T00:30:00.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36587 fp=#0x10000003755' 2024-09-21T00:30:00.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36619 fp=#0x1000000449c' 2024-09-21T00:30:00.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36651 fp=#0x10000004482' 2024-09-21T00:30:00.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36715 fp=#0x10000004730' 2024-09-21T00:30:00.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36747 fp=#0x100000046fd' 2024-09-21T00:30:00.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36779 fp=#0x10000004703' 2024-09-21T00:30:00.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36811 fp=#0x10000004739' 2024-09-21T00:30:00.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36843 fp=#0x100000047cb' 2024-09-21T00:30:00.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36875 fp=#0x1000000267a' 2024-09-21T00:30:00.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36907 fp=#0x100000035ae' 2024-09-21T00:30:00.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36939 fp=#0x10000002560' 2024-09-21T00:30:00.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36971 fp=#0x10000003561' 2024-09-21T00:30:00.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37003 fp=#0x10000003536' 2024-09-21T00:30:00.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37163 fp=#0x100000014a5' 2024-09-21T00:30:00.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37227 fp=#0x1000000319d' 2024-09-21T00:30:00.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37291 fp=#0x1000000305c' 2024-09-21T00:30:00.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37323 fp=#0x30000000c2a' 2024-09-21T00:30:00.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37355 fp=#0x30000000c02' 2024-09-21T00:30:00.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37387 fp=#0x30000001b23' 2024-09-21T00:30:00.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37419 fp=#0x30000000bf4' 2024-09-21T00:30:00.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37451 fp=#0x30000001ace' 2024-09-21T00:30:00.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37483 fp=#0x30000001008' 2024-09-21T00:30:00.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37515 fp=#0x3000000108e' 2024-09-21T00:30:00.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37547 fp=#0x30000000ff4' 2024-09-21T00:30:00.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37611 fp=#0x30000001b6b' 2024-09-21T00:30:00.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37643 fp=#0x30000000c88' 2024-09-21T00:30:00.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37675 fp=#0x30000000c83' 2024-09-21T00:30:00.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37707 fp=#0x30000000c6f' 2024-09-21T00:30:00.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37739 fp=#0x30000000c43' 2024-09-21T00:30:00.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37803 fp=#0x30000000c7b' 2024-09-21T00:30:00.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37835 fp=#0x3000000126c' 2024-09-21T00:30:00.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37899 fp=#0x30000001272' 2024-09-21T00:30:00.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37963 fp=#0x30000000dbe' 2024-09-21T00:30:00.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37995 fp=#0x30000000da3' 2024-09-21T00:30:00.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38027 fp=#0x30000001975' 2024-09-21T00:30:00.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38059 fp=#0x30000000da1' 2024-09-21T00:30:00.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38091 fp=#0x10000000773' 2024-09-21T00:30:00.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38155 fp=#0x1000000071f' 2024-09-21T00:30:00.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38187 fp=#0x10000000e29' 2024-09-21T00:30:00.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38219 fp=#0x10000000722' 2024-09-21T00:30:00.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38251 fp=#0x10000000ec7' 2024-09-21T00:30:00.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38283 fp=#0x1000000248e' 2024-09-21T00:30:00.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38315 fp=#0x30000001a85' 2024-09-21T00:30:00.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38347 fp=#0x30000001a40' 2024-09-21T00:30:00.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38411 fp=#0x10000003325' 2024-09-21T00:30:00.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38475 fp=#0x1000000326e' 2024-09-21T00:30:00.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38507 fp=#0x1000000218c' 2024-09-21T00:30:00.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38539 fp=#0x10000002191' 2024-09-21T00:30:00.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38571 fp=#0x10000002164' 2024-09-21T00:30:00.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38635 fp=#0x10000000515' 2024-09-21T00:30:00.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38667 fp=#0x30000001c2e' 2024-09-21T00:30:00.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38699 fp=#0x100000005e5' 2024-09-21T00:30:00.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38763 fp=#0x30000001c1d' 2024-09-21T00:30:00.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38795 fp=#0x300000019d5' 2024-09-21T00:30:00.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38827 fp=#0x10000002952' 2024-09-21T00:30:00.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38891 fp=#0x10000002924' 2024-09-21T00:30:00.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38923 fp=#0x30000001c69' 2024-09-21T00:30:00.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38955 fp=#0x30000001c84' 2024-09-21T00:30:00.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38987 fp=#0x30000001c58' 2024-09-21T00:30:00.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39019 fp=#0x30000001d5a' 2024-09-21T00:30:00.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39051 fp=#0x30000001c48' 2024-09-21T00:30:00.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39083 fp=#0x30000001d25' 2024-09-21T00:30:00.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39211 fp=#0x10000003540' 2024-09-21T00:30:00.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39339 fp=#0x10000003fa6' 2024-09-21T00:30:00.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39371 fp=#0x30000000824' 2024-09-21T00:30:00.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39403 fp=#0x10000003ed6' 2024-09-21T00:30:00.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39435 fp=#0x10000003f1d' 2024-09-21T00:30:00.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39467 fp=#0x30000001fb6' 2024-09-21T00:30:00.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39499 fp=#0x30000001f9d' 2024-09-21T00:30:00.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39531 fp=#0x30000001f7a' 2024-09-21T00:30:00.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39563 fp=#0x30000001f8d' 2024-09-21T00:30:00.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39595 fp=#0x10000002b38' 2024-09-21T00:30:00.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39627 fp=#0x10000002aa0' 2024-09-21T00:30:00.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39691 fp=#0x30000000ef7' 2024-09-21T00:30:00.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39723 fp=#0x30000000e6b' 2024-09-21T00:30:00.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39755 fp=#0x30000000eac' 2024-09-21T00:30:00.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39819 fp=#0x300000014fb' 2024-09-21T00:30:00.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39851 fp=#0x3000000198e' 2024-09-21T00:30:00.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39883 fp=#0x30000001984' 2024-09-21T00:30:00.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39979 fp=#0x30000001117' 2024-09-21T00:30:00.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40011 fp=#0x3000000185e' 2024-09-21T00:30:00.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40043 fp=#0x30000001838' 2024-09-21T00:30:00.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40075 fp=#0x3000000184a' 2024-09-21T00:30:00.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40107 fp=#0x3000000185c' 2024-09-21T00:30:00.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40171 fp=#0x3000000153e' 2024-09-21T00:30:00.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40203 fp=#0x10000001d6b' 2024-09-21T00:30:00.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40235 fp=#0x30000000dad' 2024-09-21T00:30:00.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40267 fp=#0x30000000d96' 2024-09-21T00:30:00.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40299 fp=#0x30000000d8d' 2024-09-21T00:30:00.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40331 fp=#0x10000001cf8' 2024-09-21T00:30:00.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40363 fp=#0x10000004e7b' 2024-09-21T00:30:00.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40395 fp=#0x10000004ca1' 2024-09-21T00:30:00.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40427 fp=#0x10000004e79' 2024-09-21T00:30:00.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40491 fp=#0x10000004cbd' 2024-09-21T00:30:00.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40555 fp=#0x20000000546' 2024-09-21T00:30:00.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40587 fp=#0x200000006d6' 2024-09-21T00:30:00.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40619 fp=#0x2000000058e' 2024-09-21T00:30:00.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40651 fp=#0x2000000044c' 2024-09-21T00:30:00.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40715 fp=#0x10000000f55' 2024-09-21T00:30:00.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40779 fp=#0x3000000145f' 2024-09-21T00:30:00.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40811 fp=#0x30000001422' 2024-09-21T00:30:00.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40843 fp=#0x10000004c71' 2024-09-21T00:30:00.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40875 fp=#0x10000004aef' 2024-09-21T00:30:00.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40907 fp=#0x10000004ad9' 2024-09-21T00:30:00.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40939 fp=#0x10000004aa4' 2024-09-21T00:30:00.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40971 fp=#0x10000004a9f' 2024-09-21T00:30:00.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41003 fp=#0x10000004cdb' 2024-09-21T00:30:00.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41035 fp=#0x2000000070f' 2024-09-21T00:30:00.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41067 fp=#0x2000000071f' 2024-09-21T00:30:00.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41099 fp=#0x1000000102f' 2024-09-21T00:30:00.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41131 fp=#0x20000000c21' 2024-09-21T00:30:00.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41163 fp=#0x10000000f79' 2024-09-21T00:30:00.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41195 fp=#0x20000000bee' 2024-09-21T00:30:00.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41227 fp=#0x10000000d8e' 2024-09-21T00:30:00.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41259 fp=#0x10000000de9' 2024-09-21T00:30:00.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41291 fp=#0x20000000471' 2024-09-21T00:30:00.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41323 fp=#0x20000000c19' 2024-09-21T00:30:00.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41387 fp=#0x1000000094a' 2024-09-21T00:30:00.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41419 fp=#0x10000000e18' 2024-09-21T00:30:00.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41451 fp=#0x10000000b3b' 2024-09-21T00:30:00.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41483 fp=#0x10000000c39' 2024-09-21T00:30:00.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41515 fp=#0x10000000c48' 2024-09-21T00:30:00.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41547 fp=#0x10000000bdf' 2024-09-21T00:30:00.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41611 fp=#0x30000001915' 2024-09-21T00:30:00.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41803 fp=#0x300000009cf' 2024-09-21T00:30:00.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41835 fp=#0x30000000943' 2024-09-21T00:30:00.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41867 fp=#0x3000000096c' 2024-09-21T00:30:00.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41899 fp=#0x300000009a7' 2024-09-21T00:30:00.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41931 fp=#0x300000008b8' 2024-09-21T00:30:00.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41963 fp=#0x3000000099c' 2024-09-21T00:30:00.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41995 fp=#0x300000007b4' 2024-09-21T00:30:00.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42027 fp=#0x30000000854' 2024-09-21T00:30:00.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42059 fp=#0x30000000945' 2024-09-21T00:30:00.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42091 fp=#0x30000000996' 2024-09-21T00:30:00.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42123 fp=#0x30000000998' 2024-09-21T00:30:00.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42155 fp=#0x3000000096e' 2024-09-21T00:30:00.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42187 fp=#0x30000001bbb' 2024-09-21T00:30:00.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42219 fp=#0x30000002094' 2024-09-21T00:30:00.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42251 fp=#0x30000001ba5' 2024-09-21T00:30:00.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42283 fp=#0x30000001b90' 2024-09-21T00:30:00.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42315 fp=#0x30000002096' 2024-09-21T00:30:00.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42347 fp=#0x30000000b38' 2024-09-21T00:30:00.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42379 fp=#0x30000000ad9' 2024-09-21T00:30:00.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42411 fp=#0x30000000b48' 2024-09-21T00:30:00.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42443 fp=#0x30000000b23' 2024-09-21T00:30:00.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42507 fp=#0x30000000a5e' 2024-09-21T00:30:00.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42667 fp=#0x100000018e3' 2024-09-21T00:30:00.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42731 fp=#0x10000003916' 2024-09-21T00:30:00.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42763 fp=#0x100000019d0' 2024-09-21T00:30:00.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42827 fp=#0x100000018dd' 2024-09-21T00:30:00.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42923 fp=#0x30000000e50' 2024-09-21T00:30:00.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43019 fp=#0x300000016be' 2024-09-21T00:30:00.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43051 fp=#0x300000016e7' 2024-09-21T00:30:00.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43083 fp=#0x30000001722' 2024-09-21T00:30:00.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43115 fp=#0x3000000171c' 2024-09-21T00:30:00.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43147 fp=#0x100000018b3' 2024-09-21T00:30:00.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43243 fp=#0x1000000186b' 2024-09-21T00:30:00.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43275 fp=#0x30000000f89' 2024-09-21T00:30:00.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43371 fp=#0x30000000f43' 2024-09-21T00:30:00.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43403 fp=#0x30000000f36' 2024-09-21T00:30:00.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43435 fp=#0x3000000237e' 2024-09-21T00:30:00.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43467 fp=#0x300000023d8' 2024-09-21T00:30:00.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43499 fp=#0x300000023c3' 2024-09-21T00:30:00.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43531 fp=#0x300000023a9' 2024-09-21T00:30:00.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43563 fp=#0x300000023c9' 2024-09-21T00:30:00.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43595 fp=#0x300000010fb' 2024-09-21T00:30:00.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43627 fp=#0x30000002159' 2024-09-21T00:30:00.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43659 fp=#0x3000000215b' 2024-09-21T00:30:00.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43755 fp=#0x30000002173' 2024-09-21T00:30:00.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43787 fp=#0x100000039eb' 2024-09-21T00:30:00.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43851 fp=#0x10000003a39' 2024-09-21T00:30:00.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43883 fp=#0x1000000298e' 2024-09-21T00:30:00.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43915 fp=#0x10000003976' 2024-09-21T00:30:00.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43947 fp=#0x300000013a3' 2024-09-21T00:30:00.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31756 fp=#0x30000001d96' 2024-09-21T00:30:00.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31820 fp=#0x1000000110e' 2024-09-21T00:30:00.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31948 fp=#0x30000000ef9' 2024-09-21T00:30:00.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34700 fp=#0x10000000b75' 2024-09-21T00:30:00.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34732 fp=#0x10000000b76' 2024-09-21T00:30:00.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34764 fp=#0x10000000b7c' 2024-09-21T00:30:00.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34860 fp=#0x100000026fd' 2024-09-21T00:30:00.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34988 fp=#0x10000003826' 2024-09-21T00:30:00.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35020 fp=#0x300000020b1' 2024-09-21T00:30:00.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35052 fp=#0x100000032f7' 2024-09-21T00:30:00.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35084 fp=#0x100000032c0' 2024-09-21T00:30:00.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35116 fp=#0x10000004207' 2024-09-21T00:30:00.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35148 fp=#0x10000003ac5' 2024-09-21T00:30:00.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35180 fp=#0x100000039e2' 2024-09-21T00:30:00.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35212 fp=#0x1000000422e' 2024-09-21T00:30:00.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35244 fp=#0x100000041ba' 2024-09-21T00:30:00.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35276 fp=#0x10000003953' 2024-09-21T00:30:00.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35308 fp=#0x100000038b7' 2024-09-21T00:30:00.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35340 fp=#0x10000003a05' 2024-09-21T00:30:00.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35372 fp=#0x10000003a37' 2024-09-21T00:30:00.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35404 fp=#0x10000003893' 2024-09-21T00:30:00.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35436 fp=#0x100000038c6' 2024-09-21T00:30:00.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35468 fp=#0x30000001197' 2024-09-21T00:30:00.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35500 fp=#0x3000000112c' 2024-09-21T00:30:00.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35532 fp=#0x3000000117c' 2024-09-21T00:30:00.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35564 fp=#0x30000001ff0' 2024-09-21T00:30:00.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35596 fp=#0x30000002044' 2024-09-21T00:30:00.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35628 fp=#0x3000000202f' 2024-09-21T00:30:00.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35660 fp=#0x30000002051' 2024-09-21T00:30:00.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35820 fp=#0x10000003417' 2024-09-21T00:30:00.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35884 fp=#0x10000003415' 2024-09-21T00:30:00.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35916 fp=#0x300000011f3' 2024-09-21T00:30:00.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35948 fp=#0x300000011cc' 2024-09-21T00:30:00.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35980 fp=#0x300000011f5' 2024-09-21T00:30:00.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36076 fp=#0x10000002d7f' 2024-09-21T00:30:00.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36108 fp=#0x10000002ca6' 2024-09-21T00:30:00.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36140 fp=#0x10000001f59' 2024-09-21T00:30:00.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36172 fp=#0x10000001f86' 2024-09-21T00:30:00.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36204 fp=#0x10000001f16' 2024-09-21T00:30:00.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36236 fp=#0x10000001f65' 2024-09-21T00:30:00.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36268 fp=#0x3000000160e' 2024-09-21T00:30:00.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36428 fp=#0x10000001d51' 2024-09-21T00:30:00.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36460 fp=#0x10000001e5b' 2024-09-21T00:30:00.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36492 fp=#0x10000004d50' 2024-09-21T00:30:00.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36524 fp=#0x10000001e73' 2024-09-21T00:30:00.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36556 fp=#0x10000001e6b' 2024-09-21T00:30:00.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36588 fp=#0x10000003839' 2024-09-21T00:30:00.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36620 fp=#0x10000004431' 2024-09-21T00:30:00.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36652 fp=#0x10000004448' 2024-09-21T00:30:00.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36684 fp=#0x10000003763' 2024-09-21T00:30:00.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36716 fp=#0x10000004708' 2024-09-21T00:30:00.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36748 fp=#0x30000001a84' 2024-09-21T00:30:00.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36780 fp=#0x30000001aa2' 2024-09-21T00:30:00.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36812 fp=#0x10000004793' 2024-09-21T00:30:00.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36844 fp=#0x30000001a93' 2024-09-21T00:30:00.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36876 fp=#0x10000003680' 2024-09-21T00:30:00.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36908 fp=#0x100000035b1' 2024-09-21T00:30:00.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36940 fp=#0x100000035a0' 2024-09-21T00:30:00.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36972 fp=#0x10000002577' 2024-09-21T00:30:00.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37004 fp=#0x100000036bd' 2024-09-21T00:30:00.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37196 fp=#0x10000003155' 2024-09-21T00:30:00.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37228 fp=#0x10000003174' 2024-09-21T00:30:00.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37292 fp=#0x100000030c2' 2024-09-21T00:30:00.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37324 fp=#0x30000000c10' 2024-09-21T00:30:00.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37356 fp=#0x30000001ab8' 2024-09-21T00:30:00.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37388 fp=#0x30000001b19' 2024-09-21T00:30:00.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37420 fp=#0x30000000bf3' 2024-09-21T00:30:00.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37452 fp=#0x30000001ade' 2024-09-21T00:30:00.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37484 fp=#0x30000001b6c' 2024-09-21T00:30:00.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37548 fp=#0x30000001b82' 2024-09-21T00:30:00.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37580 fp=#0x300000010d2' 2024-09-21T00:30:00.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37612 fp=#0x30000001b86' 2024-09-21T00:30:00.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37740 fp=#0x1000000292a' 2024-09-21T00:30:00.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37804 fp=#0x30000000c5b' 2024-09-21T00:30:00.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37836 fp=#0x300000012c3' 2024-09-21T00:30:00.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37868 fp=#0x300000012d4' 2024-09-21T00:30:00.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37932 fp=#0x3000000224e' 2024-09-21T00:30:00.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37996 fp=#0x30000000cc6' 2024-09-21T00:30:00.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38060 fp=#0x30000000ce0' 2024-09-21T00:30:00.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38092 fp=#0x1000000072b' 2024-09-21T00:30:00.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38156 fp=#0x10000000e0f' 2024-09-21T00:30:00.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38188 fp=#0x10000000719' 2024-09-21T00:30:00.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38220 fp=#0x1000000075c' 2024-09-21T00:30:00.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38252 fp=#0x10000000e38' 2024-09-21T00:30:00.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38348 fp=#0x30000001a7e' 2024-09-21T00:30:00.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38412 fp=#0x1000000331d' 2024-09-21T00:30:00.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38444 fp=#0x30000001ce7' 2024-09-21T00:30:00.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38476 fp=#0x10000003310' 2024-09-21T00:30:00.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38508 fp=#0x100000020df' 2024-09-21T00:30:00.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38604 fp=#0x100000005e7' 2024-09-21T00:30:00.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38668 fp=#0x30000001c50' 2024-09-21T00:30:00.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38700 fp=#0x100000005b8' 2024-09-21T00:30:00.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38732 fp=#0x100000005ab' 2024-09-21T00:30:00.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38764 fp=#0x30000001bfc' 2024-09-21T00:30:00.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38796 fp=#0x10000002936' 2024-09-21T00:30:00.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38828 fp=#0x300000019df' 2024-09-21T00:30:00.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38860 fp=#0x1000000291c' 2024-09-21T00:30:00.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38924 fp=#0x30000001d63' 2024-09-21T00:30:00.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38956 fp=#0x30000001d54' 2024-09-21T00:30:00.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38988 fp=#0x30000001c23' 2024-09-21T00:30:00.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39020 fp=#0x30000001c76' 2024-09-21T00:30:00.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39052 fp=#0x30000001c73' 2024-09-21T00:30:00.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39116 fp=#0x30000001d2d' 2024-09-21T00:30:00.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39180 fp=#0x10000001274' 2024-09-21T00:30:00.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39212 fp=#0x1000000353b' 2024-09-21T00:30:00.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39244 fp=#0x10000003587' 2024-09-21T00:30:00.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39340 fp=#0x10000003f49' 2024-09-21T00:30:00.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39372 fp=#0x10000003f85' 2024-09-21T00:30:00.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39404 fp=#0x10000003f57' 2024-09-21T00:30:00.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39436 fp=#0x10000003f01' 2024-09-21T00:30:00.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39468 fp=#0x30000001f01' 2024-09-21T00:30:00.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39500 fp=#0x30000001f0a' 2024-09-21T00:30:00.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39532 fp=#0x30000001f82' 2024-09-21T00:30:00.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39564 fp=#0x30000001f7f' 2024-09-21T00:30:00.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39628 fp=#0x10000002ab5' 2024-09-21T00:30:00.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39692 fp=#0x30000000e68' 2024-09-21T00:30:00.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39756 fp=#0x300000019a7' 2024-09-21T00:30:00.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39788 fp=#0x300000019ab' 2024-09-21T00:30:00.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39820 fp=#0x30000000ee1' 2024-09-21T00:30:00.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39884 fp=#0x3000000198b' 2024-09-21T00:30:00.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39916 fp=#0x30000001105' 2024-09-21T00:30:00.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40012 fp=#0x300000014d1' 2024-09-21T00:30:00.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40076 fp=#0x3000000180c' 2024-09-21T00:30:00.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40108 fp=#0x30000001676' 2024-09-21T00:30:00.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40172 fp=#0x30000001833' 2024-09-21T00:30:00.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40204 fp=#0x10000001d87' 2024-09-21T00:30:00.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40236 fp=#0x10000001d42' 2024-09-21T00:30:00.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40268 fp=#0x300000009ec' 2024-09-21T00:30:00.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40332 fp=#0x10000001d44' 2024-09-21T00:30:00.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40364 fp=#0x3000000231e' 2024-09-21T00:30:00.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40396 fp=#0x10000004cfb' 2024-09-21T00:30:00.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40428 fp=#0x10000004d23' 2024-09-21T00:30:00.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40460 fp=#0x10000004c7f' 2024-09-21T00:30:00.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40492 fp=#0x10000004cea' 2024-09-21T00:30:00.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40524 fp=#0x10000004648' 2024-09-21T00:30:00.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40556 fp=#0x10000001009' 2024-09-21T00:30:00.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40588 fp=#0x20000000587' 2024-09-21T00:30:00.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40620 fp=#0x10000000ca6' 2024-09-21T00:30:00.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40652 fp=#0x10000001001' 2024-09-21T00:30:00.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40684 fp=#0x10000000f99' 2024-09-21T00:30:00.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40716 fp=#0x20000000447' 2024-09-21T00:30:00.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40780 fp=#0x30000001448' 2024-09-21T00:30:00.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40812 fp=#0x30000001421' 2024-09-21T00:30:00.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40844 fp=#0x10000004aa1' 2024-09-21T00:30:00.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40908 fp=#0x10000004aa9' 2024-09-21T00:30:00.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40972 fp=#0x10000004abb' 2024-09-21T00:30:00.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41036 fp=#0x2000000070e' 2024-09-21T00:30:00.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41068 fp=#0x20000000717' 2024-09-21T00:30:00.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41100 fp=#0x20000000aee' 2024-09-21T00:30:00.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41132 fp=#0x10000000e10' 2024-09-21T00:30:00.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41164 fp=#0x20000000871' 2024-09-21T00:30:00.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41196 fp=#0x10000000ec5' 2024-09-21T00:30:00.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41228 fp=#0x20000000bb6' 2024-09-21T00:30:00.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41260 fp=#0x20000000476' 2024-09-21T00:30:00.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41292 fp=#0x20000000b85' 2024-09-21T00:30:00.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41324 fp=#0x10000000cef' 2024-09-21T00:30:00.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41388 fp=#0x10000000e6b' 2024-09-21T00:30:00.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41420 fp=#0x10000000c22' 2024-09-21T00:30:00.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41452 fp=#0x30000002075' 2024-09-21T00:30:00.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41484 fp=#0x100000009d6' 2024-09-21T00:30:00.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41516 fp=#0x10000000970' 2024-09-21T00:30:00.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41548 fp=#0x10000000adf' 2024-09-21T00:30:00.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41580 fp=#0x30000001913' 2024-09-21T00:30:00.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41676 fp=#0x30000000902' 2024-09-21T00:30:00.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41772 fp=#0x300000007b9' 2024-09-21T00:30:00.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41836 fp=#0x30000001383' 2024-09-21T00:30:00.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41868 fp=#0x3000000083a' 2024-09-21T00:30:00.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41900 fp=#0x3000000095d' 2024-09-21T00:30:00.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41964 fp=#0x30000000845' 2024-09-21T00:30:00.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41996 fp=#0x300000007be' 2024-09-21T00:30:00.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42028 fp=#0x30000001da6' 2024-09-21T00:30:00.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42060 fp=#0x30000000976' 2024-09-21T00:30:00.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42092 fp=#0x3000000085a' 2024-09-21T00:30:00.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42124 fp=#0x30000001d55' 2024-09-21T00:30:00.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42156 fp=#0x30000000bc0' 2024-09-21T00:30:00.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42188 fp=#0x30000001b93' 2024-09-21T00:30:00.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42220 fp=#0x30000001b9b' 2024-09-21T00:30:00.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42252 fp=#0x30000000b60' 2024-09-21T00:30:00.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42284 fp=#0x300000020a3' 2024-09-21T00:30:00.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42316 fp=#0x30000001bb0' 2024-09-21T00:30:00.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42348 fp=#0x30000000b2e' 2024-09-21T00:30:00.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42380 fp=#0x30000000af2' 2024-09-21T00:30:00.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42412 fp=#0x30000000b27' 2024-09-21T00:30:00.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42444 fp=#0x10000004130' 2024-09-21T00:30:00.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42508 fp=#0x10000004148' 2024-09-21T00:30:00.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42540 fp=#0x100000040f2' 2024-09-21T00:30:00.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42572 fp=#0x300000013dd' 2024-09-21T00:30:00.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42700 fp=#0x100000018ce' 2024-09-21T00:30:00.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42732 fp=#0x1000000191a' 2024-09-21T00:30:00.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42796 fp=#0x10000001914' 2024-09-21T00:30:00.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42828 fp=#0x30000001089' 2024-09-21T00:30:00.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42860 fp=#0x100000019f3' 2024-09-21T00:30:00.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42924 fp=#0x30000000def' 2024-09-21T00:30:00.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43020 fp=#0x30000001768' 2024-09-21T00:30:00.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43052 fp=#0x30000001733' 2024-09-21T00:30:00.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43084 fp=#0x300000016cb' 2024-09-21T00:30:00.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43116 fp=#0x300000016d0' 2024-09-21T00:30:00.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43148 fp=#0x100000018af' 2024-09-21T00:30:00.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43244 fp=#0x10000001899' 2024-09-21T00:30:00.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43276 fp=#0x30000000f01' 2024-09-21T00:30:00.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43308 fp=#0x30000000f71' 2024-09-21T00:30:00.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43340 fp=#0x30000000f88' 2024-09-21T00:30:00.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43372 fp=#0x30000000f2e' 2024-09-21T00:30:00.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43436 fp=#0x300000023f0' 2024-09-21T00:30:00.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43468 fp=#0x300000023ec' 2024-09-21T00:30:00.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43500 fp=#0x30000002380' 2024-09-21T00:30:00.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43532 fp=#0x30000002383' 2024-09-21T00:30:00.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43564 fp=#0x300000023b8' 2024-09-21T00:30:00.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43596 fp=#0x30000000fcb' 2024-09-21T00:30:00.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43628 fp=#0x30000002149' 2024-09-21T00:30:00.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43660 fp=#0x300000021cf' 2024-09-21T00:30:00.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43692 fp=#0x30000002122' 2024-09-21T00:30:00.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43724 fp=#0x30000002113' 2024-09-21T00:30:00.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43756 fp=#0x3000000213a' 2024-09-21T00:30:00.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43788 fp=#0x10000002a29' 2024-09-21T00:30:00.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43820 fp=#0x100000039ad' 2024-09-21T00:30:00.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43884 fp=#0x100000039e4' 2024-09-21T00:30:00.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43948 fp=#0x3000000134c' 2024-09-21T00:30:00.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31757 fp=#0x10000000c98' 2024-09-21T00:30:00.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31821 fp=#0x10000003ed1' 2024-09-21T00:30:00.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31917 fp=#0x10000001480' 2024-09-21T00:30:00.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31949 fp=#0x10000004155' 2024-09-21T00:30:00.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34669 fp=#0x30000002295' 2024-09-21T00:30:00.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34701 fp=#0x3000000226d' 2024-09-21T00:30:00.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34733 fp=#0x30000002277' 2024-09-21T00:30:00.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34765 fp=#0x30000002279' 2024-09-21T00:30:00.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34861 fp=#0x10000002812' 2024-09-21T00:30:00.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34925 fp=#0x100000027f5' 2024-09-21T00:30:00.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34989 fp=#0x100000032fb' 2024-09-21T00:30:00.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35021 fp=#0x3000000205f' 2024-09-21T00:30:00.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35053 fp=#0x100000032ab' 2024-09-21T00:30:00.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35085 fp=#0x1000000387c' 2024-09-21T00:30:00.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35117 fp=#0x10000003a41' 2024-09-21T00:30:00.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35149 fp=#0x10000003a1d' 2024-09-21T00:30:00.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35181 fp=#0x10000004219' 2024-09-21T00:30:00.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35213 fp=#0x10000003945' 2024-09-21T00:30:00.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35245 fp=#0x10000004163' 2024-09-21T00:30:00.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35277 fp=#0x10000004197' 2024-09-21T00:30:00.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35309 fp=#0x10000004141' 2024-09-21T00:30:00.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35341 fp=#0x10000004193' 2024-09-21T00:30:00.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35373 fp=#0x1000000393f' 2024-09-21T00:30:00.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35405 fp=#0x100000038f8' 2024-09-21T00:30:00.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35437 fp=#0x10000003987' 2024-09-21T00:30:00.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35469 fp=#0x30000001195' 2024-09-21T00:30:00.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35501 fp=#0x300000011a1' 2024-09-21T00:30:00.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35533 fp=#0x30000001129' 2024-09-21T00:30:00.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35565 fp=#0x30000002059' 2024-09-21T00:30:00.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35597 fp=#0x30000002003' 2024-09-21T00:30:00.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35629 fp=#0x3000000205a' 2024-09-21T00:30:00.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35661 fp=#0x30000002031' 2024-09-21T00:30:00.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35693 fp=#0x30000001ff4' 2024-09-21T00:30:00.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35821 fp=#0x100000033ad' 2024-09-21T00:30:00.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35853 fp=#0x1000000338d' 2024-09-21T00:30:00.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35885 fp=#0x10000003464' 2024-09-21T00:30:00.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35917 fp=#0x300000011cd' 2024-09-21T00:30:00.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35949 fp=#0x30000001201' 2024-09-21T00:30:00.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35981 fp=#0x300000011c0' 2024-09-21T00:30:00.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36077 fp=#0x10000002cc9' 2024-09-21T00:30:00.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36109 fp=#0x10000002d3d' 2024-09-21T00:30:00.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36141 fp=#0x10000002080' 2024-09-21T00:30:00.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36205 fp=#0x10000001f2a' 2024-09-21T00:30:00.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36397 fp=#0x300000015a8' 2024-09-21T00:30:00.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36429 fp=#0x10000001e71' 2024-09-21T00:30:00.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36461 fp=#0x10000001e49' 2024-09-21T00:30:00.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36493 fp=#0x30000002305' 2024-09-21T00:30:00.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36525 fp=#0x10000001e33' 2024-09-21T00:30:00.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36557 fp=#0x10000001dcc' 2024-09-21T00:30:00.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36589 fp=#0x10000004446' 2024-09-21T00:30:00.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36621 fp=#0x10000004491' 2024-09-21T00:30:00.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36653 fp=#0x10000003788' 2024-09-21T00:30:00.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36685 fp=#0x100000037ba' 2024-09-21T00:30:00.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36717 fp=#0x100000047a3' 2024-09-21T00:30:00.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36749 fp=#0x1000000471a' 2024-09-21T00:30:00.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36781 fp=#0x10000004732' 2024-09-21T00:30:00.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36813 fp=#0x100000047b7' 2024-09-21T00:30:00.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36845 fp=#0x30000001aae' 2024-09-21T00:30:00.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36877 fp=#0x10000003634' 2024-09-21T00:30:00.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36909 fp=#0x10000002543' 2024-09-21T00:30:00.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36941 fp=#0x1000000356b' 2024-09-21T00:30:00.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36973 fp=#0x10000003694' 2024-09-21T00:30:00.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37037 fp=#0x10000003677' 2024-09-21T00:30:00.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37165 fp=#0x1000000148c' 2024-09-21T00:30:00.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37197 fp=#0x10000003152' 2024-09-21T00:30:00.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37229 fp=#0x100000030fc' 2024-09-21T00:30:00.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37261 fp=#0x10000003065' 2024-09-21T00:30:00.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37293 fp=#0x10000003181' 2024-09-21T00:30:00.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37325 fp=#0x30000000bf9' 2024-09-21T00:30:00.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37357 fp=#0x30000001ac6' 2024-09-21T00:30:00.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37389 fp=#0x30000000be8' 2024-09-21T00:30:00.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37421 fp=#0x30000000c19' 2024-09-21T00:30:00.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37453 fp=#0x30000000bf8' 2024-09-21T00:30:00.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37517 fp=#0x30000001082' 2024-09-21T00:30:00.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37549 fp=#0x30000001b7a' 2024-09-21T00:30:00.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37581 fp=#0x3000000107d' 2024-09-21T00:30:00.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37613 fp=#0x30000001b77' 2024-09-21T00:30:00.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37677 fp=#0x10000002844' 2024-09-21T00:30:00.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37805 fp=#0x30000000c65' 2024-09-21T00:30:00.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37837 fp=#0x300000012d5' 2024-09-21T00:30:00.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37901 fp=#0x30000001282' 2024-09-21T00:30:00.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37933 fp=#0x30000002255' 2024-09-21T00:30:00.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37997 fp=#0x30000000cde' 2024-09-21T00:30:00.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38093 fp=#0x10000000771' 2024-09-21T00:30:00.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38157 fp=#0x10000000775' 2024-09-21T00:30:00.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38221 fp=#0x1000000073b' 2024-09-21T00:30:00.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38253 fp=#0x10000000733' 2024-09-21T00:30:00.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38317 fp=#0x10000002455' 2024-09-21T00:30:00.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38349 fp=#0x30000001a46' 2024-09-21T00:30:00.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38381 fp=#0x30000001a51' 2024-09-21T00:30:00.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38445 fp=#0x10000003346' 2024-09-21T00:30:00.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38477 fp=#0x10000003307' 2024-09-21T00:30:00.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38509 fp=#0x10000002188' 2024-09-21T00:30:00.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38573 fp=#0x30000001ef5' 2024-09-21T00:30:00.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38605 fp=#0x30000001c3f' 2024-09-21T00:30:00.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38637 fp=#0x100000005b0' 2024-09-21T00:30:00.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38669 fp=#0x30000001c11' 2024-09-21T00:30:00.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38733 fp=#0x100000005e4' 2024-09-21T00:30:00.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38765 fp=#0x30000001c07' 2024-09-21T00:30:00.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38829 fp=#0x300000019e1' 2024-09-21T00:30:00.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38893 fp=#0x300000019f7' 2024-09-21T00:30:00.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38925 fp=#0x30000001c24' 2024-09-21T00:30:00.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38957 fp=#0x30000001d3a' 2024-09-21T00:30:00.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39021 fp=#0x30000001c6d' 2024-09-21T00:30:00.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39053 fp=#0x30000001d00' 2024-09-21T00:30:00.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39181 fp=#0x1000000126e' 2024-09-21T00:30:00.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39245 fp=#0x10000003564' 2024-09-21T00:30:00.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39341 fp=#0x10000003f93' 2024-09-21T00:30:00.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39373 fp=#0x10000003fa0' 2024-09-21T00:30:00.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39405 fp=#0x10000003f10' 2024-09-21T00:30:00.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39437 fp=#0x10000003f9c' 2024-09-21T00:30:00.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39469 fp=#0x30000001f04' 2024-09-21T00:30:00.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39501 fp=#0x30000001f17' 2024-09-21T00:30:00.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39533 fp=#0x30000001f62' 2024-09-21T00:30:00.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39565 fp=#0x30000001f30' 2024-09-21T00:30:00.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39597 fp=#0x10000002b40' 2024-09-21T00:30:00.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39629 fp=#0x10000002ac2' 2024-09-21T00:30:00.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39661 fp=#0x10000002a9a' 2024-09-21T00:30:00.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39693 fp=#0x30000000e9a' 2024-09-21T00:30:00.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39725 fp=#0x30000000ee6' 2024-09-21T00:30:00.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39789 fp=#0x30000000ef5' 2024-09-21T00:30:00.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39821 fp=#0x30000000ea3' 2024-09-21T00:30:00.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39885 fp=#0x30000000e97' 2024-09-21T00:30:00.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39981 fp=#0x30000001020' 2024-09-21T00:30:00.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40013 fp=#0x30000001834' 2024-09-21T00:30:00.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40045 fp=#0x30000001828' 2024-09-21T00:30:00.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40077 fp=#0x30000001840' 2024-09-21T00:30:00.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40109 fp=#0x30000001540' 2024-09-21T00:30:00.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40141 fp=#0x30000001832' 2024-09-21T00:30:00.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40173 fp=#0x30000001862' 2024-09-21T00:30:00.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40205 fp=#0x30000000dd3' 2024-09-21T00:30:00.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40237 fp=#0x30000000d70' 2024-09-21T00:30:00.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40269 fp=#0x300000009ef' 2024-09-21T00:30:00.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40301 fp=#0x30000000d90' 2024-09-21T00:30:00.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40333 fp=#0x10000001d67' 2024-09-21T00:30:00.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40365 fp=#0x300000022d9' 2024-09-21T00:30:00.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40397 fp=#0x30000002328' 2024-09-21T00:30:00.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40429 fp=#0x10000004c8f' 2024-09-21T00:30:00.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40461 fp=#0x300000022bb' 2024-09-21T00:30:00.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40493 fp=#0x10000004e53' 2024-09-21T00:30:00.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40557 fp=#0x200000001fd' 2024-09-21T00:30:00.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40589 fp=#0x20000000589' 2024-09-21T00:30:00.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40621 fp=#0x10000000cd8' 2024-09-21T00:30:00.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40685 fp=#0x10000000d42' 2024-09-21T00:30:00.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40717 fp=#0x10000000d31' 2024-09-21T00:30:00.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40781 fp=#0x30000001424' 2024-09-21T00:30:00.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40813 fp=#0x30000001446' 2024-09-21T00:30:00.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40845 fp=#0x10000004a72' 2024-09-21T00:30:00.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40877 fp=#0x10000004aa2' 2024-09-21T00:30:00.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41005 fp=#0x10000004ae2' 2024-09-21T00:30:00.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41037 fp=#0x1000000347b' 2024-09-21T00:30:00.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41069 fp=#0x1000000347f' 2024-09-21T00:30:01.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41101 fp=#0x10000001006' 2024-09-21T00:30:01.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41133 fp=#0x10000000d58' 2024-09-21T00:30:01.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41165 fp=#0x20000000a8e' 2024-09-21T00:30:01.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41197 fp=#0x20000000479' 2024-09-21T00:30:01.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41229 fp=#0x20000000beb' 2024-09-21T00:30:01.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41261 fp=#0x10000000d29' 2024-09-21T00:30:01.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41293 fp=#0x20000000bcf' 2024-09-21T00:30:01.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41325 fp=#0x10000001016' 2024-09-21T00:30:01.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41389 fp=#0x10000000bb6' 2024-09-21T00:30:01.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41421 fp=#0x1000000097d' 2024-09-21T00:30:01.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41453 fp=#0x3000000206a' 2024-09-21T00:30:01.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41485 fp=#0x10000000b90' 2024-09-21T00:30:01.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41517 fp=#0x10000000a81' 2024-09-21T00:30:01.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41549 fp=#0x3000000207b' 2024-09-21T00:30:01.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41645 fp=#0x3000000184d' 2024-09-21T00:30:01.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41677 fp=#0x300000007bd' 2024-09-21T00:30:01.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41773 fp=#0x300000008fd' 2024-09-21T00:30:01.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41805 fp=#0x3000000082d' 2024-09-21T00:30:01.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41837 fp=#0x30000000841' 2024-09-21T00:30:01.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41869 fp=#0x3000000094c' 2024-09-21T00:30:01.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41901 fp=#0x3000000085f' 2024-09-21T00:30:01.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41933 fp=#0x300000009b0' 2024-09-21T00:30:01.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41965 fp=#0x30000001da4' 2024-09-21T00:30:01.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41997 fp=#0x30000000830' 2024-09-21T00:30:01.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42029 fp=#0x300000007ec' 2024-09-21T00:30:01.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42061 fp=#0x30000001d94' 2024-09-21T00:30:01.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42093 fp=#0x3000000084c' 2024-09-21T00:30:01.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42125 fp=#0x300000007fc' 2024-09-21T00:30:01.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42157 fp=#0x30000001ba6' 2024-09-21T00:30:01.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42189 fp=#0x30000000b8a' 2024-09-21T00:30:01.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42221 fp=#0x30000000b9c' 2024-09-21T00:30:01.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42253 fp=#0x30000000b83' 2024-09-21T00:30:01.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42317 fp=#0x30000001bba' 2024-09-21T00:30:01.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42349 fp=#0x30000000b43' 2024-09-21T00:30:01.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42381 fp=#0x30000000ada' 2024-09-21T00:30:01.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42413 fp=#0x30000000ade' 2024-09-21T00:30:01.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42445 fp=#0x30000000a52' 2024-09-21T00:30:01.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42477 fp=#0x100000040f6' 2024-09-21T00:30:01.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42509 fp=#0x30000000a6f' 2024-09-21T00:30:01.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42541 fp=#0x30000000a4f' 2024-09-21T00:30:01.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42573 fp=#0x100000040fc' 2024-09-21T00:30:01.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42637 fp=#0x30000000ca6' 2024-09-21T00:30:01.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42765 fp=#0x100000019be' 2024-09-21T00:30:01.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42829 fp=#0x10000001a36' 2024-09-21T00:30:01.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42861 fp=#0x100000019e2' 2024-09-21T00:30:01.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42925 fp=#0x30000000e3a' 2024-09-21T00:30:01.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43021 fp=#0x3000000172d' 2024-09-21T00:30:01.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43053 fp=#0x3000000170d' 2024-09-21T00:30:01.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43085 fp=#0x3000000168d' 2024-09-21T00:30:01.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43117 fp=#0x3000000174a' 2024-09-21T00:30:01.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43181 fp=#0x10000001876' 2024-09-21T00:30:01.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43277 fp=#0x30000000f33' 2024-09-21T00:30:01.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43309 fp=#0x30000000f3f' 2024-09-21T00:30:01.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43373 fp=#0x30000000f4d' 2024-09-21T00:30:01.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43405 fp=#0x30000000f08' 2024-09-21T00:30:01.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43437 fp=#0x300000023eb' 2024-09-21T00:30:01.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43469 fp=#0x300000023b6' 2024-09-21T00:30:01.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43501 fp=#0x300000023bc' 2024-09-21T00:30:01.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43533 fp=#0x3000000239f' 2024-09-21T00:30:01.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43565 fp=#0x300000023db' 2024-09-21T00:30:01.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43597 fp=#0x300000010f3' 2024-09-21T00:30:01.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43661 fp=#0x30000002119' 2024-09-21T00:30:01.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43693 fp=#0x30000002129' 2024-09-21T00:30:01.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43821 fp=#0x100000039b0' 2024-09-21T00:30:01.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43853 fp=#0x100000039b4' 2024-09-21T00:30:01.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43949 fp=#0x30000001390' 2024-09-21T00:30:01.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31758 fp=#0x10000003373' 2024-09-21T00:30:01.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31822 fp=#0x30000000fed' 2024-09-21T00:30:01.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31950 fp=#0x1000000039d' 2024-09-21T00:30:01.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34670 fp=#0x3000000228d' 2024-09-21T00:30:01.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34702 fp=#0x3000000227e' 2024-09-21T00:30:01.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34734 fp=#0x10000000c41' 2024-09-21T00:30:01.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34990 fp=#0x100000032e8' 2024-09-21T00:30:01.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35022 fp=#0x300000020af' 2024-09-21T00:30:01.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35054 fp=#0x1000000327f' 2024-09-21T00:30:01.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35086 fp=#0x1000000392a' 2024-09-21T00:30:01.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35118 fp=#0x100000038d9' 2024-09-21T00:30:01.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35150 fp=#0x10000003acd' 2024-09-21T00:30:01.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35182 fp=#0x10000004165' 2024-09-21T00:30:01.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35214 fp=#0x30000001ef2' 2024-09-21T00:30:01.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35246 fp=#0x10000003950' 2024-09-21T00:30:01.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35278 fp=#0x100000039db' 2024-09-21T00:30:01.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35310 fp=#0x100000041d3' 2024-09-21T00:30:01.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35342 fp=#0x10000004135' 2024-09-21T00:30:01.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35374 fp=#0x100000038b6' 2024-09-21T00:30:01.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35406 fp=#0x10000003897' 2024-09-21T00:30:01.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35470 fp=#0x300000011aa' 2024-09-21T00:30:01.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35534 fp=#0x3000000111e' 2024-09-21T00:30:01.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35566 fp=#0x3000000203a' 2024-09-21T00:30:01.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35598 fp=#0x3000000203e' 2024-09-21T00:30:01.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35630 fp=#0x30000002026' 2024-09-21T00:30:01.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35662 fp=#0x30000001fdb' 2024-09-21T00:30:01.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35694 fp=#0x30000001ffa' 2024-09-21T00:30:01.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35822 fp=#0x1000000346e' 2024-09-21T00:30:01.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35886 fp=#0x100000033a1' 2024-09-21T00:30:01.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35918 fp=#0x300000011ea' 2024-09-21T00:30:01.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35950 fp=#0x300000011e3' 2024-09-21T00:30:01.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36014 fp=#0x10000003c1f' 2024-09-21T00:30:01.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36078 fp=#0x10000002d74' 2024-09-21T00:30:01.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36110 fp=#0x10000002cbc' 2024-09-21T00:30:01.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36142 fp=#0x10000001ede' 2024-09-21T00:30:01.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36206 fp=#0x10000001ebf' 2024-09-21T00:30:01.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36270 fp=#0x300000015a7' 2024-09-21T00:30:01.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36398 fp=#0x300000015fc' 2024-09-21T00:30:01.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36430 fp=#0x10000001e67' 2024-09-21T00:30:01.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36462 fp=#0x10000001e19' 2024-09-21T00:30:01.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36494 fp=#0x10000001e25' 2024-09-21T00:30:01.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36526 fp=#0x10000001d75' 2024-09-21T00:30:01.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36558 fp=#0x10000001de0' 2024-09-21T00:30:01.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36590 fp=#0x1000000444c' 2024-09-21T00:30:01.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36622 fp=#0x1000000448c' 2024-09-21T00:30:01.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36654 fp=#0x1000000443e' 2024-09-21T00:30:01.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36686 fp=#0x10000004418' 2024-09-21T00:30:01.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36718 fp=#0x30000001aac' 2024-09-21T00:30:01.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36750 fp=#0x100000047c8' 2024-09-21T00:30:01.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36782 fp=#0x1000000473a' 2024-09-21T00:30:01.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36814 fp=#0x10000004748' 2024-09-21T00:30:01.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36846 fp=#0x30000001a9f' 2024-09-21T00:30:01.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36878 fp=#0x100000035d3' 2024-09-21T00:30:01.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36910 fp=#0x10000003558' 2024-09-21T00:30:01.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36942 fp=#0x1000000369b' 2024-09-21T00:30:01.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36974 fp=#0x1000000352d' 2024-09-21T00:30:01.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37006 fp=#0x100000034ff' 2024-09-21T00:30:01.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37038 fp=#0x100000035a8' 2024-09-21T00:30:01.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37070 fp=#0x30000001793' 2024-09-21T00:30:01.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37102 fp=#0x3000000180b' 2024-09-21T00:30:01.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37166 fp=#0x10000001483' 2024-09-21T00:30:01.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37230 fp=#0x1000000316f' 2024-09-21T00:30:01.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37294 fp=#0x1000000315e' 2024-09-21T00:30:01.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37326 fp=#0x30000000bf5' 2024-09-21T00:30:01.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37358 fp=#0x30000000c0d' 2024-09-21T00:30:01.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37390 fp=#0x30000000bed' 2024-09-21T00:30:01.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37422 fp=#0x30000001ad3' 2024-09-21T00:30:01.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37454 fp=#0x30000001b1c' 2024-09-21T00:30:01.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37486 fp=#0x30000001133' 2024-09-21T00:30:01.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37518 fp=#0x30000001b72' 2024-09-21T00:30:01.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37550 fp=#0x30000000ff1' 2024-09-21T00:30:01.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37582 fp=#0x3000000196d' 2024-09-21T00:30:01.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37614 fp=#0x30000000fdc' 2024-09-21T00:30:01.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37646 fp=#0x30000000c76' 2024-09-21T00:30:01.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37678 fp=#0x30000000c6c' 2024-09-21T00:30:01.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37742 fp=#0x1000000284b' 2024-09-21T00:30:01.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37806 fp=#0x30000000c3d' 2024-09-21T00:30:01.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37966 fp=#0x30000000dca' 2024-09-21T00:30:01.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37998 fp=#0x30000000cf1' 2024-09-21T00:30:01.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38062 fp=#0x30000000cfa' 2024-09-21T00:30:01.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38126 fp=#0x10000000761' 2024-09-21T00:30:01.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38158 fp=#0x10000001c23' 2024-09-21T00:30:01.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38190 fp=#0x10000000763' 2024-09-21T00:30:01.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38222 fp=#0x10000000ed6' 2024-09-21T00:30:01.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38254 fp=#0x10000000739' 2024-09-21T00:30:01.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38286 fp=#0x10000002483' 2024-09-21T00:30:01.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38350 fp=#0x30000001a49' 2024-09-21T00:30:01.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38414 fp=#0x1000000331c' 2024-09-21T00:30:01.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38446 fp=#0x30000001cf6' 2024-09-21T00:30:01.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38478 fp=#0x100000032d0' 2024-09-21T00:30:01.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38542 fp=#0x100000020e4' 2024-09-21T00:30:01.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38574 fp=#0x1000000210f' 2024-09-21T00:30:01.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38606 fp=#0x30000001c31' 2024-09-21T00:30:01.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38638 fp=#0x30000001c0e' 2024-09-21T00:30:01.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38670 fp=#0x10000000ea0' 2024-09-21T00:30:01.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38702 fp=#0x30000001c1a' 2024-09-21T00:30:01.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38798 fp=#0x300000019fa' 2024-09-21T00:30:01.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38830 fp=#0x300000019c3' 2024-09-21T00:30:01.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38862 fp=#0x300000019e3' 2024-09-21T00:30:01.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38894 fp=#0x300000019c0' 2024-09-21T00:30:01.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39022 fp=#0x30000001c5a' 2024-09-21T00:30:01.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39054 fp=#0x30000001c87' 2024-09-21T00:30:01.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39182 fp=#0x10000003577' 2024-09-21T00:30:01.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39214 fp=#0x10000001115' 2024-09-21T00:30:01.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39246 fp=#0x100000034d8' 2024-09-21T00:30:01.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39342 fp=#0x10000003fdb' 2024-09-21T00:30:01.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39374 fp=#0x10000003f8b' 2024-09-21T00:30:01.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39406 fp=#0x10000003fb6' 2024-09-21T00:30:01.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39438 fp=#0x10000003fa9' 2024-09-21T00:30:01.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39470 fp=#0x30000001f2a' 2024-09-21T00:30:01.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39502 fp=#0x30000001f21' 2024-09-21T00:30:01.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39534 fp=#0x30000001f14' 2024-09-21T00:30:01.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39566 fp=#0x30000001fa7' 2024-09-21T00:30:01.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39598 fp=#0x10000002a8b' 2024-09-21T00:30:01.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39694 fp=#0x30000000ee9' 2024-09-21T00:30:01.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39726 fp=#0x30000000eab' 2024-09-21T00:30:01.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39758 fp=#0x30000000eda' 2024-09-21T00:30:01.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39790 fp=#0x30000001980' 2024-09-21T00:30:01.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39822 fp=#0x30000000e9e' 2024-09-21T00:30:01.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39854 fp=#0x30000000e9b' 2024-09-21T00:30:01.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39886 fp=#0x30000000e9d' 2024-09-21T00:30:01.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39918 fp=#0x30000001025' 2024-09-21T00:30:01.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40046 fp=#0x300000017fa' 2024-09-21T00:30:01.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40078 fp=#0x300000014c9' 2024-09-21T00:30:01.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40110 fp=#0x30000001536' 2024-09-21T00:30:01.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40174 fp=#0x30000001842' 2024-09-21T00:30:01.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40206 fp=#0x10000001d4e' 2024-09-21T00:30:01.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40238 fp=#0x30000000d63' 2024-09-21T00:30:01.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40270 fp=#0x30000001d17' 2024-09-21T00:30:01.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40302 fp=#0x10000001d5d' 2024-09-21T00:30:01.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40334 fp=#0x30000000db5' 2024-09-21T00:30:01.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40398 fp=#0x10000004c63' 2024-09-21T00:30:01.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40430 fp=#0x300000022c7' 2024-09-21T00:30:01.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40462 fp=#0x10000004e6b' 2024-09-21T00:30:01.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40494 fp=#0x10000004e1f' 2024-09-21T00:30:01.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40558 fp=#0x200000005d7' 2024-09-21T00:30:01.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40590 fp=#0x10000000f48' 2024-09-21T00:30:01.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40654 fp=#0x10000000d60' 2024-09-21T00:30:01.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40686 fp=#0x2000000055f' 2024-09-21T00:30:01.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40718 fp=#0x20000000454' 2024-09-21T00:30:01.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40750 fp=#0x20000000610' 2024-09-21T00:30:01.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40782 fp=#0x3000000145a' 2024-09-21T00:30:01.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40814 fp=#0x30000001441' 2024-09-21T00:30:01.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40846 fp=#0x10000004a89' 2024-09-21T00:30:01.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40910 fp=#0x10000004d03' 2024-09-21T00:30:01.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40942 fp=#0x10000004cfd' 2024-09-21T00:30:01.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40974 fp=#0x10000004a99' 2024-09-21T00:30:01.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41006 fp=#0x10000004afe' 2024-09-21T00:30:01.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41038 fp=#0x1000000369d' 2024-09-21T00:30:01.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41070 fp=#0x200000006fb' 2024-09-21T00:30:01.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41102 fp=#0x10000000cc9' 2024-09-21T00:30:01.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41134 fp=#0x20000000208' 2024-09-21T00:30:01.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41166 fp=#0x10000000eca' 2024-09-21T00:30:01.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41198 fp=#0x10000000e9e' 2024-09-21T00:30:01.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41230 fp=#0x20000000ae5' 2024-09-21T00:30:01.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41262 fp=#0x10000000db5' 2024-09-21T00:30:01.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41294 fp=#0x20000000b91' 2024-09-21T00:30:01.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41326 fp=#0x10000000cbd' 2024-09-21T00:30:01.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41358 fp=#0x10000000dbd' 2024-09-21T00:30:01.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41390 fp=#0x10000000bce' 2024-09-21T00:30:01.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41422 fp=#0x10000000af8' 2024-09-21T00:30:01.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41454 fp=#0x1000000092b' 2024-09-21T00:30:01.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41486 fp=#0x10000000baa' 2024-09-21T00:30:01.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41518 fp=#0x10000000b9c' 2024-09-21T00:30:01.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41550 fp=#0x3000000205d' 2024-09-21T00:30:01.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41646 fp=#0x30000001909' 2024-09-21T00:30:01.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41678 fp=#0x30000000900' 2024-09-21T00:30:01.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41806 fp=#0x30000000870' 2024-09-21T00:30:01.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41838 fp=#0x300000009c3' 2024-09-21T00:30:01.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41870 fp=#0x30000000c31' 2024-09-21T00:30:01.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41902 fp=#0x30000000949' 2024-09-21T00:30:01.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41934 fp=#0x300000009a3' 2024-09-21T00:30:01.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41966 fp=#0x3000000099a' 2024-09-21T00:30:01.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41998 fp=#0x30000001d4f' 2024-09-21T00:30:01.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42030 fp=#0x3000000086c' 2024-09-21T00:30:01.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42062 fp=#0x300000009bc' 2024-09-21T00:30:01.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42094 fp=#0x300000007dd' 2024-09-21T00:30:01.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42126 fp=#0x30000000982' 2024-09-21T00:30:01.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42158 fp=#0x30000000bb6' 2024-09-21T00:30:01.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42190 fp=#0x30000000bb8' 2024-09-21T00:30:01.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42222 fp=#0x3000000208d' 2024-09-21T00:30:01.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42254 fp=#0x30000001ba1' 2024-09-21T00:30:01.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42286 fp=#0x30000000b55' 2024-09-21T00:30:01.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42318 fp=#0x30000001b9d' 2024-09-21T00:30:01.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42350 fp=#0x30000000b0b' 2024-09-21T00:30:01.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42382 fp=#0x30000000b20' 2024-09-21T00:30:01.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42414 fp=#0x30000000b0a' 2024-09-21T00:30:01.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42510 fp=#0x30000000a56' 2024-09-21T00:30:01.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42542 fp=#0x100000040ed' 2024-09-21T00:30:01.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42574 fp=#0x30000000a6c' 2024-09-21T00:30:01.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42670 fp=#0x10000001929' 2024-09-21T00:30:01.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42702 fp=#0x10000001934' 2024-09-21T00:30:01.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42766 fp=#0x100000018d7' 2024-09-21T00:30:01.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42830 fp=#0x100000019c3' 2024-09-21T00:30:01.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42862 fp=#0x10000001943' 2024-09-21T00:30:01.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42958 fp=#0x30000000e53' 2024-09-21T00:30:01.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42990 fp=#0x30000000e41' 2024-09-21T00:30:01.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43022 fp=#0x3000000176c' 2024-09-21T00:30:01.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43054 fp=#0x30000001760' 2024-09-21T00:30:01.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43086 fp=#0x30000001744' 2024-09-21T00:30:01.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43118 fp=#0x3000000169d' 2024-09-21T00:30:01.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43214 fp=#0x10000001862' 2024-09-21T00:30:01.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43278 fp=#0x30000000f7d' 2024-09-21T00:30:01.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43342 fp=#0x30000000f80' 2024-09-21T00:30:01.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43438 fp=#0x300000023a7' 2024-09-21T00:30:01.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43470 fp=#0x300000023e5' 2024-09-21T00:30:01.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43502 fp=#0x3000000237f' 2024-09-21T00:30:01.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43534 fp=#0x300000023ee' 2024-09-21T00:30:01.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43566 fp=#0x300000023c8' 2024-09-21T00:30:01.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43598 fp=#0x3000000219f' 2024-09-21T00:30:01.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43630 fp=#0x300000010cd' 2024-09-21T00:30:01.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43694 fp=#0x3000000210a' 2024-09-21T00:30:01.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43726 fp=#0x30000002115' 2024-09-21T00:30:01.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43758 fp=#0x300000010c8' 2024-09-21T00:30:01.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43790 fp=#0x10000002a2f' 2024-09-21T00:30:01.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43886 fp=#0x10000003984' 2024-09-21T00:30:01.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43918 fp=#0x100000039a0' 2024-09-21T00:30:01.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43950 fp=#0x300000013c7' 2024-09-21T00:30:01.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31759 fp=#0x10000002d29' 2024-09-21T00:30:01.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31823 fp=#0x30000001ef7' 2024-09-21T00:30:01.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31951 fp=#0x30000002370' 2024-09-21T00:30:01.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34671 fp=#0x300000022b9' 2024-09-21T00:30:01.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34703 fp=#0x30000002241' 2024-09-21T00:30:01.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34735 fp=#0x30000002231' 2024-09-21T00:30:01.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34895 fp=#0x10000004853' 2024-09-21T00:30:01.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34959 fp=#0x10000004860' 2024-09-21T00:30:01.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34991 fp=#0x300000020a5' 2024-09-21T00:30:01.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35023 fp=#0x100000032a3' 2024-09-21T00:30:01.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35055 fp=#0x30000002078' 2024-09-21T00:30:01.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35087 fp=#0x10000003b18' 2024-09-21T00:30:01.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35119 fp=#0x100000041be' 2024-09-21T00:30:01.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35151 fp=#0x100000041fb' 2024-09-21T00:30:01.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35183 fp=#0x10000003a92' 2024-09-21T00:30:01.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35215 fp=#0x10000003a17' 2024-09-21T00:30:01.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35247 fp=#0x10000004232' 2024-09-21T00:30:01.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35279 fp=#0x10000003995' 2024-09-21T00:30:01.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35311 fp=#0x1000000416f' 2024-09-21T00:30:01.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35343 fp=#0x100000041ad' 2024-09-21T00:30:01.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35375 fp=#0x10000003a24' 2024-09-21T00:30:01.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35407 fp=#0x10000003a30' 2024-09-21T00:30:01.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35439 fp=#0x10000003929' 2024-09-21T00:30:01.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35471 fp=#0x3000000114c' 2024-09-21T00:30:01.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35535 fp=#0x30000001171' 2024-09-21T00:30:01.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35567 fp=#0x3000000205b' 2024-09-21T00:30:01.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35599 fp=#0x3000000202c' 2024-09-21T00:30:01.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35631 fp=#0x30000002052' 2024-09-21T00:30:01.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35663 fp=#0x30000002007' 2024-09-21T00:30:01.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35823 fp=#0x1000000337b' 2024-09-21T00:30:01.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35887 fp=#0x10000003439' 2024-09-21T00:30:01.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35919 fp=#0x300000011f1' 2024-09-21T00:30:01.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35951 fp=#0x300000011e8' 2024-09-21T00:30:01.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36079 fp=#0x10000002ca3' 2024-09-21T00:30:01.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36143 fp=#0x10000001f32' 2024-09-21T00:30:01.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36175 fp=#0x10000001ed2' 2024-09-21T00:30:01.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36239 fp=#0x10000001f54' 2024-09-21T00:30:01.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36271 fp=#0x30000001602' 2024-09-21T00:30:01.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36431 fp=#0x10000001ddd' 2024-09-21T00:30:01.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36463 fp=#0x10000001dfe' 2024-09-21T00:30:01.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36495 fp=#0x10000001e0f' 2024-09-21T00:30:01.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36527 fp=#0x10000001e07' 2024-09-21T00:30:01.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36559 fp=#0x10000001e4b' 2024-09-21T00:30:01.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36591 fp=#0x10000003797' 2024-09-21T00:30:01.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36623 fp=#0x100000044b4' 2024-09-21T00:30:01.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36655 fp=#0x1000000379e' 2024-09-21T00:30:01.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36687 fp=#0x1000000446a' 2024-09-21T00:30:01.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36719 fp=#0x10000004740' 2024-09-21T00:30:01.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36751 fp=#0x100000047be' 2024-09-21T00:30:01.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36783 fp=#0x30000001a80' 2024-09-21T00:30:01.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36815 fp=#0x100000047c6' 2024-09-21T00:30:01.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36847 fp=#0x1000000471b' 2024-09-21T00:30:01.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36879 fp=#0x10000003554' 2024-09-21T00:30:01.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36911 fp=#0x1000000267f' 2024-09-21T00:30:01.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36943 fp=#0x10000003659' 2024-09-21T00:30:01.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37039 fp=#0x10000003504' 2024-09-21T00:30:01.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37103 fp=#0x300000020c1' 2024-09-21T00:30:01.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37135 fp=#0x300000020c6' 2024-09-21T00:30:01.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37167 fp=#0x10000003189' 2024-09-21T00:30:01.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37199 fp=#0x1000000134c' 2024-09-21T00:30:01.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37231 fp=#0x10000003190' 2024-09-21T00:30:01.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37295 fp=#0x10000003149' 2024-09-21T00:30:01.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37327 fp=#0x30000000c24' 2024-09-21T00:30:01.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37359 fp=#0x30000000c04' 2024-09-21T00:30:01.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37391 fp=#0x30000001af0' 2024-09-21T00:30:01.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37423 fp=#0x30000000c14' 2024-09-21T00:30:01.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37455 fp=#0x30000001b12' 2024-09-21T00:30:01.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37487 fp=#0x30000001138' 2024-09-21T00:30:01.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37519 fp=#0x30000001b71' 2024-09-21T00:30:01.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37551 fp=#0x30000001002' 2024-09-21T00:30:01.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37615 fp=#0x300000010cb' 2024-09-21T00:30:01.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37647 fp=#0x30000000c53' 2024-09-21T00:30:01.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37679 fp=#0x30000000c6d' 2024-09-21T00:30:01.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37711 fp=#0x30000000c3b' 2024-09-21T00:30:01.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37743 fp=#0x30000000c78' 2024-09-21T00:30:01.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37775 fp=#0x1000000283f' 2024-09-21T00:30:01.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37839 fp=#0x300000012d7' 2024-09-21T00:30:01.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37935 fp=#0x3000000228a' 2024-09-21T00:30:01.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37967 fp=#0x30000000dc5' 2024-09-21T00:30:01.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37999 fp=#0x30000000cf9' 2024-09-21T00:30:01.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38127 fp=#0x10000000742' 2024-09-21T00:30:01.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38159 fp=#0x10000000df3' 2024-09-21T00:30:01.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38191 fp=#0x1000000073c' 2024-09-21T00:30:01.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38223 fp=#0x10000000f0a' 2024-09-21T00:30:01.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38255 fp=#0x10000000e03' 2024-09-21T00:30:01.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38287 fp=#0x1000000247d' 2024-09-21T00:30:01.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38319 fp=#0x30000001a76' 2024-09-21T00:30:01.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38383 fp=#0x10000003284' 2024-09-21T00:30:01.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38447 fp=#0x10000003321' 2024-09-21T00:30:01.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38511 fp=#0x10000002184' 2024-09-21T00:30:01.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38575 fp=#0x10000002133' 2024-09-21T00:30:01.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38607 fp=#0x30000001c18' 2024-09-21T00:30:01.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38639 fp=#0x30000001c0f' 2024-09-21T00:30:01.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38703 fp=#0x30000001c22' 2024-09-21T00:30:01.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38735 fp=#0x30000001c0b' 2024-09-21T00:30:01.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38767 fp=#0x30000001bb6' 2024-09-21T00:30:01.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38799 fp=#0x30000000c38' 2024-09-21T00:30:01.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38831 fp=#0x10000002970' 2024-09-21T00:30:01.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38895 fp=#0x300000019ce' 2024-09-21T00:30:01.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38927 fp=#0x30000001c59' 2024-09-21T00:30:01.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38959 fp=#0x30000001c7c' 2024-09-21T00:30:01.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38991 fp=#0x30000001d03' 2024-09-21T00:30:01.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39023 fp=#0x30000001d6e' 2024-09-21T00:30:01.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39055 fp=#0x30000001d37' 2024-09-21T00:30:01.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39183 fp=#0x1000000354c' 2024-09-21T00:30:01.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39215 fp=#0x100000034c7' 2024-09-21T00:30:01.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39247 fp=#0x1000000355a' 2024-09-21T00:30:01.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39311 fp=#0x10000003510' 2024-09-21T00:30:01.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39343 fp=#0x10000003fe1' 2024-09-21T00:30:01.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39375 fp=#0x10000003fb2' 2024-09-21T00:30:01.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39407 fp=#0x10000003fab' 2024-09-21T00:30:01.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39439 fp=#0x10000003f23' 2024-09-21T00:30:01.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39471 fp=#0x30000001f9e' 2024-09-21T00:30:01.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39503 fp=#0x30000001fb7' 2024-09-21T00:30:01.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39535 fp=#0x30000001f5a' 2024-09-21T00:30:01.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39567 fp=#0x30000001f18' 2024-09-21T00:30:01.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39599 fp=#0x10000002b1d' 2024-09-21T00:30:01.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39695 fp=#0x30000000e5b' 2024-09-21T00:30:01.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39727 fp=#0x30000000edf' 2024-09-21T00:30:01.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39759 fp=#0x300000019a2' 2024-09-21T00:30:01.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39791 fp=#0x300000019af' 2024-09-21T00:30:01.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39855 fp=#0x30000000ee3' 2024-09-21T00:30:01.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40015 fp=#0x30000001554' 2024-09-21T00:30:01.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40047 fp=#0x3000000153a' 2024-09-21T00:30:01.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40079 fp=#0x3000000183b' 2024-09-21T00:30:01.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40143 fp=#0x300000017fe' 2024-09-21T00:30:01.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40175 fp=#0x3000000184e' 2024-09-21T00:30:01.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40239 fp=#0x30000001d27' 2024-09-21T00:30:01.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40271 fp=#0x10000001d81' 2024-09-21T00:30:01.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40303 fp=#0x10000001d1b' 2024-09-21T00:30:01.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40335 fp=#0x30000001d2c' 2024-09-21T00:30:01.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40367 fp=#0x10000004e7a' 2024-09-21T00:30:01.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40399 fp=#0x10000004d1f' 2024-09-21T00:30:01.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40431 fp=#0x10000004ca5' 2024-09-21T00:30:01.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40463 fp=#0x10000004c89' 2024-09-21T00:30:01.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40527 fp=#0x10000000fd9' 2024-09-21T00:30:01.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40559 fp=#0x10000000d4c' 2024-09-21T00:30:01.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40591 fp=#0x20000000547' 2024-09-21T00:30:01.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40623 fp=#0x20000000435' 2024-09-21T00:30:01.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40655 fp=#0x10000000cd4' 2024-09-21T00:30:01.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40687 fp=#0x10000000d2a' 2024-09-21T00:30:01.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40719 fp=#0x2000000059d' 2024-09-21T00:30:01.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40751 fp=#0x200000005d1' 2024-09-21T00:30:01.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40783 fp=#0x30000001439' 2024-09-21T00:30:01.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40815 fp=#0x3000000141a' 2024-09-21T00:30:01.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40847 fp=#0x10000004cca' 2024-09-21T00:30:01.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40879 fp=#0x10000004a78' 2024-09-21T00:30:01.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40911 fp=#0x10000004ccf' 2024-09-21T00:30:01.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40943 fp=#0x10000004a8b' 2024-09-21T00:30:01.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40975 fp=#0x10000004d20' 2024-09-21T00:30:01.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41007 fp=#0x10000004aab' 2024-09-21T00:30:01.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41039 fp=#0x10000003557' 2024-09-21T00:30:01.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41071 fp=#0x20000000715' 2024-09-21T00:30:01.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41103 fp=#0x10000000de0' 2024-09-21T00:30:01.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41135 fp=#0x200000006a4' 2024-09-21T00:30:01.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41167 fp=#0x10000000cc1' 2024-09-21T00:30:01.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41199 fp=#0x10000000d1f' 2024-09-21T00:30:01.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41231 fp=#0x10000000cac' 2024-09-21T00:30:01.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41263 fp=#0x200000006a2' 2024-09-21T00:30:01.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41295 fp=#0x10000000f3a' 2024-09-21T00:30:01.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41327 fp=#0x20000000c37' 2024-09-21T00:30:01.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41359 fp=#0x10000000bc2' 2024-09-21T00:30:01.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41391 fp=#0x10000000b2a' 2024-09-21T00:30:01.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41423 fp=#0x100000009b0' 2024-09-21T00:30:01.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41455 fp=#0x10000000c26' 2024-09-21T00:30:01.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41487 fp=#0x10000000b85' 2024-09-21T00:30:01.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41519 fp=#0x10000000a28' 2024-09-21T00:30:01.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41551 fp=#0x10000000c7c' 2024-09-21T00:30:01.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41711 fp=#0x300000007e2' 2024-09-21T00:30:01.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41807 fp=#0x30000000862' 2024-09-21T00:30:01.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41839 fp=#0x300000008ca' 2024-09-21T00:30:01.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41871 fp=#0x30000000849' 2024-09-21T00:30:01.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41935 fp=#0x300000008ad' 2024-09-21T00:30:01.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41967 fp=#0x300000007ea' 2024-09-21T00:30:01.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41999 fp=#0x30000001d6a' 2024-09-21T00:30:01.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42031 fp=#0x3000000087c' 2024-09-21T00:30:01.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42063 fp=#0x30000001da1' 2024-09-21T00:30:01.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42095 fp=#0x3000000082a' 2024-09-21T00:30:01.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42127 fp=#0x300000007b8' 2024-09-21T00:30:01.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42159 fp=#0x30000000b6b' 2024-09-21T00:30:01.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42191 fp=#0x30000000b7b' 2024-09-21T00:30:01.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42223 fp=#0x3000000208a' 2024-09-21T00:30:01.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42255 fp=#0x30000000b67' 2024-09-21T00:30:01.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42287 fp=#0x30000000b98' 2024-09-21T00:30:01.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42319 fp=#0x30000000b66' 2024-09-21T00:30:01.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42351 fp=#0x30000000b3c' 2024-09-21T00:30:01.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42415 fp=#0x30000000af5' 2024-09-21T00:30:01.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42511 fp=#0x30000000a48' 2024-09-21T00:30:01.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42543 fp=#0x1000000412b' 2024-09-21T00:30:01.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42671 fp=#0x10000001a4d' 2024-09-21T00:30:01.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42735 fp=#0x1000000193d' 2024-09-21T00:30:01.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42831 fp=#0x100000019d8' 2024-09-21T00:30:01.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42863 fp=#0x10000001a1f' 2024-09-21T00:30:01.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42927 fp=#0x30000000e45' 2024-09-21T00:30:01.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43023 fp=#0x300000016ce' 2024-09-21T00:30:01.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43055 fp=#0x3000000173e' 2024-09-21T00:30:01.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43087 fp=#0x300000016db' 2024-09-21T00:30:01.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43119 fp=#0x30000001726' 2024-09-21T00:30:01.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43279 fp=#0x30000000f0a' 2024-09-21T00:30:01.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43311 fp=#0x30000000f03' 2024-09-21T00:30:01.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43343 fp=#0x30000000f42' 2024-09-21T00:30:01.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43375 fp=#0x30000000eff' 2024-09-21T00:30:01.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43439 fp=#0x300000023ea' 2024-09-21T00:30:01.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43471 fp=#0x300000023dc' 2024-09-21T00:30:01.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43503 fp=#0x30000002384' 2024-09-21T00:30:01.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43535 fp=#0x300000023c5' 2024-09-21T00:30:01.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43567 fp=#0x300000023ba' 2024-09-21T00:30:01.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43599 fp=#0x30000002176' 2024-09-21T00:30:01.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43631 fp=#0x30000001006' 2024-09-21T00:30:01.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43663 fp=#0x30000000fc7' 2024-09-21T00:30:01.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43695 fp=#0x30000000fd0' 2024-09-21T00:30:01.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43727 fp=#0x30000001011' 2024-09-21T00:30:01.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43855 fp=#0x10000003a35' 2024-09-21T00:30:01.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43887 fp=#0x10000003a53' 2024-09-21T00:30:01.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43919 fp=#0x300000013bb' 2024-09-21T00:30:01.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43951 fp=#0x300000013a1' 2024-09-21T00:30:01.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31760 fp=#0x300000011bc' 2024-09-21T00:30:01.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31824 fp=#0x300000014bc' 2024-09-21T00:30:01.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31952 fp=#0x10000003e8d' 2024-09-21T00:30:01.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34672 fp=#0x30000002256' 2024-09-21T00:30:01.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34736 fp=#0x3000000223e' 2024-09-21T00:30:01.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34928 fp=#0x100000026fa' 2024-09-21T00:30:01.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34960 fp=#0x100000027f2' 2024-09-21T00:30:01.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34992 fp=#0x300000020a4' 2024-09-21T00:30:01.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35024 fp=#0x300000020b7' 2024-09-21T00:30:01.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35056 fp=#0x10000003288' 2024-09-21T00:30:01.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35088 fp=#0x1000000417d' 2024-09-21T00:30:01.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35120 fp=#0x1000000415d' 2024-09-21T00:30:01.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35152 fp=#0x100000041f7' 2024-09-21T00:30:01.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35184 fp=#0x100000038b3' 2024-09-21T00:30:01.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35216 fp=#0x10000003aaa' 2024-09-21T00:30:01.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35248 fp=#0x10000004149' 2024-09-21T00:30:01.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35280 fp=#0x10000003a0d' 2024-09-21T00:30:01.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35312 fp=#0x10000004182' 2024-09-21T00:30:01.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35344 fp=#0x1000000416a' 2024-09-21T00:30:01.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35376 fp=#0x10000003908' 2024-09-21T00:30:01.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35408 fp=#0x10000003a0f' 2024-09-21T00:30:01.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35440 fp=#0x10000003918' 2024-09-21T00:30:01.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35472 fp=#0x30000001116' 2024-09-21T00:30:01.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35504 fp=#0x30000001134' 2024-09-21T00:30:01.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35536 fp=#0x30000001146' 2024-09-21T00:30:01.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35600 fp=#0x30000002055' 2024-09-21T00:30:01.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35664 fp=#0x30000001ff2' 2024-09-21T00:30:01.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35696 fp=#0x3000000203d' 2024-09-21T00:30:01.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35824 fp=#0x10000003381' 2024-09-21T00:30:01.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35856 fp=#0x1000000338b' 2024-09-21T00:30:01.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35888 fp=#0x100000033e5' 2024-09-21T00:30:01.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35920 fp=#0x300000011c2' 2024-09-21T00:30:01.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35952 fp=#0x300000011c3' 2024-09-21T00:30:01.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36112 fp=#0x10000002d42' 2024-09-21T00:30:01.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36144 fp=#0x100000020ad' 2024-09-21T00:30:01.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36176 fp=#0x1000000206a' 2024-09-21T00:30:01.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36208 fp=#0x10000001f4a' 2024-09-21T00:30:01.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36240 fp=#0x10000002078' 2024-09-21T00:30:01.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36304 fp=#0x300000015f7' 2024-09-21T00:30:01.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36368 fp=#0x300000015fa' 2024-09-21T00:30:01.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36432 fp=#0x10000001e65' 2024-09-21T00:30:01.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36464 fp=#0x10000001e37' 2024-09-21T00:30:01.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36496 fp=#0x10000001e2f' 2024-09-21T00:30:01.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36528 fp=#0x10000001dc3' 2024-09-21T00:30:01.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36560 fp=#0x10000001d45' 2024-09-21T00:30:01.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36592 fp=#0x10000004476' 2024-09-21T00:30:01.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36624 fp=#0x10000004461' 2024-09-21T00:30:01.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36656 fp=#0x1000000446e' 2024-09-21T00:30:01.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36688 fp=#0x10000004478' 2024-09-21T00:30:01.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36720 fp=#0x30000001a94' 2024-09-21T00:30:01.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36752 fp=#0x30000001a9a' 2024-09-21T00:30:01.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36784 fp=#0x30000001aa0' 2024-09-21T00:30:01.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36816 fp=#0x10000004795' 2024-09-21T00:30:01.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36848 fp=#0x10000004705' 2024-09-21T00:30:01.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36880 fp=#0x10000002673' 2024-09-21T00:30:01.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36944 fp=#0x1000000255c' 2024-09-21T00:30:01.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37008 fp=#0x10000003532' 2024-09-21T00:30:01.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37040 fp=#0x100000035bc' 2024-09-21T00:30:01.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37168 fp=#0x10000003162' 2024-09-21T00:30:01.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37200 fp=#0x100000030eb' 2024-09-21T00:30:01.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37264 fp=#0x300000013d0' 2024-09-21T00:30:01.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37328 fp=#0x30000000c03' 2024-09-21T00:30:01.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37360 fp=#0x30000001ab2' 2024-09-21T00:30:01.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37392 fp=#0x30000001b17' 2024-09-21T00:30:01.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37424 fp=#0x30000001abe' 2024-09-21T00:30:01.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37456 fp=#0x30000001b0b' 2024-09-21T00:30:01.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37488 fp=#0x30000000fd5' 2024-09-21T00:30:01.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37520 fp=#0x30000000ffd' 2024-09-21T00:30:01.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37552 fp=#0x30000001b78' 2024-09-21T00:30:01.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37584 fp=#0x3000000108b' 2024-09-21T00:30:01.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37616 fp=#0x30000000fc8' 2024-09-21T00:30:01.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37648 fp=#0x1000000292f' 2024-09-21T00:30:01.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37712 fp=#0x10000002942' 2024-09-21T00:30:01.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37744 fp=#0x30000000c7f' 2024-09-21T00:30:01.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37776 fp=#0x10000002925' 2024-09-21T00:30:01.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37808 fp=#0x30000000c92' 2024-09-21T00:30:01.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37840 fp=#0x3000000126a' 2024-09-21T00:30:01.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37872 fp=#0x300000012d6' 2024-09-21T00:30:01.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37904 fp=#0x300000012c8' 2024-09-21T00:30:01.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37936 fp=#0x3000000228e' 2024-09-21T00:30:01.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38096 fp=#0x10000000ee9' 2024-09-21T00:30:01.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38128 fp=#0x10000000ea7' 2024-09-21T00:30:01.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38160 fp=#0x10000000e84' 2024-09-21T00:30:01.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38192 fp=#0x1000000074d' 2024-09-21T00:30:01.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38224 fp=#0x10000000edb' 2024-09-21T00:30:01.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38256 fp=#0x1000000074a' 2024-09-21T00:30:01.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38288 fp=#0x30000001a7a' 2024-09-21T00:30:01.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38320 fp=#0x30000001a41' 2024-09-21T00:30:01.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38384 fp=#0x10000003309' 2024-09-21T00:30:01.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38416 fp=#0x10000003280' 2024-09-21T00:30:01.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38448 fp=#0x100000032e7' 2024-09-21T00:30:01.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38480 fp=#0x30000001ce0' 2024-09-21T00:30:01.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38512 fp=#0x1000000219a' 2024-09-21T00:30:01.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38576 fp=#0x10000002137' 2024-09-21T00:30:01.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38608 fp=#0x30000001bea' 2024-09-21T00:30:01.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38672 fp=#0x30000001baf' 2024-09-21T00:30:01.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38704 fp=#0x30000001c04' 2024-09-21T00:30:01.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38800 fp=#0x1000000297c' 2024-09-21T00:30:01.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38864 fp=#0x1000000297f' 2024-09-21T00:30:01.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38928 fp=#0x30000001d89' 2024-09-21T00:30:01.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38960 fp=#0x30000001d1e' 2024-09-21T00:30:01.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39056 fp=#0x30000001cfd' 2024-09-21T00:30:01.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39088 fp=#0x30000001c64' 2024-09-21T00:30:01.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39120 fp=#0x30000001c60' 2024-09-21T00:30:01.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39152 fp=#0x30000001c68' 2024-09-21T00:30:01.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39216 fp=#0x100000034cc' 2024-09-21T00:30:01.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39248 fp=#0x1000000357f' 2024-09-21T00:30:01.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39312 fp=#0x100000034ab' 2024-09-21T00:30:01.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39344 fp=#0x10000003f13' 2024-09-21T00:30:01.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39376 fp=#0x10000003f4c' 2024-09-21T00:30:01.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39408 fp=#0x10000003f40' 2024-09-21T00:30:01.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39440 fp=#0x30000001f07' 2024-09-21T00:30:01.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39472 fp=#0x30000001eff' 2024-09-21T00:30:01.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39504 fp=#0x30000001f70' 2024-09-21T00:30:01.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39536 fp=#0x30000001f13' 2024-09-21T00:30:01.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39568 fp=#0x30000001f8a' 2024-09-21T00:30:01.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39632 fp=#0x10000002b35' 2024-09-21T00:30:01.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39664 fp=#0x10000002a8e' 2024-09-21T00:30:01.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39696 fp=#0x30000000e71' 2024-09-21T00:30:01.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39824 fp=#0x30000000e94' 2024-09-21T00:30:01.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39856 fp=#0x30000001995' 2024-09-21T00:30:01.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39888 fp=#0x30000000e6f' 2024-09-21T00:30:01.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39952 fp=#0x30000001131' 2024-09-21T00:30:01.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40016 fp=#0x30000001558' 2024-09-21T00:30:01.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40048 fp=#0x30000001532' 2024-09-21T00:30:01.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40080 fp=#0x30000001864' 2024-09-21T00:30:01.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40112 fp=#0x3000000182e' 2024-09-21T00:30:01.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40144 fp=#0x30000001815' 2024-09-21T00:30:01.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40176 fp=#0x30000001820' 2024-09-21T00:30:01.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40208 fp=#0x10000001d77' 2024-09-21T00:30:01.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40272 fp=#0x30000000d86' 2024-09-21T00:30:01.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40336 fp=#0x30000000dcf' 2024-09-21T00:30:01.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40368 fp=#0x10000004e83' 2024-09-21T00:30:01.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40400 fp=#0x10000004d38' 2024-09-21T00:30:01.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40464 fp=#0x30000002331' 2024-09-21T00:30:01.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40528 fp=#0x2000000056c' 2024-09-21T00:30:01.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40592 fp=#0x10000000ee6' 2024-09-21T00:30:01.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40624 fp=#0x10000001046' 2024-09-21T00:30:01.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40656 fp=#0x200000005aa' 2024-09-21T00:30:01.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40688 fp=#0x20000000456' 2024-09-21T00:30:01.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40720 fp=#0x200000005ac' 2024-09-21T00:30:01.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40752 fp=#0x10000000fc4' 2024-09-21T00:30:01.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40784 fp=#0x3000000144e' 2024-09-21T00:30:01.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40816 fp=#0x30000001437' 2024-09-21T00:30:01.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40848 fp=#0x10000004942' 2024-09-21T00:30:01.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40912 fp=#0x10000004afb' 2024-09-21T00:30:01.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41040 fp=#0x10000003683' 2024-09-21T00:30:01.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41072 fp=#0x100000034e3' 2024-09-21T00:30:01.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41104 fp=#0x20000000c56' 2024-09-21T00:30:01.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41136 fp=#0x20000000ade' 2024-09-21T00:30:01.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41168 fp=#0x20000000203' 2024-09-21T00:30:01.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41200 fp=#0x20000000b18' 2024-09-21T00:30:01.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41264 fp=#0x2000000020d' 2024-09-21T00:30:01.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41296 fp=#0x10000000ee8' 2024-09-21T00:30:01.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41328 fp=#0x10000000e42' 2024-09-21T00:30:01.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41360 fp=#0x10000000c6a' 2024-09-21T00:30:01.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41392 fp=#0x10000000c72' 2024-09-21T00:30:01.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41424 fp=#0x10000000985' 2024-09-21T00:30:01.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41456 fp=#0x30000002060' 2024-09-21T00:30:01.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41520 fp=#0x3000000207e' 2024-09-21T00:30:01.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41552 fp=#0x3000000206f' 2024-09-21T00:30:01.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41584 fp=#0x30000001900' 2024-09-21T00:30:01.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41648 fp=#0x3000000185b' 2024-09-21T00:30:01.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41680 fp=#0x3000000092b' 2024-09-21T00:30:01.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41776 fp=#0x300000007d3' 2024-09-21T00:30:01.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41808 fp=#0x30000000810' 2024-09-21T00:30:01.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41840 fp=#0x300000009bb' 2024-09-21T00:30:01.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41872 fp=#0x30000000899' 2024-09-21T00:30:01.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41904 fp=#0x3000000085b' 2024-09-21T00:30:01.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41936 fp=#0x300000009c8' 2024-09-21T00:30:01.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41968 fp=#0x30000001d5e' 2024-09-21T00:30:01.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42000 fp=#0x3000000098e' 2024-09-21T00:30:01.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42032 fp=#0x30000001d86' 2024-09-21T00:30:01.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42064 fp=#0x3000000095f' 2024-09-21T00:30:01.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42096 fp=#0x300000009a4' 2024-09-21T00:30:01.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42128 fp=#0x30000001d8b' 2024-09-21T00:30:01.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42160 fp=#0x3000000209e' 2024-09-21T00:30:01.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42192 fp=#0x30000000ba9' 2024-09-21T00:30:01.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42224 fp=#0x300000020a2' 2024-09-21T00:30:01.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42256 fp=#0x30000001bc1' 2024-09-21T00:30:01.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42288 fp=#0x30000000b6e' 2024-09-21T00:30:01.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42320 fp=#0x30000000b6c' 2024-09-21T00:30:01.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42352 fp=#0x30000000b33' 2024-09-21T00:30:01.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42384 fp=#0x30000000b25' 2024-09-21T00:30:01.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42448 fp=#0x30000001b4c' 2024-09-21T00:30:01.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42512 fp=#0x30000001b48' 2024-09-21T00:30:01.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42576 fp=#0x30000001b50' 2024-09-21T00:30:01.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42672 fp=#0x100000019f6' 2024-09-21T00:30:01.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42736 fp=#0x10000001920' 2024-09-21T00:30:01.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42832 fp=#0x1000000195e' 2024-09-21T00:30:01.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42928 fp=#0x30000000e04' 2024-09-21T00:30:01.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42960 fp=#0x30000000e4b' 2024-09-21T00:30:01.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43024 fp=#0x30000001756' 2024-09-21T00:30:01.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43056 fp=#0x300000016b5' 2024-09-21T00:30:01.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43088 fp=#0x30000001718' 2024-09-21T00:30:01.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43120 fp=#0x300000016a9' 2024-09-21T00:30:01.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43376 fp=#0x30000000f75' 2024-09-21T00:30:01.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43408 fp=#0x10000004198' 2024-09-21T00:30:01.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43440 fp=#0x30000002386' 2024-09-21T00:30:01.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43472 fp=#0x300000023c4' 2024-09-21T00:30:01.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43504 fp=#0x3000000239d' 2024-09-21T00:30:01.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43536 fp=#0x300000023ce' 2024-09-21T00:30:01.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43568 fp=#0x30000002377' 2024-09-21T00:30:01.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43600 fp=#0x30000002120' 2024-09-21T00:30:01.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43632 fp=#0x30000002116' 2024-09-21T00:30:01.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43664 fp=#0x30000001101' 2024-09-21T00:30:01.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43696 fp=#0x30000000fa8' 2024-09-21T00:30:01.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43728 fp=#0x30000000fd8' 2024-09-21T00:30:01.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43856 fp=#0x10000002a2c' 2024-09-21T00:30:01.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43920 fp=#0x300000013b1' 2024-09-21T00:30:01.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43952 fp=#0x30000001330' 2024-09-21T00:30:01.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31761 fp=#0x10000003b4d' 2024-09-21T00:30:01.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31825 fp=#0x10000002a4a' 2024-09-21T00:30:01.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31953 fp=#0x10000000460' 2024-09-21T00:30:01.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34705 fp=#0x3000000223d' 2024-09-21T00:30:01.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34737 fp=#0x3000000222c' 2024-09-21T00:30:01.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34769 fp=#0x300000022b8' 2024-09-21T00:30:01.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34865 fp=#0x100000027ef' 2024-09-21T00:30:01.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34897 fp=#0x10000004849' 2024-09-21T00:30:01.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34929 fp=#0x100000027fe' 2024-09-21T00:30:01.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34993 fp=#0x100000032e6' 2024-09-21T00:30:01.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35025 fp=#0x10000003894' 2024-09-21T00:30:01.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35057 fp=#0x1000000382d' 2024-09-21T00:30:01.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35089 fp=#0x10000003b0c' 2024-09-21T00:30:01.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35121 fp=#0x10000003b20' 2024-09-21T00:30:01.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35153 fp=#0x1000000393a' 2024-09-21T00:30:01.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35185 fp=#0x10000003ae5' 2024-09-21T00:30:01.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35217 fp=#0x100000041a2' 2024-09-21T00:30:01.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35249 fp=#0x10000003af0' 2024-09-21T00:30:01.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35281 fp=#0x10000003a6c' 2024-09-21T00:30:01.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35313 fp=#0x10000003962' 2024-09-21T00:30:01.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35345 fp=#0x1000000399b' 2024-09-21T00:30:01.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35377 fp=#0x10000003993' 2024-09-21T00:30:01.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35409 fp=#0x1000000394c' 2024-09-21T00:30:01.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35441 fp=#0x100000039c8' 2024-09-21T00:30:01.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35473 fp=#0x3000000119d' 2024-09-21T00:30:01.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35537 fp=#0x3000000116b' 2024-09-21T00:30:01.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35569 fp=#0x30000001ff7' 2024-09-21T00:30:01.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35601 fp=#0x30000002043' 2024-09-21T00:30:01.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35633 fp=#0x30000002006' 2024-09-21T00:30:01.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35697 fp=#0x3000000202d' 2024-09-21T00:30:01.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35825 fp=#0x100000033e7' 2024-09-21T00:30:01.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35857 fp=#0x10000003396' 2024-09-21T00:30:01.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35889 fp=#0x100000033bf' 2024-09-21T00:30:01.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35921 fp=#0x300000011d9' 2024-09-21T00:30:01.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35953 fp=#0x300000011ef' 2024-09-21T00:30:01.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35985 fp=#0x10000003c22' 2024-09-21T00:30:01.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36113 fp=#0x10000002cc4' 2024-09-21T00:30:01.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36145 fp=#0x10000002016' 2024-09-21T00:30:01.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36177 fp=#0x10000001f83' 2024-09-21T00:30:01.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36209 fp=#0x10000001ee3' 2024-09-21T00:30:01.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36241 fp=#0x10000001efb' 2024-09-21T00:30:01.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36273 fp=#0x30000001609' 2024-09-21T00:30:01.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36305 fp=#0x300000015b7' 2024-09-21T00:30:01.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36369 fp=#0x300000015b2' 2024-09-21T00:30:01.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36433 fp=#0x10000001d6c' 2024-09-21T00:30:01.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36465 fp=#0x10000001e3d' 2024-09-21T00:30:01.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36497 fp=#0x10000001e86' 2024-09-21T00:30:01.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36529 fp=#0x10000001e15' 2024-09-21T00:30:01.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36561 fp=#0x10000001e2b' 2024-09-21T00:30:01.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36593 fp=#0x10000004487' 2024-09-21T00:30:01.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36657 fp=#0x1000000442e' 2024-09-21T00:30:01.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36689 fp=#0x1000000446c' 2024-09-21T00:30:01.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36721 fp=#0x10000004790' 2024-09-21T00:30:01.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36753 fp=#0x30000001a99' 2024-09-21T00:30:01.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36785 fp=#0x1000000478b' 2024-09-21T00:30:01.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36817 fp=#0x1000000476b' 2024-09-21T00:30:01.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36849 fp=#0x30000001aa5' 2024-09-21T00:30:01.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36881 fp=#0x100000026a3' 2024-09-21T00:30:01.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36945 fp=#0x10000002ec3' 2024-09-21T00:30:01.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36977 fp=#0x1000000357d' 2024-09-21T00:30:01.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37009 fp=#0x100000035c2' 2024-09-21T00:30:01.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37041 fp=#0x1000000356e' 2024-09-21T00:30:01.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37105 fp=#0x300000017f9' 2024-09-21T00:30:01.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37201 fp=#0x100000030f0' 2024-09-21T00:30:01.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37265 fp=#0x10000003083' 2024-09-21T00:30:01.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37329 fp=#0x30000001b1e' 2024-09-21T00:30:01.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37361 fp=#0x30000001ac1' 2024-09-21T00:30:01.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37393 fp=#0x30000000c06' 2024-09-21T00:30:01.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37425 fp=#0x30000000bec' 2024-09-21T00:30:01.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37457 fp=#0x30000000bfd' 2024-09-21T00:30:01.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37489 fp=#0x30000001065' 2024-09-21T00:30:01.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37617 fp=#0x30000001b7d' 2024-09-21T00:30:01.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37649 fp=#0x30000000c37' 2024-09-21T00:30:01.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37681 fp=#0x30000000c54' 2024-09-21T00:30:01.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37713 fp=#0x30000000c80' 2024-09-21T00:30:01.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37745 fp=#0x30000000c62' 2024-09-21T00:30:01.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37809 fp=#0x30000000c87' 2024-09-21T00:30:01.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37841 fp=#0x300000012cb' 2024-09-21T00:30:01.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37873 fp=#0x300000012ce' 2024-09-21T00:30:01.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37905 fp=#0x30000001267' 2024-09-21T00:30:01.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37969 fp=#0x30000000dc4' 2024-09-21T00:30:01.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38097 fp=#0x10000000e1a' 2024-09-21T00:30:01.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38129 fp=#0x10000000ead' 2024-09-21T00:30:01.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38161 fp=#0x10000000e33' 2024-09-21T00:30:01.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38193 fp=#0x10000000751' 2024-09-21T00:30:01.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38225 fp=#0x10000000756' 2024-09-21T00:30:01.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38257 fp=#0x10000000e8d' 2024-09-21T00:30:01.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38289 fp=#0x30000001a70' 2024-09-21T00:30:01.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38353 fp=#0x30000001a52' 2024-09-21T00:30:01.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38385 fp=#0x30000001cf8' 2024-09-21T00:30:01.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38417 fp=#0x1000000332f' 2024-09-21T00:30:01.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38449 fp=#0x10000003640' 2024-09-21T00:30:01.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38481 fp=#0x30000001cc4' 2024-09-21T00:30:01.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38513 fp=#0x1000000211a' 2024-09-21T00:30:01.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38609 fp=#0x100000005e6' 2024-09-21T00:30:01.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38641 fp=#0x30000001bec' 2024-09-21T00:30:01.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38673 fp=#0x100000005eb' 2024-09-21T00:30:01.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38737 fp=#0x30000001c0d' 2024-09-21T00:30:01.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38769 fp=#0x30000001c3b' 2024-09-21T00:30:01.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38801 fp=#0x10000002982' 2024-09-21T00:30:01.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38833 fp=#0x300000019c4' 2024-09-21T00:30:01.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38865 fp=#0x300000019d8' 2024-09-21T00:30:01.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38897 fp=#0x1000000293e' 2024-09-21T00:30:01.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38929 fp=#0x30000001d33' 2024-09-21T00:30:01.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38993 fp=#0x30000001d11' 2024-09-21T00:30:01.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39025 fp=#0x30000001d0e' 2024-09-21T00:30:01.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39185 fp=#0x100000034f7' 2024-09-21T00:30:01.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39345 fp=#0x10000003fa2' 2024-09-21T00:30:01.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39377 fp=#0x10000003fc3' 2024-09-21T00:30:01.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39409 fp=#0x10000003f96' 2024-09-21T00:30:01.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39441 fp=#0x30000001faf' 2024-09-21T00:30:01.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39473 fp=#0x30000001f8b' 2024-09-21T00:30:01.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39505 fp=#0x30000001f64' 2024-09-21T00:30:01.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39537 fp=#0x30000001efb' 2024-09-21T00:30:01.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39569 fp=#0x30000001f02' 2024-09-21T00:30:01.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39601 fp=#0x10000002b30' 2024-09-21T00:30:01.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39665 fp=#0x10000002b03' 2024-09-21T00:30:01.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39761 fp=#0x3000000199f' 2024-09-21T00:30:01.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39793 fp=#0x30000000eec' 2024-09-21T00:30:01.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39825 fp=#0x30000000ee8' 2024-09-21T00:30:01.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39857 fp=#0x3000000197f' 2024-09-21T00:30:01.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39985 fp=#0x300000010fc' 2024-09-21T00:30:01.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40049 fp=#0x3000000183a' 2024-09-21T00:30:01.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40081 fp=#0x30000001825' 2024-09-21T00:30:01.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40113 fp=#0x30000001835' 2024-09-21T00:30:01.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40145 fp=#0x300000017f4' 2024-09-21T00:30:01.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40177 fp=#0x30000001674' 2024-09-21T00:30:01.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40209 fp=#0x30000000d7c' 2024-09-21T00:30:01.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40241 fp=#0x10000001d57' 2024-09-21T00:30:01.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40273 fp=#0x30000001d23' 2024-09-21T00:30:01.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40305 fp=#0x30000000db4' 2024-09-21T00:30:01.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40337 fp=#0x30000001d13' 2024-09-21T00:30:01.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40369 fp=#0x300000022ee' 2024-09-21T00:30:01.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40401 fp=#0x300000022fc' 2024-09-21T00:30:01.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40465 fp=#0x10000004c7b' 2024-09-21T00:30:01.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40529 fp=#0x10000000fb7' 2024-09-21T00:30:01.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40561 fp=#0x10000000ca2' 2024-09-21T00:30:01.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40593 fp=#0x1000000102d' 2024-09-21T00:30:01.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40625 fp=#0x200000005bb' 2024-09-21T00:30:01.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40657 fp=#0x200000005b8' 2024-09-21T00:30:01.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40689 fp=#0x10000000cfa' 2024-09-21T00:30:01.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40721 fp=#0x200000005a2' 2024-09-21T00:30:01.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40753 fp=#0x200000005e6' 2024-09-21T00:30:01.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40785 fp=#0x3000000144a' 2024-09-21T00:30:01.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40817 fp=#0x30000001447' 2024-09-21T00:30:01.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40913 fp=#0x10000004cc4' 2024-09-21T00:30:01.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40977 fp=#0x10000004a96' 2024-09-21T00:30:01.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41041 fp=#0x10000003621' 2024-09-21T00:30:01.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41073 fp=#0x2000000071d' 2024-09-21T00:30:01.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41105 fp=#0x200000006b4' 2024-09-21T00:30:01.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41137 fp=#0x10000000d05' 2024-09-21T00:30:01.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41169 fp=#0x10000000fcd' 2024-09-21T00:30:01.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41201 fp=#0x20000000aa2' 2024-09-21T00:30:01.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41233 fp=#0x20000000861' 2024-09-21T00:30:01.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41265 fp=#0x10000000ed4' 2024-09-21T00:30:01.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41297 fp=#0x20000000224' 2024-09-21T00:30:01.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41329 fp=#0x20000000b37' 2024-09-21T00:30:01.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41361 fp=#0x100000009c0' 2024-09-21T00:30:01.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41393 fp=#0x10000000e46' 2024-09-21T00:30:01.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41425 fp=#0x10000000a71' 2024-09-21T00:30:01.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41457 fp=#0x1000000098f' 2024-09-21T00:30:01.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41489 fp=#0x10000000942' 2024-09-21T00:30:01.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41521 fp=#0x1000000095c' 2024-09-21T00:30:01.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41553 fp=#0x10000000bfe' 2024-09-21T00:30:01.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41809 fp=#0x3000000095a' 2024-09-21T00:30:01.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41841 fp=#0x30000000813' 2024-09-21T00:30:01.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41873 fp=#0x3000000084d' 2024-09-21T00:30:01.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41937 fp=#0x3000000086d' 2024-09-21T00:30:01.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42001 fp=#0x30000001d39' 2024-09-21T00:30:01.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42033 fp=#0x30000000960' 2024-09-21T00:30:01.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42065 fp=#0x300000009ac' 2024-09-21T00:30:01.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42097 fp=#0x3000000095b' 2024-09-21T00:30:01.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42129 fp=#0x30000001d73' 2024-09-21T00:30:01.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42161 fp=#0x30000000bb3' 2024-09-21T00:30:01.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42193 fp=#0x30000000b94' 2024-09-21T00:30:01.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42225 fp=#0x30000001bb7' 2024-09-21T00:30:01.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42257 fp=#0x30000001b9a' 2024-09-21T00:30:01.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42289 fp=#0x30000001b8f' 2024-09-21T00:30:01.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42321 fp=#0x30000000b96' 2024-09-21T00:30:01.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42385 fp=#0x30000000b51' 2024-09-21T00:30:01.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42417 fp=#0x30000000ae8' 2024-09-21T00:30:01.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42481 fp=#0x30000000a73' 2024-09-21T00:30:01.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42513 fp=#0x30000000a7b' 2024-09-21T00:30:01.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42545 fp=#0x100000040f3' 2024-09-21T00:30:01.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42577 fp=#0x30000000a78' 2024-09-21T00:30:01.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42673 fp=#0x10000001a51' 2024-09-21T00:30:01.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42705 fp=#0x10000001a07' 2024-09-21T00:30:01.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42769 fp=#0x100000019ad' 2024-09-21T00:30:01.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42801 fp=#0x10000001924' 2024-09-21T00:30:01.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42833 fp=#0x10000001a3d' 2024-09-21T00:30:01.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42897 fp=#0x10000001937' 2024-09-21T00:30:01.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42929 fp=#0x30000000e3e' 2024-09-21T00:30:01.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42993 fp=#0x30000000df8' 2024-09-21T00:30:01.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43025 fp=#0x300000016a1' 2024-09-21T00:30:01.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43057 fp=#0x300000016cc' 2024-09-21T00:30:01.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43089 fp=#0x30000001720' 2024-09-21T00:30:01.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43121 fp=#0x3000000173a' 2024-09-21T00:30:01.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43153 fp=#0x100000018ad' 2024-09-21T00:30:01.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43185 fp=#0x30000000732' 2024-09-21T00:30:01.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43313 fp=#0x30000000efa' 2024-09-21T00:30:01.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43345 fp=#0x30000000f2a' 2024-09-21T00:30:01.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43409 fp=#0x1000000419e' 2024-09-21T00:30:01.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43441 fp=#0x3000000239e' 2024-09-21T00:30:01.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43473 fp=#0x300000023b2' 2024-09-21T00:30:01.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43505 fp=#0x30000002385' 2024-09-21T00:30:01.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43537 fp=#0x300000023af' 2024-09-21T00:30:01.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43569 fp=#0x300000023ed' 2024-09-21T00:30:01.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43601 fp=#0x300000020fc' 2024-09-21T00:30:01.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43633 fp=#0x30000002117' 2024-09-21T00:30:01.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43665 fp=#0x3000000212a' 2024-09-21T00:30:01.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43761 fp=#0x30000000ff8' 2024-09-21T00:30:01.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43857 fp=#0x1000000399a' 2024-09-21T00:30:01.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43889 fp=#0x100000039c5' 2024-09-21T00:30:01.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43921 fp=#0x3000000133e' 2024-09-21T00:30:01.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43953 fp=#0x3000000133b' 2024-09-21T00:30:01.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31762 fp=#0x10000002c98' 2024-09-21T00:30:01.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31826 fp=#0x10000001ca7' 2024-09-21T00:30:01.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31954 fp=#0x10000003d06' 2024-09-21T00:30:01.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34706 fp=#0x30000002265' 2024-09-21T00:30:01.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34738 fp=#0x300000022ba' 2024-09-21T00:30:01.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34770 fp=#0x30000002298' 2024-09-21T00:30:01.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34866 fp=#0x1000000270f' 2024-09-21T00:30:01.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34962 fp=#0x100000026dc' 2024-09-21T00:30:01.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34994 fp=#0x1000000388c' 2024-09-21T00:30:01.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35026 fp=#0x300000020b6' 2024-09-21T00:30:01.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35058 fp=#0x1000000385d' 2024-09-21T00:30:01.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35090 fp=#0x100000038f2' 2024-09-21T00:30:01.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35122 fp=#0x10000003b37' 2024-09-21T00:30:01.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35154 fp=#0x10000003aec' 2024-09-21T00:30:01.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35186 fp=#0x1000000420a' 2024-09-21T00:30:01.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35218 fp=#0x10000003a55' 2024-09-21T00:30:01.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35250 fp=#0x100000039f3' 2024-09-21T00:30:01.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35282 fp=#0x1000000396d' 2024-09-21T00:30:01.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35314 fp=#0x1000000412c' 2024-09-21T00:30:01.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35346 fp=#0x100000039ae' 2024-09-21T00:30:01.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35378 fp=#0x100000039de' 2024-09-21T00:30:01.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35410 fp=#0x100000038df' 2024-09-21T00:30:01.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35442 fp=#0x100000038be' 2024-09-21T00:30:01.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35474 fp=#0x3000000113d' 2024-09-21T00:30:01.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35506 fp=#0x30000001161' 2024-09-21T00:30:01.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35538 fp=#0x30000001164' 2024-09-21T00:30:01.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35570 fp=#0x3000000204f' 2024-09-21T00:30:01.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35602 fp=#0x30000001feb' 2024-09-21T00:30:01.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35634 fp=#0x30000001fe9' 2024-09-21T00:30:01.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35666 fp=#0x30000002022' 2024-09-21T00:30:01.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35890 fp=#0x10000003455' 2024-09-21T00:30:01.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35922 fp=#0x300000011d3' 2024-09-21T00:30:01.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35954 fp=#0x300000011cf' 2024-09-21T00:30:01.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36082 fp=#0x10000002d51' 2024-09-21T00:30:01.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36114 fp=#0x10000002cb5' 2024-09-21T00:30:01.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36146 fp=#0x100000020b3' 2024-09-21T00:30:01.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36178 fp=#0x10000002036' 2024-09-21T00:30:01.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36210 fp=#0x1000000206f' 2024-09-21T00:30:01.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36274 fp=#0x30000001607' 2024-09-21T00:30:01.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36434 fp=#0x10000001dbc' 2024-09-21T00:30:01.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36466 fp=#0x10000001d5e' 2024-09-21T00:30:01.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36498 fp=#0x10000001e1d' 2024-09-21T00:30:01.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36530 fp=#0x10000001d97' 2024-09-21T00:30:01.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36562 fp=#0x1000000380b' 2024-09-21T00:30:01.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36594 fp=#0x10000004468' 2024-09-21T00:30:01.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36626 fp=#0x300000020ee' 2024-09-21T00:30:01.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36658 fp=#0x100000044a7' 2024-09-21T00:30:01.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36690 fp=#0x100000037a9' 2024-09-21T00:30:01.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36722 fp=#0x100000047d2' 2024-09-21T00:30:01.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36754 fp=#0x10000004738' 2024-09-21T00:30:01.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36786 fp=#0x30000001aa8' 2024-09-21T00:30:01.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36818 fp=#0x30000001a92' 2024-09-21T00:30:01.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36850 fp=#0x30000001a82' 2024-09-21T00:30:01.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36882 fp=#0x10000003652' 2024-09-21T00:30:01.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36914 fp=#0x1000000351e' 2024-09-21T00:30:01.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36946 fp=#0x100000035b9' 2024-09-21T00:30:01.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36978 fp=#0x10000004d7c' 2024-09-21T00:30:01.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37010 fp=#0x100000035e6' 2024-09-21T00:30:01.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37074 fp=#0x3000000178f' 2024-09-21T00:30:01.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37170 fp=#0x10000003113' 2024-09-21T00:30:01.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37202 fp=#0x10000003017' 2024-09-21T00:30:01.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37234 fp=#0x100000030f4' 2024-09-21T00:30:01.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37298 fp=#0x1000000133e' 2024-09-21T00:30:01.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37330 fp=#0x30000001b13' 2024-09-21T00:30:01.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37362 fp=#0x30000001ab1' 2024-09-21T00:30:01.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37394 fp=#0x30000001b21' 2024-09-21T00:30:01.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37426 fp=#0x30000000c08' 2024-09-21T00:30:01.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37490 fp=#0x300000010d9' 2024-09-21T00:30:01.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37522 fp=#0x30000001081' 2024-09-21T00:30:01.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37554 fp=#0x30000001b80' 2024-09-21T00:30:01.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37650 fp=#0x1000000294e' 2024-09-21T00:30:01.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37778 fp=#0x30000000c69' 2024-09-21T00:30:01.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37810 fp=#0x30000000c81' 2024-09-21T00:30:01.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37842 fp=#0x300000012d2' 2024-09-21T00:30:01.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37874 fp=#0x3000000127c' 2024-09-21T00:30:01.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37906 fp=#0x3000000126d' 2024-09-21T00:30:01.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37938 fp=#0x30000002250' 2024-09-21T00:30:01.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37970 fp=#0x30000000dd6' 2024-09-21T00:30:01.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38034 fp=#0x30000000daa' 2024-09-21T00:30:01.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38098 fp=#0x10000000770' 2024-09-21T00:30:01.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38130 fp=#0x1000000075b' 2024-09-21T00:30:01.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38194 fp=#0x10000000e45' 2024-09-21T00:30:01.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38226 fp=#0x1000000074f' 2024-09-21T00:30:01.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38290 fp=#0x30000001a5f' 2024-09-21T00:30:01.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38322 fp=#0x30000001a71' 2024-09-21T00:30:01.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38418 fp=#0x100000032ef' 2024-09-21T00:30:01.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38450 fp=#0x10000003279' 2024-09-21T00:30:01.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38482 fp=#0x10000003282' 2024-09-21T00:30:01.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38514 fp=#0x1000000218f' 2024-09-21T00:30:01.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38546 fp=#0x10000002197' 2024-09-21T00:30:01.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38578 fp=#0x1000000216f' 2024-09-21T00:30:01.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38610 fp=#0x30000001ba8' 2024-09-21T00:30:01.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38674 fp=#0x30000001c27' 2024-09-21T00:30:01.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38706 fp=#0x10000000501' 2024-09-21T00:30:01.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38738 fp=#0x100000005b9' 2024-09-21T00:30:01.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38770 fp=#0x30000001c2f' 2024-09-21T00:30:01.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38802 fp=#0x10000002931' 2024-09-21T00:30:01.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38834 fp=#0x1000000292e' 2024-09-21T00:30:01.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38898 fp=#0x300000019ea' 2024-09-21T00:30:01.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38930 fp=#0x30000001d8f' 2024-09-21T00:30:01.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38962 fp=#0x30000001d29' 2024-09-21T00:30:01.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38994 fp=#0x30000001d8c' 2024-09-21T00:30:01.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39026 fp=#0x30000001c77' 2024-09-21T00:30:01.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39186 fp=#0x300000007c4' 2024-09-21T00:30:01.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39314 fp=#0x1000000356f' 2024-09-21T00:30:01.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39346 fp=#0x10000003ef2' 2024-09-21T00:30:01.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39378 fp=#0x10000003fcc' 2024-09-21T00:30:01.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39410 fp=#0x10000003ed3' 2024-09-21T00:30:01.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39442 fp=#0x30000001fa9' 2024-09-21T00:30:01.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39474 fp=#0x30000001faa' 2024-09-21T00:30:01.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39506 fp=#0x30000001f09' 2024-09-21T00:30:01.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39538 fp=#0x30000001fa5' 2024-09-21T00:30:01.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39570 fp=#0x30000001f1d' 2024-09-21T00:30:01.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39634 fp=#0x10000002b3d' 2024-09-21T00:30:01.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39730 fp=#0x30000000ea7' 2024-09-21T00:30:01.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39762 fp=#0x300000019b3' 2024-09-21T00:30:01.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39794 fp=#0x30000000eb1' 2024-09-21T00:30:01.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39890 fp=#0x30000001982' 2024-09-21T00:30:01.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39922 fp=#0x30000001016' 2024-09-21T00:30:01.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39954 fp=#0x30000001119' 2024-09-21T00:30:01.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40018 fp=#0x30000001513' 2024-09-21T00:30:01.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40050 fp=#0x30000001839' 2024-09-21T00:30:01.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40114 fp=#0x30000001818' 2024-09-21T00:30:01.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40146 fp=#0x30000001866' 2024-09-21T00:30:01.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40178 fp=#0x30000001827' 2024-09-21T00:30:01.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40210 fp=#0x30000000d83' 2024-09-21T00:30:01.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40242 fp=#0x10000001d63' 2024-09-21T00:30:01.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40274 fp=#0x300000009eb' 2024-09-21T00:30:01.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40306 fp=#0x10000001cfb' 2024-09-21T00:30:01.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40338 fp=#0x30000000d8e' 2024-09-21T00:30:01.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40370 fp=#0x10000004d13' 2024-09-21T00:30:01.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40402 fp=#0x10000004e01' 2024-09-21T00:30:01.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40434 fp=#0x10000004e3e' 2024-09-21T00:30:01.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40466 fp=#0x10000004e58' 2024-09-21T00:30:01.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40498 fp=#0x1000000465c' 2024-09-21T00:30:01.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40530 fp=#0x10000000d20' 2024-09-21T00:30:01.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40562 fp=#0x1000000103e' 2024-09-21T00:30:01.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40658 fp=#0x20000000569' 2024-09-21T00:30:01.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40690 fp=#0x10000000fcb' 2024-09-21T00:30:01.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40722 fp=#0x20000000449' 2024-09-21T00:30:01.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40786 fp=#0x30000001419' 2024-09-21T00:30:01.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40818 fp=#0x30000001454' 2024-09-21T00:30:01.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40850 fp=#0x10000004a7d' 2024-09-21T00:30:01.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40882 fp=#0x10000004aa3' 2024-09-21T00:30:01.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40914 fp=#0x10000004ce9' 2024-09-21T00:30:01.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40978 fp=#0x10000004b05' 2024-09-21T00:30:01.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41042 fp=#0x100000036d9' 2024-09-21T00:30:01.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41074 fp=#0x10000003480' 2024-09-21T00:30:01.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41106 fp=#0x10000000d39' 2024-09-21T00:30:01.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41138 fp=#0x20000000af7' 2024-09-21T00:30:01.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41170 fp=#0x10000000d98' 2024-09-21T00:30:01.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41202 fp=#0x10000000e76' 2024-09-21T00:30:01.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41234 fp=#0x10000000e52' 2024-09-21T00:30:01.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41266 fp=#0x10000000e0a' 2024-09-21T00:30:01.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41298 fp=#0x10000000ff5' 2024-09-21T00:30:01.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41330 fp=#0x20000000b68' 2024-09-21T00:30:01.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41362 fp=#0x30000002077' 2024-09-21T00:30:01.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41394 fp=#0x10000000c14' 2024-09-21T00:30:01.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41426 fp=#0x10000000c57' 2024-09-21T00:30:01.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41458 fp=#0x30000002072' 2024-09-21T00:30:01.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41522 fp=#0x10000000c01' 2024-09-21T00:30:01.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41554 fp=#0x10000000c2d' 2024-09-21T00:30:01.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41586 fp=#0x3000000185d' 2024-09-21T00:30:01.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41682 fp=#0x30000000930' 2024-09-21T00:30:01.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41714 fp=#0x3000000091e' 2024-09-21T00:30:01.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41746 fp=#0x300000007cc' 2024-09-21T00:30:01.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41778 fp=#0x300000007f5' 2024-09-21T00:30:01.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41810 fp=#0x300000009d6' 2024-09-21T00:30:01.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41842 fp=#0x30000000999' 2024-09-21T00:30:01.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41906 fp=#0x300000009c6' 2024-09-21T00:30:01.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41938 fp=#0x300000009ad' 2024-09-21T00:30:01.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41970 fp=#0x30000000826' 2024-09-21T00:30:01.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42002 fp=#0x300000007ad' 2024-09-21T00:30:01.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42034 fp=#0x3000000098d' 2024-09-21T00:30:01.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42098 fp=#0x30000000973' 2024-09-21T00:30:01.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42130 fp=#0x30000001d67' 2024-09-21T00:30:01.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42162 fp=#0x30000002083' 2024-09-21T00:30:01.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42194 fp=#0x30000000b76' 2024-09-21T00:30:01.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42226 fp=#0x30000000b5c' 2024-09-21T00:30:01.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42258 fp=#0x3000000209a' 2024-09-21T00:30:01.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42290 fp=#0x30000001bbd' 2024-09-21T00:30:01.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42322 fp=#0x30000001bad' 2024-09-21T00:30:01.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42354 fp=#0x30000000b07' 2024-09-21T00:30:01.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42418 fp=#0x30000000b3f' 2024-09-21T00:30:01.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42450 fp=#0x30000000a6d' 2024-09-21T00:30:01.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42482 fp=#0x30000000a72' 2024-09-21T00:30:01.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42546 fp=#0x10000004134' 2024-09-21T00:30:01.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42642 fp=#0x30000001262' 2024-09-21T00:30:01.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42802 fp=#0x100000018d1' 2024-09-21T00:30:01.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42834 fp=#0x100000019df' 2024-09-21T00:30:01.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42898 fp=#0x10000001948' 2024-09-21T00:30:01.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42962 fp=#0x30000000e02' 2024-09-21T00:30:01.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43026 fp=#0x30000001695' 2024-09-21T00:30:01.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43058 fp=#0x300000016b9' 2024-09-21T00:30:01.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43122 fp=#0x300000016a5' 2024-09-21T00:30:01.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43154 fp=#0x100000018bf' 2024-09-21T00:30:01.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43186 fp=#0x1000000188d' 2024-09-21T00:30:01.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43218 fp=#0x10000001893' 2024-09-21T00:30:01.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43282 fp=#0x30000000f02' 2024-09-21T00:30:01.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43314 fp=#0x30000000f83' 2024-09-21T00:30:01.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43346 fp=#0x30000000f0b' 2024-09-21T00:30:01.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43442 fp=#0x300000023f9' 2024-09-21T00:30:01.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43474 fp=#0x300000023d3' 2024-09-21T00:30:01.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43506 fp=#0x300000023ca' 2024-09-21T00:30:01.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43538 fp=#0x300000023f1' 2024-09-21T00:30:01.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43570 fp=#0x300000023a2' 2024-09-21T00:30:01.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43602 fp=#0x300000010f1' 2024-09-21T00:30:01.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43634 fp=#0x300000010bf' 2024-09-21T00:30:01.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43698 fp=#0x30000002106' 2024-09-21T00:30:01.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43794 fp=#0x10000003a31' 2024-09-21T00:30:01.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43858 fp=#0x10000003a44' 2024-09-21T00:30:01.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43890 fp=#0x100000039dd' 2024-09-21T00:30:01.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43922 fp=#0x300000013af' 2024-09-21T00:30:01.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43954 fp=#0x30000001332' 2024-09-21T00:30:01.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31763 fp=#0x30000001417' 2024-09-21T00:30:01.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31827 fp=#0x30000000e59' 2024-09-21T00:30:01.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31955 fp=#0x30000000f9a' 2024-09-21T00:30:01.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34707 fp=#0x30000002236' 2024-09-21T00:30:01.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34739 fp=#0x3000000221a' 2024-09-21T00:30:01.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34867 fp=#0x100000027da' 2024-09-21T00:30:01.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34931 fp=#0x1000000270c' 2024-09-21T00:30:01.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34963 fp=#0x100000026e5' 2024-09-21T00:30:01.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34995 fp=#0x10000003891' 2024-09-21T00:30:01.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35027 fp=#0x10000003302' 2024-09-21T00:30:01.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35059 fp=#0x300000020bf' 2024-09-21T00:30:01.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35091 fp=#0x100000039aa' 2024-09-21T00:30:01.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35123 fp=#0x10000004213' 2024-09-21T00:30:01.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35155 fp=#0x100000039cd' 2024-09-21T00:30:01.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35187 fp=#0x10000003936' 2024-09-21T00:30:01.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35219 fp=#0x1000000413c' 2024-09-21T00:30:01.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35251 fp=#0x100000039b8' 2024-09-21T00:30:01.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35283 fp=#0x100000041d7' 2024-09-21T00:30:01.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35315 fp=#0x10000003acf' 2024-09-21T00:30:01.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35347 fp=#0x100000039fd' 2024-09-21T00:30:01.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35379 fp=#0x10000003969' 2024-09-21T00:30:01.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35443 fp=#0x100000038d2' 2024-09-21T00:30:01.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35475 fp=#0x30000001173' 2024-09-21T00:30:01.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35507 fp=#0x300000011a6' 2024-09-21T00:30:01.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35539 fp=#0x30000001142' 2024-09-21T00:30:01.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35571 fp=#0x30000001fe2' 2024-09-21T00:30:01.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35603 fp=#0x30000001ff6' 2024-09-21T00:30:01.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35667 fp=#0x30000001fe5' 2024-09-21T00:30:01.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35827 fp=#0x10000003376' 2024-09-21T00:30:01.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35859 fp=#0x100000033dc' 2024-09-21T00:30:01.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35891 fp=#0x100000033c9' 2024-09-21T00:30:01.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35923 fp=#0x300000011e6' 2024-09-21T00:30:01.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35955 fp=#0x300000011f8' 2024-09-21T00:30:01.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35987 fp=#0x10000003c28' 2024-09-21T00:30:01.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36083 fp=#0x10000002ce0' 2024-09-21T00:30:01.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36147 fp=#0x10000001ea4' 2024-09-21T00:30:01.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36179 fp=#0x10000001ea9' 2024-09-21T00:30:01.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36211 fp=#0x10000003225' 2024-09-21T00:30:01.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36243 fp=#0x10000001eb4' 2024-09-21T00:30:01.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36371 fp=#0x300000015ab' 2024-09-21T00:30:01.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36435 fp=#0x10000001e84' 2024-09-21T00:30:01.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36467 fp=#0x10000001e59' 2024-09-21T00:30:01.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36499 fp=#0x10000001e75' 2024-09-21T00:30:01.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36531 fp=#0x10000001f70' 2024-09-21T00:30:01.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36563 fp=#0x1000000444a' 2024-09-21T00:30:01.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36595 fp=#0x10000004420' 2024-09-21T00:30:01.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36627 fp=#0x1000000377c' 2024-09-21T00:30:01.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36659 fp=#0x10000004416' 2024-09-21T00:30:01.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36691 fp=#0x10000004412' 2024-09-21T00:30:01.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36723 fp=#0x100000047d6' 2024-09-21T00:30:01.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36755 fp=#0x100000047bc' 2024-09-21T00:30:01.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36787 fp=#0x1000000473d' 2024-09-21T00:30:01.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36819 fp=#0x1000000470a' 2024-09-21T00:30:01.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36851 fp=#0x1000000479a' 2024-09-21T00:30:01.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36883 fp=#0x100000025f1' 2024-09-21T00:30:01.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36915 fp=#0x1000000355c' 2024-09-21T00:30:01.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36947 fp=#0x1000000364d' 2024-09-21T00:30:01.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36979 fp=#0x1000000257c' 2024-09-21T00:30:01.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37011 fp=#0x100000035c9' 2024-09-21T00:30:01.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37107 fp=#0x30000001791' 2024-09-21T00:30:01.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37171 fp=#0x1000000300d' 2024-09-21T00:30:01.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37203 fp=#0x100000016a1' 2024-09-21T00:30:01.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37267 fp=#0x10000002fe7' 2024-09-21T00:30:01.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37331 fp=#0x30000001af9' 2024-09-21T00:30:01.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37363 fp=#0x30000001afb' 2024-09-21T00:30:01.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37395 fp=#0x30000000bee' 2024-09-21T00:30:01.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37427 fp=#0x30000000c1a' 2024-09-21T00:30:01.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37459 fp=#0x30000000bfb' 2024-09-21T00:30:01.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37491 fp=#0x30000000fe7' 2024-09-21T00:30:01.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37523 fp=#0x30000000fd9' 2024-09-21T00:30:01.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37555 fp=#0x30000001005' 2024-09-21T00:30:01.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37587 fp=#0x30000001b6a' 2024-09-21T00:30:01.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37651 fp=#0x30000000c8e' 2024-09-21T00:30:01.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37683 fp=#0x10000002939' 2024-09-21T00:30:01.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37715 fp=#0x30000000c4e' 2024-09-21T00:30:01.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37779 fp=#0x30000000c6b' 2024-09-21T00:30:01.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37843 fp=#0x300000012d0' 2024-09-21T00:30:01.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37971 fp=#0x30000000ce9' 2024-09-21T00:30:01.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38035 fp=#0x30000000ce3' 2024-09-21T00:30:01.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38067 fp=#0x30000000dba' 2024-09-21T00:30:01.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38099 fp=#0x10000000779' 2024-09-21T00:30:01.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38131 fp=#0x10000000e1f' 2024-09-21T00:30:01.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38195 fp=#0x1000000071c' 2024-09-21T00:30:01.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38227 fp=#0x10000000737' 2024-09-21T00:30:01.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38291 fp=#0x30000001a3d' 2024-09-21T00:30:01.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38323 fp=#0x30000001a6e' 2024-09-21T00:30:01.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38355 fp=#0x30000001a54' 2024-09-21T00:30:01.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38387 fp=#0x100000032fa' 2024-09-21T00:30:01.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38419 fp=#0x10000003304' 2024-09-21T00:30:01.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38451 fp=#0x100000032f6' 2024-09-21T00:30:01.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38483 fp=#0x100000032e0' 2024-09-21T00:30:01.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38515 fp=#0x100000020ed' 2024-09-21T00:30:01.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38579 fp=#0x100000020f1' 2024-09-21T00:30:01.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38611 fp=#0x100000005ef' 2024-09-21T00:30:01.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38643 fp=#0x100000005e8' 2024-09-21T00:30:01.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38675 fp=#0x100000005be' 2024-09-21T00:30:01.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38771 fp=#0x30000001c29' 2024-09-21T00:30:01.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38803 fp=#0x10000002973' 2024-09-21T00:30:01.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38867 fp=#0x10000002967' 2024-09-21T00:30:01.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38899 fp=#0x300000019f3' 2024-09-21T00:30:01.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38963 fp=#0x30000001d51' 2024-09-21T00:30:01.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39027 fp=#0x30000001d75' 2024-09-21T00:30:01.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39059 fp=#0x30000001d6c' 2024-09-21T00:30:01.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39187 fp=#0x10000001289' 2024-09-21T00:30:01.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39219 fp=#0x10000001272' 2024-09-21T00:30:01.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39283 fp=#0x1000000135c' 2024-09-21T00:30:01.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39315 fp=#0x10000003548' 2024-09-21T00:30:01.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39347 fp=#0x10000003f29' 2024-09-21T00:30:01.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39379 fp=#0x10000003ed7' 2024-09-21T00:30:01.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39411 fp=#0x10000003fb9' 2024-09-21T00:30:01.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39443 fp=#0x30000001ef8' 2024-09-21T00:30:01.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39475 fp=#0x30000001f45' 2024-09-21T00:30:01.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39507 fp=#0x30000001f5c' 2024-09-21T00:30:01.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39539 fp=#0x30000001f38' 2024-09-21T00:30:01.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39571 fp=#0x30000001f9f' 2024-09-21T00:30:01.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39603 fp=#0x10000002b2b' 2024-09-21T00:30:01.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39635 fp=#0x10000002b26' 2024-09-21T00:30:01.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39699 fp=#0x30000000e69' 2024-09-21T00:30:01.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39763 fp=#0x30000001989' 2024-09-21T00:30:01.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39795 fp=#0x30000000ea9' 2024-09-21T00:30:01.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39891 fp=#0x30000001113' 2024-09-21T00:30:01.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40019 fp=#0x3000000166c' 2024-09-21T00:30:01.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40051 fp=#0x300000014cb' 2024-09-21T00:30:01.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40147 fp=#0x30000001668' 2024-09-21T00:30:01.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40179 fp=#0x30000001810' 2024-09-21T00:30:01.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40211 fp=#0x10000001d1d' 2024-09-21T00:30:01.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40243 fp=#0x30000001d0c' 2024-09-21T00:30:01.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40275 fp=#0x10000001d5a' 2024-09-21T00:30:01.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40307 fp=#0x30000000d61' 2024-09-21T00:30:01.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40339 fp=#0x300000009e9' 2024-09-21T00:30:01.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40371 fp=#0x10000004c6f' 2024-09-21T00:30:01.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40403 fp=#0x10000004e5a' 2024-09-21T00:30:01.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40435 fp=#0x10000004cc1' 2024-09-21T00:30:01.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40467 fp=#0x10000004cf3' 2024-09-21T00:30:01.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40499 fp=#0x1000000465e' 2024-09-21T00:30:01.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40531 fp=#0x20000000607' 2024-09-21T00:30:01.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40595 fp=#0x10000000f77' 2024-09-21T00:30:01.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40627 fp=#0x20000000458' 2024-09-21T00:30:01.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40723 fp=#0x10000000f2b' 2024-09-21T00:30:01.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40755 fp=#0x20000000582' 2024-09-21T00:30:01.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40787 fp=#0x30000001449' 2024-09-21T00:30:01.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40819 fp=#0x30000001420' 2024-09-21T00:30:01.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40851 fp=#0x10000004ac2' 2024-09-21T00:30:01.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40883 fp=#0x10000004a9e' 2024-09-21T00:30:01.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40979 fp=#0x10000004c83' 2024-09-21T00:30:01.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41011 fp=#0x10000004ada' 2024-09-21T00:30:01.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41043 fp=#0x1000000369a' 2024-09-21T00:30:01.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41107 fp=#0x20000000be4' 2024-09-21T00:30:01.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41139 fp=#0x10000000dc2' 2024-09-21T00:30:01.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41171 fp=#0x20000000a93' 2024-09-21T00:30:01.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41203 fp=#0x20000000c4c' 2024-09-21T00:30:01.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41235 fp=#0x10000000da6' 2024-09-21T00:30:01.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41267 fp=#0x20000000001' 2024-09-21T00:30:01.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41299 fp=#0x2000000046a' 2024-09-21T00:30:01.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41331 fp=#0x20000000aab' 2024-09-21T00:30:01.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41363 fp=#0x10000000c5c' 2024-09-21T00:30:01.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41395 fp=#0x10000000c36' 2024-09-21T00:30:01.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41427 fp=#0x100000009db' 2024-09-21T00:30:01.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41459 fp=#0x1000000092d' 2024-09-21T00:30:01.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41491 fp=#0x1000000093e' 2024-09-21T00:30:01.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41523 fp=#0x100000009d0' 2024-09-21T00:30:01.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41555 fp=#0x10000000b4e' 2024-09-21T00:30:01.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41811 fp=#0x30000000890' 2024-09-21T00:30:01.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41843 fp=#0x300000009d5' 2024-09-21T00:30:01.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41875 fp=#0x30000000941' 2024-09-21T00:30:01.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41907 fp=#0x300000008c3' 2024-09-21T00:30:01.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41939 fp=#0x30000001387' 2024-09-21T00:30:01.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41971 fp=#0x300000009b9' 2024-09-21T00:30:01.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42003 fp=#0x30000000983' 2024-09-21T00:30:01.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42035 fp=#0x30000001d79' 2024-09-21T00:30:01.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42067 fp=#0x30000000871' 2024-09-21T00:30:01.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42099 fp=#0x3000000085d' 2024-09-21T00:30:01.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42131 fp=#0x30000000806' 2024-09-21T00:30:01.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42163 fp=#0x30000000b62' 2024-09-21T00:30:01.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42195 fp=#0x30000000b85' 2024-09-21T00:30:01.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42227 fp=#0x30000000bd3' 2024-09-21T00:30:01.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42259 fp=#0x30000001b8e' 2024-09-21T00:30:01.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42291 fp=#0x30000001bc8' 2024-09-21T00:30:01.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42323 fp=#0x30000000b47' 2024-09-21T00:30:01.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42355 fp=#0x30000000b29' 2024-09-21T00:30:01.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42387 fp=#0x30000000b49' 2024-09-21T00:30:01.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42419 fp=#0x30000000aeb' 2024-09-21T00:30:01.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42451 fp=#0x1000000414c' 2024-09-21T00:30:01.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42515 fp=#0x30000000a5f' 2024-09-21T00:30:01.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42579 fp=#0x10000004127' 2024-09-21T00:30:01.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42643 fp=#0x30000001264' 2024-09-21T00:30:01.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42675 fp=#0x10000001940' 2024-09-21T00:30:01.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42707 fp=#0x100000018c5' 2024-09-21T00:30:01.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42803 fp=#0x100000019b2' 2024-09-21T00:30:01.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43059 fp=#0x300000016dd' 2024-09-21T00:30:01.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43091 fp=#0x30000001692' 2024-09-21T00:30:01.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43123 fp=#0x300000016ab' 2024-09-21T00:30:01.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43219 fp=#0x10000001874' 2024-09-21T00:30:01.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43283 fp=#0x30000000f4b' 2024-09-21T00:30:01.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43315 fp=#0x30000000f6f' 2024-09-21T00:30:01.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43347 fp=#0x30000000f49' 2024-09-21T00:30:01.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43411 fp=#0x1000000418f' 2024-09-21T00:30:01.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43443 fp=#0x300000023da' 2024-09-21T00:30:01.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43475 fp=#0x300000023b9' 2024-09-21T00:30:01.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43507 fp=#0x300000023bf' 2024-09-21T00:30:01.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43539 fp=#0x300000023d1' 2024-09-21T00:30:01.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43571 fp=#0x300000023a8' 2024-09-21T00:30:01.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43603 fp=#0x3000000110c' 2024-09-21T00:30:01.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43635 fp=#0x30000000f9f' 2024-09-21T00:30:01.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43699 fp=#0x30000000fdf' 2024-09-21T00:30:01.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43731 fp=#0x30000002126' 2024-09-21T00:30:01.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43763 fp=#0x3000000232e' 2024-09-21T00:30:01.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43795 fp=#0x10000003a0b' 2024-09-21T00:30:01.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43827 fp=#0x100000039d3' 2024-09-21T00:30:01.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43891 fp=#0x100000039c9' 2024-09-21T00:30:01.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43923 fp=#0x30000001335' 2024-09-21T00:30:01.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43955 fp=#0x30000001343' 2024-09-21T00:30:01.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_path:mds.2:31733 fp=#0x1' 2024-09-21T00:30:01.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31764 fp=#0x10000001e8f' 2024-09-21T00:30:01.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31956 fp=#0x10000002989' 2024-09-21T00:30:01.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34676 fp=#0x3000000221d' 2024-09-21T00:30:01.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34708 fp=#0x300000022b3' 2024-09-21T00:30:01.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34772 fp=#0x10000000c60' 2024-09-21T00:30:01.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34996 fp=#0x10000003837' 2024-09-21T00:30:01.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35028 fp=#0x10000003873' 2024-09-21T00:30:01.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35060 fp=#0x100000032cf' 2024-09-21T00:30:01.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35092 fp=#0x1000000423c' 2024-09-21T00:30:01.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35124 fp=#0x100000039ca' 2024-09-21T00:30:01.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35156 fp=#0x10000004199' 2024-09-21T00:30:01.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35188 fp=#0x100000041ff' 2024-09-21T00:30:01.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35220 fp=#0x10000003abc' 2024-09-21T00:30:01.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35252 fp=#0x10000003abe' 2024-09-21T00:30:01.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35284 fp=#0x100000038fa' 2024-09-21T00:30:01.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35316 fp=#0x10000003917' 2024-09-21T00:30:01.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35348 fp=#0x10000003add' 2024-09-21T00:30:01.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35380 fp=#0x100000039ac' 2024-09-21T00:30:01.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35412 fp=#0x300000013d8' 2024-09-21T00:30:01.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35444 fp=#0x10000003999' 2024-09-21T00:30:01.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35476 fp=#0x3000000116a' 2024-09-21T00:30:01.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35540 fp=#0x3000000117d' 2024-09-21T00:30:01.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35572 fp=#0x30000001fde' 2024-09-21T00:30:01.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35604 fp=#0x30000001fed' 2024-09-21T00:30:01.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35636 fp=#0x30000001fee' 2024-09-21T00:30:01.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35796 fp=#0x10000003388' 2024-09-21T00:30:01.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35828 fp=#0x1000000345d' 2024-09-21T00:30:01.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35924 fp=#0x30000001206' 2024-09-21T00:30:01.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35956 fp=#0x300000011d2' 2024-09-21T00:30:01.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36084 fp=#0x10000003627' 2024-09-21T00:30:01.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36116 fp=#0x10000002d58' 2024-09-21T00:30:01.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36148 fp=#0x10000001f51' 2024-09-21T00:30:01.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36180 fp=#0x100000020a2' 2024-09-21T00:30:01.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36244 fp=#0x10000001f0b' 2024-09-21T00:30:01.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36372 fp=#0x30000001601' 2024-09-21T00:30:01.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36436 fp=#0x10000001f58' 2024-09-21T00:30:01.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36468 fp=#0x10000001dfb' 2024-09-21T00:30:01.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36532 fp=#0x10000001da1' 2024-09-21T00:30:01.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36564 fp=#0x100000037af' 2024-09-21T00:30:01.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36596 fp=#0x100000037f9' 2024-09-21T00:30:01.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36628 fp=#0x10000003842' 2024-09-21T00:30:01.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36660 fp=#0x100000037bf' 2024-09-21T00:30:01.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36692 fp=#0x1000000447a' 2024-09-21T00:30:01.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36724 fp=#0x1000000470e' 2024-09-21T00:30:01.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36756 fp=#0x10000004713' 2024-09-21T00:30:01.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36788 fp=#0x30000001a78' 2024-09-21T00:30:01.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36820 fp=#0x10000004776' 2024-09-21T00:30:01.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36852 fp=#0x100000047a5' 2024-09-21T00:30:01.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36884 fp=#0x10000002586' 2024-09-21T00:30:01.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36980 fp=#0x1000000253d' 2024-09-21T00:30:01.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37012 fp=#0x10000003507' 2024-09-21T00:30:01.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37044 fp=#0x10000002550' 2024-09-21T00:30:01.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37172 fp=#0x1000000147e' 2024-09-21T00:30:01.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37204 fp=#0x10000002fd2' 2024-09-21T00:30:01.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37236 fp=#0x10000002fcd' 2024-09-21T00:30:01.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37300 fp=#0x1000000136b' 2024-09-21T00:30:01.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37332 fp=#0x30000001ac4' 2024-09-21T00:30:01.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37364 fp=#0x30000000c17' 2024-09-21T00:30:01.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37396 fp=#0x30000001aee' 2024-09-21T00:30:01.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37428 fp=#0x30000001ae4' 2024-09-21T00:30:01.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37460 fp=#0x30000000bfe' 2024-09-21T00:30:01.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37524 fp=#0x30000000fca' 2024-09-21T00:30:01.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37556 fp=#0x30000001b7f' 2024-09-21T00:30:01.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37588 fp=#0x30000000fd1' 2024-09-21T00:30:01.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37620 fp=#0x3000000106b' 2024-09-21T00:30:01.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37652 fp=#0x10000002841' 2024-09-21T00:30:01.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37780 fp=#0x1000000284e' 2024-09-21T00:30:01.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37812 fp=#0x10000002836' 2024-09-21T00:30:01.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37844 fp=#0x300000012cf' 2024-09-21T00:30:01.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37876 fp=#0x30000001268' 2024-09-21T00:30:01.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38036 fp=#0x30000000cd4' 2024-09-21T00:30:01.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38132 fp=#0x10000000772' 2024-09-21T00:30:01.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38164 fp=#0x10000000ec0' 2024-09-21T00:30:01.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38196 fp=#0x1000000071b' 2024-09-21T00:30:01.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38228 fp=#0x10000001663' 2024-09-21T00:30:01.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38292 fp=#0x100000031c5' 2024-09-21T00:30:01.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38324 fp=#0x30000001a8b' 2024-09-21T00:30:01.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38356 fp=#0x10000002459' 2024-09-21T00:30:01.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38388 fp=#0x10000003343' 2024-09-21T00:30:01.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38420 fp=#0x100000032f3' 2024-09-21T00:30:01.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38484 fp=#0x10000003287' 2024-09-21T00:30:01.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38516 fp=#0x1000000214b' 2024-09-21T00:30:01.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38548 fp=#0x10000002180' 2024-09-21T00:30:01.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38580 fp=#0x100000020ff' 2024-09-21T00:30:01.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38612 fp=#0x30000001c36' 2024-09-21T00:30:01.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38676 fp=#0x30000001bde' 2024-09-21T00:30:01.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38708 fp=#0x30000001c00' 2024-09-21T00:30:01.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38836 fp=#0x300000019bf' 2024-09-21T00:30:01.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38868 fp=#0x10000002958' 2024-09-21T00:30:01.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38900 fp=#0x300000019eb' 2024-09-21T00:30:01.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38964 fp=#0x30000001c79' 2024-09-21T00:30:01.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38996 fp=#0x30000001d78' 2024-09-21T00:30:01.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39028 fp=#0x30000001d19' 2024-09-21T00:30:01.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39188 fp=#0x1000000355f' 2024-09-21T00:30:01.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39252 fp=#0x100000034cf' 2024-09-21T00:30:01.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39316 fp=#0x1000000352b' 2024-09-21T00:30:01.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39348 fp=#0x10000003f6c' 2024-09-21T00:30:01.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39380 fp=#0x10000003f3a' 2024-09-21T00:30:01.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39412 fp=#0x10000003f53' 2024-09-21T00:30:01.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39444 fp=#0x30000001f31' 2024-09-21T00:30:01.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39476 fp=#0x30000001fa2' 2024-09-21T00:30:01.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39508 fp=#0x30000001f08' 2024-09-21T00:30:01.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39540 fp=#0x30000001f33' 2024-09-21T00:30:01.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39572 fp=#0x30000001ef9' 2024-09-21T00:30:01.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39604 fp=#0x10000002b28' 2024-09-21T00:30:01.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39636 fp=#0x10000002ab8' 2024-09-21T00:30:01.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39700 fp=#0x30000000eba' 2024-09-21T00:30:01.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39732 fp=#0x30000000eb3' 2024-09-21T00:30:01.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39828 fp=#0x30000000ea4' 2024-09-21T00:30:01.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39860 fp=#0x30000000e6e' 2024-09-21T00:30:01.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39892 fp=#0x30000001103' 2024-09-21T00:30:01.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39956 fp=#0x3000000101d' 2024-09-21T00:30:01.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39988 fp=#0x30000000fff' 2024-09-21T00:30:01.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40020 fp=#0x300000014bf' 2024-09-21T00:30:01.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40052 fp=#0x30000001667' 2024-09-21T00:30:01.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40084 fp=#0x3000000166b' 2024-09-21T00:30:01.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40116 fp=#0x300000014c1' 2024-09-21T00:30:01.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40180 fp=#0x3000000150d' 2024-09-21T00:30:01.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40212 fp=#0x30000000db1' 2024-09-21T00:30:01.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40244 fp=#0x30000000d6c' 2024-09-21T00:30:01.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40276 fp=#0x10000001cff' 2024-09-21T00:30:01.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40308 fp=#0x30000000d78' 2024-09-21T00:30:01.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40340 fp=#0x30000000d9f' 2024-09-21T00:30:01.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40372 fp=#0x10000004e7e' 2024-09-21T00:30:01.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40404 fp=#0x300000022da' 2024-09-21T00:30:01.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40436 fp=#0x10000004c64' 2024-09-21T00:30:01.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40468 fp=#0x10000004e48' 2024-09-21T00:30:01.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40532 fp=#0x10000000fb0' 2024-09-21T00:30:01.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40564 fp=#0x20000000566' 2024-09-21T00:30:01.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40596 fp=#0x10000000cdc' 2024-09-21T00:30:01.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40692 fp=#0x10000000ef1' 2024-09-21T00:30:01.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40724 fp=#0x10000000f04' 2024-09-21T00:30:01.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40756 fp=#0x20000000448' 2024-09-21T00:30:01.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40788 fp=#0x30000001458' 2024-09-21T00:30:01.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40820 fp=#0x3000000144d' 2024-09-21T00:30:01.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40852 fp=#0x10000004ad0' 2024-09-21T00:30:01.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40884 fp=#0x10000004c77' 2024-09-21T00:30:01.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40948 fp=#0x10000004ab5' 2024-09-21T00:30:01.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40980 fp=#0x10000004b08' 2024-09-21T00:30:01.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41044 fp=#0x100000034d2' 2024-09-21T00:30:01.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41076 fp=#0x10000003619' 2024-09-21T00:30:01.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41108 fp=#0x10000000db0' 2024-09-21T00:30:01.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41140 fp=#0x20000000bbe' 2024-09-21T00:30:01.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41172 fp=#0x10000000dd2' 2024-09-21T00:30:01.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41204 fp=#0x10000000cb4' 2024-09-21T00:30:01.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41236 fp=#0x20000000331' 2024-09-21T00:30:01.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41268 fp=#0x10000000ee5' 2024-09-21T00:30:01.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41300 fp=#0x20000000bd7' 2024-09-21T00:30:01.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41332 fp=#0x10000000dc7' 2024-09-21T00:30:01.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41364 fp=#0x1000000092f' 2024-09-21T00:30:01.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41428 fp=#0x10000000b61' 2024-09-21T00:30:01.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41460 fp=#0x10000000b18' 2024-09-21T00:30:01.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41524 fp=#0x10000000c3d' 2024-09-21T00:30:01.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41556 fp=#0x100000009a6' 2024-09-21T00:30:01.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41620 fp=#0x30000001918' 2024-09-21T00:30:01.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41684 fp=#0x300000007b6' 2024-09-21T00:30:01.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41812 fp=#0x300000009d4' 2024-09-21T00:30:01.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41876 fp=#0x30000000991' 2024-09-21T00:30:01.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41940 fp=#0x3000000087e' 2024-09-21T00:30:01.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41972 fp=#0x30000000985' 2024-09-21T00:30:01.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42004 fp=#0x300000009a0' 2024-09-21T00:30:01.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42036 fp=#0x300000007d5' 2024-09-21T00:30:01.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42068 fp=#0x3000000080e' 2024-09-21T00:30:01.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42100 fp=#0x300000009aa' 2024-09-21T00:30:01.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42132 fp=#0x3000000098f' 2024-09-21T00:30:01.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42164 fp=#0x30000000bd6' 2024-09-21T00:30:01.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42196 fp=#0x30000002089' 2024-09-21T00:30:01.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42228 fp=#0x30000002085' 2024-09-21T00:30:01.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42260 fp=#0x30000000bab' 2024-09-21T00:30:01.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42292 fp=#0x30000001bbf' 2024-09-21T00:30:01.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42324 fp=#0x30000000b42' 2024-09-21T00:30:01.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42388 fp=#0x30000000b09' 2024-09-21T00:30:01.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42420 fp=#0x30000000ae6' 2024-09-21T00:30:01.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42484 fp=#0x100000040f5' 2024-09-21T00:30:01.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42516 fp=#0x30000000a79' 2024-09-21T00:30:01.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42548 fp=#0x30000000a58' 2024-09-21T00:30:01.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42580 fp=#0x30000001b3e' 2024-09-21T00:30:01.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42644 fp=#0x30000001265' 2024-09-21T00:30:01.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42676 fp=#0x10000001a39' 2024-09-21T00:30:01.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42772 fp=#0x100000018d5' 2024-09-21T00:30:01.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42804 fp=#0x100000019e9' 2024-09-21T00:30:01.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42836 fp=#0x100000018ec' 2024-09-21T00:30:01.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42868 fp=#0x1000000391b' 2024-09-21T00:30:01.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42900 fp=#0x100000019fd' 2024-09-21T00:30:01.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42932 fp=#0x30000000e00' 2024-09-21T00:30:01.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42964 fp=#0x30000001e72' 2024-09-21T00:30:01.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42996 fp=#0x30000000df4' 2024-09-21T00:30:01.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43028 fp=#0x30000001762' 2024-09-21T00:30:01.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43060 fp=#0x3000000168b' 2024-09-21T00:30:01.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43092 fp=#0x30000001690' 2024-09-21T00:30:01.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43124 fp=#0x300000016df' 2024-09-21T00:30:01.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43156 fp=#0x1000000187e' 2024-09-21T00:30:01.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43252 fp=#0x10000001878' 2024-09-21T00:30:01.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43284 fp=#0x30000000f84' 2024-09-21T00:30:01.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43380 fp=#0x30000000f6a' 2024-09-21T00:30:01.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43412 fp=#0x10000004189' 2024-09-21T00:30:01.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43444 fp=#0x300000023fb' 2024-09-21T00:30:01.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43476 fp=#0x300000023d6' 2024-09-21T00:30:01.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43508 fp=#0x300000023cf' 2024-09-21T00:30:01.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43540 fp=#0x30000002389' 2024-09-21T00:30:01.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43572 fp=#0x300000023dd' 2024-09-21T00:30:01.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43604 fp=#0x3000000218e' 2024-09-21T00:30:01.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43636 fp=#0x300000020fd' 2024-09-21T00:30:01.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43700 fp=#0x3000000211f' 2024-09-21T00:30:01.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43732 fp=#0x30000000f9b' 2024-09-21T00:30:01.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43764 fp=#0x30000002127' 2024-09-21T00:30:01.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43796 fp=#0x100000039d0' 2024-09-21T00:30:01.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43828 fp=#0x10000003979' 2024-09-21T00:30:01.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43924 fp=#0x300000013d2' 2024-09-21T00:30:01.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43956 fp=#0x3000000134e' 2024-09-21T00:30:01.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31734 fp=#0x1' 2024-09-21T00:30:01.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31765 fp=#0x300000015a1' 2024-09-21T00:30:01.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34677 fp=#0x10000000c87' 2024-09-21T00:30:01.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34709 fp=#0x3000000222a' 2024-09-21T00:30:01.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34741 fp=#0x30000002222' 2024-09-21T00:30:01.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34773 fp=#0x300000022af' 2024-09-21T00:30:01.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34997 fp=#0x10000003293' 2024-09-21T00:30:01.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35029 fp=#0x300000020ba' 2024-09-21T00:30:01.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35061 fp=#0x30000002063' 2024-09-21T00:30:01.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35093 fp=#0x10000003b09' 2024-09-21T00:30:01.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35125 fp=#0x3000000134a' 2024-09-21T00:30:01.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35157 fp=#0x1000000419c' 2024-09-21T00:30:01.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35189 fp=#0x10000004248' 2024-09-21T00:30:01.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35221 fp=#0x10000004144' 2024-09-21T00:30:01.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35253 fp=#0x10000003ac8' 2024-09-21T00:30:01.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35285 fp=#0x10000003a09' 2024-09-21T00:30:01.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35317 fp=#0x100000038bf' 2024-09-21T00:30:01.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35349 fp=#0x1000000391a' 2024-09-21T00:30:01.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35381 fp=#0x10000003975' 2024-09-21T00:30:01.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35413 fp=#0x10000003878' 2024-09-21T00:30:01.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35445 fp=#0x100000038cb' 2024-09-21T00:30:01.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35477 fp=#0x30000001183' 2024-09-21T00:30:01.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35509 fp=#0x30000001194' 2024-09-21T00:30:01.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35541 fp=#0x3000000111a' 2024-09-21T00:30:01.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35605 fp=#0x30000001ffe' 2024-09-21T00:30:01.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35669 fp=#0x30000002048' 2024-09-21T00:30:01.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35797 fp=#0x10000003463' 2024-09-21T00:30:01.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35829 fp=#0x1000000343b' 2024-09-21T00:30:01.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35861 fp=#0x10000003398' 2024-09-21T00:30:01.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35893 fp=#0x100000033b9' 2024-09-21T00:30:01.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35925 fp=#0x300000011fd' 2024-09-21T00:30:01.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35957 fp=#0x300000011d1' 2024-09-21T00:30:01.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36117 fp=#0x10000002cc6' 2024-09-21T00:30:01.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36149 fp=#0x1000000208f' 2024-09-21T00:30:01.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36181 fp=#0x1000000205e' 2024-09-21T00:30:01.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36245 fp=#0x10000002058' 2024-09-21T00:30:01.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36437 fp=#0x10000001e88' 2024-09-21T00:30:01.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36469 fp=#0x10000001e4d' 2024-09-21T00:30:01.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36501 fp=#0x10000001e04' 2024-09-21T00:30:01.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36565 fp=#0x100000044af' 2024-09-21T00:30:01.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36597 fp=#0x100000037ec' 2024-09-21T00:30:01.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36629 fp=#0x10000004483' 2024-09-21T00:30:01.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36661 fp=#0x1000000443c' 2024-09-21T00:30:01.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36693 fp=#0x1000000443a' 2024-09-21T00:30:01.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36725 fp=#0x1000000479f' 2024-09-21T00:30:01.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36757 fp=#0x10000004706' 2024-09-21T00:30:01.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36789 fp=#0x30000001aa7' 2024-09-21T00:30:01.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36821 fp=#0x100000046fc' 2024-09-21T00:30:01.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36853 fp=#0x10000004720' 2024-09-21T00:30:01.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36885 fp=#0x10000003367' 2024-09-21T00:30:01.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36949 fp=#0x1000000368b' 2024-09-21T00:30:01.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37045 fp=#0x100000035a2' 2024-09-21T00:30:01.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37173 fp=#0x100000014a8' 2024-09-21T00:30:01.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37205 fp=#0x10000003022' 2024-09-21T00:30:01.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37301 fp=#0x1000000313a' 2024-09-21T00:30:01.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37333 fp=#0x30000000c2b' 2024-09-21T00:30:01.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37365 fp=#0x30000001acd' 2024-09-21T00:30:01.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37397 fp=#0x30000001ab7' 2024-09-21T00:30:01.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37429 fp=#0x30000000bf2' 2024-09-21T00:30:01.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37461 fp=#0x30000000c18' 2024-09-21T00:30:01.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37525 fp=#0x3000000110d' 2024-09-21T00:30:01.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37557 fp=#0x30000001135' 2024-09-21T00:30:01.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37589 fp=#0x300000010a9' 2024-09-21T00:30:01.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37621 fp=#0x30000000fe2' 2024-09-21T00:30:01.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37717 fp=#0x30000000c67' 2024-09-21T00:30:01.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37845 fp=#0x300000012d8' 2024-09-21T00:30:01.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37877 fp=#0x300000012bb' 2024-09-21T00:30:01.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38005 fp=#0x30000000dce' 2024-09-21T00:30:01.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38037 fp=#0x30000000cb9' 2024-09-21T00:30:01.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38069 fp=#0x30000000cc0' 2024-09-21T00:30:01.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38101 fp=#0x10000000741' 2024-09-21T00:30:01.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38133 fp=#0x10000000760' 2024-09-21T00:30:01.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38197 fp=#0x10000000764' 2024-09-21T00:30:01.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38229 fp=#0x3000000048e' 2024-09-21T00:30:01.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38325 fp=#0x30000001a58' 2024-09-21T00:30:01.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38357 fp=#0x30000001a65' 2024-09-21T00:30:01.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38389 fp=#0x10000003349' 2024-09-21T00:30:01.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38421 fp=#0x10000003313' 2024-09-21T00:30:01.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38453 fp=#0x10000003303' 2024-09-21T00:30:01.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38485 fp=#0x100000032da' 2024-09-21T00:30:01.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38517 fp=#0x1000000216d' 2024-09-21T00:30:01.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38549 fp=#0x10000002125' 2024-09-21T00:30:01.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38581 fp=#0x10000002140' 2024-09-21T00:30:01.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38613 fp=#0x100000005ae' 2024-09-21T00:30:01.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38677 fp=#0x10000000ac2' 2024-09-21T00:30:01.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38741 fp=#0x30000001c1e' 2024-09-21T00:30:01.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38805 fp=#0x1000000296d' 2024-09-21T00:30:01.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38837 fp=#0x300000019ee' 2024-09-21T00:30:01.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38869 fp=#0x300000019ef' 2024-09-21T00:30:01.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38901 fp=#0x10000002933' 2024-09-21T00:30:01.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38933 fp=#0x30000001c65' 2024-09-21T00:30:01.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38965 fp=#0x30000001d3f' 2024-09-21T00:30:01.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39029 fp=#0x30000001c28' 2024-09-21T00:30:01.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39061 fp=#0x30000001c5f' 2024-09-21T00:30:01.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39189 fp=#0x10000001111' 2024-09-21T00:30:01.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39349 fp=#0x10000003f5d' 2024-09-21T00:30:01.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39381 fp=#0x3000000135f' 2024-09-21T00:30:01.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39413 fp=#0x10000003fc8' 2024-09-21T00:30:01.914 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39445 fp=#0x30000001fa8' 2024-09-21T00:30:01.914 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39477 fp=#0x30000001f95' 2024-09-21T00:30:01.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39509 fp=#0x30000001f0d' 2024-09-21T00:30:01.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39541 fp=#0x30000001f0c' 2024-09-21T00:30:01.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39573 fp=#0x30000001f1b' 2024-09-21T00:30:01.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39605 fp=#0x10000002b43' 2024-09-21T00:30:01.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39701 fp=#0x30000000ef0' 2024-09-21T00:30:01.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39797 fp=#0x30000001992' 2024-09-21T00:30:01.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39829 fp=#0x30000000eeb' 2024-09-21T00:30:01.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39861 fp=#0x300000019a8' 2024-09-21T00:30:01.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39893 fp=#0x30000000ff7' 2024-09-21T00:30:01.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40053 fp=#0x30000001530' 2024-09-21T00:30:01.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40085 fp=#0x30000001552' 2024-09-21T00:30:01.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40117 fp=#0x30000001814' 2024-09-21T00:30:01.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40149 fp=#0x30000001673' 2024-09-21T00:30:01.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40181 fp=#0x3000000151a' 2024-09-21T00:30:01.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40213 fp=#0x30000000dc1' 2024-09-21T00:30:01.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40245 fp=#0x30000000dcd' 2024-09-21T00:30:01.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40277 fp=#0x30000000daf' 2024-09-21T00:30:01.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40309 fp=#0x300000009f2' 2024-09-21T00:30:01.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40341 fp=#0x30000000d6f' 2024-09-21T00:30:01.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40373 fp=#0x10000004cbb' 2024-09-21T00:30:01.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40405 fp=#0x300000022d7' 2024-09-21T00:30:01.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40437 fp=#0x30000002325' 2024-09-21T00:30:01.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40469 fp=#0x10000004e2e' 2024-09-21T00:30:01.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40533 fp=#0x200000005b5' 2024-09-21T00:30:01.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40565 fp=#0x2000000056e' 2024-09-21T00:30:01.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40597 fp=#0x20000000450' 2024-09-21T00:30:01.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40629 fp=#0x20000000579' 2024-09-21T00:30:01.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40661 fp=#0x10000000deb' 2024-09-21T00:30:01.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40693 fp=#0x200000008fb' 2024-09-21T00:30:01.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40725 fp=#0x10000000d1c' 2024-09-21T00:30:01.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40757 fp=#0x20000000596' 2024-09-21T00:30:01.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40789 fp=#0x30000001431' 2024-09-21T00:30:01.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40821 fp=#0x30000001438' 2024-09-21T00:30:01.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40853 fp=#0x10000004c7d' 2024-09-21T00:30:01.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40885 fp=#0x10000004939' 2024-09-21T00:30:01.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41013 fp=#0x20000000701' 2024-09-21T00:30:01.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41045 fp=#0x10000003508' 2024-09-21T00:30:01.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41077 fp=#0x200000006f9' 2024-09-21T00:30:01.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41109 fp=#0x2000000020e' 2024-09-21T00:30:01.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41141 fp=#0x20000000bf9' 2024-09-21T00:30:01.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41173 fp=#0x10000000da1' 2024-09-21T00:30:01.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41205 fp=#0x20000000c2b' 2024-09-21T00:30:01.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41237 fp=#0x10000000e83' 2024-09-21T00:30:01.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41269 fp=#0x20000000b32' 2024-09-21T00:30:01.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41301 fp=#0x20000000abc' 2024-09-21T00:30:01.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41333 fp=#0x10000000de4' 2024-09-21T00:30:01.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41365 fp=#0x10000000dda' 2024-09-21T00:30:01.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41397 fp=#0x10000000a5a' 2024-09-21T00:30:01.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41429 fp=#0x10000000a9d' 2024-09-21T00:30:01.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41461 fp=#0x10000000c16' 2024-09-21T00:30:01.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41493 fp=#0x10000000c44' 2024-09-21T00:30:01.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41525 fp=#0x10000000c7f' 2024-09-21T00:30:01.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41557 fp=#0x100000009c6' 2024-09-21T00:30:01.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41685 fp=#0x3000000090a' 2024-09-21T00:30:01.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41717 fp=#0x300000007cf' 2024-09-21T00:30:01.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41813 fp=#0x3000000094e' 2024-09-21T00:30:01.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41845 fp=#0x300000009c2' 2024-09-21T00:30:01.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41877 fp=#0x30000000956' 2024-09-21T00:30:01.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41909 fp=#0x300000008b6' 2024-09-21T00:30:01.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41941 fp=#0x300000009c1' 2024-09-21T00:30:01.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41973 fp=#0x300000009bd' 2024-09-21T00:30:01.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42037 fp=#0x30000001d71' 2024-09-21T00:30:01.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42069 fp=#0x30000001d4c' 2024-09-21T00:30:01.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42101 fp=#0x3000000097a' 2024-09-21T00:30:01.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42165 fp=#0x30000000b8e' 2024-09-21T00:30:01.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42197 fp=#0x30000002088' 2024-09-21T00:30:01.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42229 fp=#0x30000001b92' 2024-09-21T00:30:01.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42261 fp=#0x30000000b6f' 2024-09-21T00:30:01.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42293 fp=#0x30000000b8c' 2024-09-21T00:30:01.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42325 fp=#0x30000000afc' 2024-09-21T00:30:01.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42357 fp=#0x30000000b2c' 2024-09-21T00:30:01.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42389 fp=#0x300000015ae' 2024-09-21T00:30:01.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42421 fp=#0x30000000b36' 2024-09-21T00:30:01.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42453 fp=#0x10000004151' 2024-09-21T00:30:01.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42485 fp=#0x30000000a5b' 2024-09-21T00:30:01.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42517 fp=#0x30000001b44' 2024-09-21T00:30:01.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42549 fp=#0x30000000a5a' 2024-09-21T00:30:01.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42645 fp=#0x30000001257' 2024-09-21T00:30:01.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42709 fp=#0x100000019f1' 2024-09-21T00:30:01.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42741 fp=#0x10000001955' 2024-09-21T00:30:01.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42773 fp=#0x100000018df' 2024-09-21T00:30:01.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42901 fp=#0x100000018cd' 2024-09-21T00:30:01.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42997 fp=#0x30000000e42' 2024-09-21T00:30:01.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43029 fp=#0x300000016b6' 2024-09-21T00:30:01.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43061 fp=#0x300000016ea' 2024-09-21T00:30:01.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43093 fp=#0x30000001746' 2024-09-21T00:30:01.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43125 fp=#0x30000001707' 2024-09-21T00:30:01.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43221 fp=#0x10000001895' 2024-09-21T00:30:01.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43253 fp=#0x1000000188f' 2024-09-21T00:30:01.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43317 fp=#0x30000000f32' 2024-09-21T00:30:01.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43349 fp=#0x30000000f31' 2024-09-21T00:30:01.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43381 fp=#0x30000001ef4' 2024-09-21T00:30:01.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43413 fp=#0x10000004187' 2024-09-21T00:30:01.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43445 fp=#0x30000002373' 2024-09-21T00:30:01.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43477 fp=#0x30000002382' 2024-09-21T00:30:01.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43509 fp=#0x300000023fa' 2024-09-21T00:30:01.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43541 fp=#0x300000023b5' 2024-09-21T00:30:01.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43573 fp=#0x3000000237a' 2024-09-21T00:30:01.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43605 fp=#0x30000001003' 2024-09-21T00:30:01.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43669 fp=#0x30000002111' 2024-09-21T00:30:01.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43701 fp=#0x300000020ff' 2024-09-21T00:30:01.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43733 fp=#0x300000010d6' 2024-09-21T00:30:01.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43829 fp=#0x10000003a4b' 2024-09-21T00:30:01.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43861 fp=#0x1000000298b' 2024-09-21T00:30:01.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43925 fp=#0x30000001376' 2024-09-21T00:30:01.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43957 fp=#0x30000001352' 2024-09-21T00:30:01.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31735 fp=#0x10000000000' 2024-09-21T00:30:01.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31766 fp=#0x10000001d35' 2024-09-21T00:30:01.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31926 fp=#0x10000000923' 2024-09-21T00:30:01.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34678 fp=#0x30000002281' 2024-09-21T00:30:01.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34742 fp=#0x10000000b4f' 2024-09-21T00:30:01.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34774 fp=#0x30000002249' 2024-09-21T00:30:01.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34870 fp=#0x10000002703' 2024-09-21T00:30:01.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34902 fp=#0x1000000281b' 2024-09-21T00:30:01.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34998 fp=#0x1000000383b' 2024-09-21T00:30:01.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35030 fp=#0x100000032f4' 2024-09-21T00:30:01.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35062 fp=#0x3000000206c' 2024-09-21T00:30:01.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35094 fp=#0x10000003948' 2024-09-21T00:30:01.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35126 fp=#0x30000001ef3' 2024-09-21T00:30:01.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35158 fp=#0x10000003adf' 2024-09-21T00:30:01.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35190 fp=#0x100000041a6' 2024-09-21T00:30:01.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35222 fp=#0x1000000422a' 2024-09-21T00:30:01.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35254 fp=#0x100000038ca' 2024-09-21T00:30:01.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35286 fp=#0x10000003b03' 2024-09-21T00:30:01.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35318 fp=#0x1000000421e' 2024-09-21T00:30:01.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35350 fp=#0x10000003ab7' 2024-09-21T00:30:01.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35382 fp=#0x10000003946' 2024-09-21T00:30:01.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35414 fp=#0x1000000390e' 2024-09-21T00:30:01.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35446 fp=#0x100000038d6' 2024-09-21T00:30:01.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35478 fp=#0x30000001152' 2024-09-21T00:30:01.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35510 fp=#0x30000001150' 2024-09-21T00:30:01.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35542 fp=#0x30000001169' 2024-09-21T00:30:01.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35574 fp=#0x30000002056' 2024-09-21T00:30:01.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35606 fp=#0x30000001fd9' 2024-09-21T00:30:01.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35638 fp=#0x30000002035' 2024-09-21T00:30:01.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35670 fp=#0x30000001fe8' 2024-09-21T00:30:01.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35798 fp=#0x1000000345c' 2024-09-21T00:30:01.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35830 fp=#0x10000003433' 2024-09-21T00:30:01.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35894 fp=#0x10000003406' 2024-09-21T00:30:01.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35926 fp=#0x300000011d4' 2024-09-21T00:30:01.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35958 fp=#0x300000011fb' 2024-09-21T00:30:01.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36022 fp=#0x10000003c25' 2024-09-21T00:30:01.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36054 fp=#0x10000003b56' 2024-09-21T00:30:01.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36086 fp=#0x10000002cce' 2024-09-21T00:30:01.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36118 fp=#0x10000002d60' 2024-09-21T00:30:01.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36150 fp=#0x10000001ebc' 2024-09-21T00:30:01.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36182 fp=#0x10000001f1e' 2024-09-21T00:30:01.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36214 fp=#0x10000001eee' 2024-09-21T00:30:01.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36246 fp=#0x10000001f04' 2024-09-21T00:30:01.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36310 fp=#0x30000001608' 2024-09-21T00:30:01.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36438 fp=#0x10000001dd7' 2024-09-21T00:30:01.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36470 fp=#0x10000001d36' 2024-09-21T00:30:01.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36502 fp=#0x10000001da5' 2024-09-21T00:30:01.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36534 fp=#0x10000001d8e' 2024-09-21T00:30:01.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36566 fp=#0x10000004480' 2024-09-21T00:30:01.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36598 fp=#0x100000037dc' 2024-09-21T00:30:01.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36630 fp=#0x1000000441e' 2024-09-21T00:30:02.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36662 fp=#0x10000003753' 2024-09-21T00:30:02.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36694 fp=#0x1000000444e' 2024-09-21T00:30:02.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36726 fp=#0x100000046ff' 2024-09-21T00:30:02.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36758 fp=#0x10000004781' 2024-09-21T00:30:02.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36790 fp=#0x100000047d4' 2024-09-21T00:30:02.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36822 fp=#0x30000001ab0' 2024-09-21T00:30:02.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36854 fp=#0x1000000255a' 2024-09-21T00:30:02.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36886 fp=#0x10000002548' 2024-09-21T00:30:02.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36918 fp=#0x10000003546' 2024-09-21T00:30:02.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36950 fp=#0x1000000256f' 2024-09-21T00:30:02.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36982 fp=#0x100000025ec' 2024-09-21T00:30:02.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37046 fp=#0x300000017f5' 2024-09-21T00:30:02.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37174 fp=#0x10000003172' 2024-09-21T00:30:02.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37206 fp=#0x10000002ffa' 2024-09-21T00:30:02.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37238 fp=#0x10000002fd7' 2024-09-21T00:30:02.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37334 fp=#0x30000001b18' 2024-09-21T00:30:02.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37366 fp=#0x30000001ac3' 2024-09-21T00:30:02.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37398 fp=#0x30000001aba' 2024-09-21T00:30:02.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37430 fp=#0x30000001aeb' 2024-09-21T00:30:02.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37462 fp=#0x30000000c23' 2024-09-21T00:30:02.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37494 fp=#0x30000000fd7' 2024-09-21T00:30:02.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37558 fp=#0x30000001b83' 2024-09-21T00:30:02.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37622 fp=#0x10000002920' 2024-09-21T00:30:02.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37654 fp=#0x1000000293f' 2024-09-21T00:30:02.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37686 fp=#0x30000000c46' 2024-09-21T00:30:02.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37718 fp=#0x30000000c58' 2024-09-21T00:30:02.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37750 fp=#0x30000000c5d' 2024-09-21T00:30:02.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37814 fp=#0x10000002854' 2024-09-21T00:30:02.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37846 fp=#0x30000001275' 2024-09-21T00:30:02.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37878 fp=#0x30000001270' 2024-09-21T00:30:02.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37974 fp=#0x30000000dcc' 2024-09-21T00:30:02.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38038 fp=#0x30000000dbc' 2024-09-21T00:30:02.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38070 fp=#0x30000000cd0' 2024-09-21T00:30:02.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38102 fp=#0x10000000a1b' 2024-09-21T00:30:02.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38134 fp=#0x1000000072a' 2024-09-21T00:30:02.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38166 fp=#0x1000000072e' 2024-09-21T00:30:02.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38198 fp=#0x1000000073f' 2024-09-21T00:30:02.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38230 fp=#0x10000000e41' 2024-09-21T00:30:02.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38294 fp=#0x30000001a74' 2024-09-21T00:30:02.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38326 fp=#0x1000000248b' 2024-09-21T00:30:02.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38358 fp=#0x30000001a48' 2024-09-21T00:30:02.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38390 fp=#0x1000000328d' 2024-09-21T00:30:02.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38422 fp=#0x30000001cfc' 2024-09-21T00:30:02.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38486 fp=#0x100000032e2' 2024-09-21T00:30:02.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38518 fp=#0x1000000212d' 2024-09-21T00:30:02.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38550 fp=#0x100000020e3' 2024-09-21T00:30:02.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38582 fp=#0x100000013f8' 2024-09-21T00:30:02.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38646 fp=#0x30000001bcd' 2024-09-21T00:30:02.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38678 fp=#0x30000001c09' 2024-09-21T00:30:02.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38710 fp=#0x30000001c17' 2024-09-21T00:30:02.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38742 fp=#0x100000012bc' 2024-09-21T00:30:02.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38838 fp=#0x10000002944' 2024-09-21T00:30:02.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38870 fp=#0x300000019fb' 2024-09-21T00:30:02.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38902 fp=#0x300000019f5' 2024-09-21T00:30:02.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38934 fp=#0x30000001d70' 2024-09-21T00:30:02.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38966 fp=#0x30000001d09' 2024-09-21T00:30:02.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38998 fp=#0x30000001c80' 2024-09-21T00:30:02.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39030 fp=#0x30000001c81' 2024-09-21T00:30:02.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39062 fp=#0x30000001c6b' 2024-09-21T00:30:02.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39350 fp=#0x10000003f15' 2024-09-21T00:30:02.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39382 fp=#0x10000003f65' 2024-09-21T00:30:02.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39414 fp=#0x10000003f98' 2024-09-21T00:30:02.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39446 fp=#0x30000001f0f' 2024-09-21T00:30:02.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39478 fp=#0x30000001f66' 2024-09-21T00:30:02.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39510 fp=#0x30000001f26' 2024-09-21T00:30:02.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39542 fp=#0x30000001f28' 2024-09-21T00:30:02.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39574 fp=#0x30000001f1e' 2024-09-21T00:30:02.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39606 fp=#0x10000002aac' 2024-09-21T00:30:02.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39734 fp=#0x3000000199e' 2024-09-21T00:30:02.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39766 fp=#0x30000000ef3' 2024-09-21T00:30:02.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39798 fp=#0x30000001999' 2024-09-21T00:30:02.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39862 fp=#0x300000019a0' 2024-09-21T00:30:02.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39894 fp=#0x30000001100' 2024-09-21T00:30:02.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40022 fp=#0x30000001aea' 2024-09-21T00:30:02.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40054 fp=#0x30000001822' 2024-09-21T00:30:02.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40086 fp=#0x30000001836' 2024-09-21T00:30:02.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40118 fp=#0x3000000150f' 2024-09-21T00:30:02.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40150 fp=#0x3000000151f' 2024-09-21T00:30:02.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40182 fp=#0x3000000181e' 2024-09-21T00:30:02.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40214 fp=#0x30000000da2' 2024-09-21T00:30:02.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40278 fp=#0x30000000db9' 2024-09-21T00:30:02.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40310 fp=#0x30000001d32' 2024-09-21T00:30:02.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40342 fp=#0x10000001d21' 2024-09-21T00:30:02.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40374 fp=#0x300000022ca' 2024-09-21T00:30:02.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40406 fp=#0x30000002330' 2024-09-21T00:30:02.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40438 fp=#0x10000004e65' 2024-09-21T00:30:02.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40470 fp=#0x10000004cd1' 2024-09-21T00:30:02.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40566 fp=#0x10000000d04' 2024-09-21T00:30:02.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40598 fp=#0x20000000549' 2024-09-21T00:30:02.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40630 fp=#0x2000000054e' 2024-09-21T00:30:02.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40694 fp=#0x10000001010' 2024-09-21T00:30:02.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40726 fp=#0x10000000fa9' 2024-09-21T00:30:02.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40758 fp=#0x2000000054f' 2024-09-21T00:30:02.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40790 fp=#0x3000000143c' 2024-09-21T00:30:02.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40822 fp=#0x3000000142b' 2024-09-21T00:30:02.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40854 fp=#0x10000004ca4' 2024-09-21T00:30:02.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40886 fp=#0x10000004aad' 2024-09-21T00:30:02.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41014 fp=#0x100000036d5' 2024-09-21T00:30:02.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41046 fp=#0x100000034db' 2024-09-21T00:30:02.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41110 fp=#0x10000000f46' 2024-09-21T00:30:02.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41142 fp=#0x10000000fad' 2024-09-21T00:30:02.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41174 fp=#0x2000000047b' 2024-09-21T00:30:02.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41206 fp=#0x10000000d1b' 2024-09-21T00:30:02.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41238 fp=#0x10000000f08' 2024-09-21T00:30:02.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41270 fp=#0x10000000d46' 2024-09-21T00:30:02.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41302 fp=#0x20000000afe' 2024-09-21T00:30:02.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41334 fp=#0x10000000f6a' 2024-09-21T00:30:02.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41366 fp=#0x10000000992' 2024-09-21T00:30:02.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41398 fp=#0x10000000bc9' 2024-09-21T00:30:02.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41430 fp=#0x30000002079' 2024-09-21T00:30:02.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41462 fp=#0x10000000b8e' 2024-09-21T00:30:02.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41494 fp=#0x100000009e7' 2024-09-21T00:30:02.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41526 fp=#0x10000000c45' 2024-09-21T00:30:02.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41558 fp=#0x10000000bf0' 2024-09-21T00:30:02.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41590 fp=#0x30000001855' 2024-09-21T00:30:02.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41622 fp=#0x3000000190b' 2024-09-21T00:30:02.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41718 fp=#0x300000007eb' 2024-09-21T00:30:02.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41814 fp=#0x300000009d3' 2024-09-21T00:30:02.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41846 fp=#0x3000000088d' 2024-09-21T00:30:02.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41878 fp=#0x3000000099e' 2024-09-21T00:30:02.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41910 fp=#0x30000000843' 2024-09-21T00:30:02.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41942 fp=#0x30000000876' 2024-09-21T00:30:02.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42006 fp=#0x30000000878' 2024-09-21T00:30:02.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42038 fp=#0x30000001dab' 2024-09-21T00:30:02.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42070 fp=#0x30000001d61' 2024-09-21T00:30:02.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42102 fp=#0x300000007f0' 2024-09-21T00:30:02.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42134 fp=#0x30000000802' 2024-09-21T00:30:02.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42166 fp=#0x30000000bdb' 2024-09-21T00:30:02.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42198 fp=#0x30000000ba6' 2024-09-21T00:30:02.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42230 fp=#0x30000001bac' 2024-09-21T00:30:02.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42262 fp=#0x30000000b81' 2024-09-21T00:30:02.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42294 fp=#0x30000000b6a' 2024-09-21T00:30:02.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42326 fp=#0x30000000b41' 2024-09-21T00:30:02.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42358 fp=#0x30000000b3b' 2024-09-21T00:30:02.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42390 fp=#0x30000000af1' 2024-09-21T00:30:02.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42422 fp=#0x30000000aff' 2024-09-21T00:30:02.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42454 fp=#0x300000020e8' 2024-09-21T00:30:02.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42486 fp=#0x30000000a5c' 2024-09-21T00:30:02.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42518 fp=#0x30000000a5d' 2024-09-21T00:30:02.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42550 fp=#0x100000040f9' 2024-09-21T00:30:02.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42582 fp=#0x30000000a59' 2024-09-21T00:30:02.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42646 fp=#0x30000001260' 2024-09-21T00:30:02.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42678 fp=#0x100000018cf' 2024-09-21T00:30:02.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42710 fp=#0x10000001a30' 2024-09-21T00:30:02.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42806 fp=#0x1000000193a' 2024-09-21T00:30:02.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42838 fp=#0x10000001970' 2024-09-21T00:30:02.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42870 fp=#0x100000019ee' 2024-09-21T00:30:02.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42902 fp=#0x100000019d5' 2024-09-21T00:30:02.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43030 fp=#0x3000000175e' 2024-09-21T00:30:02.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43062 fp=#0x300000016f1' 2024-09-21T00:30:02.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43094 fp=#0x300000016e1' 2024-09-21T00:30:02.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43126 fp=#0x300000016f3' 2024-09-21T00:30:02.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43158 fp=#0x100000018b5' 2024-09-21T00:30:02.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43222 fp=#0x1000000187a' 2024-09-21T00:30:02.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43286 fp=#0x30000000f82' 2024-09-21T00:30:02.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43414 fp=#0x100000041a1' 2024-09-21T00:30:02.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43446 fp=#0x300000023a5' 2024-09-21T00:30:02.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43478 fp=#0x300000023e4' 2024-09-21T00:30:02.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43510 fp=#0x300000023c6' 2024-09-21T00:30:02.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43542 fp=#0x300000023a0' 2024-09-21T00:30:02.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43574 fp=#0x300000023cd' 2024-09-21T00:30:02.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43638 fp=#0x300000010f9' 2024-09-21T00:30:02.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43670 fp=#0x30000002101' 2024-09-21T00:30:02.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43798 fp=#0x10000002a32' 2024-09-21T00:30:02.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43830 fp=#0x10000002994' 2024-09-21T00:30:02.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43862 fp=#0x100000039d7' 2024-09-21T00:30:02.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43926 fp=#0x300000013d7' 2024-09-21T00:30:02.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43958 fp=#0x30000001396' 2024-09-21T00:30:02.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31736 fp=#0x10000000001' 2024-09-21T00:30:02.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31767 fp=#0x10000003750' 2024-09-21T00:30:02.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31927 fp=#0x10000000eef' 2024-09-21T00:30:02.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34711 fp=#0x10000000b53' 2024-09-21T00:30:02.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34743 fp=#0x3000000222b' 2024-09-21T00:30:02.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34775 fp=#0x30000002223' 2024-09-21T00:30:02.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34903 fp=#0x10000004856' 2024-09-21T00:30:02.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34967 fp=#0x10000003868' 2024-09-21T00:30:02.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34999 fp=#0x100000032b6' 2024-09-21T00:30:02.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35031 fp=#0x1000000328a' 2024-09-21T00:30:02.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35063 fp=#0x30000002065' 2024-09-21T00:30:02.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35095 fp=#0x100000041df' 2024-09-21T00:30:02.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35127 fp=#0x10000003b29' 2024-09-21T00:30:02.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35159 fp=#0x1000000390a' 2024-09-21T00:30:02.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35191 fp=#0x1000000395f' 2024-09-21T00:30:02.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35223 fp=#0x10000003b31' 2024-09-21T00:30:02.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35255 fp=#0x10000004171' 2024-09-21T00:30:02.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35287 fp=#0x10000003977' 2024-09-21T00:30:02.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35319 fp=#0x10000004203' 2024-09-21T00:30:02.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35351 fp=#0x1000000418d' 2024-09-21T00:30:02.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35383 fp=#0x1000000388b' 2024-09-21T00:30:02.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35447 fp=#0x10000003933' 2024-09-21T00:30:02.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35479 fp=#0x30000001179' 2024-09-21T00:30:02.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35511 fp=#0x30000001eef' 2024-09-21T00:30:02.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35543 fp=#0x30000001125' 2024-09-21T00:30:02.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35575 fp=#0x30000001fec' 2024-09-21T00:30:02.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35607 fp=#0x30000002005' 2024-09-21T00:30:02.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35639 fp=#0x30000001ff5' 2024-09-21T00:30:02.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35799 fp=#0x300000020e6' 2024-09-21T00:30:02.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35863 fp=#0x10000003474' 2024-09-21T00:30:02.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35895 fp=#0x100000033f4' 2024-09-21T00:30:02.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35927 fp=#0x300000011be' 2024-09-21T00:30:02.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35959 fp=#0x300000011f6' 2024-09-21T00:30:02.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36119 fp=#0x10000002cd9' 2024-09-21T00:30:02.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36151 fp=#0x10000001efe' 2024-09-21T00:30:02.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36183 fp=#0x10000001f4e' 2024-09-21T00:30:02.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36215 fp=#0x10000001eec' 2024-09-21T00:30:02.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36247 fp=#0x10000001f10' 2024-09-21T00:30:02.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36279 fp=#0x30000001606' 2024-09-21T00:30:02.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36311 fp=#0x300000015f6' 2024-09-21T00:30:02.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36375 fp=#0x300000015b4' 2024-09-21T00:30:02.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36407 fp=#0x300000015b0' 2024-09-21T00:30:02.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36439 fp=#0x10000001e6f' 2024-09-21T00:30:02.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36471 fp=#0x10000001e29' 2024-09-21T00:30:02.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36503 fp=#0x10000001db6' 2024-09-21T00:30:02.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36535 fp=#0x10000001e4f' 2024-09-21T00:30:02.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36567 fp=#0x1000000378e' 2024-09-21T00:30:02.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36599 fp=#0x1000000448a' 2024-09-21T00:30:02.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36631 fp=#0x10000004451' 2024-09-21T00:30:02.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36663 fp=#0x10000003751' 2024-09-21T00:30:02.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36695 fp=#0x1000000375c' 2024-09-21T00:30:02.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36727 fp=#0x10000004727' 2024-09-21T00:30:02.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36759 fp=#0x1000000479e' 2024-09-21T00:30:02.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36791 fp=#0x30000001aab' 2024-09-21T00:30:02.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36823 fp=#0x10000004765' 2024-09-21T00:30:02.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36855 fp=#0x10000002660' 2024-09-21T00:30:02.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36887 fp=#0x1000000358b' 2024-09-21T00:30:02.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36919 fp=#0x100000035d6' 2024-09-21T00:30:02.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36951 fp=#0x100000035b4' 2024-09-21T00:30:02.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36983 fp=#0x10000003588' 2024-09-21T00:30:02.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37015 fp=#0x10000003523' 2024-09-21T00:30:02.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37079 fp=#0x3000000178d' 2024-09-21T00:30:02.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37175 fp=#0x10000001349' 2024-09-21T00:30:02.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37207 fp=#0x10000003118' 2024-09-21T00:30:02.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37239 fp=#0x1000000318c' 2024-09-21T00:30:02.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37271 fp=#0x10000001345' 2024-09-21T00:30:02.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37335 fp=#0x30000000c1b' 2024-09-21T00:30:02.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37367 fp=#0x30000001b34' 2024-09-21T00:30:02.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37399 fp=#0x30000001b20' 2024-09-21T00:30:02.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37431 fp=#0x30000000c20' 2024-09-21T00:30:02.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37463 fp=#0x30000001b06' 2024-09-21T00:30:02.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37495 fp=#0x30000001114' 2024-09-21T00:30:02.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37559 fp=#0x300000010f8' 2024-09-21T00:30:02.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37591 fp=#0x30000001b88' 2024-09-21T00:30:02.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37623 fp=#0x30000000c63' 2024-09-21T00:30:02.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37655 fp=#0x30000000c61' 2024-09-21T00:30:02.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37687 fp=#0x30000000c55' 2024-09-21T00:30:02.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37751 fp=#0x30000000c75' 2024-09-21T00:30:02.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37783 fp=#0x30000000c7c' 2024-09-21T00:30:02.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37815 fp=#0x30000000c7d' 2024-09-21T00:30:02.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37847 fp=#0x300000012d3' 2024-09-21T00:30:02.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37943 fp=#0x30000002252' 2024-09-21T00:30:02.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37975 fp=#0x30000000cbd' 2024-09-21T00:30:02.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38103 fp=#0x10000000774' 2024-09-21T00:30:02.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38135 fp=#0x10000000e24' 2024-09-21T00:30:02.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38199 fp=#0x10000000e59' 2024-09-21T00:30:02.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38231 fp=#0x10000000e72' 2024-09-21T00:30:02.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38295 fp=#0x1000000244e' 2024-09-21T00:30:02.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38327 fp=#0x30000001a69' 2024-09-21T00:30:02.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38455 fp=#0x30000001cec' 2024-09-21T00:30:02.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38487 fp=#0x10000003315' 2024-09-21T00:30:02.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38519 fp=#0x10000002156' 2024-09-21T00:30:02.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38551 fp=#0x100000020f4' 2024-09-21T00:30:02.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38583 fp=#0x100000005e3' 2024-09-21T00:30:02.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38615 fp=#0x100000005ea' 2024-09-21T00:30:02.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38647 fp=#0x30000001bee' 2024-09-21T00:30:02.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38679 fp=#0x30000001bf6' 2024-09-21T00:30:02.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38711 fp=#0x30000001bf1' 2024-09-21T00:30:02.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38839 fp=#0x300000019ca' 2024-09-21T00:30:02.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38871 fp=#0x300000019d3' 2024-09-21T00:30:02.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38903 fp=#0x300000019cb' 2024-09-21T00:30:02.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38967 fp=#0x30000001d80' 2024-09-21T00:30:02.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38999 fp=#0x30000001c4d' 2024-09-21T00:30:02.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39031 fp=#0x30000001c5b' 2024-09-21T00:30:02.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39191 fp=#0x10000001277' 2024-09-21T00:30:02.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39223 fp=#0x100000034fd' 2024-09-21T00:30:02.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39255 fp=#0x1000000357b' 2024-09-21T00:30:02.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39319 fp=#0x1000000112e' 2024-09-21T00:30:02.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39351 fp=#0x10000003fd3' 2024-09-21T00:30:02.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39383 fp=#0x10000003f82' 2024-09-21T00:30:02.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39415 fp=#0x10000003f74' 2024-09-21T00:30:02.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39447 fp=#0x30000001f27' 2024-09-21T00:30:02.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39479 fp=#0x30000001f3e' 2024-09-21T00:30:02.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39511 fp=#0x30000001f0e' 2024-09-21T00:30:02.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39543 fp=#0x30000001fac' 2024-09-21T00:30:02.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39575 fp=#0x30000001f37' 2024-09-21T00:30:02.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39607 fp=#0x10000002b33' 2024-09-21T00:30:02.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39671 fp=#0x10000002aa3' 2024-09-21T00:30:02.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39703 fp=#0x30000000eee' 2024-09-21T00:30:02.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39735 fp=#0x3000000199d' 2024-09-21T00:30:02.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39767 fp=#0x3000000199a' 2024-09-21T00:30:02.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39799 fp=#0x30000000e5d' 2024-09-21T00:30:02.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39831 fp=#0x30000001983' 2024-09-21T00:30:02.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39863 fp=#0x3000000199b' 2024-09-21T00:30:02.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39927 fp=#0x30000001013' 2024-09-21T00:30:02.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39991 fp=#0x30000000ff6' 2024-09-21T00:30:02.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40023 fp=#0x3000000166a' 2024-09-21T00:30:02.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40055 fp=#0x30000001823' 2024-09-21T00:30:02.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40087 fp=#0x300000017f8' 2024-09-21T00:30:02.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40119 fp=#0x3000000182b' 2024-09-21T00:30:02.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40151 fp=#0x300000017f2' 2024-09-21T00:30:02.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40215 fp=#0x30000000d5c' 2024-09-21T00:30:02.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40247 fp=#0x30000000dc8' 2024-09-21T00:30:02.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40279 fp=#0x30000000d7e' 2024-09-21T00:30:02.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40343 fp=#0x10000001d31' 2024-09-21T00:30:02.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40375 fp=#0x10000004e5d' 2024-09-21T00:30:02.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40407 fp=#0x10000004c72' 2024-09-21T00:30:02.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40439 fp=#0x10000004cf9' 2024-09-21T00:30:02.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40471 fp=#0x10000004e6d' 2024-09-21T00:30:02.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40535 fp=#0x10000000cf5' 2024-09-21T00:30:02.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40567 fp=#0x200000005ab' 2024-09-21T00:30:02.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40599 fp=#0x10000000d16' 2024-09-21T00:30:02.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40663 fp=#0x10000000f13' 2024-09-21T00:30:02.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40695 fp=#0x10000000f02' 2024-09-21T00:30:02.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40727 fp=#0x20000000451' 2024-09-21T00:30:02.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40759 fp=#0x10000000d0b' 2024-09-21T00:30:02.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40791 fp=#0x30000001433' 2024-09-21T00:30:02.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40823 fp=#0x10000004a6c' 2024-09-21T00:30:02.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40887 fp=#0x10000004aa0' 2024-09-21T00:30:02.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40919 fp=#0x10000004af3' 2024-09-21T00:30:02.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41015 fp=#0x100000036e9' 2024-09-21T00:30:02.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41047 fp=#0x20000000702' 2024-09-21T00:30:02.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41079 fp=#0x10000003635' 2024-09-21T00:30:02.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41111 fp=#0x20000000215' 2024-09-21T00:30:02.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41143 fp=#0x20000000b79' 2024-09-21T00:30:02.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41175 fp=#0x20000000ba9' 2024-09-21T00:30:02.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41207 fp=#0x10000000d2f' 2024-09-21T00:30:02.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41239 fp=#0x20000000aca' 2024-09-21T00:30:02.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41271 fp=#0x20000000b3d' 2024-09-21T00:30:02.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41303 fp=#0x10000000d63' 2024-09-21T00:30:02.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41335 fp=#0x10000000d01' 2024-09-21T00:30:02.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41367 fp=#0x10000000eb3' 2024-09-21T00:30:02.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41399 fp=#0x10000000c79' 2024-09-21T00:30:02.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41431 fp=#0x10000000933' 2024-09-21T00:30:02.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41463 fp=#0x30000002071' 2024-09-21T00:30:02.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41495 fp=#0x10000000bad' 2024-09-21T00:30:02.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41527 fp=#0x10000000bd4' 2024-09-21T00:30:02.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41559 fp=#0x10000000b5c' 2024-09-21T00:30:02.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41655 fp=#0x30000001843' 2024-09-21T00:30:02.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41719 fp=#0x300000007f3' 2024-09-21T00:30:02.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41783 fp=#0x30000000906' 2024-09-21T00:30:02.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41815 fp=#0x300000009dc' 2024-09-21T00:30:02.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41879 fp=#0x300000009da' 2024-09-21T00:30:02.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41911 fp=#0x3000000089d' 2024-09-21T00:30:02.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41943 fp=#0x30000000881' 2024-09-21T00:30:02.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41975 fp=#0x3000000083e' 2024-09-21T00:30:02.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42007 fp=#0x30000000993' 2024-09-21T00:30:02.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42039 fp=#0x300000007fa' 2024-09-21T00:30:02.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42103 fp=#0x30000000875' 2024-09-21T00:30:02.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42135 fp=#0x30000000808' 2024-09-21T00:30:02.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42167 fp=#0x30000000b69' 2024-09-21T00:30:02.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42199 fp=#0x30000000b75' 2024-09-21T00:30:02.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42231 fp=#0x30000001b9c' 2024-09-21T00:30:02.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42263 fp=#0x30000001ba0' 2024-09-21T00:30:02.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42295 fp=#0x30000000bb1' 2024-09-21T00:30:02.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42327 fp=#0x30000000adb' 2024-09-21T00:30:02.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42359 fp=#0x30000000b06' 2024-09-21T00:30:02.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42423 fp=#0x30000000b30' 2024-09-21T00:30:02.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42455 fp=#0x10000004142' 2024-09-21T00:30:02.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42551 fp=#0x30000000a6b' 2024-09-21T00:30:02.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42583 fp=#0x30000000a60' 2024-09-21T00:30:02.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42647 fp=#0x3000000125d' 2024-09-21T00:30:02.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42679 fp=#0x100000018ff' 2024-09-21T00:30:02.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42711 fp=#0x10000001973' 2024-09-21T00:30:02.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42839 fp=#0x10000001931' 2024-09-21T00:30:02.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42871 fp=#0x100000018cb' 2024-09-21T00:30:02.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42967 fp=#0x30000000df1' 2024-09-21T00:30:02.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43063 fp=#0x30000001704' 2024-09-21T00:30:02.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43095 fp=#0x300000016d6' 2024-09-21T00:30:02.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43127 fp=#0x3000000172f' 2024-09-21T00:30:02.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43287 fp=#0x30000000f48' 2024-09-21T00:30:02.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43319 fp=#0x30000000f3a' 2024-09-21T00:30:02.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43351 fp=#0x30000000f73' 2024-09-21T00:30:02.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43415 fp=#0x10000004158' 2024-09-21T00:30:02.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43447 fp=#0x300000023ef' 2024-09-21T00:30:02.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43479 fp=#0x300000023b1' 2024-09-21T00:30:02.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43511 fp=#0x300000023f2' 2024-09-21T00:30:02.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43543 fp=#0x300000023c2' 2024-09-21T00:30:02.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43607 fp=#0x300000010fe' 2024-09-21T00:30:02.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43639 fp=#0x300000010ca' 2024-09-21T00:30:02.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43671 fp=#0x3000000232d' 2024-09-21T00:30:02.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43703 fp=#0x300000021a3' 2024-09-21T00:30:02.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43735 fp=#0x300000020fa' 2024-09-21T00:30:02.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43767 fp=#0x300000021cd' 2024-09-21T00:30:02.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43799 fp=#0x10000003a1e' 2024-09-21T00:30:02.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43831 fp=#0x10000003a25' 2024-09-21T00:30:02.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43863 fp=#0x100000039ef' 2024-09-21T00:30:02.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43895 fp=#0x100000039e1' 2024-09-21T00:30:02.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43927 fp=#0x3000000133c' 2024-09-21T00:30:02.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31737 fp=#0x100000001fa' 2024-09-21T00:30:02.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31768 fp=#0x100000046fa' 2024-09-21T00:30:02.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31800 fp=#0x30000001773' 2024-09-21T00:30:02.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31928 fp=#0x3000000183e' 2024-09-21T00:30:02.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34680 fp=#0x3000000222e' 2024-09-21T00:30:02.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34744 fp=#0x30000002260' 2024-09-21T00:30:02.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34776 fp=#0x3000000225b' 2024-09-21T00:30:02.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34968 fp=#0x100000032bd' 2024-09-21T00:30:02.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35000 fp=#0x1000000329d' 2024-09-21T00:30:02.204 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35032 fp=#0x30000002069' 2024-09-21T00:30:02.204 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35064 fp=#0x100000032ba' 2024-09-21T00:30:02.204 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35096 fp=#0x100000039e9' 2024-09-21T00:30:02.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35128 fp=#0x10000003913' 2024-09-21T00:30:02.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35160 fp=#0x1000000415e' 2024-09-21T00:30:02.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35192 fp=#0x1000000397d' 2024-09-21T00:30:02.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35224 fp=#0x10000004186' 2024-09-21T00:30:02.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35256 fp=#0x10000003a64' 2024-09-21T00:30:02.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35288 fp=#0x10000004159' 2024-09-21T00:30:02.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35320 fp=#0x100000041ef' 2024-09-21T00:30:02.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35352 fp=#0x1000000387b' 2024-09-21T00:30:02.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35384 fp=#0x100000039a5' 2024-09-21T00:30:02.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35416 fp=#0x100000038a4' 2024-09-21T00:30:02.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35448 fp=#0x1000000396f' 2024-09-21T00:30:02.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35480 fp=#0x3000000116e' 2024-09-21T00:30:02.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35512 fp=#0x30000001132' 2024-09-21T00:30:02.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35544 fp=#0x30000001174' 2024-09-21T00:30:02.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35576 fp=#0x30000002054' 2024-09-21T00:30:02.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35608 fp=#0x30000002009' 2024-09-21T00:30:02.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35640 fp=#0x30000001fda' 2024-09-21T00:30:02.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35672 fp=#0x30000001fe3' 2024-09-21T00:30:02.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35704 fp=#0x30000001e6b' 2024-09-21T00:30:02.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35896 fp=#0x10000003458' 2024-09-21T00:30:02.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35928 fp=#0x300000011c1' 2024-09-21T00:30:02.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35960 fp=#0x300000011ca' 2024-09-21T00:30:02.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36088 fp=#0x10000002d36' 2024-09-21T00:30:02.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36120 fp=#0x10000002ca0' 2024-09-21T00:30:02.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36152 fp=#0x10000001ef1' 2024-09-21T00:30:02.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36184 fp=#0x10000001f25' 2024-09-21T00:30:02.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36216 fp=#0x10000001ea1' 2024-09-21T00:30:02.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36248 fp=#0x10000001f73' 2024-09-21T00:30:02.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36280 fp=#0x3000000160f' 2024-09-21T00:30:02.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36312 fp=#0x300000015a6' 2024-09-21T00:30:02.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36344 fp=#0x3000000160d' 2024-09-21T00:30:02.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36376 fp=#0x30000001604' 2024-09-21T00:30:02.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36440 fp=#0x10000001d4b' 2024-09-21T00:30:02.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36472 fp=#0x10000001e7b' 2024-09-21T00:30:02.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36504 fp=#0x10000001f75' 2024-09-21T00:30:02.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36568 fp=#0x1000000384d' 2024-09-21T00:30:02.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36600 fp=#0x10000004489' 2024-09-21T00:30:02.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36632 fp=#0x10000004434' 2024-09-21T00:30:02.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36664 fp=#0x100000037cb' 2024-09-21T00:30:02.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36696 fp=#0x1000000449e' 2024-09-21T00:30:02.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36728 fp=#0x10000004763' 2024-09-21T00:30:02.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36760 fp=#0x10000004786' 2024-09-21T00:30:02.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36792 fp=#0x100000047c3' 2024-09-21T00:30:02.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36824 fp=#0x10000004700' 2024-09-21T00:30:02.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36856 fp=#0x10000003593' 2024-09-21T00:30:02.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36888 fp=#0x10000002553' 2024-09-21T00:30:02.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36952 fp=#0x10000003511' 2024-09-21T00:30:02.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37016 fp=#0x100000035e8' 2024-09-21T00:30:02.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37048 fp=#0x3000000196b' 2024-09-21T00:30:02.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37144 fp=#0x30000001789' 2024-09-21T00:30:02.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37176 fp=#0x10000001490' 2024-09-21T00:30:02.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37208 fp=#0x1000000304a' 2024-09-21T00:30:02.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37240 fp=#0x1000000149f' 2024-09-21T00:30:02.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37272 fp=#0x10000001475' 2024-09-21T00:30:02.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37304 fp=#0x100000031aa' 2024-09-21T00:30:02.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37336 fp=#0x30000000bf6' 2024-09-21T00:30:02.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37368 fp=#0x30000000bf0' 2024-09-21T00:30:02.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37400 fp=#0x30000000c29' 2024-09-21T00:30:02.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37432 fp=#0x30000000c09' 2024-09-21T00:30:02.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37560 fp=#0x3000000139f' 2024-09-21T00:30:02.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37656 fp=#0x30000000c64' 2024-09-21T00:30:02.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37720 fp=#0x30000000c9d' 2024-09-21T00:30:02.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37752 fp=#0x30000000c56' 2024-09-21T00:30:02.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37912 fp=#0x3000000228c' 2024-09-21T00:30:02.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38104 fp=#0x10000000e5e' 2024-09-21T00:30:02.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38136 fp=#0x10000000735' 2024-09-21T00:30:02.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38168 fp=#0x10000000716' 2024-09-21T00:30:02.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38200 fp=#0x1000000073a' 2024-09-21T00:30:02.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38232 fp=#0x10000000720' 2024-09-21T00:30:02.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38296 fp=#0x10000002487' 2024-09-21T00:30:02.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38328 fp=#0x30000001a50' 2024-09-21T00:30:02.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38360 fp=#0x30000001a8c' 2024-09-21T00:30:02.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38392 fp=#0x10000003305' 2024-09-21T00:30:02.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38424 fp=#0x1000000331e' 2024-09-21T00:30:02.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38456 fp=#0x10000003336' 2024-09-21T00:30:02.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38488 fp=#0x1000000332c' 2024-09-21T00:30:02.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38520 fp=#0x10000002139' 2024-09-21T00:30:02.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38584 fp=#0x30000001c05' 2024-09-21T00:30:02.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38616 fp=#0x30000001c03' 2024-09-21T00:30:02.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38648 fp=#0x1000000336f' 2024-09-21T00:30:02.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38680 fp=#0x30000001c47' 2024-09-21T00:30:02.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38712 fp=#0x1000000052d' 2024-09-21T00:30:02.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38744 fp=#0x30000001be5' 2024-09-21T00:30:02.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38776 fp=#0x100000005a6' 2024-09-21T00:30:02.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38808 fp=#0x300000019ed' 2024-09-21T00:30:02.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38840 fp=#0x300000019c1' 2024-09-21T00:30:02.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38904 fp=#0x300000019d7' 2024-09-21T00:30:02.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38936 fp=#0x30000001c3e' 2024-09-21T00:30:02.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39064 fp=#0x30000001d3c' 2024-09-21T00:30:02.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39192 fp=#0x1000000135f' 2024-09-21T00:30:02.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39256 fp=#0x1000000127f' 2024-09-21T00:30:02.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39320 fp=#0x10000003515' 2024-09-21T00:30:02.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39352 fp=#0x10000003f88' 2024-09-21T00:30:02.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39384 fp=#0x10000003fd7' 2024-09-21T00:30:02.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39416 fp=#0x10000003f0d' 2024-09-21T00:30:02.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39448 fp=#0x30000001fb8' 2024-09-21T00:30:02.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39480 fp=#0x30000001f80' 2024-09-21T00:30:02.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39512 fp=#0x30000001f56' 2024-09-21T00:30:02.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39544 fp=#0x30000001f68' 2024-09-21T00:30:02.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39576 fp=#0x30000001f6e' 2024-09-21T00:30:02.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39640 fp=#0x10000002a85' 2024-09-21T00:30:02.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39672 fp=#0x10000002ac5' 2024-09-21T00:30:02.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39704 fp=#0x300000019aa' 2024-09-21T00:30:02.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39800 fp=#0x30000000e60' 2024-09-21T00:30:02.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39864 fp=#0x30000000e63' 2024-09-21T00:30:02.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39896 fp=#0x3000000112a' 2024-09-21T00:30:02.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39960 fp=#0x30000000fee' 2024-09-21T00:30:02.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39992 fp=#0x300000010ff' 2024-09-21T00:30:02.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40024 fp=#0x3000000154e' 2024-09-21T00:30:02.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40056 fp=#0x30000001805' 2024-09-21T00:30:02.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40088 fp=#0x300000014c6' 2024-09-21T00:30:02.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40120 fp=#0x3000000180a' 2024-09-21T00:30:02.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40152 fp=#0x30000001829' 2024-09-21T00:30:02.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40184 fp=#0x30000000d81' 2024-09-21T00:30:02.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40216 fp=#0x10000003183' 2024-09-21T00:30:02.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40248 fp=#0x30000001d2f' 2024-09-21T00:30:02.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40280 fp=#0x10000001d0d' 2024-09-21T00:30:02.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40312 fp=#0x300000009e8' 2024-09-21T00:30:02.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40344 fp=#0x30000000d8b' 2024-09-21T00:30:02.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40376 fp=#0x10000004e46' 2024-09-21T00:30:02.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40408 fp=#0x10000004c91' 2024-09-21T00:30:02.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40440 fp=#0x10000004e27' 2024-09-21T00:30:02.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40472 fp=#0x10000004c75' 2024-09-21T00:30:02.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40504 fp=#0x1000000465f' 2024-09-21T00:30:02.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40536 fp=#0x1000000104e' 2024-09-21T00:30:02.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40568 fp=#0x200000005bf' 2024-09-21T00:30:02.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40600 fp=#0x10000000ce6' 2024-09-21T00:30:02.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40632 fp=#0x200000005f7' 2024-09-21T00:30:02.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40664 fp=#0x20000000559' 2024-09-21T00:30:02.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40696 fp=#0x2000000057f' 2024-09-21T00:30:02.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40728 fp=#0x200000005ae' 2024-09-21T00:30:02.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40760 fp=#0x10000000d26' 2024-09-21T00:30:02.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40792 fp=#0x30000001444' 2024-09-21T00:30:02.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40824 fp=#0x10000004acc' 2024-09-21T00:30:02.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40888 fp=#0x10000004d14' 2024-09-21T00:30:02.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40984 fp=#0x10000004c87' 2024-09-21T00:30:02.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41016 fp=#0x20000000716' 2024-09-21T00:30:02.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41048 fp=#0x1000000353f' 2024-09-21T00:30:02.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41080 fp=#0x2000000071c' 2024-09-21T00:30:02.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41112 fp=#0x10000000cdd' 2024-09-21T00:30:02.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41144 fp=#0x10000000cea' 2024-09-21T00:30:02.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41176 fp=#0x10000000dfb' 2024-09-21T00:30:02.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41208 fp=#0x20000000b5b' 2024-09-21T00:30:02.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41240 fp=#0x10000000dcb' 2024-09-21T00:30:02.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41272 fp=#0x10000000e2a' 2024-09-21T00:30:02.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41304 fp=#0x10000000f1d' 2024-09-21T00:30:02.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41336 fp=#0x2000000025b' 2024-09-21T00:30:02.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41368 fp=#0x10000000bdb' 2024-09-21T00:30:02.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41432 fp=#0x10000000a0a' 2024-09-21T00:30:02.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41464 fp=#0x10000000ec9' 2024-09-21T00:30:02.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41496 fp=#0x100000009cb' 2024-09-21T00:30:02.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41528 fp=#0x10000000ba2' 2024-09-21T00:30:02.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41560 fp=#0x10000000bec' 2024-09-21T00:30:02.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41624 fp=#0x30000001859' 2024-09-21T00:30:02.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41688 fp=#0x30000000923' 2024-09-21T00:30:02.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41816 fp=#0x300000008c7' 2024-09-21T00:30:02.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41848 fp=#0x300000008a8' 2024-09-21T00:30:02.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41880 fp=#0x3000000097c' 2024-09-21T00:30:02.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41912 fp=#0x300000009ce' 2024-09-21T00:30:02.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41944 fp=#0x300000008be' 2024-09-21T00:30:02.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41976 fp=#0x300000009a6' 2024-09-21T00:30:02.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42008 fp=#0x30000000988' 2024-09-21T00:30:02.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42040 fp=#0x30000000812' 2024-09-21T00:30:02.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42072 fp=#0x3000000081f' 2024-09-21T00:30:02.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42104 fp=#0x3000000082e' 2024-09-21T00:30:02.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42136 fp=#0x30000000857' 2024-09-21T00:30:02.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42168 fp=#0x30000000bbe' 2024-09-21T00:30:02.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42200 fp=#0x30000001bb2' 2024-09-21T00:30:02.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42232 fp=#0x3000000209b' 2024-09-21T00:30:02.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42264 fp=#0x30000000bad' 2024-09-21T00:30:02.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42296 fp=#0x30000001b8c' 2024-09-21T00:30:02.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42328 fp=#0x30000000af3' 2024-09-21T00:30:02.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42360 fp=#0x30000000b35' 2024-09-21T00:30:02.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42392 fp=#0x30000000b2b' 2024-09-21T00:30:02.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42424 fp=#0x30000000b28' 2024-09-21T00:30:02.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42456 fp=#0x1000000413d' 2024-09-21T00:30:02.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42488 fp=#0x30000000a44' 2024-09-21T00:30:02.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42520 fp=#0x30000001b42' 2024-09-21T00:30:02.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42584 fp=#0x30000000a49' 2024-09-21T00:30:02.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42648 fp=#0x30000001259' 2024-09-21T00:30:02.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42680 fp=#0x100000018f5' 2024-09-21T00:30:02.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42712 fp=#0x10000001a12' 2024-09-21T00:30:02.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42776 fp=#0x10000001926' 2024-09-21T00:30:02.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42840 fp=#0x100000019bc' 2024-09-21T00:30:02.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42904 fp=#0x30000000e49' 2024-09-21T00:30:02.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42936 fp=#0x30000000e3d' 2024-09-21T00:30:02.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42968 fp=#0x30000000e3f' 2024-09-21T00:30:02.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43000 fp=#0x30000000e3b' 2024-09-21T00:30:02.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43032 fp=#0x3000000175c' 2024-09-21T00:30:02.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43064 fp=#0x30000001737' 2024-09-21T00:30:02.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43096 fp=#0x30000001683' 2024-09-21T00:30:02.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43128 fp=#0x30000001706' 2024-09-21T00:30:02.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43224 fp=#0x1000000186f' 2024-09-21T00:30:02.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43288 fp=#0x30000000f81' 2024-09-21T00:30:02.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43352 fp=#0x30000000f39' 2024-09-21T00:30:02.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43384 fp=#0x30000000f34' 2024-09-21T00:30:02.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43416 fp=#0x10000004192' 2024-09-21T00:30:02.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43448 fp=#0x3000000237c' 2024-09-21T00:30:02.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43480 fp=#0x300000023de' 2024-09-21T00:30:02.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43512 fp=#0x300000023b4' 2024-09-21T00:30:02.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43544 fp=#0x30000002393' 2024-09-21T00:30:02.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43576 fp=#0x300000010ea' 2024-09-21T00:30:02.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43608 fp=#0x30000000fa3' 2024-09-21T00:30:02.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43640 fp=#0x30000000fc9' 2024-09-21T00:30:02.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43672 fp=#0x300000021ab' 2024-09-21T00:30:02.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43704 fp=#0x300000010d1' 2024-09-21T00:30:02.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43736 fp=#0x30000002107' 2024-09-21T00:30:02.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43768 fp=#0x3000000218a' 2024-09-21T00:30:02.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43800 fp=#0x10000002a23' 2024-09-21T00:30:02.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43832 fp=#0x1000000396e' 2024-09-21T00:30:02.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43864 fp=#0x1000000398e' 2024-09-21T00:30:02.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43896 fp=#0x1000000397c' 2024-09-21T00:30:02.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43928 fp=#0x300000013b9' 2024-09-21T00:30:02.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31738 fp=#0x100000001fb' 2024-09-21T00:30:02.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31769 fp=#0x10000004936' 2024-09-21T00:30:02.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31801 fp=#0x1000000133b' 2024-09-21T00:30:02.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31929 fp=#0x1000000302a' 2024-09-21T00:30:02.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34681 fp=#0x30000002246' 2024-09-21T00:30:02.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34745 fp=#0x30000002272' 2024-09-21T00:30:02.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34777 fp=#0x30000002239' 2024-09-21T00:30:02.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34841 fp=#0x100000027fb' 2024-09-21T00:30:02.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34905 fp=#0x10000002801' 2024-09-21T00:30:02.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34937 fp=#0x100000027e3' 2024-09-21T00:30:02.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34969 fp=#0x300000020bb' 2024-09-21T00:30:02.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35001 fp=#0x1000000382b' 2024-09-21T00:30:02.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35033 fp=#0x3000000208f' 2024-09-21T00:30:02.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35065 fp=#0x30000002073' 2024-09-21T00:30:02.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35097 fp=#0x1000000398f' 2024-09-21T00:30:02.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35129 fp=#0x10000004178' 2024-09-21T00:30:02.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35161 fp=#0x10000003ab4' 2024-09-21T00:30:02.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35193 fp=#0x100000039ed' 2024-09-21T00:30:02.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35225 fp=#0x10000004239' 2024-09-21T00:30:02.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35257 fp=#0x10000003958' 2024-09-21T00:30:02.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35289 fp=#0x10000004174' 2024-09-21T00:30:02.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35321 fp=#0x100000038ed' 2024-09-21T00:30:02.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35353 fp=#0x100000039ff' 2024-09-21T00:30:02.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35385 fp=#0x10000003943' 2024-09-21T00:30:02.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35417 fp=#0x100000039b3' 2024-09-21T00:30:02.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35449 fp=#0x10000003973' 2024-09-21T00:30:02.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35513 fp=#0x30000001167' 2024-09-21T00:30:02.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35545 fp=#0x30000001120' 2024-09-21T00:30:02.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35609 fp=#0x30000002027' 2024-09-21T00:30:02.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35641 fp=#0x30000002046' 2024-09-21T00:30:02.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35673 fp=#0x3000000203f' 2024-09-21T00:30:02.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35801 fp=#0x1000000345a' 2024-09-21T00:30:02.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35833 fp=#0x10000003391' 2024-09-21T00:30:02.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35865 fp=#0x10000003465' 2024-09-21T00:30:02.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35897 fp=#0x100000033b3' 2024-09-21T00:30:02.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35929 fp=#0x300000011ed' 2024-09-21T00:30:02.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35961 fp=#0x300000011eb' 2024-09-21T00:30:02.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36089 fp=#0x10000002ccb' 2024-09-21T00:30:02.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36121 fp=#0x10000002d44' 2024-09-21T00:30:02.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36153 fp=#0x10000001ea5' 2024-09-21T00:30:02.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36185 fp=#0x10000001f6e' 2024-09-21T00:30:02.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36217 fp=#0x10000002072' 2024-09-21T00:30:02.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36249 fp=#0x10000001fa2' 2024-09-21T00:30:02.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36409 fp=#0x300000015c1' 2024-09-21T00:30:02.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36441 fp=#0x10000001d8a' 2024-09-21T00:30:02.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36473 fp=#0x1000000251f' 2024-09-21T00:30:02.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36505 fp=#0x10000001e3b' 2024-09-21T00:30:02.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36537 fp=#0x10000002523' 2024-09-21T00:30:02.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36569 fp=#0x10000003862' 2024-09-21T00:30:02.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36601 fp=#0x100000037d9' 2024-09-21T00:30:02.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36633 fp=#0x100000044ab' 2024-09-21T00:30:02.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36665 fp=#0x10000004444' 2024-09-21T00:30:02.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36697 fp=#0x1000000448e' 2024-09-21T00:30:02.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36729 fp=#0x10000004754' 2024-09-21T00:30:02.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36761 fp=#0x30000001a8d' 2024-09-21T00:30:02.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36793 fp=#0x1000000477b' 2024-09-21T00:30:02.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36825 fp=#0x100000047aa' 2024-09-21T00:30:02.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36889 fp=#0x10000003596' 2024-09-21T00:30:02.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36921 fp=#0x10000003574' 2024-09-21T00:30:02.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36953 fp=#0x10000003526' 2024-09-21T00:30:02.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36985 fp=#0x10000003538' 2024-09-21T00:30:02.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37017 fp=#0x10000003579' 2024-09-21T00:30:02.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37049 fp=#0x300000020c2' 2024-09-21T00:30:02.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37113 fp=#0x30000001787' 2024-09-21T00:30:02.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37177 fp=#0x10000003093' 2024-09-21T00:30:02.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37209 fp=#0x10000003006' 2024-09-21T00:30:02.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37241 fp=#0x100000030ca' 2024-09-21T00:30:02.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37305 fp=#0x1000000316a' 2024-09-21T00:30:02.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37337 fp=#0x30000000c25' 2024-09-21T00:30:02.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37369 fp=#0x30000000c1e' 2024-09-21T00:30:02.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37401 fp=#0x30000001adf' 2024-09-21T00:30:02.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37433 fp=#0x30000000bfc' 2024-09-21T00:30:02.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37465 fp=#0x30000001b37' 2024-09-21T00:30:02.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37497 fp=#0x30000001106' 2024-09-21T00:30:02.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37593 fp=#0x30000001071' 2024-09-21T00:30:02.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37625 fp=#0x30000000ca1' 2024-09-21T00:30:02.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37657 fp=#0x30000000c79' 2024-09-21T00:30:02.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37689 fp=#0x30000000c3f' 2024-09-21T00:30:02.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37881 fp=#0x300000012bd' 2024-09-21T00:30:02.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38041 fp=#0x30000000ce6' 2024-09-21T00:30:02.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38105 fp=#0x10000000e6f' 2024-09-21T00:30:02.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38137 fp=#0x1000000072c' 2024-09-21T00:30:02.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38169 fp=#0x3000000070e' 2024-09-21T00:30:02.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38201 fp=#0x10000000730' 2024-09-21T00:30:02.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38233 fp=#0x10000000754' 2024-09-21T00:30:02.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38297 fp=#0x30000001a4a' 2024-09-21T00:30:02.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38329 fp=#0x30000001a42' 2024-09-21T00:30:02.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38361 fp=#0x30000001a5c' 2024-09-21T00:30:02.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38425 fp=#0x10000003327' 2024-09-21T00:30:02.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38457 fp=#0x10000003312' 2024-09-21T00:30:02.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38489 fp=#0x30000001cfe' 2024-09-21T00:30:02.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38521 fp=#0x10000002120' 2024-09-21T00:30:02.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38553 fp=#0x10000002158' 2024-09-21T00:30:02.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38585 fp=#0x100000005de' 2024-09-21T00:30:02.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38617 fp=#0x30000001c06' 2024-09-21T00:30:02.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38681 fp=#0x30000001c38' 2024-09-21T00:30:02.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38713 fp=#0x30000001c45' 2024-09-21T00:30:02.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38745 fp=#0x30000001c01' 2024-09-21T00:30:02.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38777 fp=#0x100000005bc' 2024-09-21T00:30:02.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38809 fp=#0x300000019c2' 2024-09-21T00:30:02.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38873 fp=#0x300000019cd' 2024-09-21T00:30:02.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38937 fp=#0x30000001c34' 2024-09-21T00:30:02.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38969 fp=#0x30000001d4e' 2024-09-21T00:30:02.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39001 fp=#0x30000001c56' 2024-09-21T00:30:02.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39065 fp=#0x30000001d0b' 2024-09-21T00:30:02.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39193 fp=#0x100000034a7' 2024-09-21T00:30:02.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39225 fp=#0x10000003591' 2024-09-21T00:30:02.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39257 fp=#0x100000034ae' 2024-09-21T00:30:02.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39289 fp=#0x100000034bc' 2024-09-21T00:30:02.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39353 fp=#0x10000003efb' 2024-09-21T00:30:02.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39385 fp=#0x10000003f36' 2024-09-21T00:30:02.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39417 fp=#0x10000003f7c' 2024-09-21T00:30:02.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39449 fp=#0x30000001f8f' 2024-09-21T00:30:02.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39481 fp=#0x30000001f57' 2024-09-21T00:30:02.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39513 fp=#0x30000001f6f' 2024-09-21T00:30:02.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39545 fp=#0x30000001f76' 2024-09-21T00:30:02.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39577 fp=#0x30000001f4a' 2024-09-21T00:30:02.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39641 fp=#0x10000002afc' 2024-09-21T00:30:02.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39705 fp=#0x300000019a1' 2024-09-21T00:30:02.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39737 fp=#0x30000000ede' 2024-09-21T00:30:02.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39769 fp=#0x30000000ebd' 2024-09-21T00:30:02.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39801 fp=#0x30000000eb0' 2024-09-21T00:30:02.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39833 fp=#0x30000001985' 2024-09-21T00:30:02.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40057 fp=#0x30000001549' 2024-09-21T00:30:02.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40089 fp=#0x3000000181d' 2024-09-21T00:30:02.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40121 fp=#0x300000014d4' 2024-09-21T00:30:02.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40153 fp=#0x30000001830' 2024-09-21T00:30:02.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40185 fp=#0x10000001d2d' 2024-09-21T00:30:02.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40217 fp=#0x10000001d61' 2024-09-21T00:30:02.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40249 fp=#0x30000000d99' 2024-09-21T00:30:02.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40281 fp=#0x30000000d76' 2024-09-21T00:30:02.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40313 fp=#0x10000001d3c' 2024-09-21T00:30:02.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40345 fp=#0x10000004e77' 2024-09-21T00:30:02.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40377 fp=#0x10000004e72' 2024-09-21T00:30:02.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40409 fp=#0x10000004e7f' 2024-09-21T00:30:02.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40441 fp=#0x10000004e3c' 2024-09-21T00:30:02.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40473 fp=#0x10000004e44' 2024-09-21T00:30:02.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40537 fp=#0x10000000f31' 2024-09-21T00:30:02.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40569 fp=#0x10000000f68' 2024-09-21T00:30:02.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40601 fp=#0x20000000445' 2024-09-21T00:30:02.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40633 fp=#0x200000005d9' 2024-09-21T00:30:02.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40665 fp=#0x10000000d11' 2024-09-21T00:30:02.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40697 fp=#0x10000000c99' 2024-09-21T00:30:02.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40729 fp=#0x20000000553' 2024-09-21T00:30:02.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40793 fp=#0x3000000141e' 2024-09-21T00:30:02.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40857 fp=#0x10000004b02' 2024-09-21T00:30:02.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40889 fp=#0x1000000494b' 2024-09-21T00:30:02.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40985 fp=#0x10000004ab1' 2024-09-21T00:30:02.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41017 fp=#0x10000003553' 2024-09-21T00:30:02.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41049 fp=#0x20000000722' 2024-09-21T00:30:02.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41081 fp=#0x100000036e5' 2024-09-21T00:30:02.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41113 fp=#0x10000000dd6' 2024-09-21T00:30:02.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41145 fp=#0x2000000047a' 2024-09-21T00:30:02.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41177 fp=#0x20000000ab4' 2024-09-21T00:30:02.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41209 fp=#0x20000000c1c' 2024-09-21T00:30:02.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41241 fp=#0x20000000b9c' 2024-09-21T00:30:02.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41273 fp=#0x10000000dba' 2024-09-21T00:30:02.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41305 fp=#0x10000000cf9' 2024-09-21T00:30:02.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41337 fp=#0x20000000b24' 2024-09-21T00:30:02.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41369 fp=#0x10000000ecd' 2024-09-21T00:30:02.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41401 fp=#0x10000000a22' 2024-09-21T00:30:02.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41433 fp=#0x10000000a26' 2024-09-21T00:30:02.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41465 fp=#0x10000000be1' 2024-09-21T00:30:02.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41497 fp=#0x10000000a34' 2024-09-21T00:30:02.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41529 fp=#0x100000009bb' 2024-09-21T00:30:02.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41561 fp=#0x30000001908' 2024-09-21T00:30:02.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41689 fp=#0x3000000091b' 2024-09-21T00:30:02.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41785 fp=#0x300000007c1' 2024-09-21T00:30:02.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41849 fp=#0x30000000893' 2024-09-21T00:30:02.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41881 fp=#0x300000009d2' 2024-09-21T00:30:02.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41913 fp=#0x30000000977' 2024-09-21T00:30:02.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41945 fp=#0x30000000989' 2024-09-21T00:30:02.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41977 fp=#0x300000007c2' 2024-09-21T00:30:02.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42009 fp=#0x30000000958' 2024-09-21T00:30:02.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42073 fp=#0x300000007c7' 2024-09-21T00:30:02.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42137 fp=#0x30000000964' 2024-09-21T00:30:02.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42169 fp=#0x30000000bce' 2024-09-21T00:30:02.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42201 fp=#0x30000001bb3' 2024-09-21T00:30:02.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42233 fp=#0x30000002093' 2024-09-21T00:30:02.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42265 fp=#0x30000000b72' 2024-09-21T00:30:02.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42297 fp=#0x30000002080' 2024-09-21T00:30:02.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42329 fp=#0x30000000b50' 2024-09-21T00:30:02.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42361 fp=#0x30000000b22' 2024-09-21T00:30:02.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42393 fp=#0x30000000af8' 2024-09-21T00:30:02.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42425 fp=#0x30000000afb' 2024-09-21T00:30:02.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42457 fp=#0x30000001b41' 2024-09-21T00:30:02.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42489 fp=#0x10000004140' 2024-09-21T00:30:02.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42521 fp=#0x1000000414f' 2024-09-21T00:30:02.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42553 fp=#0x30000000a41' 2024-09-21T00:30:02.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42649 fp=#0x30000001258' 2024-09-21T00:30:02.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42713 fp=#0x100000019fb' 2024-09-21T00:30:02.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42745 fp=#0x100000018e5' 2024-09-21T00:30:02.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42841 fp=#0x10000001908' 2024-09-21T00:30:02.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42905 fp=#0x30000000e44' 2024-09-21T00:30:02.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42937 fp=#0x30000000dff' 2024-09-21T00:30:02.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43001 fp=#0x30000000dfb' 2024-09-21T00:30:02.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43033 fp=#0x3000000176f' 2024-09-21T00:30:02.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43065 fp=#0x3000000169a' 2024-09-21T00:30:02.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43097 fp=#0x300000016c1' 2024-09-21T00:30:02.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43129 fp=#0x30000001752' 2024-09-21T00:30:02.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43161 fp=#0x1000000185e' 2024-09-21T00:30:02.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43193 fp=#0x3000000072e' 2024-09-21T00:30:02.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43257 fp=#0x100000018a1' 2024-09-21T00:30:02.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43289 fp=#0x30000000f8a' 2024-09-21T00:30:02.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43321 fp=#0x30000000f41' 2024-09-21T00:30:02.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43385 fp=#0x30000000f77' 2024-09-21T00:30:02.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43449 fp=#0x30000002390' 2024-09-21T00:30:02.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43481 fp=#0x30000002375' 2024-09-21T00:30:02.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43513 fp=#0x30000002397' 2024-09-21T00:30:02.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43545 fp=#0x30000002371' 2024-09-21T00:30:02.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43577 fp=#0x3000000210b' 2024-09-21T00:30:02.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43609 fp=#0x30000002109' 2024-09-21T00:30:02.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43641 fp=#0x300000020fe' 2024-09-21T00:30:02.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43673 fp=#0x30000002190' 2024-09-21T00:30:02.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43705 fp=#0x30000000ff2' 2024-09-21T00:30:02.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43737 fp=#0x300000021e0' 2024-09-21T00:30:02.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43769 fp=#0x30000000fa2' 2024-09-21T00:30:02.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43801 fp=#0x1000000398a' 2024-09-21T00:30:02.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43897 fp=#0x10000003996' 2024-09-21T00:30:02.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43929 fp=#0x30000001324' 2024-09-21T00:30:02.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31739 fp=#0x10000000005' 2024-09-21T00:30:02.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31770 fp=#0x10000003479' 2024-09-21T00:30:02.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31802 fp=#0x10000002a95' 2024-09-21T00:30:02.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31930 fp=#0x300000007b2' 2024-09-21T00:30:02.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34682 fp=#0x30000002238' 2024-09-21T00:30:02.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34714 fp=#0x3000000223c' 2024-09-21T00:30:02.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34746 fp=#0x30000002263' 2024-09-21T00:30:02.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34778 fp=#0x10000000b46' 2024-09-21T00:30:02.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34842 fp=#0x100000027ec' 2024-09-21T00:30:02.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34970 fp=#0x3000000207f' 2024-09-21T00:30:02.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35002 fp=#0x30000002068' 2024-09-21T00:30:02.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35034 fp=#0x10000003867' 2024-09-21T00:30:02.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35066 fp=#0x10000003853' 2024-09-21T00:30:02.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35098 fp=#0x10000003b2e' 2024-09-21T00:30:02.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35130 fp=#0x1000000391f' 2024-09-21T00:30:02.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35162 fp=#0x100000038e9' 2024-09-21T00:30:02.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35194 fp=#0x10000004126' 2024-09-21T00:30:02.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35226 fp=#0x10000003a8b' 2024-09-21T00:30:02.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35258 fp=#0x10000003a2f' 2024-09-21T00:30:02.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35290 fp=#0x10000003a02' 2024-09-21T00:30:02.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35322 fp=#0x10000003971' 2024-09-21T00:30:02.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35354 fp=#0x100000039d6' 2024-09-21T00:30:02.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35386 fp=#0x30000001111' 2024-09-21T00:30:02.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35418 fp=#0x1000000392b' 2024-09-21T00:30:02.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35450 fp=#0x100000039e5' 2024-09-21T00:30:02.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35482 fp=#0x3000000114e' 2024-09-21T00:30:02.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35514 fp=#0x3000000113f' 2024-09-21T00:30:02.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35546 fp=#0x30000001127' 2024-09-21T00:30:02.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35578 fp=#0x30000002053' 2024-09-21T00:30:02.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35610 fp=#0x30000002040' 2024-09-21T00:30:02.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35674 fp=#0x30000001ffc' 2024-09-21T00:30:02.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35802 fp=#0x1000000339b' 2024-09-21T00:30:02.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35834 fp=#0x1000000342e' 2024-09-21T00:30:02.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35866 fp=#0x10000003437' 2024-09-21T00:30:02.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35898 fp=#0x1000000337d' 2024-09-21T00:30:02.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35930 fp=#0x30000001203' 2024-09-21T00:30:02.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35962 fp=#0x300000011fc' 2024-09-21T00:30:02.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35994 fp=#0x10000003c2b' 2024-09-21T00:30:02.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36026 fp=#0x10000003b4e' 2024-09-21T00:30:02.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36154 fp=#0x100000020a4' 2024-09-21T00:30:02.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36186 fp=#0x10000001ee0' 2024-09-21T00:30:02.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36218 fp=#0x10000001f77' 2024-09-21T00:30:02.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36250 fp=#0x1000000201e' 2024-09-21T00:30:02.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36282 fp=#0x300000015b6' 2024-09-21T00:30:02.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36314 fp=#0x300000015af' 2024-09-21T00:30:02.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36442 fp=#0x10000001d7e' 2024-09-21T00:30:02.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36474 fp=#0x10000001dac' 2024-09-21T00:30:02.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36506 fp=#0x10000001dcf' 2024-09-21T00:30:02.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36538 fp=#0x10000001e43' 2024-09-21T00:30:02.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36602 fp=#0x1000000445b' 2024-09-21T00:30:02.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36634 fp=#0x10000004499' 2024-09-21T00:30:02.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36666 fp=#0x100000037b6' 2024-09-21T00:30:02.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36698 fp=#0x10000004485' 2024-09-21T00:30:02.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36730 fp=#0x10000004710' 2024-09-21T00:30:02.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36762 fp=#0x100000047b1' 2024-09-21T00:30:02.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36794 fp=#0x30000001a8e' 2024-09-21T00:30:02.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36826 fp=#0x30000001a86' 2024-09-21T00:30:02.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36922 fp=#0x1000000354f' 2024-09-21T00:30:02.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36954 fp=#0x10000003685' 2024-09-21T00:30:02.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36986 fp=#0x10000002568' 2024-09-21T00:30:02.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37018 fp=#0x1000000359b' 2024-09-21T00:30:02.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37082 fp=#0x300000017ff' 2024-09-21T00:30:02.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37146 fp=#0x30000001796' 2024-09-21T00:30:02.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37210 fp=#0x100000031a7' 2024-09-21T00:30:02.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37242 fp=#0x100000031a1' 2024-09-21T00:30:02.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37274 fp=#0x100000030dc' 2024-09-21T00:30:02.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37338 fp=#0x30000001ad5' 2024-09-21T00:30:02.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37370 fp=#0x30000001416' 2024-09-21T00:30:02.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37402 fp=#0x30000001b25' 2024-09-21T00:30:02.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37434 fp=#0x30000001b04' 2024-09-21T00:30:02.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37466 fp=#0x30000001b16' 2024-09-21T00:30:02.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37498 fp=#0x30000001b7e' 2024-09-21T00:30:02.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37530 fp=#0x30000001b65' 2024-09-21T00:30:02.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37562 fp=#0x30000001b66' 2024-09-21T00:30:02.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37594 fp=#0x30000001b68' 2024-09-21T00:30:02.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37658 fp=#0x30000000c82' 2024-09-21T00:30:02.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37690 fp=#0x30000000c50' 2024-09-21T00:30:02.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37722 fp=#0x30000000c8f' 2024-09-21T00:30:02.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37786 fp=#0x30000000c51' 2024-09-21T00:30:02.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37850 fp=#0x300000012c6' 2024-09-21T00:30:02.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37882 fp=#0x300000020f6' 2024-09-21T00:30:02.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37914 fp=#0x3000000228f' 2024-09-21T00:30:02.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38042 fp=#0x30000000dc0' 2024-09-21T00:30:02.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38074 fp=#0x1000000076d' 2024-09-21T00:30:02.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38106 fp=#0x10000000732' 2024-09-21T00:30:02.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38138 fp=#0x10000000dfe' 2024-09-21T00:30:02.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38170 fp=#0x10000000eb2' 2024-09-21T00:30:02.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38202 fp=#0x1000000076b' 2024-09-21T00:30:02.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38298 fp=#0x30000001a4d' 2024-09-21T00:30:02.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38330 fp=#0x30000001a7f' 2024-09-21T00:30:02.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38362 fp=#0x30000001a63' 2024-09-21T00:30:02.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38394 fp=#0x100000032d4' 2024-09-21T00:30:02.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38426 fp=#0x30000001cde' 2024-09-21T00:30:02.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38490 fp=#0x30000001cd1' 2024-09-21T00:30:02.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38554 fp=#0x10000002107' 2024-09-21T00:30:02.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38586 fp=#0x10000000596' 2024-09-21T00:30:02.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38650 fp=#0x30000001bbc' 2024-09-21T00:30:02.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38682 fp=#0x10000000a62' 2024-09-21T00:30:02.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38714 fp=#0x30000001bd6' 2024-09-21T00:30:02.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38746 fp=#0x30000001bd9' 2024-09-21T00:30:02.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38778 fp=#0x30000001c1f' 2024-09-21T00:30:02.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38810 fp=#0x1000000296a' 2024-09-21T00:30:02.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38842 fp=#0x10000002956' 2024-09-21T00:30:02.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38874 fp=#0x1000000292c' 2024-09-21T00:30:02.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38970 fp=#0x30000001c51' 2024-09-21T00:30:02.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39034 fp=#0x30000001c44' 2024-09-21T00:30:02.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39194 fp=#0x10000003506' 2024-09-21T00:30:02.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39226 fp=#0x10000001c12' 2024-09-21T00:30:02.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39258 fp=#0x1000000358a' 2024-09-21T00:30:02.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39290 fp=#0x100000034c4' 2024-09-21T00:30:02.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39354 fp=#0x10000003fe4' 2024-09-21T00:30:02.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39386 fp=#0x10000003efd' 2024-09-21T00:30:02.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39418 fp=#0x10000003ee1' 2024-09-21T00:30:02.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39450 fp=#0x30000001fbb' 2024-09-21T00:30:02.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39482 fp=#0x30000001f87' 2024-09-21T00:30:02.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39514 fp=#0x30000001f5f' 2024-09-21T00:30:02.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39546 fp=#0x30000001f39' 2024-09-21T00:30:02.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39578 fp=#0x30000001f90' 2024-09-21T00:30:02.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39642 fp=#0x10000002a9d' 2024-09-21T00:30:02.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39674 fp=#0x10000002b12' 2024-09-21T00:30:02.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39706 fp=#0x30000000eb7' 2024-09-21T00:30:02.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39738 fp=#0x30000000ea6' 2024-09-21T00:30:02.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39770 fp=#0x30000000e75' 2024-09-21T00:30:02.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39834 fp=#0x30000000eaa' 2024-09-21T00:30:02.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39866 fp=#0x30000000edc' 2024-09-21T00:30:02.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39898 fp=#0x30000001137' 2024-09-21T00:30:02.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39930 fp=#0x30000001122' 2024-09-21T00:30:02.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40026 fp=#0x30000001544' 2024-09-21T00:30:02.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40058 fp=#0x30000001665' 2024-09-21T00:30:02.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40090 fp=#0x3000000180e' 2024-09-21T00:30:02.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40122 fp=#0x30000001858' 2024-09-21T00:30:02.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40154 fp=#0x3000000182a' 2024-09-21T00:30:02.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40186 fp=#0x30000001d24' 2024-09-21T00:30:02.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40218 fp=#0x10000001d54' 2024-09-21T00:30:02.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40282 fp=#0x10000001d23' 2024-09-21T00:30:02.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40314 fp=#0x300000009f1' 2024-09-21T00:30:02.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40346 fp=#0x10000004e70' 2024-09-21T00:30:02.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40378 fp=#0x10000004d32' 2024-09-21T00:30:02.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40410 fp=#0x300000022c8' 2024-09-21T00:30:02.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40474 fp=#0x3000000231c' 2024-09-21T00:30:02.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40538 fp=#0x200000008f9' 2024-09-21T00:30:02.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40602 fp=#0x2000000045a' 2024-09-21T00:30:02.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40634 fp=#0x20000000591' 2024-09-21T00:30:02.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40666 fp=#0x200000005a3' 2024-09-21T00:30:02.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40698 fp=#0x20000000585' 2024-09-21T00:30:02.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40730 fp=#0x20000000573' 2024-09-21T00:30:02.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40762 fp=#0x10000000f42' 2024-09-21T00:30:02.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40794 fp=#0x3000000144b' 2024-09-21T00:30:02.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40826 fp=#0x10000004d0d' 2024-09-21T00:30:02.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40858 fp=#0x10000004a9c' 2024-09-21T00:30:02.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40890 fp=#0x10000004958' 2024-09-21T00:30:02.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40922 fp=#0x10000004ce4' 2024-09-21T00:30:02.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40954 fp=#0x1000000493f' 2024-09-21T00:30:02.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40986 fp=#0x10000004ad4' 2024-09-21T00:30:02.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41018 fp=#0x10000003490' 2024-09-21T00:30:02.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41050 fp=#0x10000003483' 2024-09-21T00:30:02.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41082 fp=#0x20000000720' 2024-09-21T00:30:02.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41114 fp=#0x10000001027' 2024-09-21T00:30:02.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41146 fp=#0x10000000f2a' 2024-09-21T00:30:02.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41178 fp=#0x2000000047e' 2024-09-21T00:30:02.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41210 fp=#0x10000000dab' 2024-09-21T00:30:02.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41242 fp=#0x20000000a95' 2024-09-21T00:30:02.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41274 fp=#0x10000000e4d' 2024-09-21T00:30:02.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41306 fp=#0x20000000c33' 2024-09-21T00:30:02.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41370 fp=#0x100000009b6' 2024-09-21T00:30:02.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41402 fp=#0x10000000c33' 2024-09-21T00:30:02.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41434 fp=#0x3000000206d' 2024-09-21T00:30:02.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41466 fp=#0x30000002076' 2024-09-21T00:30:02.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41498 fp=#0x10000000928' 2024-09-21T00:30:02.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41530 fp=#0x10000000c51' 2024-09-21T00:30:02.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41562 fp=#0x300000018fe' 2024-09-21T00:30:02.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41754 fp=#0x300000007d7' 2024-09-21T00:30:02.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41818 fp=#0x30000000866' 2024-09-21T00:30:02.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41850 fp=#0x300000008bb' 2024-09-21T00:30:02.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41882 fp=#0x3000000093f' 2024-09-21T00:30:02.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41914 fp=#0x3000000099d' 2024-09-21T00:30:02.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41946 fp=#0x300000008dc' 2024-09-21T00:30:02.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41978 fp=#0x300000007fe' 2024-09-21T00:30:02.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42010 fp=#0x30000000869' 2024-09-21T00:30:02.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42042 fp=#0x300000009b7' 2024-09-21T00:30:02.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42074 fp=#0x30000000942' 2024-09-21T00:30:02.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42106 fp=#0x30000000950' 2024-09-21T00:30:02.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42138 fp=#0x3000000087f' 2024-09-21T00:30:02.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42170 fp=#0x30000000b5e' 2024-09-21T00:30:02.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42202 fp=#0x30000000b97' 2024-09-21T00:30:02.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42234 fp=#0x30000001bb9' 2024-09-21T00:30:02.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42266 fp=#0x30000001ba3' 2024-09-21T00:30:02.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42298 fp=#0x30000001b9f' 2024-09-21T00:30:02.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42330 fp=#0x30000000b31' 2024-09-21T00:30:02.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42362 fp=#0x30000000b4c' 2024-09-21T00:30:02.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42394 fp=#0x30000000add' 2024-09-21T00:30:02.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42426 fp=#0x30000000b40' 2024-09-21T00:30:02.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42458 fp=#0x30000000a76' 2024-09-21T00:30:02.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42522 fp=#0x30000000a55' 2024-09-21T00:30:02.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42554 fp=#0x30000000a4e' 2024-09-21T00:30:02.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42650 fp=#0x3000000125a' 2024-09-21T00:30:02.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42682 fp=#0x100000019b7' 2024-09-21T00:30:02.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42714 fp=#0x100000019c1' 2024-09-21T00:30:02.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42746 fp=#0x1000000196a' 2024-09-21T00:30:02.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42778 fp=#0x100000018e1' 2024-09-21T00:30:02.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42810 fp=#0x100000018ea' 2024-09-21T00:30:02.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42906 fp=#0x30000000df5' 2024-09-21T00:30:02.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42970 fp=#0x30000000df9' 2024-09-21T00:30:02.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43034 fp=#0x300000016e5' 2024-09-21T00:30:02.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43066 fp=#0x300000016b3' 2024-09-21T00:30:02.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43098 fp=#0x30000001754' 2024-09-21T00:30:02.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43130 fp=#0x300000016c9' 2024-09-21T00:30:02.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43226 fp=#0x1000000185a' 2024-09-21T00:30:02.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43290 fp=#0x30000000f3e' 2024-09-21T00:30:02.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43322 fp=#0x30000000f45' 2024-09-21T00:30:02.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43450 fp=#0x3000000238c' 2024-09-21T00:30:02.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43482 fp=#0x300000023cc' 2024-09-21T00:30:02.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43514 fp=#0x3000000239c' 2024-09-21T00:30:02.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43546 fp=#0x300000023e8' 2024-09-21T00:30:02.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43578 fp=#0x300000010dd' 2024-09-21T00:30:02.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43610 fp=#0x30000000fa6' 2024-09-21T00:30:02.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43642 fp=#0x30000000fe1' 2024-09-21T00:30:02.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43770 fp=#0x300000010e1' 2024-09-21T00:30:02.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43802 fp=#0x10000003994' 2024-09-21T00:30:02.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43866 fp=#0x10000003982' 2024-09-21T00:30:02.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43930 fp=#0x300000013c9' 2024-09-21T00:30:02.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31740 fp=#0x10000000006' 2024-09-21T00:30:02.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31771 fp=#0x10000002539' 2024-09-21T00:30:02.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31803 fp=#0x30000000be6' 2024-09-21T00:30:02.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31931 fp=#0x3000000080f' 2024-09-21T00:30:02.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34683 fp=#0x300000022b4' 2024-09-21T00:30:02.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34747 fp=#0x30000002224' 2024-09-21T00:30:02.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34843 fp=#0x100000026e2' 2024-09-21T00:30:02.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34875 fp=#0x10000002700' 2024-09-21T00:30:02.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34907 fp=#0x10000002709' 2024-09-21T00:30:02.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34971 fp=#0x100000032a6' 2024-09-21T00:30:02.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35003 fp=#0x100000032d6' 2024-09-21T00:30:02.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35067 fp=#0x1000000386f' 2024-09-21T00:30:02.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35099 fp=#0x10000003ad6' 2024-09-21T00:30:02.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35131 fp=#0x1000000417f' 2024-09-21T00:30:02.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35163 fp=#0x10000003b24' 2024-09-21T00:30:02.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35195 fp=#0x10000003b2c' 2024-09-21T00:30:02.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35227 fp=#0x1000000413a' 2024-09-21T00:30:02.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35259 fp=#0x100000039f0' 2024-09-21T00:30:02.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35291 fp=#0x1000000414d' 2024-09-21T00:30:02.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35323 fp=#0x10000003ae9' 2024-09-21T00:30:02.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35355 fp=#0x1000000391c' 2024-09-21T00:30:02.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35419 fp=#0x30000001645' 2024-09-21T00:30:02.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35451 fp=#0x1000000390c' 2024-09-21T00:30:02.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35483 fp=#0x30000001165' 2024-09-21T00:30:02.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35515 fp=#0x3000000117a' 2024-09-21T00:30:02.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35547 fp=#0x30000001149' 2024-09-21T00:30:02.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35579 fp=#0x3000000205c' 2024-09-21T00:30:02.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35611 fp=#0x30000001fd0' 2024-09-21T00:30:02.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35643 fp=#0x30000001ffd' 2024-09-21T00:30:02.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35675 fp=#0x30000002039' 2024-09-21T00:30:02.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35835 fp=#0x1000000346b' 2024-09-21T00:30:02.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35867 fp=#0x100000033cb' 2024-09-21T00:30:02.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35931 fp=#0x300000011ff' 2024-09-21T00:30:02.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35963 fp=#0x300000011d0' 2024-09-21T00:30:02.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35995 fp=#0x10000003c1c' 2024-09-21T00:30:02.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36091 fp=#0x10000002d6d' 2024-09-21T00:30:02.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36155 fp=#0x10000001f22' 2024-09-21T00:30:02.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36187 fp=#0x10000004609' 2024-09-21T00:30:02.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36219 fp=#0x10000001f60' 2024-09-21T00:30:02.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36251 fp=#0x10000004607' 2024-09-21T00:30:02.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36283 fp=#0x3000000160a' 2024-09-21T00:30:02.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36443 fp=#0x10000001e11' 2024-09-21T00:30:02.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36475 fp=#0x10000001dd3' 2024-09-21T00:30:02.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36507 fp=#0x10000001d72' 2024-09-21T00:30:02.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36539 fp=#0x10000001e35' 2024-09-21T00:30:02.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36571 fp=#0x100000037a5' 2024-09-21T00:30:02.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36603 fp=#0x1000000445d' 2024-09-21T00:30:02.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36635 fp=#0x10000004463' 2024-09-21T00:30:02.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36667 fp=#0x1000000441c' 2024-09-21T00:30:02.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36699 fp=#0x1000000376e' 2024-09-21T00:30:02.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36731 fp=#0x10000004784' 2024-09-21T00:30:02.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36763 fp=#0x30000001aad' 2024-09-21T00:30:02.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36795 fp=#0x30000001a95' 2024-09-21T00:30:02.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36827 fp=#0x30000001a90' 2024-09-21T00:30:02.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36859 fp=#0x100000036a5' 2024-09-21T00:30:02.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36891 fp=#0x100000035e1' 2024-09-21T00:30:02.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36923 fp=#0x100000036aa' 2024-09-21T00:30:02.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36955 fp=#0x1000000269d' 2024-09-21T00:30:02.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37179 fp=#0x100000030b6' 2024-09-21T00:30:02.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37211 fp=#0x10000001354' 2024-09-21T00:30:02.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37275 fp=#0x100000030e7' 2024-09-21T00:30:02.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37307 fp=#0x10000003185' 2024-09-21T00:30:02.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37339 fp=#0x30000001adc' 2024-09-21T00:30:02.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37371 fp=#0x30000001b1b' 2024-09-21T00:30:02.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37403 fp=#0x30000001ab6' 2024-09-21T00:30:02.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37435 fp=#0x30000000bff' 2024-09-21T00:30:02.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37467 fp=#0x300000015b3' 2024-09-21T00:30:02.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37499 fp=#0x30000001b75' 2024-09-21T00:30:02.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37531 fp=#0x30000001076' 2024-09-21T00:30:02.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37563 fp=#0x30000001b6d' 2024-09-21T00:30:02.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37595 fp=#0x30000001066' 2024-09-21T00:30:02.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37627 fp=#0x30000000c7e' 2024-09-21T00:30:02.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37691 fp=#0x30000000c3e' 2024-09-21T00:30:02.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37723 fp=#0x30000000c70' 2024-09-21T00:30:02.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37755 fp=#0x30000000c9c' 2024-09-21T00:30:02.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37787 fp=#0x30000000c5e' 2024-09-21T00:30:02.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37851 fp=#0x3000000127f' 2024-09-21T00:30:02.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37883 fp=#0x3000000126e' 2024-09-21T00:30:02.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37979 fp=#0x30000000db8' 2024-09-21T00:30:02.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38075 fp=#0x10000000e6a' 2024-09-21T00:30:02.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38107 fp=#0x10000000f06' 2024-09-21T00:30:02.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38139 fp=#0x10000000ef6' 2024-09-21T00:30:02.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38171 fp=#0x1000000074c' 2024-09-21T00:30:02.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38203 fp=#0x1000000074b' 2024-09-21T00:30:02.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38299 fp=#0x30000001a62' 2024-09-21T00:30:02.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38331 fp=#0x30000001a75' 2024-09-21T00:30:02.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38363 fp=#0x30000001a5d' 2024-09-21T00:30:02.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38523 fp=#0x10000002152' 2024-09-21T00:30:02.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38587 fp=#0x30000001c4b' 2024-09-21T00:30:02.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38619 fp=#0x30000001c1b' 2024-09-21T00:30:02.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38651 fp=#0x30000001be3' 2024-09-21T00:30:02.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38683 fp=#0x1000000051d' 2024-09-21T00:30:02.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38747 fp=#0x30000001c52' 2024-09-21T00:30:02.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38811 fp=#0x10000002985' 2024-09-21T00:30:02.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38843 fp=#0x1000000295e' 2024-09-21T00:30:02.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38875 fp=#0x300000019ec' 2024-09-21T00:30:02.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38907 fp=#0x300000019fd' 2024-09-21T00:30:02.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38939 fp=#0x30000001d83' 2024-09-21T00:30:02.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38971 fp=#0x30000001c62' 2024-09-21T00:30:02.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39003 fp=#0x30000001d26' 2024-09-21T00:30:02.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39163 fp=#0x100000034a6' 2024-09-21T00:30:02.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39195 fp=#0x10000003530' 2024-09-21T00:30:02.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39227 fp=#0x1000000112c' 2024-09-21T00:30:02.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39291 fp=#0x10000001124' 2024-09-21T00:30:02.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39355 fp=#0x10000003fbc' 2024-09-21T00:30:02.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39387 fp=#0x10000003f32' 2024-09-21T00:30:02.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39419 fp=#0x30000001360' 2024-09-21T00:30:02.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39451 fp=#0x30000001efc' 2024-09-21T00:30:02.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39483 fp=#0x30000001f0b' 2024-09-21T00:30:02.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39515 fp=#0x30000001f79' 2024-09-21T00:30:02.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39547 fp=#0x30000001fb0' 2024-09-21T00:30:02.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39579 fp=#0x30000001f6d' 2024-09-21T00:30:02.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39675 fp=#0x10000002a82' 2024-09-21T00:30:02.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39707 fp=#0x30000000e5e' 2024-09-21T00:30:02.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39739 fp=#0x30000001986' 2024-09-21T00:30:02.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39771 fp=#0x30000001981' 2024-09-21T00:30:02.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39803 fp=#0x30000000e72' 2024-09-21T00:30:02.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39835 fp=#0x30000000e95' 2024-09-21T00:30:02.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39867 fp=#0x30000000e65' 2024-09-21T00:30:02.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39963 fp=#0x300000010f7' 2024-09-21T00:30:02.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39995 fp=#0x30000001007' 2024-09-21T00:30:02.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40027 fp=#0x30000001851' 2024-09-21T00:30:02.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40091 fp=#0x3000000185a' 2024-09-21T00:30:02.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40123 fp=#0x300000017f0' 2024-09-21T00:30:02.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40155 fp=#0x30000001675' 2024-09-21T00:30:02.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40219 fp=#0x30000000dc6' 2024-09-21T00:30:02.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40251 fp=#0x10000001d03' 2024-09-21T00:30:02.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40283 fp=#0x30000000da8' 2024-09-21T00:30:02.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40315 fp=#0x30000000da4' 2024-09-21T00:30:02.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40347 fp=#0x10000004ce5' 2024-09-21T00:30:02.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40379 fp=#0x10000004e63' 2024-09-21T00:30:02.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40411 fp=#0x10000004e13' 2024-09-21T00:30:02.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40443 fp=#0x10000004c6b' 2024-09-21T00:30:02.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40475 fp=#0x300000022d8' 2024-09-21T00:30:02.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40571 fp=#0x2000000059a' 2024-09-21T00:30:02.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40603 fp=#0x20000000441' 2024-09-21T00:30:02.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40635 fp=#0x200000005d5' 2024-09-21T00:30:02.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40667 fp=#0x2000000054a' 2024-09-21T00:30:02.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40699 fp=#0x10000000d64' 2024-09-21T00:30:02.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40731 fp=#0x10000000cad' 2024-09-21T00:30:02.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40763 fp=#0x10000000eeb' 2024-09-21T00:30:02.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40795 fp=#0x3000000144f' 2024-09-21T00:30:02.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40891 fp=#0x10000004cd5' 2024-09-21T00:30:02.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40923 fp=#0x300000020f7' 2024-09-21T00:30:02.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40955 fp=#0x10000004abe' 2024-09-21T00:30:02.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41019 fp=#0x100000036de' 2024-09-21T00:30:02.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41051 fp=#0x10000003560' 2024-09-21T00:30:02.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41083 fp=#0x1000000360c' 2024-09-21T00:30:02.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41115 fp=#0x10000000d0c' 2024-09-21T00:30:02.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41147 fp=#0x1000000100f' 2024-09-21T00:30:02.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41179 fp=#0x20000000c60' 2024-09-21T00:30:02.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41211 fp=#0x20000000c43' 2024-09-21T00:30:02.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41243 fp=#0x200000006d4' 2024-09-21T00:30:02.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41275 fp=#0x10000000e99' 2024-09-21T00:30:02.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41307 fp=#0x20000000c0f' 2024-09-21T00:30:02.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41371 fp=#0x10000000c8a' 2024-09-21T00:30:02.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41403 fp=#0x10000000be8' 2024-09-21T00:30:02.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41435 fp=#0x10000000b27' 2024-09-21T00:30:02.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41467 fp=#0x10000000aa3' 2024-09-21T00:30:02.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41499 fp=#0x10000000997' 2024-09-21T00:30:02.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41531 fp=#0x10000000c82' 2024-09-21T00:30:02.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41595 fp=#0x30000001853' 2024-09-21T00:30:02.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41691 fp=#0x30000000919' 2024-09-21T00:30:02.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41755 fp=#0x300000007c3' 2024-09-21T00:30:02.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41819 fp=#0x300000009d7' 2024-09-21T00:30:02.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41883 fp=#0x300000008d2' 2024-09-21T00:30:02.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41915 fp=#0x300000008c5' 2024-09-21T00:30:02.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41947 fp=#0x300000009b5' 2024-09-21T00:30:02.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41979 fp=#0x300000007f6' 2024-09-21T00:30:02.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42011 fp=#0x3000000097d' 2024-09-21T00:30:02.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42043 fp=#0x30000001d64' 2024-09-21T00:30:02.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42075 fp=#0x3000000097f' 2024-09-21T00:30:02.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42107 fp=#0x30000000804' 2024-09-21T00:30:02.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42139 fp=#0x30000001d5b' 2024-09-21T00:30:02.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42171 fp=#0x30000000bc9' 2024-09-21T00:30:02.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42203 fp=#0x30000001b98' 2024-09-21T00:30:02.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42235 fp=#0x30000002090' 2024-09-21T00:30:02.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42267 fp=#0x30000000b6d' 2024-09-21T00:30:02.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42299 fp=#0x30000001ba9' 2024-09-21T00:30:02.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42331 fp=#0x30000000b52' 2024-09-21T00:30:02.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42363 fp=#0x30000000b3a' 2024-09-21T00:30:02.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42395 fp=#0x30000000b3d' 2024-09-21T00:30:02.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42427 fp=#0x30000000ae4' 2024-09-21T00:30:02.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42459 fp=#0x30000000a65' 2024-09-21T00:30:02.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42491 fp=#0x30000000a45' 2024-09-21T00:30:02.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42523 fp=#0x30000001b4a' 2024-09-21T00:30:02.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42555 fp=#0x30000001b54' 2024-09-21T00:30:02.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42587 fp=#0x30000000a57' 2024-09-21T00:30:02.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42651 fp=#0x3000000125e' 2024-09-21T00:30:02.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42683 fp=#0x10000001a47' 2024-09-21T00:30:02.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42715 fp=#0x10000001967' 2024-09-21T00:30:02.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42747 fp=#0x10000001976' 2024-09-21T00:30:02.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42875 fp=#0x1000000190b' 2024-09-21T00:30:02.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42907 fp=#0x30000000e43' 2024-09-21T00:30:02.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43003 fp=#0x30000000e06' 2024-09-21T00:30:02.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43035 fp=#0x30000001698' 2024-09-21T00:30:02.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43067 fp=#0x30000001766' 2024-09-21T00:30:02.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43099 fp=#0x30000001700' 2024-09-21T00:30:02.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43131 fp=#0x300000016d4' 2024-09-21T00:30:02.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43227 fp=#0x100000018a3' 2024-09-21T00:30:02.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43355 fp=#0x30000000f4c' 2024-09-21T00:30:02.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43419 fp=#0x1000000416c' 2024-09-21T00:30:02.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43451 fp=#0x30000002374' 2024-09-21T00:30:02.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43483 fp=#0x300000023f5' 2024-09-21T00:30:02.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43515 fp=#0x3000000237d' 2024-09-21T00:30:02.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43547 fp=#0x30000002396' 2024-09-21T00:30:02.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43579 fp=#0x30000000fe8' 2024-09-21T00:30:02.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43611 fp=#0x3000000210c' 2024-09-21T00:30:02.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43643 fp=#0x30000000fcd' 2024-09-21T00:30:02.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43739 fp=#0x300000010c1' 2024-09-21T00:30:02.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43771 fp=#0x3000000219b' 2024-09-21T00:30:02.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43803 fp=#0x100000039da' 2024-09-21T00:30:02.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43835 fp=#0x10000003a10' 2024-09-21T00:30:02.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43867 fp=#0x100000039cc' 2024-09-21T00:30:02.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43899 fp=#0x10000003a29' 2024-09-21T00:30:02.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43931 fp=#0x30000001331' 2024-09-21T00:30:02.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31741 fp=#0x10000000201' 2024-09-21T00:30:02.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31804 fp=#0x30000000fc4' 2024-09-21T00:30:02.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31932 fp=#0x300000007ab' 2024-09-21T00:30:02.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34684 fp=#0x300000022ab' 2024-09-21T00:30:02.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34844 fp=#0x10000004858' 2024-09-21T00:30:02.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34940 fp=#0x10000004866' 2024-09-21T00:30:02.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34972 fp=#0x10000003306' 2024-09-21T00:30:02.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35004 fp=#0x10000003850' 2024-09-21T00:30:02.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35036 fp=#0x100000032a0' 2024-09-21T00:30:02.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35068 fp=#0x300000020b3' 2024-09-21T00:30:02.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35100 fp=#0x10000003b34' 2024-09-21T00:30:02.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35132 fp=#0x10000003997' 2024-09-21T00:30:02.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35164 fp=#0x10000003aeb' 2024-09-21T00:30:02.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35196 fp=#0x100000041eb' 2024-09-21T00:30:02.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35228 fp=#0x10000003b1d' 2024-09-21T00:30:02.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35260 fp=#0x10000003955' 2024-09-21T00:30:02.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35292 fp=#0x100000038c3' 2024-09-21T00:30:02.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35324 fp=#0x100000038f6' 2024-09-21T00:30:02.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35356 fp=#0x100000039cf' 2024-09-21T00:30:02.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35388 fp=#0x10000003a1f' 2024-09-21T00:30:02.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35420 fp=#0x100000038ac' 2024-09-21T00:30:02.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35452 fp=#0x10000003922' 2024-09-21T00:30:02.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35484 fp=#0x30000001168' 2024-09-21T00:30:02.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35516 fp=#0x30000001147' 2024-09-21T00:30:02.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35580 fp=#0x30000002002' 2024-09-21T00:30:02.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35612 fp=#0x30000001fe0' 2024-09-21T00:30:02.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35644 fp=#0x30000002030' 2024-09-21T00:30:02.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35676 fp=#0x30000002032' 2024-09-21T00:30:02.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35804 fp=#0x100000033df' 2024-09-21T00:30:02.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35836 fp=#0x100000033d0' 2024-09-21T00:30:02.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35932 fp=#0x300000011c9' 2024-09-21T00:30:02.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35964 fp=#0x30000001202' 2024-09-21T00:30:02.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36124 fp=#0x10000002cb8' 2024-09-21T00:30:02.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36156 fp=#0x1000000209e' 2024-09-21T00:30:02.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36188 fp=#0x10000001f9d' 2024-09-21T00:30:02.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36220 fp=#0x10000001ed5' 2024-09-21T00:30:02.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36252 fp=#0x10000001eaf' 2024-09-21T00:30:02.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36348 fp=#0x30000001605' 2024-09-21T00:30:02.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36380 fp=#0x300000015bd' 2024-09-21T00:30:02.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36444 fp=#0x10000001e7d' 2024-09-21T00:30:02.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36476 fp=#0x10000001db3' 2024-09-21T00:30:02.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36508 fp=#0x10000001e5d' 2024-09-21T00:30:02.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36540 fp=#0x10000001dd9' 2024-09-21T00:30:02.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36572 fp=#0x10000003808' 2024-09-21T00:30:02.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36604 fp=#0x1000000442a' 2024-09-21T00:30:02.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36636 fp=#0x10000004495' 2024-09-21T00:30:02.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36668 fp=#0x10000003792' 2024-09-21T00:30:02.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36700 fp=#0x10000003767' 2024-09-21T00:30:02.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36732 fp=#0x30000001aa6' 2024-09-21T00:30:02.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36764 fp=#0x10000004743' 2024-09-21T00:30:02.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36796 fp=#0x10000004701' 2024-09-21T00:30:02.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36828 fp=#0x30000001a87' 2024-09-21T00:30:02.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36892 fp=#0x10000003618' 2024-09-21T00:30:02.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36924 fp=#0x10000002574' 2024-09-21T00:30:02.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36956 fp=#0x1000000359e' 2024-09-21T00:30:02.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37020 fp=#0x1000000254b' 2024-09-21T00:30:02.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37052 fp=#0x30000001809' 2024-09-21T00:30:02.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37180 fp=#0x10000003123' 2024-09-21T00:30:02.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37212 fp=#0x10000001366' 2024-09-21T00:30:02.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37244 fp=#0x10000002fca' 2024-09-21T00:30:02.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37276 fp=#0x1000000135e' 2024-09-21T00:30:02.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37308 fp=#0x100000030e0' 2024-09-21T00:30:02.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37340 fp=#0x30000000c0a' 2024-09-21T00:30:02.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37372 fp=#0x30000001ac8' 2024-09-21T00:30:02.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37404 fp=#0x30000000c1f' 2024-09-21T00:30:02.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37436 fp=#0x30000001ac2' 2024-09-21T00:30:02.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37468 fp=#0x30000001b1d' 2024-09-21T00:30:02.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37500 fp=#0x30000001b64' 2024-09-21T00:30:02.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37532 fp=#0x30000001b85' 2024-09-21T00:30:02.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37564 fp=#0x3000000196a' 2024-09-21T00:30:02.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37596 fp=#0x300000010ae' 2024-09-21T00:30:02.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37692 fp=#0x30000000c77' 2024-09-21T00:30:02.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37724 fp=#0x30000000c8d' 2024-09-21T00:30:02.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37756 fp=#0x30000000c4a' 2024-09-21T00:30:02.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37820 fp=#0x30000000ca5' 2024-09-21T00:30:02.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37852 fp=#0x300000012ba' 2024-09-21T00:30:02.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37884 fp=#0x300000012ca' 2024-09-21T00:30:02.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37916 fp=#0x30000002287' 2024-09-21T00:30:02.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37948 fp=#0x30000002257' 2024-09-21T00:30:02.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38044 fp=#0x30000000cf5' 2024-09-21T00:30:02.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38076 fp=#0x10000000768' 2024-09-21T00:30:02.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38140 fp=#0x1000000073d' 2024-09-21T00:30:02.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38204 fp=#0x10000000740' 2024-09-21T00:30:02.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38300 fp=#0x30000001a6b' 2024-09-21T00:30:02.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38332 fp=#0x30000001a59' 2024-09-21T00:30:02.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38364 fp=#0x30000001a64' 2024-09-21T00:30:02.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38396 fp=#0x30000001348' 2024-09-21T00:30:02.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38428 fp=#0x30000001cce' 2024-09-21T00:30:02.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38460 fp=#0x1000000328c' 2024-09-21T00:30:02.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38492 fp=#0x1000000327c' 2024-09-21T00:30:02.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38524 fp=#0x1000000211d' 2024-09-21T00:30:02.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38556 fp=#0x1000000215f' 2024-09-21T00:30:02.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38588 fp=#0x100000005dd' 2024-09-21T00:30:02.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38620 fp=#0x30000001c2a' 2024-09-21T00:30:02.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38652 fp=#0x30000001bb8' 2024-09-21T00:30:02.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38684 fp=#0x30000001c12' 2024-09-21T00:30:02.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38716 fp=#0x100000005a1' 2024-09-21T00:30:02.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38780 fp=#0x30000001c4f' 2024-09-21T00:30:02.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38876 fp=#0x300000019c5' 2024-09-21T00:30:02.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38908 fp=#0x10000002947' 2024-09-21T00:30:02.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38940 fp=#0x30000001c53' 2024-09-21T00:30:02.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38972 fp=#0x30000001c54' 2024-09-21T00:30:02.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39004 fp=#0x30000001c61' 2024-09-21T00:30:02.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39036 fp=#0x30000001d5d' 2024-09-21T00:30:02.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39228 fp=#0x10000003569' 2024-09-21T00:30:02.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39356 fp=#0x10000003f60' 2024-09-21T00:30:02.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39388 fp=#0x10000003f43' 2024-09-21T00:30:02.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39420 fp=#0x10000003f6e' 2024-09-21T00:30:02.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39452 fp=#0x30000001f2e' 2024-09-21T00:30:02.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39484 fp=#0x30000001fa1' 2024-09-21T00:30:02.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39516 fp=#0x30000001fba' 2024-09-21T00:30:02.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39548 fp=#0x30000001f7c' 2024-09-21T00:30:02.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39580 fp=#0x30000001f59' 2024-09-21T00:30:02.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39612 fp=#0x10000002b0f' 2024-09-21T00:30:02.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39676 fp=#0x10000002a93' 2024-09-21T00:30:02.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39708 fp=#0x30000001990' 2024-09-21T00:30:02.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39740 fp=#0x30000001987' 2024-09-21T00:30:02.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39772 fp=#0x30000000eb6' 2024-09-21T00:30:02.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39804 fp=#0x30000000eb8' 2024-09-21T00:30:02.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39836 fp=#0x30000000eaf' 2024-09-21T00:30:02.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40028 fp=#0x3000000183d' 2024-09-21T00:30:02.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40060 fp=#0x30000001678' 2024-09-21T00:30:02.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40092 fp=#0x30000001848' 2024-09-21T00:30:02.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40124 fp=#0x30000001521' 2024-09-21T00:30:02.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40156 fp=#0x30000001515' 2024-09-21T00:30:02.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40188 fp=#0x30000000db6' 2024-09-21T00:30:02.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40220 fp=#0x30000000dc3' 2024-09-21T00:30:02.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40252 fp=#0x10000001d7a' 2024-09-21T00:30:02.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40284 fp=#0x30000000d6a' 2024-09-21T00:30:02.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40316 fp=#0x30000000db2' 2024-09-21T00:30:02.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40348 fp=#0x30000002327' 2024-09-21T00:30:02.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40380 fp=#0x10000004e4d' 2024-09-21T00:30:02.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40444 fp=#0x10000004c69' 2024-09-21T00:30:02.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40476 fp=#0x300000022c9' 2024-09-21T00:30:02.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40508 fp=#0x10000004646' 2024-09-21T00:30:02.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40540 fp=#0x10000000d1a' 2024-09-21T00:30:02.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40572 fp=#0x20000000459' 2024-09-21T00:30:02.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40604 fp=#0x200000005a7' 2024-09-21T00:30:02.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40636 fp=#0x10000000cf0' 2024-09-21T00:30:02.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40700 fp=#0x10000000d00' 2024-09-21T00:30:02.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40732 fp=#0x2000000043d' 2024-09-21T00:30:02.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40764 fp=#0x2000000057d' 2024-09-21T00:30:02.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40796 fp=#0x30000001418' 2024-09-21T00:30:02.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40828 fp=#0x10000004c92' 2024-09-21T00:30:02.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40892 fp=#0x10000004aa6' 2024-09-21T00:30:02.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40924 fp=#0x10000004ad7' 2024-09-21T00:30:02.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40956 fp=#0x10000004a9a' 2024-09-21T00:30:02.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40988 fp=#0x10000004cd7' 2024-09-21T00:30:02.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41020 fp=#0x100000034f6' 2024-09-21T00:30:02.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41052 fp=#0x10000003576' 2024-09-21T00:30:02.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41084 fp=#0x100000036d2' 2024-09-21T00:30:02.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41116 fp=#0x200000006b5' 2024-09-21T00:30:02.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41148 fp=#0x10000000e30' 2024-09-21T00:30:02.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41180 fp=#0x20000000bab' 2024-09-21T00:30:02.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41212 fp=#0x10000000d34' 2024-09-21T00:30:02.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41244 fp=#0x20000000c48' 2024-09-21T00:30:02.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41276 fp=#0x10000000d79' 2024-09-21T00:30:02.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41308 fp=#0x20000000bf3' 2024-09-21T00:30:02.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41436 fp=#0x10000000abf' 2024-09-21T00:30:02.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41468 fp=#0x10000000de3' 2024-09-21T00:30:02.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41500 fp=#0x10000000c86' 2024-09-21T00:30:02.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41532 fp=#0x10000000b7f' 2024-09-21T00:30:02.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41692 fp=#0x30000000936' 2024-09-21T00:30:02.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41724 fp=#0x30000000928' 2024-09-21T00:30:02.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41756 fp=#0x30000000909' 2024-09-21T00:30:02.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41820 fp=#0x3000000081e' 2024-09-21T00:30:02.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41852 fp=#0x30000000852' 2024-09-21T00:30:02.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41884 fp=#0x30000000888' 2024-09-21T00:30:02.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41916 fp=#0x300000009d1' 2024-09-21T00:30:02.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41948 fp=#0x30000000987' 2024-09-21T00:30:02.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41980 fp=#0x30000000948' 2024-09-21T00:30:02.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42012 fp=#0x30000000934' 2024-09-21T00:30:02.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42044 fp=#0x30000000968' 2024-09-21T00:30:02.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42076 fp=#0x30000000838' 2024-09-21T00:30:02.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42108 fp=#0x30000001d58' 2024-09-21T00:30:02.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42140 fp=#0x30000000986' 2024-09-21T00:30:02.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42172 fp=#0x30000002097' 2024-09-21T00:30:02.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42204 fp=#0x30000000bcc' 2024-09-21T00:30:02.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42236 fp=#0x30000000b61' 2024-09-21T00:30:02.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42268 fp=#0x30000001b91' 2024-09-21T00:30:02.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42300 fp=#0x30000000b87' 2024-09-21T00:30:02.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42332 fp=#0x30000000afa' 2024-09-21T00:30:02.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42364 fp=#0x30000000aee' 2024-09-21T00:30:02.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42396 fp=#0x30000000b00' 2024-09-21T00:30:02.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42428 fp=#0x30000000b2d' 2024-09-21T00:30:02.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42460 fp=#0x30000000a43' 2024-09-21T00:30:02.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42492 fp=#0x30000000a40' 2024-09-21T00:30:02.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42524 fp=#0x10000004136' 2024-09-21T00:30:02.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42652 fp=#0x30000001261' 2024-09-21T00:30:02.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42684 fp=#0x1000000191d' 2024-09-21T00:30:02.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42716 fp=#0x100000019e4' 2024-09-21T00:30:02.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42780 fp=#0x100000019a3' 2024-09-21T00:30:02.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42844 fp=#0x10000001a1b' 2024-09-21T00:30:02.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42876 fp=#0x10000001a0a' 2024-09-21T00:30:02.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42908 fp=#0x30000000dec' 2024-09-21T00:30:02.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42940 fp=#0x30000000e4e' 2024-09-21T00:30:02.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43036 fp=#0x30000001764' 2024-09-21T00:30:02.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43068 fp=#0x3000000170b' 2024-09-21T00:30:02.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43100 fp=#0x30000001712' 2024-09-21T00:30:02.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43132 fp=#0x30000001732' 2024-09-21T00:30:02.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43228 fp=#0x300000020ec' 2024-09-21T00:30:02.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43292 fp=#0x30000000f85' 2024-09-21T00:30:02.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43324 fp=#0x30000000f6b' 2024-09-21T00:30:02.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43388 fp=#0x30000000f07' 2024-09-21T00:30:02.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43420 fp=#0x10000004160' 2024-09-21T00:30:02.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43452 fp=#0x300000023c0' 2024-09-21T00:30:02.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43484 fp=#0x300000023e3' 2024-09-21T00:30:02.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43516 fp=#0x300000023be' 2024-09-21T00:30:02.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43548 fp=#0x30000002391' 2024-09-21T00:30:02.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43580 fp=#0x300000021bf' 2024-09-21T00:30:02.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43612 fp=#0x30000000fdb' 2024-09-21T00:30:02.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43644 fp=#0x300000020f9' 2024-09-21T00:30:02.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43676 fp=#0x3000000211a' 2024-09-21T00:30:02.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43708 fp=#0x30000002105' 2024-09-21T00:30:02.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43740 fp=#0x30000000fef' 2024-09-21T00:30:02.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43772 fp=#0x30000002114' 2024-09-21T00:30:02.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43804 fp=#0x100000039f9' 2024-09-21T00:30:02.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43836 fp=#0x10000003a04' 2024-09-21T00:30:02.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43868 fp=#0x10000003a3c' 2024-09-21T00:30:02.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43900 fp=#0x10000003a08' 2024-09-21T00:30:02.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43932 fp=#0x300000013c4' 2024-09-21T00:30:02.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31742 fp=#0x10000000208' 2024-09-21T00:30:02.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31805 fp=#0x10000002832' 2024-09-21T00:30:02.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31933 fp=#0x30000000b54' 2024-09-21T00:30:02.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34685 fp=#0x30000002259' 2024-09-21T00:30:02.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34717 fp=#0x3000000033f' 2024-09-21T00:30:02.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34845 fp=#0x100000027e9' 2024-09-21T00:30:02.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34973 fp=#0x100000032b8' 2024-09-21T00:30:02.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35005 fp=#0x10000003831' 2024-09-21T00:30:02.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35037 fp=#0x1000000385a' 2024-09-21T00:30:02.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35069 fp=#0x300000020a8' 2024-09-21T00:30:02.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35101 fp=#0x100000039f7' 2024-09-21T00:30:02.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35133 fp=#0x100000041cb' 2024-09-21T00:30:02.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35165 fp=#0x10000003980' 2024-09-21T00:30:02.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35197 fp=#0x10000003ab1' 2024-09-21T00:30:02.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35229 fp=#0x10000003a15' 2024-09-21T00:30:02.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35261 fp=#0x100000038fe' 2024-09-21T00:30:02.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35293 fp=#0x1000000424c' 2024-09-21T00:30:02.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35325 fp=#0x1000000396a' 2024-09-21T00:30:02.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35357 fp=#0x10000003899' 2024-09-21T00:30:02.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35389 fp=#0x1000000399f' 2024-09-21T00:30:02.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35421 fp=#0x100000038aa' 2024-09-21T00:30:02.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35453 fp=#0x1000000397b' 2024-09-21T00:30:02.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35485 fp=#0x30000001178' 2024-09-21T00:30:02.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35517 fp=#0x30000001162' 2024-09-21T00:30:02.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35549 fp=#0x30000001154' 2024-09-21T00:30:02.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35581 fp=#0x30000001fdf' 2024-09-21T00:30:02.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35613 fp=#0x30000001fea' 2024-09-21T00:30:02.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35645 fp=#0x30000002034' 2024-09-21T00:30:02.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35677 fp=#0x30000001ff8' 2024-09-21T00:30:02.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35805 fp=#0x10000003472' 2024-09-21T00:30:02.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35837 fp=#0x100000033ef' 2024-09-21T00:30:02.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35869 fp=#0x100000033d9' 2024-09-21T00:30:02.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35933 fp=#0x300000011fe' 2024-09-21T00:30:02.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35965 fp=#0x300000011ec' 2024-09-21T00:30:02.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36093 fp=#0x10000002ca8' 2024-09-21T00:30:02.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36157 fp=#0x1000000209b' 2024-09-21T00:30:02.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36189 fp=#0x10000002047' 2024-09-21T00:30:02.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36221 fp=#0x10000001f36' 2024-09-21T00:30:02.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36253 fp=#0x10000002089' 2024-09-21T00:30:02.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36317 fp=#0x300000015ba' 2024-09-21T00:30:02.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36381 fp=#0x300000015a2' 2024-09-21T00:30:02.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36445 fp=#0x10000001db0' 2024-09-21T00:30:02.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36477 fp=#0x10000001dec' 2024-09-21T00:30:02.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36509 fp=#0x300000020f0' 2024-09-21T00:30:02.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36541 fp=#0x10000001e63' 2024-09-21T00:30:02.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36573 fp=#0x10000003760' 2024-09-21T00:30:02.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36605 fp=#0x10000003824' 2024-09-21T00:30:02.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36637 fp=#0x1000000378b' 2024-09-21T00:30:02.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36669 fp=#0x10000003804' 2024-09-21T00:30:02.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36701 fp=#0x10000003771' 2024-09-21T00:30:02.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36733 fp=#0x30000001a9d' 2024-09-21T00:30:02.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36765 fp=#0x10000004cf8' 2024-09-21T00:30:02.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36797 fp=#0x10000004704' 2024-09-21T00:30:02.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36829 fp=#0x100000047a1' 2024-09-21T00:30:02.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36861 fp=#0x10000002571' 2024-09-21T00:30:02.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36925 fp=#0x100000025f8' 2024-09-21T00:30:02.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36957 fp=#0x1000000369f' 2024-09-21T00:30:02.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36989 fp=#0x100000025ef' 2024-09-21T00:30:02.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37021 fp=#0x10000003665' 2024-09-21T00:30:02.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37085 fp=#0x30000001788' 2024-09-21T00:30:02.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37181 fp=#0x1000000314e' 2024-09-21T00:30:02.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37213 fp=#0x10000003178' 2024-09-21T00:30:02.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37245 fp=#0x10000001479' 2024-09-21T00:30:02.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37309 fp=#0x100000030ff' 2024-09-21T00:30:02.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37341 fp=#0x30000001afe' 2024-09-21T00:30:02.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37373 fp=#0x30000001415' 2024-09-21T00:30:02.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37405 fp=#0x30000000c00' 2024-09-21T00:30:02.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37437 fp=#0x30000001ad2' 2024-09-21T00:30:02.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37469 fp=#0x30000001ae6' 2024-09-21T00:30:02.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37501 fp=#0x300000010fd' 2024-09-21T00:30:02.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37533 fp=#0x3000000100b' 2024-09-21T00:30:02.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37597 fp=#0x30000001b69' 2024-09-21T00:30:02.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37629 fp=#0x10000002945' 2024-09-21T00:30:02.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37661 fp=#0x30000000c72' 2024-09-21T00:30:02.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37693 fp=#0x10000002833' 2024-09-21T00:30:02.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37725 fp=#0x30000000c9a' 2024-09-21T00:30:02.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37789 fp=#0x30000000c4c' 2024-09-21T00:30:02.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37821 fp=#0x30000000c8c' 2024-09-21T00:30:02.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37917 fp=#0x30000002284' 2024-09-21T00:30:02.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37981 fp=#0x30000000cf7' 2024-09-21T00:30:02.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38045 fp=#0x30000000cba' 2024-09-21T00:30:02.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38077 fp=#0x10000000738' 2024-09-21T00:30:02.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38109 fp=#0x1000000075e' 2024-09-21T00:30:02.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38141 fp=#0x10000000e3c' 2024-09-21T00:30:02.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38173 fp=#0x10000000777' 2024-09-21T00:30:02.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38205 fp=#0x1000000076e' 2024-09-21T00:30:02.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38237 fp=#0x10000001c29' 2024-09-21T00:30:02.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38301 fp=#0x30000001a5b' 2024-09-21T00:30:02.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38333 fp=#0x30000001a73' 2024-09-21T00:30:02.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38365 fp=#0x30000001a45' 2024-09-21T00:30:02.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38397 fp=#0x10000003331' 2024-09-21T00:30:02.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38461 fp=#0x10000003318' 2024-09-21T00:30:02.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38493 fp=#0x30000001ce3' 2024-09-21T00:30:02.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38525 fp=#0x10000002143' 2024-09-21T00:30:02.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38557 fp=#0x10000002113' 2024-09-21T00:30:02.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38589 fp=#0x100000012c0' 2024-09-21T00:30:02.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38653 fp=#0x30000001c19' 2024-09-21T00:30:02.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38685 fp=#0x30000001c35' 2024-09-21T00:30:02.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38781 fp=#0x30000001c33' 2024-09-21T00:30:02.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38813 fp=#0x10000002950' 2024-09-21T00:30:02.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38845 fp=#0x300000019c7' 2024-09-21T00:30:02.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38909 fp=#0x10000002960' 2024-09-21T00:30:02.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38941 fp=#0x30000001d7e' 2024-09-21T00:30:02.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38973 fp=#0x30000001c71' 2024-09-21T00:30:02.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39037 fp=#0x30000001c4a' 2024-09-21T00:30:02.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39101 fp=#0x30000001c5d' 2024-09-21T00:30:02.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39325 fp=#0x10000003595' 2024-09-21T00:30:02.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39357 fp=#0x10000003fd5' 2024-09-21T00:30:02.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39389 fp=#0x10000003f90' 2024-09-21T00:30:02.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39421 fp=#0x10000003ef5' 2024-09-21T00:30:02.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39453 fp=#0x30000001fad' 2024-09-21T00:30:02.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39485 fp=#0x30000001f3c' 2024-09-21T00:30:02.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39517 fp=#0x30000001f69' 2024-09-21T00:30:02.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39549 fp=#0x30000001f12' 2024-09-21T00:30:02.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39581 fp=#0x30000001f00' 2024-09-21T00:30:02.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39613 fp=#0x10000002abf' 2024-09-21T00:30:02.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39645 fp=#0x10000002afe' 2024-09-21T00:30:02.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39709 fp=#0x30000000eed' 2024-09-21T00:30:02.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39773 fp=#0x300000019ae' 2024-09-21T00:30:02.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39837 fp=#0x30000000ebc' 2024-09-21T00:30:02.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39901 fp=#0x30000000ff3' 2024-09-21T00:30:02.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39965 fp=#0x30000001207' 2024-09-21T00:30:02.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39997 fp=#0x3000000102b' 2024-09-21T00:30:02.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40029 fp=#0x3000000182c' 2024-09-21T00:30:02.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40061 fp=#0x30000001807' 2024-09-21T00:30:02.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40093 fp=#0x300000014c5' 2024-09-21T00:30:02.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40125 fp=#0x30000001813' 2024-09-21T00:30:02.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40157 fp=#0x30000001679' 2024-09-21T00:30:02.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40189 fp=#0x10000001d14' 2024-09-21T00:30:02.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40253 fp=#0x30000000dbb' 2024-09-21T00:30:02.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40317 fp=#0x30000000da6' 2024-09-21T00:30:02.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40349 fp=#0x10000004e6e' 2024-09-21T00:30:02.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40413 fp=#0x30000002326' 2024-09-21T00:30:02.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40445 fp=#0x10000004e67' 2024-09-21T00:30:02.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40477 fp=#0x10000004c82' 2024-09-21T00:30:02.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40541 fp=#0x10000000fd2' 2024-09-21T00:30:02.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40573 fp=#0x10000000f23' 2024-09-21T00:30:02.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40605 fp=#0x10000000d3b' 2024-09-21T00:30:02.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40637 fp=#0x200000005ef' 2024-09-21T00:30:02.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40733 fp=#0x10000000f1f' 2024-09-21T00:30:02.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40765 fp=#0x30000001453' 2024-09-21T00:30:02.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40797 fp=#0x30000001426' 2024-09-21T00:30:02.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40925 fp=#0x10000004d07' 2024-09-21T00:30:02.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40989 fp=#0x10000004cb6' 2024-09-21T00:30:02.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41021 fp=#0x20000000704' 2024-09-21T00:30:02.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41053 fp=#0x20000000718' 2024-09-21T00:30:02.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41117 fp=#0x10000000d54' 2024-09-21T00:30:02.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41149 fp=#0x10000000fdb' 2024-09-21T00:30:02.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41181 fp=#0x10000000e20' 2024-09-21T00:30:02.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41213 fp=#0x10000000e5c' 2024-09-21T00:30:02.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41245 fp=#0x2000000046f' 2024-09-21T00:30:02.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41277 fp=#0x20000000bdb' 2024-09-21T00:30:02.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41309 fp=#0x10000000cc4' 2024-09-21T00:30:02.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41373 fp=#0x10000000956' 2024-09-21T00:30:02.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41405 fp=#0x100000009fc' 2024-09-21T00:30:02.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41437 fp=#0x10000000c0c' 2024-09-21T00:30:02.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41469 fp=#0x10000000a14' 2024-09-21T00:30:02.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41501 fp=#0x10000000a84' 2024-09-21T00:30:02.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41533 fp=#0x10000000bd7' 2024-09-21T00:30:02.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41565 fp=#0x30000001917' 2024-09-21T00:30:02.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41661 fp=#0x30000001849' 2024-09-21T00:30:02.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41693 fp=#0x300000007de' 2024-09-21T00:30:02.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41789 fp=#0x30000000834' 2024-09-21T00:30:02.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41821 fp=#0x30000000828' 2024-09-21T00:30:02.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41853 fp=#0x300000008ce' 2024-09-21T00:30:02.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41917 fp=#0x300000009a1' 2024-09-21T00:30:02.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41949 fp=#0x30000000946' 2024-09-21T00:30:02.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41981 fp=#0x300000009b4' 2024-09-21T00:30:02.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42013 fp=#0x30000000980' 2024-09-21T00:30:02.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42045 fp=#0x30000000953' 2024-09-21T00:30:02.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42077 fp=#0x30000001d43' 2024-09-21T00:30:02.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42141 fp=#0x30000000962' 2024-09-21T00:30:02.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42173 fp=#0x3000000208c' 2024-09-21T00:30:02.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42205 fp=#0x30000000ba4' 2024-09-21T00:30:02.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42237 fp=#0x30000002099' 2024-09-21T00:30:02.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42269 fp=#0x30000000b63' 2024-09-21T00:30:02.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42301 fp=#0x30000002095' 2024-09-21T00:30:02.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42333 fp=#0x30000000b46' 2024-09-21T00:30:02.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42365 fp=#0x30000000af9' 2024-09-21T00:30:02.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42397 fp=#0x30000000b26' 2024-09-21T00:30:02.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42429 fp=#0x30000000ae9' 2024-09-21T00:30:02.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42461 fp=#0x10000004139' 2024-09-21T00:30:02.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42493 fp=#0x30000000a66' 2024-09-21T00:30:02.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42525 fp=#0x100000040f8' 2024-09-21T00:30:02.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42557 fp=#0x100000040ee' 2024-09-21T00:30:02.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42653 fp=#0x30000001263' 2024-09-21T00:30:02.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42685 fp=#0x10000001a43' 2024-09-21T00:30:02.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42717 fp=#0x100000019a6' 2024-09-21T00:30:02.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42813 fp=#0x10000001a15' 2024-09-21T00:30:02.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42909 fp=#0x30000000e52' 2024-09-21T00:30:02.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42941 fp=#0x30000000e3c' 2024-09-21T00:30:02.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43037 fp=#0x300000016c6' 2024-09-21T00:30:02.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43101 fp=#0x300000016e3' 2024-09-21T00:30:02.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43133 fp=#0x3000000170f' 2024-09-21T00:30:02.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43229 fp=#0x100000018a9' 2024-09-21T00:30:02.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43293 fp=#0x30000000f06' 2024-09-21T00:30:02.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43325 fp=#0x30000000f0f' 2024-09-21T00:30:02.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43389 fp=#0x30000000efb' 2024-09-21T00:30:02.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43421 fp=#0x1000000415b' 2024-09-21T00:30:02.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43453 fp=#0x300000023f6' 2024-09-21T00:30:02.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43485 fp=#0x30000002399' 2024-09-21T00:30:02.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43517 fp=#0x300000023d5' 2024-09-21T00:30:02.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43549 fp=#0x300000023df' 2024-09-21T00:30:02.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43581 fp=#0x300000010db' 2024-09-21T00:30:02.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43613 fp=#0x30000002123' 2024-09-21T00:30:02.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43645 fp=#0x30000002121' 2024-09-21T00:30:02.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43677 fp=#0x3000000218c' 2024-09-21T00:30:02.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43741 fp=#0x3000000211d' 2024-09-21T00:30:02.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43773 fp=#0x3000000211b' 2024-09-21T00:30:02.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43837 fp=#0x10000003a52' 2024-09-21T00:30:02.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43901 fp=#0x100000039f6' 2024-09-21T00:30:02.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43933 fp=#0x300000013c2' 2024-09-21T00:30:02.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31743 fp=#0x10000000399' 2024-09-21T00:30:02.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31806 fp=#0x10000000ca7' 2024-09-21T00:30:02.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31934 fp=#0x30000000ad8' 2024-09-21T00:30:02.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34686 fp=#0x3000000229a' 2024-09-21T00:30:02.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34718 fp=#0x10000000b5e' 2024-09-21T00:30:02.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34750 fp=#0x30000000c35' 2024-09-21T00:30:02.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34782 fp=#0x30000002247' 2024-09-21T00:30:02.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34878 fp=#0x1000000280c' 2024-09-21T00:30:02.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34974 fp=#0x1000000329b' 2024-09-21T00:30:02.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35006 fp=#0x300000020bc' 2024-09-21T00:30:02.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35038 fp=#0x100000032b4' 2024-09-21T00:30:02.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35070 fp=#0x3000000209d' 2024-09-21T00:30:02.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35102 fp=#0x10000003902' 2024-09-21T00:30:02.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35134 fp=#0x100000041f3' 2024-09-21T00:30:02.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35166 fp=#0x1000000413f' 2024-09-21T00:30:02.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35198 fp=#0x10000004152' 2024-09-21T00:30:02.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35230 fp=#0x100000039d8' 2024-09-21T00:30:02.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35262 fp=#0x10000003af3' 2024-09-21T00:30:02.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35294 fp=#0x10000003967' 2024-09-21T00:30:02.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35326 fp=#0x10000003974' 2024-09-21T00:30:02.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35358 fp=#0x100000038fc' 2024-09-21T00:30:02.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35390 fp=#0x100000038e7' 2024-09-21T00:30:02.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35422 fp=#0x100000039b9' 2024-09-21T00:30:02.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35454 fp=#0x3000000118b' 2024-09-21T00:30:02.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35486 fp=#0x3000000114d' 2024-09-21T00:30:02.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35518 fp=#0x30000001166' 2024-09-21T00:30:02.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35550 fp=#0x3000000116d' 2024-09-21T00:30:02.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35582 fp=#0x30000002057' 2024-09-21T00:30:02.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35646 fp=#0x30000002021' 2024-09-21T00:30:02.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35678 fp=#0x30000002049' 2024-09-21T00:30:02.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35806 fp=#0x10000003475' 2024-09-21T00:30:02.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35838 fp=#0x100000033d7' 2024-09-21T00:30:02.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35870 fp=#0x10000003386' 2024-09-21T00:30:02.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35902 fp=#0x100000033c2' 2024-09-21T00:30:02.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35934 fp=#0x300000011e5' 2024-09-21T00:30:02.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35966 fp=#0x300000011df' 2024-09-21T00:30:02.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36094 fp=#0x10000002cbe' 2024-09-21T00:30:02.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36126 fp=#0x10000003e5c' 2024-09-21T00:30:02.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36158 fp=#0x100000020b5' 2024-09-21T00:30:02.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36190 fp=#0x10000001ece' 2024-09-21T00:30:02.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36222 fp=#0x1000000208c' 2024-09-21T00:30:02.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36254 fp=#0x3000000110b' 2024-09-21T00:30:02.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36286 fp=#0x300000015a3' 2024-09-21T00:30:02.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36318 fp=#0x300000015c0' 2024-09-21T00:30:02.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36350 fp=#0x300000015bb' 2024-09-21T00:30:02.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36446 fp=#0x10000001e79' 2024-09-21T00:30:02.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36478 fp=#0x10000001d6f' 2024-09-21T00:30:02.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36510 fp=#0x10000001e31' 2024-09-21T00:30:02.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36542 fp=#0x3000000231a' 2024-09-21T00:30:02.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36574 fp=#0x10000003754' 2024-09-21T00:30:02.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36606 fp=#0x1000000377f' 2024-09-21T00:30:02.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36638 fp=#0x100000044a6' 2024-09-21T00:30:02.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36670 fp=#0x100000037cf' 2024-09-21T00:30:02.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36702 fp=#0x100000037c7' 2024-09-21T00:30:02.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36734 fp=#0x10000004cf5' 2024-09-21T00:30:02.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36766 fp=#0x100000047c9' 2024-09-21T00:30:02.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36798 fp=#0x10000004788' 2024-09-21T00:30:02.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36830 fp=#0x30000001a91' 2024-09-21T00:30:02.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36862 fp=#0x100000035f6' 2024-09-21T00:30:02.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36926 fp=#0x10000002602' 2024-09-21T00:30:02.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36958 fp=#0x1000000350c' 2024-09-21T00:30:02.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36990 fp=#0x10000002668' 2024-09-21T00:30:02.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37022 fp=#0x1000000362e' 2024-09-21T00:30:02.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37054 fp=#0x3000000180d' 2024-09-21T00:30:02.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37086 fp=#0x300000020c3' 2024-09-21T00:30:02.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37150 fp=#0x300000020c4' 2024-09-21T00:30:02.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37214 fp=#0x1000000302f' 2024-09-21T00:30:02.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37246 fp=#0x10000001358' 2024-09-21T00:30:02.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37310 fp=#0x1000000303f' 2024-09-21T00:30:02.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37342 fp=#0x30000001b0c' 2024-09-21T00:30:02.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37374 fp=#0x30000001b15' 2024-09-21T00:30:02.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37406 fp=#0x30000001afc' 2024-09-21T00:30:02.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37438 fp=#0x30000001abd' 2024-09-21T00:30:02.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37470 fp=#0x30000001ab4' 2024-09-21T00:30:02.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37502 fp=#0x3000000113a' 2024-09-21T00:30:02.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37534 fp=#0x30000001086' 2024-09-21T00:30:02.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37566 fp=#0x30000000fe9' 2024-09-21T00:30:02.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37662 fp=#0x30000000c4f' 2024-09-21T00:30:02.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37694 fp=#0x30000000c5a' 2024-09-21T00:30:02.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37790 fp=#0x30000000ca9' 2024-09-21T00:30:02.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37854 fp=#0x30000001278' 2024-09-21T00:30:02.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37886 fp=#0x30000001274' 2024-09-21T00:30:02.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37982 fp=#0x30000000da7' 2024-09-21T00:30:02.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38014 fp=#0x30000000dd2' 2024-09-21T00:30:02.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38142 fp=#0x10000000e12' 2024-09-21T00:30:02.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38174 fp=#0x10000000e7c' 2024-09-21T00:30:02.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38206 fp=#0x1000000074e' 2024-09-21T00:30:02.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38238 fp=#0x10000000725' 2024-09-21T00:30:02.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38334 fp=#0x30000001a7d' 2024-09-21T00:30:02.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38366 fp=#0x30000001a4e' 2024-09-21T00:30:02.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38398 fp=#0x100000032f1' 2024-09-21T00:30:02.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38430 fp=#0x30000001d02' 2024-09-21T00:30:02.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38462 fp=#0x1000000330e' 2024-09-21T00:30:02.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38494 fp=#0x10000003301' 2024-09-21T00:30:02.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38526 fp=#0x1000000218b' 2024-09-21T00:30:02.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38558 fp=#0x10000002102' 2024-09-21T00:30:02.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38590 fp=#0x30000001c20' 2024-09-21T00:30:02.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38654 fp=#0x100000005a2' 2024-09-21T00:30:02.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38750 fp=#0x30000001bbe' 2024-09-21T00:30:02.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38782 fp=#0x100000005e0' 2024-09-21T00:30:02.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38814 fp=#0x10000002976' 2024-09-21T00:30:02.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38910 fp=#0x300000019f9' 2024-09-21T00:30:02.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38974 fp=#0x30000001c25' 2024-09-21T00:30:02.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39070 fp=#0x30000001d2a' 2024-09-21T00:30:02.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39102 fp=#0x30000001c66' 2024-09-21T00:30:02.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39166 fp=#0x1000000358e' 2024-09-21T00:30:02.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39198 fp=#0x100000034e0' 2024-09-21T00:30:02.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39262 fp=#0x10000001121' 2024-09-21T00:30:02.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39326 fp=#0x10000003556' 2024-09-21T00:30:02.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39358 fp=#0x30000000827' 2024-09-21T00:30:02.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39390 fp=#0x10000003f46' 2024-09-21T00:30:02.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39422 fp=#0x10000003fad' 2024-09-21T00:30:02.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39454 fp=#0x30000001f05' 2024-09-21T00:30:02.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39486 fp=#0x30000001f99' 2024-09-21T00:30:02.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39518 fp=#0x30000001fb1' 2024-09-21T00:30:02.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39550 fp=#0x30000001f44' 2024-09-21T00:30:02.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39582 fp=#0x30000001fab' 2024-09-21T00:30:02.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39614 fp=#0x10000002afa' 2024-09-21T00:30:02.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39646 fp=#0x10000002aa6' 2024-09-21T00:30:02.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39678 fp=#0x30000000e73' 2024-09-21T00:30:02.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39710 fp=#0x30000000ef8' 2024-09-21T00:30:02.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39742 fp=#0x30000000e6a' 2024-09-21T00:30:02.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39774 fp=#0x30000000e5f' 2024-09-21T00:30:02.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39806 fp=#0x30000000e98' 2024-09-21T00:30:02.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39870 fp=#0x30000000ea2' 2024-09-21T00:30:02.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39902 fp=#0x30000001110' 2024-09-21T00:30:02.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39934 fp=#0x30000000ff0' 2024-09-21T00:30:02.914 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40030 fp=#0x30000001802' 2024-09-21T00:30:02.914 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40062 fp=#0x30000001511' 2024-09-21T00:30:02.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40094 fp=#0x300000014ce' 2024-09-21T00:30:02.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40158 fp=#0x30000001831' 2024-09-21T00:30:02.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40190 fp=#0x10000001d7d' 2024-09-21T00:30:02.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40222 fp=#0x10000001d50' 2024-09-21T00:30:02.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40254 fp=#0x30000000d59' 2024-09-21T00:30:02.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40286 fp=#0x30000001d1d' 2024-09-21T00:30:02.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40318 fp=#0x300000009ea' 2024-09-21T00:30:02.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40350 fp=#0x10000004c61' 2024-09-21T00:30:02.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40382 fp=#0x10000004d1a' 2024-09-21T00:30:02.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40414 fp=#0x200000012ae' 2024-09-21T00:30:02.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40446 fp=#0x10000004d04' 2024-09-21T00:30:02.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40478 fp=#0x10000004e5f' 2024-09-21T00:30:02.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40574 fp=#0x10000000fbd' 2024-09-21T00:30:02.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40606 fp=#0x20000000551' 2024-09-21T00:30:02.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40670 fp=#0x200000005f2' 2024-09-21T00:30:02.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40766 fp=#0x3000000143f' 2024-09-21T00:30:02.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40798 fp=#0x30000001430' 2024-09-21T00:30:02.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40830 fp=#0x10000004a82' 2024-09-21T00:30:02.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40862 fp=#0x10000004954' 2024-09-21T00:30:02.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40990 fp=#0x10000004ca0' 2024-09-21T00:30:02.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41022 fp=#0x2000000070d' 2024-09-21T00:30:02.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41054 fp=#0x1000000360d' 2024-09-21T00:30:02.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41086 fp=#0x10000003568' 2024-09-21T00:30:02.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41118 fp=#0x10000000ce1' 2024-09-21T00:30:02.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41150 fp=#0x10000000f5d' 2024-09-21T00:30:02.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41182 fp=#0x10000000e34' 2024-09-21T00:30:02.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41214 fp=#0x10000000ebe' 2024-09-21T00:30:02.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41246 fp=#0x2000000085f' 2024-09-21T00:30:02.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41278 fp=#0x10000000cd2' 2024-09-21T00:30:02.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41310 fp=#0x10000000d7e' 2024-09-21T00:30:02.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41374 fp=#0x10000000947' 2024-09-21T00:30:02.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41406 fp=#0x10000000ba6' 2024-09-21T00:30:02.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41438 fp=#0x10000000ee3' 2024-09-21T00:30:02.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41470 fp=#0x1000000278b' 2024-09-21T00:30:02.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41502 fp=#0x10000000b71' 2024-09-21T00:30:02.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41534 fp=#0x10000000953' 2024-09-21T00:30:02.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41598 fp=#0x30000001910' 2024-09-21T00:30:02.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41790 fp=#0x300000009d0' 2024-09-21T00:30:02.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41822 fp=#0x300000008a1' 2024-09-21T00:30:02.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41854 fp=#0x300000008d9' 2024-09-21T00:30:02.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41918 fp=#0x30000000859' 2024-09-21T00:30:02.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41982 fp=#0x30000000818' 2024-09-21T00:30:02.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42014 fp=#0x30000001d6f' 2024-09-21T00:30:02.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42046 fp=#0x300000007d2' 2024-09-21T00:30:02.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42110 fp=#0x30000000997' 2024-09-21T00:30:02.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42142 fp=#0x30000000938' 2024-09-21T00:30:02.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42174 fp=#0x30000001bc3' 2024-09-21T00:30:02.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42206 fp=#0x30000000b5d' 2024-09-21T00:30:02.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42238 fp=#0x30000001bb4' 2024-09-21T00:30:02.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42270 fp=#0x30000000bbc' 2024-09-21T00:30:02.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42302 fp=#0x3000000208b' 2024-09-21T00:30:02.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42334 fp=#0x30000000ae5' 2024-09-21T00:30:02.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42366 fp=#0x30000000af0' 2024-09-21T00:30:02.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42398 fp=#0x30000000b2a' 2024-09-21T00:30:02.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42430 fp=#0x30000000b39' 2024-09-21T00:30:02.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42462 fp=#0x10000004153' 2024-09-21T00:30:02.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42494 fp=#0x100000040f0' 2024-09-21T00:30:02.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42526 fp=#0x30000000a67' 2024-09-21T00:30:02.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42558 fp=#0x30000000a62' 2024-09-21T00:30:02.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42654 fp=#0x3000000125b' 2024-09-21T00:30:02.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42718 fp=#0x100000019ec' 2024-09-21T00:30:02.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42750 fp=#0x100000019c6' 2024-09-21T00:30:02.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42814 fp=#0x10000001952' 2024-09-21T00:30:02.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42878 fp=#0x10000001905' 2024-09-21T00:30:02.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42942 fp=#0x30000000df2' 2024-09-21T00:30:02.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43038 fp=#0x30000001738' 2024-09-21T00:30:02.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43070 fp=#0x30000001684' 2024-09-21T00:30:02.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43102 fp=#0x30000001758' 2024-09-21T00:30:02.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43134 fp=#0x100000018a7' 2024-09-21T00:30:02.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43166 fp=#0x1000000189f' 2024-09-21T00:30:02.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43262 fp=#0x10000001860' 2024-09-21T00:30:02.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43294 fp=#0x30000000f09' 2024-09-21T00:30:02.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43358 fp=#0x30000000f00' 2024-09-21T00:30:02.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43422 fp=#0x1000000418c' 2024-09-21T00:30:02.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43454 fp=#0x3000000239a' 2024-09-21T00:30:02.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43486 fp=#0x300000023a4' 2024-09-21T00:30:02.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43518 fp=#0x300000023a3' 2024-09-21T00:30:02.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43550 fp=#0x300000023aa' 2024-09-21T00:30:02.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43614 fp=#0x30000002147' 2024-09-21T00:30:02.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43646 fp=#0x30000000ff5' 2024-09-21T00:30:02.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43678 fp=#0x30000000fd2' 2024-09-21T00:30:02.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43742 fp=#0x30000002124' 2024-09-21T00:30:02.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43774 fp=#0x30000002102' 2024-09-21T00:30:02.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43806 fp=#0x100000039c2' 2024-09-21T00:30:02.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43838 fp=#0x100000039b7' 2024-09-21T00:30:02.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43934 fp=#0x300000013d9' 2024-09-21T00:30:02.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'client_request(client.25376:319375 rmdir #0x10000000399/blog-60 2024-09-21T00:28:46.574643+0000 caller_uid=0, caller_gid=0{0,})' 2024-09-21T00:30:02.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31807 fp=#0x30000001266' 2024-09-21T00:30:02.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31935 fp=#0x100000040ec' 2024-09-21T00:30:02.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34687 fp=#0x300000022b2' 2024-09-21T00:30:02.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34719 fp=#0x3000000223f' 2024-09-21T00:30:02.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34751 fp=#0x10000000c4b' 2024-09-21T00:30:02.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34847 fp=#0x10000002818' 2024-09-21T00:30:02.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34943 fp=#0x100000026eb' 2024-09-21T00:30:02.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34975 fp=#0x300000020b5' 2024-09-21T00:30:02.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35007 fp=#0x100000032ea' 2024-09-21T00:30:02.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35039 fp=#0x30000002066' 2024-09-21T00:30:02.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35071 fp=#0x10000003283' 2024-09-21T00:30:02.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35103 fp=#0x100000039a3' 2024-09-21T00:30:02.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35135 fp=#0x10000003ae6' 2024-09-21T00:30:02.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35167 fp=#0x1000000415f' 2024-09-21T00:30:02.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35199 fp=#0x10000003a74' 2024-09-21T00:30:02.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35231 fp=#0x1000000397a' 2024-09-21T00:30:02.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35263 fp=#0x100000039c3' 2024-09-21T00:30:02.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35295 fp=#0x10000003ae3' 2024-09-21T00:30:02.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35327 fp=#0x100000039bf' 2024-09-21T00:30:02.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35359 fp=#0x10000003a2d' 2024-09-21T00:30:02.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35391 fp=#0x10000003911' 2024-09-21T00:30:02.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35423 fp=#0x10000003937' 2024-09-21T00:30:02.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35455 fp=#0x30000001182' 2024-09-21T00:30:02.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35487 fp=#0x30000001170' 2024-09-21T00:30:02.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35519 fp=#0x3000000117b' 2024-09-21T00:30:02.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35583 fp=#0x30000001ff3' 2024-09-21T00:30:02.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35615 fp=#0x30000002058' 2024-09-21T00:30:02.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35647 fp=#0x30000001fdd' 2024-09-21T00:30:02.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35679 fp=#0x30000001fe1' 2024-09-21T00:30:02.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35807 fp=#0x100000033f1' 2024-09-21T00:30:02.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35839 fp=#0x1000000340b' 2024-09-21T00:30:02.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35871 fp=#0x100000033ec' 2024-09-21T00:30:02.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35903 fp=#0x100000033bc' 2024-09-21T00:30:02.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35935 fp=#0x300000011f0' 2024-09-21T00:30:02.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35967 fp=#0x300000011e2' 2024-09-21T00:30:02.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35999 fp=#0x10000003b52' 2024-09-21T00:30:02.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36063 fp=#0x10000002d5d' 2024-09-21T00:30:02.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36095 fp=#0x10000002cd3' 2024-09-21T00:30:02.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36127 fp=#0x10000002d47' 2024-09-21T00:30:02.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36159 fp=#0x10000001f7e' 2024-09-21T00:30:02.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36191 fp=#0x10000003820' 2024-09-21T00:30:02.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36223 fp=#0x10000001f7c' 2024-09-21T00:30:02.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36255 fp=#0x10000001f5d' 2024-09-21T00:30:02.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36447 fp=#0x10000001e77' 2024-09-21T00:30:02.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36479 fp=#0x10000001e01' 2024-09-21T00:30:02.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36511 fp=#0x10000001e39' 2024-09-21T00:30:02.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36543 fp=#0x10000001e3f' 2024-09-21T00:30:02.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36575 fp=#0x1000000382e' 2024-09-21T00:30:02.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36607 fp=#0x10000004411' 2024-09-21T00:30:02.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36639 fp=#0x1000000445f' 2024-09-21T00:30:02.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36671 fp=#0x1000000380f' 2024-09-21T00:30:02.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36703 fp=#0x100000037f4' 2024-09-21T00:30:02.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36735 fp=#0x10000004702' 2024-09-21T00:30:02.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36767 fp=#0x30000001aa3' 2024-09-21T00:30:02.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36799 fp=#0x1000000472c' 2024-09-21T00:30:02.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36831 fp=#0x1000000475d' 2024-09-21T00:30:02.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36895 fp=#0x100000035bf' 2024-09-21T00:30:02.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36927 fp=#0x1000000365f' 2024-09-21T00:30:02.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36959 fp=#0x1000000266e' 2024-09-21T00:30:02.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36991 fp=#0x100000035a5' 2024-09-21T00:30:02.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37023 fp=#0x10000003613' 2024-09-21T00:30:02.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37087 fp=#0x30000001790' 2024-09-21T00:30:02.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37119 fp=#0x3000000178a' 2024-09-21T00:30:02.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37215 fp=#0x10000003166' 2024-09-21T00:30:02.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37247 fp=#0x1000000311e' 2024-09-21T00:30:02.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37279 fp=#0x10000003135' 2024-09-21T00:30:02.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37343 fp=#0x30000001af2' 2024-09-21T00:30:02.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37375 fp=#0x30000000bea' 2024-09-21T00:30:02.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37407 fp=#0x30000001b08' 2024-09-21T00:30:02.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37439 fp=#0x30000001b3a' 2024-09-21T00:30:02.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37471 fp=#0x30000001af1' 2024-09-21T00:30:02.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37503 fp=#0x30000001b79' 2024-09-21T00:30:02.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37535 fp=#0x30000001090' 2024-09-21T00:30:03.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37599 fp=#0x30000000fe0' 2024-09-21T00:30:03.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37631 fp=#0x1000000294b' 2024-09-21T00:30:03.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37695 fp=#0x30000000c52' 2024-09-21T00:30:03.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37791 fp=#0x30000000c36' 2024-09-21T00:30:03.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37823 fp=#0x30000000c93' 2024-09-21T00:30:03.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37887 fp=#0x300000012cd' 2024-09-21T00:30:03.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37919 fp=#0x30000002291' 2024-09-21T00:30:03.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38079 fp=#0x10000000767' 2024-09-21T00:30:03.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38111 fp=#0x10000000ece' 2024-09-21T00:30:03.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38207 fp=#0x10000000e92' 2024-09-21T00:30:03.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38239 fp=#0x1000000075f' 2024-09-21T00:30:03.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38303 fp=#0x1000000245d' 2024-09-21T00:30:03.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38335 fp=#0x30000001a57' 2024-09-21T00:30:03.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38367 fp=#0x30000001a77' 2024-09-21T00:30:03.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38399 fp=#0x30000001ce5' 2024-09-21T00:30:03.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38431 fp=#0x100000032cd' 2024-09-21T00:30:03.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38495 fp=#0x1000000330b' 2024-09-21T00:30:03.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38527 fp=#0x10000002150' 2024-09-21T00:30:03.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38559 fp=#0x100000020e9' 2024-09-21T00:30:03.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38623 fp=#0x30000001c14' 2024-09-21T00:30:03.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38655 fp=#0x30000001bf9' 2024-09-21T00:30:03.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38687 fp=#0x30000001c43' 2024-09-21T00:30:03.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38719 fp=#0x30000001bfe' 2024-09-21T00:30:03.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38751 fp=#0x1000000059d' 2024-09-21T00:30:03.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38783 fp=#0x30000001c3d' 2024-09-21T00:30:03.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38815 fp=#0x300000019d1' 2024-09-21T00:30:03.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38879 fp=#0x10000002941' 2024-09-21T00:30:03.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38943 fp=#0x30000001d7b' 2024-09-21T00:30:03.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38975 fp=#0x30000001c7f' 2024-09-21T00:30:03.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39007 fp=#0x30000001c26' 2024-09-21T00:30:03.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39039 fp=#0x30000001c41' 2024-09-21T00:30:03.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39231 fp=#0x10000003551' 2024-09-21T00:30:03.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39263 fp=#0x10000003502' 2024-09-21T00:30:03.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39295 fp=#0x10000003514' 2024-09-21T00:30:03.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39359 fp=#0x10000003f26' 2024-09-21T00:30:03.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39391 fp=#0x10000003fdd' 2024-09-21T00:30:03.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39423 fp=#0x10000003fa4' 2024-09-21T00:30:03.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39455 fp=#0x30000001f19' 2024-09-21T00:30:03.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39487 fp=#0x30000001efe' 2024-09-21T00:30:03.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39519 fp=#0x30000001f54' 2024-09-21T00:30:03.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39551 fp=#0x30000001f29' 2024-09-21T00:30:03.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39583 fp=#0x30000001f4f' 2024-09-21T00:30:03.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39647 fp=#0x10000002a97' 2024-09-21T00:30:03.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39679 fp=#0x30000000eea' 2024-09-21T00:30:03.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39711 fp=#0x300000019a5' 2024-09-21T00:30:03.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39775 fp=#0x300000019a6' 2024-09-21T00:30:03.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39807 fp=#0x3000000198f' 2024-09-21T00:30:03.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39839 fp=#0x30000001996' 2024-09-21T00:30:03.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39935 fp=#0x30000001004' 2024-09-21T00:30:03.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40031 fp=#0x30000001542' 2024-09-21T00:30:03.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40095 fp=#0x30000001826' 2024-09-21T00:30:03.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40159 fp=#0x30000001812' 2024-09-21T00:30:03.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40191 fp=#0x10000001d84' 2024-09-21T00:30:03.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40223 fp=#0x30000000d93' 2024-09-21T00:30:03.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40287 fp=#0x300000009f0' 2024-09-21T00:30:03.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40319 fp=#0x10000001d2b' 2024-09-21T00:30:03.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40351 fp=#0x300000022c6' 2024-09-21T00:30:03.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40383 fp=#0x10000004e30' 2024-09-21T00:30:03.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40415 fp=#0x10000004c5d' 2024-09-21T00:30:03.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40447 fp=#0x10000004cef' 2024-09-21T00:30:03.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40479 fp=#0x10000004c7e' 2024-09-21T00:30:03.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40511 fp=#0x10000004647' 2024-09-21T00:30:03.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40543 fp=#0x20000000439' 2024-09-21T00:30:03.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40607 fp=#0x20000000543' 2024-09-21T00:30:03.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40671 fp=#0x20000000453' 2024-09-21T00:30:03.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40703 fp=#0x10000000cc0' 2024-09-21T00:30:03.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40735 fp=#0x200000005e3' 2024-09-21T00:30:03.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40767 fp=#0x3000000141d' 2024-09-21T00:30:03.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40799 fp=#0x3000000145b' 2024-09-21T00:30:03.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40895 fp=#0x10000004cbc' 2024-09-21T00:30:03.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40927 fp=#0x10000004a70' 2024-09-21T00:30:03.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41023 fp=#0x10000003691' 2024-09-21T00:30:03.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41055 fp=#0x1000000368e' 2024-09-21T00:30:03.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41087 fp=#0x100000034c6' 2024-09-21T00:30:03.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41119 fp=#0x10000000e90' 2024-09-21T00:30:03.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41151 fp=#0x10000000ea3' 2024-09-21T00:30:03.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41183 fp=#0x20000000c2e' 2024-09-21T00:30:03.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41215 fp=#0x20000000b4f' 2024-09-21T00:30:03.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41247 fp=#0x10000000df4' 2024-09-21T00:30:03.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41279 fp=#0x10000000e47' 2024-09-21T00:30:03.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41311 fp=#0x10000000fc2' 2024-09-21T00:30:03.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41375 fp=#0x10000000e56' 2024-09-21T00:30:03.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41407 fp=#0x10000000bbb' 2024-09-21T00:30:03.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41439 fp=#0x10000000e8c' 2024-09-21T00:30:03.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41471 fp=#0x10000000c65' 2024-09-21T00:30:03.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41503 fp=#0x100000009ec' 2024-09-21T00:30:03.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41535 fp=#0x10000000bc5' 2024-09-21T00:30:03.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41567 fp=#0x3000000191a' 2024-09-21T00:30:03.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41663 fp=#0x30000001861' 2024-09-21T00:30:03.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41695 fp=#0x30000000925' 2024-09-21T00:30:03.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41759 fp=#0x300000007da' 2024-09-21T00:30:03.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41791 fp=#0x300000009cb' 2024-09-21T00:30:03.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41823 fp=#0x300000009c0' 2024-09-21T00:30:03.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41855 fp=#0x30000000981' 2024-09-21T00:30:03.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41887 fp=#0x30000000897' 2024-09-21T00:30:03.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41919 fp=#0x300000008d6' 2024-09-21T00:30:03.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41983 fp=#0x300000009b1' 2024-09-21T00:30:03.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42015 fp=#0x30000001d6d' 2024-09-21T00:30:03.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42047 fp=#0x3000000093b' 2024-09-21T00:30:03.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42079 fp=#0x300000007ee' 2024-09-21T00:30:03.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42111 fp=#0x3000000080b' 2024-09-21T00:30:03.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42143 fp=#0x30000001da9' 2024-09-21T00:30:03.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42175 fp=#0x30000001b97' 2024-09-21T00:30:03.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42207 fp=#0x30000001b8b' 2024-09-21T00:30:03.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42239 fp=#0x30000001b8a' 2024-09-21T00:30:03.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42271 fp=#0x30000000b73' 2024-09-21T00:30:03.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42303 fp=#0x30000002086' 2024-09-21T00:30:03.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42335 fp=#0x30000000ae1' 2024-09-21T00:30:03.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42367 fp=#0x30000000b01' 2024-09-21T00:30:03.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42431 fp=#0x30000000ae7' 2024-09-21T00:30:03.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42463 fp=#0x30000000a7a' 2024-09-21T00:30:03.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42495 fp=#0x30000000a4a' 2024-09-21T00:30:03.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42527 fp=#0x30000000a6e' 2024-09-21T00:30:03.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42559 fp=#0x30000000a69' 2024-09-21T00:30:03.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42655 fp=#0x30000001320' 2024-09-21T00:30:03.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42751 fp=#0x10000001a05' 2024-09-21T00:30:03.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42783 fp=#0x100000019b5' 2024-09-21T00:30:03.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42815 fp=#0x100000019dd' 2024-09-21T00:30:03.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42879 fp=#0x1000000190e' 2024-09-21T00:30:03.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42911 fp=#0x30000000e54' 2024-09-21T00:30:03.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42943 fp=#0x30000000dfa' 2024-09-21T00:30:03.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42975 fp=#0x30000000e40' 2024-09-21T00:30:03.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43007 fp=#0x30000000e46' 2024-09-21T00:30:03.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43071 fp=#0x300000016bc' 2024-09-21T00:30:03.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43103 fp=#0x3000000171a' 2024-09-21T00:30:03.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43167 fp=#0x1000000188b' 2024-09-21T00:30:03.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43263 fp=#0x1000000186d' 2024-09-21T00:30:03.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43295 fp=#0x30000000f2f' 2024-09-21T00:30:03.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43359 fp=#0x30000000f74' 2024-09-21T00:30:03.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43391 fp=#0x30000000f6d' 2024-09-21T00:30:03.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43423 fp=#0x10000004164' 2024-09-21T00:30:03.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43455 fp=#0x300000023f4' 2024-09-21T00:30:03.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43487 fp=#0x3000000239b' 2024-09-21T00:30:03.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43519 fp=#0x30000002388' 2024-09-21T00:30:03.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43551 fp=#0x300000023d9' 2024-09-21T00:30:03.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43583 fp=#0x30000002128' 2024-09-21T00:30:03.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43615 fp=#0x300000016ef' 2024-09-21T00:30:03.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43647 fp=#0x30000002103' 2024-09-21T00:30:03.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43679 fp=#0x300000021a2' 2024-09-21T00:30:03.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43711 fp=#0x30000000f9c' 2024-09-21T00:30:03.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43743 fp=#0x3000000211e' 2024-09-21T00:30:03.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43775 fp=#0x30000000fdd' 2024-09-21T00:30:03.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43935 fp=#0x30000001349' 2024-09-21T00:30:03.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31744 fp=#0x100000026e1' 2024-09-21T00:30:03.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31808 fp=#0x3000000224a' 2024-09-21T00:30:03.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31936 fp=#0x30000000ca3' 2024-09-21T00:30:03.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34752 fp=#0x30000002228' 2024-09-21T00:30:03.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34976 fp=#0x300000020a9' 2024-09-21T00:30:03.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35008 fp=#0x100000032fe' 2024-09-21T00:30:03.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35040 fp=#0x10000003843' 2024-09-21T00:30:03.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35072 fp=#0x1000000327b' 2024-09-21T00:30:03.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35104 fp=#0x10000003b16' 2024-09-21T00:30:03.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35136 fp=#0x100000038c7' 2024-09-21T00:30:03.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35168 fp=#0x100000039a1' 2024-09-21T00:30:03.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35200 fp=#0x10000003931' 2024-09-21T00:30:03.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35232 fp=#0x100000041c7' 2024-09-21T00:30:03.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35264 fp=#0x10000004175' 2024-09-21T00:30:03.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35296 fp=#0x100000038d3' 2024-09-21T00:30:03.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35328 fp=#0x10000003aa4' 2024-09-21T00:30:03.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35360 fp=#0x10000003a33' 2024-09-21T00:30:03.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35392 fp=#0x100000038ef' 2024-09-21T00:30:03.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35424 fp=#0x10000003925' 2024-09-21T00:30:03.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35456 fp=#0x30000001143' 2024-09-21T00:30:03.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35520 fp=#0x3000000114a' 2024-09-21T00:30:03.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35552 fp=#0x30000001185' 2024-09-21T00:30:03.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35584 fp=#0x30000002041' 2024-09-21T00:30:03.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35616 fp=#0x3000000202b' 2024-09-21T00:30:03.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35680 fp=#0x30000002037' 2024-09-21T00:30:03.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35808 fp=#0x10000003461' 2024-09-21T00:30:03.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35872 fp=#0x1000000339e' 2024-09-21T00:30:03.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35904 fp=#0x100000033c5' 2024-09-21T00:30:03.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35936 fp=#0x300000011e7' 2024-09-21T00:30:03.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35968 fp=#0x300000011db' 2024-09-21T00:30:03.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36064 fp=#0x10000002d4e' 2024-09-21T00:30:03.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36096 fp=#0x300000010f6' 2024-09-21T00:30:03.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36160 fp=#0x10000001ef5' 2024-09-21T00:30:03.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36192 fp=#0x10000001ef8' 2024-09-21T00:30:03.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36224 fp=#0x10000002095' 2024-09-21T00:30:03.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36256 fp=#0x1000000202d' 2024-09-21T00:30:03.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36416 fp=#0x30000001600' 2024-09-21T00:30:03.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36448 fp=#0x10000001e8a' 2024-09-21T00:30:03.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36480 fp=#0x10000001e0b' 2024-09-21T00:30:03.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36512 fp=#0x10000001e13' 2024-09-21T00:30:03.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36544 fp=#0x10000001d43' 2024-09-21T00:30:03.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36576 fp=#0x10000003783' 2024-09-21T00:30:03.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36608 fp=#0x1000000442c' 2024-09-21T00:30:03.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36640 fp=#0x1000000440f' 2024-09-21T00:30:03.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36672 fp=#0x10000004455' 2024-09-21T00:30:03.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36704 fp=#0x1000000447e' 2024-09-21T00:30:03.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36736 fp=#0x30000001a8a' 2024-09-21T00:30:03.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36768 fp=#0x30000001a9e' 2024-09-21T00:30:03.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36800 fp=#0x1000000479b' 2024-09-21T00:30:03.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36832 fp=#0x1000000471e' 2024-09-21T00:30:03.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36864 fp=#0x1000000269a' 2024-09-21T00:30:03.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36896 fp=#0x10000002546' 2024-09-21T00:30:03.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36928 fp=#0x10000003567' 2024-09-21T00:30:03.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36960 fp=#0x100000025fa' 2024-09-21T00:30:03.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36992 fp=#0x10000002579' 2024-09-21T00:30:03.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37088 fp=#0x30000001795' 2024-09-21T00:30:03.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37184 fp=#0x10000003104' 2024-09-21T00:30:03.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37216 fp=#0x1000000315b' 2024-09-21T00:30:03.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37248 fp=#0x10000003142' 2024-09-21T00:30:03.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37280 fp=#0x10000003000' 2024-09-21T00:30:03.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37376 fp=#0x30000001b11' 2024-09-21T00:30:03.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37408 fp=#0x30000001ad6' 2024-09-21T00:30:03.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37440 fp=#0x30000000bfa' 2024-09-21T00:30:03.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37472 fp=#0x30000001acc' 2024-09-21T00:30:03.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37504 fp=#0x3000000107f' 2024-09-21T00:30:03.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37536 fp=#0x30000001b7b' 2024-09-21T00:30:03.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37600 fp=#0x30000001b7c' 2024-09-21T00:30:03.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37696 fp=#0x30000000ca4' 2024-09-21T00:30:03.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37920 fp=#0x30000002289' 2024-09-21T00:30:03.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37952 fp=#0x30000000dbf' 2024-09-21T00:30:03.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38016 fp=#0x30000000dc2' 2024-09-21T00:30:03.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38080 fp=#0x10000000717' 2024-09-21T00:30:03.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38112 fp=#0x10000000df8' 2024-09-21T00:30:03.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38144 fp=#0x10000000718' 2024-09-21T00:30:03.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38176 fp=#0x1000000076f' 2024-09-21T00:30:03.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38208 fp=#0x1000000071a' 2024-09-21T00:30:03.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38240 fp=#0x10000000723' 2024-09-21T00:30:03.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38304 fp=#0x10000002452' 2024-09-21T00:30:03.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38336 fp=#0x30000001a7c' 2024-09-21T00:30:03.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38368 fp=#0x30000001a6f' 2024-09-21T00:30:03.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38400 fp=#0x30000001cc9' 2024-09-21T00:30:03.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38432 fp=#0x100000032ee' 2024-09-21T00:30:03.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38464 fp=#0x100000032fc' 2024-09-21T00:30:03.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38528 fp=#0x100000020de' 2024-09-21T00:30:03.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38560 fp=#0x10000002174' 2024-09-21T00:30:03.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38592 fp=#0x100000005ec' 2024-09-21T00:30:03.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38624 fp=#0x30000001be1' 2024-09-21T00:30:03.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38656 fp=#0x30000001be8' 2024-09-21T00:30:03.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38688 fp=#0x10000000521' 2024-09-21T00:30:03.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38752 fp=#0x30000001bd8' 2024-09-21T00:30:03.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38784 fp=#0x30000001c0c' 2024-09-21T00:30:03.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38816 fp=#0x300000019d6' 2024-09-21T00:30:03.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38880 fp=#0x300000019cc' 2024-09-21T00:30:03.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38944 fp=#0x30000001d93' 2024-09-21T00:30:03.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38976 fp=#0x30000001c86' 2024-09-21T00:30:03.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39008 fp=#0x30000001d57' 2024-09-21T00:30:03.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39040 fp=#0x30000001d42' 2024-09-21T00:30:03.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39072 fp=#0x30000001d30' 2024-09-21T00:30:03.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39168 fp=#0x10000003520' 2024-09-21T00:30:03.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39232 fp=#0x10000003543' 2024-09-21T00:30:03.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39264 fp=#0x10000003524' 2024-09-21T00:30:03.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39328 fp=#0x10000003573' 2024-09-21T00:30:03.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39360 fp=#0x10000003f99' 2024-09-21T00:30:03.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39392 fp=#0x10000003fc5' 2024-09-21T00:30:03.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39424 fp=#0x10000003eda' 2024-09-21T00:30:03.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39456 fp=#0x30000001f15' 2024-09-21T00:30:03.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39488 fp=#0x30000001f73' 2024-09-21T00:30:03.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39520 fp=#0x30000001f42' 2024-09-21T00:30:03.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39552 fp=#0x30000001f51' 2024-09-21T00:30:03.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39584 fp=#0x30000001f2b' 2024-09-21T00:30:03.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39616 fp=#0x10000002ab2' 2024-09-21T00:30:03.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39680 fp=#0x3000000197d' 2024-09-21T00:30:03.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39712 fp=#0x30000000ea5' 2024-09-21T00:30:03.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39744 fp=#0x30000000e62' 2024-09-21T00:30:03.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39776 fp=#0x30000000e74' 2024-09-21T00:30:03.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39840 fp=#0x30000000e66' 2024-09-21T00:30:03.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39872 fp=#0x30000000ead' 2024-09-21T00:30:03.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39936 fp=#0x30000001019' 2024-09-21T00:30:03.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40032 fp=#0x30000001669' 2024-09-21T00:30:03.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40064 fp=#0x30000001671' 2024-09-21T00:30:03.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40096 fp=#0x30000001844' 2024-09-21T00:30:03.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40128 fp=#0x3000000152a' 2024-09-21T00:30:03.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40160 fp=#0x3000000181b' 2024-09-21T00:30:03.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40192 fp=#0x10000001d28' 2024-09-21T00:30:03.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40224 fp=#0x30000000d94' 2024-09-21T00:30:03.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40256 fp=#0x30000000d89' 2024-09-21T00:30:03.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40288 fp=#0x10000001d3e' 2024-09-21T00:30:03.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40320 fp=#0x300000009ee' 2024-09-21T00:30:03.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40352 fp=#0x10000004cad' 2024-09-21T00:30:03.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40384 fp=#0x10000004e40' 2024-09-21T00:30:03.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40416 fp=#0x300000022ed' 2024-09-21T00:30:03.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40448 fp=#0x10000004c5b' 2024-09-21T00:30:03.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40480 fp=#0x10000004c84' 2024-09-21T00:30:03.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40512 fp=#0x1000000465b' 2024-09-21T00:30:03.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40544 fp=#0x10000000cb3' 2024-09-21T00:30:03.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40576 fp=#0x10000000f92' 2024-09-21T00:30:03.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40608 fp=#0x2000000060d' 2024-09-21T00:30:03.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40672 fp=#0x10000000f8f' 2024-09-21T00:30:03.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40704 fp=#0x200000005b1' 2024-09-21T00:30:03.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40736 fp=#0x200000005c9' 2024-09-21T00:30:03.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40768 fp=#0x30000001436' 2024-09-21T00:30:03.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40800 fp=#0x3000000141b' 2024-09-21T00:30:03.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40832 fp=#0x10000004a86' 2024-09-21T00:30:03.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40896 fp=#0x10000004cb0' 2024-09-21T00:30:03.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40960 fp=#0x10000004941' 2024-09-21T00:30:03.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40992 fp=#0x1000000493c' 2024-09-21T00:30:03.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41024 fp=#0x200000006fd' 2024-09-21T00:30:03.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41056 fp=#0x20000000713' 2024-09-21T00:30:03.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41088 fp=#0x20000000703' 2024-09-21T00:30:03.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41120 fp=#0x20000000205' 2024-09-21T00:30:03.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41152 fp=#0x10000000e1b' 2024-09-21T00:30:03.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41184 fp=#0x20000000a9a' 2024-09-21T00:30:03.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41216 fp=#0x10000000e89' 2024-09-21T00:30:03.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41248 fp=#0x10000000edf' 2024-09-21T00:30:03.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41280 fp=#0x10000000fee' 2024-09-21T00:30:03.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41312 fp=#0x10000000cce' 2024-09-21T00:30:03.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41376 fp=#0x10000000982' 2024-09-21T00:30:03.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41408 fp=#0x10000000e73' 2024-09-21T00:30:03.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41440 fp=#0x3000000207d' 2024-09-21T00:30:03.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41472 fp=#0x10000000de8' 2024-09-21T00:30:03.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41504 fp=#0x10000000dc9' 2024-09-21T00:30:03.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41536 fp=#0x3000000206e' 2024-09-21T00:30:03.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41632 fp=#0x30000001845' 2024-09-21T00:30:03.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41760 fp=#0x30000000910' 2024-09-21T00:30:03.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41792 fp=#0x300000008b0' 2024-09-21T00:30:03.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41856 fp=#0x300000009bf' 2024-09-21T00:30:03.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41888 fp=#0x3000000082f' 2024-09-21T00:30:03.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41920 fp=#0x3000000089f' 2024-09-21T00:30:03.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41952 fp=#0x3000000081a' 2024-09-21T00:30:03.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41984 fp=#0x30000001d90' 2024-09-21T00:30:03.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42016 fp=#0x300000007d9' 2024-09-21T00:30:03.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42048 fp=#0x30000001d9d' 2024-09-21T00:30:03.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42080 fp=#0x30000001d9b' 2024-09-21T00:30:03.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42144 fp=#0x30000001d7c' 2024-09-21T00:30:03.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42176 fp=#0x30000000bc7' 2024-09-21T00:30:03.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42208 fp=#0x30000001b99' 2024-09-21T00:30:03.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42240 fp=#0x30000001b8d' 2024-09-21T00:30:03.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42272 fp=#0x30000000bc2' 2024-09-21T00:30:03.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42304 fp=#0x30000000b58' 2024-09-21T00:30:03.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42368 fp=#0x30000000adc' 2024-09-21T00:30:03.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42400 fp=#0x30000000ae0' 2024-09-21T00:30:03.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42432 fp=#0x300000013c0' 2024-09-21T00:30:03.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42464 fp=#0x100000040f4' 2024-09-21T00:30:03.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42496 fp=#0x30000000a42' 2024-09-21T00:30:03.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42528 fp=#0x30000000a51' 2024-09-21T00:30:03.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42560 fp=#0x30000000a63' 2024-09-21T00:30:03.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42656 fp=#0x3000000125c' 2024-09-21T00:30:03.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42688 fp=#0x100000019b0' 2024-09-21T00:30:03.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42816 fp=#0x100000019e7' 2024-09-21T00:30:03.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42880 fp=#0x10000001958' 2024-09-21T00:30:03.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42944 fp=#0x30000000e01' 2024-09-21T00:30:03.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42976 fp=#0x30000000dfc' 2024-09-21T00:30:03.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43040 fp=#0x30000001728' 2024-09-21T00:30:03.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43072 fp=#0x30000001687' 2024-09-21T00:30:03.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43104 fp=#0x3000000168f' 2024-09-21T00:30:03.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43136 fp=#0x1000000189d' 2024-09-21T00:30:03.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43168 fp=#0x10000001891' 2024-09-21T00:30:03.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43232 fp=#0x10000001889' 2024-09-21T00:30:03.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43296 fp=#0x30000000f6c' 2024-09-21T00:30:03.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43328 fp=#0x30000000f86' 2024-09-21T00:30:03.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43360 fp=#0x30000000f3c' 2024-09-21T00:30:03.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43392 fp=#0x30000000efd' 2024-09-21T00:30:03.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43424 fp=#0x1000000416e' 2024-09-21T00:30:03.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43456 fp=#0x300000023bd' 2024-09-21T00:30:03.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43488 fp=#0x300000023ac' 2024-09-21T00:30:03.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43520 fp=#0x300000023e6' 2024-09-21T00:30:03.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43552 fp=#0x300000023d2' 2024-09-21T00:30:03.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43584 fp=#0x30000000fd4' 2024-09-21T00:30:03.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43616 fp=#0x3000000211c' 2024-09-21T00:30:03.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43648 fp=#0x300000020fb' 2024-09-21T00:30:03.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43712 fp=#0x300000010d4' 2024-09-21T00:30:03.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43776 fp=#0x30000000fea' 2024-09-21T00:30:03.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43904 fp=#0x10000002997' 2024-09-21T00:30:03.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43936 fp=#0x300000013cc' 2024-09-21T00:30:03.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31809 fp=#0x30000000cb4' 2024-09-21T00:30:03.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31937 fp=#0x30000001256' 2024-09-21T00:30:03.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34689 fp=#0x30000002254' 2024-09-21T00:30:03.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34721 fp=#0x10000000b4b' 2024-09-21T00:30:03.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34753 fp=#0x30000002274' 2024-09-21T00:30:03.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34849 fp=#0x1000000281e' 2024-09-21T00:30:03.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34881 fp=#0x100000026de' 2024-09-21T00:30:03.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34913 fp=#0x100000026dd' 2024-09-21T00:30:03.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34977 fp=#0x30000002067' 2024-09-21T00:30:03.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35009 fp=#0x300000020b2' 2024-09-21T00:30:03.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35041 fp=#0x300000020b8' 2024-09-21T00:30:03.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35073 fp=#0x10000003285' 2024-09-21T00:30:03.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35105 fp=#0x1000000412a' 2024-09-21T00:30:03.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35137 fp=#0x100000038bb' 2024-09-21T00:30:03.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35169 fp=#0x10000003986' 2024-09-21T00:30:03.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35201 fp=#0x1000000418a' 2024-09-21T00:30:03.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35233 fp=#0x100000041e3' 2024-09-21T00:30:03.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35265 fp=#0x100000039b1' 2024-09-21T00:30:03.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35297 fp=#0x10000004133' 2024-09-21T00:30:03.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35329 fp=#0x100000039e6' 2024-09-21T00:30:03.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35393 fp=#0x1000000392f' 2024-09-21T00:30:03.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35425 fp=#0x1000000388f' 2024-09-21T00:30:03.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35457 fp=#0x30000001180' 2024-09-21T00:30:03.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35489 fp=#0x30000001199' 2024-09-21T00:30:03.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35521 fp=#0x30000001145' 2024-09-21T00:30:03.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35553 fp=#0x3000000113e' 2024-09-21T00:30:03.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35585 fp=#0x30000002038' 2024-09-21T00:30:03.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35617 fp=#0x30000002023' 2024-09-21T00:30:03.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35681 fp=#0x30000001fe6' 2024-09-21T00:30:03.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35809 fp=#0x10000003384' 2024-09-21T00:30:03.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35841 fp=#0x1000000339d' 2024-09-21T00:30:03.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35905 fp=#0x1000000340e' 2024-09-21T00:30:03.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35937 fp=#0x300000015b1' 2024-09-21T00:30:03.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35969 fp=#0x300000011f7' 2024-09-21T00:30:03.204 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36065 fp=#0x10000002cb0' 2024-09-21T00:30:03.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36161 fp=#0x100000020a8' 2024-09-21T00:30:03.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36193 fp=#0x10000001f1b' 2024-09-21T00:30:03.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36257 fp=#0x10000001eb8' 2024-09-21T00:30:03.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36449 fp=#0x10000001df8' 2024-09-21T00:30:03.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36481 fp=#0x10000001e1f' 2024-09-21T00:30:03.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36513 fp=#0x10000001d7b' 2024-09-21T00:30:03.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36545 fp=#0x10000001d9a' 2024-09-21T00:30:03.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36577 fp=#0x1000000381a' 2024-09-21T00:30:03.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36609 fp=#0x300000019b9' 2024-09-21T00:30:03.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36641 fp=#0x10000004414' 2024-09-21T00:30:03.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36673 fp=#0x10000004459' 2024-09-21T00:30:03.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36705 fp=#0x100000044b2' 2024-09-21T00:30:03.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36737 fp=#0x100000047d9' 2024-09-21T00:30:03.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36769 fp=#0x10000004735' 2024-09-21T00:30:03.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36801 fp=#0x30000001a89' 2024-09-21T00:30:03.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36833 fp=#0x100000046fb' 2024-09-21T00:30:03.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36865 fp=#0x100000026a0' 2024-09-21T00:30:03.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36897 fp=#0x10000003541' 2024-09-21T00:30:03.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36929 fp=#0x10000003641' 2024-09-21T00:30:03.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36993 fp=#0x100000035aa' 2024-09-21T00:30:03.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37057 fp=#0x300000017f3' 2024-09-21T00:30:03.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37185 fp=#0x10000002ff7' 2024-09-21T00:30:03.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37217 fp=#0x100000031af' 2024-09-21T00:30:03.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37249 fp=#0x100000030a2' 2024-09-21T00:30:03.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37281 fp=#0x10000003079' 2024-09-21T00:30:03.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37345 fp=#0x30000001ac0' 2024-09-21T00:30:03.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37377 fp=#0x30000001b38' 2024-09-21T00:30:03.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37409 fp=#0x30000000c05' 2024-09-21T00:30:03.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37441 fp=#0x30000001ab3' 2024-09-21T00:30:03.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37473 fp=#0x30000000c0b' 2024-09-21T00:30:03.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37537 fp=#0x30000001b76' 2024-09-21T00:30:03.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37569 fp=#0x30000001b81' 2024-09-21T00:30:03.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37601 fp=#0x30000001b74' 2024-09-21T00:30:03.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37697 fp=#0x10000002849' 2024-09-21T00:30:03.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37729 fp=#0x30000000c6e' 2024-09-21T00:30:03.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37761 fp=#0x30000000c5c' 2024-09-21T00:30:03.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37793 fp=#0x30000000c40' 2024-09-21T00:30:03.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37825 fp=#0x30000000c6a' 2024-09-21T00:30:03.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37889 fp=#0x3000000127d' 2024-09-21T00:30:03.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37953 fp=#0x30000000db7' 2024-09-21T00:30:03.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37985 fp=#0x30000000cef' 2024-09-21T00:30:03.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38049 fp=#0x30000000dab' 2024-09-21T00:30:03.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38081 fp=#0x30000000499' 2024-09-21T00:30:03.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38113 fp=#0x10000000755' 2024-09-21T00:30:03.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38145 fp=#0x10000000744' 2024-09-21T00:30:03.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38177 fp=#0x10000000745' 2024-09-21T00:30:03.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38209 fp=#0x10000000ea1' 2024-09-21T00:30:03.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38241 fp=#0x10000000726' 2024-09-21T00:30:03.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38305 fp=#0x30000001a6d' 2024-09-21T00:30:03.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38337 fp=#0x10000002491' 2024-09-21T00:30:03.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38369 fp=#0x30000001a6a' 2024-09-21T00:30:03.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38401 fp=#0x10000003275' 2024-09-21T00:30:03.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38433 fp=#0x10000003341' 2024-09-21T00:30:03.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38465 fp=#0x1000000331a' 2024-09-21T00:30:03.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38497 fp=#0x10000002177' 2024-09-21T00:30:03.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38561 fp=#0x10000002115' 2024-09-21T00:30:03.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38657 fp=#0x30000001c4c' 2024-09-21T00:30:03.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38689 fp=#0x100000005bd' 2024-09-21T00:30:03.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38753 fp=#0x30000001c15' 2024-09-21T00:30:03.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38785 fp=#0x30000001bc0' 2024-09-21T00:30:03.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38817 fp=#0x10000002927' 2024-09-21T00:30:03.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38849 fp=#0x300000019e5' 2024-09-21T00:30:03.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38945 fp=#0x30000001c78' 2024-09-21T00:30:03.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39041 fp=#0x30000001c6e' 2024-09-21T00:30:03.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39233 fp=#0x10000003599' 2024-09-21T00:30:03.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39265 fp=#0x100000034a3' 2024-09-21T00:30:03.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39361 fp=#0x10000003ee3' 2024-09-21T00:30:03.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39393 fp=#0x10000003f7f' 2024-09-21T00:30:03.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39425 fp=#0x10000003f67' 2024-09-21T00:30:03.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39457 fp=#0x30000001efd' 2024-09-21T00:30:03.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39489 fp=#0x30000001fb4' 2024-09-21T00:30:03.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39521 fp=#0x30000001f20' 2024-09-21T00:30:03.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39553 fp=#0x30000001f63' 2024-09-21T00:30:03.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39585 fp=#0x30000001f32' 2024-09-21T00:30:03.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39649 fp=#0x10000002b17' 2024-09-21T00:30:03.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39681 fp=#0x30000000ee5' 2024-09-21T00:30:03.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39745 fp=#0x30000000eef' 2024-09-21T00:30:03.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39809 fp=#0x300000019ad' 2024-09-21T00:30:03.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39841 fp=#0x30000000eae' 2024-09-21T00:30:03.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39873 fp=#0x30000001997' 2024-09-21T00:30:03.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39905 fp=#0x30000001124' 2024-09-21T00:30:03.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39937 fp=#0x30000001027' 2024-09-21T00:30:03.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40033 fp=#0x3000000155b' 2024-09-21T00:30:03.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40065 fp=#0x300000017fc' 2024-09-21T00:30:03.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40129 fp=#0x30000001816' 2024-09-21T00:30:03.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40193 fp=#0x300000009e4' 2024-09-21T00:30:03.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40225 fp=#0x30000000d65' 2024-09-21T00:30:03.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40257 fp=#0x30000000d6d' 2024-09-21T00:30:03.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40289 fp=#0x30000000d74' 2024-09-21T00:30:03.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40321 fp=#0x30000000db3' 2024-09-21T00:30:03.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40353 fp=#0x10000004e80' 2024-09-21T00:30:03.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40385 fp=#0x20000001291' 2024-09-21T00:30:03.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40417 fp=#0x10000004ca9' 2024-09-21T00:30:03.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40449 fp=#0x30000001b02' 2024-09-21T00:30:03.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40481 fp=#0x10000004cff' 2024-09-21T00:30:03.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40513 fp=#0x1000000465d' 2024-09-21T00:30:03.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40545 fp=#0x10000000ca0' 2024-09-21T00:30:03.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40577 fp=#0x10000000f85' 2024-09-21T00:30:03.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40609 fp=#0x10000000d59' 2024-09-21T00:30:03.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40641 fp=#0x2000000058c' 2024-09-21T00:30:03.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40737 fp=#0x200000005be' 2024-09-21T00:30:03.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40769 fp=#0x3000000145c' 2024-09-21T00:30:03.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40801 fp=#0x30000001435' 2024-09-21T00:30:03.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40897 fp=#0x10000004d1d' 2024-09-21T00:30:03.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40929 fp=#0x1000000494e' 2024-09-21T00:30:03.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40961 fp=#0x10000004a6e' 2024-09-21T00:30:03.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41025 fp=#0x10000003662' 2024-09-21T00:30:03.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41057 fp=#0x20000000710' 2024-09-21T00:30:03.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41089 fp=#0x2000000070a' 2024-09-21T00:30:03.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41121 fp=#0x10000000d9c' 2024-09-21T00:30:03.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41153 fp=#0x10000000eee' 2024-09-21T00:30:03.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41185 fp=#0x10000000eaf' 2024-09-21T00:30:03.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41217 fp=#0x10000000ed9' 2024-09-21T00:30:03.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41249 fp=#0x10000000ef8' 2024-09-21T00:30:03.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41281 fp=#0x10000000d88' 2024-09-21T00:30:03.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41313 fp=#0x10000000d74' 2024-09-21T00:30:03.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41377 fp=#0x10000000e08' 2024-09-21T00:30:03.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41409 fp=#0x10000000e5b' 2024-09-21T00:30:03.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41441 fp=#0x10000000c1f' 2024-09-21T00:30:03.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41473 fp=#0x10000000bbe' 2024-09-21T00:30:03.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41505 fp=#0x10000000b94' 2024-09-21T00:30:03.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41537 fp=#0x10000000eb9' 2024-09-21T00:30:03.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41793 fp=#0x300000009ca' 2024-09-21T00:30:03.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41825 fp=#0x30000000816' 2024-09-21T00:30:03.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41857 fp=#0x3000000083d' 2024-09-21T00:30:03.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41889 fp=#0x30000000951' 2024-09-21T00:30:03.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41953 fp=#0x300000009cd' 2024-09-21T00:30:03.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41985 fp=#0x30000001d76' 2024-09-21T00:30:03.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42017 fp=#0x30000000992' 2024-09-21T00:30:03.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42049 fp=#0x30000001d84' 2024-09-21T00:30:03.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42081 fp=#0x30000000994' 2024-09-21T00:30:03.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42113 fp=#0x30000001dad' 2024-09-21T00:30:03.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42145 fp=#0x300000007b0' 2024-09-21T00:30:03.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42177 fp=#0x30000000bdd' 2024-09-21T00:30:03.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42209 fp=#0x30000000b68' 2024-09-21T00:30:03.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42241 fp=#0x30000002098' 2024-09-21T00:30:03.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42273 fp=#0x30000001bae' 2024-09-21T00:30:03.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42305 fp=#0x30000000ba2' 2024-09-21T00:30:03.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42337 fp=#0x30000000b4d' 2024-09-21T00:30:03.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42369 fp=#0x30000000b08' 2024-09-21T00:30:03.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42433 fp=#0x30000000b02' 2024-09-21T00:30:03.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42465 fp=#0x10000004145' 2024-09-21T00:30:03.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42497 fp=#0x30000001b4e' 2024-09-21T00:30:03.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42593 fp=#0x30000000d30' 2024-09-21T00:30:03.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42721 fp=#0x100000018db' 2024-09-21T00:30:03.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42753 fp=#0x100000018d9' 2024-09-21T00:30:03.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42785 fp=#0x100000019da' 2024-09-21T00:30:03.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42881 fp=#0x100000019cb' 2024-09-21T00:30:03.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42913 fp=#0x30000000df3' 2024-09-21T00:30:03.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42945 fp=#0x30000000e05' 2024-09-21T00:30:03.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42977 fp=#0x30000000df7' 2024-09-21T00:30:03.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43009 fp=#0x30000000df0' 2024-09-21T00:30:03.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43041 fp=#0x30000001750' 2024-09-21T00:30:03.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43073 fp=#0x30000001709' 2024-09-21T00:30:03.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43105 fp=#0x300000016ee' 2024-09-21T00:30:03.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43137 fp=#0x1000000186c' 2024-09-21T00:30:03.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43169 fp=#0x10000001882' 2024-09-21T00:30:03.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43233 fp=#0x1000000185c' 2024-09-21T00:30:03.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43265 fp=#0x30000000f0c' 2024-09-21T00:30:03.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43297 fp=#0x30000000f04' 2024-09-21T00:30:03.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43361 fp=#0x30000000f37' 2024-09-21T00:30:03.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43425 fp=#0x10000004196' 2024-09-21T00:30:03.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43457 fp=#0x300000023f3' 2024-09-21T00:30:03.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43489 fp=#0x30000002387' 2024-09-21T00:30:03.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43521 fp=#0x300000023ab' 2024-09-21T00:30:03.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43553 fp=#0x300000023a6' 2024-09-21T00:30:03.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43585 fp=#0x30000001104' 2024-09-21T00:30:03.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43617 fp=#0x30000000fa0' 2024-09-21T00:30:03.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43649 fp=#0x300000021cc' 2024-09-21T00:30:03.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43713 fp=#0x30000000ffb' 2024-09-21T00:30:03.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43777 fp=#0x300000010bc' 2024-09-21T00:30:03.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43841 fp=#0x10000002a26' 2024-09-21T00:30:03.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43873 fp=#0x1000000397f' 2024-09-21T00:30:03.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43905 fp=#0x100000039e8' 2024-09-21T00:30:03.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43937 fp=#0x30000001338' 2024-09-21T00:30:03.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31810 fp=#0x10000000715' 2024-09-21T00:30:03.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31938 fp=#0x100000018c1' 2024-09-21T00:30:03.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34690 fp=#0x30000002226' 2024-09-21T00:30:03.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34722 fp=#0x3000000222d' 2024-09-21T00:30:03.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34754 fp=#0x3000000223b' 2024-09-21T00:30:03.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34850 fp=#0x100000026df' 2024-09-21T00:30:03.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34914 fp=#0x1000000484e' 2024-09-21T00:30:03.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34946 fp=#0x100000026e0' 2024-09-21T00:30:03.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34978 fp=#0x10000003834' 2024-09-21T00:30:03.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35010 fp=#0x10000003846' 2024-09-21T00:30:03.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35042 fp=#0x100000032d2' 2024-09-21T00:30:03.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35074 fp=#0x100000032d8' 2024-09-21T00:30:03.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35106 fp=#0x30000001973' 2024-09-21T00:30:03.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35138 fp=#0x100000041b0' 2024-09-21T00:30:03.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35170 fp=#0x1000000412f' 2024-09-21T00:30:03.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35202 fp=#0x10000004168' 2024-09-21T00:30:03.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35234 fp=#0x1000000417b' 2024-09-21T00:30:03.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35266 fp=#0x10000004150' 2024-09-21T00:30:03.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35298 fp=#0x100000039b5' 2024-09-21T00:30:03.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35330 fp=#0x100000041db' 2024-09-21T00:30:03.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35362 fp=#0x10000003914' 2024-09-21T00:30:03.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35394 fp=#0x1000000389c' 2024-09-21T00:30:03.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35426 fp=#0x10000003901' 2024-09-21T00:30:03.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35458 fp=#0x30000001136' 2024-09-21T00:30:03.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35490 fp=#0x30000001eee' 2024-09-21T00:30:03.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35522 fp=#0x30000001139' 2024-09-21T00:30:03.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35554 fp=#0x30000001144' 2024-09-21T00:30:03.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35586 fp=#0x3000000204c' 2024-09-21T00:30:03.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35618 fp=#0x30000001fcf' 2024-09-21T00:30:03.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35650 fp=#0x30000001fdc' 2024-09-21T00:30:03.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35682 fp=#0x30000002042' 2024-09-21T00:30:03.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35714 fp=#0x30000001e6f' 2024-09-21T00:30:03.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35810 fp=#0x100000033b6' 2024-09-21T00:30:03.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35842 fp=#0x10000003436' 2024-09-21T00:30:03.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35874 fp=#0x10000003451' 2024-09-21T00:30:03.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35906 fp=#0x1000000344f' 2024-09-21T00:30:03.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35938 fp=#0x300000011ce' 2024-09-21T00:30:03.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35970 fp=#0x300000011f2' 2024-09-21T00:30:03.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36066 fp=#0x10000002d4b' 2024-09-21T00:30:03.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36130 fp=#0x10000002cc1' 2024-09-21T00:30:03.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36162 fp=#0x10000001eb2' 2024-09-21T00:30:03.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36194 fp=#0x100000020ac' 2024-09-21T00:30:03.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36226 fp=#0x10000001ee8' 2024-09-21T00:30:03.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36258 fp=#0x30000001603' 2024-09-21T00:30:03.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36386 fp=#0x300000015fb' 2024-09-21T00:30:03.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36450 fp=#0x10000001d82' 2024-09-21T00:30:03.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36482 fp=#0x10000001e51' 2024-09-21T00:30:03.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36514 fp=#0x10000001d78' 2024-09-21T00:30:03.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36546 fp=#0x10000001e55' 2024-09-21T00:30:03.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36578 fp=#0x100000044a2' 2024-09-21T00:30:03.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36610 fp=#0x10000003785' 2024-09-21T00:30:03.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36642 fp=#0x100000044a4' 2024-09-21T00:30:03.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36674 fp=#0x100000037e9' 2024-09-21T00:30:03.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36706 fp=#0x10000004493' 2024-09-21T00:30:03.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36738 fp=#0x30000001aa1' 2024-09-21T00:30:03.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36770 fp=#0x30000001a9b' 2024-09-21T00:30:03.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36802 fp=#0x10000004745' 2024-09-21T00:30:03.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36834 fp=#0x30000001aa4' 2024-09-21T00:30:03.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36866 fp=#0x10000002540' 2024-09-21T00:30:03.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36930 fp=#0x10000002563' 2024-09-21T00:30:03.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36962 fp=#0x100000035de' 2024-09-21T00:30:03.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36994 fp=#0x100000025fc' 2024-09-21T00:30:03.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37026 fp=#0x1000000253a' 2024-09-21T00:30:03.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37218 fp=#0x100000030f7' 2024-09-21T00:30:03.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37282 fp=#0x10000002fe1' 2024-09-21T00:30:03.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37314 fp=#0x100000030d6' 2024-09-21T00:30:03.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37346 fp=#0x30000000c21' 2024-09-21T00:30:03.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37378 fp=#0x30000001aef' 2024-09-21T00:30:03.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37410 fp=#0x30000001ad1' 2024-09-21T00:30:03.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37442 fp=#0x30000000c15' 2024-09-21T00:30:03.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37474 fp=#0x30000000c13' 2024-09-21T00:30:03.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37506 fp=#0x3000000111f' 2024-09-21T00:30:03.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37538 fp=#0x30000000ffa' 2024-09-21T00:30:03.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37602 fp=#0x30000001b6f' 2024-09-21T00:30:03.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37634 fp=#0x1000000291d' 2024-09-21T00:30:03.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37666 fp=#0x1000000284f' 2024-09-21T00:30:03.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37698 fp=#0x10000002852' 2024-09-21T00:30:03.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37794 fp=#0x30000000c4b' 2024-09-21T00:30:03.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37826 fp=#0x30000000c41' 2024-09-21T00:30:03.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37858 fp=#0x30000001277' 2024-09-21T00:30:03.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37890 fp=#0x300000012bf' 2024-09-21T00:30:03.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37954 fp=#0x30000000cce' 2024-09-21T00:30:03.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38018 fp=#0x30000000cf3' 2024-09-21T00:30:03.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38082 fp=#0x1000000072f' 2024-09-21T00:30:03.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38114 fp=#0x10000000769' 2024-09-21T00:30:03.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38146 fp=#0x10000000748' 2024-09-21T00:30:03.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38178 fp=#0x10000002127' 2024-09-21T00:30:03.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38210 fp=#0x10000000746' 2024-09-21T00:30:03.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38242 fp=#0x1000000073e' 2024-09-21T00:30:03.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38306 fp=#0x30000001a6c' 2024-09-21T00:30:03.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38338 fp=#0x30000001a4f' 2024-09-21T00:30:03.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38402 fp=#0x1000000334c' 2024-09-21T00:30:03.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38498 fp=#0x1000000216b' 2024-09-21T00:30:03.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38530 fp=#0x100000020f6' 2024-09-21T00:30:03.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38562 fp=#0x1000000217a' 2024-09-21T00:30:03.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38594 fp=#0x100000005ee' 2024-09-21T00:30:03.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38626 fp=#0x100000005df' 2024-09-21T00:30:03.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38658 fp=#0x10000000529' 2024-09-21T00:30:03.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38690 fp=#0x30000001c10' 2024-09-21T00:30:03.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38722 fp=#0x100000005da' 2024-09-21T00:30:03.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38754 fp=#0x30000001c1c' 2024-09-21T00:30:03.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38786 fp=#0x30000001c13' 2024-09-21T00:30:03.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38818 fp=#0x300000019f1' 2024-09-21T00:30:03.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38882 fp=#0x300000012e8' 2024-09-21T00:30:03.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38978 fp=#0x30000001d45' 2024-09-21T00:30:03.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39010 fp=#0x30000001c70' 2024-09-21T00:30:03.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39042 fp=#0x30000001d31' 2024-09-21T00:30:03.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39170 fp=#0x1000000127c' 2024-09-21T00:30:03.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39330 fp=#0x100000034bf' 2024-09-21T00:30:03.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39362 fp=#0x10000003edc' 2024-09-21T00:30:03.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39394 fp=#0x10000003ef8' 2024-09-21T00:30:03.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39426 fp=#0x10000003f20' 2024-09-21T00:30:03.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39458 fp=#0x30000001f60' 2024-09-21T00:30:03.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39490 fp=#0x30000001fa0' 2024-09-21T00:30:03.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39522 fp=#0x30000001f52' 2024-09-21T00:30:03.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39554 fp=#0x30000001f1c' 2024-09-21T00:30:03.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39650 fp=#0x10000002aa9' 2024-09-21T00:30:03.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39682 fp=#0x30000000e64' 2024-09-21T00:30:03.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39714 fp=#0x30000000ef2' 2024-09-21T00:30:03.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39746 fp=#0x30000000edd' 2024-09-21T00:30:03.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39778 fp=#0x3000000198a' 2024-09-21T00:30:03.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39810 fp=#0x30000000ee0' 2024-09-21T00:30:03.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39842 fp=#0x30000000ee2' 2024-09-21T00:30:03.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39906 fp=#0x3000000111d' 2024-09-21T00:30:03.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40002 fp=#0x30000001109' 2024-09-21T00:30:03.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40034 fp=#0x30000001846' 2024-09-21T00:30:03.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40066 fp=#0x3000000166f' 2024-09-21T00:30:03.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40098 fp=#0x30000001856' 2024-09-21T00:30:03.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40130 fp=#0x3000000151c' 2024-09-21T00:30:03.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40162 fp=#0x3000000166d' 2024-09-21T00:30:03.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40226 fp=#0x10000001d71' 2024-09-21T00:30:03.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40258 fp=#0x30000001d20' 2024-09-21T00:30:03.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40322 fp=#0x30000000d72' 2024-09-21T00:30:03.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40354 fp=#0x10000004e49' 2024-09-21T00:30:03.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40386 fp=#0x300000022cb' 2024-09-21T00:30:03.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40418 fp=#0x30000002324' 2024-09-21T00:30:03.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40450 fp=#0x10000004e75' 2024-09-21T00:30:03.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40482 fp=#0x10000004e0d' 2024-09-21T00:30:03.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40546 fp=#0x20000000433' 2024-09-21T00:30:03.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40578 fp=#0x10000000f3c' 2024-09-21T00:30:03.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40610 fp=#0x10000000d6f' 2024-09-21T00:30:03.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40674 fp=#0x10000000f58' 2024-09-21T00:30:03.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40706 fp=#0x10000000fa2' 2024-09-21T00:30:03.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40738 fp=#0x10000000cc3' 2024-09-21T00:30:03.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40770 fp=#0x3000000145e' 2024-09-21T00:30:03.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40802 fp=#0x3000000141c' 2024-09-21T00:30:03.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40866 fp=#0x10000004948' 2024-09-21T00:30:03.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40930 fp=#0x10000004c7a' 2024-09-21T00:30:03.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40994 fp=#0x1000000495b' 2024-09-21T00:30:03.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41026 fp=#0x100000036be' 2024-09-21T00:30:03.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41058 fp=#0x20000000724' 2024-09-21T00:30:03.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41122 fp=#0x20000000201' 2024-09-21T00:30:03.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41154 fp=#0x10000000e61' 2024-09-21T00:30:03.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41186 fp=#0x10000000f9d' 2024-09-21T00:30:03.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41218 fp=#0x2000000046d' 2024-09-21T00:30:03.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41250 fp=#0x20000000ac0' 2024-09-21T00:30:03.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41282 fp=#0x2000000047c' 2024-09-21T00:30:03.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41314 fp=#0x10000000d83' 2024-09-21T00:30:03.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41378 fp=#0x30000002070' 2024-09-21T00:30:03.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41410 fp=#0x10000000926' 2024-09-21T00:30:03.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41506 fp=#0x10000000bf4' 2024-09-21T00:30:03.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41538 fp=#0x10000000e09' 2024-09-21T00:30:03.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41570 fp=#0x30000001906' 2024-09-21T00:30:03.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41602 fp=#0x30000001847' 2024-09-21T00:30:03.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41762 fp=#0x300000007ef' 2024-09-21T00:30:03.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41794 fp=#0x30000000850' 2024-09-21T00:30:03.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41826 fp=#0x300000009ae' 2024-09-21T00:30:03.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41890 fp=#0x300000009a9' 2024-09-21T00:30:03.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41922 fp=#0x3000000095e' 2024-09-21T00:30:03.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41986 fp=#0x3000000093d' 2024-09-21T00:30:03.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42018 fp=#0x30000000865' 2024-09-21T00:30:03.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42050 fp=#0x300000007ce' 2024-09-21T00:30:03.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42114 fp=#0x30000001316' 2024-09-21T00:30:03.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42146 fp=#0x300000009a2' 2024-09-21T00:30:03.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42178 fp=#0x30000002091' 2024-09-21T00:30:03.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42210 fp=#0x30000000b5f' 2024-09-21T00:30:03.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42242 fp=#0x30000000bd9' 2024-09-21T00:30:03.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42274 fp=#0x30000001bb1' 2024-09-21T00:30:03.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42306 fp=#0x30000000b56' 2024-09-21T00:30:03.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42338 fp=#0x30000000aef' 2024-09-21T00:30:03.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42370 fp=#0x30000000b0c' 2024-09-21T00:30:03.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42402 fp=#0x30000000adf' 2024-09-21T00:30:03.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42434 fp=#0x30000000b1f' 2024-09-21T00:30:03.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42466 fp=#0x30000000a50' 2024-09-21T00:30:03.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42498 fp=#0x100000040f7' 2024-09-21T00:30:03.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42530 fp=#0x30000000a4d' 2024-09-21T00:30:03.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42658 fp=#0x3000000125f' 2024-09-21T00:30:03.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42722 fp=#0x10000001a0f' 2024-09-21T00:30:03.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42818 fp=#0x100000019ba' 2024-09-21T00:30:03.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42882 fp=#0x10000001979' 2024-09-21T00:30:03.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42914 fp=#0x30000000e48' 2024-09-21T00:30:03.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42978 fp=#0x30000000ded' 2024-09-21T00:30:03.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43010 fp=#0x30000000df6' 2024-09-21T00:30:03.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43042 fp=#0x300000016f0' 2024-09-21T00:30:03.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43106 fp=#0x300000016c4' 2024-09-21T00:30:03.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43266 fp=#0x30000000f7e' 2024-09-21T00:30:03.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43330 fp=#0x30000000f4e' 2024-09-21T00:30:03.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43426 fp=#0x1000000419b' 2024-09-21T00:30:03.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43458 fp=#0x300000023fc' 2024-09-21T00:30:03.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43490 fp=#0x300000023b3' 2024-09-21T00:30:03.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43522 fp=#0x300000023d0' 2024-09-21T00:30:03.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43554 fp=#0x300000023e9' 2024-09-21T00:30:03.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43682 fp=#0x30000002118' 2024-09-21T00:30:03.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43714 fp=#0x30000002108' 2024-09-21T00:30:03.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43746 fp=#0x30000000fe6' 2024-09-21T00:30:03.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43778 fp=#0x30000000fa5' 2024-09-21T00:30:03.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43810 fp=#0x10000002991' 2024-09-21T00:30:03.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43842 fp=#0x100000039f2' 2024-09-21T00:30:03.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43938 fp=#0x3000000139a' 2024-09-21T00:30:03.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31747 fp=#0x10000000b35' 2024-09-21T00:30:03.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31939 fp=#0x30000000de9' 2024-09-21T00:30:03.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34723 fp=#0x10000000c63' 2024-09-21T00:30:03.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34755 fp=#0x30000002244' 2024-09-21T00:30:03.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34851 fp=#0x100000027f8' 2024-09-21T00:30:03.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34883 fp=#0x100000026f2' 2024-09-21T00:30:03.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34915 fp=#0x100000027dd' 2024-09-21T00:30:03.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34947 fp=#0x100000027e6' 2024-09-21T00:30:03.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34979 fp=#0x10000003308' 2024-09-21T00:30:03.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35011 fp=#0x300000020ab' 2024-09-21T00:30:03.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35043 fp=#0x10000003856' 2024-09-21T00:30:03.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35075 fp=#0x10000003300' 2024-09-21T00:30:03.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35107 fp=#0x10000003923' 2024-09-21T00:30:03.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35139 fp=#0x3000000134b' 2024-09-21T00:30:03.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35171 fp=#0x10000004156' 2024-09-21T00:30:03.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35203 fp=#0x10000003a36' 2024-09-21T00:30:03.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35235 fp=#0x100000038d7' 2024-09-21T00:30:03.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35267 fp=#0x1000000393d' 2024-09-21T00:30:03.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35299 fp=#0x1000000414b' 2024-09-21T00:30:03.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35331 fp=#0x10000003ad8' 2024-09-21T00:30:03.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35363 fp=#0x100000039f5' 2024-09-21T00:30:03.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35395 fp=#0x1000000394a' 2024-09-21T00:30:03.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35427 fp=#0x100000039ec' 2024-09-21T00:30:03.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35459 fp=#0x300000011a3' 2024-09-21T00:30:03.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35491 fp=#0x3000000112d' 2024-09-21T00:30:03.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35523 fp=#0x30000001187' 2024-09-21T00:30:03.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35555 fp=#0x3000000116f' 2024-09-21T00:30:03.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35587 fp=#0x30000002008' 2024-09-21T00:30:03.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35651 fp=#0x30000002047' 2024-09-21T00:30:03.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35683 fp=#0x30000001fe4' 2024-09-21T00:30:03.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35811 fp=#0x100000033aa' 2024-09-21T00:30:03.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35843 fp=#0x1000000344b' 2024-09-21T00:30:03.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35875 fp=#0x10000003408' 2024-09-21T00:30:03.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35939 fp=#0x30000001200' 2024-09-21T00:30:03.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35971 fp=#0x300000011d8' 2024-09-21T00:30:03.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36099 fp=#0x10000002d76' 2024-09-21T00:30:03.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36131 fp=#0x10000002ce7' 2024-09-21T00:30:03.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36163 fp=#0x10000001eb9' 2024-09-21T00:30:03.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36195 fp=#0x10000001eca' 2024-09-21T00:30:03.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36227 fp=#0x10000001f93' 2024-09-21T00:30:03.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36451 fp=#0x10000001e7f' 2024-09-21T00:30:03.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36483 fp=#0x10000001d5b' 2024-09-21T00:30:03.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36515 fp=#0x10000001e5f' 2024-09-21T00:30:03.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36547 fp=#0x10000001d92' 2024-09-21T00:30:03.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36579 fp=#0x1000000448f' 2024-09-21T00:30:03.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36611 fp=#0x10000004422' 2024-09-21T00:30:03.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36643 fp=#0x10000003857' 2024-09-21T00:30:03.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36675 fp=#0x10000003794' 2024-09-21T00:30:03.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36707 fp=#0x100000037fd' 2024-09-21T00:30:03.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36739 fp=#0x100000047ba' 2024-09-21T00:30:03.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36771 fp=#0x30000001aaf' 2024-09-21T00:30:03.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36803 fp=#0x30000001aa9' 2024-09-21T00:30:03.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36835 fp=#0x30000001a9c' 2024-09-21T00:30:03.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36899 fp=#0x10000002600' 2024-09-21T00:30:03.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36931 fp=#0x1000000362b' 2024-09-21T00:30:03.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36963 fp=#0x100000035fd' 2024-09-21T00:30:03.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37059 fp=#0x30000001801' 2024-09-21T00:30:03.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37091 fp=#0x300000020c5' 2024-09-21T00:30:03.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37219 fp=#0x10000001495' 2024-09-21T00:30:03.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37283 fp=#0x100000031b1' 2024-09-21T00:30:03.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37315 fp=#0x30000000be7' 2024-09-21T00:30:03.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37347 fp=#0x30000000c1d' 2024-09-21T00:30:03.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37379 fp=#0x30000000bf7' 2024-09-21T00:30:03.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37411 fp=#0x30000000c0c' 2024-09-21T00:30:03.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37443 fp=#0x30000001abc' 2024-09-21T00:30:03.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37475 fp=#0x30000001ae3' 2024-09-21T00:30:03.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37507 fp=#0x30000000fde' 2024-09-21T00:30:03.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37539 fp=#0x30000001b73' 2024-09-21T00:30:03.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37571 fp=#0x30000001073' 2024-09-21T00:30:03.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37603 fp=#0x30000000fcc' 2024-09-21T00:30:03.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37699 fp=#0x30000000c90' 2024-09-21T00:30:03.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37731 fp=#0x30000000c44' 2024-09-21T00:30:03.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37795 fp=#0x30000000c73' 2024-09-21T00:30:03.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37827 fp=#0x30000000c71' 2024-09-21T00:30:03.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37923 fp=#0x30000002253' 2024-09-21T00:30:03.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37955 fp=#0x30000000db0' 2024-09-21T00:30:03.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38083 fp=#0x10000000776' 2024-09-21T00:30:03.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38179 fp=#0x10000000eff' 2024-09-21T00:30:03.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38211 fp=#0x10000000731' 2024-09-21T00:30:03.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38243 fp=#0x30000001ee8' 2024-09-21T00:30:03.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38307 fp=#0x30000000809' 2024-09-21T00:30:03.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38339 fp=#0x30000001a81' 2024-09-21T00:30:03.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38371 fp=#0x30000001a66' 2024-09-21T00:30:03.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38403 fp=#0x30000001cd5' 2024-09-21T00:30:03.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38435 fp=#0x30000001cf5' 2024-09-21T00:30:03.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38467 fp=#0x10000003323' 2024-09-21T00:30:03.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38499 fp=#0x100000020fe' 2024-09-21T00:30:03.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38531 fp=#0x100000020e2' 2024-09-21T00:30:03.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38563 fp=#0x10000002129' 2024-09-21T00:30:03.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38595 fp=#0x10000000ed7' 2024-09-21T00:30:03.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38627 fp=#0x100000005ba' 2024-09-21T00:30:03.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38659 fp=#0x100000005b6' 2024-09-21T00:30:03.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38691 fp=#0x30000001bca' 2024-09-21T00:30:03.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38723 fp=#0x100000005a7' 2024-09-21T00:30:03.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38787 fp=#0x30000001bf8' 2024-09-21T00:30:03.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38851 fp=#0x10000002979' 2024-09-21T00:30:03.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38883 fp=#0x300000019cf' 2024-09-21T00:30:03.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38915 fp=#0x300000019dd' 2024-09-21T00:30:03.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38947 fp=#0x30000001c37' 2024-09-21T00:30:03.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39011 fp=#0x30000001d16' 2024-09-21T00:30:03.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39043 fp=#0x30000001d1c' 2024-09-21T00:30:03.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39107 fp=#0x30000001d28' 2024-09-21T00:30:03.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39139 fp=#0x30000001c63' 2024-09-21T00:30:03.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39203 fp=#0x10000001128' 2024-09-21T00:30:03.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39235 fp=#0x100000034d5' 2024-09-21T00:30:03.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39299 fp=#0x10000001357' 2024-09-21T00:30:03.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39331 fp=#0x100000034dd' 2024-09-21T00:30:03.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39363 fp=#0x10000003f1b' 2024-09-21T00:30:03.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39395 fp=#0x10000003eff' 2024-09-21T00:30:03.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39427 fp=#0x10000003ede' 2024-09-21T00:30:03.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39459 fp=#0x30000001fb5' 2024-09-21T00:30:03.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39491 fp=#0x30000001f22' 2024-09-21T00:30:03.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39523 fp=#0x30000001f25' 2024-09-21T00:30:03.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39555 fp=#0x30000001efa' 2024-09-21T00:30:03.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39587 fp=#0x30000001f98' 2024-09-21T00:30:03.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39651 fp=#0x10000002b23' 2024-09-21T00:30:03.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39683 fp=#0x30000000e9c' 2024-09-21T00:30:03.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39715 fp=#0x30000000e6d' 2024-09-21T00:30:03.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39779 fp=#0x300000019a4' 2024-09-21T00:30:03.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39811 fp=#0x30000000ea0' 2024-09-21T00:30:03.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40035 fp=#0x3000000152e' 2024-09-21T00:30:03.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40067 fp=#0x3000000182d' 2024-09-21T00:30:03.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40099 fp=#0x300000014d0' 2024-09-21T00:30:03.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40131 fp=#0x300000017f6' 2024-09-21T00:30:03.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40227 fp=#0x10000001d05' 2024-09-21T00:30:03.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40259 fp=#0x30000001d08' 2024-09-21T00:30:03.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40323 fp=#0x30000000d84' 2024-09-21T00:30:03.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40355 fp=#0x10000004e82' 2024-09-21T00:30:03.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40387 fp=#0x10000004e62' 2024-09-21T00:30:03.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40419 fp=#0x10000004c5f' 2024-09-21T00:30:03.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40483 fp=#0x10000004d3a' 2024-09-21T00:30:03.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40611 fp=#0x200000005de' 2024-09-21T00:30:03.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40643 fp=#0x2000000044e' 2024-09-21T00:30:03.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40675 fp=#0x10000000efd' 2024-09-21T00:30:03.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40707 fp=#0x10000000ccc' 2024-09-21T00:30:03.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40739 fp=#0x10000000f6f' 2024-09-21T00:30:03.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40803 fp=#0x3000000145d' 2024-09-21T00:30:03.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40835 fp=#0x10000004a6b' 2024-09-21T00:30:03.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40899 fp=#0x10000004af8' 2024-09-21T00:30:03.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40931 fp=#0x10000004c9a' 2024-09-21T00:30:03.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40963 fp=#0x10000004ade' 2024-09-21T00:30:03.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40995 fp=#0x10000004c8e' 2024-09-21T00:30:03.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41027 fp=#0x10000003571' 2024-09-21T00:30:03.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41059 fp=#0x200000006fa' 2024-09-21T00:30:03.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41091 fp=#0x100000034e2' 2024-09-21T00:30:03.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41123 fp=#0x20000000c3b' 2024-09-21T00:30:03.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41155 fp=#0x10000000f12' 2024-09-21T00:30:03.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41187 fp=#0x20000000c16' 2024-09-21T00:30:03.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41219 fp=#0x20000000c3f' 2024-09-21T00:30:03.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41251 fp=#0x10000000e94' 2024-09-21T00:30:03.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41283 fp=#0x20000000ad1' 2024-09-21T00:30:03.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41315 fp=#0x10000000dff' 2024-09-21T00:30:03.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41379 fp=#0x10000000ecf' 2024-09-21T00:30:03.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41411 fp=#0x10000000c30' 2024-09-21T00:30:03.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41443 fp=#0x1000000094e' 2024-09-21T00:30:03.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41475 fp=#0x1000000098b' 2024-09-21T00:30:03.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41507 fp=#0x1000000093b' 2024-09-21T00:30:03.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41539 fp=#0x10000000968' 2024-09-21T00:30:03.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41603 fp=#0x30000001857' 2024-09-21T00:30:03.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41827 fp=#0x300000009cc' 2024-09-21T00:30:03.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41891 fp=#0x300000008a6' 2024-09-21T00:30:03.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41923 fp=#0x3000000087a' 2024-09-21T00:30:03.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41955 fp=#0x300000008a3' 2024-09-21T00:30:03.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41987 fp=#0x30000001d46' 2024-09-21T00:30:03.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42019 fp=#0x3000000098b' 2024-09-21T00:30:03.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42051 fp=#0x300000007df' 2024-09-21T00:30:03.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42083 fp=#0x30000000851' 2024-09-21T00:30:03.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42115 fp=#0x3000000098c' 2024-09-21T00:30:03.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42147 fp=#0x30000001d8d' 2024-09-21T00:30:03.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42179 fp=#0x30000000b74' 2024-09-21T00:30:03.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42211 fp=#0x30000001b94' 2024-09-21T00:30:03.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42243 fp=#0x30000001bab' 2024-09-21T00:30:03.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42275 fp=#0x30000000b79' 2024-09-21T00:30:03.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42307 fp=#0x30000000b59' 2024-09-21T00:30:03.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42339 fp=#0x30000000b4b' 2024-09-21T00:30:03.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42371 fp=#0x30000000b1d' 2024-09-21T00:30:03.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42403 fp=#0x30000000b3e' 2024-09-21T00:30:03.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42435 fp=#0x30000000b0d' 2024-09-21T00:30:03.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42467 fp=#0x30000000a53' 2024-09-21T00:30:03.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42499 fp=#0x100000040fb' 2024-09-21T00:30:03.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42531 fp=#0x30000000a64' 2024-09-21T00:30:03.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42659 fp=#0x100000018d3' 2024-09-21T00:30:03.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42691 fp=#0x10000001a41' 2024-09-21T00:30:03.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42723 fp=#0x10000001964' 2024-09-21T00:30:03.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42819 fp=#0x100000018f2' 2024-09-21T00:30:03.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42851 fp=#0x10000001902' 2024-09-21T00:30:03.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42979 fp=#0x30000000e47' 2024-09-21T00:30:03.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43043 fp=#0x30000001743' 2024-09-21T00:30:03.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43075 fp=#0x30000001715' 2024-09-21T00:30:03.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43139 fp=#0x1000000189b' 2024-09-21T00:30:03.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43235 fp=#0x10000001897' 2024-09-21T00:30:03.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43267 fp=#0x30000000f79' 2024-09-21T00:30:03.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43299 fp=#0x30000000f7a' 2024-09-21T00:30:03.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43331 fp=#0x30000000f0e' 2024-09-21T00:30:03.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43363 fp=#0x30000000f2d' 2024-09-21T00:30:03.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43395 fp=#0x30000000f3b' 2024-09-21T00:30:03.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43459 fp=#0x300000023b0' 2024-09-21T00:30:03.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43491 fp=#0x300000023b7' 2024-09-21T00:30:03.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43523 fp=#0x300000023d4' 2024-09-21T00:30:03.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43555 fp=#0x3000000238f' 2024-09-21T00:30:03.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43587 fp=#0x3000000110a' 2024-09-21T00:30:03.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43619 fp=#0x300000010df' 2024-09-21T00:30:03.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43651 fp=#0x3000000100c' 2024-09-21T00:30:03.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43715 fp=#0x30000000fec' 2024-09-21T00:30:03.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43747 fp=#0x300000010d8' 2024-09-21T00:30:03.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43875 fp=#0x10000003988' 2024-09-21T00:30:03.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43939 fp=#0x30000001369' 2024-09-21T00:30:03.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31748 fp=#0x10000004c58' 2024-09-21T00:30:03.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31812 fp=#0x1000000244b' 2024-09-21T00:30:03.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31940 fp=#0x30000001680' 2024-09-21T00:30:03.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34692 fp=#0x30000002229' 2024-09-21T00:30:03.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34756 fp=#0x3000000221f' 2024-09-21T00:30:03.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34884 fp=#0x1000000485d' 2024-09-21T00:30:03.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34916 fp=#0x100000026ef' 2024-09-21T00:30:03.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34980 fp=#0x300000020b0' 2024-09-21T00:30:03.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35012 fp=#0x300000020a7' 2024-09-21T00:30:03.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35076 fp=#0x3000000207a' 2024-09-21T00:30:03.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35108 fp=#0x10000003964' 2024-09-21T00:30:03.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35140 fp=#0x10000004154' 2024-09-21T00:30:03.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35172 fp=#0x1000000412e' 2024-09-21T00:30:03.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35204 fp=#0x10000004235' 2024-09-21T00:30:03.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35236 fp=#0x100000038e1' 2024-09-21T00:30:03.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35268 fp=#0x10000003992' 2024-09-21T00:30:03.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35300 fp=#0x100000041b4' 2024-09-21T00:30:03.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35332 fp=#0x10000003a9e' 2024-09-21T00:30:03.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35364 fp=#0x10000003875' 2024-09-21T00:30:03.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35396 fp=#0x1000000396c' 2024-09-21T00:30:03.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35428 fp=#0x1000000393b' 2024-09-21T00:30:03.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35460 fp=#0x3000000116c' 2024-09-21T00:30:03.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35492 fp=#0x30000001140' 2024-09-21T00:30:03.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35588 fp=#0x30000002045' 2024-09-21T00:30:03.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35620 fp=#0x30000001fce' 2024-09-21T00:30:03.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35652 fp=#0x30000002000' 2024-09-21T00:30:03.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35684 fp=#0x30000001fe7' 2024-09-21T00:30:03.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35812 fp=#0x1000000346c' 2024-09-21T00:30:03.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35876 fp=#0x100000033a7' 2024-09-21T00:30:03.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35940 fp=#0x300000020c0' 2024-09-21T00:30:03.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35972 fp=#0x300000011bf' 2024-09-21T00:30:03.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36068 fp=#0x10000002d78' 2024-09-21T00:30:03.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36164 fp=#0x10000001f3f' 2024-09-21T00:30:03.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36196 fp=#0x10000001ec8' 2024-09-21T00:30:03.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36228 fp=#0x10000001f3b' 2024-09-21T00:30:03.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36260 fp=#0x300000015fe' 2024-09-21T00:30:03.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36452 fp=#0x10000001d3f' 2024-09-21T00:30:03.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36484 fp=#0x10000001d85' 2024-09-21T00:30:03.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36516 fp=#0x10000001df2' 2024-09-21T00:30:03.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36548 fp=#0x10000001d9e' 2024-09-21T00:30:03.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36612 fp=#0x10000004472' 2024-09-21T00:30:03.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36644 fp=#0x10000004450' 2024-09-21T00:30:03.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36676 fp=#0x1000000376d' 2024-09-21T00:30:03.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36708 fp=#0x100000044a0' 2024-09-21T00:30:03.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36740 fp=#0x10000004757' 2024-09-21T00:30:03.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36772 fp=#0x30000001a7b' 2024-09-21T00:30:03.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36804 fp=#0x100000047ae' 2024-09-21T00:30:03.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36836 fp=#0x30000001a97' 2024-09-21T00:30:03.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36932 fp=#0x100000036b8' 2024-09-21T00:30:03.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36964 fp=#0x1000000358f' 2024-09-21T00:30:03.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37060 fp=#0x300000017fd' 2024-09-21T00:30:03.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37092 fp=#0x30000001806' 2024-09-21T00:30:03.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37220 fp=#0x10000003145' 2024-09-21T00:30:03.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37252 fp=#0x10000003072' 2024-09-21T00:30:03.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37316 fp=#0x30000001ad7' 2024-09-21T00:30:03.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37348 fp=#0x30000001b0f' 2024-09-21T00:30:03.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37380 fp=#0x30000001978' 2024-09-21T00:30:03.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37412 fp=#0x30000001af7' 2024-09-21T00:30:03.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37444 fp=#0x30000001af3' 2024-09-21T00:30:03.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37476 fp=#0x30000000feb' 2024-09-21T00:30:03.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37508 fp=#0x30000000fe4' 2024-09-21T00:30:03.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37540 fp=#0x30000001b87' 2024-09-21T00:30:03.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37636 fp=#0x30000000c3c' 2024-09-21T00:30:03.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37700 fp=#0x30000000c57' 2024-09-21T00:30:03.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37796 fp=#0x30000000c7a' 2024-09-21T00:30:03.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37828 fp=#0x30000000c59' 2024-09-21T00:30:03.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37892 fp=#0x3000000126f' 2024-09-21T00:30:03.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38084 fp=#0x10000000757' 2024-09-21T00:30:03.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38116 fp=#0x10000000747' 2024-09-21T00:30:03.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38148 fp=#0x10000000e2f' 2024-09-21T00:30:03.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38180 fp=#0x10000001c22' 2024-09-21T00:30:03.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38212 fp=#0x10000000724' 2024-09-21T00:30:03.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38244 fp=#0x10000001c04' 2024-09-21T00:30:03.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38308 fp=#0x30000001a55' 2024-09-21T00:30:03.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38372 fp=#0x30000001a88' 2024-09-21T00:30:03.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38404 fp=#0x1000000331b' 2024-09-21T00:30:03.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38436 fp=#0x10000003276' 2024-09-21T00:30:03.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38468 fp=#0x100000032fd' 2024-09-21T00:30:03.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38532 fp=#0x1000000210d' 2024-09-21T00:30:03.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38564 fp=#0x100000020eb' 2024-09-21T00:30:03.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38596 fp=#0x100000005e2' 2024-09-21T00:30:03.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38660 fp=#0x100000005bb' 2024-09-21T00:30:03.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38724 fp=#0x30000001c02' 2024-09-21T00:30:03.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38756 fp=#0x30000001c16' 2024-09-21T00:30:03.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38852 fp=#0x300000019fc' 2024-09-21T00:30:03.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38884 fp=#0x1000000293b' 2024-09-21T00:30:03.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38916 fp=#0x300000019e7' 2024-09-21T00:30:03.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38948 fp=#0x30000001d85' 2024-09-21T00:30:03.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38980 fp=#0x30000001c39' 2024-09-21T00:30:03.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39012 fp=#0x30000001d22' 2024-09-21T00:30:03.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39044 fp=#0x30000001c67' 2024-09-21T00:30:03.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39172 fp=#0x10000001283' 2024-09-21T00:30:03.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39364 fp=#0x10000003fca' 2024-09-21T00:30:03.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39396 fp=#0x10000003f8e' 2024-09-21T00:30:03.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39428 fp=#0x10000003f07' 2024-09-21T00:30:03.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39492 fp=#0x30000001f2d' 2024-09-21T00:30:03.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39524 fp=#0x30000001f06' 2024-09-21T00:30:03.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39556 fp=#0x30000001f50' 2024-09-21T00:30:03.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39588 fp=#0x30000001f03' 2024-09-21T00:30:03.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39620 fp=#0x10000002aaf' 2024-09-21T00:30:03.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39684 fp=#0x300000019b0' 2024-09-21T00:30:03.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39716 fp=#0x30000000e70' 2024-09-21T00:30:03.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39748 fp=#0x30000000e99' 2024-09-21T00:30:03.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39780 fp=#0x30000000e92' 2024-09-21T00:30:03.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39844 fp=#0x300000019b4' 2024-09-21T00:30:03.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39876 fp=#0x30000000eb4' 2024-09-21T00:30:03.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39908 fp=#0x3000000111b' 2024-09-21T00:30:03.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39972 fp=#0x300000010fa' 2024-09-21T00:30:03.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40004 fp=#0x30000000ff9' 2024-09-21T00:30:03.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40068 fp=#0x30000001524' 2024-09-21T00:30:03.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40100 fp=#0x30000001664' 2024-09-21T00:30:03.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40132 fp=#0x300000014cc' 2024-09-21T00:30:03.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40164 fp=#0x3000000184c' 2024-09-21T00:30:03.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40196 fp=#0x10000001d74' 2024-09-21T00:30:03.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40228 fp=#0x300000009ed' 2024-09-21T00:30:03.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40292 fp=#0x10000001d47' 2024-09-21T00:30:03.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40356 fp=#0x10000004cc9' 2024-09-21T00:30:03.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40388 fp=#0x10000004d16' 2024-09-21T00:30:03.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40420 fp=#0x10000004e10' 2024-09-21T00:30:03.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40452 fp=#0x10000004e23' 2024-09-21T00:30:03.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40484 fp=#0x2000000126a' 2024-09-21T00:30:03.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40548 fp=#0x10000001036' 2024-09-21T00:30:03.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40580 fp=#0x10000000d56' 2024-09-21T00:30:03.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40612 fp=#0x2000000054c' 2024-09-21T00:30:03.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40644 fp=#0x10000000ef5' 2024-09-21T00:30:03.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40676 fp=#0x10000000f0c' 2024-09-21T00:30:03.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40708 fp=#0x2000000045b' 2024-09-21T00:30:03.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40740 fp=#0x10000000cbc' 2024-09-21T00:30:03.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40772 fp=#0x3000000143d' 2024-09-21T00:30:03.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40804 fp=#0x30000001455' 2024-09-21T00:30:03.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40932 fp=#0x10000004ac7' 2024-09-21T00:30:03.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40964 fp=#0x10000004ae7' 2024-09-21T00:30:03.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41028 fp=#0x100000036a2' 2024-09-21T00:30:03.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41060 fp=#0x10000003492' 2024-09-21T00:30:03.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41124 fp=#0x20000000c13' 2024-09-21T00:30:03.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41156 fp=#0x20000000c02' 2024-09-21T00:30:03.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41188 fp=#0x10000000d21' 2024-09-21T00:30:03.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41220 fp=#0x20000000212' 2024-09-21T00:30:03.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41252 fp=#0x2000000025c' 2024-09-21T00:30:03.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41284 fp=#0x10000000e3d' 2024-09-21T00:30:03.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41316 fp=#0x10000000eb8' 2024-09-21T00:30:03.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41412 fp=#0x100000009f7' 2024-09-21T00:30:03.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41444 fp=#0x10000000979' 2024-09-21T00:30:03.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41476 fp=#0x10000000bb3' 2024-09-21T00:30:03.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41508 fp=#0x10000000c05' 2024-09-21T00:30:03.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41540 fp=#0x10000000c1b' 2024-09-21T00:30:03.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41604 fp=#0x3000000185f' 2024-09-21T00:30:03.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41668 fp=#0x30000001902' 2024-09-21T00:30:03.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41700 fp=#0x30000000903' 2024-09-21T00:30:03.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41796 fp=#0x3000000089b' 2024-09-21T00:30:03.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41956 fp=#0x300000008b3' 2024-09-21T00:30:03.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41988 fp=#0x300000009af' 2024-09-21T00:30:03.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42020 fp=#0x3000000096d' 2024-09-21T00:30:03.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42052 fp=#0x30000001d7f' 2024-09-21T00:30:03.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42084 fp=#0x30000000978' 2024-09-21T00:30:03.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42116 fp=#0x3000000084e' 2024-09-21T00:30:03.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42180 fp=#0x30000000bd1' 2024-09-21T00:30:03.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42212 fp=#0x30000001bc4' 2024-09-21T00:30:03.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42244 fp=#0x3000000209c' 2024-09-21T00:30:03.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42276 fp=#0x30000000b64' 2024-09-21T00:30:03.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42308 fp=#0x30000000b71' 2024-09-21T00:30:03.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42372 fp=#0x30000000b37' 2024-09-21T00:30:03.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42404 fp=#0x30000000b03' 2024-09-21T00:30:03.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42436 fp=#0x30000000b32' 2024-09-21T00:30:03.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42468 fp=#0x30000000a68' 2024-09-21T00:30:03.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42500 fp=#0x30000000a7c' 2024-09-21T00:30:03.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42532 fp=#0x30000000a46' 2024-09-21T00:30:03.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42564 fp=#0x30000000a6a' 2024-09-21T00:30:03.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42660 fp=#0x10000001a3b' 2024-09-21T00:30:03.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42692 fp=#0x10000001a88' 2024-09-21T00:30:03.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42788 fp=#0x10000001a4f' 2024-09-21T00:30:03.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42820 fp=#0x100000019ab' 2024-09-21T00:30:03.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42852 fp=#0x10000001a02' 2024-09-21T00:30:03.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42884 fp=#0x100000019f8' 2024-09-21T00:30:03.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42948 fp=#0x30000000e51' 2024-09-21T00:30:03.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43044 fp=#0x30000001730' 2024-09-21T00:30:03.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43076 fp=#0x3000000176a' 2024-09-21T00:30:03.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43108 fp=#0x300000016a7' 2024-09-21T00:30:03.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43172 fp=#0x10000001880' 2024-09-21T00:30:03.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43268 fp=#0x30000000efc' 2024-09-21T00:30:03.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43300 fp=#0x30000000f44' 2024-09-21T00:30:03.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43332 fp=#0x30000000f47' 2024-09-21T00:30:03.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43364 fp=#0x30000000f7c' 2024-09-21T00:30:03.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43428 fp=#0x10000004166' 2024-09-21T00:30:03.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43460 fp=#0x3000000238d' 2024-09-21T00:30:03.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43492 fp=#0x300000023c7' 2024-09-21T00:30:03.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43524 fp=#0x300000023c1' 2024-09-21T00:30:03.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43556 fp=#0x300000023d7' 2024-09-21T00:30:03.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43588 fp=#0x30000000f9d' 2024-09-21T00:30:03.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43620 fp=#0x30000001014' 2024-09-21T00:30:03.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43652 fp=#0x30000001017' 2024-09-21T00:30:03.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43684 fp=#0x300000021a0' 2024-09-21T00:30:03.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43748 fp=#0x30000002125' 2024-09-21T00:30:03.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43780 fp=#0x30000002112' 2024-09-21T00:30:03.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43940 fp=#0x300000013b3' 2024-09-21T00:30:03.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31749 fp=#0x100000026a7' 2024-09-21T00:30:03.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31813 fp=#0x1000000326d' 2024-09-21T00:30:03.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:31941 fp=#0x30000001321' 2024-09-21T00:30:03.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34693 fp=#0x300000022b1' 2024-09-21T00:30:03.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34885 fp=#0x10000002706' 2024-09-21T00:30:03.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:34981 fp=#0x100000032e3' 2024-09-21T00:30:03.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35013 fp=#0x300000020be' 2024-09-21T00:30:03.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35045 fp=#0x100000032db' 2024-09-21T00:30:03.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35077 fp=#0x300000020bd' 2024-09-21T00:30:03.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35109 fp=#0x1000000395a' 2024-09-21T00:30:03.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35141 fp=#0x10000003ad1' 2024-09-21T00:30:03.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35173 fp=#0x100000041c3' 2024-09-21T00:30:03.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35205 fp=#0x100000038dd' 2024-09-21T00:30:03.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35237 fp=#0x10000003a11' 2024-09-21T00:30:03.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35269 fp=#0x10000003b13' 2024-09-21T00:30:03.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35301 fp=#0x100000041cf' 2024-09-21T00:30:03.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35333 fp=#0x10000003a27' 2024-09-21T00:30:03.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35365 fp=#0x100000038c2' 2024-09-21T00:30:03.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35397 fp=#0x1000000398d' 2024-09-21T00:30:03.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35429 fp=#0x10000003a07' 2024-09-21T00:30:03.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35461 fp=#0x300000011a8' 2024-09-21T00:30:03.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35493 fp=#0x30000001130' 2024-09-21T00:30:03.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35557 fp=#0x30000001fcd' 2024-09-21T00:30:03.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35589 fp=#0x3000000203c' 2024-09-21T00:30:03.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35621 fp=#0x30000001fd4' 2024-09-21T00:30:03.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35653 fp=#0x30000001ffb' 2024-09-21T00:30:03.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35685 fp=#0x30000001fff' 2024-09-21T00:30:03.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35813 fp=#0x100000033d3' 2024-09-21T00:30:03.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35877 fp=#0x100000033a4' 2024-09-21T00:30:03.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35909 fp=#0x10000003379' 2024-09-21T00:30:03.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35941 fp=#0x300000011c8' 2024-09-21T00:30:03.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:35973 fp=#0x300000011e0' 2024-09-21T00:30:03.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36133 fp=#0x10000002d55' 2024-09-21T00:30:03.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36165 fp=#0x1000000204f' 2024-09-21T00:30:03.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36197 fp=#0x10000001ed8' 2024-09-21T00:30:03.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36229 fp=#0x10000001f07' 2024-09-21T00:30:03.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36293 fp=#0x300000015ff' 2024-09-21T00:30:03.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36453 fp=#0x100000020aa' 2024-09-21T00:30:03.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36485 fp=#0x10000001da9' 2024-09-21T00:30:03.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36517 fp=#0x10000001de3' 2024-09-21T00:30:03.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36549 fp=#0x10000001def' 2024-09-21T00:30:03.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36581 fp=#0x10000004470' 2024-09-21T00:30:03.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36613 fp=#0x100000037c3' 2024-09-21T00:30:03.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36645 fp=#0x100000044ad' 2024-09-21T00:30:03.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36677 fp=#0x1000000447c' 2024-09-21T00:30:03.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36709 fp=#0x10000004457' 2024-09-21T00:30:03.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36741 fp=#0x100000047cd' 2024-09-21T00:30:03.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36773 fp=#0x1000000475a' 2024-09-21T00:30:03.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36805 fp=#0x100000046fe' 2024-09-21T00:30:03.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36837 fp=#0x10000004707' 2024-09-21T00:30:03.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36869 fp=#0x10000002555' 2024-09-21T00:30:03.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36901 fp=#0x100000025f6' 2024-09-21T00:30:03.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36933 fp=#0x100000035b6' 2024-09-21T00:30:03.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36965 fp=#0x10000004d78' 2024-09-21T00:30:03.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:36997 fp=#0x100000035ec' 2024-09-21T00:30:03.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37029 fp=#0x100000025e9' 2024-09-21T00:30:03.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37125 fp=#0x3000000178b' 2024-09-21T00:30:03.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37189 fp=#0x10000001362' 2024-09-21T00:30:03.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37253 fp=#0x10000003110' 2024-09-21T00:30:03.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37285 fp=#0x10000002fdd' 2024-09-21T00:30:03.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37317 fp=#0x30000000c12' 2024-09-21T00:30:03.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37349 fp=#0x30000001b0d' 2024-09-21T00:30:03.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37381 fp=#0x30000000c27' 2024-09-21T00:30:03.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37413 fp=#0x30000001ac7' 2024-09-21T00:30:03.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37445 fp=#0x30000001aff' 2024-09-21T00:30:03.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37477 fp=#0x300000010e2' 2024-09-21T00:30:03.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37733 fp=#0x30000000c48' 2024-09-21T00:30:03.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37797 fp=#0x30000000c74' 2024-09-21T00:30:03.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37861 fp=#0x300000012d1' 2024-09-21T00:30:03.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37957 fp=#0x30000000dd0' 2024-09-21T00:30:03.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:37989 fp=#0x30000000da5' 2024-09-21T00:30:03.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38053 fp=#0x30000000cd8' 2024-09-21T00:30:03.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38085 fp=#0x10000000778' 2024-09-21T00:30:03.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38117 fp=#0x10000000762' 2024-09-21T00:30:03.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38149 fp=#0x1000000075d' 2024-09-21T00:30:03.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38181 fp=#0x10000000e64' 2024-09-21T00:30:03.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38213 fp=#0x10000001bf9' 2024-09-21T00:30:03.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38245 fp=#0x10000000758' 2024-09-21T00:30:03.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38309 fp=#0x30000001a56' 2024-09-21T00:30:03.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38341 fp=#0x10000002458' 2024-09-21T00:30:03.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38373 fp=#0x30000001a72' 2024-09-21T00:30:03.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38405 fp=#0x100000032cb' 2024-09-21T00:30:03.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38437 fp=#0x30000001cdb' 2024-09-21T00:30:03.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38469 fp=#0x1000000363b' 2024-09-21T00:30:03.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38501 fp=#0x10000002169' 2024-09-21T00:30:03.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38597 fp=#0x30000001bb5' 2024-09-21T00:30:03.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38661 fp=#0x30000001c3a' 2024-09-21T00:30:03.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38693 fp=#0x30000001bd0' 2024-09-21T00:30:03.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38725 fp=#0x1000000059c' 2024-09-21T00:30:03.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38757 fp=#0x30000001bc2' 2024-09-21T00:30:03.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38789 fp=#0x1000000059a' 2024-09-21T00:30:03.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38821 fp=#0x300000019e8' 2024-09-21T00:30:03.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38885 fp=#0x10000002922' 2024-09-21T00:30:03.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38917 fp=#0x300000019c9' 2024-09-21T00:30:03.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38949 fp=#0x30000001c5e' 2024-09-21T00:30:03.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:38981 fp=#0x30000001c4e' 2024-09-21T00:30:03.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39045 fp=#0x30000001d60' 2024-09-21T00:30:03.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39077 fp=#0x30000001d21' 2024-09-21T00:30:03.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39269 fp=#0x10000001353' 2024-09-21T00:30:03.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39301 fp=#0x1000000359a' 2024-09-21T00:30:03.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39365 fp=#0x10000003fc1' 2024-09-21T00:30:03.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39397 fp=#0x10000003f2c' 2024-09-21T00:30:03.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39429 fp=#0x10000003f77' 2024-09-21T00:30:03.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39461 fp=#0x30000001fb3' 2024-09-21T00:30:03.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39493 fp=#0x30000001f24' 2024-09-21T00:30:03.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39525 fp=#0x30000001f5e' 2024-09-21T00:30:03.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39557 fp=#0x30000001fa6' 2024-09-21T00:30:03.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39589 fp=#0x30000001f74' 2024-09-21T00:30:03.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39653 fp=#0x10000002abc' 2024-09-21T00:30:03.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39685 fp=#0x30000000ee4' 2024-09-21T00:30:03.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39717 fp=#0x3000000197e' 2024-09-21T00:30:03.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39781 fp=#0x300000019ac' 2024-09-21T00:30:03.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39877 fp=#0x300000019b2' 2024-09-21T00:30:03.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39909 fp=#0x3000000113c' 2024-09-21T00:30:03.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:39973 fp=#0x3000000100a' 2024-09-21T00:30:03.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40037 fp=#0x3000000154f' 2024-09-21T00:30:03.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40069 fp=#0x30000001811' 2024-09-21T00:30:03.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40101 fp=#0x30000001824' 2024-09-21T00:30:03.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40165 fp=#0x3000000183c' 2024-09-21T00:30:03.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40197 fp=#0x10000001d08' 2024-09-21T00:30:03.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40229 fp=#0x30000000d67' 2024-09-21T00:30:03.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40293 fp=#0x30000000d7a' 2024-09-21T00:30:03.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40325 fp=#0x300000009e5' 2024-09-21T00:30:03.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40357 fp=#0x10000004e76' 2024-09-21T00:30:03.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40389 fp=#0x10000004e55' 2024-09-21T00:30:03.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40453 fp=#0x10000004d0a' 2024-09-21T00:30:03.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40485 fp=#0x10000004e4b' 2024-09-21T00:30:03.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40549 fp=#0x10000000ce0' 2024-09-21T00:30:03.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40581 fp=#0x10000000f09' 2024-09-21T00:30:03.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40645 fp=#0x20000000452' 2024-09-21T00:30:03.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40677 fp=#0x20000000556' 2024-09-21T00:30:03.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40709 fp=#0x2000000055b' 2024-09-21T00:30:03.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40741 fp=#0x10000000cc8' 2024-09-21T00:30:03.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40773 fp=#0x20000000f81' 2024-09-21T00:30:03.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40805 fp=#0x3000000144c' 2024-09-21T00:30:03.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:40965 fp=#0x10000004951' 2024-09-21T00:30:03.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41029 fp=#0x1000000367a' 2024-09-21T00:30:03.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41061 fp=#0x100000034cd' 2024-09-21T00:30:03.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41093 fp=#0x10000003655' 2024-09-21T00:30:03.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41125 fp=#0x10000000e7d' 2024-09-21T00:30:03.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41157 fp=#0x20000000bfd' 2024-09-21T00:30:03.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41189 fp=#0x20000000c05' 2024-09-21T00:30:03.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41221 fp=#0x20000000b0b' 2024-09-21T00:30:03.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41253 fp=#0x10000000e6c' 2024-09-21T00:30:03.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41285 fp=#0x20000000473' 2024-09-21T00:30:03.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41317 fp=#0x10000000e39' 2024-09-21T00:30:03.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41381 fp=#0x10000000c54' 2024-09-21T00:30:03.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41413 fp=#0x10000000c0a' 2024-09-21T00:30:03.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41445 fp=#0x10000000e87' 2024-09-21T00:30:03.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41477 fp=#0x10000000b98' 2024-09-21T00:30:03.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41509 fp=#0x10000000a1c' 2024-09-21T00:30:03.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41541 fp=#0x10000000960' 2024-09-21T00:30:03.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41573 fp=#0x30000001912' 2024-09-21T00:30:03.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41733 fp=#0x3000000092e' 2024-09-21T00:30:03.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41797 fp=#0x300000009d9' 2024-09-21T00:30:03.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41829 fp=#0x300000008d8' 2024-09-21T00:30:03.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41925 fp=#0x300000009c7' 2024-09-21T00:30:03.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:41989 fp=#0x300000009c9' 2024-09-21T00:30:03.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42021 fp=#0x300000009b2' 2024-09-21T00:30:03.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42053 fp=#0x30000000823' 2024-09-21T00:30:03.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42085 fp=#0x3000000097e' 2024-09-21T00:30:03.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42117 fp=#0x30000001d3d' 2024-09-21T00:30:03.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42149 fp=#0x30000001d36' 2024-09-21T00:30:03.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42181 fp=#0x30000001ba4' 2024-09-21T00:30:03.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42213 fp=#0x30000000b70' 2024-09-21T00:30:03.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42245 fp=#0x30000001b89' 2024-09-21T00:30:03.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42277 fp=#0x30000000b57' 2024-09-21T00:30:03.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42309 fp=#0x30000000b7f' 2024-09-21T00:30:03.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42341 fp=#0x30000000b4a' 2024-09-21T00:30:03.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42373 fp=#0x30000000ae3' 2024-09-21T00:30:03.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42405 fp=#0x30000000afe' 2024-09-21T00:30:03.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42437 fp=#0x30000000b1e' 2024-09-21T00:30:03.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42469 fp=#0x30000000a71' 2024-09-21T00:30:03.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42501 fp=#0x30000000a47' 2024-09-21T00:30:03.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42533 fp=#0x30000001b46' 2024-09-21T00:30:03.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42565 fp=#0x30000000a4b' 2024-09-21T00:30:03.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42757 fp=#0x100000018fb' 2024-09-21T00:30:03.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42821 fp=#0x100000018ee' 2024-09-21T00:30:03.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42885 fp=#0x100000019c9' 2024-09-21T00:30:03.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:42917 fp=#0x30000000e4f' 2024-09-21T00:30:03.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43077 fp=#0x30000001702' 2024-09-21T00:30:03.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43109 fp=#0x30000001748' 2024-09-21T00:30:03.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43141 fp=#0x10000001858' 2024-09-21T00:30:03.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43205 fp=#0x100000018bb' 2024-09-21T00:30:03.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43269 fp=#0x30000000f35' 2024-09-21T00:30:03.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43301 fp=#0x30000000f72' 2024-09-21T00:30:03.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43333 fp=#0x30000000f0d' 2024-09-21T00:30:03.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43365 fp=#0x30000000f50' 2024-09-21T00:30:03.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43397 fp=#0x30000000f46' 2024-09-21T00:30:03.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43461 fp=#0x300000023f7' 2024-09-21T00:30:03.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43493 fp=#0x300000023e0' 2024-09-21T00:30:03.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43525 fp=#0x3000000238b' 2024-09-21T00:30:03.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43557 fp=#0x30000002379' 2024-09-21T00:30:03.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43589 fp=#0x30000000fe3' 2024-09-21T00:30:03.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43621 fp=#0x300000010cf' 2024-09-21T00:30:03.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43653 fp=#0x30000002194' 2024-09-21T00:30:03.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43685 fp=#0x30000001108' 2024-09-21T00:30:03.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43717 fp=#0x30000000fd6' 2024-09-21T00:30:03.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43749 fp=#0x30000002104' 2024-09-21T00:30:03.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43845 fp=#0x1000000399c' 2024-09-21T00:30:03.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43909 fp=#0x10000003a4f' 2024-09-21T00:30:03.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for 47099ce0-1: 'internal op quiesce_inode:mds.2:43941 fp=#0x30000001351' 2024-09-21T00:30:03.715 INFO:tasks.quiescer.fs.[cephfs]:Pulled 12227 ops from rank 2 (i) into home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310/ops-47099ce0-1-mds.i.json 2024-09-21T00:30:03.715 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -- /var/run/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ops-47099ce0-1-mds.i.json 2024-09-21T00:30:03.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:29:59 smithi027 ceph-mon[30745]: pgmap v642: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 6 op/s 2024-09-21T00:30:03.717 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:29:59 smithi089 ceph-mon[34481]: pgmap v642: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 6 op/s 2024-09-21T00:30:03.717 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:00 smithi089 ceph-mon[34481]: overall HEALTH_OK 2024-09-21T00:30:03.717 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:01 smithi089 ceph-mon[34481]: pgmap v643: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-09-21T00:30:03.717 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:29:59 smithi098 ceph-mon[34725]: pgmap v642: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 6 op/s 2024-09-21T00:30:03.717 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:00 smithi098 ceph-mon[34725]: overall HEALTH_OK 2024-09-21T00:30:03.717 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:01 smithi098 ceph-mon[34725]: pgmap v643: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-09-21T00:30:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 -- 172.21.15.27:0/818133337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f58104740 msgr2=0x7f8f58104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 --2- 172.21.15.27:0/818133337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f58104740 0x7f8f58104b50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009900 tx=0x7f8f4c02a3f0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 -- 172.21.15.27:0/818133337 shutdown_connections 2024-09-21T00:30:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 --2- 172.21.15.27:0/818133337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f5810dba0 0x7f8f58110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 --2- 172.21.15.27:0/818133337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f58105090 0x7f8f5810d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 --2- 172.21.15.27:0/818133337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f58104740 0x7f8f58104b50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 -- 172.21.15.27:0/818133337 >> 172.21.15.27:0/818133337 conn(0x7f8f580fe210 msgr2=0x7f8f58100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 -- 172.21.15.27:0/818133337 shutdown_connections 2024-09-21T00:30:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 -- 172.21.15.27:0/818133337 wait complete. 2024-09-21T00:30:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 Processor -- start 2024-09-21T00:30:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 -- start start 2024-09-21T00:30:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f58104740 0x7f8f5819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f58105090 0x7f8f5819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f5810dba0 0x7f8f5819fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8f58113870 con 0x7f8f58105090 2024-09-21T00:30:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581136f0 con 0x7f8f5810dba0 2024-09-21T00:30:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581139f0 con 0x7f8f58104740 2024-09-21T00:30:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f57e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f58105090 0x7f8f5819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f57e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f58105090 0x7f8f5819a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48422/0 (socket says 172.21.15.27:48422) 2024-09-21T00:30:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f57e00640 1 -- 172.21.15.27:0/2413574466 learned_addr learned my addr 172.21.15.27:0/2413574466 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5cc00640 1 --2- 172.21.15.27:0/2413574466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f58104740 0x7f8f5819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5d600640 1 --2- 172.21.15.27:0/2413574466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f5810dba0 0x7f8f5819fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f57e00640 1 -- 172.21.15.27:0/2413574466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f58104740 msgr2=0x7f8f5819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f57e00640 1 --2- 172.21.15.27:0/2413574466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f58104740 0x7f8f5819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f57e00640 1 -- 172.21.15.27:0/2413574466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f5810dba0 msgr2=0x7f8f5819fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f57e00640 1 --2- 172.21.15.27:0/2413574466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f5810dba0 0x7f8f5819fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f57e00640 1 -- 172.21.15.27:0/2413574466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f581a01c0 con 0x7f8f58105090 2024-09-21T00:30:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f57e00640 1 --2- 172.21.15.27:0/2413574466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f58105090 0x7f8f5819a830 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400e970 tx=0x7f8f4400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f55600640 1 -- 172.21.15.27:0/2413574466 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4400cd80 con 0x7f8f58105090 2024-09-21T00:30:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 -- 172.21.15.27:0/2413574466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f581a04b0 con 0x7f8f58105090 2024-09-21T00:30:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f55600640 1 -- 172.21.15.27:0/2413574466 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f440035f0 con 0x7f8f58105090 2024-09-21T00:30:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f55600640 1 -- 172.21.15.27:0/2413574466 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44010690 con 0x7f8f58105090 2024-09-21T00:30:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.616+0000 7f8f5e000640 1 -- 172.21.15.27:0/2413574466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f581a6c40 con 0x7f8f58105090 2024-09-21T00:30:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.621+0000 7f8f55600640 1 -- 172.21.15.27:0/2413574466 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8f44003aa0 con 0x7f8f58105090 2024-09-21T00:30:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.621+0000 7f8f6663a740 1 Processor -- start 2024-09-21T00:30:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.621+0000 7f8f6663a740 1 -- start start 2024-09-21T00:30:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.621+0000 7f8f6663a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 0x55db28e07e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.621+0000 7f8f6663a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a3a2f0 0x55db28e08350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.621+0000 7f8f6663a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a424d0 0x55db28e08890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.621+0000 7f8f6663a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55db28d7d6b0 con 0x55db28a3a2f0 2024-09-21T00:30:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.621+0000 7f8f6663a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55db28d6cae0 con 0x55db28a424d0 2024-09-21T00:30:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.621+0000 7f8f6663a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55db28d708f0 con 0x55db28a631b0 2024-09-21T00:30:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.621+0000 7f8f5d600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 0x55db28e07e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.621+0000 7f8f5d600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 0x55db28e07e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41742/0 (socket says 172.21.15.27:41742) 2024-09-21T00:30:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.621+0000 7f8f5d600640 1 -- 172.21.15.27:0/1069073297 learned_addr learned my addr 172.21.15.27:0/1069073297 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.621+0000 7f8f5cc00640 1 --2- 172.21.15.27:0/1069073297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a3a2f0 0x55db28e08350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.621+0000 7f8f57e00640 1 --2- 172.21.15.27:0/1069073297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a424d0 0x55db28e08890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5d600640 1 -- 172.21.15.27:0/1069073297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a424d0 msgr2=0x55db28e08890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5d600640 1 --2- 172.21.15.27:0/1069073297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a424d0 0x55db28e08890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5d600640 1 -- 172.21.15.27:0/1069073297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a3a2f0 msgr2=0x55db28e08350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5d600640 1 --2- 172.21.15.27:0/1069073297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a3a2f0 0x55db28e08350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5d600640 1 -- 172.21.15.27:0/1069073297 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db28d39780 con 0x55db28a631b0 2024-09-21T00:30:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5d600640 1 --2- 172.21.15.27:0/1069073297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 0x55db28e07e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f50011440 tx=0x7f8f50011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=616d5aad1f046930 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f49c00640 1 -- 172.21.15.27:0/1069073297 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50004d20 con 0x55db28a631b0 2024-09-21T00:30:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f49c00640 1 -- 172.21.15.27:0/1069073297 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f50004050 con 0x55db28a631b0 2024-09-21T00:30:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f49c00640 1 -- 172.21.15.27:0/1069073297 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5000f760 con 0x55db28a631b0 2024-09-21T00:30:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f55600640 1 --2- 172.21.15.27:0/2413574466 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8f400b6670 0x7f8f400b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f55600640 1 -- 172.21.15.27:0/2413574466 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f8f440e7dd0 con 0x7f8f58105090 2024-09-21T00:30:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5cc00640 1 --2- 172.21.15.27:0/2413574466 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8f400b6670 0x7f8f400b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 -- 172.21.15.27:0/1069073297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 msgr2=0x55db28e07e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 --2- 172.21.15.27:0/1069073297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 0x55db28e07e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f50011440 tx=0x7f8f50011900 comp rx=0 tx=0).stop 2024-09-21T00:30:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 -- 172.21.15.27:0/1069073297 shutdown_connections 2024-09-21T00:30:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 --2- 172.21.15.27:0/1069073297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a424d0 0x55db28e08890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 --2- 172.21.15.27:0/1069073297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a3a2f0 0x55db28e08350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 --2- 172.21.15.27:0/1069073297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 0x55db28e07e10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 -- 172.21.15.27:0/1069073297 >> 172.21.15.27:0/1069073297 conn(0x55db28a57300 msgr2=0x55db28e05d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 -- 172.21.15.27:0/1069073297 shutdown_connections 2024-09-21T00:30:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 -- 172.21.15.27:0/1069073297 wait complete. 2024-09-21T00:30:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5cc00640 1 --2- 172.21.15.27:0/2413574466 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8f400b6670 0x7f8f400b8b80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c005ab0 tx=0x7f8f4c005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 -- start start 2024-09-21T00:30:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 Processor -- start 2024-09-21T00:30:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a3a2f0 0x55db28e17770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a424d0 0x55db28e17cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 0x55db28e181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55db28ca4ab0 con 0x55db28a3a2f0 2024-09-21T00:30:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55db28d13570 con 0x55db28a424d0 2024-09-21T00:30:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55db28e1a970 con 0x55db28a631b0 2024-09-21T00:30:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f6663a740 10 client.0 _ll_register_callbacks cb 0x55db28d22100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a3a2f0 0x55db28e17770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a3a2f0 0x55db28e17770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48442/0 (socket says 172.21.15.27:48442) 2024-09-21T00:30:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5cc00640 1 -- 172.21.15.27:0/3737811670 learned_addr learned my addr 172.21.15.27:0/3737811670 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5d600640 1 --2- 172.21.15.27:0/3737811670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 0x55db28e181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f57e00640 1 --2- 172.21.15.27:0/3737811670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a424d0 0x55db28e17cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5d600640 1 -- 172.21.15.27:0/3737811670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a424d0 msgr2=0x55db28e17cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5d600640 1 --2- 172.21.15.27:0/3737811670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a424d0 0x55db28e17cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5d600640 1 -- 172.21.15.27:0/3737811670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a3a2f0 msgr2=0x55db28e17770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5d600640 1 --2- 172.21.15.27:0/3737811670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a3a2f0 0x55db28e17770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.626+0000 7f8f5d600640 1 -- 172.21.15.27:0/3737811670 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db28ace790 con 0x55db28a631b0 2024-09-21T00:30:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f5cc00640 1 --2- 172.21.15.27:0/3737811670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a3a2f0 0x55db28e17770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:30:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f5d600640 1 --2- 172.21.15.27:0/3737811670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 0x55db28e181f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f5001b040 tx=0x7f8f50004200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=641a6f95f6bfb249 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f3e000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:30:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f3e000640 1 -- 172.21.15.27:0/3737811670 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5001d040 con 0x55db28a631b0 2024-09-21T00:30:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 1 -- 172.21.15.27:0/3737811670 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f50004050 con 0x55db28a631b0 2024-09-21T00:30:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f3e000640 1 -- 172.21.15.27:0/3737811670 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f8f50004a00 con 0x55db28a631b0 2024-09-21T00:30:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f3e000640 1 -- 172.21.15.27:0/3737811670 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8f50031050 con 0x55db28a631b0 2024-09-21T00:30:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 10 client.26332 fetch_fsmap learned FSMap version 38 2024-09-21T00:30:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 1 -- 172.21.15.27:0/3737811670 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55db28b47620 con 0x55db28a631b0 2024-09-21T00:30:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f3e000640 1 -- 172.21.15.27:0/3737811670 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f50004050 con 0x55db28a631b0 2024-09-21T00:30:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f3e000640 1 -- 172.21.15.27:0/3737811670 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f8f50020070 con 0x55db28a631b0 2024-09-21T00:30:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 10 client.26332 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:30:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 10 client.26332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:30:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 20 client.26332 populate_metadata read hostname 'smithi027' 2024-09-21T00:30:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 1 --2- 172.21.15.27:0/3737811670 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55db28a21d80 0x55db28e297e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 4 client.26332 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:30:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 1 -- 172.21.15.27:0/3737811670 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db28d7f600 con 0x55db28a21d80 2024-09-21T00:30:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f5cc00640 1 --2- 172.21.15.27:0/3737811670 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55db28a21d80 0x55db28e297e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f3e000640 10 client.26332 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f5cc00640 1 --2- 172.21.15.27:0/3737811670 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55db28a21d80 0x55db28e297e0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b304f06f48defd server_cookie=c38e71156b0dc826 in_seq=0 out_seq=0 2024-09-21T00:30:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f3e000640 1 -- 172.21.15.27:0/3737811670 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55db28d7f600 con 0x55db28a21d80 2024-09-21T00:30:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f3e000640 10 client.26332 handle_command_reply: tid=0 2024-09-21T00:30:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 1 client.26332 shutdown 2024-09-21T00:30:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 1 -- 172.21.15.27:0/3737811670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 msgr2=0x55db28e181f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 1 --2- 172.21.15.27:0/3737811670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 0x55db28e181f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f5001b040 tx=0x7f8f50004200 comp rx=0 tx=0).stop 2024-09-21T00:30:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 1 -- 172.21.15.27:0/3737811670 shutdown_connections 2024-09-21T00:30:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 1 --2- 172.21.15.27:0/3737811670 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55db28a21d80 0x55db28e297e0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 1 --2- 172.21.15.27:0/3737811670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 0x55db28e181f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 1 --2- 172.21.15.27:0/3737811670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a424d0 0x55db28e17cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f3e000640 0 client.26332 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 1 --2- 172.21.15.27:0/3737811670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a3a2f0 0x55db28e17770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 1 -- 172.21.15.27:0/3737811670 >> 172.21.15.27:0/3737811670 conn(0x55db28a57300 msgr2=0x55db28e0afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 1 -- 172.21.15.27:0/3737811670 shutdown_connections 2024-09-21T00:30:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 1 -- 172.21.15.27:0/3737811670 wait complete. 2024-09-21T00:30:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.631+0000 7f8f6663a740 20 client.26332 trim_cache size 0 max 16384 2024-09-21T00:30:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 Processor -- start 2024-09-21T00:30:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 -- start start 2024-09-21T00:30:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a3a2f0 0x55db28e12c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a424d0 0x55db28e14940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 0x55db28e14e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581b14a0 con 0x55db28a424d0 2024-09-21T00:30:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581aed10 con 0x55db28a3a2f0 2024-09-21T00:30:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581acb80 con 0x55db28a631b0 2024-09-21T00:30:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f57e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a424d0 0x55db28e14940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f57e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a424d0 0x55db28e14940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48446/0 (socket says 172.21.15.27:48446) 2024-09-21T00:30:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f57e00640 1 -- 172.21.15.27:0/1510743088 learned_addr learned my addr 172.21.15.27:0/1510743088 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f5cc00640 1 --2- 172.21.15.27:0/1510743088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a3a2f0 0x55db28e12c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f5d600640 1 --2- 172.21.15.27:0/1510743088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 0x55db28e14e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f57e00640 1 -- 172.21.15.27:0/1510743088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 msgr2=0x55db28e14e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f57e00640 1 --2- 172.21.15.27:0/1510743088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 0x55db28e14e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f57e00640 1 -- 172.21.15.27:0/1510743088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a3a2f0 msgr2=0x55db28e12c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f57e00640 1 --2- 172.21.15.27:0/1510743088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a3a2f0 0x55db28e12c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f57e00640 1 -- 172.21.15.27:0/1510743088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db28e15c50 con 0x55db28a424d0 2024-09-21T00:30:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f57e00640 1 --2- 172.21.15.27:0/1510743088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a424d0 0x55db28e14940 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400c6a0 tx=0x7f8f44003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c255158b36a1b301 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f49200640 1 -- 172.21.15.27:0/1510743088 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f440ad300 con 0x55db28a424d0 2024-09-21T00:30:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f49200640 1 -- 172.21.15.27:0/1510743088 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f440adef0 con 0x55db28a424d0 2024-09-21T00:30:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f49200640 1 -- 172.21.15.27:0/1510743088 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f440a93a0 con 0x55db28a424d0 2024-09-21T00:30:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 -- 172.21.15.27:0/1510743088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a424d0 msgr2=0x55db28e14940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 --2- 172.21.15.27:0/1510743088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a424d0 0x55db28e14940 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400c6a0 tx=0x7f8f44003a10 comp rx=0 tx=0).stop 2024-09-21T00:30:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 -- 172.21.15.27:0/1510743088 shutdown_connections 2024-09-21T00:30:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 --2- 172.21.15.27:0/1510743088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a631b0 0x55db28e14e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 --2- 172.21.15.27:0/1510743088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a424d0 0x55db28e14940 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 --2- 172.21.15.27:0/1510743088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a3a2f0 0x55db28e12c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 -- 172.21.15.27:0/1510743088 >> 172.21.15.27:0/1510743088 conn(0x55db28a57300 msgr2=0x55db28e12550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 -- 172.21.15.27:0/1510743088 shutdown_connections 2024-09-21T00:30:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 -- 172.21.15.27:0/1510743088 wait complete. 2024-09-21T00:30:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 -- start start 2024-09-21T00:30:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 Processor -- start 2024-09-21T00:30:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a3a2f0 0x55db28e1ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a424d0 0x55db28e1d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a631b0 0x55db28e8c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581ad1d0 con 0x55db28a631b0 2024-09-21T00:30:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581ae640 con 0x55db28a3a2f0 2024-09-21T00:30:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581ad8a0 con 0x55db28a424d0 2024-09-21T00:30:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f6663a740 10 client.0 _ll_register_callbacks cb 0x7f8f500093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f5cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a3a2f0 0x55db28e1ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f5d600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a631b0 0x55db28e8c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f5cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a3a2f0 0x55db28e1ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:46226/0 (socket says 172.21.15.27:46226) 2024-09-21T00:30:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f5cc00640 1 -- 172.21.15.27:0/3163076311 learned_addr learned my addr 172.21.15.27:0/3163076311 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f5cc00640 1 -- 172.21.15.27:0/3163076311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a424d0 msgr2=0x55db28e1d010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:30:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f5cc00640 1 --2- 172.21.15.27:0/3163076311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a424d0 0x55db28e1d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f5cc00640 1 -- 172.21.15.27:0/3163076311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a631b0 msgr2=0x55db28e8c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f5cc00640 1 --2- 172.21.15.27:0/3163076311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a631b0 0x55db28e8c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.651+0000 7f8f5cc00640 1 -- 172.21.15.27:0/3163076311 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db28e8c670 con 0x55db28a3a2f0 2024-09-21T00:30:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f5d600640 1 --2- 172.21.15.27:0/3163076311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a631b0 0x55db28e8c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:30:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f5cc00640 1 --2- 172.21.15.27:0/3163076311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a3a2f0 0x55db28e1ca50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c02a9e0 tx=0x7f8f4c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8521548f89ea3415 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f3e000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:30:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f3e000640 1 -- 172.21.15.27:0/3163076311 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c060060 con 0x55db28a3a2f0 2024-09-21T00:30:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f6663a740 1 -- 172.21.15.27:0/3163076311 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f440adef0 con 0x55db28a3a2f0 2024-09-21T00:30:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f3e000640 1 -- 172.21.15.27:0/3163076311 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9503+0+0 (secure 0 0 0) 0x7f8f4c0035f0 con 0x55db28a3a2f0 2024-09-21T00:30:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f3e000640 1 -- 172.21.15.27:0/3163076311 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8f4c067050 con 0x55db28a3a2f0 2024-09-21T00:30:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f6663a740 10 client.26330 fetch_fsmap learned FSMap version 38 2024-09-21T00:30:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f6663a740 1 -- 172.21.15.27:0/3163076311 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55db28e93a00 con 0x55db28a3a2f0 2024-09-21T00:30:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f3e000640 1 -- 172.21.15.27:0/3163076311 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f440adef0 con 0x55db28a3a2f0 2024-09-21T00:30:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f3e000640 1 -- 172.21.15.27:0/3163076311 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f8f4c032710 con 0x55db28a3a2f0 2024-09-21T00:30:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f6663a740 10 client.26330 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:30:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f6663a740 10 client.26330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:30:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f6663a740 20 client.26330 populate_metadata read hostname 'smithi027' 2024-09-21T00:30:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f6663a740 1 --2- 172.21.15.27:0/3163076311 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55db289f71c0 0x55db28e961a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f6663a740 4 client.26330 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:30:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f6663a740 1 -- 172.21.15.27:0/3163076311 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db28d7b0b0 con 0x55db289f71c0 2024-09-21T00:30:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f57e00640 1 --2- 172.21.15.27:0/3163076311 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55db289f71c0 0x55db28e961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f57e00640 1 --2- 172.21.15.27:0/3163076311 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55db289f71c0 0x55db28e961a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6162b34aa8489f2c server_cookie=a514c9925b2ab0b9 in_seq=0 out_seq=0 2024-09-21T00:30:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.656+0000 7f8f3e000640 10 client.26330 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f3e000640 1 -- 172.21.15.27:0/3163076311 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db28d7b0b0 con 0x55db289f71c0 2024-09-21T00:30:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f3e000640 10 client.26330 handle_command_reply: tid=0 2024-09-21T00:30:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f6663a740 1 client.26330 shutdown 2024-09-21T00:30:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f6663a740 1 -- 172.21.15.27:0/3163076311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a3a2f0 msgr2=0x55db28e1ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f6663a740 1 --2- 172.21.15.27:0/3163076311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a3a2f0 0x55db28e1ca50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c02a9e0 tx=0x7f8f4c009260 comp rx=0 tx=0).stop 2024-09-21T00:30:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f6663a740 1 -- 172.21.15.27:0/3163076311 shutdown_connections 2024-09-21T00:30:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f6663a740 1 --2- 172.21.15.27:0/3163076311 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55db289f71c0 0x55db28e961a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f6663a740 1 --2- 172.21.15.27:0/3163076311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55db28a631b0 0x55db28e8c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f6663a740 1 --2- 172.21.15.27:0/3163076311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db28a424d0 0x55db28e1d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f6663a740 1 --2- 172.21.15.27:0/3163076311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55db28a3a2f0 0x55db28e1ca50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f3e000640 0 client.26330 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f6663a740 1 -- 172.21.15.27:0/3163076311 >> 172.21.15.27:0/3163076311 conn(0x55db28a57300 msgr2=0x55db28e04360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f6663a740 1 -- 172.21.15.27:0/3163076311 shutdown_connections 2024-09-21T00:30:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f6663a740 1 -- 172.21.15.27:0/3163076311 wait complete. 2024-09-21T00:30:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f6663a740 20 client.26330 trim_cache size 0 max 16384 2024-09-21T00:30:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:30:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:30:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:30:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:30:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f5e000640 1 -- 172.21.15.27:0/2413574466 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8f400b6670 msgr2=0x7f8f400b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f5e000640 1 --2- 172.21.15.27:0/2413574466 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8f400b6670 0x7f8f400b8b80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c005ab0 tx=0x7f8f4c005a20 comp rx=0 tx=0).stop 2024-09-21T00:30:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f5e000640 1 -- 172.21.15.27:0/2413574466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f58105090 msgr2=0x7f8f5819a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f5e000640 1 --2- 172.21.15.27:0/2413574466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f58105090 0x7f8f5819a830 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400e970 tx=0x7f8f4400ee30 comp rx=0 tx=0).stop 2024-09-21T00:30:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f5e000640 1 -- 172.21.15.27:0/2413574466 shutdown_connections 2024-09-21T00:30:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f5e000640 1 --2- 172.21.15.27:0/2413574466 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8f400b6670 0x7f8f400b8b80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f5e000640 1 --2- 172.21.15.27:0/2413574466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f5810dba0 0x7f8f5819fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f5e000640 1 --2- 172.21.15.27:0/2413574466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f58105090 0x7f8f5819a830 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f5e000640 1 --2- 172.21.15.27:0/2413574466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f58104740 0x7f8f5819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f5e000640 1 -- 172.21.15.27:0/2413574466 >> 172.21.15.27:0/2413574466 conn(0x7f8f580fe210 msgr2=0x7f8f581076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f5e000640 1 -- 172.21.15.27:0/2413574466 shutdown_connections 2024-09-21T00:30:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:00.675+0000 7f8f5e000640 1 -- 172.21.15.27:0/2413574466 wait complete. 2024-09-21T00:30:03.746 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:00 smithi027 ceph-mon[30745]: overall HEALTH_OK 2024-09-21T00:30:03.746 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:01 smithi027 ceph-mon[30745]: pgmap v643: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-09-21T00:30:03.750 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-09-21T00:30:03.786 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.SNkGsDFv7P 2024-09-21T00:30:03.786 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /var/run/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ops-47099ce0-1-mds.a.json /tmp/tmp.SNkGsDFv7P 2024-09-21T00:30:03.974 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:03 smithi027 ceph-mon[30745]: pgmap v644: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 60 KiB/s wr, 9 op/s 2024-09-21T00:30:03.977 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.SNkGsDFv7P 2024-09-21T00:30:03.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:03 smithi098 ceph-mon[34725]: pgmap v644: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 60 KiB/s wr, 9 op/s 2024-09-21T00:30:04.136 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.SNkGsDFv7P is 21MB 2024-09-21T00:30:04.234 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:03 smithi089 ceph-mon[34481]: pgmap v644: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 60 KiB/s wr, 9 op/s 2024-09-21T00:30:04.603 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.SNkGsDFv7P 2024-09-21T00:30:04.682 DEBUG:tasks.quiescer.fs.[cephfs]:read 21545261B of ops from '/var/run/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ops-47099ce0-1-mds.a.json' on mds.1 (a) 2024-09-21T00:30:05.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22726 fp=#0x10000000001' 2024-09-21T00:30:05.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22790 fp=#0x10000004aae' 2024-09-21T00:30:05.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22822 fp=#0x10000004f04' 2024-09-21T00:30:05.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26694 fp=#0x20000000a15' 2024-09-21T00:30:05.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26726 fp=#0x200000007f5' 2024-09-21T00:30:05.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26790 fp=#0x100000043e3' 2024-09-21T00:30:05.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26886 fp=#0x1000000461a' 2024-09-21T00:30:05.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26918 fp=#0x100000043ef' 2024-09-21T00:30:05.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26982 fp=#0x20000000dee' 2024-09-21T00:30:05.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27014 fp=#0x20000000ff5' 2024-09-21T00:30:05.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27046 fp=#0x20000000ed7' 2024-09-21T00:30:05.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27078 fp=#0x20000000d54' 2024-09-21T00:30:05.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27110 fp=#0x20000000ee0' 2024-09-21T00:30:05.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27142 fp=#0x10000002501' 2024-09-21T00:30:05.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27174 fp=#0x10000002223' 2024-09-21T00:30:05.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27206 fp=#0x1000000251d' 2024-09-21T00:30:05.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27238 fp=#0x20000000f42' 2024-09-21T00:30:05.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27270 fp=#0x1000000226c' 2024-09-21T00:30:05.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27302 fp=#0x10000002513' 2024-09-21T00:30:05.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27334 fp=#0x20000000f27' 2024-09-21T00:30:05.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27366 fp=#0x10000002283' 2024-09-21T00:30:05.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27398 fp=#0x10000003b80' 2024-09-21T00:30:05.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27430 fp=#0x10000003c53' 2024-09-21T00:30:05.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27462 fp=#0x10000003b5b' 2024-09-21T00:30:05.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27494 fp=#0x10000003b92' 2024-09-21T00:30:05.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27526 fp=#0x10000003cb3' 2024-09-21T00:30:05.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27654 fp=#0x10000002093' 2024-09-21T00:30:05.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27686 fp=#0x1000000209f' 2024-09-21T00:30:05.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27782 fp=#0x200000008c0' 2024-09-21T00:30:05.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27814 fp=#0x200000008fa' 2024-09-21T00:30:05.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27846 fp=#0x200000008bc' 2024-09-21T00:30:05.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27878 fp=#0x2000000096d' 2024-09-21T00:30:05.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27910 fp=#0x2000000112c' 2024-09-21T00:30:05.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27942 fp=#0x20000001095' 2024-09-21T00:30:05.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27974 fp=#0x100000049d9' 2024-09-21T00:30:05.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28006 fp=#0x100000049d5' 2024-09-21T00:30:05.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28038 fp=#0x100000049e0' 2024-09-21T00:30:05.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28070 fp=#0x2000000135b' 2024-09-21T00:30:05.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28102 fp=#0x200000012b4' 2024-09-21T00:30:05.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28166 fp=#0x10000004b6a' 2024-09-21T00:30:05.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28198 fp=#0x10000004bc5' 2024-09-21T00:30:05.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28230 fp=#0x10000004bda' 2024-09-21T00:30:05.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28262 fp=#0x20000000e8f' 2024-09-21T00:30:05.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28294 fp=#0x10000003091' 2024-09-21T00:30:05.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28326 fp=#0x10000003354' 2024-09-21T00:30:05.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28358 fp=#0x100000030ad' 2024-09-21T00:30:05.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28390 fp=#0x20000000e81' 2024-09-21T00:30:05.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28422 fp=#0x10000003154' 2024-09-21T00:30:05.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28454 fp=#0x10000004bd8' 2024-09-21T00:30:05.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28486 fp=#0x20000000f49' 2024-09-21T00:30:05.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28518 fp=#0x20000000f6e' 2024-09-21T00:30:05.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28550 fp=#0x20000000fad' 2024-09-21T00:30:05.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28582 fp=#0x20000001028' 2024-09-21T00:30:05.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28614 fp=#0x2000000102c' 2024-09-21T00:30:05.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28678 fp=#0x20000000e37' 2024-09-21T00:30:05.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28710 fp=#0x100000016db' 2024-09-21T00:30:05.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28742 fp=#0x100000016c4' 2024-09-21T00:30:05.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28774 fp=#0x1000000168c' 2024-09-21T00:30:05.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28806 fp=#0x100000016d8' 2024-09-21T00:30:05.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28838 fp=#0x200000009f5' 2024-09-21T00:30:05.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28870 fp=#0x200000009cf' 2024-09-21T00:30:05.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28934 fp=#0x10000001631' 2024-09-21T00:30:05.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28966 fp=#0x10000001640' 2024-09-21T00:30:05.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28998 fp=#0x1000000165d' 2024-09-21T00:30:05.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29030 fp=#0x20000000844' 2024-09-21T00:30:05.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29062 fp=#0x100000043b4' 2024-09-21T00:30:05.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29094 fp=#0x200000013e9' 2024-09-21T00:30:05.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29126 fp=#0x1000000436b' 2024-09-21T00:30:05.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29158 fp=#0x2000000140c' 2024-09-21T00:30:05.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29190 fp=#0x100000043af' 2024-09-21T00:30:05.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29222 fp=#0x10000004385' 2024-09-21T00:30:05.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29254 fp=#0x200000013f7' 2024-09-21T00:30:05.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29414 fp=#0x10000004e3b' 2024-09-21T00:30:05.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29446 fp=#0x10000004cae' 2024-09-21T00:30:05.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29510 fp=#0x10000004ce0' 2024-09-21T00:30:05.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29542 fp=#0x10000004c76' 2024-09-21T00:30:05.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29574 fp=#0x10000002a8d' 2024-09-21T00:30:05.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29606 fp=#0x100000029a2' 2024-09-21T00:30:05.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29638 fp=#0x100000040c0' 2024-09-21T00:30:05.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29670 fp=#0x100000040af' 2024-09-21T00:30:05.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29702 fp=#0x20000000ecd' 2024-09-21T00:30:05.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29766 fp=#0x10000004585' 2024-09-21T00:30:05.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29798 fp=#0x1000000453a' 2024-09-21T00:30:05.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29830 fp=#0x10000004556' 2024-09-21T00:30:05.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29862 fp=#0x10000004d79' 2024-09-21T00:30:05.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29894 fp=#0x10000004ce7' 2024-09-21T00:30:05.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29926 fp=#0x20000000c61' 2024-09-21T00:30:05.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29958 fp=#0x20000000e3e' 2024-09-21T00:30:05.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29990 fp=#0x20000000d93' 2024-09-21T00:30:05.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30022 fp=#0x20000000c87' 2024-09-21T00:30:05.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30054 fp=#0x20000000c4d' 2024-09-21T00:30:05.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30118 fp=#0x20000000a47' 2024-09-21T00:30:05.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30150 fp=#0x20000000be1' 2024-09-21T00:30:05.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30182 fp=#0x10000001c92' 2024-09-21T00:30:05.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30214 fp=#0x10000001b83' 2024-09-21T00:30:05.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30246 fp=#0x10000001bdc' 2024-09-21T00:30:05.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30278 fp=#0x10000002981' 2024-09-21T00:30:05.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30310 fp=#0x10000004d28' 2024-09-21T00:30:05.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30342 fp=#0x100000028b7' 2024-09-21T00:30:05.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30374 fp=#0x100000029b9' 2024-09-21T00:30:05.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30406 fp=#0x100000028e7' 2024-09-21T00:30:05.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30438 fp=#0x20000000a35' 2024-09-21T00:30:05.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22727 fp=#0x100000001fa' 2024-09-21T00:30:05.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22823 fp=#0x200000009ad' 2024-09-21T00:30:05.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26631 fp=#0x2000000082f' 2024-09-21T00:30:05.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26695 fp=#0x20000000785' 2024-09-21T00:30:05.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26727 fp=#0x2000000079d' 2024-09-21T00:30:05.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26791 fp=#0x10000004488' 2024-09-21T00:30:05.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26887 fp=#0x10000004638' 2024-09-21T00:30:05.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26919 fp=#0x10000004618' 2024-09-21T00:30:05.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26951 fp=#0x1000000444f' 2024-09-21T00:30:05.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26983 fp=#0x20000000d6f' 2024-09-21T00:30:05.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27015 fp=#0x20000001383' 2024-09-21T00:30:05.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27047 fp=#0x20000000fdc' 2024-09-21T00:30:05.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27079 fp=#0x20000000db2' 2024-09-21T00:30:05.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27111 fp=#0x20000000fd0' 2024-09-21T00:30:05.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27143 fp=#0x1000000223b' 2024-09-21T00:30:05.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27175 fp=#0x20000000f38' 2024-09-21T00:30:05.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27207 fp=#0x100000023ef' 2024-09-21T00:30:05.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27239 fp=#0x20000000f1f' 2024-09-21T00:30:05.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27271 fp=#0x10000002228' 2024-09-21T00:30:05.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27303 fp=#0x100000023a2' 2024-09-21T00:30:05.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27335 fp=#0x20000000f3b' 2024-09-21T00:30:05.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27367 fp=#0x1000000232f' 2024-09-21T00:30:05.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27399 fp=#0x10000003d10' 2024-09-21T00:30:05.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27431 fp=#0x10000003cb9' 2024-09-21T00:30:05.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27463 fp=#0x10000003c38' 2024-09-21T00:30:05.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27527 fp=#0x10000003b55' 2024-09-21T00:30:05.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27655 fp=#0x10000001ee2' 2024-09-21T00:30:05.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27687 fp=#0x10000001efd' 2024-09-21T00:30:05.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27719 fp=#0x10000001f89' 2024-09-21T00:30:05.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27751 fp=#0x20000001024' 2024-09-21T00:30:05.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27783 fp=#0x200000008d5' 2024-09-21T00:30:05.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27847 fp=#0x200000008eb' 2024-09-21T00:30:05.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27879 fp=#0x200000008d8' 2024-09-21T00:30:05.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27911 fp=#0x20000001123' 2024-09-21T00:30:05.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27943 fp=#0x2000000107c' 2024-09-21T00:30:05.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27975 fp=#0x100000049d0' 2024-09-21T00:30:05.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28007 fp=#0x100000049ed' 2024-09-21T00:30:05.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28039 fp=#0x10000004983' 2024-09-21T00:30:05.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28071 fp=#0x20000001385' 2024-09-21T00:30:05.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28103 fp=#0x20000001352' 2024-09-21T00:30:05.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28135 fp=#0x10000004c04' 2024-09-21T00:30:05.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28167 fp=#0x10000004b77' 2024-09-21T00:30:05.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28199 fp=#0x10000004b47' 2024-09-21T00:30:05.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28231 fp=#0x10000004b54' 2024-09-21T00:30:05.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28263 fp=#0x10000003125' 2024-09-21T00:30:05.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28295 fp=#0x20000000e3b' 2024-09-21T00:30:05.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28327 fp=#0x100000032d9' 2024-09-21T00:30:05.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28359 fp=#0x20000000e48' 2024-09-21T00:30:05.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28391 fp=#0x20000000e91' 2024-09-21T00:30:05.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28423 fp=#0x10000004c49' 2024-09-21T00:30:05.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28455 fp=#0x20000000e6f' 2024-09-21T00:30:05.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28487 fp=#0x20000000ea8' 2024-09-21T00:30:05.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28551 fp=#0x20000000fe2' 2024-09-21T00:30:05.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28583 fp=#0x20000000fa4' 2024-09-21T00:30:05.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28615 fp=#0x20000000fd1' 2024-09-21T00:30:05.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28647 fp=#0x20000001189' 2024-09-21T00:30:05.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28679 fp=#0x20000000e41' 2024-09-21T00:30:05.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28711 fp=#0x10000001639' 2024-09-21T00:30:05.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28743 fp=#0x10000001632' 2024-09-21T00:30:05.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28775 fp=#0x20000000391' 2024-09-21T00:30:05.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28807 fp=#0x100000015e1' 2024-09-21T00:30:05.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28839 fp=#0x200000009f4' 2024-09-21T00:30:05.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28903 fp=#0x200000009c3' 2024-09-21T00:30:05.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28935 fp=#0x100000016a0' 2024-09-21T00:30:05.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28967 fp=#0x1000000166d' 2024-09-21T00:30:05.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28999 fp=#0x10000001620' 2024-09-21T00:30:05.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29031 fp=#0x10000001616' 2024-09-21T00:30:05.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29063 fp=#0x100000043d5' 2024-09-21T00:30:05.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29095 fp=#0x100000043bb' 2024-09-21T00:30:05.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29159 fp=#0x1000000434e' 2024-09-21T00:30:05.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29223 fp=#0x10000004370' 2024-09-21T00:30:05.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29255 fp=#0x200000013e7' 2024-09-21T00:30:05.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29415 fp=#0x10000004e52' 2024-09-21T00:30:05.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29447 fp=#0x10000004c8c' 2024-09-21T00:30:05.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29479 fp=#0x10000004e4e' 2024-09-21T00:30:05.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29511 fp=#0x10000004e37' 2024-09-21T00:30:05.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29543 fp=#0x10000004e07' 2024-09-21T00:30:05.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29575 fp=#0x1000000297a' 2024-09-21T00:30:05.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29671 fp=#0x10000002974' 2024-09-21T00:30:05.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29735 fp=#0x10000002983' 2024-09-21T00:30:05.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29767 fp=#0x1000000452e' 2024-09-21T00:30:05.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29799 fp=#0x10000004553' 2024-09-21T00:30:05.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29831 fp=#0x10000004549' 2024-09-21T00:30:05.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29863 fp=#0x10000004d80' 2024-09-21T00:30:05.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29927 fp=#0x20000000dfc' 2024-09-21T00:30:05.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29959 fp=#0x20000000e2b' 2024-09-21T00:30:05.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29991 fp=#0x20000000cd7' 2024-09-21T00:30:05.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30023 fp=#0x20000000c90' 2024-09-21T00:30:05.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30055 fp=#0x20000000c75' 2024-09-21T00:30:05.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30087 fp=#0x20000000c66' 2024-09-21T00:30:05.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30119 fp=#0x20000000a2d' 2024-09-21T00:30:05.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30183 fp=#0x10000001b67' 2024-09-21T00:30:05.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30215 fp=#0x10000001b9a' 2024-09-21T00:30:05.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30247 fp=#0x10000001b49' 2024-09-21T00:30:05.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30279 fp=#0x20000000b1b' 2024-09-21T00:30:05.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30311 fp=#0x10000002921' 2024-09-21T00:30:05.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30343 fp=#0x10000002918' 2024-09-21T00:30:05.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30375 fp=#0x20000000a82' 2024-09-21T00:30:05.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30407 fp=#0x1000000298d' 2024-09-21T00:30:05.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30439 fp=#0x20000000b0a' 2024-09-21T00:30:05.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30471 fp=#0x100000023b3' 2024-09-21T00:30:05.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30503 fp=#0x10000002383' 2024-09-21T00:30:05.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30535 fp=#0x10000002388' 2024-09-21T00:30:05.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22728 fp=#0x100000001fb' 2024-09-21T00:30:05.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22824 fp=#0x100000051af' 2024-09-21T00:30:05.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:25256 fp=#0x20000001382' 2024-09-21T00:30:05.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26664 fp=#0x200000007ed' 2024-09-21T00:30:05.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26696 fp=#0x2000000078f' 2024-09-21T00:30:05.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26728 fp=#0x200000007ef' 2024-09-21T00:30:05.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26792 fp=#0x10000004449' 2024-09-21T00:30:05.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26824 fp=#0x10000004441' 2024-09-21T00:30:05.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26856 fp=#0x1000000460c' 2024-09-21T00:30:05.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26888 fp=#0x10000004462' 2024-09-21T00:30:05.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26920 fp=#0x1000000461f' 2024-09-21T00:30:05.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26984 fp=#0x20000000f09' 2024-09-21T00:30:05.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27016 fp=#0x20000000df8' 2024-09-21T00:30:05.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27048 fp=#0x20000000f0d' 2024-09-21T00:30:05.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27080 fp=#0x20000000efc' 2024-09-21T00:30:05.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27112 fp=#0x20000000faa' 2024-09-21T00:30:05.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27144 fp=#0x20000000f9c' 2024-09-21T00:30:05.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27176 fp=#0x10000002276' 2024-09-21T00:30:05.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27208 fp=#0x10000002264' 2024-09-21T00:30:05.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27240 fp=#0x10000002357' 2024-09-21T00:30:05.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27272 fp=#0x1000000235e' 2024-09-21T00:30:05.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27304 fp=#0x20000000f48' 2024-09-21T00:30:05.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27336 fp=#0x10000002250' 2024-09-21T00:30:05.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27368 fp=#0x100000023d2' 2024-09-21T00:30:05.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27400 fp=#0x10000003b51' 2024-09-21T00:30:05.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27432 fp=#0x10000003b63' 2024-09-21T00:30:05.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27464 fp=#0x10000003c8e' 2024-09-21T00:30:05.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27496 fp=#0x10000003cd4' 2024-09-21T00:30:05.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27528 fp=#0x10000003c66' 2024-09-21T00:30:05.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27656 fp=#0x10000001eea' 2024-09-21T00:30:05.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27688 fp=#0x10000001eb3' 2024-09-21T00:30:05.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27720 fp=#0x10000001f4b' 2024-09-21T00:30:05.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27784 fp=#0x2000000096f' 2024-09-21T00:30:05.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27816 fp=#0x200000008df' 2024-09-21T00:30:05.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27848 fp=#0x200000008d1' 2024-09-21T00:30:05.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27880 fp=#0x200000008c1' 2024-09-21T00:30:05.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27912 fp=#0x20000001178' 2024-09-21T00:30:05.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27944 fp=#0x200000010f8' 2024-09-21T00:30:05.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27976 fp=#0x1000000498c' 2024-09-21T00:30:05.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28008 fp=#0x10000004a83' 2024-09-21T00:30:05.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28040 fp=#0x100000049e8' 2024-09-21T00:30:05.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28072 fp=#0x200000012b8' 2024-09-21T00:30:05.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28104 fp=#0x200000012d0' 2024-09-21T00:30:05.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28136 fp=#0x10000004c17' 2024-09-21T00:30:05.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28168 fp=#0x10000004ba8' 2024-09-21T00:30:05.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28200 fp=#0x10000004add' 2024-09-21T00:30:05.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28232 fp=#0x10000004bcd' 2024-09-21T00:30:05.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28264 fp=#0x10000003361' 2024-09-21T00:30:05.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28296 fp=#0x10000003134' 2024-09-21T00:30:05.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28328 fp=#0x1000000310d' 2024-09-21T00:30:05.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28360 fp=#0x10000004c54' 2024-09-21T00:30:05.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28392 fp=#0x20000000e30' 2024-09-21T00:30:05.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28424 fp=#0x100000030b7' 2024-09-21T00:30:05.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28456 fp=#0x20000000e86' 2024-09-21T00:30:05.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28488 fp=#0x10000003080' 2024-09-21T00:30:05.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28520 fp=#0x20000001211' 2024-09-21T00:30:05.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28552 fp=#0x20000000fb9' 2024-09-21T00:30:05.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28648 fp=#0x20000001129' 2024-09-21T00:30:05.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28680 fp=#0x20000000e1c' 2024-09-21T00:30:05.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28712 fp=#0x10000001690' 2024-09-21T00:30:05.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28744 fp=#0x100000015d7' 2024-09-21T00:30:05.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28776 fp=#0x1000000169c' 2024-09-21T00:30:05.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28808 fp=#0x10000001666' 2024-09-21T00:30:05.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28840 fp=#0x200000009fe' 2024-09-21T00:30:05.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28936 fp=#0x20000000304' 2024-09-21T00:30:05.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28968 fp=#0x1000000168b' 2024-09-21T00:30:05.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29000 fp=#0x2000000030a' 2024-09-21T00:30:05.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29032 fp=#0x20000000f50' 2024-09-21T00:30:05.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29064 fp=#0x10000004389' 2024-09-21T00:30:05.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29128 fp=#0x200000013ca' 2024-09-21T00:30:05.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29160 fp=#0x10000004353' 2024-09-21T00:30:05.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29192 fp=#0x20000000f2e' 2024-09-21T00:30:05.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29224 fp=#0x1000000438f' 2024-09-21T00:30:05.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29256 fp=#0x1000000433d' 2024-09-21T00:30:05.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29544 fp=#0x10000004c66' 2024-09-21T00:30:05.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29576 fp=#0x10000002a69' 2024-09-21T00:30:05.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29640 fp=#0x10000002a84' 2024-09-21T00:30:05.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29672 fp=#0x100000040c6' 2024-09-21T00:30:05.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29704 fp=#0x10000002988' 2024-09-21T00:30:05.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29768 fp=#0x10000004531' 2024-09-21T00:30:05.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29800 fp=#0x1000000453b' 2024-09-21T00:30:05.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29832 fp=#0x10000004527' 2024-09-21T00:30:05.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29928 fp=#0x20000000d0f' 2024-09-21T00:30:05.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29960 fp=#0x20000000e7b' 2024-09-21T00:30:05.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29992 fp=#0x20000000cc5' 2024-09-21T00:30:05.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30024 fp=#0x20000000cd4' 2024-09-21T00:30:05.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30056 fp=#0x20000000a5c' 2024-09-21T00:30:05.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30120 fp=#0x20000000c36' 2024-09-21T00:30:05.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30184 fp=#0x10000001b72' 2024-09-21T00:30:05.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30216 fp=#0x10000001c24' 2024-09-21T00:30:05.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30248 fp=#0x10000001c01' 2024-09-21T00:30:05.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30280 fp=#0x100000028e5' 2024-09-21T00:30:05.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30312 fp=#0x10000002972' 2024-09-21T00:30:05.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30344 fp=#0x10000002923' 2024-09-21T00:30:05.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30376 fp=#0x20000000ba5' 2024-09-21T00:30:05.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30408 fp=#0x10000002937' 2024-09-21T00:30:05.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30440 fp=#0x20000000a67' 2024-09-21T00:30:05.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30472 fp=#0x10000002372' 2024-09-21T00:30:05.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30504 fp=#0x200000006f4' 2024-09-21T00:30:05.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30536 fp=#0x10000002379' 2024-09-21T00:30:05.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22729 fp=#0x10000000005' 2024-09-21T00:30:05.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22825 fp=#0x100000015f8' 2024-09-21T00:30:05.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:25257 fp=#0x200000013b4' 2024-09-21T00:30:05.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26633 fp=#0x2000000076b' 2024-09-21T00:30:05.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26793 fp=#0x1000000448d' 2024-09-21T00:30:05.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26825 fp=#0x10000004629' 2024-09-21T00:30:05.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26889 fp=#0x100000043f0' 2024-09-21T00:30:05.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26921 fp=#0x10000004619' 2024-09-21T00:30:05.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26985 fp=#0x20000000dba' 2024-09-21T00:30:05.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27017 fp=#0x20000000e0c' 2024-09-21T00:30:05.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27049 fp=#0x20000000fef' 2024-09-21T00:30:05.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27081 fp=#0x20000000ecc' 2024-09-21T00:30:05.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27113 fp=#0x20000000f8e' 2024-09-21T00:30:05.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27145 fp=#0x10000002270' 2024-09-21T00:30:05.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27177 fp=#0x20000000f82' 2024-09-21T00:30:05.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27209 fp=#0x20000000f19' 2024-09-21T00:30:05.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27241 fp=#0x20000001015' 2024-09-21T00:30:05.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27273 fp=#0x100000023ad' 2024-09-21T00:30:05.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27305 fp=#0x10000002242' 2024-09-21T00:30:05.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27337 fp=#0x100000023b8' 2024-09-21T00:30:05.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27369 fp=#0x10000002221' 2024-09-21T00:30:05.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27401 fp=#0x10000003b57' 2024-09-21T00:30:05.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27433 fp=#0x20000001187' 2024-09-21T00:30:05.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27465 fp=#0x10000003bc3' 2024-09-21T00:30:05.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27497 fp=#0x10000003b9c' 2024-09-21T00:30:05.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27657 fp=#0x10000001f8e' 2024-09-21T00:30:05.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27689 fp=#0x10000001efa' 2024-09-21T00:30:05.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27721 fp=#0x10000001ecc' 2024-09-21T00:30:05.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27753 fp=#0x200000008e7' 2024-09-21T00:30:05.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27817 fp=#0x2000000101b' 2024-09-21T00:30:05.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27881 fp=#0x200000008cd' 2024-09-21T00:30:05.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27913 fp=#0x200000010bc' 2024-09-21T00:30:05.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27945 fp=#0x200000010b7' 2024-09-21T00:30:05.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27977 fp=#0x10000004a6f' 2024-09-21T00:30:05.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28009 fp=#0x100000049ae' 2024-09-21T00:30:05.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28041 fp=#0x100000049db' 2024-09-21T00:30:05.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28073 fp=#0x2000000136e' 2024-09-21T00:30:05.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28105 fp=#0x2000000125d' 2024-09-21T00:30:05.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28137 fp=#0x10000004b65' 2024-09-21T00:30:05.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28201 fp=#0x10000004be7' 2024-09-21T00:30:05.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28233 fp=#0x10000004bbf' 2024-09-21T00:30:05.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28265 fp=#0x100000032c2' 2024-09-21T00:30:05.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28297 fp=#0x20000000e83' 2024-09-21T00:30:05.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28329 fp=#0x10000004bd1' 2024-09-21T00:30:05.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28361 fp=#0x10000003226' 2024-09-21T00:30:05.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28393 fp=#0x10000003024' 2024-09-21T00:30:05.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28425 fp=#0x10000003129' 2024-09-21T00:30:05.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28457 fp=#0x20000000e74' 2024-09-21T00:30:05.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28489 fp=#0x100000032f2' 2024-09-21T00:30:05.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28521 fp=#0x20000000fb3' 2024-09-21T00:30:05.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28553 fp=#0x20000001004' 2024-09-21T00:30:05.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28585 fp=#0x20000000fbc' 2024-09-21T00:30:05.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28617 fp=#0x200000010ec' 2024-09-21T00:30:05.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28649 fp=#0x20000000e14' 2024-09-21T00:30:05.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28681 fp=#0x20000000da4' 2024-09-21T00:30:05.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28713 fp=#0x10000001742' 2024-09-21T00:30:05.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28745 fp=#0x100000016c2' 2024-09-21T00:30:05.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28777 fp=#0x1000000163f' 2024-09-21T00:30:05.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28809 fp=#0x1000000161d' 2024-09-21T00:30:05.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28841 fp=#0x200000009cb' 2024-09-21T00:30:05.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28905 fp=#0x10000001638' 2024-09-21T00:30:05.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28937 fp=#0x2000000037a' 2024-09-21T00:30:05.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28969 fp=#0x2000000031b' 2024-09-21T00:30:05.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29001 fp=#0x100000016c1' 2024-09-21T00:30:05.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29033 fp=#0x20000000305' 2024-09-21T00:30:05.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29065 fp=#0x10000004342' 2024-09-21T00:30:05.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29097 fp=#0x10000004345' 2024-09-21T00:30:05.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29129 fp=#0x1000000437c' 2024-09-21T00:30:05.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29161 fp=#0x200000013b0' 2024-09-21T00:30:05.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29193 fp=#0x200000013c3' 2024-09-21T00:30:05.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29225 fp=#0x1000000439c' 2024-09-21T00:30:05.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29257 fp=#0x10000004364' 2024-09-21T00:30:05.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29417 fp=#0x10000004ce3' 2024-09-21T00:30:05.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29449 fp=#0x10000004cd4' 2024-09-21T00:30:05.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29545 fp=#0x10000004c79' 2024-09-21T00:30:05.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29641 fp=#0x100000040d1' 2024-09-21T00:30:05.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29673 fp=#0x100000040a8' 2024-09-21T00:30:05.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29705 fp=#0x100000040c4' 2024-09-21T00:30:05.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29769 fp=#0x10000004557' 2024-09-21T00:30:05.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29801 fp=#0x20000000a30' 2024-09-21T00:30:05.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29833 fp=#0x10000004574' 2024-09-21T00:30:05.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29865 fp=#0x10000004d6b' 2024-09-21T00:30:05.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29929 fp=#0x20000000cac' 2024-09-21T00:30:05.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29961 fp=#0x20000000d84' 2024-09-21T00:30:05.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29993 fp=#0x20000000ca7' 2024-09-21T00:30:05.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30025 fp=#0x20000000cf6' 2024-09-21T00:30:05.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30057 fp=#0x20000000c62' 2024-09-21T00:30:05.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30153 fp=#0x20000000c31' 2024-09-21T00:30:05.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30185 fp=#0x10000001c36' 2024-09-21T00:30:05.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30217 fp=#0x100000055aa' 2024-09-21T00:30:05.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30249 fp=#0x10000001b0d' 2024-09-21T00:30:05.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30281 fp=#0x100000029b6' 2024-09-21T00:30:05.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30313 fp=#0x1000000295b' 2024-09-21T00:30:05.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30345 fp=#0x20000000b75' 2024-09-21T00:30:05.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30377 fp=#0x20000000a2c' 2024-09-21T00:30:05.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30409 fp=#0x1000000290c' 2024-09-21T00:30:05.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30441 fp=#0x20000000a80' 2024-09-21T00:30:05.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30473 fp=#0x200000006f3' 2024-09-21T00:30:05.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22730 fp=#0x10000000006' 2024-09-21T00:30:05.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22826 fp=#0x100000021fd' 2024-09-21T00:30:05.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26666 fp=#0x2000000078b' 2024-09-21T00:30:05.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26698 fp=#0x20000000a11' 2024-09-21T00:30:05.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26730 fp=#0x2000000079b' 2024-09-21T00:30:05.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26762 fp=#0x20000000779' 2024-09-21T00:30:05.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26826 fp=#0x10000004628' 2024-09-21T00:30:05.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26858 fp=#0x10000004639' 2024-09-21T00:30:05.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26922 fp=#0x10000004447' 2024-09-21T00:30:05.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26954 fp=#0x1000000460d' 2024-09-21T00:30:05.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26986 fp=#0x20000000f63' 2024-09-21T00:30:05.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27018 fp=#0x20000000fcb' 2024-09-21T00:30:05.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27050 fp=#0x20000000ece' 2024-09-21T00:30:05.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27082 fp=#0x20000000ed6' 2024-09-21T00:30:05.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27114 fp=#0x20000000d89' 2024-09-21T00:30:05.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27178 fp=#0x20000000faf' 2024-09-21T00:30:05.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27210 fp=#0x1000000227c' 2024-09-21T00:30:05.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27242 fp=#0x20000000fe3' 2024-09-21T00:30:05.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27274 fp=#0x20000000f2d' 2024-09-21T00:30:05.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27306 fp=#0x20000000f4f' 2024-09-21T00:30:05.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27338 fp=#0x1000000221e' 2024-09-21T00:30:05.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27370 fp=#0x10000002392' 2024-09-21T00:30:05.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27402 fp=#0x10000003bbd' 2024-09-21T00:30:05.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27434 fp=#0x10000003b83' 2024-09-21T00:30:05.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27466 fp=#0x10000003b71' 2024-09-21T00:30:05.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27658 fp=#0x10000001ea2' 2024-09-21T00:30:05.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27690 fp=#0x10000001f0d' 2024-09-21T00:30:05.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27722 fp=#0x10000001f56' 2024-09-21T00:30:05.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27818 fp=#0x200000008c9' 2024-09-21T00:30:05.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27882 fp=#0x200000008f5' 2024-09-21T00:30:05.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27914 fp=#0x20000001051' 2024-09-21T00:30:05.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27946 fp=#0x2000000110e' 2024-09-21T00:30:05.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27978 fp=#0x100000049f2' 2024-09-21T00:30:05.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28010 fp=#0x1000000496b' 2024-09-21T00:30:05.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28042 fp=#0x10000004962' 2024-09-21T00:30:05.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28074 fp=#0x20000001267' 2024-09-21T00:30:05.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28106 fp=#0x20000001305' 2024-09-21T00:30:05.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28138 fp=#0x10000004af1' 2024-09-21T00:30:05.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28170 fp=#0x20000000f84' 2024-09-21T00:30:05.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28202 fp=#0x10000004b56' 2024-09-21T00:30:05.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28234 fp=#0x10000004b4c' 2024-09-21T00:30:05.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28266 fp=#0x10000003366' 2024-09-21T00:30:05.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28298 fp=#0x20000000e97' 2024-09-21T00:30:05.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28330 fp=#0x10000004c51' 2024-09-21T00:30:05.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28362 fp=#0x20000000f22' 2024-09-21T00:30:05.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28394 fp=#0x10000003037' 2024-09-21T00:30:05.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28426 fp=#0x100000032f5' 2024-09-21T00:30:05.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28458 fp=#0x10000004bcc' 2024-09-21T00:30:05.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28490 fp=#0x1000000303b' 2024-09-21T00:30:05.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28522 fp=#0x20000001206' 2024-09-21T00:30:05.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28554 fp=#0x20000000f55' 2024-09-21T00:30:05.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28618 fp=#0x20000000f92' 2024-09-21T00:30:05.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28650 fp=#0x20000000e00' 2024-09-21T00:30:05.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28682 fp=#0x20000000d48' 2024-09-21T00:30:05.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28714 fp=#0x1000000174f' 2024-09-21T00:30:05.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28746 fp=#0x10000001615' 2024-09-21T00:30:05.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28778 fp=#0x100000016b2' 2024-09-21T00:30:05.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28810 fp=#0x100000015e6' 2024-09-21T00:30:05.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28842 fp=#0x200000009f8' 2024-09-21T00:30:05.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28906 fp=#0x20000000313' 2024-09-21T00:30:05.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28938 fp=#0x100000015fa' 2024-09-21T00:30:05.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28970 fp=#0x100000016e6' 2024-09-21T00:30:05.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29002 fp=#0x100000016b5' 2024-09-21T00:30:05.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29034 fp=#0x1000000161b' 2024-09-21T00:30:05.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29066 fp=#0x10000004375' 2024-09-21T00:30:05.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29162 fp=#0x20000001405' 2024-09-21T00:30:05.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29226 fp=#0x10000004363' 2024-09-21T00:30:05.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29258 fp=#0x100000043bc' 2024-09-21T00:30:05.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29418 fp=#0x10000004e47' 2024-09-21T00:30:05.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29450 fp=#0x20000000ff7' 2024-09-21T00:30:05.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29546 fp=#0x10000004cd9' 2024-09-21T00:30:05.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29610 fp=#0x100000040d5' 2024-09-21T00:30:05.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29674 fp=#0x20000000ec0' 2024-09-21T00:30:05.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29706 fp=#0x100000040cb' 2024-09-21T00:30:05.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29738 fp=#0x100000040b9' 2024-09-21T00:30:05.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29770 fp=#0x1000000451a' 2024-09-21T00:30:05.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29802 fp=#0x10000004588' 2024-09-21T00:30:05.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29834 fp=#0x1000000454c' 2024-09-21T00:30:05.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29866 fp=#0x10000004d62' 2024-09-21T00:30:05.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29930 fp=#0x20000000dce' 2024-09-21T00:30:05.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29962 fp=#0x20000000c65' 2024-09-21T00:30:05.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29994 fp=#0x20000000dc6' 2024-09-21T00:30:05.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30026 fp=#0x20000000de6' 2024-09-21T00:30:05.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30058 fp=#0x20000000a32' 2024-09-21T00:30:05.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30186 fp=#0x10000001b11' 2024-09-21T00:30:05.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30218 fp=#0x10000001b9d' 2024-09-21T00:30:05.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30250 fp=#0x100000055c5' 2024-09-21T00:30:05.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30282 fp=#0x10000002957' 2024-09-21T00:30:05.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30314 fp=#0x10000002915' 2024-09-21T00:30:05.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30346 fp=#0x20000000a5b' 2024-09-21T00:30:05.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30378 fp=#0x10000002998' 2024-09-21T00:30:05.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30410 fp=#0x10000002996' 2024-09-21T00:30:05.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30442 fp=#0x100000028c7' 2024-09-21T00:30:05.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30506 fp=#0x1000000239c' 2024-09-21T00:30:05.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22731 fp=#0x10000000201' 2024-09-21T00:30:05.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22827 fp=#0x1000000432f' 2024-09-21T00:30:05.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26699 fp=#0x20000000791' 2024-09-21T00:30:05.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26795 fp=#0x100000043ea' 2024-09-21T00:30:05.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26827 fp=#0x1000000445e' 2024-09-21T00:30:05.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26859 fp=#0x10000004631' 2024-09-21T00:30:05.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26955 fp=#0x20000000eff' 2024-09-21T00:30:05.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26987 fp=#0x20000000f94' 2024-09-21T00:30:05.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27019 fp=#0x20000000ecf' 2024-09-21T00:30:05.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27051 fp=#0x20000000f02' 2024-09-21T00:30:05.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27083 fp=#0x20000000deb' 2024-09-21T00:30:05.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27115 fp=#0x20000000eec' 2024-09-21T00:30:05.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27147 fp=#0x100000023c5' 2024-09-21T00:30:05.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27179 fp=#0x100000023e0' 2024-09-21T00:30:05.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27211 fp=#0x10000002267' 2024-09-21T00:30:05.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27243 fp=#0x1000000222c' 2024-09-21T00:30:05.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27275 fp=#0x10000002279' 2024-09-21T00:30:05.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27307 fp=#0x100000022a2' 2024-09-21T00:30:05.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27339 fp=#0x20000000fed' 2024-09-21T00:30:05.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27371 fp=#0x20000000f45' 2024-09-21T00:30:05.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27403 fp=#0x10000003cbd' 2024-09-21T00:30:05.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27435 fp=#0x10000003ba2' 2024-09-21T00:30:05.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27467 fp=#0x2000000113a' 2024-09-21T00:30:05.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27499 fp=#0x20000001193' 2024-09-21T00:30:05.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27627 fp=#0x10000001ef7' 2024-09-21T00:30:05.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27659 fp=#0x10000001e9e' 2024-09-21T00:30:05.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27691 fp=#0x10000001f80' 2024-09-21T00:30:05.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27723 fp=#0x10000001ea0' 2024-09-21T00:30:05.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27819 fp=#0x200000008de' 2024-09-21T00:30:05.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27883 fp=#0x20000000ae0' 2024-09-21T00:30:05.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27915 fp=#0x2000000114a' 2024-09-21T00:30:05.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27947 fp=#0x20000001046' 2024-09-21T00:30:05.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27979 fp=#0x10000004a56' 2024-09-21T00:30:05.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28011 fp=#0x100000049ab' 2024-09-21T00:30:05.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28043 fp=#0x10000004a21' 2024-09-21T00:30:05.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28075 fp=#0x20000001334' 2024-09-21T00:30:05.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28107 fp=#0x2000000129e' 2024-09-21T00:30:05.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28139 fp=#0x10000004c0c' 2024-09-21T00:30:05.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28171 fp=#0x10000004abd' 2024-09-21T00:30:05.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28203 fp=#0x10000004bba' 2024-09-21T00:30:05.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28235 fp=#0x10000004bed' 2024-09-21T00:30:05.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28267 fp=#0x10000003197' 2024-09-21T00:30:05.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28299 fp=#0x100000032d3' 2024-09-21T00:30:05.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28331 fp=#0x10000003141' 2024-09-21T00:30:05.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28363 fp=#0x10000003030' 2024-09-21T00:30:05.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28395 fp=#0x100000031d7' 2024-09-21T00:30:05.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28427 fp=#0x10000003277' 2024-09-21T00:30:05.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28459 fp=#0x100000032bf' 2024-09-21T00:30:05.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28491 fp=#0x20000000f00' 2024-09-21T00:30:05.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28555 fp=#0x20000001023' 2024-09-21T00:30:05.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28587 fp=#0x2000000115c' 2024-09-21T00:30:05.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28651 fp=#0x20000000d65' 2024-09-21T00:30:05.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28683 fp=#0x20000000de7' 2024-09-21T00:30:05.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28715 fp=#0x10000001611' 2024-09-21T00:30:05.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28747 fp=#0x10000001602' 2024-09-21T00:30:05.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28779 fp=#0x20000000423' 2024-09-21T00:30:05.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28811 fp=#0x10000001607' 2024-09-21T00:30:05.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28907 fp=#0x2000000030e' 2024-09-21T00:30:05.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28939 fp=#0x10000001627' 2024-09-21T00:30:05.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28971 fp=#0x1000000163c' 2024-09-21T00:30:05.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29003 fp=#0x10000001643' 2024-09-21T00:30:05.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29035 fp=#0x1000000167c' 2024-09-21T00:30:05.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29067 fp=#0x100000043c9' 2024-09-21T00:30:05.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29099 fp=#0x100000043b2' 2024-09-21T00:30:05.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29163 fp=#0x1000000436d' 2024-09-21T00:30:05.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29195 fp=#0x10000004357' 2024-09-21T00:30:05.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29227 fp=#0x200000013d9' 2024-09-21T00:30:05.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29259 fp=#0x1000000433f' 2024-09-21T00:30:05.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29419 fp=#0x20000001025' 2024-09-21T00:30:05.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29483 fp=#0x10000004e39' 2024-09-21T00:30:05.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29515 fp=#0x10000004cf6' 2024-09-21T00:30:05.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29579 fp=#0x10000002a7e' 2024-09-21T00:30:05.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29611 fp=#0x100000040d4' 2024-09-21T00:30:05.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29643 fp=#0x100000040a0' 2024-09-21T00:30:05.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29675 fp=#0x10000002a4e' 2024-09-21T00:30:05.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29707 fp=#0x100000040c5' 2024-09-21T00:30:05.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29739 fp=#0x10000004534' 2024-09-21T00:30:05.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29771 fp=#0x10000004517' 2024-09-21T00:30:05.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29803 fp=#0x10000004580' 2024-09-21T00:30:05.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29835 fp=#0x10000004519' 2024-09-21T00:30:05.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29867 fp=#0x10000004d5e' 2024-09-21T00:30:05.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29899 fp=#0x10000004cfa' 2024-09-21T00:30:05.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29931 fp=#0x20000000e10' 2024-09-21T00:30:05.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29963 fp=#0x20000000c96' 2024-09-21T00:30:05.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29995 fp=#0x20000000cda' 2024-09-21T00:30:05.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30027 fp=#0x20000000d0e' 2024-09-21T00:30:05.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30059 fp=#0x20000000bff' 2024-09-21T00:30:05.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30091 fp=#0x20000000a29' 2024-09-21T00:30:05.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30123 fp=#0x20000000a58' 2024-09-21T00:30:05.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30155 fp=#0x20000000a36' 2024-09-21T00:30:05.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30187 fp=#0x10000001b04' 2024-09-21T00:30:05.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30219 fp=#0x100000055a6' 2024-09-21T00:30:05.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30251 fp=#0x10000001bd1' 2024-09-21T00:30:05.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30283 fp=#0x100000029bc' 2024-09-21T00:30:05.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30315 fp=#0x10000002946' 2024-09-21T00:30:05.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30347 fp=#0x10000002963' 2024-09-21T00:30:05.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30379 fp=#0x1000000291b' 2024-09-21T00:30:05.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30411 fp=#0x10000002940' 2024-09-21T00:30:05.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30443 fp=#0x10000002951' 2024-09-21T00:30:05.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30507 fp=#0x100000023ac' 2024-09-21T00:30:05.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22732 fp=#0x10000000208' 2024-09-21T00:30:05.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22764 fp=#0x20000001241' 2024-09-21T00:30:05.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26636 fp=#0x2000000077f' 2024-09-21T00:30:05.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26732 fp=#0x20000001274' 2024-09-21T00:30:05.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26796 fp=#0x10000004473' 2024-09-21T00:30:05.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26828 fp=#0x100000043f6' 2024-09-21T00:30:05.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26860 fp=#0x10000004615' 2024-09-21T00:30:05.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26924 fp=#0x100000043eb' 2024-09-21T00:30:05.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26956 fp=#0x20000000ef7' 2024-09-21T00:30:05.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26988 fp=#0x20000000eeb' 2024-09-21T00:30:05.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27020 fp=#0x20000000eea' 2024-09-21T00:30:05.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27084 fp=#0x20000000f01' 2024-09-21T00:30:05.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27116 fp=#0x20000000d81' 2024-09-21T00:30:05.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27180 fp=#0x20000000f6b' 2024-09-21T00:30:05.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27212 fp=#0x20000000f12' 2024-09-21T00:30:05.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27244 fp=#0x10000002237' 2024-09-21T00:30:05.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27276 fp=#0x20000000f76' 2024-09-21T00:30:05.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27308 fp=#0x10000002274' 2024-09-21T00:30:05.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27340 fp=#0x20000000fc1' 2024-09-21T00:30:05.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27372 fp=#0x10000003b5d' 2024-09-21T00:30:05.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27404 fp=#0x10000003ca5' 2024-09-21T00:30:05.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27436 fp=#0x10000003d16' 2024-09-21T00:30:05.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27468 fp=#0x10000003ccd' 2024-09-21T00:30:05.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27500 fp=#0x10000003b8c' 2024-09-21T00:30:05.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27628 fp=#0x10000002053' 2024-09-21T00:30:05.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27660 fp=#0x200000008ae' 2024-09-21T00:30:05.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27692 fp=#0x10000001f11' 2024-09-21T00:30:05.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27724 fp=#0x10000001e93' 2024-09-21T00:30:05.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27756 fp=#0x20000000972' 2024-09-21T00:30:05.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27788 fp=#0x20000000976' 2024-09-21T00:30:05.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27820 fp=#0x200000008e5' 2024-09-21T00:30:05.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27852 fp=#0x200000008e4' 2024-09-21T00:30:05.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27884 fp=#0x200000008cf' 2024-09-21T00:30:05.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27916 fp=#0x20000001081' 2024-09-21T00:30:05.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27948 fp=#0x200000010da' 2024-09-21T00:30:05.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27980 fp=#0x10000004968' 2024-09-21T00:30:05.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28012 fp=#0x100000049b9' 2024-09-21T00:30:05.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28044 fp=#0x10000004a15' 2024-09-21T00:30:05.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28076 fp=#0x200000012d4' 2024-09-21T00:30:05.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28108 fp=#0x20000001287' 2024-09-21T00:30:05.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28172 fp=#0x10000004af5' 2024-09-21T00:30:05.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28204 fp=#0x10000004bc3' 2024-09-21T00:30:05.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28236 fp=#0x10000004af7' 2024-09-21T00:30:05.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28268 fp=#0x1000000305a' 2024-09-21T00:30:05.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28300 fp=#0x1000000301a' 2024-09-21T00:30:05.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28332 fp=#0x10000003117' 2024-09-21T00:30:05.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28364 fp=#0x20000000ef2' 2024-09-21T00:30:05.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28396 fp=#0x1000000311c' 2024-09-21T00:30:05.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28428 fp=#0x1000000329a' 2024-09-21T00:30:05.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28460 fp=#0x100000032a9' 2024-09-21T00:30:05.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28492 fp=#0x100000031ad' 2024-09-21T00:30:05.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28524 fp=#0x20000001200' 2024-09-21T00:30:05.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28556 fp=#0x20000001035' 2024-09-21T00:30:05.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28588 fp=#0x20000000fdd' 2024-09-21T00:30:05.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28620 fp=#0x200000011c8' 2024-09-21T00:30:05.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28652 fp=#0x20000000dfd' 2024-09-21T00:30:05.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28684 fp=#0x20000000d73' 2024-09-21T00:30:05.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28716 fp=#0x100000016a2' 2024-09-21T00:30:05.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28748 fp=#0x1000000172e' 2024-09-21T00:30:05.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28780 fp=#0x100000015ea' 2024-09-21T00:30:05.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28812 fp=#0x20000000376' 2024-09-21T00:30:05.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28876 fp=#0x200000009c6' 2024-09-21T00:30:05.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28908 fp=#0x1000000161a' 2024-09-21T00:30:05.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28940 fp=#0x10000001740' 2024-09-21T00:30:05.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28972 fp=#0x1000000160f' 2024-09-21T00:30:05.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29004 fp=#0x1000000169a' 2024-09-21T00:30:05.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29036 fp=#0x10000001681' 2024-09-21T00:30:05.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29068 fp=#0x1000000434b' 2024-09-21T00:30:05.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29100 fp=#0x100000043c6' 2024-09-21T00:30:05.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29132 fp=#0x20000001425' 2024-09-21T00:30:05.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29164 fp=#0x2000000141c' 2024-09-21T00:30:05.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29196 fp=#0x2000000141e' 2024-09-21T00:30:05.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29228 fp=#0x10000004340' 2024-09-21T00:30:05.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29260 fp=#0x10000004361' 2024-09-21T00:30:05.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29452 fp=#0x10000004cf1' 2024-09-21T00:30:05.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29484 fp=#0x10000004cf4' 2024-09-21T00:30:05.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29516 fp=#0x10000004c9d' 2024-09-21T00:30:05.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29548 fp=#0x10000004d06' 2024-09-21T00:30:05.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29676 fp=#0x1000000298c' 2024-09-21T00:30:05.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29740 fp=#0x1000000457e' 2024-09-21T00:30:05.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29772 fp=#0x1000000452c' 2024-09-21T00:30:05.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29804 fp=#0x1000000456b' 2024-09-21T00:30:05.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29836 fp=#0x10000004547' 2024-09-21T00:30:05.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29868 fp=#0x10000004d83' 2024-09-21T00:30:05.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29932 fp=#0x20000000d47' 2024-09-21T00:30:05.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29964 fp=#0x20000000e04' 2024-09-21T00:30:05.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29996 fp=#0x20000000dd5' 2024-09-21T00:30:05.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30028 fp=#0x20000000e78' 2024-09-21T00:30:05.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30060 fp=#0x20000000bd2' 2024-09-21T00:30:05.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30124 fp=#0x20000000c44' 2024-09-21T00:30:05.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30156 fp=#0x20000000a45' 2024-09-21T00:30:05.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30188 fp=#0x10000001b5e' 2024-09-21T00:30:05.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30220 fp=#0x10000001c9d' 2024-09-21T00:30:05.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30252 fp=#0x10000001b1e' 2024-09-21T00:30:05.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30284 fp=#0x100000028f4' 2024-09-21T00:30:05.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30316 fp=#0x1000000294f' 2024-09-21T00:30:05.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30348 fp=#0x20000000b54' 2024-09-21T00:30:05.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30380 fp=#0x100000028d7' 2024-09-21T00:30:05.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30412 fp=#0x20000000a8c' 2024-09-21T00:30:05.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30444 fp=#0x20000000b7b' 2024-09-21T00:30:05.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30476 fp=#0x100000023d5' 2024-09-21T00:30:05.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30540 fp=#0x1000000238e' 2024-09-21T00:30:05.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22733 fp=#0x10000000399' 2024-09-21T00:30:05.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22829 fp=#0x10000004c55' 2024-09-21T00:30:05.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26701 fp=#0x200000007ec' 2024-09-21T00:30:05.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26829 fp=#0x10000004611' 2024-09-21T00:30:05.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26861 fp=#0x1000000462f' 2024-09-21T00:30:05.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26925 fp=#0x10000004467' 2024-09-21T00:30:05.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26957 fp=#0x20000000dd0' 2024-09-21T00:30:05.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26989 fp=#0x20000000f47' 2024-09-21T00:30:05.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27021 fp=#0x20000000fb2' 2024-09-21T00:30:05.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27053 fp=#0x20000000dff' 2024-09-21T00:30:05.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27085 fp=#0x20000000ed9' 2024-09-21T00:30:05.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27117 fp=#0x20000000d8d' 2024-09-21T00:30:05.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27149 fp=#0x1000000230d' 2024-09-21T00:30:05.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27181 fp=#0x10000002218' 2024-09-21T00:30:05.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27213 fp=#0x20000000f7b' 2024-09-21T00:30:05.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27245 fp=#0x20000000f62' 2024-09-21T00:30:05.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27277 fp=#0x20000000f1b' 2024-09-21T00:30:05.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27309 fp=#0x10000002342' 2024-09-21T00:30:05.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27341 fp=#0x20000000fa9' 2024-09-21T00:30:05.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27373 fp=#0x200000008e1' 2024-09-21T00:30:05.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27405 fp=#0x10000003ce4' 2024-09-21T00:30:05.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27437 fp=#0x10000003c5f' 2024-09-21T00:30:05.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27501 fp=#0x10000003cb7' 2024-09-21T00:30:05.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27629 fp=#0x10000001e9a' 2024-09-21T00:30:05.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27661 fp=#0x10000001edc' 2024-09-21T00:30:05.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27725 fp=#0x10000001f40' 2024-09-21T00:30:05.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27885 fp=#0x200000008f3' 2024-09-21T00:30:05.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27917 fp=#0x200000010e3' 2024-09-21T00:30:05.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27949 fp=#0x2000000107e' 2024-09-21T00:30:05.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27981 fp=#0x10000004a54' 2024-09-21T00:30:05.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28013 fp=#0x100000049fb' 2024-09-21T00:30:05.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28045 fp=#0x100000049d7' 2024-09-21T00:30:05.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28077 fp=#0x200000012ed' 2024-09-21T00:30:05.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28109 fp=#0x20000001310' 2024-09-21T00:30:05.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28141 fp=#0x10000004b3f' 2024-09-21T00:30:05.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28173 fp=#0x10000004b1b' 2024-09-21T00:30:05.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28205 fp=#0x10000004b99' 2024-09-21T00:30:05.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28237 fp=#0x10000004bc9' 2024-09-21T00:30:05.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28269 fp=#0x10000003144' 2024-09-21T00:30:05.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28301 fp=#0x20000000e6c' 2024-09-21T00:30:05.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28333 fp=#0x10000004c4c' 2024-09-21T00:30:05.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28365 fp=#0x100000030a5' 2024-09-21T00:30:05.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28397 fp=#0x1000000309c' 2024-09-21T00:30:05.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28429 fp=#0x1000000316c' 2024-09-21T00:30:05.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28461 fp=#0x10000003148' 2024-09-21T00:30:05.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28493 fp=#0x20000000e51' 2024-09-21T00:30:05.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28525 fp=#0x20000001232' 2024-09-21T00:30:05.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28557 fp=#0x20000001102' 2024-09-21T00:30:05.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28621 fp=#0x20000000fa0' 2024-09-21T00:30:05.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28653 fp=#0x20000000d3e' 2024-09-21T00:30:05.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28685 fp=#0x20000000def' 2024-09-21T00:30:05.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28717 fp=#0x10000001703' 2024-09-21T00:30:05.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28749 fp=#0x10000001685' 2024-09-21T00:30:05.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28781 fp=#0x10000001646' 2024-09-21T00:30:05.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28813 fp=#0x20000000a08' 2024-09-21T00:30:05.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28909 fp=#0x100000016b3' 2024-09-21T00:30:05.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28941 fp=#0x2000000030f' 2024-09-21T00:30:05.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28973 fp=#0x1000000163a' 2024-09-21T00:30:05.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29005 fp=#0x20000000311' 2024-09-21T00:30:05.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29037 fp=#0x100000016bf' 2024-09-21T00:30:05.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29069 fp=#0x1000000435f' 2024-09-21T00:30:05.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29101 fp=#0x10000004386' 2024-09-21T00:30:05.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29133 fp=#0x10000004356' 2024-09-21T00:30:05.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29165 fp=#0x10000004354' 2024-09-21T00:30:05.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29197 fp=#0x100000043c0' 2024-09-21T00:30:05.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29229 fp=#0x1000000438e' 2024-09-21T00:30:05.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29261 fp=#0x10000004365' 2024-09-21T00:30:05.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29421 fp=#0x10000004ca2' 2024-09-21T00:30:05.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29453 fp=#0x10000004d01' 2024-09-21T00:30:05.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29485 fp=#0x10000004c80' 2024-09-21T00:30:05.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29581 fp=#0x10000002a78' 2024-09-21T00:30:05.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29613 fp=#0x1000000299f' 2024-09-21T00:30:05.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29645 fp=#0x100000040b8' 2024-09-21T00:30:05.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29741 fp=#0x10000004513' 2024-09-21T00:30:05.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29773 fp=#0x10000004579' 2024-09-21T00:30:05.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29805 fp=#0x1000000454d' 2024-09-21T00:30:05.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29837 fp=#0x1000000451d' 2024-09-21T00:30:05.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29869 fp=#0x10000004cce' 2024-09-21T00:30:05.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29901 fp=#0x20000000dcc' 2024-09-21T00:30:05.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29933 fp=#0x20000000d96' 2024-09-21T00:30:05.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29965 fp=#0x20000000e01' 2024-09-21T00:30:05.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29997 fp=#0x20000000d42' 2024-09-21T00:30:05.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30029 fp=#0x20000000e43' 2024-09-21T00:30:05.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30061 fp=#0x20000000c2d' 2024-09-21T00:30:05.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30189 fp=#0x10000001c89' 2024-09-21T00:30:05.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30221 fp=#0x100000055b6' 2024-09-21T00:30:05.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30253 fp=#0x10000001c52' 2024-09-21T00:30:05.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30285 fp=#0x10000002990' 2024-09-21T00:30:05.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30317 fp=#0x20000000b3c' 2024-09-21T00:30:05.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30349 fp=#0x20000000b42' 2024-09-21T00:30:05.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30381 fp=#0x20000000af0' 2024-09-21T00:30:05.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30413 fp=#0x100000028bb' 2024-09-21T00:30:05.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30445 fp=#0x20000000b47' 2024-09-21T00:30:05.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30477 fp=#0x100000023df' 2024-09-21T00:30:05.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30509 fp=#0x200000006f6' 2024-09-21T00:30:05.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30541 fp=#0x100000023b7' 2024-09-21T00:30:05.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22766 fp=#0x10000002b46' 2024-09-21T00:30:05.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22830 fp=#0x10000002966' 2024-09-21T00:30:06.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26638 fp=#0x2000000079f' 2024-09-21T00:30:06.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26670 fp=#0x2000000076f' 2024-09-21T00:30:06.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26702 fp=#0x200000007fe' 2024-09-21T00:30:06.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26734 fp=#0x2000000078d' 2024-09-21T00:30:06.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26798 fp=#0x10000004481' 2024-09-21T00:30:06.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26830 fp=#0x10000004612' 2024-09-21T00:30:06.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26862 fp=#0x100000043e2' 2024-09-21T00:30:06.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26894 fp=#0x10000004464' 2024-09-21T00:30:06.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26926 fp=#0x10000004620' 2024-09-21T00:30:06.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26958 fp=#0x20000000ff1' 2024-09-21T00:30:06.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26990 fp=#0x20000000ee1' 2024-09-21T00:30:06.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27022 fp=#0x200000010be' 2024-09-21T00:30:06.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27054 fp=#0x20000000d94' 2024-09-21T00:30:06.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27086 fp=#0x20000000d62' 2024-09-21T00:30:06.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27118 fp=#0x20000000de4' 2024-09-21T00:30:06.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27150 fp=#0x1000000220f' 2024-09-21T00:30:06.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27182 fp=#0x20000000f10' 2024-09-21T00:30:06.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27214 fp=#0x1000000228e' 2024-09-21T00:30:06.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27246 fp=#0x20000000fda' 2024-09-21T00:30:06.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27278 fp=#0x1000000223f' 2024-09-21T00:30:06.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27310 fp=#0x10000002369' 2024-09-21T00:30:06.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27342 fp=#0x10000002200' 2024-09-21T00:30:06.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27374 fp=#0x10000003cdf' 2024-09-21T00:30:06.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27438 fp=#0x10000003c47' 2024-09-21T00:30:06.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27470 fp=#0x10000003bdb' 2024-09-21T00:30:06.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27502 fp=#0x10000003bd8' 2024-09-21T00:30:06.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27630 fp=#0x10000002013' 2024-09-21T00:30:06.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27662 fp=#0x10000002025' 2024-09-21T00:30:06.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27694 fp=#0x10000002062' 2024-09-21T00:30:06.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27726 fp=#0x10000001ea8' 2024-09-21T00:30:06.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27758 fp=#0x200000008d2' 2024-09-21T00:30:06.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27790 fp=#0x2000000095e' 2024-09-21T00:30:06.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27822 fp=#0x20000001013' 2024-09-21T00:30:06.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27886 fp=#0x200000008f6' 2024-09-21T00:30:06.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27918 fp=#0x200000010c7' 2024-09-21T00:30:06.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27950 fp=#0x2000000105f' 2024-09-21T00:30:06.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27982 fp=#0x10000004aa8' 2024-09-21T00:30:06.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28014 fp=#0x100000049bc' 2024-09-21T00:30:06.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28046 fp=#0x1000000499b' 2024-09-21T00:30:06.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28078 fp=#0x200000012c1' 2024-09-21T00:30:06.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28110 fp=#0x200000012ac' 2024-09-21T00:30:06.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28142 fp=#0x10000004bde' 2024-09-21T00:30:06.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28238 fp=#0x10000004b0e' 2024-09-21T00:30:06.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28270 fp=#0x100000032f8' 2024-09-21T00:30:06.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28302 fp=#0x1000000307c' 2024-09-21T00:30:06.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28334 fp=#0x20000000e7e' 2024-09-21T00:30:06.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28366 fp=#0x100000030b9' 2024-09-21T00:30:06.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28398 fp=#0x1000000324a' 2024-09-21T00:30:06.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28430 fp=#0x10000003345' 2024-09-21T00:30:06.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28462 fp=#0x100000032eb' 2024-09-21T00:30:06.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28494 fp=#0x10000004bca' 2024-09-21T00:30:06.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28526 fp=#0x20000001017' 2024-09-21T00:30:06.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28558 fp=#0x20000001198' 2024-09-21T00:30:06.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28654 fp=#0x20000000e3c' 2024-09-21T00:30:06.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28686 fp=#0x20000000d7e' 2024-09-21T00:30:06.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28718 fp=#0x100000015eb' 2024-09-21T00:30:06.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28750 fp=#0x1000000169f' 2024-09-21T00:30:06.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28782 fp=#0x100000015dc' 2024-09-21T00:30:06.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28814 fp=#0x10000001609' 2024-09-21T00:30:06.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28846 fp=#0x200000009ed' 2024-09-21T00:30:06.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28910 fp=#0x2000000030d' 2024-09-21T00:30:06.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28942 fp=#0x100000016d5' 2024-09-21T00:30:06.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28974 fp=#0x10000001645' 2024-09-21T00:30:06.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29006 fp=#0x100000016da' 2024-09-21T00:30:06.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29038 fp=#0x100000016e1' 2024-09-21T00:30:06.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29070 fp=#0x1000000435b' 2024-09-21T00:30:06.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29102 fp=#0x100000043bf' 2024-09-21T00:30:06.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29134 fp=#0x10000004383' 2024-09-21T00:30:06.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29166 fp=#0x1000000438a' 2024-09-21T00:30:06.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29198 fp=#0x1000000437b' 2024-09-21T00:30:06.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29230 fp=#0x100000043c2' 2024-09-21T00:30:06.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29262 fp=#0x1000000437d' 2024-09-21T00:30:06.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29518 fp=#0x10000004c73' 2024-09-21T00:30:06.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29550 fp=#0x10000004df1' 2024-09-21T00:30:06.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29582 fp=#0x20000000eb9' 2024-09-21T00:30:06.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29614 fp=#0x100000040bd' 2024-09-21T00:30:06.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29646 fp=#0x100000040aa' 2024-09-21T00:30:06.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29710 fp=#0x1000000297d' 2024-09-21T00:30:06.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29742 fp=#0x10000004577' 2024-09-21T00:30:06.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29774 fp=#0x1000000456d' 2024-09-21T00:30:06.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29806 fp=#0x1000000451e' 2024-09-21T00:30:06.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29838 fp=#0x10000004560' 2024-09-21T00:30:06.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29870 fp=#0x10000004d6f' 2024-09-21T00:30:06.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29902 fp=#0x20000000cfd' 2024-09-21T00:30:06.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29934 fp=#0x20000000dbb' 2024-09-21T00:30:06.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29966 fp=#0x20000000d33' 2024-09-21T00:30:06.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29998 fp=#0x20000000d58' 2024-09-21T00:30:06.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30030 fp=#0x20000000dbe' 2024-09-21T00:30:06.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30126 fp=#0x20000000a65' 2024-09-21T00:30:06.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30158 fp=#0x20000000be7' 2024-09-21T00:30:06.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30190 fp=#0x10000001c75' 2024-09-21T00:30:06.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30222 fp=#0x10000001c7d' 2024-09-21T00:30:06.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30254 fp=#0x10000002bad' 2024-09-21T00:30:06.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30286 fp=#0x100000028c9' 2024-09-21T00:30:06.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30318 fp=#0x20000000b36' 2024-09-21T00:30:06.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30350 fp=#0x20000000bc2' 2024-09-21T00:30:06.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30382 fp=#0x20000000b23' 2024-09-21T00:30:06.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30414 fp=#0x20000000a97' 2024-09-21T00:30:06.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30446 fp=#0x20000000b2c' 2024-09-21T00:30:06.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30478 fp=#0x1000000236d' 2024-09-21T00:30:06.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22831 fp=#0x10000004512' 2024-09-21T00:30:06.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26639 fp=#0x2000000076d' 2024-09-21T00:30:06.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26671 fp=#0x20000000800' 2024-09-21T00:30:06.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26703 fp=#0x200000007f3' 2024-09-21T00:30:06.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26799 fp=#0x100000043df' 2024-09-21T00:30:06.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26831 fp=#0x10000004452' 2024-09-21T00:30:06.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26863 fp=#0x10000004637' 2024-09-21T00:30:06.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26895 fp=#0x100000043f3' 2024-09-21T00:30:06.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26927 fp=#0x10000004443' 2024-09-21T00:30:06.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26959 fp=#0x20000000ef5' 2024-09-21T00:30:06.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26991 fp=#0x20000000ef8' 2024-09-21T00:30:06.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27023 fp=#0x20000000fa6' 2024-09-21T00:30:06.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27055 fp=#0x20000000f5f' 2024-09-21T00:30:06.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27087 fp=#0x20000000e03' 2024-09-21T00:30:06.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27119 fp=#0x20000000ed1' 2024-09-21T00:30:06.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27151 fp=#0x1000000227e' 2024-09-21T00:30:06.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27183 fp=#0x100000023bf' 2024-09-21T00:30:06.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27215 fp=#0x20000000f23' 2024-09-21T00:30:06.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27247 fp=#0x1000000226e' 2024-09-21T00:30:06.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27279 fp=#0x2000000100d' 2024-09-21T00:30:06.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27311 fp=#0x10000002259' 2024-09-21T00:30:06.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27343 fp=#0x1000000225a' 2024-09-21T00:30:06.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27375 fp=#0x10000003bcf' 2024-09-21T00:30:06.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27407 fp=#0x10000003bfc' 2024-09-21T00:30:06.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27439 fp=#0x10000003b61' 2024-09-21T00:30:06.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27471 fp=#0x10000003c76' 2024-09-21T00:30:06.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27503 fp=#0x10000003ca8' 2024-09-21T00:30:06.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27631 fp=#0x10000001ec3' 2024-09-21T00:30:06.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27663 fp=#0x10000001e96' 2024-09-21T00:30:06.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27695 fp=#0x10000001feb' 2024-09-21T00:30:06.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27727 fp=#0x10000001fb9' 2024-09-21T00:30:06.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27791 fp=#0x2000000096a' 2024-09-21T00:30:06.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27823 fp=#0x200000008e6' 2024-09-21T00:30:06.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27855 fp=#0x2000000094a' 2024-09-21T00:30:06.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27887 fp=#0x20000001281' 2024-09-21T00:30:06.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27919 fp=#0x200000010c1' 2024-09-21T00:30:06.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27951 fp=#0x2000000139a' 2024-09-21T00:30:06.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27983 fp=#0x10000004a31' 2024-09-21T00:30:06.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28015 fp=#0x10000004a97' 2024-09-21T00:30:06.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28047 fp=#0x10000004974' 2024-09-21T00:30:06.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28079 fp=#0x200000012e8' 2024-09-21T00:30:06.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28111 fp=#0x200000012dd' 2024-09-21T00:30:06.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28143 fp=#0x10000004bcb' 2024-09-21T00:30:06.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28175 fp=#0x10000004c0f' 2024-09-21T00:30:06.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28239 fp=#0x10000004bdf' 2024-09-21T00:30:06.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28271 fp=#0x1000000301e' 2024-09-21T00:30:06.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28303 fp=#0x10000004c4b' 2024-09-21T00:30:06.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28335 fp=#0x10000003160' 2024-09-21T00:30:06.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28367 fp=#0x20000000e65' 2024-09-21T00:30:06.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28399 fp=#0x100000032b0' 2024-09-21T00:30:06.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28431 fp=#0x100000032a5' 2024-09-21T00:30:06.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28463 fp=#0x10000004bd2' 2024-09-21T00:30:06.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28495 fp=#0x20000000e69' 2024-09-21T00:30:06.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28559 fp=#0x20000000f40' 2024-09-21T00:30:06.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28591 fp=#0x20000000f58' 2024-09-21T00:30:06.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28623 fp=#0x20000000f83' 2024-09-21T00:30:06.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28655 fp=#0x20000000e45' 2024-09-21T00:30:06.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28687 fp=#0x20000000d6b' 2024-09-21T00:30:06.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28719 fp=#0x10000001622' 2024-09-21T00:30:06.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28751 fp=#0x20000000a0a' 2024-09-21T00:30:06.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28783 fp=#0x10000001644' 2024-09-21T00:30:06.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28815 fp=#0x1000000166f' 2024-09-21T00:30:06.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28847 fp=#0x200000009d1' 2024-09-21T00:30:06.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28879 fp=#0x200000009c9' 2024-09-21T00:30:06.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28911 fp=#0x1000000164d' 2024-09-21T00:30:06.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28943 fp=#0x20000000308' 2024-09-21T00:30:06.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28975 fp=#0x10000001696' 2024-09-21T00:30:06.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29007 fp=#0x100000016fb' 2024-09-21T00:30:06.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29039 fp=#0x100000016d9' 2024-09-21T00:30:06.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29071 fp=#0x10000004343' 2024-09-21T00:30:06.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29103 fp=#0x100000043b6' 2024-09-21T00:30:06.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29135 fp=#0x10000004387' 2024-09-21T00:30:06.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29167 fp=#0x10000004392' 2024-09-21T00:30:06.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29199 fp=#0x100000043aa' 2024-09-21T00:30:06.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29231 fp=#0x10000004366' 2024-09-21T00:30:06.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29263 fp=#0x100000043a3' 2024-09-21T00:30:06.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29583 fp=#0x100000040d8' 2024-09-21T00:30:06.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29615 fp=#0x100000040be' 2024-09-21T00:30:06.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29679 fp=#0x100000040ae' 2024-09-21T00:30:06.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29711 fp=#0x10000002a57' 2024-09-21T00:30:06.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29743 fp=#0x10000004525' 2024-09-21T00:30:06.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29775 fp=#0x10000004571' 2024-09-21T00:30:06.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29807 fp=#0x10000004521' 2024-09-21T00:30:06.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29839 fp=#0x1000000455c' 2024-09-21T00:30:06.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29903 fp=#0x20000000db9' 2024-09-21T00:30:06.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29935 fp=#0x20000000cef' 2024-09-21T00:30:06.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29967 fp=#0x20000000e6b' 2024-09-21T00:30:06.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29999 fp=#0x200000011e5' 2024-09-21T00:30:06.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30031 fp=#0x20000000e5a' 2024-09-21T00:30:06.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30063 fp=#0x20000000a68' 2024-09-21T00:30:06.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30127 fp=#0x20000000a2f' 2024-09-21T00:30:06.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30191 fp=#0x10000001b90' 2024-09-21T00:30:06.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30223 fp=#0x10000001c08' 2024-09-21T00:30:06.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30255 fp=#0x10000001c46' 2024-09-21T00:30:06.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30287 fp=#0x100000028c1' 2024-09-21T00:30:06.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30319 fp=#0x1000000296f' 2024-09-21T00:30:06.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30351 fp=#0x20000000aac' 2024-09-21T00:30:06.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30383 fp=#0x20000000a74' 2024-09-21T00:30:06.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30415 fp=#0x1000000291e' 2024-09-21T00:30:06.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30447 fp=#0x100000028d1' 2024-09-21T00:30:06.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30543 fp=#0x100000023a4' 2024-09-21T00:30:06.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22768 fp=#0x10000002e6a' 2024-09-21T00:30:06.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22832 fp=#0x10000004cc7' 2024-09-21T00:30:06.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:25296 fp=#0x200000013af' 2024-09-21T00:30:06.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26640 fp=#0x200000007f4' 2024-09-21T00:30:06.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26704 fp=#0x20000000803' 2024-09-21T00:30:06.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26768 fp=#0x200000007fd' 2024-09-21T00:30:06.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26800 fp=#0x1000000447d' 2024-09-21T00:30:06.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26896 fp=#0x100000043ee' 2024-09-21T00:30:06.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26928 fp=#0x10000004633' 2024-09-21T00:30:06.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26960 fp=#0x20000000d5c' 2024-09-21T00:30:06.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26992 fp=#0x20000000e07' 2024-09-21T00:30:06.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27024 fp=#0x20000000ffe' 2024-09-21T00:30:06.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27056 fp=#0x20000000f85' 2024-09-21T00:30:06.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27088 fp=#0x20000000db8' 2024-09-21T00:30:06.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27120 fp=#0x20000000e12' 2024-09-21T00:30:06.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27152 fp=#0x10000002511' 2024-09-21T00:30:06.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27184 fp=#0x10000002509' 2024-09-21T00:30:06.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27216 fp=#0x20000000f15' 2024-09-21T00:30:06.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27248 fp=#0x20000001007' 2024-09-21T00:30:06.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27280 fp=#0x100000023fa' 2024-09-21T00:30:06.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27312 fp=#0x20000000f93' 2024-09-21T00:30:06.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27344 fp=#0x100000023e6' 2024-09-21T00:30:06.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27376 fp=#0x20000001162' 2024-09-21T00:30:06.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27408 fp=#0x10000003ccb' 2024-09-21T00:30:06.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27440 fp=#0x10000003c32' 2024-09-21T00:30:06.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27472 fp=#0x10000003c8a' 2024-09-21T00:30:06.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27504 fp=#0x10000003c80' 2024-09-21T00:30:06.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27632 fp=#0x10000001faa' 2024-09-21T00:30:06.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27696 fp=#0x10000001f5b' 2024-09-21T00:30:06.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27728 fp=#0x10000001fa5' 2024-09-21T00:30:06.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27792 fp=#0x200000008e3' 2024-09-21T00:30:06.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27824 fp=#0x2000000127b' 2024-09-21T00:30:06.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27920 fp=#0x20000001136' 2024-09-21T00:30:06.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27952 fp=#0x2000000105a' 2024-09-21T00:30:06.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27984 fp=#0x100000049d3' 2024-09-21T00:30:06.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28016 fp=#0x10000004a51' 2024-09-21T00:30:06.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28048 fp=#0x100000049e3' 2024-09-21T00:30:06.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28080 fp=#0x200000012bd' 2024-09-21T00:30:06.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28112 fp=#0x2000000133b' 2024-09-21T00:30:06.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28144 fp=#0x10000004bf5' 2024-09-21T00:30:06.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28176 fp=#0x10000004bb1' 2024-09-21T00:30:06.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28208 fp=#0x10000004ae5' 2024-09-21T00:30:06.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28240 fp=#0x10000004b0a' 2024-09-21T00:30:06.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28272 fp=#0x10000003098' 2024-09-21T00:30:06.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28304 fp=#0x100000032bb' 2024-09-21T00:30:06.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28336 fp=#0x10000004c5a' 2024-09-21T00:30:06.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28368 fp=#0x20000000eab' 2024-09-21T00:30:06.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28400 fp=#0x10000004c62' 2024-09-21T00:30:06.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28432 fp=#0x20000000e4d' 2024-09-21T00:30:06.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28464 fp=#0x20000000e88' 2024-09-21T00:30:06.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28496 fp=#0x10000004c57' 2024-09-21T00:30:06.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28528 fp=#0x20000000f77' 2024-09-21T00:30:06.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28592 fp=#0x200000011b8' 2024-09-21T00:30:06.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28624 fp=#0x20000001192' 2024-09-21T00:30:06.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28656 fp=#0x20000000d4c' 2024-09-21T00:30:06.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28688 fp=#0x20000000d40' 2024-09-21T00:30:06.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28720 fp=#0x1000000166a' 2024-09-21T00:30:06.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28752 fp=#0x100000015d9' 2024-09-21T00:30:06.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28784 fp=#0x1000000160e' 2024-09-21T00:30:06.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28816 fp=#0x10000001678' 2024-09-21T00:30:06.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28880 fp=#0x200000009c4' 2024-09-21T00:30:06.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28912 fp=#0x200000003e7' 2024-09-21T00:30:06.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28944 fp=#0x1000000175e' 2024-09-21T00:30:06.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28976 fp=#0x10000001649' 2024-09-21T00:30:06.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29008 fp=#0x100000016c3' 2024-09-21T00:30:06.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29040 fp=#0x20000000843' 2024-09-21T00:30:06.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29072 fp=#0x1000000439a' 2024-09-21T00:30:06.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29104 fp=#0x1000000439e' 2024-09-21T00:30:06.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29136 fp=#0x2000000140f' 2024-09-21T00:30:06.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29168 fp=#0x100000043a4' 2024-09-21T00:30:06.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29200 fp=#0x100000043ae' 2024-09-21T00:30:06.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29264 fp=#0x10000004388' 2024-09-21T00:30:06.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29392 fp=#0x10000004e35' 2024-09-21T00:30:06.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29424 fp=#0x10000004e04' 2024-09-21T00:30:06.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29456 fp=#0x10000004e00' 2024-09-21T00:30:06.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29488 fp=#0x2000000102b' 2024-09-21T00:30:06.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29520 fp=#0x10000004e1d' 2024-09-21T00:30:06.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29648 fp=#0x20000000ec2' 2024-09-21T00:30:06.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29712 fp=#0x100000040cc' 2024-09-21T00:30:06.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29744 fp=#0x1000000455d' 2024-09-21T00:30:06.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29776 fp=#0x10000004569' 2024-09-21T00:30:06.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29808 fp=#0x10000004552' 2024-09-21T00:30:06.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29840 fp=#0x10000004575' 2024-09-21T00:30:06.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29904 fp=#0x20000000cbd' 2024-09-21T00:30:06.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29936 fp=#0x20000000d70' 2024-09-21T00:30:06.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29968 fp=#0x20000000e4f' 2024-09-21T00:30:06.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30000 fp=#0x20000000ca9' 2024-09-21T00:30:06.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30032 fp=#0x20000000d4a' 2024-09-21T00:30:06.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30064 fp=#0x20000000c15' 2024-09-21T00:30:06.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30160 fp=#0x10000001b80' 2024-09-21T00:30:06.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30192 fp=#0x10000003d1b' 2024-09-21T00:30:06.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30224 fp=#0x10000001b22' 2024-09-21T00:30:06.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30256 fp=#0x10000001b00' 2024-09-21T00:30:06.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30288 fp=#0x100000029ab' 2024-09-21T00:30:06.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30320 fp=#0x10000002912' 2024-09-21T00:30:06.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30352 fp=#0x100000028c5' 2024-09-21T00:30:06.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30416 fp=#0x100000028e9' 2024-09-21T00:30:06.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30448 fp=#0x10000002969' 2024-09-21T00:30:06.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30512 fp=#0x100000023d1' 2024-09-21T00:30:06.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22833 fp=#0x20000000c5b' 2024-09-21T00:30:06.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26705 fp=#0x20000000a0e' 2024-09-21T00:30:06.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26737 fp=#0x200000007f9' 2024-09-21T00:30:06.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26769 fp=#0x200000007db' 2024-09-21T00:30:06.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26801 fp=#0x10000004635' 2024-09-21T00:30:06.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26833 fp=#0x1000000447f' 2024-09-21T00:30:06.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26897 fp=#0x100000043e4' 2024-09-21T00:30:06.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26929 fp=#0x1000000462b' 2024-09-21T00:30:06.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26961 fp=#0x20000000f75' 2024-09-21T00:30:06.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26993 fp=#0x20000000eef' 2024-09-21T00:30:06.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27121 fp=#0x20000000f72' 2024-09-21T00:30:06.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27153 fp=#0x10000002225' 2024-09-21T00:30:06.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27185 fp=#0x100000022a5' 2024-09-21T00:30:06.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27217 fp=#0x20000001026' 2024-09-21T00:30:06.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27249 fp=#0x10000002239' 2024-09-21T00:30:06.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27281 fp=#0x10000002303' 2024-09-21T00:30:06.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27313 fp=#0x1000000220b' 2024-09-21T00:30:06.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27345 fp=#0x1000000220a' 2024-09-21T00:30:06.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27377 fp=#0x10000003cdc' 2024-09-21T00:30:06.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27409 fp=#0x10000003cb2' 2024-09-21T00:30:06.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27441 fp=#0x20000001145' 2024-09-21T00:30:06.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27473 fp=#0x10000003c41' 2024-09-21T00:30:06.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27505 fp=#0x10000003bcc' 2024-09-21T00:30:06.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27633 fp=#0x10000002005' 2024-09-21T00:30:06.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27665 fp=#0x10000001fe0' 2024-09-21T00:30:06.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27697 fp=#0x10000001eaa' 2024-09-21T00:30:06.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27729 fp=#0x10000001e9d' 2024-09-21T00:30:06.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27761 fp=#0x200000008c5' 2024-09-21T00:30:06.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27825 fp=#0x2000000094e' 2024-09-21T00:30:06.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27889 fp=#0x20000000974' 2024-09-21T00:30:06.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27921 fp=#0x2000000106d' 2024-09-21T00:30:06.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27953 fp=#0x20000001099' 2024-09-21T00:30:06.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27985 fp=#0x10000004a79' 2024-09-21T00:30:06.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28017 fp=#0x10000004a3f' 2024-09-21T00:30:06.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28049 fp=#0x10000004980' 2024-09-21T00:30:06.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28081 fp=#0x20000001363' 2024-09-21T00:30:06.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28113 fp=#0x2000000126f' 2024-09-21T00:30:06.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28145 fp=#0x10000004b75' 2024-09-21T00:30:06.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28177 fp=#0x10000004b9f' 2024-09-21T00:30:06.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28241 fp=#0x10000004b10' 2024-09-21T00:30:06.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28273 fp=#0x100000030e9' 2024-09-21T00:30:06.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28305 fp=#0x100000032f0' 2024-09-21T00:30:06.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28337 fp=#0x10000004c4e' 2024-09-21T00:30:06.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28369 fp=#0x10000003120' 2024-09-21T00:30:06.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28401 fp=#0x1000000313d' 2024-09-21T00:30:06.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28433 fp=#0x100000030d0' 2024-09-21T00:30:06.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28465 fp=#0x10000004c5c' 2024-09-21T00:30:06.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28497 fp=#0x10000003138' 2024-09-21T00:30:06.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28529 fp=#0x20000001217' 2024-09-21T00:30:06.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28561 fp=#0x20000000f7a' 2024-09-21T00:30:06.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28625 fp=#0x20000000ffd' 2024-09-21T00:30:06.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28657 fp=#0x20000000e0f' 2024-09-21T00:30:06.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28689 fp=#0x20000000e0b' 2024-09-21T00:30:06.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28721 fp=#0x1000000164a' 2024-09-21T00:30:06.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28753 fp=#0x100000016b7' 2024-09-21T00:30:06.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28785 fp=#0x100000016bb' 2024-09-21T00:30:06.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28817 fp=#0x100000016ba' 2024-09-21T00:30:06.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28913 fp=#0x2000000031c' 2024-09-21T00:30:06.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28945 fp=#0x100000016cb' 2024-09-21T00:30:06.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28977 fp=#0x2000000050a' 2024-09-21T00:30:06.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29009 fp=#0x20000000314' 2024-09-21T00:30:06.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29041 fp=#0x100000016df' 2024-09-21T00:30:06.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29073 fp=#0x100000043d0' 2024-09-21T00:30:06.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29105 fp=#0x10000004382' 2024-09-21T00:30:06.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29265 fp=#0x200000013db' 2024-09-21T00:30:06.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29393 fp=#0x20000001030' 2024-09-21T00:30:06.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29489 fp=#0x10000004c90' 2024-09-21T00:30:06.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29521 fp=#0x10000004c7c' 2024-09-21T00:30:06.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29585 fp=#0x100000040c7' 2024-09-21T00:30:06.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29617 fp=#0x100000040a6' 2024-09-21T00:30:06.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29649 fp=#0x20000000eaf' 2024-09-21T00:30:06.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29713 fp=#0x100000040ad' 2024-09-21T00:30:06.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29745 fp=#0x10000004576' 2024-09-21T00:30:06.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29777 fp=#0x10000004538' 2024-09-21T00:30:06.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29809 fp=#0x10000004533' 2024-09-21T00:30:06.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29841 fp=#0x10000004516' 2024-09-21T00:30:06.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29873 fp=#0x10000004ceb' 2024-09-21T00:30:06.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29905 fp=#0x20000000e71' 2024-09-21T00:30:06.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29937 fp=#0x20000000d35' 2024-09-21T00:30:06.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29969 fp=#0x20000000d88' 2024-09-21T00:30:06.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30001 fp=#0x20000000c8a' 2024-09-21T00:30:06.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30033 fp=#0x20000000d90' 2024-09-21T00:30:06.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30065 fp=#0x20000000be5' 2024-09-21T00:30:06.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30097 fp=#0x20000000c6f' 2024-09-21T00:30:06.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30161 fp=#0x10000001af4' 2024-09-21T00:30:06.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30193 fp=#0x10000001c6f' 2024-09-21T00:30:06.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30225 fp=#0x10000001b2e' 2024-09-21T00:30:06.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30257 fp=#0x10000005238' 2024-09-21T00:30:06.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30289 fp=#0x100000028dd' 2024-09-21T00:30:06.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30321 fp=#0x20000000a9c' 2024-09-21T00:30:06.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30353 fp=#0x100000028d9' 2024-09-21T00:30:06.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30385 fp=#0x100000028e1' 2024-09-21T00:30:06.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30417 fp=#0x100000028cf' 2024-09-21T00:30:06.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30449 fp=#0x100000028cd' 2024-09-21T00:30:06.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22834 fp=#0x20000000a0c' 2024-09-21T00:30:06.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26706 fp=#0x2000000082e' 2024-09-21T00:30:06.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26738 fp=#0x20000000802' 2024-09-21T00:30:06.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26802 fp=#0x1000000462c' 2024-09-21T00:30:06.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26834 fp=#0x1000000445c' 2024-09-21T00:30:06.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26898 fp=#0x10000004471' 2024-09-21T00:30:06.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26930 fp=#0x10000004626' 2024-09-21T00:30:06.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26962 fp=#0x20000000dad' 2024-09-21T00:30:06.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26994 fp=#0x20000000ee5' 2024-09-21T00:30:06.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27026 fp=#0x20000000d72' 2024-09-21T00:30:06.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27058 fp=#0x20000000da5' 2024-09-21T00:30:06.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27122 fp=#0x20000000ee4' 2024-09-21T00:30:06.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27154 fp=#0x10000002253' 2024-09-21T00:30:06.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27186 fp=#0x100000023f6' 2024-09-21T00:30:06.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27218 fp=#0x1000000223d' 2024-09-21T00:30:06.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27250 fp=#0x1000000231b' 2024-09-21T00:30:06.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27282 fp=#0x10000002289' 2024-09-21T00:30:06.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27314 fp=#0x1000000233e' 2024-09-21T00:30:06.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27346 fp=#0x10000002257' 2024-09-21T00:30:06.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27378 fp=#0x10000003b6d' 2024-09-21T00:30:06.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27410 fp=#0x10000003c4a' 2024-09-21T00:30:06.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27442 fp=#0x10000003b5f' 2024-09-21T00:30:06.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27474 fp=#0x10000003b7a' 2024-09-21T00:30:06.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27506 fp=#0x10000003cd7' 2024-09-21T00:30:06.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27634 fp=#0x10000001f14' 2024-09-21T00:30:06.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27666 fp=#0x10000002488' 2024-09-21T00:30:06.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27698 fp=#0x10000001eb5' 2024-09-21T00:30:06.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27730 fp=#0x10000001f39' 2024-09-21T00:30:06.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27762 fp=#0x200000008db' 2024-09-21T00:30:06.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27826 fp=#0x2000000100f' 2024-09-21T00:30:06.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27858 fp=#0x20000000955' 2024-09-21T00:30:06.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27890 fp=#0x2000000102d' 2024-09-21T00:30:06.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27922 fp=#0x20000001088' 2024-09-21T00:30:06.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27954 fp=#0x2000000108d' 2024-09-21T00:30:06.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27986 fp=#0x10000004998' 2024-09-21T00:30:06.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28018 fp=#0x100000049ea' 2024-09-21T00:30:06.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28050 fp=#0x100000049de' 2024-09-21T00:30:06.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28082 fp=#0x200000012e2' 2024-09-21T00:30:06.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28114 fp=#0x20000001293' 2024-09-21T00:30:06.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28146 fp=#0x10000004be5' 2024-09-21T00:30:06.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28178 fp=#0x10000004aba' 2024-09-21T00:30:06.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28210 fp=#0x10000004be9' 2024-09-21T00:30:06.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28242 fp=#0x10000004b5c' 2024-09-21T00:30:06.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28274 fp=#0x100000030da' 2024-09-21T00:30:06.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28306 fp=#0x10000003186' 2024-09-21T00:30:06.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28338 fp=#0x20000000f3a' 2024-09-21T00:30:06.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28370 fp=#0x20000000f14' 2024-09-21T00:30:06.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28402 fp=#0x20000000e77' 2024-09-21T00:30:06.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28434 fp=#0x20000000e54' 2024-09-21T00:30:06.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28466 fp=#0x20000000e40' 2024-09-21T00:30:06.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28498 fp=#0x10000003150' 2024-09-21T00:30:06.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28530 fp=#0x20000000fd7' 2024-09-21T00:30:06.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28562 fp=#0x20000000fa8' 2024-09-21T00:30:06.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28594 fp=#0x20000001000' 2024-09-21T00:30:06.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28626 fp=#0x20000001174' 2024-09-21T00:30:06.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28658 fp=#0x20000000e2f' 2024-09-21T00:30:06.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28690 fp=#0x20000000d82' 2024-09-21T00:30:06.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28722 fp=#0x10000001613' 2024-09-21T00:30:06.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28754 fp=#0x1000000163b' 2024-09-21T00:30:06.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28786 fp=#0x100000016ae' 2024-09-21T00:30:06.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28818 fp=#0x20000000424' 2024-09-21T00:30:06.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28850 fp=#0x200000009cd' 2024-09-21T00:30:06.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28882 fp=#0x200000009ef' 2024-09-21T00:30:06.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28914 fp=#0x10000001750' 2024-09-21T00:30:06.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28946 fp=#0x10000001722' 2024-09-21T00:30:06.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28978 fp=#0x10000001701' 2024-09-21T00:30:06.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29010 fp=#0x10000001712' 2024-09-21T00:30:06.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29042 fp=#0x10000001628' 2024-09-21T00:30:06.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29074 fp=#0x1000000436a' 2024-09-21T00:30:06.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29106 fp=#0x100000043ad' 2024-09-21T00:30:06.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29138 fp=#0x100000043a1' 2024-09-21T00:30:06.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29170 fp=#0x10000004399' 2024-09-21T00:30:06.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29202 fp=#0x1000000439b' 2024-09-21T00:30:06.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29234 fp=#0x1000000438c' 2024-09-21T00:30:06.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29266 fp=#0x100000043b5' 2024-09-21T00:30:06.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29394 fp=#0x10000004e26' 2024-09-21T00:30:06.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29426 fp=#0x10000004de9' 2024-09-21T00:30:06.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29522 fp=#0x20000001014' 2024-09-21T00:30:06.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29554 fp=#0x10000004e2c' 2024-09-21T00:30:06.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29586 fp=#0x100000040a5' 2024-09-21T00:30:06.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29650 fp=#0x100000040c1' 2024-09-21T00:30:06.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29682 fp=#0x10000002a51' 2024-09-21T00:30:06.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29714 fp=#0x100000040b6' 2024-09-21T00:30:06.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29746 fp=#0x1000000453c' 2024-09-21T00:30:06.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29778 fp=#0x10000004570' 2024-09-21T00:30:06.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29810 fp=#0x10000004559' 2024-09-21T00:30:06.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29842 fp=#0x10000004545' 2024-09-21T00:30:06.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29906 fp=#0x20000000da8' 2024-09-21T00:30:06.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29938 fp=#0x20000000ce2' 2024-09-21T00:30:06.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29970 fp=#0x20000000e49' 2024-09-21T00:30:06.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30002 fp=#0x20000000ddd' 2024-09-21T00:30:06.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30034 fp=#0x20000000caf' 2024-09-21T00:30:06.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30162 fp=#0x10000001b41' 2024-09-21T00:30:06.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30194 fp=#0x10000001b5b' 2024-09-21T00:30:06.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30226 fp=#0x10000001c1f' 2024-09-21T00:30:06.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30258 fp=#0x10000001b6a' 2024-09-21T00:30:06.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30290 fp=#0x100000029a0' 2024-09-21T00:30:06.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30322 fp=#0x20000000a9f' 2024-09-21T00:30:06.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30354 fp=#0x100000029b3' 2024-09-21T00:30:06.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30386 fp=#0x1000000294c' 2024-09-21T00:30:06.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30418 fp=#0x20000000b0d' 2024-09-21T00:30:06.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30450 fp=#0x100000028d3' 2024-09-21T00:30:06.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30482 fp=#0x100000023ca' 2024-09-21T00:30:06.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30514 fp=#0x200000006f1' 2024-09-21T00:30:06.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22835 fp=#0x10000003351' 2024-09-21T00:30:06.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:25235 fp=#0x200000013b9' 2024-09-21T00:30:06.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26707 fp=#0x20000000a17' 2024-09-21T00:30:06.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26771 fp=#0x200000007fc' 2024-09-21T00:30:06.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26835 fp=#0x20000000833' 2024-09-21T00:30:06.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26867 fp=#0x10000004613' 2024-09-21T00:30:06.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26931 fp=#0x100000043d9' 2024-09-21T00:30:06.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26963 fp=#0x20000000f0b' 2024-09-21T00:30:06.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26995 fp=#0x20000000ecb' 2024-09-21T00:30:06.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27027 fp=#0x20000000d98' 2024-09-21T00:30:06.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27091 fp=#0x20000000d85' 2024-09-21T00:30:06.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27123 fp=#0x20000000ed0' 2024-09-21T00:30:06.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27155 fp=#0x10000002248' 2024-09-21T00:30:06.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27187 fp=#0x100000023cb' 2024-09-21T00:30:06.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27219 fp=#0x1000000229e' 2024-09-21T00:30:06.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27251 fp=#0x1000000234d' 2024-09-21T00:30:06.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27283 fp=#0x1000000239f' 2024-09-21T00:30:06.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27315 fp=#0x10000002400' 2024-09-21T00:30:06.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27347 fp=#0x1000000225c' 2024-09-21T00:30:06.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27379 fp=#0x10000003bb4' 2024-09-21T00:30:06.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27411 fp=#0x10000003bf0' 2024-09-21T00:30:06.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27443 fp=#0x10000003b69' 2024-09-21T00:30:06.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27475 fp=#0x10000003c2f' 2024-09-21T00:30:06.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27507 fp=#0x10000003b86' 2024-09-21T00:30:06.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27635 fp=#0x10000001f82' 2024-09-21T00:30:06.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27667 fp=#0x200000011b7' 2024-09-21T00:30:06.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27699 fp=#0x10000001f79' 2024-09-21T00:30:06.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27731 fp=#0x10000001ee5' 2024-09-21T00:30:06.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27763 fp=#0x200000008f1' 2024-09-21T00:30:06.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27859 fp=#0x200000008ea' 2024-09-21T00:30:06.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27891 fp=#0x200000008d6' 2024-09-21T00:30:06.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27923 fp=#0x20000001072' 2024-09-21T00:30:06.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27955 fp=#0x20000001055' 2024-09-21T00:30:06.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27987 fp=#0x100000049a1' 2024-09-21T00:30:06.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28019 fp=#0x10000004965' 2024-09-21T00:30:06.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28051 fp=#0x100000049e5' 2024-09-21T00:30:06.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28083 fp=#0x2000000127c' 2024-09-21T00:30:06.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28115 fp=#0x10000004ab4' 2024-09-21T00:30:06.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28147 fp=#0x10000004bd5' 2024-09-21T00:30:06.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28211 fp=#0x10000004b5e' 2024-09-21T00:30:06.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28243 fp=#0x10000004bac' 2024-09-21T00:30:06.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28275 fp=#0x1000000305f' 2024-09-21T00:30:06.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28307 fp=#0x100000032df' 2024-09-21T00:30:06.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28339 fp=#0x100000030c5' 2024-09-21T00:30:06.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28371 fp=#0x100000030bb' 2024-09-21T00:30:06.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28403 fp=#0x10000004c52' 2024-09-21T00:30:06.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28435 fp=#0x10000003182' 2024-09-21T00:30:06.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28467 fp=#0x100000030e5' 2024-09-21T00:30:06.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28499 fp=#0x1000000319c' 2024-09-21T00:30:06.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28531 fp=#0x2000000119e' 2024-09-21T00:30:06.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28595 fp=#0x2000000113e' 2024-09-21T00:30:06.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28627 fp=#0x2000000114b' 2024-09-21T00:30:06.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28659 fp=#0x20000000e26' 2024-09-21T00:30:06.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28691 fp=#0x20000000e18' 2024-09-21T00:30:06.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28723 fp=#0x10000001732' 2024-09-21T00:30:06.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28755 fp=#0x100000016c7' 2024-09-21T00:30:06.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28787 fp=#0x1000000165b' 2024-09-21T00:30:06.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28819 fp=#0x10000001619' 2024-09-21T00:30:06.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28883 fp=#0x200000009ca' 2024-09-21T00:30:06.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28915 fp=#0x2000000031e' 2024-09-21T00:30:06.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28947 fp=#0x10000001733' 2024-09-21T00:30:06.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28979 fp=#0x2000000031d' 2024-09-21T00:30:06.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29011 fp=#0x1000000165a' 2024-09-21T00:30:06.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29043 fp=#0x20000000310' 2024-09-21T00:30:06.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29075 fp=#0x100000043ce' 2024-09-21T00:30:06.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29107 fp=#0x10000004393' 2024-09-21T00:30:06.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29139 fp=#0x100000043ba' 2024-09-21T00:30:06.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29171 fp=#0x100000043a9' 2024-09-21T00:30:06.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29203 fp=#0x1000000435a' 2024-09-21T00:30:06.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29235 fp=#0x20000001428' 2024-09-21T00:30:06.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29267 fp=#0x100000043a2' 2024-09-21T00:30:06.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29395 fp=#0x10000004cbe' 2024-09-21T00:30:06.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29427 fp=#0x10000004e29' 2024-09-21T00:30:06.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29491 fp=#0x10000004de1' 2024-09-21T00:30:06.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29587 fp=#0x10000002a75' 2024-09-21T00:30:06.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29619 fp=#0x10000002a7b' 2024-09-21T00:30:06.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29651 fp=#0x100000040db' 2024-09-21T00:30:06.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29683 fp=#0x10000002995' 2024-09-21T00:30:06.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29747 fp=#0x10000004554' 2024-09-21T00:30:06.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29779 fp=#0x10000004568' 2024-09-21T00:30:06.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29811 fp=#0x1000000453e' 2024-09-21T00:30:06.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29843 fp=#0x20000001233' 2024-09-21T00:30:06.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29907 fp=#0x20000000ded' 2024-09-21T00:30:06.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29939 fp=#0x20000000e20' 2024-09-21T00:30:06.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29971 fp=#0x20000000ca1' 2024-09-21T00:30:06.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30003 fp=#0x20000000d9b' 2024-09-21T00:30:06.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30035 fp=#0x20000000cc0' 2024-09-21T00:30:06.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30067 fp=#0x20000000a5f' 2024-09-21T00:30:06.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30099 fp=#0x20000000c49' 2024-09-21T00:30:06.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30163 fp=#0x10000001c49' 2024-09-21T00:30:06.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30195 fp=#0x10000001b37' 2024-09-21T00:30:06.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30227 fp=#0x10000001b7b' 2024-09-21T00:30:06.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30259 fp=#0x10000001b08' 2024-09-21T00:30:06.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30291 fp=#0x20000000b89' 2024-09-21T00:30:06.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30323 fp=#0x1000000295f' 2024-09-21T00:30:06.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30355 fp=#0x20000000b26' 2024-09-21T00:30:06.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30387 fp=#0x20000000a6a' 2024-09-21T00:30:06.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30419 fp=#0x1000000297b' 2024-09-21T00:30:06.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30451 fp=#0x10000002907' 2024-09-21T00:30:06.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30547 fp=#0x200000006f7' 2024-09-21T00:30:06.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22836 fp=#0x10000001aee' 2024-09-21T00:30:06.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26644 fp=#0x20000000a0b' 2024-09-21T00:30:06.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26708 fp=#0x200000007fa' 2024-09-21T00:30:06.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26772 fp=#0x200000007ee' 2024-09-21T00:30:06.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26804 fp=#0x1000000461b' 2024-09-21T00:30:06.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26836 fp=#0x100000043e9' 2024-09-21T00:30:06.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26932 fp=#0x1000000445a' 2024-09-21T00:30:06.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26964 fp=#0x20000000ee3' 2024-09-21T00:30:06.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26996 fp=#0x20000000df4' 2024-09-21T00:30:06.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27060 fp=#0x20000000edd' 2024-09-21T00:30:06.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27092 fp=#0x20000000f6d' 2024-09-21T00:30:06.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27156 fp=#0x10000002213' 2024-09-21T00:30:06.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27188 fp=#0x1000000236e' 2024-09-21T00:30:06.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27220 fp=#0x100000022a6' 2024-09-21T00:30:06.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27252 fp=#0x10000002384' 2024-09-21T00:30:06.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27284 fp=#0x100000023ab' 2024-09-21T00:30:06.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27316 fp=#0x1000000225e' 2024-09-21T00:30:06.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27348 fp=#0x10000002290' 2024-09-21T00:30:06.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27380 fp=#0x10000003d1a' 2024-09-21T00:30:06.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27412 fp=#0x10000003c85' 2024-09-21T00:30:06.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27444 fp=#0x10000003bb1' 2024-09-21T00:30:06.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27508 fp=#0x10000003c9d' 2024-09-21T00:30:06.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27636 fp=#0x100000020ab' 2024-09-21T00:30:06.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27668 fp=#0x10000001f05' 2024-09-21T00:30:06.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27700 fp=#0x200000011bd' 2024-09-21T00:30:06.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27732 fp=#0x10000001e9f' 2024-09-21T00:30:06.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27764 fp=#0x200000008ed' 2024-09-21T00:30:06.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27796 fp=#0x200000008d4' 2024-09-21T00:30:06.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27828 fp=#0x20000001021' 2024-09-21T00:30:06.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27924 fp=#0x200000010a6' 2024-09-21T00:30:06.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27956 fp=#0x20000001061' 2024-09-21T00:30:06.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27988 fp=#0x10000004a06' 2024-09-21T00:30:06.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28020 fp=#0x1000000496d' 2024-09-21T00:30:06.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28052 fp=#0x10000004a28' 2024-09-21T00:30:06.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28084 fp=#0x2000000123c' 2024-09-21T00:30:06.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28116 fp=#0x10000004bff' 2024-09-21T00:30:06.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28148 fp=#0x10000004ba1' 2024-09-21T00:30:06.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28180 fp=#0x10000004ab8' 2024-09-21T00:30:06.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28212 fp=#0x10000004b52' 2024-09-21T00:30:06.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28244 fp=#0x10000004b78' 2024-09-21T00:30:06.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28276 fp=#0x10000003267' 2024-09-21T00:30:06.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28308 fp=#0x100000032c7' 2024-09-21T00:30:06.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28340 fp=#0x1000000317d' 2024-09-21T00:30:06.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28372 fp=#0x10000003198' 2024-09-21T00:30:06.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28404 fp=#0x100000030cc' 2024-09-21T00:30:06.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28436 fp=#0x1000000312f' 2024-09-21T00:30:06.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28468 fp=#0x100000032b7' 2024-09-21T00:30:06.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28500 fp=#0x100000032c9' 2024-09-21T00:30:06.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28532 fp=#0x2000000116e' 2024-09-21T00:30:06.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28564 fp=#0x2000000121e' 2024-09-21T00:30:06.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28596 fp=#0x20000001156' 2024-09-21T00:30:06.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28628 fp=#0x20000000feb' 2024-09-21T00:30:06.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28660 fp=#0x20000000e21' 2024-09-21T00:30:06.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28692 fp=#0x20000000da7' 2024-09-21T00:30:06.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28724 fp=#0x10000001688' 2024-09-21T00:30:06.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28756 fp=#0x100000016d0' 2024-09-21T00:30:06.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28788 fp=#0x10000001693' 2024-09-21T00:30:06.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28820 fp=#0x100000016f2' 2024-09-21T00:30:06.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28884 fp=#0x200000009f2' 2024-09-21T00:30:06.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28916 fp=#0x10000001623' 2024-09-21T00:30:06.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28948 fp=#0x10000001641' 2024-09-21T00:30:06.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28980 fp=#0x100000016dc' 2024-09-21T00:30:06.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29012 fp=#0x200000006e4' 2024-09-21T00:30:06.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29044 fp=#0x10000001612' 2024-09-21T00:30:06.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29076 fp=#0x20000001418' 2024-09-21T00:30:06.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29108 fp=#0x100000043b0' 2024-09-21T00:30:06.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29140 fp=#0x200000013fe' 2024-09-21T00:30:06.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29172 fp=#0x2000000141a' 2024-09-21T00:30:06.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29204 fp=#0x100000053af' 2024-09-21T00:30:06.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29236 fp=#0x10000004360' 2024-09-21T00:30:06.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29268 fp=#0x10000004394' 2024-09-21T00:30:06.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29428 fp=#0x10000004cf7' 2024-09-21T00:30:06.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29492 fp=#0x2000000100e' 2024-09-21T00:30:06.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29524 fp=#0x10000004c60' 2024-09-21T00:30:06.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29588 fp=#0x10000002a90' 2024-09-21T00:30:06.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29652 fp=#0x20000000ebd' 2024-09-21T00:30:06.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29684 fp=#0x100000040b1' 2024-09-21T00:30:06.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29716 fp=#0x20000000eb4' 2024-09-21T00:30:06.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29748 fp=#0x10000004587' 2024-09-21T00:30:06.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29780 fp=#0x1000000452b' 2024-09-21T00:30:06.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29812 fp=#0x10000004572' 2024-09-21T00:30:06.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29844 fp=#0x10000004518' 2024-09-21T00:30:06.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29908 fp=#0x20000000ca4' 2024-09-21T00:30:06.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29940 fp=#0x20000000e17' 2024-09-21T00:30:06.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29972 fp=#0x20000000e64' 2024-09-21T00:30:06.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30004 fp=#0x20000000c93' 2024-09-21T00:30:06.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30036 fp=#0x20000000d38' 2024-09-21T00:30:06.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30132 fp=#0x20000000c22' 2024-09-21T00:30:06.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30164 fp=#0x100000055b9' 2024-09-21T00:30:06.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30196 fp=#0x10000001c13' 2024-09-21T00:30:06.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30228 fp=#0x10000001c3c' 2024-09-21T00:30:06.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30260 fp=#0x10000001b3c' 2024-09-21T00:30:06.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30292 fp=#0x20000000b4e' 2024-09-21T00:30:06.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30324 fp=#0x20000000a51' 2024-09-21T00:30:06.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30356 fp=#0x10000002930' 2024-09-21T00:30:06.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30388 fp=#0x100000028ef' 2024-09-21T00:30:06.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30420 fp=#0x100000028d5' 2024-09-21T00:30:06.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30484 fp=#0x100000023c4' 2024-09-21T00:30:06.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22837 fp=#0x10000005152' 2024-09-21T00:30:06.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:25237 fp=#0x20000001399' 2024-09-21T00:30:06.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26645 fp=#0x200000007f1' 2024-09-21T00:30:06.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26741 fp=#0x20000000805' 2024-09-21T00:30:06.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26773 fp=#0x20000000a1d' 2024-09-21T00:30:06.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26805 fp=#0x1000000447b' 2024-09-21T00:30:06.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26837 fp=#0x100000043e0' 2024-09-21T00:30:06.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26901 fp=#0x100000043f4' 2024-09-21T00:30:06.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26933 fp=#0x100000043da' 2024-09-21T00:30:06.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26965 fp=#0x20000000f0f' 2024-09-21T00:30:06.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26997 fp=#0x20000000edb' 2024-09-21T00:30:06.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27029 fp=#0x20000000f0a' 2024-09-21T00:30:06.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27061 fp=#0x20000000dc1' 2024-09-21T00:30:06.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27125 fp=#0x20000000ffb' 2024-09-21T00:30:06.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27157 fp=#0x10000002363' 2024-09-21T00:30:06.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27189 fp=#0x1000000229c' 2024-09-21T00:30:06.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27221 fp=#0x20000000fe8' 2024-09-21T00:30:06.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27253 fp=#0x20000000f59' 2024-09-21T00:30:06.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27285 fp=#0x20000000f2b' 2024-09-21T00:30:06.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27317 fp=#0x20000000f13' 2024-09-21T00:30:06.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27349 fp=#0x10000002382' 2024-09-21T00:30:06.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27381 fp=#0x10000003cac' 2024-09-21T00:30:06.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27445 fp=#0x2000000112e' 2024-09-21T00:30:06.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27477 fp=#0x10000003b77' 2024-09-21T00:30:06.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27509 fp=#0x10000003b53' 2024-09-21T00:30:06.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27669 fp=#0x10000001e97' 2024-09-21T00:30:06.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27701 fp=#0x10000001f96' 2024-09-21T00:30:06.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27733 fp=#0x10000001ec0' 2024-09-21T00:30:06.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27765 fp=#0x200000008d3' 2024-09-21T00:30:06.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27797 fp=#0x20000000941' 2024-09-21T00:30:06.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27893 fp=#0x200000008cb' 2024-09-21T00:30:06.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27925 fp=#0x200000010cf' 2024-09-21T00:30:06.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27957 fp=#0x2000000105d' 2024-09-21T00:30:06.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27989 fp=#0x10000004992' 2024-09-21T00:30:06.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28021 fp=#0x100000049f8' 2024-09-21T00:30:06.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28053 fp=#0x10000004a49' 2024-09-21T00:30:06.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28085 fp=#0x200000012a7' 2024-09-21T00:30:06.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28117 fp=#0x10000004bf2' 2024-09-21T00:30:06.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28149 fp=#0x10000004b69' 2024-09-21T00:30:06.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28181 fp=#0x10000004ac9' 2024-09-21T00:30:06.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28213 fp=#0x10000004ac4' 2024-09-21T00:30:06.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28245 fp=#0x10000004bd3' 2024-09-21T00:30:06.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28277 fp=#0x10000003357' 2024-09-21T00:30:06.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28309 fp=#0x10000003270' 2024-09-21T00:30:06.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28341 fp=#0x1000000318b' 2024-09-21T00:30:06.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28373 fp=#0x100000031e9' 2024-09-21T00:30:06.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28405 fp=#0x20000000f1a' 2024-09-21T00:30:06.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28437 fp=#0x100000031c3' 2024-09-21T00:30:06.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28469 fp=#0x10000003087' 2024-09-21T00:30:06.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28501 fp=#0x1000000308c' 2024-09-21T00:30:06.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28533 fp=#0x200000011da' 2024-09-21T00:30:06.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28565 fp=#0x20000001121' 2024-09-21T00:30:06.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28629 fp=#0x200000011cc' 2024-09-21T00:30:06.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28661 fp=#0x20000000e47' 2024-09-21T00:30:06.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28693 fp=#0x20000000d43' 2024-09-21T00:30:06.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28725 fp=#0x10000001729' 2024-09-21T00:30:06.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28757 fp=#0x100000015df' 2024-09-21T00:30:06.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28789 fp=#0x100000016f7' 2024-09-21T00:30:06.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28821 fp=#0x100000015f9' 2024-09-21T00:30:06.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28885 fp=#0x200000009d0' 2024-09-21T00:30:06.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28917 fp=#0x100000016af' 2024-09-21T00:30:06.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28949 fp=#0x20000000307' 2024-09-21T00:30:06.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28981 fp=#0x20000000315' 2024-09-21T00:30:06.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29013 fp=#0x1000000169d' 2024-09-21T00:30:06.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29045 fp=#0x100000016a3' 2024-09-21T00:30:06.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29077 fp=#0x200000013fa' 2024-09-21T00:30:06.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29109 fp=#0x200000013f0' 2024-09-21T00:30:06.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29141 fp=#0x10000004350' 2024-09-21T00:30:06.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29173 fp=#0x100000043d4' 2024-09-21T00:30:06.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29205 fp=#0x10000004359' 2024-09-21T00:30:06.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29237 fp=#0x100000043b9' 2024-09-21T00:30:06.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29269 fp=#0x20000001426' 2024-09-21T00:30:06.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29397 fp=#0x10000004e24' 2024-09-21T00:30:06.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29429 fp=#0x10000004e12' 2024-09-21T00:30:06.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29461 fp=#0x10000004c81' 2024-09-21T00:30:06.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29493 fp=#0x10000004caa' 2024-09-21T00:30:06.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29525 fp=#0x10000004dfe' 2024-09-21T00:30:06.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29589 fp=#0x20000000eaa' 2024-09-21T00:30:06.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29621 fp=#0x100000040a7' 2024-09-21T00:30:06.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29685 fp=#0x10000002986' 2024-09-21T00:30:06.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29717 fp=#0x100000040d7' 2024-09-21T00:30:06.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29749 fp=#0x1000000458a' 2024-09-21T00:30:06.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29781 fp=#0x200000012ce' 2024-09-21T00:30:06.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29813 fp=#0x10000004562' 2024-09-21T00:30:06.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29845 fp=#0x10000004524' 2024-09-21T00:30:06.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29877 fp=#0x10000004cd3' 2024-09-21T00:30:06.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29909 fp=#0x20000000e08' 2024-09-21T00:30:06.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29941 fp=#0x20000000d63' 2024-09-21T00:30:06.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29973 fp=#0x20000000d7f' 2024-09-21T00:30:06.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30005 fp=#0x20000000e7f' 2024-09-21T00:30:06.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30037 fp=#0x20000000d39' 2024-09-21T00:30:06.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30101 fp=#0x20000000a64' 2024-09-21T00:30:06.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30165 fp=#0x10000001ba2' 2024-09-21T00:30:06.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30197 fp=#0x10000001c3f' 2024-09-21T00:30:06.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30229 fp=#0x10000001baa' 2024-09-21T00:30:06.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30261 fp=#0x10000001be0' 2024-09-21T00:30:06.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30293 fp=#0x20000000adf' 2024-09-21T00:30:06.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30325 fp=#0x10000002932' 2024-09-21T00:30:06.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30357 fp=#0x20000000a46' 2024-09-21T00:30:06.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30389 fp=#0x20000000ad9' 2024-09-21T00:30:06.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30421 fp=#0x20000000a4b' 2024-09-21T00:30:06.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30453 fp=#0x10000002926' 2024-09-21T00:30:06.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30485 fp=#0x200000006ed' 2024-09-21T00:30:06.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30549 fp=#0x1000000237c' 2024-09-21T00:30:06.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22838 fp=#0x100000028b5' 2024-09-21T00:30:06.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:25302 fp=#0x20000001395' 2024-09-21T00:30:06.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26614 fp=#0x10000001e76' 2024-09-21T00:30:06.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26678 fp=#0x20000000a1b' 2024-09-21T00:30:06.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26710 fp=#0x200000007eb' 2024-09-21T00:30:06.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26742 fp=#0x20000000799' 2024-09-21T00:30:06.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26774 fp=#0x20000000774' 2024-09-21T00:30:06.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26806 fp=#0x10000004490' 2024-09-21T00:30:06.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26838 fp=#0x1000000461c' 2024-09-21T00:30:06.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26870 fp=#0x10000004636' 2024-09-21T00:30:06.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26902 fp=#0x10000004477' 2024-09-21T00:30:06.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26934 fp=#0x100000043f2' 2024-09-21T00:30:06.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27030 fp=#0x20000000f89' 2024-09-21T00:30:06.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27062 fp=#0x20000000f53' 2024-09-21T00:30:06.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27094 fp=#0x20000000ef3' 2024-09-21T00:30:06.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27126 fp=#0x20000000fb7' 2024-09-21T00:30:06.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27190 fp=#0x100000023a8' 2024-09-21T00:30:06.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27222 fp=#0x20000000f1d' 2024-09-21T00:30:06.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27254 fp=#0x10000002361' 2024-09-21T00:30:06.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27286 fp=#0x10000002367' 2024-09-21T00:30:06.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27318 fp=#0x20000000f2f' 2024-09-21T00:30:06.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27350 fp=#0x100000022a9' 2024-09-21T00:30:06.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27382 fp=#0x10000003d22' 2024-09-21T00:30:06.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27414 fp=#0x10000003c96' 2024-09-21T00:30:06.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27446 fp=#0x10000003b6b' 2024-09-21T00:30:06.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27478 fp=#0x10000003cd1' 2024-09-21T00:30:06.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27510 fp=#0x10000003b96' 2024-09-21T00:30:06.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27638 fp=#0x10000001ed1' 2024-09-21T00:30:06.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27670 fp=#0x10000001fda' 2024-09-21T00:30:06.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27702 fp=#0x10000001f1c' 2024-09-21T00:30:06.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27734 fp=#0x10000001f4f' 2024-09-21T00:30:06.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27798 fp=#0x200000008cc' 2024-09-21T00:30:06.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27862 fp=#0x200000008f4' 2024-09-21T00:30:06.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27926 fp=#0x200000010e9' 2024-09-21T00:30:06.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27958 fp=#0x20000001064' 2024-09-21T00:30:06.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27990 fp=#0x10000004a44' 2024-09-21T00:30:06.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28022 fp=#0x100000049a6' 2024-09-21T00:30:06.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28054 fp=#0x10000004995' 2024-09-21T00:30:06.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28086 fp=#0x200000012c5' 2024-09-21T00:30:06.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28118 fp=#0x10000004b4e' 2024-09-21T00:30:06.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28150 fp=#0x10000004bc1' 2024-09-21T00:30:06.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28182 fp=#0x10000004b33' 2024-09-21T00:30:06.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28214 fp=#0x10000004b38' 2024-09-21T00:30:06.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28246 fp=#0x20000000fb0' 2024-09-21T00:30:06.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28278 fp=#0x10000003112' 2024-09-21T00:30:06.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28310 fp=#0x10000003179' 2024-09-21T00:30:06.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28342 fp=#0x20000000ef6' 2024-09-21T00:30:06.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28374 fp=#0x10000003286' 2024-09-21T00:30:06.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28406 fp=#0x10000004bd6' 2024-09-21T00:30:06.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28438 fp=#0x1000000326b' 2024-09-21T00:30:06.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28470 fp=#0x20000000efb' 2024-09-21T00:30:06.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28502 fp=#0x10000003298' 2024-09-21T00:30:06.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28566 fp=#0x200000010ed' 2024-09-21T00:30:06.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28598 fp=#0x20000000f66' 2024-09-21T00:30:06.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28630 fp=#0x20000000f7c' 2024-09-21T00:30:06.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28662 fp=#0x20000000d86' 2024-09-21T00:30:06.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28694 fp=#0x20000000df6' 2024-09-21T00:30:06.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28726 fp=#0x100000015f1' 2024-09-21T00:30:06.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28758 fp=#0x2000000072c' 2024-09-21T00:30:06.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28790 fp=#0x100000015da' 2024-09-21T00:30:06.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28822 fp=#0x1000000169b' 2024-09-21T00:30:06.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28854 fp=#0x200000009cc' 2024-09-21T00:30:06.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28918 fp=#0x20000000312' 2024-09-21T00:30:06.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28950 fp=#0x100000016c8' 2024-09-21T00:30:06.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28982 fp=#0x2000000050b' 2024-09-21T00:30:06.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29014 fp=#0x1000000167d' 2024-09-21T00:30:06.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29046 fp=#0x100000016b4' 2024-09-21T00:30:06.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29078 fp=#0x10000004397' 2024-09-21T00:30:06.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29110 fp=#0x200000013ec' 2024-09-21T00:30:06.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29142 fp=#0x200000013f3' 2024-09-21T00:30:06.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29206 fp=#0x10000004341' 2024-09-21T00:30:06.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29238 fp=#0x10000004331' 2024-09-21T00:30:06.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29270 fp=#0x20000001401' 2024-09-21T00:30:06.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29430 fp=#0x10000004df4' 2024-09-21T00:30:06.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29462 fp=#0x10000004e4a' 2024-09-21T00:30:06.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29526 fp=#0x10000004ccc' 2024-09-21T00:30:06.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29558 fp=#0x20000000ff3' 2024-09-21T00:30:06.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29590 fp=#0x10000002b1b' 2024-09-21T00:30:06.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29622 fp=#0x10000002977' 2024-09-21T00:30:06.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29718 fp=#0x100000040d2' 2024-09-21T00:30:06.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29750 fp=#0x10000004529' 2024-09-21T00:30:06.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29782 fp=#0x10000004583' 2024-09-21T00:30:06.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29814 fp=#0x1000000451b' 2024-09-21T00:30:06.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29846 fp=#0x1000000454a' 2024-09-21T00:30:06.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29878 fp=#0x10000004d68' 2024-09-21T00:30:06.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29910 fp=#0x20000000cfb' 2024-09-21T00:30:06.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29942 fp=#0x20000000cdf' 2024-09-21T00:30:06.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29974 fp=#0x20000000e61' 2024-09-21T00:30:06.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30006 fp=#0x20000000c7a' 2024-09-21T00:30:06.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30038 fp=#0x20000000c3e' 2024-09-21T00:30:06.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30070 fp=#0x20000000bca' 2024-09-21T00:30:06.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30166 fp=#0x10000001c2b' 2024-09-21T00:30:06.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30198 fp=#0x10000001c1c' 2024-09-21T00:30:06.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30230 fp=#0x100000055ae' 2024-09-21T00:30:06.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30262 fp=#0x10000001b44' 2024-09-21T00:30:06.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30294 fp=#0x20000000a4f' 2024-09-21T00:30:06.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30326 fp=#0x100000029a6' 2024-09-21T00:30:06.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30358 fp=#0x20000000a86' 2024-09-21T00:30:06.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30390 fp=#0x20000000a59' 2024-09-21T00:30:06.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30422 fp=#0x20000000a6c' 2024-09-21T00:30:06.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30454 fp=#0x20000000b16' 2024-09-21T00:30:06.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30486 fp=#0x100000023be' 2024-09-21T00:30:06.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30518 fp=#0x200000006ef' 2024-09-21T00:30:06.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30550 fp=#0x10000002370' 2024-09-21T00:30:06.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22775 fp=#0x10000002300' 2024-09-21T00:30:06.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22839 fp=#0x1000000527d' 2024-09-21T00:30:06.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:25239 fp=#0x2000000137d' 2024-09-21T00:30:06.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26615 fp=#0x10000001dea' 2024-09-21T00:30:06.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26647 fp=#0x20000000830' 2024-09-21T00:30:06.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26679 fp=#0x20000000795' 2024-09-21T00:30:06.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26743 fp=#0x200000007d7' 2024-09-21T00:30:06.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26807 fp=#0x10000004630' 2024-09-21T00:30:06.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26839 fp=#0x100000043f1' 2024-09-21T00:30:06.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26871 fp=#0x1000000448b' 2024-09-21T00:30:06.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26967 fp=#0x20000000dc8' 2024-09-21T00:30:06.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26999 fp=#0x20000000ede' 2024-09-21T00:30:06.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27031 fp=#0x20000000ed3' 2024-09-21T00:30:06.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27063 fp=#0x20000000d60' 2024-09-21T00:30:06.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27095 fp=#0x20000000d91' 2024-09-21T00:30:06.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27127 fp=#0x20000000efa' 2024-09-21T00:30:06.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27159 fp=#0x10000002265' 2024-09-21T00:30:06.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27191 fp=#0x1000000234f' 2024-09-21T00:30:06.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27223 fp=#0x1000000237b' 2024-09-21T00:30:06.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27255 fp=#0x1000000239a' 2024-09-21T00:30:06.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27287 fp=#0x10000002246' 2024-09-21T00:30:06.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27319 fp=#0x10000002338' 2024-09-21T00:30:06.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27351 fp=#0x20000000f97' 2024-09-21T00:30:06.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27415 fp=#0x10000003b65' 2024-09-21T00:30:06.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27479 fp=#0x10000003be4' 2024-09-21T00:30:06.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27511 fp=#0x10000003cc0' 2024-09-21T00:30:06.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27639 fp=#0x10000001fa0' 2024-09-21T00:30:06.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27671 fp=#0x10000001ed7' 2024-09-21T00:30:06.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27703 fp=#0x10000001fbd' 2024-09-21T00:30:06.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27735 fp=#0x10000001ec7' 2024-09-21T00:30:06.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27767 fp=#0x200000008fe' 2024-09-21T00:30:06.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27831 fp=#0x200000008f8' 2024-09-21T00:30:06.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27863 fp=#0x200000008d0' 2024-09-21T00:30:06.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27927 fp=#0x20000001053' 2024-09-21T00:30:06.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27959 fp=#0x200000010ab' 2024-09-21T00:30:06.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27991 fp=#0x10000004a38' 2024-09-21T00:30:06.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28023 fp=#0x10000004a0d' 2024-09-21T00:30:06.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28055 fp=#0x10000004970' 2024-09-21T00:30:06.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28087 fp=#0x200000012d7' 2024-09-21T00:30:06.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28119 fp=#0x10000004bef' 2024-09-21T00:30:06.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28183 fp=#0x10000004acd' 2024-09-21T00:30:06.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28215 fp=#0x10000004bfb' 2024-09-21T00:30:06.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28279 fp=#0x10000003350' 2024-09-21T00:30:06.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28311 fp=#0x100000032ac' 2024-09-21T00:30:06.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28343 fp=#0x10000004c53' 2024-09-21T00:30:06.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28375 fp=#0x20000000e5f' 2024-09-21T00:30:06.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28407 fp=#0x100000032e1' 2024-09-21T00:30:06.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28439 fp=#0x100000030f5' 2024-09-21T00:30:06.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28471 fp=#0x20000000ea1' 2024-09-21T00:30:06.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28503 fp=#0x20000000e4a' 2024-09-21T00:30:06.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28535 fp=#0x20000001033' 2024-09-21T00:30:06.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28567 fp=#0x20000000f3d' 2024-09-21T00:30:06.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28631 fp=#0x20000001169' 2024-09-21T00:30:06.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28663 fp=#0x20000000e33' 2024-09-21T00:30:06.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28695 fp=#0x20000000d6e' 2024-09-21T00:30:06.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28727 fp=#0x10000001724' 2024-09-21T00:30:06.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28759 fp=#0x10000001655' 2024-09-21T00:30:06.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28791 fp=#0x10000001656' 2024-09-21T00:30:06.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28823 fp=#0x100000016ac' 2024-09-21T00:30:06.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28919 fp=#0x10000001603' 2024-09-21T00:30:06.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28951 fp=#0x20000000301' 2024-09-21T00:30:06.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28983 fp=#0x10000001689' 2024-09-21T00:30:06.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29015 fp=#0x1000000161e' 2024-09-21T00:30:06.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29047 fp=#0x20000000300' 2024-09-21T00:30:06.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29079 fp=#0x10000004335' 2024-09-21T00:30:06.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29111 fp=#0x10000004351' 2024-09-21T00:30:06.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29143 fp=#0x20000001427' 2024-09-21T00:30:06.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29175 fp=#0x20000001422' 2024-09-21T00:30:06.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29207 fp=#0x100000043c1' 2024-09-21T00:30:06.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29239 fp=#0x1000000437a' 2024-09-21T00:30:06.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29271 fp=#0x10000004346' 2024-09-21T00:30:06.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29399 fp=#0x10000004c96' 2024-09-21T00:30:06.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29559 fp=#0x10000004ca6' 2024-09-21T00:30:06.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29591 fp=#0x10000002a81' 2024-09-21T00:30:06.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29623 fp=#0x100000040a2' 2024-09-21T00:30:06.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29655 fp=#0x100000040bf' 2024-09-21T00:30:06.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29687 fp=#0x10000002a66' 2024-09-21T00:30:06.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29751 fp=#0x1000000455b' 2024-09-21T00:30:06.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29783 fp=#0x1000000454f' 2024-09-21T00:30:06.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29815 fp=#0x10000004541' 2024-09-21T00:30:06.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29847 fp=#0x10000004526' 2024-09-21T00:30:06.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29911 fp=#0x20000000e22' 2024-09-21T00:30:06.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29943 fp=#0x20000000d75' 2024-09-21T00:30:06.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29975 fp=#0x2000000110b' 2024-09-21T00:30:06.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30007 fp=#0x20000000d41' 2024-09-21T00:30:06.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30039 fp=#0x20000000c2a' 2024-09-21T00:30:06.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30071 fp=#0x20000000c0c' 2024-09-21T00:30:06.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30167 fp=#0x10000001c9a' 2024-09-21T00:30:06.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30199 fp=#0x10000005726' 2024-09-21T00:30:06.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30231 fp=#0x10000001c54' 2024-09-21T00:30:06.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30263 fp=#0x10000001bd6' 2024-09-21T00:30:06.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30295 fp=#0x1000000299b' 2024-09-21T00:30:06.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30327 fp=#0x1000000296c' 2024-09-21T00:30:06.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30359 fp=#0x20000000a5d' 2024-09-21T00:30:06.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30391 fp=#0x10000003693' 2024-09-21T00:30:06.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30423 fp=#0x20000000a42' 2024-09-21T00:30:06.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30455 fp=#0x10000002959' 2024-09-21T00:30:06.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30487 fp=#0x100000023e2' 2024-09-21T00:30:06.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22840 fp=#0x100000014ac' 2024-09-21T00:30:06.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26616 fp=#0x10000001e8b' 2024-09-21T00:30:06.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26776 fp=#0x20000000a09' 2024-09-21T00:30:06.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26808 fp=#0x100000043f5' 2024-09-21T00:30:06.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26840 fp=#0x1000000463c' 2024-09-21T00:30:06.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26872 fp=#0x1000000460b' 2024-09-21T00:30:06.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26904 fp=#0x1000000460e' 2024-09-21T00:30:06.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26936 fp=#0x100000043fd' 2024-09-21T00:30:06.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26968 fp=#0x20000000efd' 2024-09-21T00:30:06.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27000 fp=#0x20000000f9d' 2024-09-21T00:30:06.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27032 fp=#0x20000000ec8' 2024-09-21T00:30:06.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27064 fp=#0x20000000fe5' 2024-09-21T00:30:06.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27128 fp=#0x20000000fd3' 2024-09-21T00:30:06.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27160 fp=#0x1000000251a' 2024-09-21T00:30:06.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27192 fp=#0x100000023b4' 2024-09-21T00:30:06.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27224 fp=#0x100000023db' 2024-09-21T00:30:06.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27256 fp=#0x20000001001' 2024-09-21T00:30:06.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27288 fp=#0x10000002355' 2024-09-21T00:30:06.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27320 fp=#0x20000000d4d' 2024-09-21T00:30:06.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27352 fp=#0x100000023c8' 2024-09-21T00:30:06.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27384 fp=#0x10000003bc9' 2024-09-21T00:30:06.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27416 fp=#0x10000003cc7' 2024-09-21T00:30:06.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27480 fp=#0x10000003bd5' 2024-09-21T00:30:06.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27512 fp=#0x10000003b8f' 2024-09-21T00:30:06.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27640 fp=#0x10000002049' 2024-09-21T00:30:06.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27672 fp=#0x10000001ebd' 2024-09-21T00:30:06.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27704 fp=#0x10000001ea3' 2024-09-21T00:30:06.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27736 fp=#0x10000001f46' 2024-09-21T00:30:06.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27832 fp=#0x200000008c7' 2024-09-21T00:30:06.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27896 fp=#0x20000000977' 2024-09-21T00:30:06.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27928 fp=#0x2000000106b' 2024-09-21T00:30:06.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27960 fp=#0x200000010f2' 2024-09-21T00:30:06.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27992 fp=#0x100000049b4' 2024-09-21T00:30:06.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28024 fp=#0x100000049c6' 2024-09-21T00:30:06.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28056 fp=#0x10000004a0a' 2024-09-21T00:30:06.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28088 fp=#0x20000001373' 2024-09-21T00:30:06.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28120 fp=#0x10000004ad1' 2024-09-21T00:30:06.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28152 fp=#0x10000004bc7' 2024-09-21T00:30:06.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28216 fp=#0x10000004ba2' 2024-09-21T00:30:06.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28248 fp=#0x10000004bf8' 2024-09-21T00:30:06.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28280 fp=#0x20000000f07' 2024-09-21T00:30:06.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28312 fp=#0x100000031f7' 2024-09-21T00:30:06.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28344 fp=#0x1000000327d' 2024-09-21T00:30:06.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28376 fp=#0x10000003262' 2024-09-21T00:30:06.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28408 fp=#0x10000003192' 2024-09-21T00:30:06.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28440 fp=#0x20000000e8b' 2024-09-21T00:30:06.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28472 fp=#0x20000000e95' 2024-09-21T00:30:06.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28504 fp=#0x200000011f1' 2024-09-21T00:30:06.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28536 fp=#0x200000011b1' 2024-09-21T00:30:06.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28632 fp=#0x20000000f95' 2024-09-21T00:30:06.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28664 fp=#0x20000000d45' 2024-09-21T00:30:06.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28696 fp=#0x20000000d5a' 2024-09-21T00:30:06.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28728 fp=#0x10000001752' 2024-09-21T00:30:06.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28760 fp=#0x10000001682' 2024-09-21T00:30:06.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28792 fp=#0x100000016be' 2024-09-21T00:30:06.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28824 fp=#0x1000000161f' 2024-09-21T00:30:06.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28856 fp=#0x200000009f6' 2024-09-21T00:30:06.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28920 fp=#0x10000001633' 2024-09-21T00:30:06.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28952 fp=#0x10000001626' 2024-09-21T00:30:06.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28984 fp=#0x1000000160c' 2024-09-21T00:30:06.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29016 fp=#0x2000000030b' 2024-09-21T00:30:06.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29048 fp=#0x20000000a79' 2024-09-21T00:30:06.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29080 fp=#0x200000013c7' 2024-09-21T00:30:06.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29112 fp=#0x100000043be' 2024-09-21T00:30:06.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29144 fp=#0x200000013aa' 2024-09-21T00:30:06.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29176 fp=#0x100000043a0' 2024-09-21T00:30:06.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29208 fp=#0x1000000437e' 2024-09-21T00:30:06.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29240 fp=#0x200000013be' 2024-09-21T00:30:06.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29272 fp=#0x100000043c7' 2024-09-21T00:30:06.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29400 fp=#0x10000004e4c' 2024-09-21T00:30:06.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29432 fp=#0x10000004ced' 2024-09-21T00:30:06.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29496 fp=#0x10000004e15' 2024-09-21T00:30:06.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29560 fp=#0x10000004cc2' 2024-09-21T00:30:06.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29592 fp=#0x100000040b2' 2024-09-21T00:30:06.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29624 fp=#0x100000040c8' 2024-09-21T00:30:06.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29720 fp=#0x10000002992' 2024-09-21T00:30:06.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29752 fp=#0x1000000457c' 2024-09-21T00:30:06.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29784 fp=#0x1000000455a' 2024-09-21T00:30:06.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29816 fp=#0x10000004514' 2024-09-21T00:30:06.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29848 fp=#0x10000004548' 2024-09-21T00:30:06.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29880 fp=#0x10000004cdd' 2024-09-21T00:30:06.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29912 fp=#0x20000000cb6' 2024-09-21T00:30:06.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29944 fp=#0x20000000d7b' 2024-09-21T00:30:06.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29976 fp=#0x20000000d32' 2024-09-21T00:30:06.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30008 fp=#0x20000000ccb' 2024-09-21T00:30:06.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30040 fp=#0x20000000a43' 2024-09-21T00:30:06.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30072 fp=#0x20000000a5e' 2024-09-21T00:30:06.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30136 fp=#0x20000000bf5' 2024-09-21T00:30:06.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30168 fp=#0x10000001ca0' 2024-09-21T00:30:06.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30200 fp=#0x10000001c59' 2024-09-21T00:30:06.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30232 fp=#0x10000001b15' 2024-09-21T00:30:06.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30264 fp=#0x10000001bfc' 2024-09-21T00:30:06.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30296 fp=#0x100000029bf' 2024-09-21T00:30:06.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30328 fp=#0x100000028db' 2024-09-21T00:30:06.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30392 fp=#0x20000000b9d' 2024-09-21T00:30:06.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30424 fp=#0x20000000bc8' 2024-09-21T00:30:06.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30488 fp=#0x10000002391' 2024-09-21T00:30:06.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22841 fp=#0x1000000077a' 2024-09-21T00:30:06.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26617 fp=#0x10000001dc4' 2024-09-21T00:30:06.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26649 fp=#0x200000007a1' 2024-09-21T00:30:06.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26681 fp=#0x200000007da' 2024-09-21T00:30:06.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26713 fp=#0x200000007ea' 2024-09-21T00:30:06.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26809 fp=#0x10000004484' 2024-09-21T00:30:06.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26841 fp=#0x100000043f8' 2024-09-21T00:30:06.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26873 fp=#0x1000000463a' 2024-09-21T00:30:06.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26905 fp=#0x10000004610' 2024-09-21T00:30:06.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26937 fp=#0x10000004622' 2024-09-21T00:30:06.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26969 fp=#0x20000000d79' 2024-09-21T00:30:06.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27001 fp=#0x20000000f98' 2024-09-21T00:30:06.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27033 fp=#0x20000000d51' 2024-09-21T00:30:06.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27065 fp=#0x20000000ef0' 2024-09-21T00:30:06.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27097 fp=#0x20000001002' 2024-09-21T00:30:06.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27161 fp=#0x20000000ff9' 2024-09-21T00:30:06.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27193 fp=#0x20000000fb8' 2024-09-21T00:30:06.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27225 fp=#0x20000000fcf' 2024-09-21T00:30:06.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27289 fp=#0x10000002244' 2024-09-21T00:30:06.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27321 fp=#0x20000001031' 2024-09-21T00:30:06.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27353 fp=#0x1000000237e' 2024-09-21T00:30:06.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27385 fp=#0x10000003cf0' 2024-09-21T00:30:06.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27417 fp=#0x10000003bed' 2024-09-21T00:30:06.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27449 fp=#0x10000003c5c' 2024-09-21T00:30:06.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27481 fp=#0x10000003b67' 2024-09-21T00:30:06.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27513 fp=#0x10000003b99' 2024-09-21T00:30:06.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27641 fp=#0x10000001ea6' 2024-09-21T00:30:06.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27673 fp=#0x1000000207d' 2024-09-21T00:30:06.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27705 fp=#0x10000001f26' 2024-09-21T00:30:06.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27737 fp=#0x10000001f53' 2024-09-21T00:30:06.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27769 fp=#0x2000000100a' 2024-09-21T00:30:06.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27801 fp=#0x200000008fc' 2024-09-21T00:30:06.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27833 fp=#0x200000008ca' 2024-09-21T00:30:06.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27865 fp=#0x200000008f7' 2024-09-21T00:30:06.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27897 fp=#0x200000008d9' 2024-09-21T00:30:06.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27929 fp=#0x20000001154' 2024-09-21T00:30:06.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27961 fp=#0x20000001083' 2024-09-21T00:30:06.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27993 fp=#0x10000004a34' 2024-09-21T00:30:06.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28025 fp=#0x10000004a24' 2024-09-21T00:30:06.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28057 fp=#0x10000004a1b' 2024-09-21T00:30:06.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28089 fp=#0x20000001248' 2024-09-21T00:30:06.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28121 fp=#0x10000004b37' 2024-09-21T00:30:06.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28153 fp=#0x10000004abf' 2024-09-21T00:30:06.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28185 fp=#0x10000004bae' 2024-09-21T00:30:06.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28217 fp=#0x10000004bb7' 2024-09-21T00:30:06.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28249 fp=#0x10000004b44' 2024-09-21T00:30:06.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28281 fp=#0x20000000ea4' 2024-09-21T00:30:06.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28313 fp=#0x100000032b3' 2024-09-21T00:30:06.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28345 fp=#0x100000032d1' 2024-09-21T00:30:06.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28377 fp=#0x10000004bcf' 2024-09-21T00:30:06.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28409 fp=#0x1000000315c' 2024-09-21T00:30:06.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28441 fp=#0x10000003052' 2024-09-21T00:30:06.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28473 fp=#0x10000003046' 2024-09-21T00:30:06.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28505 fp=#0x200000011d5' 2024-09-21T00:30:06.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28537 fp=#0x20000001185' 2024-09-21T00:30:06.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28569 fp=#0x20000000f39' 2024-09-21T00:30:06.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28601 fp=#0x20000000f60' 2024-09-21T00:30:06.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28633 fp=#0x200000011a3' 2024-09-21T00:30:06.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28665 fp=#0x20000000db7' 2024-09-21T00:30:06.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28697 fp=#0x20000000dfa' 2024-09-21T00:30:06.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28729 fp=#0x100000016c0' 2024-09-21T00:30:06.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28761 fp=#0x1000000173c' 2024-09-21T00:30:06.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28793 fp=#0x10000001714' 2024-09-21T00:30:06.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28825 fp=#0x20000000374' 2024-09-21T00:30:06.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28921 fp=#0x10000001677' 2024-09-21T00:30:06.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28953 fp=#0x100000016e9' 2024-09-21T00:30:06.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28985 fp=#0x10000001610' 2024-09-21T00:30:06.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29017 fp=#0x10000001686' 2024-09-21T00:30:06.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29049 fp=#0x100000016f1' 2024-09-21T00:30:06.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29113 fp=#0x20000000b39' 2024-09-21T00:30:06.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29145 fp=#0x1000000435d' 2024-09-21T00:30:06.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29177 fp=#0x100000043cc' 2024-09-21T00:30:06.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29241 fp=#0x200000013d4' 2024-09-21T00:30:06.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29273 fp=#0x1000000433b' 2024-09-21T00:30:06.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29401 fp=#0x10000004ded' 2024-09-21T00:30:06.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29433 fp=#0x10000004ddd' 2024-09-21T00:30:06.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29497 fp=#0x10000004cd0' 2024-09-21T00:30:06.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29561 fp=#0x10000004dfa' 2024-09-21T00:30:06.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29593 fp=#0x100000040b5' 2024-09-21T00:30:06.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29625 fp=#0x1000000298f' 2024-09-21T00:30:06.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29753 fp=#0x1000000451c' 2024-09-21T00:30:06.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29785 fp=#0x10000004551' 2024-09-21T00:30:06.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29817 fp=#0x10000004540' 2024-09-21T00:30:06.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29849 fp=#0x20000000a34' 2024-09-21T00:30:06.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29881 fp=#0x10000004cf0' 2024-09-21T00:30:06.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29913 fp=#0x20000000dc9' 2024-09-21T00:30:06.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29945 fp=#0x20000000d3f' 2024-09-21T00:30:06.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29977 fp=#0x20000000e6e' 2024-09-21T00:30:06.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30009 fp=#0x20000000d99' 2024-09-21T00:30:06.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30041 fp=#0x20000000c26' 2024-09-21T00:30:06.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30073 fp=#0x20000000bef' 2024-09-21T00:30:06.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30137 fp=#0x20000000a49' 2024-09-21T00:30:06.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30169 fp=#0x10000001b51' 2024-09-21T00:30:06.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30201 fp=#0x100000055bc' 2024-09-21T00:30:06.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30233 fp=#0x10000001bb7' 2024-09-21T00:30:06.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30265 fp=#0x10000001c5e' 2024-09-21T00:30:06.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30297 fp=#0x20000000b67' 2024-09-21T00:30:06.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30329 fp=#0x20000000a31' 2024-09-21T00:30:06.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30361 fp=#0x20000000ba2' 2024-09-21T00:30:06.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30393 fp=#0x20000000a3b' 2024-09-21T00:30:06.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30425 fp=#0x100000028cb' 2024-09-21T00:30:06.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30457 fp=#0x20000000bba' 2024-09-21T00:30:06.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30521 fp=#0x200000006ee' 2024-09-21T00:30:06.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22778 fp=#0x10000001d89' 2024-09-21T00:30:06.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22842 fp=#0x10000001142' 2024-09-21T00:30:06.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26618 fp=#0x10000001e4a' 2024-09-21T00:30:06.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26682 fp=#0x200000007fb' 2024-09-21T00:30:06.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26746 fp=#0x200000007f6' 2024-09-21T00:30:06.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26810 fp=#0x100000043e6' 2024-09-21T00:30:06.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26842 fp=#0x100000043fc' 2024-09-21T00:30:06.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26906 fp=#0x100000043fa' 2024-09-21T00:30:06.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26938 fp=#0x1000000444b' 2024-09-21T00:30:06.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26970 fp=#0x20000000d6c' 2024-09-21T00:30:06.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27002 fp=#0x20000000d53' 2024-09-21T00:30:06.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27098 fp=#0x20000000d7d' 2024-09-21T00:30:06.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27130 fp=#0x20000000dbd' 2024-09-21T00:30:06.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27162 fp=#0x20000000fc8' 2024-09-21T00:30:06.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27194 fp=#0x20000000fb5' 2024-09-21T00:30:06.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27226 fp=#0x10000002230' 2024-09-21T00:30:06.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27258 fp=#0x20000000f3f' 2024-09-21T00:30:06.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27290 fp=#0x1000000220c' 2024-09-21T00:30:06.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27322 fp=#0x10000002254' 2024-09-21T00:30:06.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27354 fp=#0x10000002352' 2024-09-21T00:30:06.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27386 fp=#0x10000003b59' 2024-09-21T00:30:06.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27418 fp=#0x10000003cb6' 2024-09-21T00:30:06.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27450 fp=#0x10000003c3b' 2024-09-21T00:30:06.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27482 fp=#0x10000003bb7' 2024-09-21T00:30:06.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27514 fp=#0x10000003be1' 2024-09-21T00:30:06.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27642 fp=#0x10000001ef0' 2024-09-21T00:30:06.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27674 fp=#0x10000001f69' 2024-09-21T00:30:06.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27706 fp=#0x10000001e9b' 2024-09-21T00:30:06.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27738 fp=#0x10000001fc7' 2024-09-21T00:30:06.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27770 fp=#0x200000008c8' 2024-09-21T00:30:06.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27866 fp=#0x200000008c2' 2024-09-21T00:30:06.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27898 fp=#0x20000000946' 2024-09-21T00:30:06.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27930 fp=#0x20000001077' 2024-09-21T00:30:06.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27962 fp=#0x20000001091' 2024-09-21T00:30:06.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27994 fp=#0x1000000498f' 2024-09-21T00:30:06.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28026 fp=#0x100000049cd' 2024-09-21T00:30:06.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28058 fp=#0x1000000499e' 2024-09-21T00:30:06.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28090 fp=#0x2000000137b' 2024-09-21T00:30:06.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28122 fp=#0x10000004c11' 2024-09-21T00:30:06.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28154 fp=#0x10000004be2' 2024-09-21T00:30:06.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28186 fp=#0x10000004b91' 2024-09-21T00:30:06.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28218 fp=#0x10000004b62' 2024-09-21T00:30:06.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28250 fp=#0x10000004b4f' 2024-09-21T00:30:06.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28282 fp=#0x10000003234' 2024-09-21T00:30:06.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28314 fp=#0x20000000e9c' 2024-09-21T00:30:06.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28346 fp=#0x20000000ea5' 2024-09-21T00:30:06.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28378 fp=#0x1000000329e' 2024-09-21T00:30:06.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28410 fp=#0x100000032a2' 2024-09-21T00:30:06.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28442 fp=#0x100000031b9' 2024-09-21T00:30:06.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28474 fp=#0x100000030fa' 2024-09-21T00:30:06.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28506 fp=#0x20000000ff0' 2024-09-21T00:30:06.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28538 fp=#0x200000010f3' 2024-09-21T00:30:06.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28570 fp=#0x20000000f4a' 2024-09-21T00:30:06.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28602 fp=#0x200000011c2' 2024-09-21T00:30:06.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28634 fp=#0x20000000f8b' 2024-09-21T00:30:06.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28666 fp=#0x20000000e05' 2024-09-21T00:30:06.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28698 fp=#0x20000000dbf' 2024-09-21T00:30:06.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28730 fp=#0x1000000164f' 2024-09-21T00:30:06.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28762 fp=#0x1000000162c' 2024-09-21T00:30:06.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28794 fp=#0x10000001642' 2024-09-21T00:30:06.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28826 fp=#0x200000009f1' 2024-09-21T00:30:06.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28890 fp=#0x200000009c5' 2024-09-21T00:30:06.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28922 fp=#0x10000001664' 2024-09-21T00:30:06.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28954 fp=#0x10000001614' 2024-09-21T00:30:06.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28986 fp=#0x10000001694' 2024-09-21T00:30:06.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29018 fp=#0x100000015fc' 2024-09-21T00:30:06.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29050 fp=#0x1000000434d' 2024-09-21T00:30:06.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29082 fp=#0x100000043cd' 2024-09-21T00:30:06.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29146 fp=#0x10000004368' 2024-09-21T00:30:06.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29178 fp=#0x10000004391' 2024-09-21T00:30:06.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29210 fp=#0x10000004379' 2024-09-21T00:30:06.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29242 fp=#0x200000013bf' 2024-09-21T00:30:06.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29274 fp=#0x20000001416' 2024-09-21T00:30:06.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29402 fp=#0x10000004e4f' 2024-09-21T00:30:06.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29434 fp=#0x10000004de4' 2024-09-21T00:30:06.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29530 fp=#0x10000004deb' 2024-09-21T00:30:06.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29626 fp=#0x100000040ac' 2024-09-21T00:30:06.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29658 fp=#0x20000000ebb' 2024-09-21T00:30:06.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29690 fp=#0x1000000299a' 2024-09-21T00:30:06.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29722 fp=#0x100000029b1' 2024-09-21T00:30:06.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29754 fp=#0x10000004532' 2024-09-21T00:30:06.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29786 fp=#0x10000004563' 2024-09-21T00:30:06.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29818 fp=#0x10000004528' 2024-09-21T00:30:06.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29850 fp=#0x10000004564' 2024-09-21T00:30:06.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29914 fp=#0x20000000c6c' 2024-09-21T00:30:06.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29946 fp=#0x20000000df2' 2024-09-21T00:30:06.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29978 fp=#0x20000000dd9' 2024-09-21T00:30:06.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30010 fp=#0x20000000dd1' 2024-09-21T00:30:06.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30042 fp=#0x20000000a2b' 2024-09-21T00:30:06.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30106 fp=#0x20000000bc6' 2024-09-21T00:30:06.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30138 fp=#0x20000000bd5' 2024-09-21T00:30:06.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30170 fp=#0x10000001c43' 2024-09-21T00:30:06.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30202 fp=#0x10000001ba7' 2024-09-21T00:30:06.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30234 fp=#0x10000001af8' 2024-09-21T00:30:06.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30266 fp=#0x10000001c8f' 2024-09-21T00:30:06.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30298 fp=#0x1000000290f' 2024-09-21T00:30:06.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30330 fp=#0x20000000a54' 2024-09-21T00:30:06.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30362 fp=#0x20000000b6f' 2024-09-21T00:30:06.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30394 fp=#0x10000002975' 2024-09-21T00:30:06.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30426 fp=#0x10000002961' 2024-09-21T00:30:06.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30458 fp=#0x20000000b5a' 2024-09-21T00:30:06.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30490 fp=#0x100000023cd' 2024-09-21T00:30:06.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30522 fp=#0x100000023dc' 2024-09-21T00:30:06.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22779 fp=#0x2000000076a' 2024-09-21T00:30:06.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26619 fp=#0x10000001e56' 2024-09-21T00:30:06.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26651 fp=#0x20000000777' 2024-09-21T00:30:06.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26715 fp=#0x20000000781' 2024-09-21T00:30:06.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26811 fp=#0x1000000461e' 2024-09-21T00:30:06.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26843 fp=#0x10000004632' 2024-09-21T00:30:06.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26875 fp=#0x10000004479' 2024-09-21T00:30:06.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26907 fp=#0x100000043e5' 2024-09-21T00:30:06.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27003 fp=#0x20000000eee' 2024-09-21T00:30:06.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27035 fp=#0x20000000fea' 2024-09-21T00:30:06.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27067 fp=#0x20000001016' 2024-09-21T00:30:06.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27099 fp=#0x20000000ee9' 2024-09-21T00:30:06.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27131 fp=#0x20000000dd4' 2024-09-21T00:30:06.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27163 fp=#0x10000002358' 2024-09-21T00:30:06.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27195 fp=#0x10000002231' 2024-09-21T00:30:06.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27227 fp=#0x20000000fbd' 2024-09-21T00:30:06.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27259 fp=#0x20000001019' 2024-09-21T00:30:06.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27291 fp=#0x100000023fe' 2024-09-21T00:30:06.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27323 fp=#0x100000023d8' 2024-09-21T00:30:06.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27355 fp=#0x2000000102a' 2024-09-21T00:30:06.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27387 fp=#0x10000003b89' 2024-09-21T00:30:06.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27419 fp=#0x10000003c62' 2024-09-21T00:30:06.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27451 fp=#0x10000003c71' 2024-09-21T00:30:06.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27483 fp=#0x10000003cec' 2024-09-21T00:30:06.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27515 fp=#0x10000003c59' 2024-09-21T00:30:06.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27643 fp=#0x10000001f3d' 2024-09-21T00:30:06.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27675 fp=#0x10000001f9b' 2024-09-21T00:30:06.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27707 fp=#0x10000001f23' 2024-09-21T00:30:06.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27835 fp=#0x200000008dc' 2024-09-21T00:30:06.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27867 fp=#0x20000001034' 2024-09-21T00:30:06.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27899 fp=#0x20000001106' 2024-09-21T00:30:06.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27931 fp=#0x20000001167' 2024-09-21T00:30:06.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27963 fp=#0x100000049a8' 2024-09-21T00:30:06.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27995 fp=#0x10000004973' 2024-09-21T00:30:06.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28027 fp=#0x100000049ca' 2024-09-21T00:30:06.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28059 fp=#0x2000000134e' 2024-09-21T00:30:06.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28091 fp=#0x20000001356' 2024-09-21T00:30:06.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28123 fp=#0x10000004bd0' 2024-09-21T00:30:06.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28187 fp=#0x10000004bb9' 2024-09-21T00:30:06.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28219 fp=#0x10000004c01' 2024-09-21T00:30:06.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28251 fp=#0x10000004bd7' 2024-09-21T00:30:06.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28283 fp=#0x10000003027' 2024-09-21T00:30:06.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28315 fp=#0x20000000e99' 2024-09-21T00:30:06.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28347 fp=#0x100000030af' 2024-09-21T00:30:06.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28379 fp=#0x10000004bd4' 2024-09-21T00:30:06.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28411 fp=#0x100000032af' 2024-09-21T00:30:06.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28443 fp=#0x100000032e9' 2024-09-21T00:30:06.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28475 fp=#0x100000032d5' 2024-09-21T00:30:06.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28507 fp=#0x200000011d2' 2024-09-21T00:30:06.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28539 fp=#0x20000001020' 2024-09-21T00:30:06.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28571 fp=#0x20000000fc0' 2024-09-21T00:30:06.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28603 fp=#0x2000000100c' 2024-09-21T00:30:06.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28635 fp=#0x20000000f99' 2024-09-21T00:30:06.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28667 fp=#0x20000000dab' 2024-09-21T00:30:06.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28699 fp=#0x20000000d7a' 2024-09-21T00:30:06.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28731 fp=#0x10000001736' 2024-09-21T00:30:06.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28763 fp=#0x100000015f3' 2024-09-21T00:30:06.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28795 fp=#0x200000010a4' 2024-09-21T00:30:06.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28827 fp=#0x200000009ec' 2024-09-21T00:30:06.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28923 fp=#0x10000001625' 2024-09-21T00:30:06.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28955 fp=#0x20000000318' 2024-09-21T00:30:06.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28987 fp=#0x10000001636' 2024-09-21T00:30:06.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29019 fp=#0x10000001679' 2024-09-21T00:30:06.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29051 fp=#0x100000043ca' 2024-09-21T00:30:06.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29083 fp=#0x100000043d6' 2024-09-21T00:30:06.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29147 fp=#0x100000043d2' 2024-09-21T00:30:06.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29179 fp=#0x10000004384' 2024-09-21T00:30:06.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29211 fp=#0x10000004352' 2024-09-21T00:30:06.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29243 fp=#0x100000043b3' 2024-09-21T00:30:06.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29275 fp=#0x1000000438d' 2024-09-21T00:30:06.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29403 fp=#0x10000004cb2' 2024-09-21T00:30:06.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29467 fp=#0x20000000fee' 2024-09-21T00:30:06.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29499 fp=#0x10000004ddf' 2024-09-21T00:30:06.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29531 fp=#0x10000004dd7' 2024-09-21T00:30:06.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29595 fp=#0x100000040ca' 2024-09-21T00:30:06.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29627 fp=#0x100000040a4' 2024-09-21T00:30:06.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29659 fp=#0x10000002a8a' 2024-09-21T00:30:06.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29691 fp=#0x10000002a6f' 2024-09-21T00:30:06.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29723 fp=#0x100000040ce' 2024-09-21T00:30:06.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29755 fp=#0x10000004546' 2024-09-21T00:30:06.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29787 fp=#0x1000000453d' 2024-09-21T00:30:06.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29819 fp=#0x10000004573' 2024-09-21T00:30:06.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29851 fp=#0x1000000456f' 2024-09-21T00:30:06.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29883 fp=#0x10000004d75' 2024-09-21T00:30:06.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29915 fp=#0x20000000c5e' 2024-09-21T00:30:06.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29947 fp=#0x20000000d56' 2024-09-21T00:30:06.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29979 fp=#0x20000000ce9' 2024-09-21T00:30:06.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30011 fp=#0x20000000da3' 2024-09-21T00:30:06.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30043 fp=#0x20000000c6b' 2024-09-21T00:30:06.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30075 fp=#0x20000000c5a' 2024-09-21T00:30:06.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30107 fp=#0x20000000a39' 2024-09-21T00:30:06.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30171 fp=#0x10000001b1a' 2024-09-21T00:30:06.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30203 fp=#0x10000001c79' 2024-09-21T00:30:06.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30235 fp=#0x10000001bb4' 2024-09-21T00:30:06.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30267 fp=#0x100000055bf' 2024-09-21T00:30:06.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30299 fp=#0x100000028c3' 2024-09-21T00:30:06.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30363 fp=#0x20000000ab1' 2024-09-21T00:30:06.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30395 fp=#0x10000002902' 2024-09-21T00:30:06.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30427 fp=#0x20000000a26' 2024-09-21T00:30:06.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30459 fp=#0x1000000298a' 2024-09-21T00:30:06.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30523 fp=#0x200000006f2' 2024-09-21T00:30:06.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22780 fp=#0x100000043d7' 2024-09-21T00:30:06.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26620 fp=#0x10000001df3' 2024-09-21T00:30:06.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26652 fp=#0x2000000077d' 2024-09-21T00:30:06.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26684 fp=#0x20000000789' 2024-09-21T00:30:06.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26716 fp=#0x200000007f2' 2024-09-21T00:30:06.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26812 fp=#0x10000004627' 2024-09-21T00:30:06.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26972 fp=#0x20000000f08' 2024-09-21T00:30:06.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27004 fp=#0x20000000df1' 2024-09-21T00:30:06.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27036 fp=#0x20000000fc6' 2024-09-21T00:30:06.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27068 fp=#0x20000000ed8' 2024-09-21T00:30:06.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27100 fp=#0x20000000ddb' 2024-09-21T00:30:06.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27164 fp=#0x20000000f78' 2024-09-21T00:30:06.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27196 fp=#0x100000023f2' 2024-09-21T00:30:06.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27228 fp=#0x2000000102e' 2024-09-21T00:30:06.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27260 fp=#0x10000002377' 2024-09-21T00:30:06.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27292 fp=#0x10000002292' 2024-09-21T00:30:06.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27324 fp=#0x10000002287' 2024-09-21T00:30:06.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27356 fp=#0x20000000fd4' 2024-09-21T00:30:06.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27388 fp=#0x10000003b7d' 2024-09-21T00:30:06.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27420 fp=#0x10000003d07' 2024-09-21T00:30:06.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27452 fp=#0x2000000111e' 2024-09-21T00:30:06.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27484 fp=#0x10000003c4d' 2024-09-21T00:30:06.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27516 fp=#0x10000003c02' 2024-09-21T00:30:06.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27644 fp=#0x10000001f2c' 2024-09-21T00:30:07.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27676 fp=#0x10000001f74' 2024-09-21T00:30:07.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27708 fp=#0x10000001ed0' 2024-09-21T00:30:07.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27740 fp=#0x10000001edf' 2024-09-21T00:30:07.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27804 fp=#0x20000000ae9' 2024-09-21T00:30:07.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27836 fp=#0x20000000966' 2024-09-21T00:30:07.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27868 fp=#0x200000008f2' 2024-09-21T00:30:07.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27900 fp=#0x20000001090' 2024-09-21T00:30:07.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27932 fp=#0x20000001118' 2024-09-21T00:30:07.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27964 fp=#0x10000004a4a' 2024-09-21T00:30:07.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27996 fp=#0x10000004a59' 2024-09-21T00:30:07.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28028 fp=#0x10000004986' 2024-09-21T00:30:07.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28060 fp=#0x20000001325' 2024-09-21T00:30:07.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28092 fp=#0x200000012cc' 2024-09-21T00:30:07.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28124 fp=#0x10000004c16' 2024-09-21T00:30:07.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28156 fp=#0x10000004b67' 2024-09-21T00:30:07.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28188 fp=#0x10000004c12' 2024-09-21T00:30:07.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28220 fp=#0x10000004bbd' 2024-09-21T00:30:07.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28252 fp=#0x10000004bb5' 2024-09-21T00:30:07.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28284 fp=#0x20000000e7a' 2024-09-21T00:30:07.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28316 fp=#0x100000030b3' 2024-09-21T00:30:07.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28348 fp=#0x20000000e9f' 2024-09-21T00:30:07.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28380 fp=#0x20000000f0c' 2024-09-21T00:30:07.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28412 fp=#0x1000000301c' 2024-09-21T00:30:07.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28444 fp=#0x10000003102' 2024-09-21T00:30:07.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28476 fp=#0x1000000304f' 2024-09-21T00:30:07.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28508 fp=#0x20000000f51' 2024-09-21T00:30:07.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28540 fp=#0x2000000114f' 2024-09-21T00:30:07.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28572 fp=#0x20000000f4d' 2024-09-21T00:30:07.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28604 fp=#0x20000000ff6' 2024-09-21T00:30:07.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28636 fp=#0x20000001009' 2024-09-21T00:30:07.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28668 fp=#0x20000000ddc' 2024-09-21T00:30:07.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28700 fp=#0x20000000db4' 2024-09-21T00:30:07.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28732 fp=#0x1000000161c' 2024-09-21T00:30:07.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28764 fp=#0x1000000162a' 2024-09-21T00:30:07.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28796 fp=#0x10000001697' 2024-09-21T00:30:07.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28828 fp=#0x200000009c2' 2024-09-21T00:30:07.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28892 fp=#0x200000009c1' 2024-09-21T00:30:07.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28924 fp=#0x20000000319' 2024-09-21T00:30:07.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28956 fp=#0x20000000306' 2024-09-21T00:30:07.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28988 fp=#0x1000000172f' 2024-09-21T00:30:07.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29020 fp=#0x20000000302' 2024-09-21T00:30:07.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29084 fp=#0x20000001408' 2024-09-21T00:30:07.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29116 fp=#0x100000043b8' 2024-09-21T00:30:07.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29148 fp=#0x200000013e4' 2024-09-21T00:30:07.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29180 fp=#0x10000004367' 2024-09-21T00:30:07.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29212 fp=#0x1000000436f' 2024-09-21T00:30:07.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29244 fp=#0x100000043ac' 2024-09-21T00:30:07.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29404 fp=#0x10000004e33' 2024-09-21T00:30:07.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29500 fp=#0x20000001008' 2024-09-21T00:30:07.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29532 fp=#0x10000004cb4' 2024-09-21T00:30:07.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29564 fp=#0x10000002a6c' 2024-09-21T00:30:07.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29596 fp=#0x100000040d3' 2024-09-21T00:30:07.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29628 fp=#0x20000000ec7' 2024-09-21T00:30:07.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29660 fp=#0x100000040b7' 2024-09-21T00:30:07.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29756 fp=#0x10000004542' 2024-09-21T00:30:07.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29788 fp=#0x1000000456a' 2024-09-21T00:30:07.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29820 fp=#0x10000004565' 2024-09-21T00:30:07.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29852 fp=#0x1000000452d' 2024-09-21T00:30:07.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29884 fp=#0x10000004d7d' 2024-09-21T00:30:07.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29916 fp=#0x20000000df9' 2024-09-21T00:30:07.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29948 fp=#0x20000000c97' 2024-09-21T00:30:07.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29980 fp=#0x20000000c95' 2024-09-21T00:30:07.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30012 fp=#0x20000000cb7' 2024-09-21T00:30:07.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30044 fp=#0x20000000bdc' 2024-09-21T00:30:07.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30076 fp=#0x20000000c08' 2024-09-21T00:30:07.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30140 fp=#0x20000000a41' 2024-09-21T00:30:07.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30172 fp=#0x10000001b76' 2024-09-21T00:30:07.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30204 fp=#0x10000001c8c' 2024-09-21T00:30:07.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30236 fp=#0x10000001b4e' 2024-09-21T00:30:07.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30268 fp=#0x10000001b63' 2024-09-21T00:30:07.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30300 fp=#0x100000029af' 2024-09-21T00:30:07.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30332 fp=#0x100000028f2' 2024-09-21T00:30:07.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30364 fp=#0x20000000b61' 2024-09-21T00:30:07.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30396 fp=#0x1000000293d' 2024-09-21T00:30:07.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30428 fp=#0x10000002900' 2024-09-21T00:30:07.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30460 fp=#0x20000000b81' 2024-09-21T00:30:07.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30524 fp=#0x100000023ba' 2024-09-21T00:30:07.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30556 fp=#0x10000002385' 2024-09-21T00:30:07.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22781 fp=#0x20000000d4f' 2024-09-21T00:30:07.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:25309 fp=#0x20000001388' 2024-09-21T00:30:07.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26621 fp=#0x20000000783' 2024-09-21T00:30:07.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26653 fp=#0x2000000113c' 2024-09-21T00:30:07.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26685 fp=#0x20000000a1e' 2024-09-21T00:30:07.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26813 fp=#0x10000004458' 2024-09-21T00:30:07.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26845 fp=#0x10000004456' 2024-09-21T00:30:07.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26877 fp=#0x100000043f9' 2024-09-21T00:30:07.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26941 fp=#0x1000000463d' 2024-09-21T00:30:07.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26973 fp=#0x20000000d9d' 2024-09-21T00:30:07.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27005 fp=#0x20000000ee8' 2024-09-21T00:30:07.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27037 fp=#0x20000000d55' 2024-09-21T00:30:07.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27069 fp=#0x20000000edc' 2024-09-21T00:30:07.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27101 fp=#0x20000000f4b' 2024-09-21T00:30:07.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27133 fp=#0x20000000ee6' 2024-09-21T00:30:07.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27165 fp=#0x20000000f17' 2024-09-21T00:30:07.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27229 fp=#0x20000000f5e' 2024-09-21T00:30:07.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27261 fp=#0x10000002515' 2024-09-21T00:30:07.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27293 fp=#0x20000000f32' 2024-09-21T00:30:07.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27325 fp=#0x20000000d4b' 2024-09-21T00:30:07.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27357 fp=#0x1000000221a' 2024-09-21T00:30:07.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27389 fp=#0x10000003c3e' 2024-09-21T00:30:07.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27421 fp=#0x10000003cc4' 2024-09-21T00:30:07.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27453 fp=#0x10000003d1e' 2024-09-21T00:30:07.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27485 fp=#0x10000003c6b' 2024-09-21T00:30:07.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27517 fp=#0x10000003cb0' 2024-09-21T00:30:07.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27645 fp=#0x10000001ed4' 2024-09-21T00:30:07.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27677 fp=#0x10000001e98' 2024-09-21T00:30:07.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27709 fp=#0x10000001fce' 2024-09-21T00:30:07.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27741 fp=#0x10000002031' 2024-09-21T00:30:07.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27805 fp=#0x20000001029' 2024-09-21T00:30:07.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27901 fp=#0x20000001079' 2024-09-21T00:30:07.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27933 fp=#0x200000010d4' 2024-09-21T00:30:07.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27965 fp=#0x1000000497d' 2024-09-21T00:30:07.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27997 fp=#0x100000049ef' 2024-09-21T00:30:07.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28029 fp=#0x10000004a2b' 2024-09-21T00:30:07.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28061 fp=#0x2000000128d' 2024-09-21T00:30:07.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28093 fp=#0x20000001242' 2024-09-21T00:30:07.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28125 fp=#0x10000004b49' 2024-09-21T00:30:07.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28157 fp=#0x10000004bd9' 2024-09-21T00:30:07.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28189 fp=#0x10000004ba5' 2024-09-21T00:30:07.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28221 fp=#0x10000004ae0' 2024-09-21T00:30:07.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28253 fp=#0x100000030a0' 2024-09-21T00:30:07.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28285 fp=#0x10000004c4f' 2024-09-21T00:30:07.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28317 fp=#0x100000032dc' 2024-09-21T00:30:07.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28349 fp=#0x20000000f5a' 2024-09-21T00:30:07.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28381 fp=#0x10000003363' 2024-09-21T00:30:07.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28413 fp=#0x10000004c5e' 2024-09-21T00:30:07.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28445 fp=#0x10000004c4d' 2024-09-21T00:30:07.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28477 fp=#0x100000030f2' 2024-09-21T00:30:07.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28605 fp=#0x20000000f46' 2024-09-21T00:30:07.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28637 fp=#0x20000001110' 2024-09-21T00:30:07.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28669 fp=#0x20000000d8e' 2024-09-21T00:30:07.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28701 fp=#0x20000000e09' 2024-09-21T00:30:07.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28733 fp=#0x10000001621' 2024-09-21T00:30:07.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28765 fp=#0x10000001635' 2024-09-21T00:30:07.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28797 fp=#0x200000006a0' 2024-09-21T00:30:07.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28829 fp=#0x200000009eb' 2024-09-21T00:30:07.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28861 fp=#0x200000009c7' 2024-09-21T00:30:07.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28893 fp=#0x200000009ea' 2024-09-21T00:30:07.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28925 fp=#0x10000001691' 2024-09-21T00:30:07.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28957 fp=#0x1000000168e' 2024-09-21T00:30:07.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28989 fp=#0x100000016aa' 2024-09-21T00:30:07.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29021 fp=#0x10000001605' 2024-09-21T00:30:07.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29053 fp=#0x100000043c5' 2024-09-21T00:30:07.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29085 fp=#0x10000004380' 2024-09-21T00:30:07.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29117 fp=#0x10000004344' 2024-09-21T00:30:07.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29149 fp=#0x100000043a5' 2024-09-21T00:30:07.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29181 fp=#0x200000013ce' 2024-09-21T00:30:07.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29213 fp=#0x100000043c8' 2024-09-21T00:30:07.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29277 fp=#0x200000013b5' 2024-09-21T00:30:07.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29405 fp=#0x10000004c59' 2024-09-21T00:30:07.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29437 fp=#0x10000004e0f' 2024-09-21T00:30:07.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29533 fp=#0x10000004e21' 2024-09-21T00:30:07.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29565 fp=#0x100000040b4' 2024-09-21T00:30:07.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29629 fp=#0x100000029a4' 2024-09-21T00:30:07.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29661 fp=#0x20000000ec4' 2024-09-21T00:30:07.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29693 fp=#0x100000040ba' 2024-09-21T00:30:07.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29757 fp=#0x1000000452a' 2024-09-21T00:30:07.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29789 fp=#0x1000000456e' 2024-09-21T00:30:07.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29821 fp=#0x1000000455f' 2024-09-21T00:30:07.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29853 fp=#0x1000000457a' 2024-09-21T00:30:07.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29885 fp=#0x10000004d55' 2024-09-21T00:30:07.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29917 fp=#0x20000001111' 2024-09-21T00:30:07.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29949 fp=#0x20000000cb3' 2024-09-21T00:30:07.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29981 fp=#0x20000000dc3' 2024-09-21T00:30:07.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30013 fp=#0x20000000d9f' 2024-09-21T00:30:07.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30045 fp=#0x20000000c72' 2024-09-21T00:30:07.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30109 fp=#0x20000000c18' 2024-09-21T00:30:07.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30141 fp=#0x20000000c24' 2024-09-21T00:30:07.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30173 fp=#0x10000001bce' 2024-09-21T00:30:07.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30205 fp=#0x10000001afb' 2024-09-21T00:30:07.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30237 fp=#0x10000001c61' 2024-09-21T00:30:07.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30269 fp=#0x10000001b33' 2024-09-21T00:30:07.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30301 fp=#0x20000000a71' 2024-09-21T00:30:07.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30333 fp=#0x100000028df' 2024-09-21T00:30:07.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30365 fp=#0x100000028e3' 2024-09-21T00:30:07.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30397 fp=#0x10000002949' 2024-09-21T00:30:07.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30429 fp=#0x20000000a3e' 2024-09-21T00:30:07.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30461 fp=#0x20000000a92' 2024-09-21T00:30:07.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30525 fp=#0x100000023a0' 2024-09-21T00:30:07.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30557 fp=#0x100000023ae' 2024-09-21T00:30:07.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22782 fp=#0x100000021f6' 2024-09-21T00:30:07.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:25278 fp=#0x2000000138b' 2024-09-21T00:30:07.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26686 fp=#0x200000007a3' 2024-09-21T00:30:07.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26782 fp=#0x2000000077b' 2024-09-21T00:30:07.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26814 fp=#0x10000004617' 2024-09-21T00:30:07.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26846 fp=#0x1000000462d' 2024-09-21T00:30:07.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26878 fp=#0x100000043dc' 2024-09-21T00:30:07.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26910 fp=#0x100000043d8' 2024-09-21T00:30:07.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26942 fp=#0x1000000462e' 2024-09-21T00:30:07.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26974 fp=#0x20000000f04' 2024-09-21T00:30:07.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27006 fp=#0x20000000ed4' 2024-09-21T00:30:07.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27038 fp=#0x20000000fbb' 2024-09-21T00:30:07.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27102 fp=#0x20000000f67' 2024-09-21T00:30:07.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27134 fp=#0x20000000ed5' 2024-09-21T00:30:07.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27166 fp=#0x10000002517' 2024-09-21T00:30:07.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27198 fp=#0x10000002298' 2024-09-21T00:30:07.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27230 fp=#0x1000000224d' 2024-09-21T00:30:07.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27262 fp=#0x10000002349' 2024-09-21T00:30:07.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27294 fp=#0x10000002285' 2024-09-21T00:30:07.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27326 fp=#0x20000000f70' 2024-09-21T00:30:07.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27358 fp=#0x1000000250c' 2024-09-21T00:30:07.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27390 fp=#0x10000003d0d' 2024-09-21T00:30:07.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27422 fp=#0x200000008e2' 2024-09-21T00:30:07.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27454 fp=#0x20000001157' 2024-09-21T00:30:07.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27486 fp=#0x10000003be7' 2024-09-21T00:30:07.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27518 fp=#0x10000003c56' 2024-09-21T00:30:07.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27646 fp=#0x10000002087' 2024-09-21T00:30:07.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27678 fp=#0x10000001fc2' 2024-09-21T00:30:07.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27710 fp=#0x10000001fb4' 2024-09-21T00:30:07.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27742 fp=#0x10000001eba' 2024-09-21T00:30:07.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27774 fp=#0x200000008ee' 2024-09-21T00:30:07.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27806 fp=#0x200000008e8' 2024-09-21T00:30:07.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27838 fp=#0x200000008e9' 2024-09-21T00:30:07.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27870 fp=#0x200000008c6' 2024-09-21T00:30:07.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27902 fp=#0x2000000115f' 2024-09-21T00:30:07.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27934 fp=#0x20000001068' 2024-09-21T00:30:07.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27966 fp=#0x10000004a18' 2024-09-21T00:30:07.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27998 fp=#0x10000004a0f' 2024-09-21T00:30:07.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28030 fp=#0x10000004a1e' 2024-09-21T00:30:07.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28062 fp=#0x2000000131d' 2024-09-21T00:30:07.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28094 fp=#0x20000001275' 2024-09-21T00:30:07.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28126 fp=#0x10000004bfd' 2024-09-21T00:30:07.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28158 fp=#0x10000004bb3' 2024-09-21T00:30:07.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28190 fp=#0x10000004c03' 2024-09-21T00:30:07.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28222 fp=#0x10000004b6e' 2024-09-21T00:30:07.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28254 fp=#0x10000003342' 2024-09-21T00:30:07.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28286 fp=#0x1000000334d' 2024-09-21T00:30:07.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28318 fp=#0x10000003176' 2024-09-21T00:30:07.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28350 fp=#0x20000000e62' 2024-09-21T00:30:07.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28382 fp=#0x20000000e44' 2024-09-21T00:30:07.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28414 fp=#0x1000000326a' 2024-09-21T00:30:07.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28446 fp=#0x20000000e1d' 2024-09-21T00:30:07.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28478 fp=#0x100000030fe' 2024-09-21T00:30:07.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28510 fp=#0x20000000fc5' 2024-09-21T00:30:07.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28542 fp=#0x20000001163' 2024-09-21T00:30:07.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28606 fp=#0x20000000fcc' 2024-09-21T00:30:07.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28638 fp=#0x20000001037' 2024-09-21T00:30:07.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28670 fp=#0x20000000dd2' 2024-09-21T00:30:07.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28702 fp=#0x20000000d46' 2024-09-21T00:30:07.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28734 fp=#0x10000001699' 2024-09-21T00:30:07.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28766 fp=#0x1000000168d' 2024-09-21T00:30:07.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28798 fp=#0x1000000160a' 2024-09-21T00:30:07.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28862 fp=#0x200000009ce' 2024-09-21T00:30:07.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28926 fp=#0x20000000317' 2024-09-21T00:30:07.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28958 fp=#0x100000016ad' 2024-09-21T00:30:07.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28990 fp=#0x10000001654' 2024-09-21T00:30:07.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29022 fp=#0x100000016b6' 2024-09-21T00:30:07.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29054 fp=#0x1000000433e' 2024-09-21T00:30:07.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29086 fp=#0x1000000435c' 2024-09-21T00:30:07.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29182 fp=#0x200000013e0' 2024-09-21T00:30:07.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29214 fp=#0x1000000439f' 2024-09-21T00:30:07.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29246 fp=#0x100000043c3' 2024-09-21T00:30:07.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29438 fp=#0x10000004ce8' 2024-09-21T00:30:07.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29502 fp=#0x10000004e19' 2024-09-21T00:30:07.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29566 fp=#0x10000002a5d' 2024-09-21T00:30:07.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29598 fp=#0x100000040c3' 2024-09-21T00:30:07.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29630 fp=#0x100000029ae' 2024-09-21T00:30:07.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29694 fp=#0x100000040bc' 2024-09-21T00:30:07.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29758 fp=#0x10000004584' 2024-09-21T00:30:07.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29790 fp=#0x10000004515' 2024-09-21T00:30:07.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29822 fp=#0x10000004539' 2024-09-21T00:30:07.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29854 fp=#0x10000004522' 2024-09-21T00:30:07.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29886 fp=#0x20000000f79' 2024-09-21T00:30:07.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29918 fp=#0x20000000dea' 2024-09-21T00:30:07.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29950 fp=#0x20000000c9d' 2024-09-21T00:30:07.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29982 fp=#0x20000000df5' 2024-09-21T00:30:07.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30014 fp=#0x20000000dd8' 2024-09-21T00:30:07.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30078 fp=#0x20000000a3c' 2024-09-21T00:30:07.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30142 fp=#0x20000000bce' 2024-09-21T00:30:07.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30174 fp=#0x10000001bbc' 2024-09-21T00:30:07.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30206 fp=#0x10000001b8d' 2024-09-21T00:30:07.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30238 fp=#0x10000001c4e' 2024-09-21T00:30:07.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30270 fp=#0x10000001c39' 2024-09-21T00:30:07.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30302 fp=#0x20000000a7d' 2024-09-21T00:30:07.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30334 fp=#0x20000000a24' 2024-09-21T00:30:07.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30366 fp=#0x20000000b93' 2024-09-21T00:30:07.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30398 fp=#0x20000000a69' 2024-09-21T00:30:07.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30430 fp=#0x20000000acb' 2024-09-21T00:30:07.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30462 fp=#0x20000000a28' 2024-09-21T00:30:07.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30494 fp=#0x100000023a9' 2024-09-21T00:30:07.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30558 fp=#0x200000006f5' 2024-09-21T00:30:07.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22783 fp=#0x10000003b4f' 2024-09-21T00:30:07.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26655 fp=#0x20000000771' 2024-09-21T00:30:07.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26719 fp=#0x20000000797' 2024-09-21T00:30:07.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26783 fp=#0x20000000801' 2024-09-21T00:30:07.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26815 fp=#0x10000004445' 2024-09-21T00:30:07.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26847 fp=#0x1000000462a' 2024-09-21T00:30:07.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26879 fp=#0x1000000461d' 2024-09-21T00:30:07.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26911 fp=#0x10000004621' 2024-09-21T00:30:07.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26943 fp=#0x1000000446f' 2024-09-21T00:30:07.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26975 fp=#0x20000000fd8' 2024-09-21T00:30:07.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27007 fp=#0x20000000f05' 2024-09-21T00:30:07.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27039 fp=#0x20000000daa' 2024-09-21T00:30:07.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27071 fp=#0x20000000d69' 2024-09-21T00:30:07.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27103 fp=#0x20000000f4e' 2024-09-21T00:30:07.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27135 fp=#0x1000000222b' 2024-09-21T00:30:07.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27199 fp=#0x20000000f52' 2024-09-21T00:30:07.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27231 fp=#0x10000002262' 2024-09-21T00:30:07.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27263 fp=#0x100000022a0' 2024-09-21T00:30:07.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27295 fp=#0x20000000d4e' 2024-09-21T00:30:07.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27327 fp=#0x20000000f8a' 2024-09-21T00:30:07.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27359 fp=#0x100000021fb' 2024-09-21T00:30:07.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27391 fp=#0x10000003ba8' 2024-09-21T00:30:07.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27423 fp=#0x10000003bc6' 2024-09-21T00:30:07.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27455 fp=#0x10000003c50' 2024-09-21T00:30:07.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27487 fp=#0x10000003cf7' 2024-09-21T00:30:07.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27647 fp=#0x10000001f6f' 2024-09-21T00:30:07.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27679 fp=#0x10000001f17' 2024-09-21T00:30:07.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27711 fp=#0x10000001f08' 2024-09-21T00:30:07.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27743 fp=#0x10000001f91' 2024-09-21T00:30:07.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27839 fp=#0x20000001018' 2024-09-21T00:30:07.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27903 fp=#0x2000000116d' 2024-09-21T00:30:07.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27935 fp=#0x20000001074' 2024-09-21T00:30:07.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27967 fp=#0x100000049bf' 2024-09-21T00:30:07.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27999 fp=#0x100000049f5' 2024-09-21T00:30:07.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28031 fp=#0x100000049b6' 2024-09-21T00:30:07.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28063 fp=#0x2000000124d' 2024-09-21T00:30:07.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28095 fp=#0x200000012f2' 2024-09-21T00:30:07.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28127 fp=#0x10000004ab0' 2024-09-21T00:30:07.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28159 fp=#0x10000004c09' 2024-09-21T00:30:07.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28223 fp=#0x10000004b3c' 2024-09-21T00:30:07.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28255 fp=#0x20000000e36' 2024-09-21T00:30:07.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28319 fp=#0x20000000e57' 2024-09-21T00:30:07.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28351 fp=#0x100000030e1' 2024-09-21T00:30:07.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28383 fp=#0x20000000f29' 2024-09-21T00:30:07.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28415 fp=#0x100000030d7' 2024-09-21T00:30:07.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28447 fp=#0x100000031a5' 2024-09-21T00:30:07.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28479 fp=#0x10000003163' 2024-09-21T00:30:07.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28511 fp=#0x20000001221' 2024-09-21T00:30:07.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28543 fp=#0x20000000f43' 2024-09-21T00:30:07.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28575 fp=#0x2000000111b' 2024-09-21T00:30:07.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28607 fp=#0x200000011e6' 2024-09-21T00:30:07.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28639 fp=#0x2000000117e' 2024-09-21T00:30:07.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28671 fp=#0x20000000d8a' 2024-09-21T00:30:07.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28703 fp=#0x20000000d68' 2024-09-21T00:30:07.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28735 fp=#0x100000015e9' 2024-09-21T00:30:07.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28767 fp=#0x10000001637' 2024-09-21T00:30:07.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28799 fp=#0x100000016d6' 2024-09-21T00:30:07.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28831 fp=#0x200000009fa' 2024-09-21T00:30:07.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28863 fp=#0x20000001168' 2024-09-21T00:30:07.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28927 fp=#0x1000000160d' 2024-09-21T00:30:07.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28959 fp=#0x10000001692' 2024-09-21T00:30:07.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28991 fp=#0x20000000309' 2024-09-21T00:30:07.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29023 fp=#0x1000000174c' 2024-09-21T00:30:07.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29055 fp=#0x10000004377' 2024-09-21T00:30:07.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29087 fp=#0x100000043d1' 2024-09-21T00:30:07.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29119 fp=#0x100000043a6' 2024-09-21T00:30:07.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29151 fp=#0x1000000439d' 2024-09-21T00:30:07.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29215 fp=#0x100000043a8' 2024-09-21T00:30:07.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29247 fp=#0x200000013d6' 2024-09-21T00:30:07.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29279 fp=#0x10000004372' 2024-09-21T00:30:07.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29407 fp=#0x10000004e45' 2024-09-21T00:30:07.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29503 fp=#0x10000004cdc' 2024-09-21T00:30:07.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29535 fp=#0x20000000fff' 2024-09-21T00:30:07.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29567 fp=#0x10000002980' 2024-09-21T00:30:07.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29599 fp=#0x100000040a1' 2024-09-21T00:30:07.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29631 fp=#0x20000000eb2' 2024-09-21T00:30:07.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29663 fp=#0x10000002a72' 2024-09-21T00:30:07.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29695 fp=#0x100000029a9' 2024-09-21T00:30:07.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29727 fp=#0x20000000ec9' 2024-09-21T00:30:07.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29759 fp=#0x10000004550' 2024-09-21T00:30:07.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29791 fp=#0x10000004537' 2024-09-21T00:30:07.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29823 fp=#0x10000004555' 2024-09-21T00:30:07.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29855 fp=#0x10000004558' 2024-09-21T00:30:07.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29919 fp=#0x20000000e5e' 2024-09-21T00:30:07.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29951 fp=#0x20000000db5' 2024-09-21T00:30:07.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29983 fp=#0x20000000cf7' 2024-09-21T00:30:07.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30015 fp=#0x20000000d30' 2024-09-21T00:30:07.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30047 fp=#0x20000000a3f' 2024-09-21T00:30:07.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30111 fp=#0x20000000a4c' 2024-09-21T00:30:07.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30175 fp=#0x10000001b56' 2024-09-21T00:30:07.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30207 fp=#0x100000055c2' 2024-09-21T00:30:07.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30239 fp=#0x10000001b95' 2024-09-21T00:30:07.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30271 fp=#0x10000001c4c' 2024-09-21T00:30:07.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30303 fp=#0x100000028eb' 2024-09-21T00:30:07.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30335 fp=#0x20000000a4a' 2024-09-21T00:30:07.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30399 fp=#0x100000028bf' 2024-09-21T00:30:07.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30431 fp=#0x20000000b00' 2024-09-21T00:30:07.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30463 fp=#0x20000000ab8' 2024-09-21T00:30:07.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30495 fp=#0x10000002397' 2024-09-21T00:30:07.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30559 fp=#0x200000006ec' 2024-09-21T00:30:07.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26720 fp=#0x20000000a13' 2024-09-21T00:30:07.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26752 fp=#0x20000000a20' 2024-09-21T00:30:07.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26784 fp=#0x10000004475' 2024-09-21T00:30:07.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26816 fp=#0x1000000446d' 2024-09-21T00:30:07.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26848 fp=#0x1000000463e' 2024-09-21T00:30:07.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26880 fp=#0x10000004624' 2024-09-21T00:30:07.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26944 fp=#0x10000004634' 2024-09-21T00:30:07.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26976 fp=#0x20000000fae' 2024-09-21T00:30:07.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27008 fp=#0x20000000eed' 2024-09-21T00:30:07.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27040 fp=#0x20000000fdf' 2024-09-21T00:30:07.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27104 fp=#0x20000000ee2' 2024-09-21T00:30:07.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27136 fp=#0x10000002512' 2024-09-21T00:30:07.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27168 fp=#0x20000000fde' 2024-09-21T00:30:07.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27200 fp=#0x20000000f56' 2024-09-21T00:30:07.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27264 fp=#0x10000002260' 2024-09-21T00:30:07.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27296 fp=#0x10000002233' 2024-09-21T00:30:07.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27328 fp=#0x20000000f74' 2024-09-21T00:30:07.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27360 fp=#0x20000000ff4' 2024-09-21T00:30:07.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27392 fp=#0x10000003d03' 2024-09-21T00:30:07.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27424 fp=#0x10000003bab' 2024-09-21T00:30:07.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27456 fp=#0x10000003cfc' 2024-09-21T00:30:07.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27488 fp=#0x10000003c7b' 2024-09-21T00:30:07.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27520 fp=#0x2000000117d' 2024-09-21T00:30:07.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27648 fp=#0x10000002076' 2024-09-21T00:30:07.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27680 fp=#0x10000001fe5' 2024-09-21T00:30:07.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27712 fp=#0x10000001f85' 2024-09-21T00:30:07.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27744 fp=#0x10000001faf' 2024-09-21T00:30:07.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27872 fp=#0x20000000970' 2024-09-21T00:30:07.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27904 fp=#0x2000000108b' 2024-09-21T00:30:07.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27936 fp=#0x20000001086' 2024-09-21T00:30:07.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27968 fp=#0x100000049fd' 2024-09-21T00:30:07.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28000 fp=#0x10000004a26' 2024-09-21T00:30:07.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28032 fp=#0x10000004a00' 2024-09-21T00:30:07.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28064 fp=#0x20000001376' 2024-09-21T00:30:07.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28096 fp=#0x200000012a3' 2024-09-21T00:30:07.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28128 fp=#0x20000000b08' 2024-09-21T00:30:07.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28160 fp=#0x10000004be1' 2024-09-21T00:30:07.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28192 fp=#0x10000004b6c' 2024-09-21T00:30:07.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28224 fp=#0x10000004b9c' 2024-09-21T00:30:07.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28256 fp=#0x100000032ed' 2024-09-21T00:30:07.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28288 fp=#0x20000000edf' 2024-09-21T00:30:07.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28320 fp=#0x20000000e5b' 2024-09-21T00:30:07.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28352 fp=#0x1000000310a' 2024-09-21T00:30:07.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28384 fp=#0x1000000327a' 2024-09-21T00:30:07.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28416 fp=#0x10000003215' 2024-09-21T00:30:07.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28448 fp=#0x10000004c48' 2024-09-21T00:30:07.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28480 fp=#0x10000003281' 2024-09-21T00:30:07.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28512 fp=#0x2000000117a' 2024-09-21T00:30:07.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28544 fp=#0x200000011ac' 2024-09-21T00:30:07.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28608 fp=#0x200000010f9' 2024-09-21T00:30:07.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28640 fp=#0x2000000110a' 2024-09-21T00:30:07.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28672 fp=#0x20000000e2a' 2024-09-21T00:30:07.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28704 fp=#0x10000001629' 2024-09-21T00:30:07.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28736 fp=#0x100000015e4' 2024-09-21T00:30:07.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28768 fp=#0x10000001748' 2024-09-21T00:30:07.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28800 fp=#0x1000000167b' 2024-09-21T00:30:07.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28928 fp=#0x20000000316' 2024-09-21T00:30:07.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28960 fp=#0x100000016bc' 2024-09-21T00:30:07.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28992 fp=#0x200000006e3' 2024-09-21T00:30:07.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29024 fp=#0x10000001719' 2024-09-21T00:30:07.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29056 fp=#0x20000001424' 2024-09-21T00:30:07.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29088 fp=#0x10000004347' 2024-09-21T00:30:07.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29120 fp=#0x1000000433c' 2024-09-21T00:30:07.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29152 fp=#0x100000053b0' 2024-09-21T00:30:07.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29184 fp=#0x10000004333' 2024-09-21T00:30:07.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29216 fp=#0x1000000437f' 2024-09-21T00:30:07.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29280 fp=#0x10000004378' 2024-09-21T00:30:07.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29408 fp=#0x10000004c88' 2024-09-21T00:30:07.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29472 fp=#0x10000004c94' 2024-09-21T00:30:07.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29504 fp=#0x2000000101c' 2024-09-21T00:30:07.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29568 fp=#0x20000000ec6' 2024-09-21T00:30:07.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29632 fp=#0x100000040a9' 2024-09-21T00:30:07.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29664 fp=#0x100000029a7' 2024-09-21T00:30:07.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29696 fp=#0x200000010c5' 2024-09-21T00:30:07.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29728 fp=#0x100000040da' 2024-09-21T00:30:07.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29760 fp=#0x10000004582' 2024-09-21T00:30:07.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29792 fp=#0x10000004544' 2024-09-21T00:30:07.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29824 fp=#0x1000000457b' 2024-09-21T00:30:07.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29856 fp=#0x1000000455e' 2024-09-21T00:30:07.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29920 fp=#0x20000000e39' 2024-09-21T00:30:07.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29952 fp=#0x20000000cd1' 2024-09-21T00:30:07.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29984 fp=#0x20000000c9f' 2024-09-21T00:30:07.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30016 fp=#0x20000000c69' 2024-09-21T00:30:07.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30048 fp=#0x20000000c3a' 2024-09-21T00:30:07.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30080 fp=#0x20000000a4e' 2024-09-21T00:30:07.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30144 fp=#0x20000000c04' 2024-09-21T00:30:07.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30176 fp=#0x10000001c17' 2024-09-21T00:30:07.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30208 fp=#0x10000001bc4' 2024-09-21T00:30:07.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30240 fp=#0x10000001c2e' 2024-09-21T00:30:07.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30272 fp=#0x10000001b6e' 2024-09-21T00:30:07.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30304 fp=#0x20000000af8' 2024-09-21T00:30:07.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30336 fp=#0x20000000a22' 2024-09-21T00:30:07.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30368 fp=#0x1000000292d' 2024-09-21T00:30:07.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30400 fp=#0x100000028bd' 2024-09-21T00:30:07.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30432 fp=#0x20000000ad2' 2024-09-21T00:30:07.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30464 fp=#0x20000000a60' 2024-09-21T00:30:07.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22785 fp=#0x10000001e91' 2024-09-21T00:30:07.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:25217 fp=#0x2000000139f' 2024-09-21T00:30:07.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26625 fp=#0x20000000a19' 2024-09-21T00:30:07.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26689 fp=#0x2000000082c' 2024-09-21T00:30:07.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26785 fp=#0x1000000446b' 2024-09-21T00:30:07.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26817 fp=#0x100000043fb' 2024-09-21T00:30:07.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26849 fp=#0x10000004614' 2024-09-21T00:30:07.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26913 fp=#0x10000004465' 2024-09-21T00:30:07.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27009 fp=#0x20000000d9a' 2024-09-21T00:30:07.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27041 fp=#0x20000000d59' 2024-09-21T00:30:07.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27105 fp=#0x20000000eda' 2024-09-21T00:30:07.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27137 fp=#0x20000000f36' 2024-09-21T00:30:07.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27169 fp=#0x10000002297' 2024-09-21T00:30:07.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27201 fp=#0x100000023d4' 2024-09-21T00:30:07.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27233 fp=#0x20000000fa5' 2024-09-21T00:30:07.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27265 fp=#0x20000000f7f' 2024-09-21T00:30:07.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27297 fp=#0x10000002272' 2024-09-21T00:30:07.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27329 fp=#0x100000023c2' 2024-09-21T00:30:07.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27361 fp=#0x1000000233a' 2024-09-21T00:30:07.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27393 fp=#0x20000001112' 2024-09-21T00:30:07.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27425 fp=#0x10000003bde' 2024-09-21T00:30:07.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27489 fp=#0x10000003b73' 2024-09-21T00:30:07.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27521 fp=#0x10000003ce8' 2024-09-21T00:30:07.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27649 fp=#0x10000001eb0' 2024-09-21T00:30:07.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27713 fp=#0x1000000203c' 2024-09-21T00:30:07.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27745 fp=#0x10000001ee7' 2024-09-21T00:30:07.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27777 fp=#0x200000008d7' 2024-09-21T00:30:07.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27809 fp=#0x200000008bd' 2024-09-21T00:30:07.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27873 fp=#0x20000000959' 2024-09-21T00:30:07.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27905 fp=#0x200000010ff' 2024-09-21T00:30:07.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27937 fp=#0x20000001043' 2024-09-21T00:30:07.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27969 fp=#0x10000004a8a' 2024-09-21T00:30:07.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28001 fp=#0x100000049c2' 2024-09-21T00:30:07.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28033 fp=#0x10000004a2e' 2024-09-21T00:30:07.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28065 fp=#0x2000000137f' 2024-09-21T00:30:07.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28097 fp=#0x200000012fc' 2024-09-21T00:30:07.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28129 fp=#0x10000004aed' 2024-09-21T00:30:07.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28161 fp=#0x10000004b27' 2024-09-21T00:30:07.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28193 fp=#0x10000004b17' 2024-09-21T00:30:07.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28225 fp=#0x10000004b0d' 2024-09-21T00:30:07.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28257 fp=#0x10000003049' 2024-09-21T00:30:07.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28289 fp=#0x10000003170' 2024-09-21T00:30:07.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28321 fp=#0x100000032cc' 2024-09-21T00:30:07.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28353 fp=#0x1000000335e' 2024-09-21T00:30:07.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28385 fp=#0x10000004bce' 2024-09-21T00:30:07.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28417 fp=#0x100000030d3' 2024-09-21T00:30:07.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28449 fp=#0x20000000f64' 2024-09-21T00:30:07.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28513 fp=#0x2000000122d' 2024-09-21T00:30:07.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28545 fp=#0x2000000101a' 2024-09-21T00:30:07.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28577 fp=#0x20000001137' 2024-09-21T00:30:07.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28609 fp=#0x20000001130' 2024-09-21T00:30:07.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28673 fp=#0x20000000dc7' 2024-09-21T00:30:07.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28705 fp=#0x1000000170d' 2024-09-21T00:30:07.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28737 fp=#0x10000001617' 2024-09-21T00:30:07.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28769 fp=#0x1000000171c' 2024-09-21T00:30:07.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28801 fp=#0x100000015e8' 2024-09-21T00:30:07.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28833 fp=#0x200000009fd' 2024-09-21T00:30:07.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28929 fp=#0x2000000031f' 2024-09-21T00:30:07.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28961 fp=#0x10000001650' 2024-09-21T00:30:07.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28993 fp=#0x100000016b9' 2024-09-21T00:30:07.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29025 fp=#0x100000016b1' 2024-09-21T00:30:07.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29057 fp=#0x100000043c4' 2024-09-21T00:30:07.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29121 fp=#0x100000043cf' 2024-09-21T00:30:07.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29185 fp=#0x10000004339' 2024-09-21T00:30:07.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29217 fp=#0x100000043cb' 2024-09-21T00:30:07.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29281 fp=#0x1000000438b' 2024-09-21T00:30:07.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29409 fp=#0x10000004e3d' 2024-09-21T00:30:07.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29441 fp=#0x10000004dfc' 2024-09-21T00:30:07.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29505 fp=#0x10000004cc6' 2024-09-21T00:30:07.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29537 fp=#0x10000004c56' 2024-09-21T00:30:07.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29569 fp=#0x10000002a5a' 2024-09-21T00:30:07.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29601 fp=#0x10000002a60' 2024-09-21T00:30:07.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29665 fp=#0x100000040c9' 2024-09-21T00:30:07.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29729 fp=#0x10000002a63' 2024-09-21T00:30:07.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29761 fp=#0x1000000451f' 2024-09-21T00:30:07.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29793 fp=#0x1000000454e' 2024-09-21T00:30:07.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29825 fp=#0x10000004566' 2024-09-21T00:30:07.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29857 fp=#0x10000004530' 2024-09-21T00:30:07.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29921 fp=#0x20000000db1' 2024-09-21T00:30:07.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29953 fp=#0x20000000e75' 2024-09-21T00:30:07.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29985 fp=#0x20000000cb1' 2024-09-21T00:30:07.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30017 fp=#0x20000000e56' 2024-09-21T00:30:07.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30081 fp=#0x20000000a62' 2024-09-21T00:30:07.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30113 fp=#0x20000000bc1' 2024-09-21T00:30:07.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30177 fp=#0x1000000559d' 2024-09-21T00:30:07.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30209 fp=#0x10000001c0e' 2024-09-21T00:30:07.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30241 fp=#0x10000001af0' 2024-09-21T00:30:07.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30273 fp=#0x10000002993' 2024-09-21T00:30:07.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30305 fp=#0x20000000b31' 2024-09-21T00:30:07.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30337 fp=#0x20000000ae8' 2024-09-21T00:30:07.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30369 fp=#0x20000000aa8' 2024-09-21T00:30:07.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30401 fp=#0x10000002978' 2024-09-21T00:30:07.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30433 fp=#0x20000000a2a' 2024-09-21T00:30:07.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30465 fp=#0x100000028ed' 2024-09-21T00:30:07.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30497 fp=#0x10000002393' 2024-09-21T00:30:07.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30561 fp=#0x10000002376' 2024-09-21T00:30:07.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22786 fp=#0x200000008bb' 2024-09-21T00:30:07.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22818 fp=#0x10000003019' 2024-09-21T00:30:07.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26658 fp=#0x200000007d8' 2024-09-21T00:30:07.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26690 fp=#0x200000007ff' 2024-09-21T00:30:07.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26786 fp=#0x100000043ec' 2024-09-21T00:30:07.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26818 fp=#0x10000004460' 2024-09-21T00:30:07.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26850 fp=#0x100000043dd' 2024-09-21T00:30:07.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26882 fp=#0x100000043e1' 2024-09-21T00:30:07.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26914 fp=#0x100000043db' 2024-09-21T00:30:07.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26946 fp=#0x1000000460f' 2024-09-21T00:30:07.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26978 fp=#0x20000000f57' 2024-09-21T00:30:07.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27010 fp=#0x20000000dc4' 2024-09-21T00:30:07.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27074 fp=#0x20000000d66' 2024-09-21T00:30:07.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27106 fp=#0x20000000dcb' 2024-09-21T00:30:07.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27138 fp=#0x10000002505' 2024-09-21T00:30:07.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27170 fp=#0x1000000224a' 2024-09-21T00:30:07.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27202 fp=#0x10000002215' 2024-09-21T00:30:07.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27234 fp=#0x10000002386' 2024-09-21T00:30:07.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27266 fp=#0x20000000fa2' 2024-09-21T00:30:07.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27298 fp=#0x1000000250e' 2024-09-21T00:30:07.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27330 fp=#0x100000023a5' 2024-09-21T00:30:07.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27362 fp=#0x20000000f4c' 2024-09-21T00:30:07.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27394 fp=#0x10000003c35' 2024-09-21T00:30:07.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27426 fp=#0x2000000116f' 2024-09-21T00:30:07.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27458 fp=#0x10000003bf6' 2024-09-21T00:30:07.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27490 fp=#0x10000003bf9' 2024-09-21T00:30:07.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27650 fp=#0x10000002069' 2024-09-21T00:30:07.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27682 fp=#0x10000001e9c' 2024-09-21T00:30:07.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27714 fp=#0x10000001e99' 2024-09-21T00:30:07.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27746 fp=#0x200000008dd' 2024-09-21T00:30:07.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27778 fp=#0x200000008da' 2024-09-21T00:30:07.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27842 fp=#0x20000000952' 2024-09-21T00:30:07.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27906 fp=#0x20000001058' 2024-09-21T00:30:07.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27938 fp=#0x2000000109f' 2024-09-21T00:30:07.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27970 fp=#0x10000004a9d' 2024-09-21T00:30:07.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28002 fp=#0x10000004a3d' 2024-09-21T00:30:07.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28034 fp=#0x10000004a4d' 2024-09-21T00:30:07.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28066 fp=#0x20000001282' 2024-09-21T00:30:07.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28098 fp=#0x200000012b1' 2024-09-21T00:30:07.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28130 fp=#0x10000004b95' 2024-09-21T00:30:07.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28162 fp=#0x10000004b48' 2024-09-21T00:30:07.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28226 fp=#0x10000004beb' 2024-09-21T00:30:07.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28258 fp=#0x1000000314c' 2024-09-21T00:30:07.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28290 fp=#0x100000030dd' 2024-09-21T00:30:07.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28322 fp=#0x10000003077' 2024-09-21T00:30:07.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28354 fp=#0x20000000e92' 2024-09-21T00:30:07.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28386 fp=#0x10000003348' 2024-09-21T00:30:07.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28418 fp=#0x10000003056' 2024-09-21T00:30:07.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28450 fp=#0x100000030ec' 2024-09-21T00:30:07.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28482 fp=#0x20000000e8d' 2024-09-21T00:30:07.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28546 fp=#0x2000000118c' 2024-09-21T00:30:07.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28578 fp=#0x20000001228' 2024-09-21T00:30:07.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28610 fp=#0x20000001012' 2024-09-21T00:30:07.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28642 fp=#0x200000011e0' 2024-09-21T00:30:07.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28674 fp=#0x20000000d49' 2024-09-21T00:30:07.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28706 fp=#0x20000000730' 2024-09-21T00:30:07.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28738 fp=#0x100000016eb' 2024-09-21T00:30:07.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28770 fp=#0x100000016ca' 2024-09-21T00:30:07.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28802 fp=#0x1000000166c' 2024-09-21T00:30:07.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28834 fp=#0x200000009c0' 2024-09-21T00:30:07.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28866 fp=#0x200000009f9' 2024-09-21T00:30:07.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28898 fp=#0x200000009c8' 2024-09-21T00:30:07.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28930 fp=#0x100000016c5' 2024-09-21T00:30:07.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28962 fp=#0x100000016cd' 2024-09-21T00:30:07.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28994 fp=#0x1000000170a' 2024-09-21T00:30:07.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29026 fp=#0x2000000030c' 2024-09-21T00:30:07.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29058 fp=#0x10000004355' 2024-09-21T00:30:07.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29090 fp=#0x1000000434a' 2024-09-21T00:30:07.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29122 fp=#0x100000043bd' 2024-09-21T00:30:07.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29186 fp=#0x10000004390' 2024-09-21T00:30:07.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29218 fp=#0x100000043ab' 2024-09-21T00:30:07.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29250 fp=#0x1000000435e' 2024-09-21T00:30:07.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29282 fp=#0x100000043b7' 2024-09-21T00:30:07.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29506 fp=#0x10000004c99' 2024-09-21T00:30:07.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29538 fp=#0x10000004de6' 2024-09-21T00:30:07.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29570 fp=#0x100000040cf' 2024-09-21T00:30:07.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29602 fp=#0x100000029ac' 2024-09-21T00:30:07.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29666 fp=#0x100000040d0' 2024-09-21T00:30:07.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29698 fp=#0x100000040cd' 2024-09-21T00:30:07.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29730 fp=#0x20000000eb6' 2024-09-21T00:30:07.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29762 fp=#0x10000004581' 2024-09-21T00:30:07.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29794 fp=#0x10000004586' 2024-09-21T00:30:07.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29826 fp=#0x1000000457f' 2024-09-21T00:30:07.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29922 fp=#0x20000000de0' 2024-09-21T00:30:07.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29954 fp=#0x20000000ce6' 2024-09-21T00:30:07.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29986 fp=#0x20000000daf' 2024-09-21T00:30:07.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30018 fp=#0x20000000e34' 2024-09-21T00:30:07.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30050 fp=#0x20000000c5c' 2024-09-21T00:30:07.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30082 fp=#0x20000000a6b' 2024-09-21T00:30:07.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30178 fp=#0x100000034cb' 2024-09-21T00:30:07.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30210 fp=#0x10000001c96' 2024-09-21T00:30:07.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30242 fp=#0x10000001c65' 2024-09-21T00:30:07.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30274 fp=#0x20000000a7a' 2024-09-21T00:30:07.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30306 fp=#0x10000002965' 2024-09-21T00:30:07.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30338 fp=#0x10000002905' 2024-09-21T00:30:07.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30370 fp=#0x20000000a2e' 2024-09-21T00:30:07.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30402 fp=#0x1000000290a' 2024-09-21T00:30:07.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30434 fp=#0x10000002955' 2024-09-21T00:30:07.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30466 fp=#0x100000028fc' 2024-09-21T00:30:07.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30498 fp=#0x100000023c7' 2024-09-21T00:30:07.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_path:mds.1:22723 fp=#0x1' 2024-09-21T00:30:07.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22787 fp=#0x20000001042' 2024-09-21T00:30:07.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22819 fp=#0x20000000f37' 2024-09-21T00:30:07.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:25315 fp=#0x200000013a9' 2024-09-21T00:30:07.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26627 fp=#0x20000000793' 2024-09-21T00:30:07.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26691 fp=#0x200000007f7' 2024-09-21T00:30:07.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26787 fp=#0x1000000463b' 2024-09-21T00:30:07.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26819 fp=#0x1000000444d' 2024-09-21T00:30:07.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26851 fp=#0x10000004486' 2024-09-21T00:30:07.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26947 fp=#0x10000004616' 2024-09-21T00:30:07.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26979 fp=#0x20000000f03' 2024-09-21T00:30:07.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27011 fp=#0x20000000da1' 2024-09-21T00:30:07.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27075 fp=#0x20000000ef1' 2024-09-21T00:30:07.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27107 fp=#0x20000000ef4' 2024-09-21T00:30:07.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27139 fp=#0x10000002203' 2024-09-21T00:30:07.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27203 fp=#0x20000000f18' 2024-09-21T00:30:07.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27235 fp=#0x20000000d57' 2024-09-21T00:30:07.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27267 fp=#0x20000000f34' 2024-09-21T00:30:07.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27299 fp=#0x10000002374' 2024-09-21T00:30:07.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27331 fp=#0x10000002280' 2024-09-21T00:30:07.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27363 fp=#0x20000000f68' 2024-09-21T00:30:07.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27395 fp=#0x10000003cff' 2024-09-21T00:30:07.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27427 fp=#0x10000003b6f' 2024-09-21T00:30:07.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27459 fp=#0x10000003b9f' 2024-09-21T00:30:07.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27491 fp=#0x10000003bba' 2024-09-21T00:30:07.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27523 fp=#0x10000003bc0' 2024-09-21T00:30:07.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27683 fp=#0x200000008af' 2024-09-21T00:30:07.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27715 fp=#0x10000001fd1' 2024-09-21T00:30:07.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27811 fp=#0x20000000a3a' 2024-09-21T00:30:07.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27843 fp=#0x20000000962' 2024-09-21T00:30:07.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27907 fp=#0x2000000104e' 2024-09-21T00:30:07.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27939 fp=#0x2000000104c' 2024-09-21T00:30:07.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27971 fp=#0x10000004989' 2024-09-21T00:30:07.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28003 fp=#0x10000004a45' 2024-09-21T00:30:07.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28035 fp=#0x100000049c8' 2024-09-21T00:30:07.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28067 fp=#0x20000001347' 2024-09-21T00:30:07.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28099 fp=#0x20000001298' 2024-09-21T00:30:07.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28131 fp=#0x10000004c0b' 2024-09-21T00:30:07.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28163 fp=#0x10000004b2b' 2024-09-21T00:30:07.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28195 fp=#0x10000004b30' 2024-09-21T00:30:07.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28227 fp=#0x10000004b13' 2024-09-21T00:30:07.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28259 fp=#0x20000000f30' 2024-09-21T00:30:07.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28291 fp=#0x1000000335a' 2024-09-21T00:30:07.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28323 fp=#0x10000004c50' 2024-09-21T00:30:07.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28355 fp=#0x1000000328b' 2024-09-21T00:30:07.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28387 fp=#0x100000031dd' 2024-09-21T00:30:07.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28419 fp=#0x100000032e5' 2024-09-21T00:30:07.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28451 fp=#0x100000032ce' 2024-09-21T00:30:07.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28483 fp=#0x100000032c5' 2024-09-21T00:30:07.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28515 fp=#0x20000000fe7' 2024-09-21T00:30:07.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28547 fp=#0x20000001126' 2024-09-21T00:30:07.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28579 fp=#0x200000010fc' 2024-09-21T00:30:07.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28611 fp=#0x200000011f6' 2024-09-21T00:30:07.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28675 fp=#0x20000000d77' 2024-09-21T00:30:07.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28707 fp=#0x100000016e4' 2024-09-21T00:30:07.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28739 fp=#0x100000015d4' 2024-09-21T00:30:07.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28771 fp=#0x1000000167e' 2024-09-21T00:30:07.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28803 fp=#0x100000015ee' 2024-09-21T00:30:07.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28835 fp=#0x200000009f0' 2024-09-21T00:30:07.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28899 fp=#0x200000009d3' 2024-09-21T00:30:07.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28931 fp=#0x10000001669' 2024-09-21T00:30:07.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28963 fp=#0x100000016d7' 2024-09-21T00:30:07.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28995 fp=#0x1000000172b' 2024-09-21T00:30:07.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29027 fp=#0x10000001671' 2024-09-21T00:30:07.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29059 fp=#0x200000013de' 2024-09-21T00:30:07.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29123 fp=#0x10000004369' 2024-09-21T00:30:07.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29155 fp=#0x10000004396' 2024-09-21T00:30:07.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29187 fp=#0x1000000436c' 2024-09-21T00:30:07.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29251 fp=#0x100000043b1' 2024-09-21T00:30:07.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29283 fp=#0x10000004349' 2024-09-21T00:30:07.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29443 fp=#0x10000004e41' 2024-09-21T00:30:07.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29507 fp=#0x10000004e2f' 2024-09-21T00:30:07.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29635 fp=#0x100000040d9' 2024-09-21T00:30:07.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29667 fp=#0x100000040b3' 2024-09-21T00:30:07.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29699 fp=#0x10000002a54' 2024-09-21T00:30:07.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29731 fp=#0x100000040bb' 2024-09-21T00:30:07.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29795 fp=#0x10000004536' 2024-09-21T00:30:07.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29827 fp=#0x200000012cb' 2024-09-21T00:30:07.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29923 fp=#0x20000000e2c' 2024-09-21T00:30:07.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29955 fp=#0x20000000cf2' 2024-09-21T00:30:07.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29987 fp=#0x20000000e67' 2024-09-21T00:30:07.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30019 fp=#0x20000000e0d' 2024-09-21T00:30:07.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30147 fp=#0x20000000a53' 2024-09-21T00:30:07.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30179 fp=#0x10000001ca3' 2024-09-21T00:30:07.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30211 fp=#0x10000001b88' 2024-09-21T00:30:07.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30243 fp=#0x10000001baf' 2024-09-21T00:30:07.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30275 fp=#0x10000002984' 2024-09-21T00:30:07.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30307 fp=#0x20000000ac1' 2024-09-21T00:30:07.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30339 fp=#0x20000000a63' 2024-09-21T00:30:07.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30371 fp=#0x1000000293a' 2024-09-21T00:30:07.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30403 fp=#0x20000000a6e' 2024-09-21T00:30:07.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30435 fp=#0x10000002943' 2024-09-21T00:30:07.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30467 fp=#0x1000000295d' 2024-09-21T00:30:07.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30531 fp=#0x200000006f0' 2024-09-21T00:30:07.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22724 fp=#0x1' 2024-09-21T00:30:07.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22788 fp=#0x1000000495f' 2024-09-21T00:30:07.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22820 fp=#0x20000000d3d' 2024-09-21T00:30:07.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26692 fp=#0x20000000787' 2024-09-21T00:30:07.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26724 fp=#0x20000000804' 2024-09-21T00:30:07.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26788 fp=#0x10000004469' 2024-09-21T00:30:07.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26852 fp=#0x10000004625' 2024-09-21T00:30:07.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26916 fp=#0x1000000463f' 2024-09-21T00:30:07.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26948 fp=#0x100000043e7' 2024-09-21T00:30:07.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26980 fp=#0x20000000f11' 2024-09-21T00:30:07.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27012 fp=#0x20000000f5b' 2024-09-21T00:30:07.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27044 fp=#0x20000000ed2' 2024-09-21T00:30:07.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27076 fp=#0x20000000de8' 2024-09-21T00:30:07.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27108 fp=#0x20000000f6a' 2024-09-21T00:30:07.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27140 fp=#0x10000002282' 2024-09-21T00:30:07.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27172 fp=#0x1000000221c' 2024-09-21T00:30:07.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27204 fp=#0x1000000238c' 2024-09-21T00:30:07.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27236 fp=#0x1000000236a' 2024-09-21T00:30:07.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27268 fp=#0x100000021f7' 2024-09-21T00:30:07.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27300 fp=#0x10000002398' 2024-09-21T00:30:07.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27332 fp=#0x100000024fe' 2024-09-21T00:30:07.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27364 fp=#0x20000000f21' 2024-09-21T00:30:07.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27396 fp=#0x10000003d26' 2024-09-21T00:30:07.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27428 fp=#0x10000003bf3' 2024-09-21T00:30:07.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27524 fp=#0x10000003bd2' 2024-09-21T00:30:07.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27652 fp=#0x10000001fd6' 2024-09-21T00:30:07.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27684 fp=#0x10000001f33' 2024-09-21T00:30:07.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27716 fp=#0x1000000205b' 2024-09-21T00:30:07.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27812 fp=#0x20000001032' 2024-09-21T00:30:07.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27844 fp=#0x200000008c4' 2024-09-21T00:30:07.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27876 fp=#0x200000008be' 2024-09-21T00:30:07.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27908 fp=#0x20000001141' 2024-09-21T00:30:07.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27940 fp=#0x20000001049' 2024-09-21T00:30:07.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27972 fp=#0x10000004a12' 2024-09-21T00:30:07.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28004 fp=#0x100000049b1' 2024-09-21T00:30:07.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28036 fp=#0x10000004a41' 2024-09-21T00:30:07.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28068 fp=#0x20000001369' 2024-09-21T00:30:07.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28100 fp=#0x20000001254' 2024-09-21T00:30:07.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28132 fp=#0x10000004b29' 2024-09-21T00:30:07.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28164 fp=#0x10000004b59' 2024-09-21T00:30:07.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28196 fp=#0x10000004ab3' 2024-09-21T00:30:07.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28260 fp=#0x100000032ec' 2024-09-21T00:30:07.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28292 fp=#0x10000003082' 2024-09-21T00:30:07.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28324 fp=#0x10000003291' 2024-09-21T00:30:07.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28356 fp=#0x10000004c4a' 2024-09-21T00:30:07.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28388 fp=#0x1000000317f' 2024-09-21T00:30:07.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28420 fp=#0x10000003167' 2024-09-21T00:30:07.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28452 fp=#0x100000036cf' 2024-09-21T00:30:07.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28484 fp=#0x1000000327e' 2024-09-21T00:30:07.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28516 fp=#0x200000011ed' 2024-09-21T00:30:07.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28548 fp=#0x2000000120c' 2024-09-21T00:30:07.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28580 fp=#0x200000011fb' 2024-09-21T00:30:07.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28612 fp=#0x20000001142' 2024-09-21T00:30:07.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28644 fp=#0x20000000fdb' 2024-09-21T00:30:07.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28676 fp=#0x20000000da0' 2024-09-21T00:30:07.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28708 fp=#0x10000001608' 2024-09-21T00:30:07.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28740 fp=#0x100000016ce' 2024-09-21T00:30:07.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28772 fp=#0x20000000383' 2024-09-21T00:30:07.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28804 fp=#0x100000016b0' 2024-09-21T00:30:07.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28836 fp=#0x200000009f7' 2024-09-21T00:30:07.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28868 fp=#0x200000009fc' 2024-09-21T00:30:07.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28932 fp=#0x2000000031a' 2024-09-21T00:30:07.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28964 fp=#0x100000016dd' 2024-09-21T00:30:07.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28996 fp=#0x10000001600' 2024-09-21T00:30:07.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29028 fp=#0x1000000173a' 2024-09-21T00:30:07.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29060 fp=#0x1000000436e' 2024-09-21T00:30:07.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29092 fp=#0x10000004371' 2024-09-21T00:30:07.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29156 fp=#0x20000001420' 2024-09-21T00:30:07.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29188 fp=#0x1000000434c' 2024-09-21T00:30:07.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29220 fp=#0x10000004358' 2024-09-21T00:30:07.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29252 fp=#0x10000004348' 2024-09-21T00:30:07.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29284 fp=#0x100000043a7' 2024-09-21T00:30:07.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29444 fp=#0x10000004e09' 2024-09-21T00:30:07.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29572 fp=#0x10000002b16' 2024-09-21T00:30:07.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29636 fp=#0x100000040d6' 2024-09-21T00:30:07.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29700 fp=#0x100000040a3' 2024-09-21T00:30:07.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29732 fp=#0x100000040c2' 2024-09-21T00:30:07.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29764 fp=#0x10000004567' 2024-09-21T00:30:07.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29796 fp=#0x10000004520' 2024-09-21T00:30:07.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29828 fp=#0x1000000453f' 2024-09-21T00:30:07.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29892 fp=#0x10000004ccb' 2024-09-21T00:30:07.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29924 fp=#0x20000000e4c' 2024-09-21T00:30:07.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29956 fp=#0x20000000e53' 2024-09-21T00:30:07.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29988 fp=#0x20000000d8c' 2024-09-21T00:30:07.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30020 fp=#0x20000000c8c' 2024-09-21T00:30:07.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30052 fp=#0x20000000c52' 2024-09-21T00:30:07.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30084 fp=#0x20000000bfa' 2024-09-21T00:30:07.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30116 fp=#0x20000000c1d' 2024-09-21T00:30:07.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30180 fp=#0x10000001c6b' 2024-09-21T00:30:07.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30212 fp=#0x100000055b3' 2024-09-21T00:30:07.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30244 fp=#0x10000001c84' 2024-09-21T00:30:07.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30276 fp=#0x100000028f9' 2024-09-21T00:30:07.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30308 fp=#0x20000000a48' 2024-09-21T00:30:07.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30340 fp=#0x20000000a4d' 2024-09-21T00:30:07.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30372 fp=#0x20000000a38' 2024-09-21T00:30:07.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30404 fp=#0x100000028fe' 2024-09-21T00:30:07.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30436 fp=#0x100000028f7' 2024-09-21T00:30:07.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30468 fp=#0x10000002935' 2024-09-21T00:30:07.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30532 fp=#0x1000000237f' 2024-09-21T00:30:07.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22725 fp=#0x10000000000' 2024-09-21T00:30:07.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22789 fp=#0x20000001237' 2024-09-21T00:30:07.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:22821 fp=#0x100000015d1' 2024-09-21T00:30:07.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26629 fp=#0x2000000082d' 2024-09-21T00:30:07.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26661 fp=#0x200000007f8' 2024-09-21T00:30:07.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26693 fp=#0x200000007f0' 2024-09-21T00:30:07.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26789 fp=#0x10000004623' 2024-09-21T00:30:07.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26821 fp=#0x100000043f7' 2024-09-21T00:30:07.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26885 fp=#0x100000043e8' 2024-09-21T00:30:07.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26917 fp=#0x10000004454' 2024-09-21T00:30:07.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26949 fp=#0x100000043ed' 2024-09-21T00:30:07.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:26981 fp=#0x20000000fbf' 2024-09-21T00:30:07.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27013 fp=#0x20000000de1' 2024-09-21T00:30:07.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27077 fp=#0x200000012bc' 2024-09-21T00:30:07.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27109 fp=#0x20000000d76' 2024-09-21T00:30:07.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27141 fp=#0x20000001022' 2024-09-21T00:30:07.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27173 fp=#0x20000000f33' 2024-09-21T00:30:07.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27205 fp=#0x100000021fc' 2024-09-21T00:30:07.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27237 fp=#0x1000000222e' 2024-09-21T00:30:07.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27269 fp=#0x10000002345' 2024-09-21T00:30:07.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27301 fp=#0x10000002294' 2024-09-21T00:30:07.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27333 fp=#0x20000000f8f' 2024-09-21T00:30:07.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27365 fp=#0x10000002277' 2024-09-21T00:30:07.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27397 fp=#0x10000003bea' 2024-09-21T00:30:07.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27429 fp=#0x10000003bff' 2024-09-21T00:30:07.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27461 fp=#0x10000003bae' 2024-09-21T00:30:07.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27493 fp=#0x10000003c44' 2024-09-21T00:30:07.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27653 fp=#0x10000001f2f' 2024-09-21T00:30:07.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27685 fp=#0x10000001f62' 2024-09-21T00:30:07.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27717 fp=#0x10000001f1f' 2024-09-21T00:30:07.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27749 fp=#0x200000008ce' 2024-09-21T00:30:07.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27781 fp=#0x200000008c3' 2024-09-21T00:30:07.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27813 fp=#0x200000008ef' 2024-09-21T00:30:07.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27845 fp=#0x200000008ec' 2024-09-21T00:30:07.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27877 fp=#0x200000008f0' 2024-09-21T00:30:07.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27909 fp=#0x2000000106f' 2024-09-21T00:30:07.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27941 fp=#0x200000010b1' 2024-09-21T00:30:07.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:27973 fp=#0x100000049a3' 2024-09-21T00:30:07.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28037 fp=#0x10000004a04' 2024-09-21T00:30:07.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28069 fp=#0x2000000135e' 2024-09-21T00:30:07.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28101 fp=#0x20000001342' 2024-09-21T00:30:07.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28133 fp=#0x10000004c07' 2024-09-21T00:30:07.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28165 fp=#0x10000004ad3' 2024-09-21T00:30:07.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28229 fp=#0x10000004b03' 2024-09-21T00:30:07.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28261 fp=#0x100000030bf' 2024-09-21T00:30:07.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28293 fp=#0x20000000e28' 2024-09-21T00:30:07.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28325 fp=#0x10000003040' 2024-09-21T00:30:07.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28357 fp=#0x1000000326f' 2024-09-21T00:30:07.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28389 fp=#0x10000003105' 2024-09-21T00:30:07.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28421 fp=#0x20000000e24' 2024-09-21T00:30:07.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28453 fp=#0x10000003157' 2024-09-21T00:30:07.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28485 fp=#0x10000003203' 2024-09-21T00:30:07.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28517 fp=#0x20000000f80' 2024-09-21T00:30:07.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28549 fp=#0x200000011a6' 2024-09-21T00:30:07.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28613 fp=#0x20000000f5c' 2024-09-21T00:30:07.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28645 fp=#0x20000000ff2' 2024-09-21T00:30:07.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28677 fp=#0x20000000dae' 2024-09-21T00:30:07.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28709 fp=#0x100000016a7' 2024-09-21T00:30:07.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28741 fp=#0x2000000072b' 2024-09-21T00:30:07.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28773 fp=#0x100000015f7' 2024-09-21T00:30:07.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28805 fp=#0x100000016a5' 2024-09-21T00:30:07.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28869 fp=#0x200000009f3' 2024-09-21T00:30:07.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28901 fp=#0x200000009ee' 2024-09-21T00:30:07.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28933 fp=#0x1000000162b' 2024-09-21T00:30:07.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28965 fp=#0x20000000303' 2024-09-21T00:30:07.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:28997 fp=#0x10000001660' 2024-09-21T00:30:07.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29029 fp=#0x1000000162d' 2024-09-21T00:30:07.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29061 fp=#0x20000000f2c' 2024-09-21T00:30:07.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29093 fp=#0x200000013d1' 2024-09-21T00:30:07.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29125 fp=#0x10000004374' 2024-09-21T00:30:07.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29157 fp=#0x10000004337' 2024-09-21T00:30:07.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29189 fp=#0x10000004398' 2024-09-21T00:30:07.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29253 fp=#0x20000001413' 2024-09-21T00:30:07.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29445 fp=#0x10000004c85' 2024-09-21T00:30:07.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29637 fp=#0x20000000eca' 2024-09-21T00:30:07.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29701 fp=#0x100000040b0' 2024-09-21T00:30:07.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29733 fp=#0x100000040ab' 2024-09-21T00:30:07.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29765 fp=#0x10000004543' 2024-09-21T00:30:07.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29797 fp=#0x10000004535' 2024-09-21T00:30:07.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29829 fp=#0x1000000456c' 2024-09-21T00:30:07.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29893 fp=#0x10000004d52' 2024-09-21T00:30:07.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29925 fp=#0x20000000cdd' 2024-09-21T00:30:07.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29957 fp=#0x20000000d44' 2024-09-21T00:30:07.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:29989 fp=#0x20000000ccf' 2024-09-21T00:30:07.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30021 fp=#0x20000000d50' 2024-09-21T00:30:07.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30117 fp=#0x20000000a56' 2024-09-21T00:30:07.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30149 fp=#0x20000000a6d' 2024-09-21T00:30:07.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30181 fp=#0x10000001bc1' 2024-09-21T00:30:07.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30213 fp=#0x10000001b2a' 2024-09-21T00:30:07.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30245 fp=#0x10000002bb0' 2024-09-21T00:30:07.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30277 fp=#0x20000000bae' 2024-09-21T00:30:07.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30309 fp=#0x10000002953' 2024-09-21T00:30:07.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30341 fp=#0x20000000bb4' 2024-09-21T00:30:07.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30373 fp=#0x20000000b8c' 2024-09-21T00:30:07.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30405 fp=#0x20000000a44' 2024-09-21T00:30:07.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30437 fp=#0x1000000297e' 2024-09-21T00:30:07.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (a) for 47099ce0-1: 'internal op quiesce_inode:mds.1:30469 fp=#0x20000000a89' 2024-09-21T00:30:07.673 INFO:tasks.quiescer.fs.[cephfs]:Pulled 7841 ops from rank 1 (a) into home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310/ops-47099ce0-1-mds.a.json 2024-09-21T00:30:07.673 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -- /var/run/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ops-47099ce0-1-mds.a.json 2024-09-21T00:30:07.674 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:05 smithi098 ceph-mon[34725]: pgmap v645: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 KiB/s wr, 8 op/s 2024-09-21T00:30:07.674 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:05 smithi089 ceph-mon[34481]: pgmap v645: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 KiB/s wr, 8 op/s 2024-09-21T00:30:07.674 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:05 smithi027 ceph-mon[30745]: pgmap v645: 129 pgs: 129 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 KiB/s wr, 8 op/s 2024-09-21T00:30:07.738 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2024-09-21T00:30:07.757 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.XTxO3INE0P 2024-09-21T00:30:07.758 DEBUG:teuthology.orchestra.run.smithi089:> sudo cp /var/run/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ops-47099ce0-1-mds.b.json /tmp/tmp.XTxO3INE0P 2024-09-21T00:30:07.978 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.XTxO3INE0P 2024-09-21T00:30:08.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:07 smithi089 ceph-mon[34481]: pgmap v646: 129 pgs: 129 active+clean; 981 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 89 KiB/s wr, 28 op/s 2024-09-21T00:30:08.047 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:07 smithi098 ceph-mon[34725]: pgmap v646: 129 pgs: 129 active+clean; 981 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 89 KiB/s wr, 28 op/s 2024-09-21T00:30:08.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:07 smithi027 ceph-mon[30745]: pgmap v646: 129 pgs: 129 active+clean; 981 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 89 KiB/s wr, 28 op/s 2024-09-21T00:30:08.084 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.XTxO3INE0P is 27MB 2024-09-21T00:30:08.654 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.XTxO3INE0P 2024-09-21T00:30:08.719 DEBUG:tasks.quiescer.fs.[cephfs]:read 28105396B of ops from '/var/run/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ops-47099ce0-1-mds.b.json' on mds.0 (b) 2024-09-21T00:30:09.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30789 fp=#0x10000001b34' 2024-09-21T00:30:09.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30821 fp=#0x10000002b3a' 2024-09-21T00:30:09.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30885 fp=#0x10000003031' 2024-09-21T00:30:09.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32709 fp=#0x10000005787' 2024-09-21T00:30:09.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32741 fp=#0x1000000573b' 2024-09-21T00:30:09.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32773 fp=#0x10000005df7' 2024-09-21T00:30:09.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32901 fp=#0x1000000504f' 2024-09-21T00:30:09.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32933 fp=#0x1000000507e' 2024-09-21T00:30:09.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32965 fp=#0x10000005064' 2024-09-21T00:30:09.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32997 fp=#0x10000005c67' 2024-09-21T00:30:09.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33029 fp=#0x10000005c38' 2024-09-21T00:30:09.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33061 fp=#0x10000005cb4' 2024-09-21T00:30:09.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33093 fp=#0x10000005d03' 2024-09-21T00:30:09.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33125 fp=#0x10000005cb6' 2024-09-21T00:30:09.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33157 fp=#0x10000005ddb' 2024-09-21T00:30:09.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33189 fp=#0x10000006040' 2024-09-21T00:30:09.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33317 fp=#0x1000000682d' 2024-09-21T00:30:09.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33349 fp=#0x10000006823' 2024-09-21T00:30:09.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33381 fp=#0x100000067cd' 2024-09-21T00:30:09.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33413 fp=#0x10000003ed9' 2024-09-21T00:30:09.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33445 fp=#0x10000003dc7' 2024-09-21T00:30:09.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33477 fp=#0x10000003e07' 2024-09-21T00:30:09.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33509 fp=#0x10000003e8e' 2024-09-21T00:30:09.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33541 fp=#0x10000005676' 2024-09-21T00:30:09.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33637 fp=#0x10000005a16' 2024-09-21T00:30:09.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33669 fp=#0x100000055bd' 2024-09-21T00:30:09.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33733 fp=#0x100000067c2' 2024-09-21T00:30:09.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33765 fp=#0x100000067b8' 2024-09-21T00:30:09.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33797 fp=#0x1000000676d' 2024-09-21T00:30:09.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33925 fp=#0x1000000643f' 2024-09-21T00:30:09.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33957 fp=#0x10000004731' 2024-09-21T00:30:09.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34021 fp=#0x10000004773' 2024-09-21T00:30:09.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34053 fp=#0x10000004721' 2024-09-21T00:30:09.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34085 fp=#0x10000003202' 2024-09-21T00:30:09.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34117 fp=#0x10000003238' 2024-09-21T00:30:09.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34149 fp=#0x100000061f8' 2024-09-21T00:30:09.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34181 fp=#0x10000006475' 2024-09-21T00:30:09.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34213 fp=#0x10000006464' 2024-09-21T00:30:09.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34245 fp=#0x10000006232' 2024-09-21T00:30:09.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34277 fp=#0x10000006294' 2024-09-21T00:30:09.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34309 fp=#0x1000000251e' 2024-09-21T00:30:09.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34373 fp=#0x100000061b4' 2024-09-21T00:30:09.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34405 fp=#0x10000002503' 2024-09-21T00:30:09.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34437 fp=#0x1000000619a' 2024-09-21T00:30:09.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34469 fp=#0x10000006629' 2024-09-21T00:30:09.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34501 fp=#0x10000003066' 2024-09-21T00:30:09.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34533 fp=#0x10000002573' 2024-09-21T00:30:09.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34565 fp=#0x100000063b1' 2024-09-21T00:30:09.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34597 fp=#0x10000006335' 2024-09-21T00:30:09.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34629 fp=#0x1000000634e' 2024-09-21T00:30:09.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34661 fp=#0x10000006387' 2024-09-21T00:30:09.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34757 fp=#0x100000057da' 2024-09-21T00:30:09.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34789 fp=#0x100000057d9' 2024-09-21T00:30:09.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34821 fp=#0x100000064fb' 2024-09-21T00:30:09.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34853 fp=#0x100000057e4' 2024-09-21T00:30:09.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34885 fp=#0x100000057ed' 2024-09-21T00:30:09.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34917 fp=#0x10000006962' 2024-09-21T00:30:09.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34949 fp=#0x1000000696d' 2024-09-21T00:30:09.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34981 fp=#0x10000005311' 2024-09-21T00:30:09.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35013 fp=#0x10000006970' 2024-09-21T00:30:09.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35077 fp=#0x100000060db' 2024-09-21T00:30:09.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35109 fp=#0x100000060ab' 2024-09-21T00:30:09.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35205 fp=#0x100000060cd' 2024-09-21T00:30:09.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35269 fp=#0x10000003f45' 2024-09-21T00:30:09.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35301 fp=#0x10000003fc0' 2024-09-21T00:30:09.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35365 fp=#0x10000005394' 2024-09-21T00:30:09.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35461 fp=#0x10000005ff7' 2024-09-21T00:30:09.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35557 fp=#0x10000005f93' 2024-09-21T00:30:09.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35621 fp=#0x10000005b72' 2024-09-21T00:30:09.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35653 fp=#0x10000005b74' 2024-09-21T00:30:09.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35685 fp=#0x10000005b7a' 2024-09-21T00:30:09.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35717 fp=#0x10000004920' 2024-09-21T00:30:09.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35749 fp=#0x10000005025' 2024-09-21T00:30:09.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35781 fp=#0x10000004f75' 2024-09-21T00:30:09.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35813 fp=#0x10000004f50' 2024-09-21T00:30:09.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35845 fp=#0x10000004fee' 2024-09-21T00:30:09.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35877 fp=#0x1000000670f' 2024-09-21T00:30:09.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35909 fp=#0x10000006704' 2024-09-21T00:30:09.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35941 fp=#0x1000000674e' 2024-09-21T00:30:09.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35973 fp=#0x10000006841' 2024-09-21T00:30:09.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36005 fp=#0x10000006868' 2024-09-21T00:30:09.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36037 fp=#0x10000005d6d' 2024-09-21T00:30:09.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36101 fp=#0x10000005f3c' 2024-09-21T00:30:09.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36165 fp=#0x10000005f49' 2024-09-21T00:30:09.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36197 fp=#0x100000068e2' 2024-09-21T00:30:09.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36229 fp=#0x100000068c6' 2024-09-21T00:30:09.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36261 fp=#0x1000000660d' 2024-09-21T00:30:09.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36293 fp=#0x100000065bc' 2024-09-21T00:30:09.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36357 fp=#0x10000006609' 2024-09-21T00:30:09.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36389 fp=#0x1000000659b' 2024-09-21T00:30:09.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36421 fp=#0x10000006134' 2024-09-21T00:30:09.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36453 fp=#0x1000000615f' 2024-09-21T00:30:09.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36485 fp=#0x1000000604c' 2024-09-21T00:30:09.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36517 fp=#0x10000006001' 2024-09-21T00:30:09.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36581 fp=#0x10000005a8f' 2024-09-21T00:30:09.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36613 fp=#0x10000005a98' 2024-09-21T00:30:09.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36677 fp=#0x10000004933' 2024-09-21T00:30:09.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36709 fp=#0x100000047b2' 2024-09-21T00:30:09.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36741 fp=#0x100000047bb' 2024-09-21T00:30:09.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36773 fp=#0x10000004864' 2024-09-21T00:30:09.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36805 fp=#0x10000005973' 2024-09-21T00:30:09.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36837 fp=#0x10000005af9' 2024-09-21T00:30:09.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36869 fp=#0x10000005b14' 2024-09-21T00:30:09.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36901 fp=#0x1000000591a' 2024-09-21T00:30:09.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36933 fp=#0x100000058e2' 2024-09-21T00:30:09.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36997 fp=#0x1000000525d' 2024-09-21T00:30:09.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37029 fp=#0x10000005225' 2024-09-21T00:30:09.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37061 fp=#0x100000051aa' 2024-09-21T00:30:09.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37093 fp=#0x1000000525e' 2024-09-21T00:30:09.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37189 fp=#0x10000003d02' 2024-09-21T00:30:09.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37221 fp=#0x10000003ce1' 2024-09-21T00:30:09.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37317 fp=#0x10000005c22' 2024-09-21T00:30:09.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37349 fp=#0x10000005bd4' 2024-09-21T00:30:09.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37381 fp=#0x10000005ee7' 2024-09-21T00:30:09.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37413 fp=#0x10000005eca' 2024-09-21T00:30:09.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37445 fp=#0x10000004adc' 2024-09-21T00:30:09.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37477 fp=#0x10000004b88' 2024-09-21T00:30:09.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37509 fp=#0x10000004bc4' 2024-09-21T00:30:09.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37541 fp=#0x10000004aac' 2024-09-21T00:30:09.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37573 fp=#0x10000004b58' 2024-09-21T00:30:09.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37605 fp=#0x10000005476' 2024-09-21T00:30:09.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37637 fp=#0x100000054fd' 2024-09-21T00:30:09.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37669 fp=#0x100000054e3' 2024-09-21T00:30:09.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37701 fp=#0x10000004224' 2024-09-21T00:30:09.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37733 fp=#0x100000042e6' 2024-09-21T00:30:09.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37765 fp=#0x10000005829' 2024-09-21T00:30:09.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37797 fp=#0x10000005803' 2024-09-21T00:30:09.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37829 fp=#0x1000000429f' 2024-09-21T00:30:09.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37861 fp=#0x100000042ea' 2024-09-21T00:30:09.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37893 fp=#0x10000001bc8' 2024-09-21T00:30:09.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37925 fp=#0x10000001b55' 2024-09-21T00:30:09.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37957 fp=#0x10000001bed' 2024-09-21T00:30:09.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37989 fp=#0x10000001b43' 2024-09-21T00:30:09.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38021 fp=#0x10000001cc3' 2024-09-21T00:30:09.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38085 fp=#0x10000003b3a' 2024-09-21T00:30:09.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38213 fp=#0x1000000617e' 2024-09-21T00:30:09.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38245 fp=#0x10000005852' 2024-09-21T00:30:09.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38277 fp=#0x100000059aa' 2024-09-21T00:30:09.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38309 fp=#0x1000000584d' 2024-09-21T00:30:09.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38341 fp=#0x10000005859' 2024-09-21T00:30:09.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38373 fp=#0x10000005966' 2024-09-21T00:30:09.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38405 fp=#0x10000002b9a' 2024-09-21T00:30:09.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38437 fp=#0x10000002bcd' 2024-09-21T00:30:09.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38469 fp=#0x10000002b3b' 2024-09-21T00:30:09.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38501 fp=#0x100000029e9' 2024-09-21T00:30:09.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38533 fp=#0x10000002ac9' 2024-09-21T00:30:09.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38565 fp=#0x10000002b41' 2024-09-21T00:30:09.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38597 fp=#0x10000002a0e' 2024-09-21T00:30:09.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38629 fp=#0x10000002b3e' 2024-09-21T00:30:09.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38693 fp=#0x1000000109a' 2024-09-21T00:30:09.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38725 fp=#0x100000010da' 2024-09-21T00:30:09.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38757 fp=#0x10000001fc4' 2024-09-21T00:30:09.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38789 fp=#0x10000001fe2' 2024-09-21T00:30:09.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38853 fp=#0x1000000406e' 2024-09-21T00:30:09.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38885 fp=#0x10000004162' 2024-09-21T00:30:09.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38981 fp=#0x100000041ce' 2024-09-21T00:30:09.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39045 fp=#0x10000001869' 2024-09-21T00:30:09.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39077 fp=#0x10000001d80' 2024-09-21T00:30:09.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39109 fp=#0x100000017f2' 2024-09-21T00:30:09.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39141 fp=#0x10000001bd7' 2024-09-21T00:30:09.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39173 fp=#0x1000000449d' 2024-09-21T00:30:09.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39237 fp=#0x100000044a8' 2024-09-21T00:30:09.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39269 fp=#0x100000044f2' 2024-09-21T00:30:09.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39301 fp=#0x100000048b8' 2024-09-21T00:30:09.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39333 fp=#0x100000048a8' 2024-09-21T00:30:09.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39365 fp=#0x100000048a2' 2024-09-21T00:30:09.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39397 fp=#0x10000005a3a' 2024-09-21T00:30:09.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39429 fp=#0x100000059e6' 2024-09-21T00:30:09.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39461 fp=#0x10000005e10' 2024-09-21T00:30:09.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39493 fp=#0x100000059c1' 2024-09-21T00:30:09.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39525 fp=#0x10000005ab1' 2024-09-21T00:30:09.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39557 fp=#0x10000005aa6' 2024-09-21T00:30:09.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39589 fp=#0x100000050d8' 2024-09-21T00:30:09.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39621 fp=#0x100000050a7' 2024-09-21T00:30:09.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39653 fp=#0x100000050e4' 2024-09-21T00:30:09.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39685 fp=#0x10000005d88' 2024-09-21T00:30:09.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39749 fp=#0x10000005b80' 2024-09-21T00:30:09.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39781 fp=#0x10000005b33' 2024-09-21T00:30:09.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39813 fp=#0x1000000638f' 2024-09-21T00:30:09.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39845 fp=#0x100000062cf' 2024-09-21T00:30:09.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39877 fp=#0x100000062d0' 2024-09-21T00:30:09.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39909 fp=#0x10000004eaf' 2024-09-21T00:30:09.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40037 fp=#0x100000062a7' 2024-09-21T00:30:09.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40069 fp=#0x100000062b2' 2024-09-21T00:30:09.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40101 fp=#0x100000062a4' 2024-09-21T00:30:09.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40133 fp=#0x10000002609' 2024-09-21T00:30:09.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40165 fp=#0x100000026c6' 2024-09-21T00:30:09.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40197 fp=#0x10000004a47' 2024-09-21T00:30:09.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40229 fp=#0x1000000131c' 2024-09-21T00:30:09.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40261 fp=#0x10000000a4e' 2024-09-21T00:30:09.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40293 fp=#0x10000004332' 2024-09-21T00:30:09.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40325 fp=#0x10000004298' 2024-09-21T00:30:09.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40357 fp=#0x10000000874' 2024-09-21T00:30:09.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40389 fp=#0x10000004250' 2024-09-21T00:30:09.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40421 fp=#0x10000000ad4' 2024-09-21T00:30:09.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40453 fp=#0x100000058f4' 2024-09-21T00:30:09.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40517 fp=#0x100000028a0' 2024-09-21T00:30:09.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40677 fp=#0x1000000495c' 2024-09-21T00:30:09.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40709 fp=#0x10000005ea9' 2024-09-21T00:30:09.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40741 fp=#0x1000000493b' 2024-09-21T00:30:09.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40773 fp=#0x1000000481a' 2024-09-21T00:30:09.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40805 fp=#0x10000005ec9' 2024-09-21T00:30:09.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40837 fp=#0x100000063d2' 2024-09-21T00:30:09.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40869 fp=#0x100000063e1' 2024-09-21T00:30:09.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40901 fp=#0x100000063d1' 2024-09-21T00:30:09.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40997 fp=#0x10000005133' 2024-09-21T00:30:09.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41029 fp=#0x100000049e4' 2024-09-21T00:30:09.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41061 fp=#0x10000005113' 2024-09-21T00:30:09.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41093 fp=#0x100000049fa' 2024-09-21T00:30:09.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41125 fp=#0x10000005119' 2024-09-21T00:30:09.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30790 fp=#0x10000005589' 2024-09-21T00:30:09.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30822 fp=#0x10000005317' 2024-09-21T00:30:09.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30886 fp=#0x10000005ff8' 2024-09-21T00:30:09.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32742 fp=#0x1000000579f' 2024-09-21T00:30:09.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32774 fp=#0x1000000577c' 2024-09-21T00:30:09.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32870 fp=#0x1000000502c' 2024-09-21T00:30:09.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32934 fp=#0x10000005042' 2024-09-21T00:30:09.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32998 fp=#0x10000005c37' 2024-09-21T00:30:09.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33030 fp=#0x10000005cba' 2024-09-21T00:30:09.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33062 fp=#0x10000005d07' 2024-09-21T00:30:09.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33094 fp=#0x10000005cbf' 2024-09-21T00:30:09.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33126 fp=#0x10000005ca8' 2024-09-21T00:30:09.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33158 fp=#0x10000006682' 2024-09-21T00:30:09.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33190 fp=#0x10000006039' 2024-09-21T00:30:09.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33318 fp=#0x10000006811' 2024-09-21T00:30:09.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33350 fp=#0x10000006815' 2024-09-21T00:30:09.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33382 fp=#0x1000000681d' 2024-09-21T00:30:09.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33446 fp=#0x10000003deb' 2024-09-21T00:30:09.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33478 fp=#0x10000003e96' 2024-09-21T00:30:09.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33606 fp=#0x10000005a42' 2024-09-21T00:30:09.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33734 fp=#0x1000000678a' 2024-09-21T00:30:09.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33766 fp=#0x1000000678f' 2024-09-21T00:30:09.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33798 fp=#0x100000067a6' 2024-09-21T00:30:09.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33926 fp=#0x1000000642e' 2024-09-21T00:30:09.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33958 fp=#0x100000063fa' 2024-09-21T00:30:09.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33990 fp=#0x100000063ff' 2024-09-21T00:30:09.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34022 fp=#0x100000047a6' 2024-09-21T00:30:09.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34054 fp=#0x10000004719' 2024-09-21T00:30:09.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34086 fp=#0x10000003268' 2024-09-21T00:30:09.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34118 fp=#0x100000031fa' 2024-09-21T00:30:09.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34150 fp=#0x10000006474' 2024-09-21T00:30:09.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34182 fp=#0x1000000627b' 2024-09-21T00:30:09.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34214 fp=#0x100000064a8' 2024-09-21T00:30:09.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34246 fp=#0x10000006469' 2024-09-21T00:30:09.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34278 fp=#0x10000006481' 2024-09-21T00:30:09.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34310 fp=#0x10000002518' 2024-09-21T00:30:09.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34342 fp=#0x100000025ac' 2024-09-21T00:30:09.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34374 fp=#0x1000000258f' 2024-09-21T00:30:09.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34406 fp=#0x100000061ad' 2024-09-21T00:30:09.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34438 fp=#0x10000002576' 2024-09-21T00:30:09.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34470 fp=#0x100000061a6' 2024-09-21T00:30:09.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34502 fp=#0x100000056a5' 2024-09-21T00:30:09.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34534 fp=#0x10000005671' 2024-09-21T00:30:09.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34566 fp=#0x10000006357' 2024-09-21T00:30:09.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34598 fp=#0x100000062c1' 2024-09-21T00:30:09.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34630 fp=#0x10000006375' 2024-09-21T00:30:09.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34662 fp=#0x10000006379' 2024-09-21T00:30:09.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34758 fp=#0x100000057b1' 2024-09-21T00:30:09.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34790 fp=#0x100000057d0' 2024-09-21T00:30:09.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34822 fp=#0x100000057ba' 2024-09-21T00:30:09.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34854 fp=#0x100000064d0' 2024-09-21T00:30:09.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34886 fp=#0x100000064e5' 2024-09-21T00:30:09.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34918 fp=#0x10000006956' 2024-09-21T00:30:09.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34950 fp=#0x100000052b8' 2024-09-21T00:30:09.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34982 fp=#0x100000052d9' 2024-09-21T00:30:09.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35014 fp=#0x100000052cc' 2024-09-21T00:30:09.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35046 fp=#0x100000060b3' 2024-09-21T00:30:09.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35110 fp=#0x100000060a3' 2024-09-21T00:30:09.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35142 fp=#0x100000060c7' 2024-09-21T00:30:09.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35206 fp=#0x10000006091' 2024-09-21T00:30:09.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35270 fp=#0x10000003f97' 2024-09-21T00:30:09.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35334 fp=#0x1000000539a' 2024-09-21T00:30:09.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35526 fp=#0x10000005ff5' 2024-09-21T00:30:09.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35558 fp=#0x10000005f7e' 2024-09-21T00:30:09.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35590 fp=#0x10000005f9b' 2024-09-21T00:30:09.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35622 fp=#0x10000005bb6' 2024-09-21T00:30:09.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35654 fp=#0x10000005bc3' 2024-09-21T00:30:09.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35686 fp=#0x10000005bad' 2024-09-21T00:30:09.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35718 fp=#0x10000004940' 2024-09-21T00:30:09.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35750 fp=#0x10000004f9f' 2024-09-21T00:30:09.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35782 fp=#0x10000004f97' 2024-09-21T00:30:09.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35814 fp=#0x10000004ffc' 2024-09-21T00:30:09.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35846 fp=#0x10000004f13' 2024-09-21T00:30:09.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35878 fp=#0x1000000672d' 2024-09-21T00:30:09.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35910 fp=#0x1000000670d' 2024-09-21T00:30:09.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35942 fp=#0x10000006733' 2024-09-21T00:30:09.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35974 fp=#0x10000006870' 2024-09-21T00:30:09.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36006 fp=#0x10000006842' 2024-09-21T00:30:09.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36102 fp=#0x10000005f56' 2024-09-21T00:30:09.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36198 fp=#0x10000006896' 2024-09-21T00:30:09.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36230 fp=#0x100000068a6' 2024-09-21T00:30:09.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36262 fp=#0x100000065ad' 2024-09-21T00:30:09.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36294 fp=#0x10000006607' 2024-09-21T00:30:09.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36326 fp=#0x100000065a3' 2024-09-21T00:30:09.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36358 fp=#0x100000065b4' 2024-09-21T00:30:09.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36390 fp=#0x100000065b3' 2024-09-21T00:30:09.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36422 fp=#0x10000003fff' 2024-09-21T00:30:09.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36486 fp=#0x1000000604a' 2024-09-21T00:30:09.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36550 fp=#0x100000059d4' 2024-09-21T00:30:09.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36582 fp=#0x100000059b3' 2024-09-21T00:30:09.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36614 fp=#0x10000005a10' 2024-09-21T00:30:09.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36646 fp=#0x10000005ab9' 2024-09-21T00:30:09.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36678 fp=#0x1000000481d' 2024-09-21T00:30:09.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36710 fp=#0x10000004d70' 2024-09-21T00:30:09.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36742 fp=#0x1000000474f' 2024-09-21T00:30:09.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36774 fp=#0x10000004794' 2024-09-21T00:30:09.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36838 fp=#0x10000005a23' 2024-09-21T00:30:09.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36966 fp=#0x10000005900' 2024-09-21T00:30:09.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36998 fp=#0x1000000524e' 2024-09-21T00:30:09.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37030 fp=#0x10000005196' 2024-09-21T00:30:09.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37062 fp=#0x10000005266' 2024-09-21T00:30:09.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37094 fp=#0x100000051b9' 2024-09-21T00:30:09.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37190 fp=#0x10000003cf3' 2024-09-21T00:30:09.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37222 fp=#0x10000003d0a' 2024-09-21T00:30:09.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37254 fp=#0x10000005bcd' 2024-09-21T00:30:09.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37286 fp=#0x10000005c66' 2024-09-21T00:30:09.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37350 fp=#0x10000005c45' 2024-09-21T00:30:09.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37382 fp=#0x10000005f07' 2024-09-21T00:30:09.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37414 fp=#0x10000005ef8' 2024-09-21T00:30:09.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37446 fp=#0x10000004bc2' 2024-09-21T00:30:09.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37478 fp=#0x10000004a85' 2024-09-21T00:30:09.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37510 fp=#0x10000004b35' 2024-09-21T00:30:09.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37542 fp=#0x10000004a60' 2024-09-21T00:30:09.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37574 fp=#0x100000054a9' 2024-09-21T00:30:09.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37606 fp=#0x10000005498' 2024-09-21T00:30:09.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37638 fp=#0x10000005480' 2024-09-21T00:30:09.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37670 fp=#0x1000000549e' 2024-09-21T00:30:09.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37702 fp=#0x10000004268' 2024-09-21T00:30:09.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37734 fp=#0x100000042fb' 2024-09-21T00:30:09.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37766 fp=#0x10000004273' 2024-09-21T00:30:09.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37798 fp=#0x1000000422c' 2024-09-21T00:30:09.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37830 fp=#0x1000000581d' 2024-09-21T00:30:09.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37862 fp=#0x1000000581e' 2024-09-21T00:30:09.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37894 fp=#0x10000001bb6' 2024-09-21T00:30:09.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37926 fp=#0x10000001cc0' 2024-09-21T00:30:09.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37958 fp=#0x10000001c09' 2024-09-21T00:30:09.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37990 fp=#0x10000001d26' 2024-09-21T00:30:09.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38022 fp=#0x10000003c91' 2024-09-21T00:30:09.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38054 fp=#0x10000003c43' 2024-09-21T00:30:09.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38150 fp=#0x10000003adc' 2024-09-21T00:30:09.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38214 fp=#0x100000058fc' 2024-09-21T00:30:09.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38246 fp=#0x1000000596a' 2024-09-21T00:30:09.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38278 fp=#0x10000006177' 2024-09-21T00:30:09.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38310 fp=#0x100000058fd' 2024-09-21T00:30:09.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38342 fp=#0x10000005887' 2024-09-21T00:30:09.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38374 fp=#0x10000005840' 2024-09-21T00:30:09.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38406 fp=#0x100000029f8' 2024-09-21T00:30:09.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38438 fp=#0x100000029d4' 2024-09-21T00:30:09.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38470 fp=#0x10000002bca' 2024-09-21T00:30:09.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38502 fp=#0x100000045ce' 2024-09-21T00:30:09.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38534 fp=#0x10000002a33' 2024-09-21T00:30:09.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38566 fp=#0x10000002a62' 2024-09-21T00:30:09.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38598 fp=#0x10000002b5c' 2024-09-21T00:30:09.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38630 fp=#0x100000045cf' 2024-09-21T00:30:09.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38694 fp=#0x100000010d0' 2024-09-21T00:30:09.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38726 fp=#0x100000048ac' 2024-09-21T00:30:09.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38758 fp=#0x100000010d4' 2024-09-21T00:30:09.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38854 fp=#0x100000041f8' 2024-09-21T00:30:09.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38918 fp=#0x100000053a3' 2024-09-21T00:30:09.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38982 fp=#0x1000000408e' 2024-09-21T00:30:09.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39046 fp=#0x1000000186e' 2024-09-21T00:30:09.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39078 fp=#0x100000017e2' 2024-09-21T00:30:09.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39110 fp=#0x10000001d8b' 2024-09-21T00:30:09.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39142 fp=#0x10000001bec' 2024-09-21T00:30:09.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39174 fp=#0x10000004509' 2024-09-21T00:30:09.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39206 fp=#0x100000044f5' 2024-09-21T00:30:09.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39238 fp=#0x10000004503' 2024-09-21T00:30:09.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39302 fp=#0x100000048ce' 2024-09-21T00:30:09.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39334 fp=#0x100000048b3' 2024-09-21T00:30:09.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39366 fp=#0x100000048c1' 2024-09-21T00:30:09.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39398 fp=#0x100000059c6' 2024-09-21T00:30:09.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39430 fp=#0x10000005e19' 2024-09-21T00:30:09.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39494 fp=#0x10000005adb' 2024-09-21T00:30:09.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39526 fp=#0x10000005a13' 2024-09-21T00:30:09.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39558 fp=#0x10000005e0a' 2024-09-21T00:30:09.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39590 fp=#0x100000050e6' 2024-09-21T00:30:09.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39622 fp=#0x100000050e8' 2024-09-21T00:30:09.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39654 fp=#0x10000005096' 2024-09-21T00:30:09.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39718 fp=#0x10000005d82' 2024-09-21T00:30:09.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39750 fp=#0x10000005b58' 2024-09-21T00:30:09.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39782 fp=#0x10000005b32' 2024-09-21T00:30:09.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39814 fp=#0x100000062fd' 2024-09-21T00:30:09.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39846 fp=#0x100000062e2' 2024-09-21T00:30:09.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39878 fp=#0x10000006343' 2024-09-21T00:30:09.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39910 fp=#0x10000004e8c' 2024-09-21T00:30:09.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39974 fp=#0x10000004ed5' 2024-09-21T00:30:09.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40038 fp=#0x100000062ca' 2024-09-21T00:30:09.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40070 fp=#0x10000006217' 2024-09-21T00:30:09.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40102 fp=#0x1000000622c' 2024-09-21T00:30:09.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40134 fp=#0x10000002697' 2024-09-21T00:30:09.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40166 fp=#0x1000000262e' 2024-09-21T00:30:09.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40198 fp=#0x1000000260e' 2024-09-21T00:30:09.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40230 fp=#0x10000000b2e' 2024-09-21T00:30:09.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40262 fp=#0x10000000acc' 2024-09-21T00:30:09.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40294 fp=#0x10000000864' 2024-09-21T00:30:09.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40326 fp=#0x10000004328' 2024-09-21T00:30:09.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40390 fp=#0x100000008e1' 2024-09-21T00:30:09.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40422 fp=#0x10000004325' 2024-09-21T00:30:09.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40486 fp=#0x100000058f8' 2024-09-21T00:30:09.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40518 fp=#0x10000002896' 2024-09-21T00:30:09.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40550 fp=#0x1000000283a' 2024-09-21T00:30:09.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40582 fp=#0x100000036d1' 2024-09-21T00:30:09.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40614 fp=#0x100000036ca' 2024-09-21T00:30:09.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40646 fp=#0x100000036d6' 2024-09-21T00:30:09.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40678 fp=#0x10000005ec0' 2024-09-21T00:30:09.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40710 fp=#0x10000004861' 2024-09-21T00:30:09.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40742 fp=#0x1000000487b' 2024-09-21T00:30:09.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40774 fp=#0x10000005edf' 2024-09-21T00:30:09.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40806 fp=#0x10000005e9c' 2024-09-21T00:30:09.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40838 fp=#0x100000063db' 2024-09-21T00:30:09.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40870 fp=#0x100000063de' 2024-09-21T00:30:09.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40902 fp=#0x100000063d8' 2024-09-21T00:30:09.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40934 fp=#0x100000063f5' 2024-09-21T00:30:09.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40998 fp=#0x10000004a01' 2024-09-21T00:30:09.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41030 fp=#0x1000000513e' 2024-09-21T00:30:09.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41062 fp=#0x10000004990' 2024-09-21T00:30:09.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41094 fp=#0x100000049c1' 2024-09-21T00:30:09.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30791 fp=#0x10000006757' 2024-09-21T00:30:09.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30823 fp=#0x10000005093' 2024-09-21T00:30:09.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30887 fp=#0x10000005983' 2024-09-21T00:30:09.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32711 fp=#0x1000000570f' 2024-09-21T00:30:09.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32743 fp=#0x1000000567c' 2024-09-21T00:30:09.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32807 fp=#0x10000005089' 2024-09-21T00:30:09.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32839 fp=#0x10000005074' 2024-09-21T00:30:09.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32871 fp=#0x100000066d0' 2024-09-21T00:30:09.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32967 fp=#0x10000005071' 2024-09-21T00:30:09.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32999 fp=#0x10000005cae' 2024-09-21T00:30:09.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33031 fp=#0x10000005d10' 2024-09-21T00:30:09.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33063 fp=#0x10000005ca5' 2024-09-21T00:30:09.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33095 fp=#0x10000005c62' 2024-09-21T00:30:09.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33127 fp=#0x10000005c3c' 2024-09-21T00:30:09.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33191 fp=#0x10000006070' 2024-09-21T00:30:09.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33223 fp=#0x1000000668c' 2024-09-21T00:30:09.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33255 fp=#0x1000000605c' 2024-09-21T00:30:09.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33287 fp=#0x10000006695' 2024-09-21T00:30:09.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33319 fp=#0x100000067e9' 2024-09-21T00:30:09.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33351 fp=#0x100000067f4' 2024-09-21T00:30:09.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33383 fp=#0x10000006802' 2024-09-21T00:30:09.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33415 fp=#0x10000003d76' 2024-09-21T00:30:09.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33447 fp=#0x10000003edf' 2024-09-21T00:30:09.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33479 fp=#0x10000003e3c' 2024-09-21T00:30:09.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33511 fp=#0x10000003e12' 2024-09-21T00:30:09.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33575 fp=#0x100000055b4' 2024-09-21T00:30:09.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33703 fp=#0x10000005ab5' 2024-09-21T00:30:09.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33735 fp=#0x10000006761' 2024-09-21T00:30:09.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33767 fp=#0x10000006770' 2024-09-21T00:30:09.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33799 fp=#0x10000006768' 2024-09-21T00:30:09.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33927 fp=#0x1000000640c' 2024-09-21T00:30:09.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33959 fp=#0x10000006409' 2024-09-21T00:30:09.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33991 fp=#0x10000006402' 2024-09-21T00:30:09.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34023 fp=#0x10000006421' 2024-09-21T00:30:09.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34055 fp=#0x10000004722' 2024-09-21T00:30:09.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34087 fp=#0x10000003244' 2024-09-21T00:30:09.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34119 fp=#0x1000000321f' 2024-09-21T00:30:09.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34151 fp=#0x100000062a9' 2024-09-21T00:30:09.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34183 fp=#0x10000006455' 2024-09-21T00:30:09.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34215 fp=#0x100000062a8' 2024-09-21T00:30:09.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34247 fp=#0x1000000628d' 2024-09-21T00:30:09.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34279 fp=#0x100000064ab' 2024-09-21T00:30:09.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34311 fp=#0x10000005688' 2024-09-21T00:30:09.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34343 fp=#0x1000000269b' 2024-09-21T00:30:09.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34375 fp=#0x10000006197' 2024-09-21T00:30:09.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34407 fp=#0x100000061c7' 2024-09-21T00:30:09.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34439 fp=#0x100000061b2' 2024-09-21T00:30:09.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34471 fp=#0x1000000250f' 2024-09-21T00:30:09.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34503 fp=#0x100000061b8' 2024-09-21T00:30:09.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34535 fp=#0x10000002584' 2024-09-21T00:30:09.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34567 fp=#0x10000006321' 2024-09-21T00:30:09.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34599 fp=#0x10000006350' 2024-09-21T00:30:09.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34631 fp=#0x1000000637d' 2024-09-21T00:30:09.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34663 fp=#0x10000006344' 2024-09-21T00:30:09.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34759 fp=#0x100000057c5' 2024-09-21T00:30:09.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34791 fp=#0x100000064b8' 2024-09-21T00:30:09.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34823 fp=#0x100000064d5' 2024-09-21T00:30:09.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34855 fp=#0x100000057be' 2024-09-21T00:30:09.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34887 fp=#0x10000006501' 2024-09-21T00:30:09.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34951 fp=#0x1000000530f' 2024-09-21T00:30:09.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35015 fp=#0x100000052fd' 2024-09-21T00:30:09.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35079 fp=#0x100000060cb' 2024-09-21T00:30:09.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35111 fp=#0x100000060f0' 2024-09-21T00:30:09.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35303 fp=#0x10000003f3f' 2024-09-21T00:30:09.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35399 fp=#0x10000005318' 2024-09-21T00:30:09.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35463 fp=#0x10000005f8e' 2024-09-21T00:30:09.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35495 fp=#0x10000005f8a' 2024-09-21T00:30:09.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35559 fp=#0x10000005f71' 2024-09-21T00:30:09.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35655 fp=#0x10000005b82' 2024-09-21T00:30:09.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35751 fp=#0x10000004f5a' 2024-09-21T00:30:09.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35783 fp=#0x10000004f4f' 2024-09-21T00:30:09.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35815 fp=#0x10000004f95' 2024-09-21T00:30:09.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35847 fp=#0x10000004f93' 2024-09-21T00:30:09.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35879 fp=#0x10000006752' 2024-09-21T00:30:09.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35911 fp=#0x1000000672c' 2024-09-21T00:30:09.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35943 fp=#0x1000000673a' 2024-09-21T00:30:09.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35975 fp=#0x10000006831' 2024-09-21T00:30:09.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36007 fp=#0x1000000685e' 2024-09-21T00:30:09.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36039 fp=#0x10000005d72' 2024-09-21T00:30:09.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36071 fp=#0x10000005f50' 2024-09-21T00:30:09.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36103 fp=#0x10000005f5f' 2024-09-21T00:30:09.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36199 fp=#0x10000006899' 2024-09-21T00:30:09.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36231 fp=#0x100000068cf' 2024-09-21T00:30:09.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36263 fp=#0x1000000660c' 2024-09-21T00:30:09.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36295 fp=#0x100000065fe' 2024-09-21T00:30:09.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36359 fp=#0x100000065af' 2024-09-21T00:30:09.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36423 fp=#0x10000003ff3' 2024-09-21T00:30:09.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36455 fp=#0x10000003fed' 2024-09-21T00:30:09.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36487 fp=#0x10000006066' 2024-09-21T00:30:09.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36551 fp=#0x10000005acf' 2024-09-21T00:30:09.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36583 fp=#0x100000059e8' 2024-09-21T00:30:09.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36615 fp=#0x100000059fc' 2024-09-21T00:30:09.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36679 fp=#0x100000047bf' 2024-09-21T00:30:09.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36711 fp=#0x10000004833' 2024-09-21T00:30:09.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36743 fp=#0x100000047b9' 2024-09-21T00:30:09.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36775 fp=#0x1000000475b' 2024-09-21T00:30:09.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36807 fp=#0x10000005908' 2024-09-21T00:30:09.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36935 fp=#0x10000005f3a' 2024-09-21T00:30:09.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36967 fp=#0x10000005910' 2024-09-21T00:30:09.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36999 fp=#0x1000000526d' 2024-09-21T00:30:09.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37063 fp=#0x100000051ea' 2024-09-21T00:30:09.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37095 fp=#0x100000051bf' 2024-09-21T00:30:09.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37159 fp=#0x100000053c7' 2024-09-21T00:30:09.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37191 fp=#0x10000003cef' 2024-09-21T00:30:09.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37255 fp=#0x10000005c2f' 2024-09-21T00:30:09.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37287 fp=#0x10000005c19' 2024-09-21T00:30:09.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37319 fp=#0x10000005ba9' 2024-09-21T00:30:09.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37351 fp=#0x10000005bb1' 2024-09-21T00:30:09.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37383 fp=#0x10000005f06' 2024-09-21T00:30:09.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37415 fp=#0x10000005ed0' 2024-09-21T00:30:09.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37447 fp=#0x10000004b8e' 2024-09-21T00:30:09.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37479 fp=#0x10000004b9b' 2024-09-21T00:30:09.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37511 fp=#0x10000004bb4' 2024-09-21T00:30:09.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37543 fp=#0x10000004ba3' 2024-09-21T00:30:09.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37575 fp=#0x1000000554c' 2024-09-21T00:30:09.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37607 fp=#0x10000005544' 2024-09-21T00:30:09.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37639 fp=#0x10000005484' 2024-09-21T00:30:09.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37671 fp=#0x10000005478' 2024-09-21T00:30:09.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37703 fp=#0x10000004294' 2024-09-21T00:30:09.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37735 fp=#0x100000042de' 2024-09-21T00:30:09.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37767 fp=#0x10000005826' 2024-09-21T00:30:09.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37799 fp=#0x10000004236' 2024-09-21T00:30:09.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37831 fp=#0x1000000424d' 2024-09-21T00:30:09.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37863 fp=#0x10000005839' 2024-09-21T00:30:09.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37895 fp=#0x10000001b50' 2024-09-21T00:30:09.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37927 fp=#0x10000001d70' 2024-09-21T00:30:09.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37959 fp=#0x100000031ab' 2024-09-21T00:30:09.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37991 fp=#0x10000001beb' 2024-09-21T00:30:09.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38023 fp=#0x10000003c84' 2024-09-21T00:30:09.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38087 fp=#0x10000003b35' 2024-09-21T00:30:09.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38151 fp=#0x10000003a9d' 2024-09-21T00:30:09.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38183 fp=#0x10000003ada' 2024-09-21T00:30:09.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38215 fp=#0x1000000616d' 2024-09-21T00:30:09.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38247 fp=#0x10000005881' 2024-09-21T00:30:09.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38279 fp=#0x10000005931' 2024-09-21T00:30:09.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38311 fp=#0x10000005949' 2024-09-21T00:30:09.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38343 fp=#0x10000005870' 2024-09-21T00:30:09.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38375 fp=#0x10000005862' 2024-09-21T00:30:09.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38407 fp=#0x100000045eb' 2024-09-21T00:30:09.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38439 fp=#0x100000045c7' 2024-09-21T00:30:09.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38503 fp=#0x100000045bf' 2024-09-21T00:30:09.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38535 fp=#0x100000029e1' 2024-09-21T00:30:09.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38567 fp=#0x10000002bc0' 2024-09-21T00:30:09.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38599 fp=#0x10000005dcd' 2024-09-21T00:30:09.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38663 fp=#0x100000010f6' 2024-09-21T00:30:09.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38855 fp=#0x10000004129' 2024-09-21T00:30:09.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38887 fp=#0x10000004015' 2024-09-21T00:30:09.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38919 fp=#0x10000004096' 2024-09-21T00:30:09.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38951 fp=#0x100000041de' 2024-09-21T00:30:09.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38983 fp=#0x100000041da' 2024-09-21T00:30:09.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39047 fp=#0x10000001856' 2024-09-21T00:30:09.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39079 fp=#0x10000001c98' 2024-09-21T00:30:09.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39111 fp=#0x10000001dc6' 2024-09-21T00:30:09.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39143 fp=#0x10000001da8' 2024-09-21T00:30:09.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39207 fp=#0x1000000449a' 2024-09-21T00:30:09.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39271 fp=#0x100000048e7' 2024-09-21T00:30:09.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39303 fp=#0x10000004893' 2024-09-21T00:30:09.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39335 fp=#0x10000004817' 2024-09-21T00:30:09.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39367 fp=#0x100000048e3' 2024-09-21T00:30:09.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39399 fp=#0x10000005af0' 2024-09-21T00:30:09.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39431 fp=#0x10000005a2b' 2024-09-21T00:30:09.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39463 fp=#0x10000005aaa' 2024-09-21T00:30:09.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39495 fp=#0x10000005a32' 2024-09-21T00:30:09.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39527 fp=#0x10000005add' 2024-09-21T00:30:09.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39559 fp=#0x100000050f2' 2024-09-21T00:30:09.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39591 fp=#0x100000050b5' 2024-09-21T00:30:09.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39623 fp=#0x100000050af' 2024-09-21T00:30:09.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39655 fp=#0x10000005099' 2024-09-21T00:30:09.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39719 fp=#0x10000005de5' 2024-09-21T00:30:09.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39751 fp=#0x10000005b35' 2024-09-21T00:30:09.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39783 fp=#0x10000005b56' 2024-09-21T00:30:09.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39815 fp=#0x10000006376' 2024-09-21T00:30:09.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39847 fp=#0x1000000638a' 2024-09-21T00:30:09.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39879 fp=#0x100000062da' 2024-09-21T00:30:09.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39911 fp=#0x10000004efe' 2024-09-21T00:30:09.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39975 fp=#0x10000004eae' 2024-09-21T00:30:09.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40007 fp=#0x10000004ef4' 2024-09-21T00:30:09.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40039 fp=#0x10000006279' 2024-09-21T00:30:09.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40071 fp=#0x100000061e8' 2024-09-21T00:30:09.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40103 fp=#0x10000006284' 2024-09-21T00:30:09.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40135 fp=#0x1000000263e' 2024-09-21T00:30:09.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40167 fp=#0x10000002653' 2024-09-21T00:30:09.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40199 fp=#0x10000002667' 2024-09-21T00:30:09.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40231 fp=#0x100000008cd' 2024-09-21T00:30:09.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40263 fp=#0x10000000903' 2024-09-21T00:30:09.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40295 fp=#0x10000000c61' 2024-09-21T00:30:09.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40327 fp=#0x10000000b28' 2024-09-21T00:30:09.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40391 fp=#0x1000000429e' 2024-09-21T00:30:09.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40423 fp=#0x100000042db' 2024-09-21T00:30:09.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40679 fp=#0x1000000486f' 2024-09-21T00:30:09.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40711 fp=#0x10000005ed9' 2024-09-21T00:30:09.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40743 fp=#0x1000000481c' 2024-09-21T00:30:09.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40775 fp=#0x10000004805' 2024-09-21T00:30:09.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40807 fp=#0x10000005ea3' 2024-09-21T00:30:09.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40871 fp=#0x100000063f4' 2024-09-21T00:30:09.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40967 fp=#0x100000063c1' 2024-09-21T00:30:09.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41031 fp=#0x10000005109' 2024-09-21T00:30:09.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41063 fp=#0x1000000510f' 2024-09-21T00:30:09.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41095 fp=#0x10000004e88' 2024-09-21T00:30:09.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30792 fp=#0x10000003c2e' 2024-09-21T00:30:09.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30824 fp=#0x10000001a8f' 2024-09-21T00:30:09.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30888 fp=#0x10000004976' 2024-09-21T00:30:09.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32712 fp=#0x1000000572c' 2024-09-21T00:30:09.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32744 fp=#0x10000005759' 2024-09-21T00:30:09.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32776 fp=#0x1000000579d' 2024-09-21T00:30:09.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32808 fp=#0x10000005092' 2024-09-21T00:30:09.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32904 fp=#0x10000005034' 2024-09-21T00:30:09.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32968 fp=#0x1000000675c' 2024-09-21T00:30:09.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33000 fp=#0x10000005d14' 2024-09-21T00:30:09.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33032 fp=#0x10000005ce7' 2024-09-21T00:30:09.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33064 fp=#0x10000005c93' 2024-09-21T00:30:09.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33096 fp=#0x10000005c94' 2024-09-21T00:30:09.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33128 fp=#0x10000005cfa' 2024-09-21T00:30:09.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33192 fp=#0x1000000604b' 2024-09-21T00:30:09.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33256 fp=#0x10000006060' 2024-09-21T00:30:09.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33288 fp=#0x1000000667e' 2024-09-21T00:30:09.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33320 fp=#0x10000006828' 2024-09-21T00:30:09.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33352 fp=#0x100000067fa' 2024-09-21T00:30:09.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33384 fp=#0x10000006808' 2024-09-21T00:30:09.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33416 fp=#0x10000003d84' 2024-09-21T00:30:09.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33448 fp=#0x10000003d95' 2024-09-21T00:30:09.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33480 fp=#0x10000003ea8' 2024-09-21T00:30:09.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33512 fp=#0x10000003e24' 2024-09-21T00:30:09.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33544 fp=#0x1000000558d' 2024-09-21T00:30:09.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33672 fp=#0x10000005a6b' 2024-09-21T00:30:09.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33704 fp=#0x10000005a90' 2024-09-21T00:30:09.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33736 fp=#0x100000067bd' 2024-09-21T00:30:09.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33768 fp=#0x10000006776' 2024-09-21T00:30:09.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33800 fp=#0x1000000676e' 2024-09-21T00:30:09.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33928 fp=#0x100000047b8' 2024-09-21T00:30:09.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33960 fp=#0x100000063f9' 2024-09-21T00:30:09.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33992 fp=#0x10000006449' 2024-09-21T00:30:09.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34024 fp=#0x10000006423' 2024-09-21T00:30:09.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34056 fp=#0x10000004766' 2024-09-21T00:30:09.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34088 fp=#0x1000000321b' 2024-09-21T00:30:09.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34120 fp=#0x10000003248' 2024-09-21T00:30:09.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34152 fp=#0x100000062b8' 2024-09-21T00:30:09.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34184 fp=#0x100000064aa' 2024-09-21T00:30:09.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34216 fp=#0x1000000628b' 2024-09-21T00:30:09.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34248 fp=#0x10000006265' 2024-09-21T00:30:09.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34280 fp=#0x10000006499' 2024-09-21T00:30:09.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34344 fp=#0x10000002680' 2024-09-21T00:30:09.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34376 fp=#0x100000056ca' 2024-09-21T00:30:09.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34408 fp=#0x10000002564' 2024-09-21T00:30:09.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34440 fp=#0x10000006193' 2024-09-21T00:30:09.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34472 fp=#0x100000061a8' 2024-09-21T00:30:09.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34504 fp=#0x10000006198' 2024-09-21T00:30:09.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34568 fp=#0x100000063ac' 2024-09-21T00:30:09.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34600 fp=#0x10000006620' 2024-09-21T00:30:09.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34632 fp=#0x100000062cd' 2024-09-21T00:30:09.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34664 fp=#0x1000000639e' 2024-09-21T00:30:09.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34760 fp=#0x100000057c1' 2024-09-21T00:30:09.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34792 fp=#0x100000064de' 2024-09-21T00:30:09.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34824 fp=#0x100000064b1' 2024-09-21T00:30:09.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34856 fp=#0x100000064d2' 2024-09-21T00:30:09.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34888 fp=#0x100000064f2' 2024-09-21T00:30:09.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34920 fp=#0x10000005339' 2024-09-21T00:30:09.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34952 fp=#0x10000005316' 2024-09-21T00:30:09.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34984 fp=#0x10000006948' 2024-09-21T00:30:09.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35048 fp=#0x100000060ef' 2024-09-21T00:30:09.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35080 fp=#0x100000060a0' 2024-09-21T00:30:09.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35112 fp=#0x1000000657f' 2024-09-21T00:30:09.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35144 fp=#0x10000006096' 2024-09-21T00:30:09.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35240 fp=#0x100000060b9' 2024-09-21T00:30:09.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35304 fp=#0x10000003fc7' 2024-09-21T00:30:09.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35464 fp=#0x10000005f6b' 2024-09-21T00:30:09.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35624 fp=#0x10000005ba2' 2024-09-21T00:30:09.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35656 fp=#0x10000005bc5' 2024-09-21T00:30:09.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35688 fp=#0x10000004937' 2024-09-21T00:30:09.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35720 fp=#0x10000004934' 2024-09-21T00:30:09.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35752 fp=#0x1000000501e' 2024-09-21T00:30:09.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35784 fp=#0x10000004fa5' 2024-09-21T00:30:09.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35816 fp=#0x10000004fd0' 2024-09-21T00:30:09.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35848 fp=#0x10000004fe2' 2024-09-21T00:30:09.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35880 fp=#0x10000006754' 2024-09-21T00:30:09.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35912 fp=#0x1000000670e' 2024-09-21T00:30:09.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35944 fp=#0x1000000683d' 2024-09-21T00:30:09.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35976 fp=#0x10000006885' 2024-09-21T00:30:09.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36008 fp=#0x1000000687c' 2024-09-21T00:30:09.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36072 fp=#0x10000005f4f' 2024-09-21T00:30:09.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36200 fp=#0x100000068c4' 2024-09-21T00:30:09.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36232 fp=#0x100000068a9' 2024-09-21T00:30:09.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36360 fp=#0x100000065a5' 2024-09-21T00:30:09.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36392 fp=#0x100000065be' 2024-09-21T00:30:09.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36424 fp=#0x10000006150' 2024-09-21T00:30:09.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36456 fp=#0x10000004008' 2024-09-21T00:30:09.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36488 fp=#0x10000006009' 2024-09-21T00:30:09.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36520 fp=#0x1000000602e' 2024-09-21T00:30:09.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36552 fp=#0x10000005a74' 2024-09-21T00:30:09.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36584 fp=#0x100000059bd' 2024-09-21T00:30:09.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36616 fp=#0x100000059d7' 2024-09-21T00:30:09.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36648 fp=#0x100000059e5' 2024-09-21T00:30:09.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36680 fp=#0x10000004924' 2024-09-21T00:30:09.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36712 fp=#0x10000004753' 2024-09-21T00:30:09.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36744 fp=#0x1000000477c' 2024-09-21T00:30:09.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36776 fp=#0x10000004772' 2024-09-21T00:30:09.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36808 fp=#0x10000005b06' 2024-09-21T00:30:09.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36840 fp=#0x1000000592d' 2024-09-21T00:30:09.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36904 fp=#0x1000000589e' 2024-09-21T00:30:09.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36936 fp=#0x10000005886' 2024-09-21T00:30:09.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37000 fp=#0x100000051fd' 2024-09-21T00:30:09.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37032 fp=#0x10000005194' 2024-09-21T00:30:09.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37064 fp=#0x100000051e2' 2024-09-21T00:30:09.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37096 fp=#0x100000051e8' 2024-09-21T00:30:09.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37128 fp=#0x100000053c0' 2024-09-21T00:30:09.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37192 fp=#0x10000003d28' 2024-09-21T00:30:09.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37224 fp=#0x10000005fe9' 2024-09-21T00:30:09.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37256 fp=#0x10000005c26' 2024-09-21T00:30:09.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37288 fp=#0x10000005c11' 2024-09-21T00:30:09.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37352 fp=#0x10000005c25' 2024-09-21T00:30:09.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37384 fp=#0x10000005f0f' 2024-09-21T00:30:09.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37416 fp=#0x10000005ece' 2024-09-21T00:30:09.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37448 fp=#0x10000004bc6' 2024-09-21T00:30:09.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37480 fp=#0x10000004b1a' 2024-09-21T00:30:09.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37512 fp=#0x10000004b21' 2024-09-21T00:30:09.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37544 fp=#0x10000004ab6' 2024-09-21T00:30:09.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37576 fp=#0x10000005542' 2024-09-21T00:30:09.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37608 fp=#0x10000005538' 2024-09-21T00:30:09.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37640 fp=#0x1000000550b' 2024-09-21T00:30:09.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37672 fp=#0x10000005489' 2024-09-21T00:30:09.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37704 fp=#0x10000004289' 2024-09-21T00:30:09.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37736 fp=#0x10000005825' 2024-09-21T00:30:09.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37768 fp=#0x1000000583a' 2024-09-21T00:30:09.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37800 fp=#0x100000042b2' 2024-09-21T00:30:09.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37832 fp=#0x10000004301' 2024-09-21T00:30:09.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37864 fp=#0x1000000582a' 2024-09-21T00:30:09.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37896 fp=#0x10000001c97' 2024-09-21T00:30:09.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37928 fp=#0x10000003d01' 2024-09-21T00:30:09.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37960 fp=#0x10000001c90' 2024-09-21T00:30:09.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37992 fp=#0x10000001bcc' 2024-09-21T00:30:09.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38024 fp=#0x10000003c37' 2024-09-21T00:30:09.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38088 fp=#0x10000003b32' 2024-09-21T00:30:09.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38120 fp=#0x10000003b3f' 2024-09-21T00:30:09.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38184 fp=#0x1000000584e' 2024-09-21T00:30:09.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38216 fp=#0x100000059a0' 2024-09-21T00:30:09.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38248 fp=#0x1000000595b' 2024-09-21T00:30:09.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38280 fp=#0x10000005918' 2024-09-21T00:30:09.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38312 fp=#0x10000005873' 2024-09-21T00:30:09.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38344 fp=#0x100000058dd' 2024-09-21T00:30:09.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38376 fp=#0x10000005865' 2024-09-21T00:30:09.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38440 fp=#0x100000029dc' 2024-09-21T00:30:09.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38472 fp=#0x10000002b85' 2024-09-21T00:30:09.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38504 fp=#0x10000002ae0' 2024-09-21T00:30:09.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38536 fp=#0x10000002af1' 2024-09-21T00:30:09.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38568 fp=#0x10000002b4a' 2024-09-21T00:30:09.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38600 fp=#0x100000045fc' 2024-09-21T00:30:09.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38632 fp=#0x10000001f6d' 2024-09-21T00:30:09.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38664 fp=#0x10000001f72' 2024-09-21T00:30:09.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38728 fp=#0x10000004897' 2024-09-21T00:30:09.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38760 fp=#0x100000048a9' 2024-09-21T00:30:09.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38792 fp=#0x100000010c3' 2024-09-21T00:30:09.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38856 fp=#0x10000004200' 2024-09-21T00:30:09.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38888 fp=#0x1000000411d' 2024-09-21T00:30:09.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38920 fp=#0x1000000409e' 2024-09-21T00:30:09.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38952 fp=#0x10000004098' 2024-09-21T00:30:09.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39016 fp=#0x10000003ffd' 2024-09-21T00:30:09.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39048 fp=#0x10000001865' 2024-09-21T00:30:09.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39080 fp=#0x10000001c5c' 2024-09-21T00:30:09.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39112 fp=#0x10000001cde' 2024-09-21T00:30:09.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39144 fp=#0x10000001ca4' 2024-09-21T00:30:09.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39176 fp=#0x10000004508' 2024-09-21T00:30:09.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39208 fp=#0x1000000450d' 2024-09-21T00:30:09.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39272 fp=#0x100000048de' 2024-09-21T00:30:09.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39304 fp=#0x100000048c5' 2024-09-21T00:30:09.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39336 fp=#0x10000004814' 2024-09-21T00:30:09.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39368 fp=#0x10000004862' 2024-09-21T00:30:09.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39400 fp=#0x10000005e03' 2024-09-21T00:30:09.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39432 fp=#0x10000005dfd' 2024-09-21T00:30:09.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39464 fp=#0x10000005e34' 2024-09-21T00:30:09.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39496 fp=#0x10000005e02' 2024-09-21T00:30:09.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39528 fp=#0x10000005e1d' 2024-09-21T00:30:09.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39560 fp=#0x100000050ed' 2024-09-21T00:30:09.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39592 fp=#0x100000050e0' 2024-09-21T00:30:09.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39656 fp=#0x100000050b1' 2024-09-21T00:30:09.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39784 fp=#0x10000005b51' 2024-09-21T00:30:09.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39816 fp=#0x100000062d2' 2024-09-21T00:30:09.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39848 fp=#0x100000062ec' 2024-09-21T00:30:09.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39880 fp=#0x1000000635a' 2024-09-21T00:30:09.914 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39912 fp=#0x10000004ea0' 2024-09-21T00:30:09.914 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39944 fp=#0x10000004e9c' 2024-09-21T00:30:09.914 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39976 fp=#0x10000004ef8' 2024-09-21T00:30:09.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40040 fp=#0x10000006234' 2024-09-21T00:30:09.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40072 fp=#0x10000006215' 2024-09-21T00:30:09.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40104 fp=#0x10000006255' 2024-09-21T00:30:09.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40136 fp=#0x1000000267c' 2024-09-21T00:30:09.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40168 fp=#0x1000000266d' 2024-09-21T00:30:09.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40232 fp=#0x10000000ae4' 2024-09-21T00:30:09.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40264 fp=#0x10000000a9a' 2024-09-21T00:30:09.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40296 fp=#0x10000000907' 2024-09-21T00:30:09.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40328 fp=#0x100000042d3' 2024-09-21T00:30:09.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40360 fp=#0x1000000086f' 2024-09-21T00:30:09.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40392 fp=#0x100000042ed' 2024-09-21T00:30:09.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40424 fp=#0x10000004254' 2024-09-21T00:30:09.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40488 fp=#0x10000005f2e' 2024-09-21T00:30:09.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40520 fp=#0x10000002893' 2024-09-21T00:30:09.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40680 fp=#0x100000047fc' 2024-09-21T00:30:09.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40712 fp=#0x10000004877' 2024-09-21T00:30:09.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40744 fp=#0x10000005eaa' 2024-09-21T00:30:09.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40776 fp=#0x10000004903' 2024-09-21T00:30:09.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40808 fp=#0x1000000483c' 2024-09-21T00:30:09.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40840 fp=#0x100000063cb' 2024-09-21T00:30:09.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40968 fp=#0x10000004a30' 2024-09-21T00:30:09.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41000 fp=#0x10000005123' 2024-09-21T00:30:09.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41064 fp=#0x100000049aa' 2024-09-21T00:30:09.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41096 fp=#0x10000005124' 2024-09-21T00:30:09.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30825 fp=#0x10000005f68' 2024-09-21T00:30:09.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30889 fp=#0x1000000474b' 2024-09-21T00:30:09.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32713 fp=#0x100000057a9' 2024-09-21T00:30:09.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32745 fp=#0x100000057a2' 2024-09-21T00:30:09.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32809 fp=#0x10000006708' 2024-09-21T00:30:09.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32841 fp=#0x1000000508b' 2024-09-21T00:30:09.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33001 fp=#0x10000005c79' 2024-09-21T00:30:09.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33033 fp=#0x10000005c78' 2024-09-21T00:30:09.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33065 fp=#0x10000005c76' 2024-09-21T00:30:09.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33097 fp=#0x10000005c7f' 2024-09-21T00:30:09.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33129 fp=#0x10000005ca1' 2024-09-21T00:30:09.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33161 fp=#0x10000006053' 2024-09-21T00:30:09.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33193 fp=#0x1000000603c' 2024-09-21T00:30:09.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33225 fp=#0x10000006059' 2024-09-21T00:30:09.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33321 fp=#0x100000067ce' 2024-09-21T00:30:09.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33353 fp=#0x100000067d5' 2024-09-21T00:30:09.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33385 fp=#0x100000067d0' 2024-09-21T00:30:09.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33417 fp=#0x10000003e0f' 2024-09-21T00:30:09.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33481 fp=#0x10000003db3' 2024-09-21T00:30:09.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33513 fp=#0x10000003d80' 2024-09-21T00:30:09.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33545 fp=#0x10000005659' 2024-09-21T00:30:09.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33609 fp=#0x1000000566b' 2024-09-21T00:30:09.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33641 fp=#0x10000005a06' 2024-09-21T00:30:09.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33705 fp=#0x10000005654' 2024-09-21T00:30:09.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33737 fp=#0x1000000677b' 2024-09-21T00:30:09.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33769 fp=#0x10000006796' 2024-09-21T00:30:09.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33801 fp=#0x10000006787' 2024-09-21T00:30:09.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33929 fp=#0x100000047f5' 2024-09-21T00:30:09.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33961 fp=#0x1000000642f' 2024-09-21T00:30:09.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33993 fp=#0x100000047ed' 2024-09-21T00:30:09.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34025 fp=#0x10000004783' 2024-09-21T00:30:09.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34057 fp=#0x10000004789' 2024-09-21T00:30:09.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34121 fp=#0x100000031ed' 2024-09-21T00:30:09.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34153 fp=#0x1000000649b' 2024-09-21T00:30:09.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34185 fp=#0x1000000622f' 2024-09-21T00:30:09.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34217 fp=#0x10000006254' 2024-09-21T00:30:09.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34249 fp=#0x1000000624d' 2024-09-21T00:30:09.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34281 fp=#0x10000006212' 2024-09-21T00:30:09.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34313 fp=#0x100000026ce' 2024-09-21T00:30:09.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34345 fp=#0x10000002643' 2024-09-21T00:30:09.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34377 fp=#0x100000061d4' 2024-09-21T00:30:09.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34409 fp=#0x100000061cf' 2024-09-21T00:30:09.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34473 fp=#0x100000025aa' 2024-09-21T00:30:09.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34505 fp=#0x100000061a2' 2024-09-21T00:30:09.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34537 fp=#0x10000002683' 2024-09-21T00:30:09.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34569 fp=#0x100000062f6' 2024-09-21T00:30:09.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34601 fp=#0x1000000635e' 2024-09-21T00:30:09.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34633 fp=#0x1000000631f' 2024-09-21T00:30:09.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34665 fp=#0x100000062c7' 2024-09-21T00:30:09.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34761 fp=#0x100000057cf' 2024-09-21T00:30:09.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34793 fp=#0x100000057e1' 2024-09-21T00:30:09.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34825 fp=#0x100000064fa' 2024-09-21T00:30:09.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34857 fp=#0x100000057ea' 2024-09-21T00:30:09.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34889 fp=#0x100000064f8' 2024-09-21T00:30:09.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34921 fp=#0x1000000535a' 2024-09-21T00:30:09.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34953 fp=#0x1000000695f' 2024-09-21T00:30:09.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35113 fp=#0x100000060e6' 2024-09-21T00:30:09.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35241 fp=#0x10000003fc4' 2024-09-21T00:30:09.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35273 fp=#0x10000003fd0' 2024-09-21T00:30:09.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35337 fp=#0x1000000538e' 2024-09-21T00:30:09.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35369 fp=#0x1000000531a' 2024-09-21T00:30:09.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35465 fp=#0x10000005fef' 2024-09-21T00:30:09.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35529 fp=#0x10000005fe0' 2024-09-21T00:30:09.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35561 fp=#0x10000005fc0' 2024-09-21T00:30:09.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35593 fp=#0x10000005fcf' 2024-09-21T00:30:09.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35625 fp=#0x10000005bd9' 2024-09-21T00:30:09.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35657 fp=#0x10000005b89' 2024-09-21T00:30:09.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35689 fp=#0x10000004918' 2024-09-21T00:30:09.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35721 fp=#0x10000004949' 2024-09-21T00:30:09.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35753 fp=#0x10000004f32' 2024-09-21T00:30:09.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35785 fp=#0x10000004fae' 2024-09-21T00:30:09.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35817 fp=#0x10000004fd8' 2024-09-21T00:30:09.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35849 fp=#0x10000004fd4' 2024-09-21T00:30:09.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35881 fp=#0x10000006737' 2024-09-21T00:30:09.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35913 fp=#0x10000006753' 2024-09-21T00:30:09.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35945 fp=#0x1000000687e' 2024-09-21T00:30:09.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35977 fp=#0x10000006858' 2024-09-21T00:30:09.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36009 fp=#0x1000000686e' 2024-09-21T00:30:09.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36201 fp=#0x10000006894' 2024-09-21T00:30:09.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36233 fp=#0x100000068a8' 2024-09-21T00:30:09.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36265 fp=#0x100000065a4' 2024-09-21T00:30:09.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36297 fp=#0x100000065b8' 2024-09-21T00:30:09.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36361 fp=#0x10000006610' 2024-09-21T00:30:09.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36393 fp=#0x100000065fd' 2024-09-21T00:30:09.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36425 fp=#0x1000000614f' 2024-09-21T00:30:09.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36457 fp=#0x10000003ffc' 2024-09-21T00:30:09.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36489 fp=#0x1000000605d' 2024-09-21T00:30:09.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36521 fp=#0x10000006006' 2024-09-21T00:30:09.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36553 fp=#0x10000005ad3' 2024-09-21T00:30:09.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36585 fp=#0x10000005a08' 2024-09-21T00:30:09.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36617 fp=#0x10000005abc' 2024-09-21T00:30:09.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36649 fp=#0x100000059f6' 2024-09-21T00:30:09.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36681 fp=#0x10000004791' 2024-09-21T00:30:09.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36713 fp=#0x100000048bb' 2024-09-21T00:30:09.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36745 fp=#0x100000048f4' 2024-09-21T00:30:09.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36777 fp=#0x100000047cf' 2024-09-21T00:30:09.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36809 fp=#0x100000058a0' 2024-09-21T00:30:09.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36969 fp=#0x10000005a28' 2024-09-21T00:30:09.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37001 fp=#0x10000005263' 2024-09-21T00:30:09.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37033 fp=#0x1000000549d' 2024-09-21T00:30:09.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37065 fp=#0x1000000519e' 2024-09-21T00:30:09.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37097 fp=#0x10000005230' 2024-09-21T00:30:09.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37193 fp=#0x10000003cbe' 2024-09-21T00:30:09.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37225 fp=#0x10000003cbb' 2024-09-21T00:30:09.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37257 fp=#0x10000005c4b' 2024-09-21T00:30:09.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37289 fp=#0x10000005b95' 2024-09-21T00:30:09.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37385 fp=#0x10000005efd' 2024-09-21T00:30:09.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37417 fp=#0x10000005efb' 2024-09-21T00:30:09.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37449 fp=#0x10000004a64' 2024-09-21T00:30:09.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37481 fp=#0x10000004b93' 2024-09-21T00:30:09.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37513 fp=#0x10000004ab7' 2024-09-21T00:30:09.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37545 fp=#0x10000004aa7' 2024-09-21T00:30:09.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37577 fp=#0x1000000548a' 2024-09-21T00:30:09.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37609 fp=#0x10000005531' 2024-09-21T00:30:09.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37641 fp=#0x100000054a6' 2024-09-21T00:30:09.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37673 fp=#0x100000054f3' 2024-09-21T00:30:09.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37705 fp=#0x10000004257' 2024-09-21T00:30:09.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37737 fp=#0x10000005824' 2024-09-21T00:30:09.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37769 fp=#0x10000005810' 2024-09-21T00:30:09.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37801 fp=#0x1000000426a' 2024-09-21T00:30:09.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37833 fp=#0x100000042a6' 2024-09-21T00:30:09.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37865 fp=#0x1000000425e' 2024-09-21T00:30:09.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37897 fp=#0x10000001d73' 2024-09-21T00:30:09.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37929 fp=#0x10000001bf6' 2024-09-21T00:30:09.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37961 fp=#0x10000001cdf' 2024-09-21T00:30:09.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37993 fp=#0x10000001d01' 2024-09-21T00:30:09.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38025 fp=#0x10000003c82' 2024-09-21T00:30:09.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38057 fp=#0x10000003c4c' 2024-09-21T00:30:09.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38089 fp=#0x10000003b48' 2024-09-21T00:30:09.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38121 fp=#0x10000003ad5' 2024-09-21T00:30:09.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38185 fp=#0x10000005999' 2024-09-21T00:30:09.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38217 fp=#0x100000059b1' 2024-09-21T00:30:09.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38249 fp=#0x10000005845' 2024-09-21T00:30:09.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38281 fp=#0x10000005850' 2024-09-21T00:30:09.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38313 fp=#0x1000000616c' 2024-09-21T00:30:09.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38345 fp=#0x10000005914' 2024-09-21T00:30:09.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38377 fp=#0x1000000587d' 2024-09-21T00:30:09.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38409 fp=#0x10000002a37' 2024-09-21T00:30:09.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38441 fp=#0x100000045cb' 2024-09-21T00:30:09.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38473 fp=#0x10000002a2a' 2024-09-21T00:30:09.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38505 fp=#0x10000004602' 2024-09-21T00:30:09.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38537 fp=#0x10000002b56' 2024-09-21T00:30:09.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38569 fp=#0x100000045d0' 2024-09-21T00:30:09.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38601 fp=#0x100000045f3' 2024-09-21T00:30:10.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38665 fp=#0x10000000f21' 2024-09-21T00:30:10.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38697 fp=#0x10000001082' 2024-09-21T00:30:10.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38761 fp=#0x100000010a5' 2024-09-21T00:30:10.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38793 fp=#0x100000010ee' 2024-09-21T00:30:10.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38857 fp=#0x1000000400c' 2024-09-21T00:30:10.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38889 fp=#0x100000041c6' 2024-09-21T00:30:10.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38953 fp=#0x10000004060' 2024-09-21T00:30:10.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38985 fp=#0x10000004043' 2024-09-21T00:30:10.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39017 fp=#0x1000000405a' 2024-09-21T00:30:10.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39049 fp=#0x10000001861' 2024-09-21T00:30:10.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39081 fp=#0x10000001ff9' 2024-09-21T00:30:10.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39145 fp=#0x10000001cfc' 2024-09-21T00:30:10.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39177 fp=#0x10000004511' 2024-09-21T00:30:10.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39241 fp=#0x100000044bf' 2024-09-21T00:30:10.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39273 fp=#0x10000004875' 2024-09-21T00:30:10.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39305 fp=#0x100000048f1' 2024-09-21T00:30:10.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39337 fp=#0x100000048d6' 2024-09-21T00:30:10.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39369 fp=#0x10000004878' 2024-09-21T00:30:10.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39401 fp=#0x10000005e06' 2024-09-21T00:30:10.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39433 fp=#0x10000005e3a' 2024-09-21T00:30:10.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39465 fp=#0x10000005af5' 2024-09-21T00:30:10.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39497 fp=#0x10000005a22' 2024-09-21T00:30:10.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39529 fp=#0x10000005dfe' 2024-09-21T00:30:10.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39561 fp=#0x100000050ab' 2024-09-21T00:30:10.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39625 fp=#0x100000050d6' 2024-09-21T00:30:10.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39721 fp=#0x10000005d8e' 2024-09-21T00:30:10.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39753 fp=#0x10000005b49' 2024-09-21T00:30:10.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39817 fp=#0x10000006386' 2024-09-21T00:30:10.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39849 fp=#0x1000000638e' 2024-09-21T00:30:10.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39881 fp=#0x100000062d6' 2024-09-21T00:30:10.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39913 fp=#0x10000004ee8' 2024-09-21T00:30:10.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39977 fp=#0x10000004ed8' 2024-09-21T00:30:10.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40009 fp=#0x10000004ea1' 2024-09-21T00:30:10.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40041 fp=#0x100000062b0' 2024-09-21T00:30:10.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40073 fp=#0x1000000621f' 2024-09-21T00:30:10.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40105 fp=#0x1000000623d' 2024-09-21T00:30:10.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40137 fp=#0x100000026cf' 2024-09-21T00:30:10.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40169 fp=#0x1000000262b' 2024-09-21T00:30:10.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40201 fp=#0x1000000265a' 2024-09-21T00:30:10.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40233 fp=#0x10000000888' 2024-09-21T00:30:10.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40265 fp=#0x10000000a64' 2024-09-21T00:30:10.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40297 fp=#0x10000004322' 2024-09-21T00:30:10.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40329 fp=#0x1000000432e' 2024-09-21T00:30:10.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40361 fp=#0x10000000aa8' 2024-09-21T00:30:10.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40393 fp=#0x100000042fc' 2024-09-21T00:30:10.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40425 fp=#0x100000042e9' 2024-09-21T00:30:10.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40457 fp=#0x100000058f2' 2024-09-21T00:30:10.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40681 fp=#0x1000000494c' 2024-09-21T00:30:10.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40713 fp=#0x10000004884' 2024-09-21T00:30:10.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40745 fp=#0x10000005eb1' 2024-09-21T00:30:10.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40777 fp=#0x10000004859' 2024-09-21T00:30:10.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40809 fp=#0x10000004852' 2024-09-21T00:30:10.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40873 fp=#0x100000063c8' 2024-09-21T00:30:10.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40937 fp=#0x100000063cc' 2024-09-21T00:30:10.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40969 fp=#0x10000005114' 2024-09-21T00:30:10.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41001 fp=#0x10000005101' 2024-09-21T00:30:10.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41033 fp=#0x100000049fe' 2024-09-21T00:30:10.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41065 fp=#0x10000005126' 2024-09-21T00:30:10.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41097 fp=#0x1000000512b' 2024-09-21T00:30:10.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30794 fp=#0x1000000470b' 2024-09-21T00:30:10.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30826 fp=#0x10000005d76' 2024-09-21T00:30:10.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32682 fp=#0x10000005699' 2024-09-21T00:30:10.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32778 fp=#0x1000000573f' 2024-09-21T00:30:10.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32810 fp=#0x1000000505c' 2024-09-21T00:30:10.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32842 fp=#0x10000005079' 2024-09-21T00:30:10.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32970 fp=#0x10000005083' 2024-09-21T00:30:10.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33002 fp=#0x10000005d0d' 2024-09-21T00:30:10.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33034 fp=#0x10000005c83' 2024-09-21T00:30:10.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33066 fp=#0x10000005ca4' 2024-09-21T00:30:10.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33098 fp=#0x10000005ca3' 2024-09-21T00:30:10.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33130 fp=#0x10000005c81' 2024-09-21T00:30:10.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33162 fp=#0x10000006056' 2024-09-21T00:30:10.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33194 fp=#0x10000006694' 2024-09-21T00:30:10.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33226 fp=#0x10000006692' 2024-09-21T00:30:10.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33290 fp=#0x10000006689' 2024-09-21T00:30:10.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33322 fp=#0x100000067d1' 2024-09-21T00:30:10.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33354 fp=#0x10000006803' 2024-09-21T00:30:10.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33386 fp=#0x100000053ae' 2024-09-21T00:30:10.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33418 fp=#0x10000003e68' 2024-09-21T00:30:10.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33482 fp=#0x10000003e50' 2024-09-21T00:30:10.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33514 fp=#0x100000047f6' 2024-09-21T00:30:10.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33610 fp=#0x10000005a76' 2024-09-21T00:30:10.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33642 fp=#0x10000005a53' 2024-09-21T00:30:10.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33674 fp=#0x1000000564d' 2024-09-21T00:30:10.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33706 fp=#0x10000005aa0' 2024-09-21T00:30:10.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33738 fp=#0x100000067a5' 2024-09-21T00:30:10.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33770 fp=#0x1000000677d' 2024-09-21T00:30:10.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33802 fp=#0x10000006795' 2024-09-21T00:30:10.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33930 fp=#0x10000006443' 2024-09-21T00:30:10.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33962 fp=#0x1000000475f' 2024-09-21T00:30:10.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33994 fp=#0x1000000641e' 2024-09-21T00:30:10.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34026 fp=#0x1000000473e' 2024-09-21T00:30:10.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34058 fp=#0x10000006435' 2024-09-21T00:30:10.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34090 fp=#0x10000003209' 2024-09-21T00:30:10.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34154 fp=#0x10000006283' 2024-09-21T00:30:10.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34186 fp=#0x10000006289' 2024-09-21T00:30:10.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34218 fp=#0x10000006476' 2024-09-21T00:30:10.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34250 fp=#0x1000000647b' 2024-09-21T00:30:10.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34282 fp=#0x1000000629c' 2024-09-21T00:30:10.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34314 fp=#0x1000000255b' 2024-09-21T00:30:10.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34346 fp=#0x100000025a0' 2024-09-21T00:30:10.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34378 fp=#0x1000000374f' 2024-09-21T00:30:10.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34410 fp=#0x10000002506' 2024-09-21T00:30:10.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34474 fp=#0x1000000256e' 2024-09-21T00:30:10.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34506 fp=#0x1000000251c' 2024-09-21T00:30:10.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34538 fp=#0x10000003745' 2024-09-21T00:30:10.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34570 fp=#0x100000062fe' 2024-09-21T00:30:10.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34602 fp=#0x10000006368' 2024-09-21T00:30:10.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34634 fp=#0x100000062c5' 2024-09-21T00:30:10.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34666 fp=#0x10000006324' 2024-09-21T00:30:10.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34762 fp=#0x100000064f9' 2024-09-21T00:30:10.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34794 fp=#0x100000064c2' 2024-09-21T00:30:10.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34826 fp=#0x100000057ee' 2024-09-21T00:30:10.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34858 fp=#0x100000057d5' 2024-09-21T00:30:10.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34890 fp=#0x100000064d1' 2024-09-21T00:30:10.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34922 fp=#0x1000000695c' 2024-09-21T00:30:10.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34986 fp=#0x100000052e2' 2024-09-21T00:30:10.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35050 fp=#0x100000060d7' 2024-09-21T00:30:10.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35114 fp=#0x100000060a8' 2024-09-21T00:30:10.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35146 fp=#0x1000000609a' 2024-09-21T00:30:10.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35178 fp=#0x100000060b0' 2024-09-21T00:30:10.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35210 fp=#0x10000006099' 2024-09-21T00:30:10.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35242 fp=#0x1000000650b' 2024-09-21T00:30:10.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35274 fp=#0x10000003fa5' 2024-09-21T00:30:10.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35306 fp=#0x10000003f52' 2024-09-21T00:30:10.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35370 fp=#0x10000005327' 2024-09-21T00:30:10.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35530 fp=#0x10000005f92' 2024-09-21T00:30:10.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35562 fp=#0x10000005f74' 2024-09-21T00:30:10.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35626 fp=#0x10000005b99' 2024-09-21T00:30:10.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35658 fp=#0x10000005b7c' 2024-09-21T00:30:10.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35690 fp=#0x1000000496f' 2024-09-21T00:30:10.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35722 fp=#0x10000004969' 2024-09-21T00:30:10.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35754 fp=#0x10000004f37' 2024-09-21T00:30:10.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35786 fp=#0x10000004fbe' 2024-09-21T00:30:10.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35818 fp=#0x10000004f11' 2024-09-21T00:30:10.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35850 fp=#0x10000004f8b' 2024-09-21T00:30:10.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35882 fp=#0x100000066ff' 2024-09-21T00:30:10.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35914 fp=#0x10000006749' 2024-09-21T00:30:10.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35946 fp=#0x10000006866' 2024-09-21T00:30:10.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35978 fp=#0x10000006863' 2024-09-21T00:30:10.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36010 fp=#0x1000000687f' 2024-09-21T00:30:10.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36170 fp=#0x10000005f65' 2024-09-21T00:30:10.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36202 fp=#0x100000068dc' 2024-09-21T00:30:10.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36234 fp=#0x10000006890' 2024-09-21T00:30:10.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36266 fp=#0x1000000661b' 2024-09-21T00:30:10.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36298 fp=#0x100000065f5' 2024-09-21T00:30:10.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36426 fp=#0x10000003ff1' 2024-09-21T00:30:10.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36458 fp=#0x10000006148' 2024-09-21T00:30:10.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36522 fp=#0x10000005ffe' 2024-09-21T00:30:10.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36554 fp=#0x100000059ea' 2024-09-21T00:30:10.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36650 fp=#0x10000005a7d' 2024-09-21T00:30:10.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36682 fp=#0x10000004798' 2024-09-21T00:30:10.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36714 fp=#0x100000047a7' 2024-09-21T00:30:10.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36746 fp=#0x10000004818' 2024-09-21T00:30:10.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36778 fp=#0x100000048cd' 2024-09-21T00:30:10.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36874 fp=#0x100000058a7' 2024-09-21T00:30:10.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36938 fp=#0x10000005a34' 2024-09-21T00:30:10.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37002 fp=#0x10000005256' 2024-09-21T00:30:10.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37034 fp=#0x100000051ce' 2024-09-21T00:30:10.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37066 fp=#0x100000051d3' 2024-09-21T00:30:10.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37098 fp=#0x10000005200' 2024-09-21T00:30:10.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37194 fp=#0x10000003cde' 2024-09-21T00:30:10.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37226 fp=#0x10000003cce' 2024-09-21T00:30:10.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37258 fp=#0x10000005bf6' 2024-09-21T00:30:10.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37322 fp=#0x10000005ba8' 2024-09-21T00:30:10.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37354 fp=#0x10000005bba' 2024-09-21T00:30:10.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37386 fp=#0x10000005ed4' 2024-09-21T00:30:10.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37418 fp=#0x10000005ef6' 2024-09-21T00:30:10.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37450 fp=#0x10000004af6' 2024-09-21T00:30:10.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37482 fp=#0x10000004a68' 2024-09-21T00:30:10.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37514 fp=#0x10000004a55' 2024-09-21T00:30:10.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37546 fp=#0x10000004b94' 2024-09-21T00:30:10.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37578 fp=#0x1000000551a' 2024-09-21T00:30:10.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37610 fp=#0x100000054b6' 2024-09-21T00:30:10.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37642 fp=#0x1000000579a' 2024-09-21T00:30:10.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37674 fp=#0x1000000548b' 2024-09-21T00:30:10.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37706 fp=#0x1000000431a' 2024-09-21T00:30:10.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37738 fp=#0x10000004275' 2024-09-21T00:30:10.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37770 fp=#0x1000000428e' 2024-09-21T00:30:10.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37802 fp=#0x100000057ff' 2024-09-21T00:30:10.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37834 fp=#0x100000057f3' 2024-09-21T00:30:10.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37866 fp=#0x1000000430d' 2024-09-21T00:30:10.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37898 fp=#0x10000001bfd' 2024-09-21T00:30:10.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37930 fp=#0x10000001c33' 2024-09-21T00:30:10.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37962 fp=#0x10000001c30' 2024-09-21T00:30:10.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37994 fp=#0x10000001c45' 2024-09-21T00:30:10.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38026 fp=#0x10000003cab' 2024-09-21T00:30:10.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38058 fp=#0x10000003c3d' 2024-09-21T00:30:10.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38090 fp=#0x10000003aa0' 2024-09-21T00:30:10.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38154 fp=#0x10000003a42' 2024-09-21T00:30:10.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38186 fp=#0x1000000598d' 2024-09-21T00:30:10.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38218 fp=#0x10000006182' 2024-09-21T00:30:10.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38250 fp=#0x10000005934' 2024-09-21T00:30:10.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38282 fp=#0x100000058f7' 2024-09-21T00:30:10.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38346 fp=#0x10000005863' 2024-09-21T00:30:10.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38378 fp=#0x1000000592c' 2024-09-21T00:30:10.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38410 fp=#0x10000002bd1' 2024-09-21T00:30:10.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38442 fp=#0x10000002a3d' 2024-09-21T00:30:10.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38506 fp=#0x100000045fa' 2024-09-21T00:30:10.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38538 fp=#0x100000045c4' 2024-09-21T00:30:10.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38570 fp=#0x100000029ff' 2024-09-21T00:30:10.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38602 fp=#0x100000045ee' 2024-09-21T00:30:10.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38666 fp=#0x10000001fd5' 2024-09-21T00:30:10.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38698 fp=#0x100000010be' 2024-09-21T00:30:10.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38730 fp=#0x10000001fad' 2024-09-21T00:30:10.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38794 fp=#0x100000010af' 2024-09-21T00:30:10.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38826 fp=#0x10000001fe7' 2024-09-21T00:30:10.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38858 fp=#0x10000005e26' 2024-09-21T00:30:10.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38890 fp=#0x1000000409c' 2024-09-21T00:30:10.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39018 fp=#0x1000000416b' 2024-09-21T00:30:10.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39050 fp=#0x10000001fd7' 2024-09-21T00:30:10.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39082 fp=#0x100000017f0' 2024-09-21T00:30:10.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39114 fp=#0x10000001caf' 2024-09-21T00:30:10.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39146 fp=#0x10000001fd0' 2024-09-21T00:30:10.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39178 fp=#0x100000044c0' 2024-09-21T00:30:10.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39242 fp=#0x10000004506' 2024-09-21T00:30:10.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39274 fp=#0x100000048dc' 2024-09-21T00:30:10.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39306 fp=#0x100000048cc' 2024-09-21T00:30:10.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39338 fp=#0x1000000488b' 2024-09-21T00:30:10.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39370 fp=#0x100000048c4' 2024-09-21T00:30:10.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39402 fp=#0x100000059fe' 2024-09-21T00:30:10.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39434 fp=#0x100000059b7' 2024-09-21T00:30:10.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39466 fp=#0x10000005e08' 2024-09-21T00:30:10.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39498 fp=#0x100000059f8' 2024-09-21T00:30:10.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39530 fp=#0x10000005e07' 2024-09-21T00:30:10.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39562 fp=#0x100000050ec' 2024-09-21T00:30:10.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39594 fp=#0x100000050f7' 2024-09-21T00:30:10.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39658 fp=#0x100000050bc' 2024-09-21T00:30:10.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39690 fp=#0x10000005d86' 2024-09-21T00:30:10.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39754 fp=#0x10000005b5e' 2024-09-21T00:30:10.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39786 fp=#0x10000005b47' 2024-09-21T00:30:10.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39818 fp=#0x100000062e6' 2024-09-21T00:30:10.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39850 fp=#0x1000000637c' 2024-09-21T00:30:10.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39882 fp=#0x100000062dc' 2024-09-21T00:30:10.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39914 fp=#0x10000004edf' 2024-09-21T00:30:10.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39978 fp=#0x10000004e85' 2024-09-21T00:30:10.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40010 fp=#0x10000004ea7' 2024-09-21T00:30:10.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40042 fp=#0x100000061ff' 2024-09-21T00:30:10.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40074 fp=#0x1000000624e' 2024-09-21T00:30:10.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40106 fp=#0x100000061f1' 2024-09-21T00:30:10.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40138 fp=#0x10000002611' 2024-09-21T00:30:10.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40170 fp=#0x100000025fe' 2024-09-21T00:30:10.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40202 fp=#0x1000000269f' 2024-09-21T00:30:10.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40234 fp=#0x10000000878' 2024-09-21T00:30:10.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40298 fp=#0x100000042b7' 2024-09-21T00:30:10.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40330 fp=#0x100000009a9' 2024-09-21T00:30:10.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40362 fp=#0x100000008ec' 2024-09-21T00:30:10.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40394 fp=#0x100000042f0' 2024-09-21T00:30:10.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40426 fp=#0x100000042bb' 2024-09-21T00:30:10.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40586 fp=#0x100000036c4' 2024-09-21T00:30:10.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40682 fp=#0x10000004839' 2024-09-21T00:30:10.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40714 fp=#0x10000005ec5' 2024-09-21T00:30:10.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40746 fp=#0x1000000482b' 2024-09-21T00:30:10.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40778 fp=#0x10000005ea5' 2024-09-21T00:30:10.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40810 fp=#0x100000048fd' 2024-09-21T00:30:10.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40906 fp=#0x100000063ce' 2024-09-21T00:30:10.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40938 fp=#0x100000063d3' 2024-09-21T00:30:10.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40970 fp=#0x10000004a20' 2024-09-21T00:30:10.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41002 fp=#0x10000004a22' 2024-09-21T00:30:10.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41034 fp=#0x10000005139' 2024-09-21T00:30:10.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41066 fp=#0x10000005108' 2024-09-21T00:30:10.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41098 fp=#0x10000005125' 2024-09-21T00:30:10.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30795 fp=#0x100000031d8' 2024-09-21T00:30:10.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30827 fp=#0x10000001480' 2024-09-21T00:30:10.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32683 fp=#0x10000005691' 2024-09-21T00:30:10.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32747 fp=#0x10000005765' 2024-09-21T00:30:10.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32779 fp=#0x10000005749' 2024-09-21T00:30:10.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32811 fp=#0x10000005039' 2024-09-21T00:30:10.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32843 fp=#0x10000005045' 2024-09-21T00:30:10.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32875 fp=#0x1000000502b' 2024-09-21T00:30:10.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32907 fp=#0x1000000507c' 2024-09-21T00:30:10.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32939 fp=#0x10000005f36' 2024-09-21T00:30:10.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33003 fp=#0x10000005cab' 2024-09-21T00:30:10.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33035 fp=#0x10000005c7a' 2024-09-21T00:30:10.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33067 fp=#0x10000005c7b' 2024-09-21T00:30:10.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33099 fp=#0x10000005cb0' 2024-09-21T00:30:10.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33131 fp=#0x10000005c87' 2024-09-21T00:30:10.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33163 fp=#0x10000006072' 2024-09-21T00:30:10.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33195 fp=#0x100000067b1' 2024-09-21T00:30:10.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33259 fp=#0x10000006075' 2024-09-21T00:30:10.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33291 fp=#0x1000000605e' 2024-09-21T00:30:10.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33323 fp=#0x10000006800' 2024-09-21T00:30:10.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33355 fp=#0x100000067e4' 2024-09-21T00:30:10.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33387 fp=#0x10000003ea3' 2024-09-21T00:30:10.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33419 fp=#0x10000003d6f' 2024-09-21T00:30:10.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33451 fp=#0x10000003d92' 2024-09-21T00:30:10.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33483 fp=#0x10000003e0b' 2024-09-21T00:30:10.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33515 fp=#0x100000047fb' 2024-09-21T00:30:10.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33579 fp=#0x10000005a82' 2024-09-21T00:30:10.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33643 fp=#0x100000059eb' 2024-09-21T00:30:10.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33675 fp=#0x10000005a0a' 2024-09-21T00:30:10.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33739 fp=#0x10000006799' 2024-09-21T00:30:10.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33771 fp=#0x1000000675b' 2024-09-21T00:30:10.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33803 fp=#0x1000000679e' 2024-09-21T00:30:10.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33931 fp=#0x10000004775' 2024-09-21T00:30:10.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33963 fp=#0x10000006413' 2024-09-21T00:30:10.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33995 fp=#0x1000000644d' 2024-09-21T00:30:10.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34027 fp=#0x1000000471d' 2024-09-21T00:30:10.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34059 fp=#0x10000006450' 2024-09-21T00:30:10.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34123 fp=#0x10000006220' 2024-09-21T00:30:10.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34155 fp=#0x1000000623e' 2024-09-21T00:30:10.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34187 fp=#0x10000006292' 2024-09-21T00:30:10.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34219 fp=#0x10000006484' 2024-09-21T00:30:10.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34251 fp=#0x10000006208' 2024-09-21T00:30:10.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34283 fp=#0x10000006200' 2024-09-21T00:30:10.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34315 fp=#0x100000026c8' 2024-09-21T00:30:10.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34347 fp=#0x10000002669' 2024-09-21T00:30:10.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34379 fp=#0x100000025a2' 2024-09-21T00:30:10.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34443 fp=#0x1000000250d' 2024-09-21T00:30:10.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34507 fp=#0x1000000568c' 2024-09-21T00:30:10.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34539 fp=#0x100000061a5' 2024-09-21T00:30:10.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34571 fp=#0x1000000633e' 2024-09-21T00:30:10.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34603 fp=#0x100000062c6' 2024-09-21T00:30:10.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34635 fp=#0x1000000631d' 2024-09-21T00:30:10.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34667 fp=#0x10000006383' 2024-09-21T00:30:10.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34763 fp=#0x100000064f0' 2024-09-21T00:30:10.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34795 fp=#0x100000064ba' 2024-09-21T00:30:10.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34827 fp=#0x100000064cd' 2024-09-21T00:30:10.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34859 fp=#0x100000057c8' 2024-09-21T00:30:10.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34891 fp=#0x100000064d8' 2024-09-21T00:30:10.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34923 fp=#0x1000000530a' 2024-09-21T00:30:10.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34955 fp=#0x10000006957' 2024-09-21T00:30:10.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34987 fp=#0x100000052a4' 2024-09-21T00:30:10.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35019 fp=#0x100000052f5' 2024-09-21T00:30:10.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35147 fp=#0x100000060ea' 2024-09-21T00:30:10.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35179 fp=#0x1000000609e' 2024-09-21T00:30:10.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35211 fp=#0x100000060d4' 2024-09-21T00:30:10.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35243 fp=#0x10000003f28' 2024-09-21T00:30:10.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35275 fp=#0x10000003f00' 2024-09-21T00:30:10.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35307 fp=#0x10000003f11' 2024-09-21T00:30:10.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35371 fp=#0x1000000532c' 2024-09-21T00:30:10.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35499 fp=#0x10000005f91' 2024-09-21T00:30:10.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35531 fp=#0x10000005f75' 2024-09-21T00:30:10.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35627 fp=#0x10000005bbf' 2024-09-21T00:30:10.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35659 fp=#0x10000005b70' 2024-09-21T00:30:10.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35691 fp=#0x10000004931' 2024-09-21T00:30:10.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35723 fp=#0x10000004928' 2024-09-21T00:30:10.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35755 fp=#0x10000004f81' 2024-09-21T00:30:10.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35787 fp=#0x10000004ff0' 2024-09-21T00:30:10.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35819 fp=#0x10000004fb2' 2024-09-21T00:30:10.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35851 fp=#0x10000005002' 2024-09-21T00:30:10.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35883 fp=#0x1000000671b' 2024-09-21T00:30:10.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35915 fp=#0x10000006725' 2024-09-21T00:30:10.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35947 fp=#0x10000006845' 2024-09-21T00:30:10.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35979 fp=#0x1000000685a' 2024-09-21T00:30:10.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36011 fp=#0x10000006848' 2024-09-21T00:30:10.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36171 fp=#0x10000005f55' 2024-09-21T00:30:10.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36203 fp=#0x100000068d7' 2024-09-21T00:30:10.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36235 fp=#0x100000068d2' 2024-09-21T00:30:10.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36267 fp=#0x100000065fc' 2024-09-21T00:30:10.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36331 fp=#0x100000065f1' 2024-09-21T00:30:10.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36363 fp=#0x100000065f0' 2024-09-21T00:30:10.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36427 fp=#0x10000004014' 2024-09-21T00:30:10.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36523 fp=#0x10000006041' 2024-09-21T00:30:10.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36555 fp=#0x10000005abd' 2024-09-21T00:30:10.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36587 fp=#0x10000006627' 2024-09-21T00:30:10.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36619 fp=#0x10000005a14' 2024-09-21T00:30:10.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36683 fp=#0x100000047ee' 2024-09-21T00:30:10.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36715 fp=#0x100000047c2' 2024-09-21T00:30:10.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36747 fp=#0x1000000480c' 2024-09-21T00:30:10.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36779 fp=#0x10000004777' 2024-09-21T00:30:10.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36907 fp=#0x10000005890' 2024-09-21T00:30:10.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36939 fp=#0x100000058a2' 2024-09-21T00:30:10.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37003 fp=#0x10000005227' 2024-09-21T00:30:10.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37035 fp=#0x100000051a2' 2024-09-21T00:30:10.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37067 fp=#0x10000005270' 2024-09-21T00:30:10.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37099 fp=#0x10000005254' 2024-09-21T00:30:10.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37195 fp=#0x10000005fe8' 2024-09-21T00:30:10.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37259 fp=#0x10000005c3d' 2024-09-21T00:30:10.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37323 fp=#0x10000005c41' 2024-09-21T00:30:10.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37355 fp=#0x10000005c32' 2024-09-21T00:30:10.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37387 fp=#0x10000005f0a' 2024-09-21T00:30:10.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37451 fp=#0x10000004baf' 2024-09-21T00:30:10.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37483 fp=#0x10000004b50' 2024-09-21T00:30:10.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37515 fp=#0x10000004ad2' 2024-09-21T00:30:10.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37547 fp=#0x10000004b0b' 2024-09-21T00:30:10.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37579 fp=#0x10000005540' 2024-09-21T00:30:10.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37611 fp=#0x10000005530' 2024-09-21T00:30:10.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37643 fp=#0x100000054cb' 2024-09-21T00:30:10.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37675 fp=#0x100000054ec' 2024-09-21T00:30:10.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37707 fp=#0x100000042ae' 2024-09-21T00:30:10.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37739 fp=#0x100000042f9' 2024-09-21T00:30:10.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37771 fp=#0x100000042aa' 2024-09-21T00:30:10.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37803 fp=#0x100000042c0' 2024-09-21T00:30:10.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37835 fp=#0x10000005800' 2024-09-21T00:30:10.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37867 fp=#0x1000000423a' 2024-09-21T00:30:10.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37899 fp=#0x10000001d6d' 2024-09-21T00:30:10.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37931 fp=#0x10000003d0c' 2024-09-21T00:30:10.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37963 fp=#0x10000001ba5' 2024-09-21T00:30:10.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37995 fp=#0x10000001cec' 2024-09-21T00:30:10.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38027 fp=#0x10000003cae' 2024-09-21T00:30:10.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38059 fp=#0x10000003c8c' 2024-09-21T00:30:10.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38155 fp=#0x10000003aac' 2024-09-21T00:30:10.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38219 fp=#0x10000005880' 2024-09-21T00:30:10.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38283 fp=#0x1000000618b' 2024-09-21T00:30:10.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38347 fp=#0x10000006173' 2024-09-21T00:30:10.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38379 fp=#0x1000000588e' 2024-09-21T00:30:10.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38411 fp=#0x10000002b68' 2024-09-21T00:30:10.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38443 fp=#0x100000045e4' 2024-09-21T00:30:10.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38475 fp=#0x10000003374' 2024-09-21T00:30:10.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38507 fp=#0x100000029e6' 2024-09-21T00:30:10.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38539 fp=#0x10000002a41' 2024-09-21T00:30:10.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38571 fp=#0x100000045d2' 2024-09-21T00:30:10.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38603 fp=#0x10000002a1b' 2024-09-21T00:30:10.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38635 fp=#0x100000048b1' 2024-09-21T00:30:10.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38667 fp=#0x10000004876' 2024-09-21T00:30:10.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38699 fp=#0x1000000107f' 2024-09-21T00:30:10.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38731 fp=#0x10000004891' 2024-09-21T00:30:10.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38795 fp=#0x100000048bc' 2024-09-21T00:30:10.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38859 fp=#0x100000041d2' 2024-09-21T00:30:10.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38891 fp=#0x1000000418e' 2024-09-21T00:30:10.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38923 fp=#0x10000004157' 2024-09-21T00:30:10.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38987 fp=#0x1000000409a' 2024-09-21T00:30:10.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39019 fp=#0x10000004003' 2024-09-21T00:30:10.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39051 fp=#0x10000001fb0' 2024-09-21T00:30:10.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39083 fp=#0x100000017f7' 2024-09-21T00:30:10.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39115 fp=#0x10000001d66' 2024-09-21T00:30:10.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39147 fp=#0x10000001fab' 2024-09-21T00:30:10.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39179 fp=#0x1000000450c' 2024-09-21T00:30:10.204 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39275 fp=#0x10000004809' 2024-09-21T00:30:10.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39307 fp=#0x10000004841' 2024-09-21T00:30:10.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39339 fp=#0x1000000487c' 2024-09-21T00:30:10.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39371 fp=#0x10000005ae4' 2024-09-21T00:30:10.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39403 fp=#0x10000005e1b' 2024-09-21T00:30:10.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39435 fp=#0x10000005a45' 2024-09-21T00:30:10.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39467 fp=#0x10000005e38' 2024-09-21T00:30:10.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39499 fp=#0x10000005ae2' 2024-09-21T00:30:10.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39531 fp=#0x100000059c8' 2024-09-21T00:30:10.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39563 fp=#0x100000050a2' 2024-09-21T00:30:10.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39595 fp=#0x100000050e5' 2024-09-21T00:30:10.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39659 fp=#0x100000050dd' 2024-09-21T00:30:10.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39755 fp=#0x10000005b5a' 2024-09-21T00:30:10.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39787 fp=#0x10000005b57' 2024-09-21T00:30:10.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39819 fp=#0x10000006381' 2024-09-21T00:30:10.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39851 fp=#0x1000000631c' 2024-09-21T00:30:10.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39883 fp=#0x1000000630f' 2024-09-21T00:30:10.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39915 fp=#0x10000004ef3' 2024-09-21T00:30:10.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39979 fp=#0x10000004eb4' 2024-09-21T00:30:10.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40043 fp=#0x10000006205' 2024-09-21T00:30:10.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40075 fp=#0x10000006224' 2024-09-21T00:30:10.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40107 fp=#0x10000006291' 2024-09-21T00:30:10.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40139 fp=#0x10000002690' 2024-09-21T00:30:10.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40171 fp=#0x10000002628' 2024-09-21T00:30:10.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40203 fp=#0x10000002693' 2024-09-21T00:30:10.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40267 fp=#0x100000008f5' 2024-09-21T00:30:10.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40299 fp=#0x100000009b3' 2024-09-21T00:30:10.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40331 fp=#0x10000004271' 2024-09-21T00:30:10.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40363 fp=#0x100000042c7' 2024-09-21T00:30:10.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40395 fp=#0x10000004259' 2024-09-21T00:30:10.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40427 fp=#0x100000008c2' 2024-09-21T00:30:10.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40555 fp=#0x10000002888' 2024-09-21T00:30:10.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40683 fp=#0x10000005eab' 2024-09-21T00:30:10.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40747 fp=#0x100000048e5' 2024-09-21T00:30:10.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40779 fp=#0x10000005eac' 2024-09-21T00:30:10.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40875 fp=#0x100000031cd' 2024-09-21T00:30:10.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40939 fp=#0x100000063cd' 2024-09-21T00:30:10.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40971 fp=#0x100000049b7' 2024-09-21T00:30:10.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41003 fp=#0x100000049e2' 2024-09-21T00:30:10.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41035 fp=#0x10000005136' 2024-09-21T00:30:10.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41067 fp=#0x10000005144' 2024-09-21T00:30:10.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41099 fp=#0x100000049e9' 2024-09-21T00:30:10.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30796 fp=#0x100000061ef' 2024-09-21T00:30:10.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30828 fp=#0x10000005b1e' 2024-09-21T00:30:10.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32748 fp=#0x1000000573d' 2024-09-21T00:30:10.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32780 fp=#0x10000005780' 2024-09-21T00:30:10.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32812 fp=#0x1000000508d' 2024-09-21T00:30:10.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32876 fp=#0x10000005030' 2024-09-21T00:30:10.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32908 fp=#0x10000005057' 2024-09-21T00:30:10.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32940 fp=#0x10000005075' 2024-09-21T00:30:10.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33004 fp=#0x10000005d09' 2024-09-21T00:30:10.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33036 fp=#0x10000005c8b' 2024-09-21T00:30:10.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33068 fp=#0x10000005c49' 2024-09-21T00:30:10.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33100 fp=#0x10000005c72' 2024-09-21T00:30:10.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33164 fp=#0x1000000666f' 2024-09-21T00:30:10.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33196 fp=#0x10000005d43' 2024-09-21T00:30:10.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33292 fp=#0x1000000606a' 2024-09-21T00:30:10.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33324 fp=#0x100000067cc' 2024-09-21T00:30:10.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33356 fp=#0x1000000680b' 2024-09-21T00:30:10.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33388 fp=#0x10000003e86' 2024-09-21T00:30:10.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33420 fp=#0x10000003e89' 2024-09-21T00:30:10.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33452 fp=#0x10000003da1' 2024-09-21T00:30:10.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33484 fp=#0x1000000514f' 2024-09-21T00:30:10.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33516 fp=#0x100000047ef' 2024-09-21T00:30:10.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33548 fp=#0x10000006630' 2024-09-21T00:30:10.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33580 fp=#0x10000005a7f' 2024-09-21T00:30:10.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33708 fp=#0x10000005a37' 2024-09-21T00:30:10.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33740 fp=#0x100000067b2' 2024-09-21T00:30:10.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33772 fp=#0x100000067ab' 2024-09-21T00:30:10.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33804 fp=#0x100000067b3' 2024-09-21T00:30:10.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33932 fp=#0x10000004711' 2024-09-21T00:30:10.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33964 fp=#0x1000000640b' 2024-09-21T00:30:10.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33996 fp=#0x100000063fe' 2024-09-21T00:30:10.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34028 fp=#0x10000006434' 2024-09-21T00:30:10.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34060 fp=#0x10000006441' 2024-09-21T00:30:10.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34092 fp=#0x100000063ee' 2024-09-21T00:30:10.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34124 fp=#0x100000062a1' 2024-09-21T00:30:10.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34156 fp=#0x100000062b1' 2024-09-21T00:30:10.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34188 fp=#0x100000064a0' 2024-09-21T00:30:10.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34220 fp=#0x10000006460' 2024-09-21T00:30:10.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34252 fp=#0x10000006252' 2024-09-21T00:30:10.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34284 fp=#0x100000064a2' 2024-09-21T00:30:10.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34316 fp=#0x100000061cd' 2024-09-21T00:30:10.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34380 fp=#0x100000025af' 2024-09-21T00:30:10.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34444 fp=#0x100000061d0' 2024-09-21T00:30:10.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34508 fp=#0x10000002674' 2024-09-21T00:30:10.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34540 fp=#0x10000006302' 2024-09-21T00:30:10.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34572 fp=#0x10000006395' 2024-09-21T00:30:10.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34604 fp=#0x10000006394' 2024-09-21T00:30:10.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34636 fp=#0x100000062e9' 2024-09-21T00:30:10.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34668 fp=#0x100000062f7' 2024-09-21T00:30:10.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34764 fp=#0x100000064dd' 2024-09-21T00:30:10.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34796 fp=#0x10000006500' 2024-09-21T00:30:10.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34828 fp=#0x100000064fe' 2024-09-21T00:30:10.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34860 fp=#0x100000064e3' 2024-09-21T00:30:10.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34892 fp=#0x100000064c1' 2024-09-21T00:30:10.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34924 fp=#0x100000052c4' 2024-09-21T00:30:10.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34956 fp=#0x100000052ac' 2024-09-21T00:30:10.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34988 fp=#0x100000052f0' 2024-09-21T00:30:10.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35020 fp=#0x1000000531c' 2024-09-21T00:30:10.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35052 fp=#0x10000002d1c' 2024-09-21T00:30:10.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35084 fp=#0x10000002c15' 2024-09-21T00:30:10.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35148 fp=#0x10000002d1e' 2024-09-21T00:30:10.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35212 fp=#0x100000060ac' 2024-09-21T00:30:10.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35244 fp=#0x10000003fb5' 2024-09-21T00:30:10.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35276 fp=#0x10000003f1c' 2024-09-21T00:30:10.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35340 fp=#0x10000005395' 2024-09-21T00:30:10.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35468 fp=#0x10000005f7f' 2024-09-21T00:30:10.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35500 fp=#0x10000005f98' 2024-09-21T00:30:10.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35628 fp=#0x10000005bab' 2024-09-21T00:30:10.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35660 fp=#0x10000005bdd' 2024-09-21T00:30:10.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35692 fp=#0x1000000490e' 2024-09-21T00:30:10.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35724 fp=#0x1000000493a' 2024-09-21T00:30:10.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35756 fp=#0x10000004ff2' 2024-09-21T00:30:10.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35788 fp=#0x10000004f43' 2024-09-21T00:30:10.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35820 fp=#0x10000004f56' 2024-09-21T00:30:10.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35852 fp=#0x10000004f5e' 2024-09-21T00:30:10.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35884 fp=#0x10000006716' 2024-09-21T00:30:10.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35916 fp=#0x1000000672b' 2024-09-21T00:30:10.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35948 fp=#0x1000000685d' 2024-09-21T00:30:10.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35980 fp=#0x1000000686b' 2024-09-21T00:30:10.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36012 fp=#0x10000006874' 2024-09-21T00:30:10.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36140 fp=#0x10000005cc8' 2024-09-21T00:30:10.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36204 fp=#0x100000068a0' 2024-09-21T00:30:10.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36236 fp=#0x100000068cd' 2024-09-21T00:30:10.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36268 fp=#0x1000000661d' 2024-09-21T00:30:10.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36300 fp=#0x100000065f8' 2024-09-21T00:30:10.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36332 fp=#0x1000000661c' 2024-09-21T00:30:10.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36364 fp=#0x100000065b5' 2024-09-21T00:30:10.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36492 fp=#0x1000000603b' 2024-09-21T00:30:10.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36588 fp=#0x10000005a33' 2024-09-21T00:30:10.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36620 fp=#0x10000005ac2' 2024-09-21T00:30:10.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36652 fp=#0x100000047a0' 2024-09-21T00:30:10.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36684 fp=#0x100000048c6' 2024-09-21T00:30:10.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36716 fp=#0x1000000663a' 2024-09-21T00:30:10.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36748 fp=#0x100000047ab' 2024-09-21T00:30:10.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36780 fp=#0x100000047ca' 2024-09-21T00:30:10.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36940 fp=#0x10000005915' 2024-09-21T00:30:10.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36972 fp=#0x10000005274' 2024-09-21T00:30:10.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37004 fp=#0x100000051f4' 2024-09-21T00:30:10.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37068 fp=#0x10000005228' 2024-09-21T00:30:10.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37100 fp=#0x1000000522e' 2024-09-21T00:30:10.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37132 fp=#0x1000000566e' 2024-09-21T00:30:10.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37196 fp=#0x10000003cf6' 2024-09-21T00:30:10.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37228 fp=#0x10000005c51' 2024-09-21T00:30:10.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37260 fp=#0x10000005c2a' 2024-09-21T00:30:10.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37292 fp=#0x10000005b9b' 2024-09-21T00:30:10.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37324 fp=#0x10000005be0' 2024-09-21T00:30:10.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37356 fp=#0x10000005bb7' 2024-09-21T00:30:10.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37388 fp=#0x10000005eee' 2024-09-21T00:30:10.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37420 fp=#0x10000005eec' 2024-09-21T00:30:10.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37452 fp=#0x10000004a77' 2024-09-21T00:30:10.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37484 fp=#0x10000004a5a' 2024-09-21T00:30:10.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37516 fp=#0x10000004a7a' 2024-09-21T00:30:10.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37548 fp=#0x10000004b8c' 2024-09-21T00:30:10.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37580 fp=#0x100000054c3' 2024-09-21T00:30:10.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37612 fp=#0x10000005496' 2024-09-21T00:30:10.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37644 fp=#0x10000005524' 2024-09-21T00:30:10.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37676 fp=#0x100000054f6' 2024-09-21T00:30:10.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37708 fp=#0x10000004314' 2024-09-21T00:30:10.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37740 fp=#0x100000042dc' 2024-09-21T00:30:10.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37772 fp=#0x10000004320' 2024-09-21T00:30:10.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37804 fp=#0x10000005816' 2024-09-21T00:30:10.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37836 fp=#0x1000000583b' 2024-09-21T00:30:10.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37868 fp=#0x10000005830' 2024-09-21T00:30:10.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37900 fp=#0x10000001c8d' 2024-09-21T00:30:10.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37932 fp=#0x10000001c4b' 2024-09-21T00:30:10.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37964 fp=#0x10000001d18' 2024-09-21T00:30:10.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37996 fp=#0x10000001cd3' 2024-09-21T00:30:10.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38028 fp=#0x10000003c31' 2024-09-21T00:30:10.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38060 fp=#0x10000003c4f' 2024-09-21T00:30:10.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38092 fp=#0x10000003b3d' 2024-09-21T00:30:10.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38188 fp=#0x10000005877' 2024-09-21T00:30:10.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38252 fp=#0x100000059a4' 2024-09-21T00:30:10.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38284 fp=#0x10000006183' 2024-09-21T00:30:10.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38316 fp=#0x10000005858' 2024-09-21T00:30:10.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38348 fp=#0x10000005841' 2024-09-21T00:30:10.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38380 fp=#0x10000006174' 2024-09-21T00:30:10.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38412 fp=#0x10000002bd5' 2024-09-21T00:30:10.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38444 fp=#0x100000045ef' 2024-09-21T00:30:10.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38476 fp=#0x10000002a30' 2024-09-21T00:30:10.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38508 fp=#0x100000029f3' 2024-09-21T00:30:10.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38540 fp=#0x100000045e0' 2024-09-21T00:30:10.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38572 fp=#0x10000002a74' 2024-09-21T00:30:10.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38604 fp=#0x10000002b82' 2024-09-21T00:30:10.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38700 fp=#0x10000001fcc' 2024-09-21T00:30:10.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38732 fp=#0x10000000f2e' 2024-09-21T00:30:10.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38828 fp=#0x10000000e19' 2024-09-21T00:30:10.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38860 fp=#0x10000004025' 2024-09-21T00:30:10.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38892 fp=#0x10000004138' 2024-09-21T00:30:10.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38988 fp=#0x10000004073' 2024-09-21T00:30:10.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39020 fp=#0x10000004056' 2024-09-21T00:30:10.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39052 fp=#0x10000001d6a' 2024-09-21T00:30:10.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39084 fp=#0x10000001fcb' 2024-09-21T00:30:10.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39116 fp=#0x10000001c78' 2024-09-21T00:30:10.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39148 fp=#0x10000001f92' 2024-09-21T00:30:10.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39180 fp=#0x100000044aa' 2024-09-21T00:30:10.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39212 fp=#0x100000044a3' 2024-09-21T00:30:10.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39244 fp=#0x100000044c5' 2024-09-21T00:30:10.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39276 fp=#0x10000004854' 2024-09-21T00:30:10.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39308 fp=#0x1000000486b' 2024-09-21T00:30:10.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39340 fp=#0x10000004857' 2024-09-21T00:30:10.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39372 fp=#0x10000005e05' 2024-09-21T00:30:10.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39404 fp=#0x10000005e24' 2024-09-21T00:30:10.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39436 fp=#0x10000005a51' 2024-09-21T00:30:10.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39532 fp=#0x10000005abe' 2024-09-21T00:30:10.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39564 fp=#0x100000050fb' 2024-09-21T00:30:10.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39596 fp=#0x1000000509d' 2024-09-21T00:30:10.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39628 fp=#0x100000050e9' 2024-09-21T00:30:10.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39724 fp=#0x10000005d78' 2024-09-21T00:30:10.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39788 fp=#0x10000005b31' 2024-09-21T00:30:10.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39820 fp=#0x1000000637e' 2024-09-21T00:30:10.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39852 fp=#0x100000062e0' 2024-09-21T00:30:10.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39884 fp=#0x10000006330' 2024-09-21T00:30:10.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39916 fp=#0x10000004eb5' 2024-09-21T00:30:10.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39948 fp=#0x10000004ed6' 2024-09-21T00:30:10.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39980 fp=#0x10000004e9f' 2024-09-21T00:30:10.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40012 fp=#0x10000004ee4' 2024-09-21T00:30:10.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40044 fp=#0x10000006257' 2024-09-21T00:30:10.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40076 fp=#0x100000062bc' 2024-09-21T00:30:10.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40108 fp=#0x10000006243' 2024-09-21T00:30:10.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40140 fp=#0x10000002623' 2024-09-21T00:30:10.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40172 fp=#0x10000002637' 2024-09-21T00:30:10.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40204 fp=#0x10000002655' 2024-09-21T00:30:10.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40236 fp=#0x10000000b12' 2024-09-21T00:30:10.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40268 fp=#0x10000000a25' 2024-09-21T00:30:10.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40300 fp=#0x10000000a7a' 2024-09-21T00:30:10.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40332 fp=#0x10000000af9' 2024-09-21T00:30:10.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40364 fp=#0x100000008dd' 2024-09-21T00:30:10.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40396 fp=#0x10000000a52' 2024-09-21T00:30:10.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40428 fp=#0x100000008d9' 2024-09-21T00:30:10.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40460 fp=#0x10000005977' 2024-09-21T00:30:10.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40556 fp=#0x100000028a7' 2024-09-21T00:30:10.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40684 fp=#0x10000005eb4' 2024-09-21T00:30:10.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40716 fp=#0x10000005f1d' 2024-09-21T00:30:10.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40748 fp=#0x1000000483f' 2024-09-21T00:30:10.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40780 fp=#0x10000005ea6' 2024-09-21T00:30:10.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40844 fp=#0x1000000305d' 2024-09-21T00:30:10.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40876 fp=#0x100000063d9' 2024-09-21T00:30:10.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40908 fp=#0x100000063ed' 2024-09-21T00:30:10.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40972 fp=#0x10000005147' 2024-09-21T00:30:10.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41004 fp=#0x10000004a0c' 2024-09-21T00:30:10.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41036 fp=#0x1000000514a' 2024-09-21T00:30:10.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41068 fp=#0x10000004e87' 2024-09-21T00:30:10.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41100 fp=#0x100000049b3' 2024-09-21T00:30:10.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30765 fp=#0x100000053bf' 2024-09-21T00:30:10.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30797 fp=#0x10000003a3b' 2024-09-21T00:30:10.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30829 fp=#0x100000062ce' 2024-09-21T00:30:10.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32685 fp=#0x1000000576f' 2024-09-21T00:30:10.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32749 fp=#0x1000000571e' 2024-09-21T00:30:10.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32781 fp=#0x10000006766' 2024-09-21T00:30:10.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32909 fp=#0x10000005066' 2024-09-21T00:30:10.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32941 fp=#0x10000005056' 2024-09-21T00:30:10.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33005 fp=#0x10000005d36' 2024-09-21T00:30:10.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33037 fp=#0x10000005c5d' 2024-09-21T00:30:10.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33069 fp=#0x10000005cac' 2024-09-21T00:30:10.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33101 fp=#0x10000005c33' 2024-09-21T00:30:10.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33133 fp=#0x10000005cd3' 2024-09-21T00:30:10.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33197 fp=#0x10000006685' 2024-09-21T00:30:10.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33261 fp=#0x1000000603a' 2024-09-21T00:30:10.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33293 fp=#0x10000006048' 2024-09-21T00:30:10.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33325 fp=#0x1000000681c' 2024-09-21T00:30:10.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33357 fp=#0x100000067e7' 2024-09-21T00:30:10.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33389 fp=#0x10000003d4a' 2024-09-21T00:30:10.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33421 fp=#0x10000003e53' 2024-09-21T00:30:10.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33485 fp=#0x10000003dd5' 2024-09-21T00:30:10.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33517 fp=#0x100000047fd' 2024-09-21T00:30:10.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33549 fp=#0x10000005664' 2024-09-21T00:30:10.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33613 fp=#0x10000005a2e' 2024-09-21T00:30:10.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33677 fp=#0x10000005aba' 2024-09-21T00:30:10.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33709 fp=#0x10000005a48' 2024-09-21T00:30:10.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33741 fp=#0x10000006793' 2024-09-21T00:30:10.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33773 fp=#0x10000006785' 2024-09-21T00:30:10.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33805 fp=#0x1000000677c' 2024-09-21T00:30:10.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33933 fp=#0x100000047da' 2024-09-21T00:30:10.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33965 fp=#0x1000000644f' 2024-09-21T00:30:10.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33997 fp=#0x100000047b0' 2024-09-21T00:30:10.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34029 fp=#0x10000005f15' 2024-09-21T00:30:10.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34061 fp=#0x1000000478d' 2024-09-21T00:30:10.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34093 fp=#0x10000003264' 2024-09-21T00:30:10.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34125 fp=#0x10000006461' 2024-09-21T00:30:10.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34157 fp=#0x10000006213' 2024-09-21T00:30:10.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34189 fp=#0x10000006498' 2024-09-21T00:30:10.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34221 fp=#0x100000061f7' 2024-09-21T00:30:10.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34253 fp=#0x10000006459' 2024-09-21T00:30:10.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34285 fp=#0x10000006468' 2024-09-21T00:30:10.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34317 fp=#0x100000061c4' 2024-09-21T00:30:10.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34349 fp=#0x10000002671' 2024-09-21T00:30:10.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34381 fp=#0x100000061ca' 2024-09-21T00:30:10.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34413 fp=#0x10000005675' 2024-09-21T00:30:10.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34445 fp=#0x100000056b5' 2024-09-21T00:30:10.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34477 fp=#0x10000002526' 2024-09-21T00:30:10.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34509 fp=#0x100000061cb' 2024-09-21T00:30:10.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34541 fp=#0x100000063a5' 2024-09-21T00:30:10.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34573 fp=#0x100000062cc' 2024-09-21T00:30:10.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34605 fp=#0x10000006308' 2024-09-21T00:30:10.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34637 fp=#0x1000000639b' 2024-09-21T00:30:10.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34669 fp=#0x100000062c9' 2024-09-21T00:30:10.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34765 fp=#0x100000064bb' 2024-09-21T00:30:10.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34797 fp=#0x100000057bc' 2024-09-21T00:30:10.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34829 fp=#0x100000057e9' 2024-09-21T00:30:10.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34861 fp=#0x100000057cb' 2024-09-21T00:30:10.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34893 fp=#0x100000057d7' 2024-09-21T00:30:10.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34925 fp=#0x10000005349' 2024-09-21T00:30:10.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34957 fp=#0x10000006952' 2024-09-21T00:30:10.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35117 fp=#0x100000060e1' 2024-09-21T00:30:10.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35149 fp=#0x100000060b6' 2024-09-21T00:30:10.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35213 fp=#0x100000060b5' 2024-09-21T00:30:10.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35245 fp=#0x10000003f0c' 2024-09-21T00:30:10.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35277 fp=#0x10000003f05' 2024-09-21T00:30:10.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35309 fp=#0x10000003fb1' 2024-09-21T00:30:10.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35341 fp=#0x10000005393' 2024-09-21T00:30:10.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35629 fp=#0x10000005b90' 2024-09-21T00:30:10.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35661 fp=#0x10000005b8b' 2024-09-21T00:30:10.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35693 fp=#0x10000004972' 2024-09-21T00:30:10.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35725 fp=#0x10000004f3d' 2024-09-21T00:30:10.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35757 fp=#0x10000006581' 2024-09-21T00:30:10.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35789 fp=#0x10000004f5c' 2024-09-21T00:30:10.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35821 fp=#0x10000006623' 2024-09-21T00:30:10.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35853 fp=#0x10000004fde' 2024-09-21T00:30:10.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35885 fp=#0x1000000674f' 2024-09-21T00:30:10.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35917 fp=#0x10000006700' 2024-09-21T00:30:10.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35949 fp=#0x1000000688a' 2024-09-21T00:30:10.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35981 fp=#0x10000006846' 2024-09-21T00:30:10.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36013 fp=#0x1000000685f' 2024-09-21T00:30:10.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36045 fp=#0x10000005d67' 2024-09-21T00:30:10.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36173 fp=#0x10000005f5b' 2024-09-21T00:30:10.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36205 fp=#0x100000068d5' 2024-09-21T00:30:10.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36237 fp=#0x100000068a1' 2024-09-21T00:30:10.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36301 fp=#0x10000006606' 2024-09-21T00:30:10.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36365 fp=#0x10000006613' 2024-09-21T00:30:10.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36397 fp=#0x1000000660f' 2024-09-21T00:30:10.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36429 fp=#0x10000006141' 2024-09-21T00:30:10.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36461 fp=#0x10000006166' 2024-09-21T00:30:10.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36493 fp=#0x1000000600f' 2024-09-21T00:30:10.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36525 fp=#0x10000006008' 2024-09-21T00:30:10.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36653 fp=#0x10000004910' 2024-09-21T00:30:10.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36685 fp=#0x10000004760' 2024-09-21T00:30:10.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36717 fp=#0x10000004926' 2024-09-21T00:30:10.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36749 fp=#0x100000047e9' 2024-09-21T00:30:10.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36781 fp=#0x100000047d1' 2024-09-21T00:30:10.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36845 fp=#0x10000005af7' 2024-09-21T00:30:10.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36877 fp=#0x100000058f0' 2024-09-21T00:30:10.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36909 fp=#0x10000005964' 2024-09-21T00:30:10.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36941 fp=#0x10000005a1d' 2024-09-21T00:30:10.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36973 fp=#0x1000000526c' 2024-09-21T00:30:10.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37005 fp=#0x10000005246' 2024-09-21T00:30:10.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37069 fp=#0x1000000523a' 2024-09-21T00:30:10.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37197 fp=#0x10000003d05' 2024-09-21T00:30:10.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37229 fp=#0x10000005c48' 2024-09-21T00:30:10.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37261 fp=#0x10000005c12' 2024-09-21T00:30:10.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37293 fp=#0x10000005bc8' 2024-09-21T00:30:10.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37325 fp=#0x10000005bd8' 2024-09-21T00:30:10.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37357 fp=#0x10000005bbc' 2024-09-21T00:30:10.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37389 fp=#0x10000005f03' 2024-09-21T00:30:10.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37421 fp=#0x10000005efc' 2024-09-21T00:30:10.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37453 fp=#0x10000004aa5' 2024-09-21T00:30:10.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37485 fp=#0x10000004bbb' 2024-09-21T00:30:10.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37517 fp=#0x10000004b87' 2024-09-21T00:30:10.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37549 fp=#0x10000004b7f' 2024-09-21T00:30:10.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37581 fp=#0x1000000550d' 2024-09-21T00:30:10.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37613 fp=#0x10000005487' 2024-09-21T00:30:10.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37645 fp=#0x1000000547c' 2024-09-21T00:30:10.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37677 fp=#0x10000005528' 2024-09-21T00:30:10.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37709 fp=#0x10000005831' 2024-09-21T00:30:10.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37741 fp=#0x1000000580d' 2024-09-21T00:30:10.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37773 fp=#0x10000005821' 2024-09-21T00:30:10.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37805 fp=#0x100000057f8' 2024-09-21T00:30:10.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37837 fp=#0x10000004299' 2024-09-21T00:30:10.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37869 fp=#0x1000000580b' 2024-09-21T00:30:10.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37901 fp=#0x10000001d69' 2024-09-21T00:30:10.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37933 fp=#0x10000001ba1' 2024-09-21T00:30:10.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37965 fp=#0x10000001c48' 2024-09-21T00:30:10.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37997 fp=#0x10000001c38' 2024-09-21T00:30:10.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38029 fp=#0x10000003c8f' 2024-09-21T00:30:10.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38061 fp=#0x10000003c93' 2024-09-21T00:30:10.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38189 fp=#0x1000000618d' 2024-09-21T00:30:10.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38221 fp=#0x1000000585d' 2024-09-21T00:30:10.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38285 fp=#0x1000000584f' 2024-09-21T00:30:10.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38317 fp=#0x1000000618a' 2024-09-21T00:30:10.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38349 fp=#0x100000058db' 2024-09-21T00:30:10.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38381 fp=#0x10000005955' 2024-09-21T00:30:10.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38445 fp=#0x10000002b7f' 2024-09-21T00:30:10.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38477 fp=#0x100000045e2' 2024-09-21T00:30:10.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38509 fp=#0x100000045d9' 2024-09-21T00:30:10.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38573 fp=#0x10000002a3b' 2024-09-21T00:30:10.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38605 fp=#0x10000002a16' 2024-09-21T00:30:10.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38669 fp=#0x10000001fb6' 2024-09-21T00:30:10.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38733 fp=#0x100000048a6' 2024-09-21T00:30:10.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38829 fp=#0x100000048b7' 2024-09-21T00:30:10.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38861 fp=#0x10000004052' 2024-09-21T00:30:10.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38893 fp=#0x10000004095' 2024-09-21T00:30:10.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38925 fp=#0x10000004033' 2024-09-21T00:30:10.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38957 fp=#0x10000003fe5' 2024-09-21T00:30:10.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38989 fp=#0x10000004058' 2024-09-21T00:30:10.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39021 fp=#0x1000000405c' 2024-09-21T00:30:10.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39053 fp=#0x10000001c62' 2024-09-21T00:30:10.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39085 fp=#0x10000001c80' 2024-09-21T00:30:10.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39117 fp=#0x10000001fe3' 2024-09-21T00:30:10.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39149 fp=#0x100000017e4' 2024-09-21T00:30:10.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39181 fp=#0x100000044f6' 2024-09-21T00:30:10.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39245 fp=#0x100000044a9' 2024-09-21T00:30:10.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39277 fp=#0x100000048fb' 2024-09-21T00:30:10.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39309 fp=#0x10000004847' 2024-09-21T00:30:10.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39341 fp=#0x100000048e4' 2024-09-21T00:30:10.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39373 fp=#0x10000005adf' 2024-09-21T00:30:10.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39405 fp=#0x10000005ad5' 2024-09-21T00:30:10.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39437 fp=#0x10000005e30' 2024-09-21T00:30:10.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39469 fp=#0x10000005ae6' 2024-09-21T00:30:10.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39501 fp=#0x10000005aa8' 2024-09-21T00:30:10.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39533 fp=#0x10000005e31' 2024-09-21T00:30:10.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39565 fp=#0x100000050dc' 2024-09-21T00:30:10.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39597 fp=#0x100000050a8' 2024-09-21T00:30:10.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39629 fp=#0x100000050b2' 2024-09-21T00:30:10.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39661 fp=#0x100000050ef' 2024-09-21T00:30:10.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39693 fp=#0x10000005de9' 2024-09-21T00:30:10.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39757 fp=#0x10000005b59' 2024-09-21T00:30:10.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39789 fp=#0x10000005b5d' 2024-09-21T00:30:10.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39821 fp=#0x10000006391' 2024-09-21T00:30:10.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39853 fp=#0x100000062ea' 2024-09-21T00:30:10.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39885 fp=#0x1000000634a' 2024-09-21T00:30:10.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39917 fp=#0x10000004eec' 2024-09-21T00:30:10.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39949 fp=#0x10000004f01' 2024-09-21T00:30:10.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40013 fp=#0x10000004ed4' 2024-09-21T00:30:10.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40045 fp=#0x100000061e2' 2024-09-21T00:30:10.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40077 fp=#0x100000062a5' 2024-09-21T00:30:10.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40109 fp=#0x100000061e1' 2024-09-21T00:30:10.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40141 fp=#0x10000002658' 2024-09-21T00:30:10.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40173 fp=#0x10000002679' 2024-09-21T00:30:10.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40205 fp=#0x1000000261c' 2024-09-21T00:30:10.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40237 fp=#0x100000008a6' 2024-09-21T00:30:10.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40301 fp=#0x1000000431b' 2024-09-21T00:30:10.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40365 fp=#0x100000042a3' 2024-09-21T00:30:10.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40397 fp=#0x10000000a20' 2024-09-21T00:30:10.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40429 fp=#0x10000000a6a' 2024-09-21T00:30:10.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40525 fp=#0x10000002886' 2024-09-21T00:30:10.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40685 fp=#0x10000004909' 2024-09-21T00:30:10.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40717 fp=#0x10000005eba' 2024-09-21T00:30:10.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40749 fp=#0x10000005ea7' 2024-09-21T00:30:10.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40781 fp=#0x100000048f2' 2024-09-21T00:30:10.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40813 fp=#0x100000031a9' 2024-09-21T00:30:10.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40973 fp=#0x10000004a1d' 2024-09-21T00:30:10.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41005 fp=#0x100000049f1' 2024-09-21T00:30:10.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41037 fp=#0x10000005120' 2024-09-21T00:30:10.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41069 fp=#0x10000004a10' 2024-09-21T00:30:10.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41101 fp=#0x10000004a19' 2024-09-21T00:30:10.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30798 fp=#0x100000024fa' 2024-09-21T00:30:10.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30830 fp=#0x10000000bd8' 2024-09-21T00:30:10.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32686 fp=#0x1000000579c' 2024-09-21T00:30:10.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32718 fp=#0x10000005737' 2024-09-21T00:30:10.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32782 fp=#0x1000000502a' 2024-09-21T00:30:10.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32814 fp=#0x10000005027' 2024-09-21T00:30:10.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32846 fp=#0x10000005050' 2024-09-21T00:30:10.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32910 fp=#0x1000000506a' 2024-09-21T00:30:10.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33006 fp=#0x10000005c8f' 2024-09-21T00:30:10.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33038 fp=#0x10000005c92' 2024-09-21T00:30:10.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33070 fp=#0x10000005cc5' 2024-09-21T00:30:10.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33102 fp=#0x10000005ca2' 2024-09-21T00:30:10.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33134 fp=#0x10000005c43' 2024-09-21T00:30:10.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33166 fp=#0x1000000606b' 2024-09-21T00:30:10.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33198 fp=#0x10000006044' 2024-09-21T00:30:10.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33262 fp=#0x10000006051' 2024-09-21T00:30:10.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33326 fp=#0x10000006817' 2024-09-21T00:30:10.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33358 fp=#0x100000067e6' 2024-09-21T00:30:10.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33390 fp=#0x10000003e21' 2024-09-21T00:30:10.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33422 fp=#0x10000003e74' 2024-09-21T00:30:10.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33454 fp=#0x10000003dff' 2024-09-21T00:30:10.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33486 fp=#0x10000003d52' 2024-09-21T00:30:10.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33518 fp=#0x100000047f1' 2024-09-21T00:30:10.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33550 fp=#0x10000005aa3' 2024-09-21T00:30:10.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33582 fp=#0x100000055b1' 2024-09-21T00:30:10.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33614 fp=#0x100000059f2' 2024-09-21T00:30:10.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33646 fp=#0x10000005aa9' 2024-09-21T00:30:10.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33710 fp=#0x100000059f5' 2024-09-21T00:30:10.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33742 fp=#0x100000067aa' 2024-09-21T00:30:10.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33774 fp=#0x1000000675f' 2024-09-21T00:30:10.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33806 fp=#0x10000006782' 2024-09-21T00:30:10.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33934 fp=#0x10000006419' 2024-09-21T00:30:10.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33966 fp=#0x1000000477d' 2024-09-21T00:30:10.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33998 fp=#0x10000004780' 2024-09-21T00:30:10.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34030 fp=#0x1000000640a' 2024-09-21T00:30:10.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34062 fp=#0x10000006447' 2024-09-21T00:30:10.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34126 fp=#0x10000006297' 2024-09-21T00:30:10.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34158 fp=#0x1000000645f' 2024-09-21T00:30:10.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34190 fp=#0x10000006223' 2024-09-21T00:30:10.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34222 fp=#0x100000061fb' 2024-09-21T00:30:10.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34254 fp=#0x100000064af' 2024-09-21T00:30:10.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34286 fp=#0x1000000646f' 2024-09-21T00:30:10.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34350 fp=#0x100000061c1' 2024-09-21T00:30:10.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34382 fp=#0x10000006194' 2024-09-21T00:30:10.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34414 fp=#0x10000002652' 2024-09-21T00:30:10.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34446 fp=#0x1000000269e' 2024-09-21T00:30:10.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34478 fp=#0x100000061b9' 2024-09-21T00:30:10.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34510 fp=#0x100000061c3' 2024-09-21T00:30:10.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34542 fp=#0x100000062c4' 2024-09-21T00:30:10.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34574 fp=#0x100000062c8' 2024-09-21T00:30:10.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34606 fp=#0x10000006322' 2024-09-21T00:30:10.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34638 fp=#0x10000006354' 2024-09-21T00:30:10.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34670 fp=#0x100000062e1' 2024-09-21T00:30:10.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34766 fp=#0x100000057f0' 2024-09-21T00:30:10.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34798 fp=#0x100000057e8' 2024-09-21T00:30:10.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34830 fp=#0x100000057cc' 2024-09-21T00:30:10.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34862 fp=#0x100000064b9' 2024-09-21T00:30:10.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34894 fp=#0x100000057dd' 2024-09-21T00:30:10.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34926 fp=#0x1000000529d' 2024-09-21T00:30:10.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34958 fp=#0x1000000696f' 2024-09-21T00:30:10.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34990 fp=#0x100000052ba' 2024-09-21T00:30:10.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35054 fp=#0x10000002d24' 2024-09-21T00:30:10.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35118 fp=#0x100000060de' 2024-09-21T00:30:10.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35150 fp=#0x100000060ee' 2024-09-21T00:30:10.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35214 fp=#0x10000006088' 2024-09-21T00:30:10.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35246 fp=#0x10000003fd9' 2024-09-21T00:30:10.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35278 fp=#0x10000003f37' 2024-09-21T00:30:10.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35438 fp=#0x10000005fda' 2024-09-21T00:30:10.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35598 fp=#0x10000005f80' 2024-09-21T00:30:10.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35630 fp=#0x10000005bbd' 2024-09-21T00:30:10.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35662 fp=#0x10000005be4' 2024-09-21T00:30:10.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35694 fp=#0x10000004953' 2024-09-21T00:30:10.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35726 fp=#0x10000005010' 2024-09-21T00:30:10.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35758 fp=#0x10000004f0a' 2024-09-21T00:30:10.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35790 fp=#0x10000004f6f' 2024-09-21T00:30:10.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35822 fp=#0x10000004f54' 2024-09-21T00:30:10.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35854 fp=#0x10000004f33' 2024-09-21T00:30:10.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35886 fp=#0x10000006729' 2024-09-21T00:30:10.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35918 fp=#0x10000006734' 2024-09-21T00:30:10.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35950 fp=#0x1000000688c' 2024-09-21T00:30:10.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35982 fp=#0x10000006872' 2024-09-21T00:30:10.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36014 fp=#0x10000006852' 2024-09-21T00:30:10.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36110 fp=#0x10000005f5e' 2024-09-21T00:30:10.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36206 fp=#0x10000006893' 2024-09-21T00:30:10.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36238 fp=#0x100000068c2' 2024-09-21T00:30:10.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36302 fp=#0x100000065b7' 2024-09-21T00:30:10.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36334 fp=#0x10000006614' 2024-09-21T00:30:10.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36366 fp=#0x100000065f3' 2024-09-21T00:30:10.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36430 fp=#0x10000006163' 2024-09-21T00:30:10.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36462 fp=#0x10000006136' 2024-09-21T00:30:10.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36526 fp=#0x10000006047' 2024-09-21T00:30:10.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36558 fp=#0x10000005aca' 2024-09-21T00:30:10.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36590 fp=#0x10000005a88' 2024-09-21T00:30:10.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36654 fp=#0x1000000474d' 2024-09-21T00:30:10.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36686 fp=#0x10000004758' 2024-09-21T00:30:10.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36718 fp=#0x100000047a4' 2024-09-21T00:30:10.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36750 fp=#0x1000000490a' 2024-09-21T00:30:10.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36782 fp=#0x10000004815' 2024-09-21T00:30:10.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36846 fp=#0x10000005889' 2024-09-21T00:30:10.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36974 fp=#0x100000051df' 2024-09-21T00:30:10.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37038 fp=#0x10000005231' 2024-09-21T00:30:10.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37070 fp=#0x100000051ed' 2024-09-21T00:30:10.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37102 fp=#0x1000000526e' 2024-09-21T00:30:10.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37230 fp=#0x10000005bd5' 2024-09-21T00:30:10.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37262 fp=#0x10000005bea' 2024-09-21T00:30:10.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37294 fp=#0x10000005b9d' 2024-09-21T00:30:10.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37326 fp=#0x10000005b9a' 2024-09-21T00:30:10.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37358 fp=#0x10000005bdf' 2024-09-21T00:30:10.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37390 fp=#0x10000005eff' 2024-09-21T00:30:10.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37422 fp=#0x10000005ecc' 2024-09-21T00:30:10.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37454 fp=#0x10000004a93' 2024-09-21T00:30:10.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37486 fp=#0x10000004b9a' 2024-09-21T00:30:10.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37518 fp=#0x10000004af0' 2024-09-21T00:30:10.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37550 fp=#0x10000004af9' 2024-09-21T00:30:10.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37582 fp=#0x10000005568' 2024-09-21T00:30:10.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37614 fp=#0x10000005555' 2024-09-21T00:30:10.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37646 fp=#0x100000054d7' 2024-09-21T00:30:10.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37678 fp=#0x1000000553c' 2024-09-21T00:30:10.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37710 fp=#0x10000005828' 2024-09-21T00:30:10.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37742 fp=#0x1000000580e' 2024-09-21T00:30:10.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37774 fp=#0x10000004270' 2024-09-21T00:30:10.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37806 fp=#0x10000004281' 2024-09-21T00:30:10.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37838 fp=#0x100000042ef' 2024-09-21T00:30:10.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37870 fp=#0x10000005812' 2024-09-21T00:30:10.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37902 fp=#0x10000001d86' 2024-09-21T00:30:10.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37934 fp=#0x10000001c6c' 2024-09-21T00:30:10.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37966 fp=#0x10000001ccd' 2024-09-21T00:30:10.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37998 fp=#0x10000001d2c' 2024-09-21T00:30:10.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38030 fp=#0x10000003ca0' 2024-09-21T00:30:10.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38062 fp=#0x10000003c61' 2024-09-21T00:30:10.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38126 fp=#0x10000003ade' 2024-09-21T00:30:10.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38190 fp=#0x1000000598b' 2024-09-21T00:30:10.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38222 fp=#0x100000059a8' 2024-09-21T00:30:10.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38254 fp=#0x10000005968' 2024-09-21T00:30:10.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38286 fp=#0x10000005854' 2024-09-21T00:30:10.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38318 fp=#0x1000000596f' 2024-09-21T00:30:10.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38350 fp=#0x10000005972' 2024-09-21T00:30:10.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38382 fp=#0x10000005929' 2024-09-21T00:30:10.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38414 fp=#0x100000029c5' 2024-09-21T00:30:10.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38478 fp=#0x10000002a59' 2024-09-21T00:30:10.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38510 fp=#0x100000045f8' 2024-09-21T00:30:10.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38606 fp=#0x10000002a1e' 2024-09-21T00:30:10.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38638 fp=#0x10000004869' 2024-09-21T00:30:10.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38670 fp=#0x100000010bb' 2024-09-21T00:30:10.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38702 fp=#0x100000010e1' 2024-09-21T00:30:10.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38766 fp=#0x100000010e6' 2024-09-21T00:30:10.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38798 fp=#0x10000001086' 2024-09-21T00:30:10.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38830 fp=#0x1000000488f' 2024-09-21T00:30:10.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38862 fp=#0x10000004049' 2024-09-21T00:30:10.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38894 fp=#0x10000004114' 2024-09-21T00:30:10.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38958 fp=#0x10000003ff4' 2024-09-21T00:30:10.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39022 fp=#0x1000000408a' 2024-09-21T00:30:10.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39054 fp=#0x1000000185f' 2024-09-21T00:30:10.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39086 fp=#0x10000001fb5' 2024-09-21T00:30:10.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39118 fp=#0x10000001bf7' 2024-09-21T00:30:10.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39150 fp=#0x100000017f9' 2024-09-21T00:30:10.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39278 fp=#0x100000048c0' 2024-09-21T00:30:10.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39310 fp=#0x10000004885' 2024-09-21T00:30:10.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39342 fp=#0x1000000488e' 2024-09-21T00:30:10.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39374 fp=#0x10000005a17' 2024-09-21T00:30:10.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39406 fp=#0x10000005e14' 2024-09-21T00:30:10.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39502 fp=#0x10000005a36' 2024-09-21T00:30:10.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39534 fp=#0x10000005e29' 2024-09-21T00:30:10.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39566 fp=#0x100000050fd' 2024-09-21T00:30:10.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39598 fp=#0x1000000509f' 2024-09-21T00:30:10.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39630 fp=#0x100000050ad' 2024-09-21T00:30:10.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39662 fp=#0x100000050a6' 2024-09-21T00:30:10.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39758 fp=#0x10000005b41' 2024-09-21T00:30:10.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39790 fp=#0x10000005b48' 2024-09-21T00:30:10.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39822 fp=#0x10000006314' 2024-09-21T00:30:10.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39854 fp=#0x10000006336' 2024-09-21T00:30:10.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39886 fp=#0x10000006372' 2024-09-21T00:30:10.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39918 fp=#0x10000004ee3' 2024-09-21T00:30:10.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40014 fp=#0x10000006214' 2024-09-21T00:30:10.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40046 fp=#0x100000061df' 2024-09-21T00:30:10.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40078 fp=#0x10000006282' 2024-09-21T00:30:10.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40110 fp=#0x1000000623b' 2024-09-21T00:30:10.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40142 fp=#0x1000000264d' 2024-09-21T00:30:10.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40174 fp=#0x1000000263a' 2024-09-21T00:30:10.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40206 fp=#0x10000002606' 2024-09-21T00:30:10.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40238 fp=#0x10000000b06' 2024-09-21T00:30:10.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40270 fp=#0x10000000a3c' 2024-09-21T00:30:10.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40302 fp=#0x10000004316' 2024-09-21T00:30:10.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40334 fp=#0x100000008fb' 2024-09-21T00:30:10.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40366 fp=#0x10000001687' 2024-09-21T00:30:10.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40398 fp=#0x100000008d1' 2024-09-21T00:30:10.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40430 fp=#0x100000009ff' 2024-09-21T00:30:10.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40462 fp=#0x100000058f5' 2024-09-21T00:30:10.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40526 fp=#0x10000002898' 2024-09-21T00:30:10.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40686 fp=#0x10000005ea4' 2024-09-21T00:30:10.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40718 fp=#0x10000005eb7' 2024-09-21T00:30:10.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40750 fp=#0x10000005ed1' 2024-09-21T00:30:10.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40782 fp=#0x100000048df' 2024-09-21T00:30:10.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40974 fp=#0x1000000512f' 2024-09-21T00:30:10.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41006 fp=#0x100000049d8' 2024-09-21T00:30:10.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41070 fp=#0x100000049c4' 2024-09-21T00:30:10.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41102 fp=#0x1000000512c' 2024-09-21T00:30:10.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30767 fp=#0x10000005885' 2024-09-21T00:30:10.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30799 fp=#0x1000000583c' 2024-09-21T00:30:10.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30831 fp=#0x10000004e84' 2024-09-21T00:30:10.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32687 fp=#0x10000005766' 2024-09-21T00:30:10.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32751 fp=#0x10000005797' 2024-09-21T00:30:10.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32783 fp=#0x10000005086' 2024-09-21T00:30:10.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32847 fp=#0x10000005047' 2024-09-21T00:30:10.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32879 fp=#0x10000005049' 2024-09-21T00:30:10.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32943 fp=#0x100000066df' 2024-09-21T00:30:10.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32975 fp=#0x1000000504e' 2024-09-21T00:30:10.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33039 fp=#0x10000005c96' 2024-09-21T00:30:10.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33071 fp=#0x10000005c82' 2024-09-21T00:30:10.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33103 fp=#0x10000005cf1' 2024-09-21T00:30:10.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33135 fp=#0x10000005cbb' 2024-09-21T00:30:10.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33167 fp=#0x10000006037' 2024-09-21T00:30:10.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33231 fp=#0x10000006071' 2024-09-21T00:30:10.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33263 fp=#0x10000006034' 2024-09-21T00:30:10.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33327 fp=#0x10000006813' 2024-09-21T00:30:10.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33359 fp=#0x1000000680e' 2024-09-21T00:30:10.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33423 fp=#0x10000003e47' 2024-09-21T00:30:10.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33487 fp=#0x10000003e7d' 2024-09-21T00:30:10.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33519 fp=#0x100000047e5' 2024-09-21T00:30:10.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33551 fp=#0x10000005a8a' 2024-09-21T00:30:10.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33583 fp=#0x10000005a3b' 2024-09-21T00:30:10.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33679 fp=#0x1000000559e' 2024-09-21T00:30:10.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33711 fp=#0x10000005597' 2024-09-21T00:30:10.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33743 fp=#0x100000067a0' 2024-09-21T00:30:10.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33775 fp=#0x1000000675e' 2024-09-21T00:30:10.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33807 fp=#0x100000067a1' 2024-09-21T00:30:10.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33935 fp=#0x1000000641c' 2024-09-21T00:30:10.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33967 fp=#0x10000004787' 2024-09-21T00:30:10.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33999 fp=#0x10000004728' 2024-09-21T00:30:10.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34031 fp=#0x10000006417' 2024-09-21T00:30:10.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34063 fp=#0x10000006422' 2024-09-21T00:30:10.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34095 fp=#0x100000031db' 2024-09-21T00:30:10.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34127 fp=#0x10000006206' 2024-09-21T00:30:10.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34159 fp=#0x1000000621a' 2024-09-21T00:30:10.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34191 fp=#0x1000000648c' 2024-09-21T00:30:10.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34223 fp=#0x1000000625b' 2024-09-21T00:30:10.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34255 fp=#0x10000006249' 2024-09-21T00:30:10.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34287 fp=#0x10000006269' 2024-09-21T00:30:10.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34351 fp=#0x100000061c0' 2024-09-21T00:30:10.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34383 fp=#0x1000000264a' 2024-09-21T00:30:10.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34415 fp=#0x100000061ab' 2024-09-21T00:30:10.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34511 fp=#0x100000061be' 2024-09-21T00:30:10.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34543 fp=#0x100000063a0' 2024-09-21T00:30:10.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34575 fp=#0x100000062f3' 2024-09-21T00:30:10.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34607 fp=#0x1000000632f' 2024-09-21T00:30:10.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34639 fp=#0x1000000634b' 2024-09-21T00:30:10.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34671 fp=#0x1000000632a' 2024-09-21T00:30:10.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34767 fp=#0x100000064f4' 2024-09-21T00:30:10.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34799 fp=#0x100000057ec' 2024-09-21T00:30:10.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34831 fp=#0x100000064df' 2024-09-21T00:30:10.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34863 fp=#0x100000064c6' 2024-09-21T00:30:10.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34895 fp=#0x100000057af' 2024-09-21T00:30:10.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34991 fp=#0x10000006967' 2024-09-21T00:30:10.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35023 fp=#0x100000052c8' 2024-09-21T00:30:10.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35119 fp=#0x100000060f2' 2024-09-21T00:30:10.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35279 fp=#0x10000003f4b' 2024-09-21T00:30:10.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35311 fp=#0x10000003f9a' 2024-09-21T00:30:10.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35343 fp=#0x10000005392' 2024-09-21T00:30:10.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35407 fp=#0x1000000531d' 2024-09-21T00:30:10.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35471 fp=#0x10000005faa' 2024-09-21T00:30:10.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35535 fp=#0x10000005f7a' 2024-09-21T00:30:10.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35567 fp=#0x10000005f6f' 2024-09-21T00:30:10.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35599 fp=#0x10000005f86' 2024-09-21T00:30:10.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35663 fp=#0x10000005b97' 2024-09-21T00:30:10.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35695 fp=#0x10000004925' 2024-09-21T00:30:10.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35727 fp=#0x10000005006' 2024-09-21T00:30:10.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35759 fp=#0x10000004f2e' 2024-09-21T00:30:10.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35791 fp=#0x10000005020' 2024-09-21T00:30:10.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35823 fp=#0x10000004f27' 2024-09-21T00:30:10.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35855 fp=#0x10000004f0b' 2024-09-21T00:30:10.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35887 fp=#0x10000006755' 2024-09-21T00:30:10.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35919 fp=#0x10000006711' 2024-09-21T00:30:10.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35951 fp=#0x1000000682f' 2024-09-21T00:30:10.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35983 fp=#0x10000006876' 2024-09-21T00:30:10.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36015 fp=#0x10000006855' 2024-09-21T00:30:10.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36047 fp=#0x10000005f5c' 2024-09-21T00:30:10.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36111 fp=#0x10000005d6f' 2024-09-21T00:30:10.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36143 fp=#0x10000005f48' 2024-09-21T00:30:10.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36175 fp=#0x10000005f3d' 2024-09-21T00:30:10.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36207 fp=#0x100000068e0' 2024-09-21T00:30:10.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36239 fp=#0x100000068d3' 2024-09-21T00:30:10.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36271 fp=#0x100000065ab' 2024-09-21T00:30:10.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36303 fp=#0x10000006600' 2024-09-21T00:30:10.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36367 fp=#0x1000000658f' 2024-09-21T00:30:10.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36399 fp=#0x100000065a8' 2024-09-21T00:30:10.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36431 fp=#0x10000003ff6' 2024-09-21T00:30:10.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36463 fp=#0x10000006146' 2024-09-21T00:30:10.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36559 fp=#0x100000059b5' 2024-09-21T00:30:10.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36623 fp=#0x10000005a2c' 2024-09-21T00:30:10.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36655 fp=#0x10000004902' 2024-09-21T00:30:10.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36687 fp=#0x1000000478c' 2024-09-21T00:30:10.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36719 fp=#0x1000000476c' 2024-09-21T00:30:10.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36751 fp=#0x1000000474c' 2024-09-21T00:30:10.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36783 fp=#0x10000004837' 2024-09-21T00:30:10.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36815 fp=#0x10000005b12' 2024-09-21T00:30:10.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36847 fp=#0x1000000592b' 2024-09-21T00:30:10.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36943 fp=#0x1000000590a' 2024-09-21T00:30:10.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36975 fp=#0x1000000526b' 2024-09-21T00:30:10.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37007 fp=#0x1000000524c' 2024-09-21T00:30:10.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37071 fp=#0x10000005275' 2024-09-21T00:30:10.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37135 fp=#0x10000005444' 2024-09-21T00:30:10.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37199 fp=#0x10000005fea' 2024-09-21T00:30:10.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37231 fp=#0x10000005c46' 2024-09-21T00:30:10.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37263 fp=#0x10000005c21' 2024-09-21T00:30:10.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37327 fp=#0x10000005bcb' 2024-09-21T00:30:10.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37359 fp=#0x10000005c17' 2024-09-21T00:30:10.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37423 fp=#0x10000005f02' 2024-09-21T00:30:10.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37455 fp=#0x10000004a66' 2024-09-21T00:30:10.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37487 fp=#0x10000004ac0' 2024-09-21T00:30:10.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37519 fp=#0x10000004a8f' 2024-09-21T00:30:10.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37551 fp=#0x10000004ba4' 2024-09-21T00:30:10.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37583 fp=#0x1000000557a' 2024-09-21T00:30:10.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37615 fp=#0x10000005503' 2024-09-21T00:30:10.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37647 fp=#0x100000054d3' 2024-09-21T00:30:10.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37679 fp=#0x1000000549b' 2024-09-21T00:30:10.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37711 fp=#0x10000004230' 2024-09-21T00:30:10.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37743 fp=#0x100000042ee' 2024-09-21T00:30:10.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37775 fp=#0x10000005808' 2024-09-21T00:30:10.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37807 fp=#0x10000005834' 2024-09-21T00:30:10.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37839 fp=#0x10000004228' 2024-09-21T00:30:10.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37903 fp=#0x10000001c5f' 2024-09-21T00:30:10.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37935 fp=#0x10000001c7e' 2024-09-21T00:30:10.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37967 fp=#0x10000001cdb' 2024-09-21T00:30:10.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38031 fp=#0x10000003c9a' 2024-09-21T00:30:10.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38063 fp=#0x10000003c7d' 2024-09-21T00:30:10.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38127 fp=#0x10000003b44' 2024-09-21T00:30:10.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38191 fp=#0x10000005856' 2024-09-21T00:30:10.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38223 fp=#0x10000005848' 2024-09-21T00:30:10.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38255 fp=#0x10000005869' 2024-09-21T00:30:10.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38287 fp=#0x10000006167' 2024-09-21T00:30:10.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38351 fp=#0x10000005866' 2024-09-21T00:30:10.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38383 fp=#0x100000058ed' 2024-09-21T00:30:10.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38447 fp=#0x100000045dc' 2024-09-21T00:30:10.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38479 fp=#0x100000045c1' 2024-09-21T00:30:10.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38511 fp=#0x10000002a35' 2024-09-21T00:30:10.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38543 fp=#0x100000029fd' 2024-09-21T00:30:10.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38575 fp=#0x10000002b09' 2024-09-21T00:30:10.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38607 fp=#0x10000002a5c' 2024-09-21T00:30:10.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38703 fp=#0x100000010c7' 2024-09-21T00:30:10.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38863 fp=#0x1000000400f' 2024-09-21T00:30:10.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38895 fp=#0x10000004118' 2024-09-21T00:30:10.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38959 fp=#0x1000000406a' 2024-09-21T00:30:10.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38991 fp=#0x10000004006' 2024-09-21T00:30:10.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39023 fp=#0x10000004295' 2024-09-21T00:30:10.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39055 fp=#0x10000001870' 2024-09-21T00:30:10.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39087 fp=#0x10000001fa6' 2024-09-21T00:30:10.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39119 fp=#0x100000017e0' 2024-09-21T00:30:10.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39151 fp=#0x10000001dba' 2024-09-21T00:30:10.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39183 fp=#0x100000044ed' 2024-09-21T00:30:10.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39279 fp=#0x10000004901' 2024-09-21T00:30:10.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39311 fp=#0x1000000480d' 2024-09-21T00:30:10.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39343 fp=#0x100000048e9' 2024-09-21T00:30:10.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39375 fp=#0x10000005e39' 2024-09-21T00:30:10.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39407 fp=#0x10000005ac0' 2024-09-21T00:30:10.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39439 fp=#0x10000005ad2' 2024-09-21T00:30:10.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39471 fp=#0x10000005a47' 2024-09-21T00:30:10.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39503 fp=#0x10000005ae5' 2024-09-21T00:30:10.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39535 fp=#0x10000005e23' 2024-09-21T00:30:10.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39567 fp=#0x100000050a9' 2024-09-21T00:30:10.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39599 fp=#0x100000050b0' 2024-09-21T00:30:10.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39631 fp=#0x100000050a3' 2024-09-21T00:30:10.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39663 fp=#0x100000050ac' 2024-09-21T00:30:10.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39695 fp=#0x10000005d7c' 2024-09-21T00:30:10.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39759 fp=#0x10000005b4c' 2024-09-21T00:30:10.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39791 fp=#0x10000005b3b' 2024-09-21T00:30:10.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39823 fp=#0x10000006388' 2024-09-21T00:30:10.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39855 fp=#0x100000062fa' 2024-09-21T00:30:10.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39887 fp=#0x100000062f9' 2024-09-21T00:30:10.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39951 fp=#0x10000004ef9' 2024-09-21T00:30:10.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39983 fp=#0x10000004e92' 2024-09-21T00:30:10.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40015 fp=#0x100000062a2' 2024-09-21T00:30:10.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40047 fp=#0x100000061fd' 2024-09-21T00:30:10.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40079 fp=#0x10000006244' 2024-09-21T00:30:10.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40111 fp=#0x100000061e5' 2024-09-21T00:30:10.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40143 fp=#0x10000002613' 2024-09-21T00:30:10.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40175 fp=#0x100000026cc' 2024-09-21T00:30:10.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40207 fp=#0x10000002662' 2024-09-21T00:30:10.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40239 fp=#0x10000004327' 2024-09-21T00:30:10.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40271 fp=#0x10000004315' 2024-09-21T00:30:10.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40303 fp=#0x10000004336' 2024-09-21T00:30:10.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40335 fp=#0x1000000089d' 2024-09-21T00:30:10.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40367 fp=#0x100000008b7' 2024-09-21T00:30:10.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40399 fp=#0x10000000ab5' 2024-09-21T00:30:10.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40431 fp=#0x100000042d7' 2024-09-21T00:30:10.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40463 fp=#0x100000058ff' 2024-09-21T00:30:10.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40527 fp=#0x100000028a3' 2024-09-21T00:30:10.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40687 fp=#0x100000048f0' 2024-09-21T00:30:10.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40719 fp=#0x10000005edd' 2024-09-21T00:30:10.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40751 fp=#0x10000004919' 2024-09-21T00:30:10.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40783 fp=#0x10000004848' 2024-09-21T00:30:10.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40815 fp=#0x100000063f1' 2024-09-21T00:30:10.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40943 fp=#0x100000063d4' 2024-09-21T00:30:10.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40975 fp=#0x10000004996' 2024-09-21T00:30:10.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41007 fp=#0x10000005f29' 2024-09-21T00:30:10.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41039 fp=#0x10000004984' 2024-09-21T00:30:10.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41071 fp=#0x1000000510e' 2024-09-21T00:30:10.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41103 fp=#0x1000000510d' 2024-09-21T00:30:10.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_path:mds.0:30608 fp=#0x1' 2024-09-21T00:30:10.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30768 fp=#0x1000000067e' 2024-09-21T00:30:10.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30800 fp=#0x100000062b9' 2024-09-21T00:30:10.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30832 fp=#0x10000005b5f' 2024-09-21T00:30:10.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32688 fp=#0x10000005794' 2024-09-21T00:30:10.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32720 fp=#0x100000057ab' 2024-09-21T00:30:10.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32752 fp=#0x10000005729' 2024-09-21T00:30:10.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32784 fp=#0x100000066cc' 2024-09-21T00:30:10.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32816 fp=#0x1000000504d' 2024-09-21T00:30:10.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32912 fp=#0x10000005037' 2024-09-21T00:30:10.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32976 fp=#0x10000005054' 2024-09-21T00:30:10.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33008 fp=#0x10000005c69' 2024-09-21T00:30:10.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33040 fp=#0x10000005cb5' 2024-09-21T00:30:10.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33072 fp=#0x10000005c61' 2024-09-21T00:30:10.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33104 fp=#0x10000005c75' 2024-09-21T00:30:10.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33232 fp=#0x10000006078' 2024-09-21T00:30:10.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33264 fp=#0x10000006697' 2024-09-21T00:30:10.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33296 fp=#0x1000000605b' 2024-09-21T00:30:10.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33328 fp=#0x100000067db' 2024-09-21T00:30:10.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33360 fp=#0x10000006809' 2024-09-21T00:30:10.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33392 fp=#0x10000003e83' 2024-09-21T00:30:10.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33424 fp=#0x10000003d9b' 2024-09-21T00:30:10.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33456 fp=#0x10000003da6' 2024-09-21T00:30:10.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33488 fp=#0x10000003de3' 2024-09-21T00:30:10.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33584 fp=#0x10000005a3d' 2024-09-21T00:30:10.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33616 fp=#0x10000005aac' 2024-09-21T00:30:10.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33680 fp=#0x10000005a4b' 2024-09-21T00:30:10.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33744 fp=#0x1000000678c' 2024-09-21T00:30:10.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33776 fp=#0x100000067ac' 2024-09-21T00:30:10.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33936 fp=#0x1000000473f' 2024-09-21T00:30:10.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33968 fp=#0x10000006445' 2024-09-21T00:30:10.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34000 fp=#0x10000006430' 2024-09-21T00:30:10.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34032 fp=#0x10000006405' 2024-09-21T00:30:10.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34064 fp=#0x100000063fc' 2024-09-21T00:30:10.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34096 fp=#0x100000031f6' 2024-09-21T00:30:10.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34128 fp=#0x10000006271' 2024-09-21T00:30:10.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34160 fp=#0x10000006462' 2024-09-21T00:30:10.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34192 fp=#0x100000064ac' 2024-09-21T00:30:10.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34224 fp=#0x1000000621d' 2024-09-21T00:30:10.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34256 fp=#0x100000061f5' 2024-09-21T00:30:10.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34288 fp=#0x10000006272' 2024-09-21T00:30:10.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34320 fp=#0x100000061b3' 2024-09-21T00:30:10.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34352 fp=#0x1000000306e' 2024-09-21T00:30:10.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34384 fp=#0x1000000268c' 2024-09-21T00:30:10.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34448 fp=#0x1000000258c' 2024-09-21T00:30:10.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34480 fp=#0x100000026a4' 2024-09-21T00:30:10.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34512 fp=#0x1000000252b' 2024-09-21T00:30:10.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34544 fp=#0x100000062e3' 2024-09-21T00:30:10.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34576 fp=#0x10000006385' 2024-09-21T00:30:10.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34608 fp=#0x100000063ad' 2024-09-21T00:30:10.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34640 fp=#0x1000000630c' 2024-09-21T00:30:10.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34672 fp=#0x10000006396' 2024-09-21T00:30:10.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34768 fp=#0x100000057e5' 2024-09-21T00:30:10.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34800 fp=#0x100000064f6' 2024-09-21T00:30:10.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34832 fp=#0x100000064e8' 2024-09-21T00:30:10.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34864 fp=#0x100000064b4' 2024-09-21T00:30:10.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34896 fp=#0x100000064cf' 2024-09-21T00:30:10.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34928 fp=#0x100000052a0' 2024-09-21T00:30:10.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34960 fp=#0x100000052c6' 2024-09-21T00:30:10.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34992 fp=#0x100000052ff' 2024-09-21T00:30:10.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35024 fp=#0x1000000529e' 2024-09-21T00:30:10.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35152 fp=#0x10000006093' 2024-09-21T00:30:10.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35216 fp=#0x100000060e7' 2024-09-21T00:30:10.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35248 fp=#0x10000003fdc' 2024-09-21T00:30:10.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35280 fp=#0x10000003f59' 2024-09-21T00:30:10.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35312 fp=#0x10000003f31' 2024-09-21T00:30:10.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35344 fp=#0x1000000539b' 2024-09-21T00:30:10.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35408 fp=#0x1000000538d' 2024-09-21T00:30:10.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35440 fp=#0x10000005fc7' 2024-09-21T00:30:10.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35504 fp=#0x10000005fa8' 2024-09-21T00:30:10.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35568 fp=#0x10000005fa6' 2024-09-21T00:30:10.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35600 fp=#0x10000005fa1' 2024-09-21T00:30:10.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35632 fp=#0x10000005b8f' 2024-09-21T00:30:10.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35664 fp=#0x10000005b71' 2024-09-21T00:30:10.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35696 fp=#0x10000004963' 2024-09-21T00:30:10.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35728 fp=#0x10000004f21' 2024-09-21T00:30:10.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35760 fp=#0x10000004fe0' 2024-09-21T00:30:10.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35792 fp=#0x10000004fc6' 2024-09-21T00:30:10.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35824 fp=#0x10000004ffe' 2024-09-21T00:30:10.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35856 fp=#0x10000004f20' 2024-09-21T00:30:10.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35888 fp=#0x10000006744' 2024-09-21T00:30:10.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35920 fp=#0x1000000673e' 2024-09-21T00:30:10.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35952 fp=#0x10000006864' 2024-09-21T00:30:10.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35984 fp=#0x10000006840' 2024-09-21T00:30:10.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36016 fp=#0x10000006850' 2024-09-21T00:30:10.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36048 fp=#0x10000005f53' 2024-09-21T00:30:10.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36080 fp=#0x10000005f43' 2024-09-21T00:30:10.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36112 fp=#0x10000005f62' 2024-09-21T00:30:10.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36208 fp=#0x100000068bc' 2024-09-21T00:30:10.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36240 fp=#0x100000068a7' 2024-09-21T00:30:10.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36272 fp=#0x10000006611' 2024-09-21T00:30:10.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36304 fp=#0x100000065ee' 2024-09-21T00:30:10.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36336 fp=#0x100000065ba' 2024-09-21T00:30:10.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36368 fp=#0x100000065bb' 2024-09-21T00:30:10.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36400 fp=#0x100000065ae' 2024-09-21T00:30:10.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36432 fp=#0x10000004002' 2024-09-21T00:30:10.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36464 fp=#0x10000006139' 2024-09-21T00:30:10.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36496 fp=#0x1000000600b' 2024-09-21T00:30:10.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36528 fp=#0x1000000600e' 2024-09-21T00:30:10.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36560 fp=#0x10000005ac7' 2024-09-21T00:30:10.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36592 fp=#0x100000059b2' 2024-09-21T00:30:10.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36624 fp=#0x100000059dc' 2024-09-21T00:30:10.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36656 fp=#0x10000004774' 2024-09-21T00:30:10.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36688 fp=#0x1000000486a' 2024-09-21T00:30:10.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36720 fp=#0x100000047a2' 2024-09-21T00:30:10.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36752 fp=#0x10000004819' 2024-09-21T00:30:10.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36784 fp=#0x1000000488d' 2024-09-21T00:30:10.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36816 fp=#0x100000058e7' 2024-09-21T00:30:10.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36848 fp=#0x10000005a2d' 2024-09-21T00:30:10.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36912 fp=#0x10000005afb' 2024-09-21T00:30:10.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36944 fp=#0x10000005afd' 2024-09-21T00:30:10.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36976 fp=#0x100000051a4' 2024-09-21T00:30:10.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37008 fp=#0x10000005199' 2024-09-21T00:30:10.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37040 fp=#0x1000000523d' 2024-09-21T00:30:10.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37072 fp=#0x10000005241' 2024-09-21T00:30:10.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37104 fp=#0x100000051d7' 2024-09-21T00:30:10.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37168 fp=#0x100000053cb' 2024-09-21T00:30:10.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37200 fp=#0x10000003d32' 2024-09-21T00:30:10.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37232 fp=#0x10000005b9f' 2024-09-21T00:30:10.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37328 fp=#0x10000005c4d' 2024-09-21T00:30:10.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37360 fp=#0x10000005bf8' 2024-09-21T00:30:10.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37392 fp=#0x10000005efe' 2024-09-21T00:30:10.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37424 fp=#0x10000005eed' 2024-09-21T00:30:10.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37456 fp=#0x10000004bbc' 2024-09-21T00:30:10.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37488 fp=#0x10000004af2' 2024-09-21T00:30:10.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37520 fp=#0x10000004b9e' 2024-09-21T00:30:10.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37552 fp=#0x10000004a98' 2024-09-21T00:30:10.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37584 fp=#0x10000005491' 2024-09-21T00:30:10.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37616 fp=#0x10000005516' 2024-09-21T00:30:10.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37648 fp=#0x1000000548e' 2024-09-21T00:30:10.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37680 fp=#0x10000005547' 2024-09-21T00:30:10.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37712 fp=#0x10000005817' 2024-09-21T00:30:10.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37744 fp=#0x1000000425a' 2024-09-21T00:30:10.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37776 fp=#0x100000057fa' 2024-09-21T00:30:10.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37808 fp=#0x100000042fd' 2024-09-21T00:30:10.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37840 fp=#0x100000042ce' 2024-09-21T00:30:10.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37872 fp=#0x100000057fb' 2024-09-21T00:30:10.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37904 fp=#0x10000001bbb' 2024-09-21T00:30:10.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37936 fp=#0x10000001cb0' 2024-09-21T00:30:10.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37968 fp=#0x10000001c9b' 2024-09-21T00:30:10.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38000 fp=#0x10000001bc3' 2024-09-21T00:30:10.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38032 fp=#0x10000003c98' 2024-09-21T00:30:10.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38064 fp=#0x10000003c40' 2024-09-21T00:30:10.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38192 fp=#0x10000006175' 2024-09-21T00:30:10.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38224 fp=#0x1000000584b' 2024-09-21T00:30:10.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38288 fp=#0x10000006181' 2024-09-21T00:30:10.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38320 fp=#0x1000000587b' 2024-09-21T00:30:10.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38416 fp=#0x10000002a45' 2024-09-21T00:30:10.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38448 fp=#0x100000029ce' 2024-09-21T00:30:10.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38480 fp=#0x10000004605' 2024-09-21T00:30:10.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38512 fp=#0x10000004600' 2024-09-21T00:30:10.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38544 fp=#0x10000004601' 2024-09-21T00:30:10.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38576 fp=#0x10000002a18' 2024-09-21T00:30:10.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38608 fp=#0x10000002b32' 2024-09-21T00:30:10.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38704 fp=#0x100000048b4' 2024-09-21T00:30:10.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38736 fp=#0x10000001fa3' 2024-09-21T00:30:10.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38800 fp=#0x100000010aa' 2024-09-21T00:30:10.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38864 fp=#0x100000041ec' 2024-09-21T00:30:10.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38960 fp=#0x10000004028' 2024-09-21T00:30:10.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38992 fp=#0x100000041a8' 2024-09-21T00:30:10.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39056 fp=#0x10000001fc3' 2024-09-21T00:30:10.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39088 fp=#0x10000002004' 2024-09-21T00:30:10.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39120 fp=#0x10000001cab' 2024-09-21T00:30:10.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39152 fp=#0x10000001fff' 2024-09-21T00:30:10.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39184 fp=#0x10000004501' 2024-09-21T00:30:10.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39280 fp=#0x1000000486e' 2024-09-21T00:30:10.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39312 fp=#0x10000004896' 2024-09-21T00:30:10.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39344 fp=#0x100000048ab' 2024-09-21T00:30:10.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39376 fp=#0x10000005ade' 2024-09-21T00:30:10.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39408 fp=#0x10000005ae0' 2024-09-21T00:30:10.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39440 fp=#0x10000005e2b' 2024-09-21T00:30:10.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39472 fp=#0x100000059de' 2024-09-21T00:30:10.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39504 fp=#0x10000005aaf' 2024-09-21T00:30:10.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39536 fp=#0x100000059d5' 2024-09-21T00:30:10.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39568 fp=#0x100000050f1' 2024-09-21T00:30:10.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39600 fp=#0x100000050b7' 2024-09-21T00:30:10.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39632 fp=#0x100000050f0' 2024-09-21T00:30:10.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39664 fp=#0x100000050df' 2024-09-21T00:30:10.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39696 fp=#0x10000005d8a' 2024-09-21T00:30:10.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39760 fp=#0x10000005b43' 2024-09-21T00:30:10.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39792 fp=#0x10000005b3e' 2024-09-21T00:30:10.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39824 fp=#0x100000062f8' 2024-09-21T00:30:10.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39856 fp=#0x10000006363' 2024-09-21T00:30:10.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39888 fp=#0x10000006305' 2024-09-21T00:30:10.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39920 fp=#0x10000004eb1' 2024-09-21T00:30:10.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40016 fp=#0x100000061db' 2024-09-21T00:30:10.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40048 fp=#0x1000000629a' 2024-09-21T00:30:10.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40080 fp=#0x100000061e6' 2024-09-21T00:30:10.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40112 fp=#0x1000000626a' 2024-09-21T00:30:10.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40144 fp=#0x10000002672' 2024-09-21T00:30:10.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40176 fp=#0x100000026bd' 2024-09-21T00:30:10.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40208 fp=#0x1000000260c' 2024-09-21T00:30:10.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40240 fp=#0x10000004319' 2024-09-21T00:30:10.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40272 fp=#0x1000000430c' 2024-09-21T00:30:10.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40304 fp=#0x100000042e5' 2024-09-21T00:30:10.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40336 fp=#0x100000042f8' 2024-09-21T00:30:10.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40368 fp=#0x10000000adc' 2024-09-21T00:30:10.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40400 fp=#0x10000004293' 2024-09-21T00:30:10.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40528 fp=#0x10000002838' 2024-09-21T00:30:10.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40656 fp=#0x10000004935' 2024-09-21T00:30:10.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40688 fp=#0x1000000492b' 2024-09-21T00:30:10.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40720 fp=#0x10000005ea1' 2024-09-21T00:30:10.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40752 fp=#0x1000000485e' 2024-09-21T00:30:10.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40784 fp=#0x10000004921' 2024-09-21T00:30:10.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40848 fp=#0x1000000304c' 2024-09-21T00:30:10.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40880 fp=#0x100000063c0' 2024-09-21T00:30:10.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40976 fp=#0x10000004a42' 2024-09-21T00:30:10.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41008 fp=#0x10000005135' 2024-09-21T00:30:10.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41040 fp=#0x10000004a40' 2024-09-21T00:30:10.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41072 fp=#0x10000004a14' 2024-09-21T00:30:10.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41104 fp=#0x10000005116' 2024-09-21T00:30:10.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30609 fp=#0x1' 2024-09-21T00:30:10.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30769 fp=#0x10000005192' 2024-09-21T00:30:10.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30801 fp=#0x100000054a1' 2024-09-21T00:30:10.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30833 fp=#0x100000020bb' 2024-09-21T00:30:10.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30865 fp=#0x10000003534' 2024-09-21T00:30:10.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32689 fp=#0x1000000577e' 2024-09-21T00:30:10.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32721 fp=#0x100000057a3' 2024-09-21T00:30:10.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32753 fp=#0x10000005695' 2024-09-21T00:30:10.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32785 fp=#0x10000005081' 2024-09-21T00:30:10.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32881 fp=#0x10000005031' 2024-09-21T00:30:10.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32913 fp=#0x10000005036' 2024-09-21T00:30:10.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33009 fp=#0x10000005d16' 2024-09-21T00:30:10.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33041 fp=#0x10000005cde' 2024-09-21T00:30:10.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33073 fp=#0x10000005cee' 2024-09-21T00:30:10.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33137 fp=#0x10000006055' 2024-09-21T00:30:10.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33201 fp=#0x1000000667b' 2024-09-21T00:30:10.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33233 fp=#0x10000006054' 2024-09-21T00:30:10.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33297 fp=#0x100000067d7' 2024-09-21T00:30:10.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33329 fp=#0x10000006812' 2024-09-21T00:30:10.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33361 fp=#0x100000067dc' 2024-09-21T00:30:10.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33393 fp=#0x10000003da9' 2024-09-21T00:30:10.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33425 fp=#0x10000003e71' 2024-09-21T00:30:10.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33521 fp=#0x100000047dd' 2024-09-21T00:30:10.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33617 fp=#0x10000005a97' 2024-09-21T00:30:10.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33649 fp=#0x100000055a2' 2024-09-21T00:30:10.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33681 fp=#0x10000005a66' 2024-09-21T00:30:10.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33745 fp=#0x10000006797' 2024-09-21T00:30:10.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33777 fp=#0x10000006788' 2024-09-21T00:30:10.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33937 fp=#0x10000006431' 2024-09-21T00:30:10.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33969 fp=#0x10000006410' 2024-09-21T00:30:10.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34001 fp=#0x10000006437' 2024-09-21T00:30:10.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34033 fp=#0x10000006452' 2024-09-21T00:30:10.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34065 fp=#0x10000006429' 2024-09-21T00:30:10.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34097 fp=#0x1000000320c' 2024-09-21T00:30:10.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34129 fp=#0x100000064a9' 2024-09-21T00:30:10.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34161 fp=#0x10000006267' 2024-09-21T00:30:10.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34193 fp=#0x1000000646d' 2024-09-21T00:30:10.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34225 fp=#0x10000006458' 2024-09-21T00:30:10.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34257 fp=#0x10000006276' 2024-09-21T00:30:10.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34289 fp=#0x10000006219' 2024-09-21T00:30:10.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34321 fp=#0x100000056cc' 2024-09-21T00:30:10.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34353 fp=#0x10000002698' 2024-09-21T00:30:10.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34385 fp=#0x100000061c5' 2024-09-21T00:30:10.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34417 fp=#0x100000056a2' 2024-09-21T00:30:10.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34449 fp=#0x10000002661' 2024-09-21T00:30:10.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34481 fp=#0x10000002534' 2024-09-21T00:30:10.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34513 fp=#0x10000002692' 2024-09-21T00:30:10.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34545 fp=#0x10000006347' 2024-09-21T00:30:10.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34577 fp=#0x10000006621' 2024-09-21T00:30:10.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34609 fp=#0x1000000636c' 2024-09-21T00:30:10.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34641 fp=#0x1000000633b' 2024-09-21T00:30:10.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34673 fp=#0x100000062e5' 2024-09-21T00:30:10.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34769 fp=#0x100000057c2' 2024-09-21T00:30:10.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34801 fp=#0x100000064bf' 2024-09-21T00:30:10.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34833 fp=#0x100000064c9' 2024-09-21T00:30:10.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34865 fp=#0x10000006504' 2024-09-21T00:30:10.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34897 fp=#0x100000052a6' 2024-09-21T00:30:10.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34961 fp=#0x100000052da' 2024-09-21T00:30:10.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34993 fp=#0x100000052c0' 2024-09-21T00:30:10.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35025 fp=#0x10000006960' 2024-09-21T00:30:10.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35121 fp=#0x100000060c8' 2024-09-21T00:30:10.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35185 fp=#0x100000060bb' 2024-09-21T00:30:10.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35217 fp=#0x100000060df' 2024-09-21T00:30:10.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35249 fp=#0x10000003f22' 2024-09-21T00:30:10.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35281 fp=#0x10000003f9d' 2024-09-21T00:30:10.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35345 fp=#0x10000005320' 2024-09-21T00:30:10.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35377 fp=#0x10000005397' 2024-09-21T00:30:10.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35441 fp=#0x10000005f85' 2024-09-21T00:30:10.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35537 fp=#0x10000005f81' 2024-09-21T00:30:10.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35569 fp=#0x10000005f88' 2024-09-21T00:30:10.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35633 fp=#0x10000005baf' 2024-09-21T00:30:10.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35665 fp=#0x10000005b63' 2024-09-21T00:30:10.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35697 fp=#0x10000004957' 2024-09-21T00:30:10.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35729 fp=#0x10000004f8d' 2024-09-21T00:30:10.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35761 fp=#0x10000005008' 2024-09-21T00:30:10.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35793 fp=#0x10000004fb6' 2024-09-21T00:30:10.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35825 fp=#0x10000004f9b' 2024-09-21T00:30:10.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35857 fp=#0x10000004f67' 2024-09-21T00:30:10.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35889 fp=#0x10000006717' 2024-09-21T00:30:10.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35921 fp=#0x10000006715' 2024-09-21T00:30:10.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35953 fp=#0x1000000687d' 2024-09-21T00:30:10.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35985 fp=#0x1000000684d' 2024-09-21T00:30:10.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36017 fp=#0x10000006869' 2024-09-21T00:30:10.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36081 fp=#0x10000005f64' 2024-09-21T00:30:10.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36209 fp=#0x100000068c8' 2024-09-21T00:30:10.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36241 fp=#0x100000068db' 2024-09-21T00:30:10.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36305 fp=#0x10000006617' 2024-09-21T00:30:10.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36337 fp=#0x1000000659d' 2024-09-21T00:30:10.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36369 fp=#0x100000065a6' 2024-09-21T00:30:10.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36433 fp=#0x10000006159' 2024-09-21T00:30:10.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36529 fp=#0x1000000602f' 2024-09-21T00:30:10.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36561 fp=#0x10000005ac5' 2024-09-21T00:30:10.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36593 fp=#0x10000005ace' 2024-09-21T00:30:10.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36657 fp=#0x10000004807' 2024-09-21T00:30:10.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36689 fp=#0x10000004905' 2024-09-21T00:30:10.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36721 fp=#0x100000047ac' 2024-09-21T00:30:10.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36753 fp=#0x10000004912' 2024-09-21T00:30:10.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36785 fp=#0x1000000478e' 2024-09-21T00:30:10.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36817 fp=#0x10000005b0c' 2024-09-21T00:30:10.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36913 fp=#0x10000005922' 2024-09-21T00:30:10.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36945 fp=#0x10000005895' 2024-09-21T00:30:10.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36977 fp=#0x100000051d0' 2024-09-21T00:30:10.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37009 fp=#0x10000005261' 2024-09-21T00:30:10.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37041 fp=#0x10000005292' 2024-09-21T00:30:10.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37073 fp=#0x100000051a9' 2024-09-21T00:30:10.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37105 fp=#0x100000051e1' 2024-09-21T00:30:10.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37137 fp=#0x100000053c2' 2024-09-21T00:30:10.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37201 fp=#0x10000005fdf' 2024-09-21T00:30:10.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37233 fp=#0x10000005bc9' 2024-09-21T00:30:10.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37265 fp=#0x10000005c24' 2024-09-21T00:30:10.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37297 fp=#0x10000005c18' 2024-09-21T00:30:10.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37329 fp=#0x10000005c13' 2024-09-21T00:30:10.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37361 fp=#0x10000005c29' 2024-09-21T00:30:10.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37393 fp=#0x10000005f08' 2024-09-21T00:30:10.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37425 fp=#0x10000005ee0' 2024-09-21T00:30:10.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37457 fp=#0x10000004ac5' 2024-09-21T00:30:10.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37489 fp=#0x10000004acb' 2024-09-21T00:30:10.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37521 fp=#0x10000004b0f' 2024-09-21T00:30:10.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37553 fp=#0x10000004b8a' 2024-09-21T00:30:10.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37585 fp=#0x10000005517' 2024-09-21T00:30:10.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37617 fp=#0x10000005509' 2024-09-21T00:30:10.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37649 fp=#0x1000000552a' 2024-09-21T00:30:10.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37681 fp=#0x10000005486' 2024-09-21T00:30:10.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37713 fp=#0x100000057f5' 2024-09-21T00:30:10.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37745 fp=#0x100000057f6' 2024-09-21T00:30:10.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37777 fp=#0x10000005835' 2024-09-21T00:30:10.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37809 fp=#0x100000042d0' 2024-09-21T00:30:10.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37841 fp=#0x10000005814' 2024-09-21T00:30:10.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37873 fp=#0x10000005653' 2024-09-21T00:30:10.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37905 fp=#0x10000001d76' 2024-09-21T00:30:10.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37937 fp=#0x10000001d04' 2024-09-21T00:30:10.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37969 fp=#0x10000001bb2' 2024-09-21T00:30:10.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38001 fp=#0x10000001ce6' 2024-09-21T00:30:10.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38033 fp=#0x10000003cb1' 2024-09-21T00:30:10.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38065 fp=#0x10000003c34' 2024-09-21T00:30:10.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38097 fp=#0x10000003aae' 2024-09-21T00:30:10.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38161 fp=#0x100000060f6' 2024-09-21T00:30:10.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38225 fp=#0x10000005871' 2024-09-21T00:30:10.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38257 fp=#0x10000005874' 2024-09-21T00:30:10.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38289 fp=#0x1000000586d' 2024-09-21T00:30:10.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38321 fp=#0x1000000593d' 2024-09-21T00:30:10.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38353 fp=#0x10000006191' 2024-09-21T00:30:10.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38385 fp=#0x1000000618f' 2024-09-21T00:30:10.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38417 fp=#0x10000002bb9' 2024-09-21T00:30:10.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38449 fp=#0x10000005dc8' 2024-09-21T00:30:10.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38481 fp=#0x100000029fa' 2024-09-21T00:30:10.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38513 fp=#0x100000029f5' 2024-09-21T00:30:10.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38545 fp=#0x10000002b8e' 2024-09-21T00:30:10.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38577 fp=#0x100000045e9' 2024-09-21T00:30:10.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38609 fp=#0x10000002a7d' 2024-09-21T00:30:10.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38737 fp=#0x10000001096' 2024-09-21T00:30:10.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38801 fp=#0x10000001fbb' 2024-09-21T00:30:10.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38833 fp=#0x1000000489f' 2024-09-21T00:30:10.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38865 fp=#0x10000004066' 2024-09-21T00:30:10.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38929 fp=#0x1000000404e' 2024-09-21T00:30:10.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38961 fp=#0x10000004117' 2024-09-21T00:30:10.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38993 fp=#0x1000000405e' 2024-09-21T00:30:10.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39025 fp=#0x10000004131' 2024-09-21T00:30:10.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39057 fp=#0x100000017ea' 2024-09-21T00:30:10.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39089 fp=#0x10000001d49' 2024-09-21T00:30:10.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39153 fp=#0x10000001fba' 2024-09-21T00:30:10.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39185 fp=#0x100000053a4' 2024-09-21T00:30:10.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39217 fp=#0x1000000450f' 2024-09-21T00:30:10.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39249 fp=#0x100000044fd' 2024-09-21T00:30:10.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39281 fp=#0x100000048e6' 2024-09-21T00:30:10.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39313 fp=#0x100000048a0' 2024-09-21T00:30:10.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39345 fp=#0x10000004812' 2024-09-21T00:30:10.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39377 fp=#0x10000005e20' 2024-09-21T00:30:10.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39409 fp=#0x10000005a50' 2024-09-21T00:30:10.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39441 fp=#0x10000005e28' 2024-09-21T00:30:10.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39473 fp=#0x10000005e1a' 2024-09-21T00:30:10.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39505 fp=#0x10000005e0b' 2024-09-21T00:30:10.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39537 fp=#0x10000005ad6' 2024-09-21T00:30:10.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39569 fp=#0x10000005094' 2024-09-21T00:30:10.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39601 fp=#0x100000050bb' 2024-09-21T00:30:10.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39697 fp=#0x10000005d92' 2024-09-21T00:30:10.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39761 fp=#0x10000005b54' 2024-09-21T00:30:10.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39793 fp=#0x10000005b39' 2024-09-21T00:30:10.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39825 fp=#0x10000006358' 2024-09-21T00:30:10.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39857 fp=#0x1000000630b' 2024-09-21T00:30:10.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39889 fp=#0x1000000634f' 2024-09-21T00:30:10.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39921 fp=#0x10000004eda' 2024-09-21T00:30:10.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39953 fp=#0x10000004eb3' 2024-09-21T00:30:10.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39985 fp=#0x10000004eef' 2024-09-21T00:30:10.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40017 fp=#0x10000006298' 2024-09-21T00:30:10.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40049 fp=#0x10000006286' 2024-09-21T00:30:10.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40081 fp=#0x100000061e9' 2024-09-21T00:30:10.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40113 fp=#0x10000006263' 2024-09-21T00:30:10.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40145 fp=#0x100000025fb' 2024-09-21T00:30:10.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40177 fp=#0x1000000266a' 2024-09-21T00:30:10.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40209 fp=#0x1000000265d' 2024-09-21T00:30:10.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40273 fp=#0x10000001430' 2024-09-21T00:30:10.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40305 fp=#0x1000000087f' 2024-09-21T00:30:10.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40337 fp=#0x10000004306' 2024-09-21T00:30:10.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40369 fp=#0x1000000142a' 2024-09-21T00:30:10.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40401 fp=#0x100000042fe' 2024-09-21T00:30:10.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40593 fp=#0x100000037ef' 2024-09-21T00:30:10.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40657 fp=#0x10000004927' 2024-09-21T00:30:10.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40689 fp=#0x10000004880' 2024-09-21T00:30:10.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40721 fp=#0x100000047fe' 2024-09-21T00:30:10.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40753 fp=#0x100000048d8' 2024-09-21T00:30:10.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40817 fp=#0x100000063d7' 2024-09-21T00:30:10.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40977 fp=#0x100000049ec' 2024-09-21T00:30:10.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41009 fp=#0x10000005134' 2024-09-21T00:30:10.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41041 fp=#0x10000005131' 2024-09-21T00:30:10.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41073 fp=#0x100000049ce' 2024-09-21T00:30:10.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41105 fp=#0x1000000513f' 2024-09-21T00:30:10.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30610 fp=#0x10000000000' 2024-09-21T00:30:10.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30802 fp=#0x100000029c3' 2024-09-21T00:30:10.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30834 fp=#0x10000004908' 2024-09-21T00:30:10.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30866 fp=#0x100000061d8' 2024-09-21T00:30:10.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32690 fp=#0x1000000575f' 2024-09-21T00:30:10.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32722 fp=#0x10000005762' 2024-09-21T00:30:10.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32754 fp=#0x10000005717' 2024-09-21T00:30:10.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32786 fp=#0x10000005053' 2024-09-21T00:30:10.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32850 fp=#0x10000005058' 2024-09-21T00:30:10.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32978 fp=#0x10000005073' 2024-09-21T00:30:10.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33010 fp=#0x10000005c3e' 2024-09-21T00:30:10.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33042 fp=#0x10000005c52' 2024-09-21T00:30:10.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33106 fp=#0x10000005c70' 2024-09-21T00:30:10.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33202 fp=#0x10000006675' 2024-09-21T00:30:10.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33266 fp=#0x10000006032' 2024-09-21T00:30:10.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33298 fp=#0x10000006820' 2024-09-21T00:30:10.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33330 fp=#0x100000067cb' 2024-09-21T00:30:10.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33362 fp=#0x100000067fd' 2024-09-21T00:30:10.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33394 fp=#0x10000003e03' 2024-09-21T00:30:10.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33458 fp=#0x10000003ef4' 2024-09-21T00:30:10.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33490 fp=#0x10000003dd1' 2024-09-21T00:30:10.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33522 fp=#0x100000047e7' 2024-09-21T00:30:10.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33554 fp=#0x10000005a5a' 2024-09-21T00:30:10.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33618 fp=#0x10000005a44' 2024-09-21T00:30:10.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33682 fp=#0x10000005a4e' 2024-09-21T00:30:10.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33746 fp=#0x100000067a9' 2024-09-21T00:30:10.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33778 fp=#0x10000006783' 2024-09-21T00:30:10.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33938 fp=#0x1000000643a' 2024-09-21T00:30:10.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33970 fp=#0x10000006440' 2024-09-21T00:30:10.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34002 fp=#0x1000000641a' 2024-09-21T00:30:10.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34034 fp=#0x10000004718' 2024-09-21T00:30:10.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34098 fp=#0x1000000321c' 2024-09-21T00:30:10.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34130 fp=#0x10000006295' 2024-09-21T00:30:10.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34162 fp=#0x10000006477' 2024-09-21T00:30:10.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34194 fp=#0x1000000623f' 2024-09-21T00:30:10.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34226 fp=#0x1000000626c' 2024-09-21T00:30:10.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34258 fp=#0x10000006471' 2024-09-21T00:30:10.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34290 fp=#0x10000006466' 2024-09-21T00:30:10.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34322 fp=#0x100000026c5' 2024-09-21T00:30:10.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34354 fp=#0x10000005680' 2024-09-21T00:30:10.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34386 fp=#0x10000003074' 2024-09-21T00:30:10.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34418 fp=#0x1000000265e' 2024-09-21T00:30:10.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34450 fp=#0x1000000266c' 2024-09-21T00:30:10.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34482 fp=#0x100000056c8' 2024-09-21T00:30:10.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34514 fp=#0x1000000662a' 2024-09-21T00:30:10.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34546 fp=#0x1000000639f' 2024-09-21T00:30:10.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34578 fp=#0x100000062bd' 2024-09-21T00:30:10.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34610 fp=#0x10000006306' 2024-09-21T00:30:10.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34642 fp=#0x100000063a3' 2024-09-21T00:30:10.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34674 fp=#0x10000006328' 2024-09-21T00:30:10.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34770 fp=#0x100000064c8' 2024-09-21T00:30:10.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34802 fp=#0x100000064f1' 2024-09-21T00:30:10.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34834 fp=#0x100000057d1' 2024-09-21T00:30:10.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34866 fp=#0x100000057c6' 2024-09-21T00:30:10.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34898 fp=#0x1000000532e' 2024-09-21T00:30:10.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34930 fp=#0x100000052eb' 2024-09-21T00:30:10.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34962 fp=#0x1000000534d' 2024-09-21T00:30:10.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34994 fp=#0x1000000645d' 2024-09-21T00:30:10.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35026 fp=#0x10000006955' 2024-09-21T00:30:10.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35058 fp=#0x10000006086' 2024-09-21T00:30:10.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35186 fp=#0x100000060c0' 2024-09-21T00:30:10.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35218 fp=#0x100000060da' 2024-09-21T00:30:10.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35250 fp=#0x10000003fc2' 2024-09-21T00:30:10.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35282 fp=#0x10000003fd6' 2024-09-21T00:30:10.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35474 fp=#0x10000005fcb' 2024-09-21T00:30:10.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35602 fp=#0x10000005be2' 2024-09-21T00:30:10.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35634 fp=#0x10000005beb' 2024-09-21T00:30:10.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35666 fp=#0x10000005bca' 2024-09-21T00:30:10.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35698 fp=#0x10000004956' 2024-09-21T00:30:10.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35730 fp=#0x10000005004' 2024-09-21T00:30:10.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35762 fp=#0x10000004fac' 2024-09-21T00:30:10.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35794 fp=#0x10000004f41' 2024-09-21T00:30:10.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35826 fp=#0x10000004f91' 2024-09-21T00:30:10.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35858 fp=#0x10000004ff4' 2024-09-21T00:30:10.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35890 fp=#0x10000006750' 2024-09-21T00:30:10.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35922 fp=#0x10000006714' 2024-09-21T00:30:10.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35954 fp=#0x1000000683b' 2024-09-21T00:30:10.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35986 fp=#0x10000006857' 2024-09-21T00:30:10.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36018 fp=#0x10000006865' 2024-09-21T00:30:10.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36050 fp=#0x10000005f42' 2024-09-21T00:30:10.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36082 fp=#0x10000005cc3' 2024-09-21T00:30:10.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36210 fp=#0x100000068be' 2024-09-21T00:30:10.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36242 fp=#0x100000068c3' 2024-09-21T00:30:10.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36274 fp=#0x10000006596' 2024-09-21T00:30:10.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36402 fp=#0x100000065ff' 2024-09-21T00:30:10.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36434 fp=#0x10000006154' 2024-09-21T00:30:10.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36466 fp=#0x10000006147' 2024-09-21T00:30:10.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36562 fp=#0x10000005ad4' 2024-09-21T00:30:10.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36626 fp=#0x10000005a9b' 2024-09-21T00:30:10.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36658 fp=#0x100000048ff' 2024-09-21T00:30:10.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36690 fp=#0x10000004830' 2024-09-21T00:30:10.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36722 fp=#0x100000047e4' 2024-09-21T00:30:10.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36754 fp=#0x10000004840' 2024-09-21T00:30:10.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36786 fp=#0x10000004907' 2024-09-21T00:30:10.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36914 fp=#0x10000005912' 2024-09-21T00:30:10.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36978 fp=#0x10000005290' 2024-09-21T00:30:10.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37010 fp=#0x100000051f2' 2024-09-21T00:30:10.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37042 fp=#0x10000005229' 2024-09-21T00:30:10.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37074 fp=#0x100000051f9' 2024-09-21T00:30:10.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37138 fp=#0x100000053ca' 2024-09-21T00:30:10.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37202 fp=#0x10000005fd9' 2024-09-21T00:30:10.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37234 fp=#0x10000005c68' 2024-09-21T00:30:10.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37266 fp=#0x10000005b98' 2024-09-21T00:30:10.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37330 fp=#0x10000005c1c' 2024-09-21T00:30:10.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37362 fp=#0x10000005c16' 2024-09-21T00:30:10.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37394 fp=#0x10000005ee6' 2024-09-21T00:30:10.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37426 fp=#0x10000005ee3' 2024-09-21T00:30:10.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37458 fp=#0x10000004bb8' 2024-09-21T00:30:10.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37490 fp=#0x10000004b12' 2024-09-21T00:30:10.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37522 fp=#0x10000004b6f' 2024-09-21T00:30:10.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37554 fp=#0x10000004a73' 2024-09-21T00:30:10.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37586 fp=#0x1000000554a' 2024-09-21T00:30:10.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37618 fp=#0x10000005526' 2024-09-21T00:30:10.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37650 fp=#0x1000000551f' 2024-09-21T00:30:10.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37682 fp=#0x10000005513' 2024-09-21T00:30:10.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37714 fp=#0x10000004313' 2024-09-21T00:30:10.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37746 fp=#0x1000000423e' 2024-09-21T00:30:10.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37778 fp=#0x10000004233' 2024-09-21T00:30:10.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37810 fp=#0x10000004220' 2024-09-21T00:30:10.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37842 fp=#0x1000000581b' 2024-09-21T00:30:10.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37874 fp=#0x100000042ca' 2024-09-21T00:30:10.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37906 fp=#0x10000001b62' 2024-09-21T00:30:10.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37938 fp=#0x10000001b87' 2024-09-21T00:30:10.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37970 fp=#0x10000001c86' 2024-09-21T00:30:10.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38002 fp=#0x10000001b66' 2024-09-21T00:30:10.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38034 fp=#0x10000003c52' 2024-09-21T00:30:10.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38066 fp=#0x10000003c7f' 2024-09-21T00:30:10.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38130 fp=#0x10000003aa9' 2024-09-21T00:30:10.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38194 fp=#0x1000000586e' 2024-09-21T00:30:10.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38226 fp=#0x10000006170' 2024-09-21T00:30:10.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38258 fp=#0x1000000586b' 2024-09-21T00:30:10.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38290 fp=#0x10000006189' 2024-09-21T00:30:10.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38322 fp=#0x10000005946' 2024-09-21T00:30:10.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38354 fp=#0x10000005861' 2024-09-21T00:30:10.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38386 fp=#0x10000006180' 2024-09-21T00:30:10.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38450 fp=#0x10000002ba0' 2024-09-21T00:30:10.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38482 fp=#0x10000002a6e' 2024-09-21T00:30:10.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38610 fp=#0x100000045ea' 2024-09-21T00:30:10.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38642 fp=#0x10000001647' 2024-09-21T00:30:10.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38802 fp=#0x10000001fd9' 2024-09-21T00:30:10.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38834 fp=#0x10000001f8d' 2024-09-21T00:30:10.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38866 fp=#0x100000041e6' 2024-09-21T00:30:10.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38898 fp=#0x10000004035' 2024-09-21T00:30:10.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38930 fp=#0x10000004062' 2024-09-21T00:30:10.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39058 fp=#0x100000036c5' 2024-09-21T00:30:10.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39090 fp=#0x10000001868' 2024-09-21T00:30:10.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39122 fp=#0x100000017e6' 2024-09-21T00:30:10.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39154 fp=#0x10000001852' 2024-09-21T00:30:10.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39186 fp=#0x100000044fa' 2024-09-21T00:30:10.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39250 fp=#0x10000004496' 2024-09-21T00:30:10.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39282 fp=#0x10000004826' 2024-09-21T00:30:10.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39314 fp=#0x100000048c8' 2024-09-21T00:30:10.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39346 fp=#0x1000000489b' 2024-09-21T00:30:10.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39378 fp=#0x100000059f4' 2024-09-21T00:30:10.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39410 fp=#0x10000005ad9' 2024-09-21T00:30:10.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39442 fp=#0x10000005e3c' 2024-09-21T00:30:10.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39474 fp=#0x10000005e21' 2024-09-21T00:30:10.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39506 fp=#0x100000059bf' 2024-09-21T00:30:10.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39538 fp=#0x100000059cf' 2024-09-21T00:30:10.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39666 fp=#0x10000005de4' 2024-09-21T00:30:10.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39730 fp=#0x10000005d7d' 2024-09-21T00:30:10.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39794 fp=#0x10000005b52' 2024-09-21T00:30:10.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39826 fp=#0x10000006345' 2024-09-21T00:30:10.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39858 fp=#0x10000006332' 2024-09-21T00:30:10.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39890 fp=#0x10000004ef7' 2024-09-21T00:30:10.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39954 fp=#0x10000004e96' 2024-09-21T00:30:10.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39986 fp=#0x10000004e95' 2024-09-21T00:30:10.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40018 fp=#0x100000061ed' 2024-09-21T00:30:10.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40050 fp=#0x1000000628c' 2024-09-21T00:30:10.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40082 fp=#0x1000000624c' 2024-09-21T00:30:10.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40114 fp=#0x10000006293' 2024-09-21T00:30:10.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40146 fp=#0x100000026d2' 2024-09-21T00:30:10.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40178 fp=#0x100000025f9' 2024-09-21T00:30:10.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40210 fp=#0x10000002664' 2024-09-21T00:30:10.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40242 fp=#0x100000042f4' 2024-09-21T00:30:10.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40274 fp=#0x10000000a95' 2024-09-21T00:30:10.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40306 fp=#0x100000008a0' 2024-09-21T00:30:10.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40338 fp=#0x100000042cb' 2024-09-21T00:30:10.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40370 fp=#0x10000000a05' 2024-09-21T00:30:10.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40402 fp=#0x100000042bf' 2024-09-21T00:30:10.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40466 fp=#0x10000005f2d' 2024-09-21T00:30:10.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40498 fp=#0x10000005902' 2024-09-21T00:30:10.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40530 fp=#0x100000028aa' 2024-09-21T00:30:10.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40658 fp=#0x1000000484d' 2024-09-21T00:30:10.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40690 fp=#0x10000004917' 2024-09-21T00:30:10.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40722 fp=#0x1000000481f' 2024-09-21T00:30:10.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40754 fp=#0x100000048e2' 2024-09-21T00:30:10.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40786 fp=#0x10000005ead' 2024-09-21T00:30:10.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40882 fp=#0x100000063ef' 2024-09-21T00:30:10.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40946 fp=#0x100000063e8' 2024-09-21T00:30:10.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40978 fp=#0x10000004a48' 2024-09-21T00:30:10.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41010 fp=#0x10000004a09' 2024-09-21T00:30:10.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41042 fp=#0x10000005100' 2024-09-21T00:30:10.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41074 fp=#0x10000005111' 2024-09-21T00:30:10.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41106 fp=#0x10000005137' 2024-09-21T00:30:10.914 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30611 fp=#0x10000000001' 2024-09-21T00:30:10.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30771 fp=#0x10000005672' 2024-09-21T00:30:10.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30803 fp=#0x100000057ae' 2024-09-21T00:30:10.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30867 fp=#0x10000004f08' 2024-09-21T00:30:10.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32723 fp=#0x1000000572b' 2024-09-21T00:30:10.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32755 fp=#0x10000005784' 2024-09-21T00:30:10.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32819 fp=#0x10000005077' 2024-09-21T00:30:10.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32915 fp=#0x1000000507d' 2024-09-21T00:30:10.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32947 fp=#0x1000000506f' 2024-09-21T00:30:10.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33011 fp=#0x10000005c44' 2024-09-21T00:30:10.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33043 fp=#0x10000005c6a' 2024-09-21T00:30:10.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33075 fp=#0x10000005c8a' 2024-09-21T00:30:10.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33107 fp=#0x10000005cd8' 2024-09-21T00:30:10.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33203 fp=#0x10000006696' 2024-09-21T00:30:10.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33267 fp=#0x1000000606c' 2024-09-21T00:30:10.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33299 fp=#0x1000000681b' 2024-09-21T00:30:10.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33331 fp=#0x10000006826' 2024-09-21T00:30:10.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33363 fp=#0x1000000681e' 2024-09-21T00:30:10.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33395 fp=#0x10000003ee5' 2024-09-21T00:30:10.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33427 fp=#0x10000003e42' 2024-09-21T00:30:10.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33491 fp=#0x10000003e60' 2024-09-21T00:30:10.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33523 fp=#0x100000047f4' 2024-09-21T00:30:10.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33555 fp=#0x10000005661' 2024-09-21T00:30:10.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33587 fp=#0x10000005666' 2024-09-21T00:30:10.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33619 fp=#0x10000005a92' 2024-09-21T00:30:10.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33715 fp=#0x10000005a31' 2024-09-21T00:30:10.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33747 fp=#0x1000000678b' 2024-09-21T00:30:10.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33779 fp=#0x10000006779' 2024-09-21T00:30:10.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33939 fp=#0x1000000642a' 2024-09-21T00:30:10.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34035 fp=#0x100000063fb' 2024-09-21T00:30:10.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34067 fp=#0x10000006412' 2024-09-21T00:30:10.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34099 fp=#0x10000003269' 2024-09-21T00:30:10.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34131 fp=#0x1000000622d' 2024-09-21T00:30:10.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34163 fp=#0x10000006489' 2024-09-21T00:30:10.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34195 fp=#0x10000006256' 2024-09-21T00:30:10.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34227 fp=#0x10000006473' 2024-09-21T00:30:10.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34259 fp=#0x10000006478' 2024-09-21T00:30:10.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34291 fp=#0x100000054ff' 2024-09-21T00:30:10.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34323 fp=#0x10000005686' 2024-09-21T00:30:10.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34355 fp=#0x1000000259d' 2024-09-21T00:30:10.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34387 fp=#0x100000061c2' 2024-09-21T00:30:10.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34419 fp=#0x100000056a9' 2024-09-21T00:30:10.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34451 fp=#0x10000006199' 2024-09-21T00:30:10.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34483 fp=#0x1000000619b' 2024-09-21T00:30:10.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34515 fp=#0x10000002530' 2024-09-21T00:30:10.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34547 fp=#0x10000006312' 2024-09-21T00:30:10.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34579 fp=#0x10000006366' 2024-09-21T00:30:10.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34611 fp=#0x100000062d9' 2024-09-21T00:30:10.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34643 fp=#0x100000062c2' 2024-09-21T00:30:10.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34675 fp=#0x1000000632b' 2024-09-21T00:30:10.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34771 fp=#0x100000057b0' 2024-09-21T00:30:10.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34803 fp=#0x100000057b6' 2024-09-21T00:30:10.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34835 fp=#0x100000057b2' 2024-09-21T00:30:10.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34867 fp=#0x100000057df' 2024-09-21T00:30:10.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34963 fp=#0x1000000694a' 2024-09-21T00:30:10.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34995 fp=#0x100000052be' 2024-09-21T00:30:10.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35027 fp=#0x1000000694b' 2024-09-21T00:30:10.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35059 fp=#0x10000006083' 2024-09-21T00:30:10.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35091 fp=#0x100000060dd' 2024-09-21T00:30:10.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35155 fp=#0x100000060d1' 2024-09-21T00:30:10.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35187 fp=#0x10000002c13' 2024-09-21T00:30:10.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35251 fp=#0x10000003fd2' 2024-09-21T00:30:10.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35315 fp=#0x10000003f25' 2024-09-21T00:30:10.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35347 fp=#0x10000005396' 2024-09-21T00:30:10.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35411 fp=#0x1000000531f' 2024-09-21T00:30:10.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35443 fp=#0x10000005fc5' 2024-09-21T00:30:10.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35475 fp=#0x10000005f94' 2024-09-21T00:30:10.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35603 fp=#0x10000005bd7' 2024-09-21T00:30:10.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35635 fp=#0x10000005bbb' 2024-09-21T00:30:10.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35667 fp=#0x10000005b67' 2024-09-21T00:30:10.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35699 fp=#0x10000006505' 2024-09-21T00:30:10.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35731 fp=#0x10000004f4d' 2024-09-21T00:30:10.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35763 fp=#0x10000004ffa' 2024-09-21T00:30:10.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35795 fp=#0x10000004f18' 2024-09-21T00:30:10.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35827 fp=#0x10000004f46' 2024-09-21T00:30:10.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35859 fp=#0x10000004f23' 2024-09-21T00:30:10.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35891 fp=#0x100000066fe' 2024-09-21T00:30:10.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35923 fp=#0x10000006742' 2024-09-21T00:30:10.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35955 fp=#0x1000000684f' 2024-09-21T00:30:10.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35987 fp=#0x10000006889' 2024-09-21T00:30:10.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36019 fp=#0x1000000684e' 2024-09-21T00:30:10.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36083 fp=#0x10000005cce' 2024-09-21T00:30:10.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36147 fp=#0x10000005f67' 2024-09-21T00:30:10.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36179 fp=#0x1000000689c' 2024-09-21T00:30:10.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36211 fp=#0x100000068d0' 2024-09-21T00:30:10.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36243 fp=#0x100000068b5' 2024-09-21T00:30:10.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36275 fp=#0x10000006592' 2024-09-21T00:30:10.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36307 fp=#0x10000006605' 2024-09-21T00:30:10.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36339 fp=#0x100000065a2' 2024-09-21T00:30:10.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36403 fp=#0x100000065ef' 2024-09-21T00:30:10.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36435 fp=#0x10000006151' 2024-09-21T00:30:10.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36467 fp=#0x1000000615a' 2024-09-21T00:30:10.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36531 fp=#0x10000006000' 2024-09-21T00:30:10.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36563 fp=#0x10000005a18' 2024-09-21T00:30:10.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36595 fp=#0x100000059d1' 2024-09-21T00:30:10.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36627 fp=#0x100000059ae' 2024-09-21T00:30:10.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36659 fp=#0x100000047af' 2024-09-21T00:30:10.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36691 fp=#0x100000048e8' 2024-09-21T00:30:10.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36723 fp=#0x100000047ad' 2024-09-21T00:30:10.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36755 fp=#0x100000047b6' 2024-09-21T00:30:10.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36787 fp=#0x10000004770' 2024-09-21T00:30:10.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36819 fp=#0x100000058a6' 2024-09-21T00:30:10.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36979 fp=#0x10000005252' 2024-09-21T00:30:10.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37011 fp=#0x1000000525a' 2024-09-21T00:30:10.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37043 fp=#0x10000005278' 2024-09-21T00:30:10.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37075 fp=#0x100000051d4' 2024-09-21T00:30:10.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37107 fp=#0x10000005258' 2024-09-21T00:30:10.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37203 fp=#0x10000005fd6' 2024-09-21T00:30:10.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37235 fp=#0x10000005c28' 2024-09-21T00:30:10.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37267 fp=#0x10000005c3b' 2024-09-21T00:30:10.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37299 fp=#0x10000005c1d' 2024-09-21T00:30:10.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37331 fp=#0x10000005be3' 2024-09-21T00:30:10.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37395 fp=#0x10000005ef1' 2024-09-21T00:30:10.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37427 fp=#0x10000005edc' 2024-09-21T00:30:10.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37459 fp=#0x10000004bb6' 2024-09-21T00:30:10.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37491 fp=#0x10000004a8e' 2024-09-21T00:30:10.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37523 fp=#0x10000004b86' 2024-09-21T00:30:10.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37555 fp=#0x10000004a5c' 2024-09-21T00:30:10.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37587 fp=#0x1000000547d' 2024-09-21T00:30:10.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37619 fp=#0x100000054c6' 2024-09-21T00:30:10.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37651 fp=#0x100000054b9' 2024-09-21T00:30:10.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37683 fp=#0x1000000551c' 2024-09-21T00:30:10.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37715 fp=#0x1000000432c' 2024-09-21T00:30:10.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37747 fp=#0x10000004317' 2024-09-21T00:30:10.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37779 fp=#0x1000000582d' 2024-09-21T00:30:10.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37811 fp=#0x100000057fe' 2024-09-21T00:30:10.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37843 fp=#0x10000005815' 2024-09-21T00:30:10.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37875 fp=#0x100000042ec' 2024-09-21T00:30:10.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37907 fp=#0x10000001b6d' 2024-09-21T00:30:10.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37939 fp=#0x10000001bbf' 2024-09-21T00:30:10.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37971 fp=#0x10000001bae' 2024-09-21T00:30:10.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38003 fp=#0x10000001b36' 2024-09-21T00:30:10.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38035 fp=#0x10000003ca2' 2024-09-21T00:30:10.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38195 fp=#0x100000059ab' 2024-09-21T00:30:10.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38227 fp=#0x10000006179' 2024-09-21T00:30:10.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38259 fp=#0x1000000616e' 2024-09-21T00:30:10.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38291 fp=#0x10000005855' 2024-09-21T00:30:10.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38323 fp=#0x1000000590f' 2024-09-21T00:30:10.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38355 fp=#0x1000000595e' 2024-09-21T00:30:10.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38387 fp=#0x10000005847' 2024-09-21T00:30:10.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38419 fp=#0x100000029df' 2024-09-21T00:30:10.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38483 fp=#0x10000002a7a' 2024-09-21T00:30:10.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38515 fp=#0x10000002acf' 2024-09-21T00:30:10.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38547 fp=#0x10000002a56' 2024-09-21T00:30:10.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38579 fp=#0x10000002b94' 2024-09-21T00:30:10.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38611 fp=#0x10000002b6d' 2024-09-21T00:30:10.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38739 fp=#0x100000010a1' 2024-09-21T00:30:10.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38803 fp=#0x10000001fbf' 2024-09-21T00:30:10.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38835 fp=#0x1000000108a' 2024-09-21T00:30:10.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38899 fp=#0x10000004181' 2024-09-21T00:30:10.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38931 fp=#0x100000041f4' 2024-09-21T00:30:10.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38963 fp=#0x10000004078' 2024-09-21T00:30:10.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39059 fp=#0x10000001cd8' 2024-09-21T00:30:10.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39091 fp=#0x10000001f8c' 2024-09-21T00:30:10.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39123 fp=#0x10000001cc9' 2024-09-21T00:30:10.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39155 fp=#0x10000001fd3' 2024-09-21T00:30:10.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39187 fp=#0x100000044f1' 2024-09-21T00:30:10.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39219 fp=#0x10000004507' 2024-09-21T00:30:10.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39251 fp=#0x100000044f4' 2024-09-21T00:30:10.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39283 fp=#0x1000000481b' 2024-09-21T00:30:10.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39347 fp=#0x1000000485c' 2024-09-21T00:30:10.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39379 fp=#0x10000005af3' 2024-09-21T00:30:10.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39411 fp=#0x10000005ac9' 2024-09-21T00:30:10.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39443 fp=#0x10000005e11' 2024-09-21T00:30:10.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39475 fp=#0x10000005e04' 2024-09-21T00:30:10.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39507 fp=#0x10000005e0d' 2024-09-21T00:30:10.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39539 fp=#0x100000059d8' 2024-09-21T00:30:10.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39571 fp=#0x100000050f8' 2024-09-21T00:30:10.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39603 fp=#0x100000050e2' 2024-09-21T00:30:10.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39635 fp=#0x100000050b3' 2024-09-21T00:30:10.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39763 fp=#0x10000005b5b' 2024-09-21T00:30:10.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39795 fp=#0x10000005b4e' 2024-09-21T00:30:10.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39827 fp=#0x10000006370' 2024-09-21T00:30:10.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39859 fp=#0x10000006365' 2024-09-21T00:30:10.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39891 fp=#0x10000004ef2' 2024-09-21T00:30:10.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39923 fp=#0x10000004edd' 2024-09-21T00:30:10.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40019 fp=#0x10000006264' 2024-09-21T00:30:11.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40051 fp=#0x1000000627d' 2024-09-21T00:30:11.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40083 fp=#0x1000000620e' 2024-09-21T00:30:11.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40115 fp=#0x100000061de' 2024-09-21T00:30:11.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40147 fp=#0x100000026b1' 2024-09-21T00:30:11.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40179 fp=#0x100000025ff' 2024-09-21T00:30:11.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40211 fp=#0x100000026a8' 2024-09-21T00:30:11.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40243 fp=#0x100000042af' 2024-09-21T00:30:11.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40275 fp=#0x100000008f2' 2024-09-21T00:30:11.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40307 fp=#0x10000000b20' 2024-09-21T00:30:11.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40371 fp=#0x10000000fa1' 2024-09-21T00:30:11.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40403 fp=#0x100000042d1' 2024-09-21T00:30:11.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40435 fp=#0x1000000597c' 2024-09-21T00:30:11.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40467 fp=#0x1000000597b' 2024-09-21T00:30:11.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40563 fp=#0x10000003801' 2024-09-21T00:30:11.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40595 fp=#0x100000036c7' 2024-09-21T00:30:11.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40659 fp=#0x10000005ed7' 2024-09-21T00:30:11.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40691 fp=#0x100000048fa' 2024-09-21T00:30:11.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40723 fp=#0x1000000494f' 2024-09-21T00:30:11.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40787 fp=#0x100000047ff' 2024-09-21T00:30:11.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40819 fp=#0x100000031cf' 2024-09-21T00:30:11.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40883 fp=#0x100000063e5' 2024-09-21T00:30:11.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40947 fp=#0x100000063df' 2024-09-21T00:30:11.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40979 fp=#0x100000049b0' 2024-09-21T00:30:11.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41011 fp=#0x100000049d6' 2024-09-21T00:30:11.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41043 fp=#0x10000005118' 2024-09-21T00:30:11.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41075 fp=#0x100000049be' 2024-09-21T00:30:11.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41107 fp=#0x10000005132' 2024-09-21T00:30:11.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30612 fp=#0x100000001fa' 2024-09-21T00:30:11.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30772 fp=#0x100000053bd' 2024-09-21T00:30:11.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30804 fp=#0x10000002e09' 2024-09-21T00:30:11.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30868 fp=#0x10000001d8d' 2024-09-21T00:30:11.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32692 fp=#0x10000005793' 2024-09-21T00:30:11.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32724 fp=#0x100000056a8' 2024-09-21T00:30:11.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32756 fp=#0x100000056a6' 2024-09-21T00:30:11.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32788 fp=#0x10000005080' 2024-09-21T00:30:11.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32884 fp=#0x10000005065' 2024-09-21T00:30:11.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32916 fp=#0x1000000505a' 2024-09-21T00:30:11.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32948 fp=#0x10000005068' 2024-09-21T00:30:11.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32980 fp=#0x10000005c4a' 2024-09-21T00:30:11.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33012 fp=#0x10000005c80' 2024-09-21T00:30:11.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33044 fp=#0x10000005c77' 2024-09-21T00:30:11.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33076 fp=#0x10000005cbe' 2024-09-21T00:30:11.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33108 fp=#0x10000005c89' 2024-09-21T00:30:11.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33204 fp=#0x10000006068' 2024-09-21T00:30:11.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33236 fp=#0x10000006050' 2024-09-21T00:30:11.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33268 fp=#0x10000006073' 2024-09-21T00:30:11.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33300 fp=#0x10000006807' 2024-09-21T00:30:11.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33332 fp=#0x100000067f6' 2024-09-21T00:30:11.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33364 fp=#0x10000006810' 2024-09-21T00:30:11.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33396 fp=#0x10000003ef7' 2024-09-21T00:30:11.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33428 fp=#0x10000003e5d' 2024-09-21T00:30:11.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33492 fp=#0x10000003e2b' 2024-09-21T00:30:11.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33524 fp=#0x100000047f9' 2024-09-21T00:30:11.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33556 fp=#0x1000000567d' 2024-09-21T00:30:11.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33620 fp=#0x10000005aa7' 2024-09-21T00:30:11.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33652 fp=#0x10000005a02' 2024-09-21T00:30:11.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33716 fp=#0x100000067b6' 2024-09-21T00:30:11.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33748 fp=#0x100000067a2' 2024-09-21T00:30:11.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33780 fp=#0x100000067b4' 2024-09-21T00:30:11.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33940 fp=#0x10000006408' 2024-09-21T00:30:11.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33972 fp=#0x1000000643d' 2024-09-21T00:30:11.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34004 fp=#0x10000004733' 2024-09-21T00:30:11.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34036 fp=#0x10000004759' 2024-09-21T00:30:11.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34068 fp=#0x10000004741' 2024-09-21T00:30:11.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34132 fp=#0x10000006482' 2024-09-21T00:30:11.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34164 fp=#0x10000006470' 2024-09-21T00:30:11.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34196 fp=#0x100000062b3' 2024-09-21T00:30:11.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34228 fp=#0x10000006231' 2024-09-21T00:30:11.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34260 fp=#0x10000006472' 2024-09-21T00:30:11.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34292 fp=#0x1000000619f' 2024-09-21T00:30:11.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34324 fp=#0x100000026db' 2024-09-21T00:30:11.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34356 fp=#0x100000061a9' 2024-09-21T00:30:11.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34388 fp=#0x100000061d6' 2024-09-21T00:30:11.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34420 fp=#0x100000061ce' 2024-09-21T00:30:11.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34516 fp=#0x10000002592' 2024-09-21T00:30:11.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34548 fp=#0x1000000633a' 2024-09-21T00:30:11.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34580 fp=#0x10000006399' 2024-09-21T00:30:11.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34612 fp=#0x10000006304' 2024-09-21T00:30:11.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34644 fp=#0x10000006371' 2024-09-21T00:30:11.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34676 fp=#0x10000006352' 2024-09-21T00:30:11.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34772 fp=#0x100000064cb' 2024-09-21T00:30:11.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34804 fp=#0x100000064ee' 2024-09-21T00:30:11.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34836 fp=#0x100000064c5' 2024-09-21T00:30:11.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34868 fp=#0x100000064b3' 2024-09-21T00:30:11.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34900 fp=#0x10000005321' 2024-09-21T00:30:11.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34932 fp=#0x1000000694d' 2024-09-21T00:30:11.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34964 fp=#0x100000052ab' 2024-09-21T00:30:11.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34996 fp=#0x10000005305' 2024-09-21T00:30:11.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35028 fp=#0x100000052d1' 2024-09-21T00:30:11.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35092 fp=#0x100000060dc' 2024-09-21T00:30:11.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35156 fp=#0x100000060d8' 2024-09-21T00:30:11.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35252 fp=#0x10000003f03' 2024-09-21T00:30:11.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35284 fp=#0x10000003f0a' 2024-09-21T00:30:11.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35316 fp=#0x10000003fb3' 2024-09-21T00:30:11.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35412 fp=#0x10000005390' 2024-09-21T00:30:11.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35476 fp=#0x10000005fb9' 2024-09-21T00:30:11.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35604 fp=#0x10000005b79' 2024-09-21T00:30:11.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35636 fp=#0x10000005b6a' 2024-09-21T00:30:11.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35668 fp=#0x10000005b60' 2024-09-21T00:30:11.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35700 fp=#0x1000000490b' 2024-09-21T00:30:11.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35732 fp=#0x10000004f7b' 2024-09-21T00:30:11.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35764 fp=#0x10000004fe8' 2024-09-21T00:30:11.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35796 fp=#0x10000004f3e' 2024-09-21T00:30:11.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35828 fp=#0x10000004f7d' 2024-09-21T00:30:11.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35860 fp=#0x10000004f64' 2024-09-21T00:30:11.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35892 fp=#0x10000006732' 2024-09-21T00:30:11.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35924 fp=#0x1000000673b' 2024-09-21T00:30:11.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35956 fp=#0x1000000684b' 2024-09-21T00:30:11.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35988 fp=#0x1000000683f' 2024-09-21T00:30:11.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36020 fp=#0x10000006837' 2024-09-21T00:30:11.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36052 fp=#0x10000005d73' 2024-09-21T00:30:11.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36084 fp=#0x10000005f59' 2024-09-21T00:30:11.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36116 fp=#0x10000005cc6' 2024-09-21T00:30:11.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36180 fp=#0x100000068cb' 2024-09-21T00:30:11.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36212 fp=#0x100000068a5' 2024-09-21T00:30:11.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36244 fp=#0x1000000688f' 2024-09-21T00:30:11.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36308 fp=#0x1000000659f' 2024-09-21T00:30:11.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36372 fp=#0x10000006599' 2024-09-21T00:30:11.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36404 fp=#0x10000006162' 2024-09-21T00:30:11.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36436 fp=#0x10000006165' 2024-09-21T00:30:11.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36468 fp=#0x10000006152' 2024-09-21T00:30:11.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36500 fp=#0x1000000600a' 2024-09-21T00:30:11.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36660 fp=#0x100000047e6' 2024-09-21T00:30:11.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36692 fp=#0x10000004898' 2024-09-21T00:30:11.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36724 fp=#0x1000000483a' 2024-09-21T00:30:11.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36756 fp=#0x1000000482d' 2024-09-21T00:30:11.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36788 fp=#0x100000047f0' 2024-09-21T00:30:11.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36820 fp=#0x10000005b0b' 2024-09-21T00:30:11.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36852 fp=#0x10000005b0e' 2024-09-21T00:30:11.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36884 fp=#0x10000005b17' 2024-09-21T00:30:11.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36916 fp=#0x100000058f9' 2024-09-21T00:30:11.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36980 fp=#0x10000005294' 2024-09-21T00:30:11.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37012 fp=#0x10000005234' 2024-09-21T00:30:11.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37108 fp=#0x10000005237' 2024-09-21T00:30:11.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37204 fp=#0x10000003d38' 2024-09-21T00:30:11.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37236 fp=#0x10000005c6d' 2024-09-21T00:30:11.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37268 fp=#0x10000005be8' 2024-09-21T00:30:11.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37300 fp=#0x10000005c6b' 2024-09-21T00:30:11.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37332 fp=#0x10000005c54' 2024-09-21T00:30:11.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37364 fp=#0x10000005c4c' 2024-09-21T00:30:11.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37396 fp=#0x10000005ee8' 2024-09-21T00:30:11.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37428 fp=#0x10000005ef7' 2024-09-21T00:30:11.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37460 fp=#0x10000004bc8' 2024-09-21T00:30:11.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37492 fp=#0x10000004b1f' 2024-09-21T00:30:11.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37524 fp=#0x10000004a92' 2024-09-21T00:30:11.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37556 fp=#0x10000004aaf' 2024-09-21T00:30:11.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37588 fp=#0x100000054a4' 2024-09-21T00:30:11.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37620 fp=#0x10000005533' 2024-09-21T00:30:11.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37652 fp=#0x1000000550e' 2024-09-21T00:30:11.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37684 fp=#0x1000000549f' 2024-09-21T00:30:11.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37716 fp=#0x1000000582c' 2024-09-21T00:30:11.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37748 fp=#0x100000042f7' 2024-09-21T00:30:11.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37780 fp=#0x1000000426e' 2024-09-21T00:30:11.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37812 fp=#0x10000004287' 2024-09-21T00:30:11.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37844 fp=#0x100000042c6' 2024-09-21T00:30:11.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37876 fp=#0x10000004264' 2024-09-21T00:30:11.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37908 fp=#0x10000001c28' 2024-09-21T00:30:11.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37940 fp=#0x10000001bf2' 2024-09-21T00:30:11.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37972 fp=#0x10000001b48' 2024-09-21T00:30:11.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38004 fp=#0x10000001bda' 2024-09-21T00:30:11.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38068 fp=#0x10000003c49' 2024-09-21T00:30:11.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38196 fp=#0x10000005953' 2024-09-21T00:30:11.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38228 fp=#0x10000005962' 2024-09-21T00:30:11.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38260 fp=#0x1000000587c' 2024-09-21T00:30:11.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38324 fp=#0x1000000585b' 2024-09-21T00:30:11.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38356 fp=#0x1000000587a' 2024-09-21T00:30:11.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38388 fp=#0x10000005992' 2024-09-21T00:30:11.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38420 fp=#0x10000002a11' 2024-09-21T00:30:11.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38452 fp=#0x10000002a77' 2024-09-21T00:30:11.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38484 fp=#0x100000045fb' 2024-09-21T00:30:11.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38516 fp=#0x10000002b37' 2024-09-21T00:30:11.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38548 fp=#0x10000002b47' 2024-09-21T00:30:11.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38580 fp=#0x10000002a21' 2024-09-21T00:30:11.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38612 fp=#0x10000002b2a' 2024-09-21T00:30:11.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38708 fp=#0x1000000489a' 2024-09-21T00:30:11.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38740 fp=#0x100000048a4' 2024-09-21T00:30:11.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38772 fp=#0x100000010f2' 2024-09-21T00:30:11.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38804 fp=#0x1000000108e' 2024-09-21T00:30:11.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38836 fp=#0x10000001fb2' 2024-09-21T00:30:11.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38900 fp=#0x10000004093' 2024-09-21T00:30:11.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39028 fp=#0x10000004017' 2024-09-21T00:30:11.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39060 fp=#0x10000001cef' 2024-09-21T00:30:11.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39092 fp=#0x10000001c1e' 2024-09-21T00:30:11.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39124 fp=#0x10000001c18' 2024-09-21T00:30:11.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39156 fp=#0x10000001d27' 2024-09-21T00:30:11.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39252 fp=#0x100000044ba' 2024-09-21T00:30:11.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39284 fp=#0x100000048f3' 2024-09-21T00:30:11.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39316 fp=#0x10000004820' 2024-09-21T00:30:11.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39348 fp=#0x1000000482f' 2024-09-21T00:30:11.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39380 fp=#0x10000005ac4' 2024-09-21T00:30:11.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39444 fp=#0x100000059c7' 2024-09-21T00:30:11.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39508 fp=#0x10000005a4d' 2024-09-21T00:30:11.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39540 fp=#0x10000005a2f' 2024-09-21T00:30:11.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39572 fp=#0x1000000509e' 2024-09-21T00:30:11.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39604 fp=#0x1000000509c' 2024-09-21T00:30:11.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39636 fp=#0x100000050f3' 2024-09-21T00:30:11.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39700 fp=#0x10000005ded' 2024-09-21T00:30:11.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39732 fp=#0x10000005d93' 2024-09-21T00:30:11.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39764 fp=#0x10000005b61' 2024-09-21T00:30:11.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39796 fp=#0x10000005b37' 2024-09-21T00:30:11.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39828 fp=#0x10000006325' 2024-09-21T00:30:11.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39860 fp=#0x10000006310' 2024-09-21T00:30:11.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39892 fp=#0x10000004ea6' 2024-09-21T00:30:11.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39924 fp=#0x10000004eeb' 2024-09-21T00:30:11.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39956 fp=#0x10000004e9b' 2024-09-21T00:30:11.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39988 fp=#0x10000004e90' 2024-09-21T00:30:11.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40020 fp=#0x10000006296' 2024-09-21T00:30:11.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40052 fp=#0x1000000621c' 2024-09-21T00:30:11.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40084 fp=#0x1000000625e' 2024-09-21T00:30:11.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40116 fp=#0x10000006238' 2024-09-21T00:30:11.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40148 fp=#0x1000000264f' 2024-09-21T00:30:11.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40180 fp=#0x10000002677' 2024-09-21T00:30:11.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40212 fp=#0x100000026c9' 2024-09-21T00:30:11.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40244 fp=#0x10000000c5d' 2024-09-21T00:30:11.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40276 fp=#0x100000042df' 2024-09-21T00:30:11.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40308 fp=#0x10000004308' 2024-09-21T00:30:11.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40340 fp=#0x100000008ad' 2024-09-21T00:30:11.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40372 fp=#0x10000000af1' 2024-09-21T00:30:11.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40404 fp=#0x10000000884' 2024-09-21T00:30:11.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40532 fp=#0x100000028af' 2024-09-21T00:30:11.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40596 fp=#0x100000036d4' 2024-09-21T00:30:11.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40660 fp=#0x10000004923' 2024-09-21T00:30:11.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40724 fp=#0x10000005eb2' 2024-09-21T00:30:11.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40756 fp=#0x1000000491d' 2024-09-21T00:30:11.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40788 fp=#0x100000048ed' 2024-09-21T00:30:11.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40852 fp=#0x100000063dd' 2024-09-21T00:30:11.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40884 fp=#0x100000063c9' 2024-09-21T00:30:11.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40948 fp=#0x100000063da' 2024-09-21T00:30:11.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40980 fp=#0x10000004a2d' 2024-09-21T00:30:11.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41012 fp=#0x1000000511d' 2024-09-21T00:30:11.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41044 fp=#0x1000000510a' 2024-09-21T00:30:11.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41076 fp=#0x100000050ff' 2024-09-21T00:30:11.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41108 fp=#0x1000000497a' 2024-09-21T00:30:11.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30645 fp=#0x10000000399' 2024-09-21T00:30:11.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30773 fp=#0x10000003cb4' 2024-09-21T00:30:11.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30805 fp=#0x100000005f0' 2024-09-21T00:30:11.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30869 fp=#0x1000000194f' 2024-09-21T00:30:11.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32693 fp=#0x10000005673' 2024-09-21T00:30:11.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32757 fp=#0x10000005751' 2024-09-21T00:30:11.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32789 fp=#0x10000005032' 2024-09-21T00:30:11.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32821 fp=#0x100000066de' 2024-09-21T00:30:11.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32949 fp=#0x10000005052' 2024-09-21T00:30:11.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32981 fp=#0x10000005d01' 2024-09-21T00:30:11.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33013 fp=#0x10000005ce3' 2024-09-21T00:30:11.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33077 fp=#0x10000005cc4' 2024-09-21T00:30:11.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33109 fp=#0x10000005ccc' 2024-09-21T00:30:11.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33141 fp=#0x10000006693' 2024-09-21T00:30:11.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33237 fp=#0x1000000606e' 2024-09-21T00:30:11.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33269 fp=#0x1000000606d' 2024-09-21T00:30:11.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33301 fp=#0x1000000681a' 2024-09-21T00:30:11.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33333 fp=#0x10000006804' 2024-09-21T00:30:11.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33365 fp=#0x10000006822' 2024-09-21T00:30:11.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33397 fp=#0x10000003d63' 2024-09-21T00:30:11.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33429 fp=#0x10000003d6b' 2024-09-21T00:30:11.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33461 fp=#0x10000003e4d' 2024-09-21T00:30:11.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33493 fp=#0x10000003e1e' 2024-09-21T00:30:11.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33525 fp=#0x100000047e0' 2024-09-21T00:30:11.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33557 fp=#0x10000005a95' 2024-09-21T00:30:11.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33685 fp=#0x10000005590' 2024-09-21T00:30:11.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33717 fp=#0x100000067b0' 2024-09-21T00:30:11.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33749 fp=#0x100000067bb' 2024-09-21T00:30:11.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33781 fp=#0x10000006789' 2024-09-21T00:30:11.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33909 fp=#0x1000000479c' 2024-09-21T00:30:11.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33941 fp=#0x10000004785' 2024-09-21T00:30:11.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33973 fp=#0x10000006451' 2024-09-21T00:30:11.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34005 fp=#0x10000006416' 2024-09-21T00:30:11.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34037 fp=#0x10000006404' 2024-09-21T00:30:11.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34069 fp=#0x10000004752' 2024-09-21T00:30:11.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34101 fp=#0x10000003272' 2024-09-21T00:30:11.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34133 fp=#0x1000000625f' 2024-09-21T00:30:11.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34165 fp=#0x10000006211' 2024-09-21T00:30:11.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34197 fp=#0x10000006485' 2024-09-21T00:30:11.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34229 fp=#0x100000064a6' 2024-09-21T00:30:11.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34261 fp=#0x1000000626f' 2024-09-21T00:30:11.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34293 fp=#0x100000026ad' 2024-09-21T00:30:11.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34325 fp=#0x100000061c8' 2024-09-21T00:30:11.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34357 fp=#0x100000061aa' 2024-09-21T00:30:11.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34421 fp=#0x100000026d7' 2024-09-21T00:30:11.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34453 fp=#0x10000002516' 2024-09-21T00:30:11.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34485 fp=#0x100000056bb' 2024-09-21T00:30:11.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34517 fp=#0x1000000264e' 2024-09-21T00:30:11.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34549 fp=#0x100000063ae' 2024-09-21T00:30:11.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34581 fp=#0x1000000638d' 2024-09-21T00:30:11.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34613 fp=#0x1000000630e' 2024-09-21T00:30:11.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34645 fp=#0x10000006355' 2024-09-21T00:30:11.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34677 fp=#0x1000000636a' 2024-09-21T00:30:11.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34773 fp=#0x100000057d6' 2024-09-21T00:30:11.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34805 fp=#0x10000006503' 2024-09-21T00:30:11.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34837 fp=#0x100000057d8' 2024-09-21T00:30:11.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34869 fp=#0x100000064da' 2024-09-21T00:30:11.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34901 fp=#0x10000005b1c' 2024-09-21T00:30:11.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35029 fp=#0x100000052ce' 2024-09-21T00:30:11.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35125 fp=#0x10000006084' 2024-09-21T00:30:11.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35157 fp=#0x100000060b2' 2024-09-21T00:30:11.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35253 fp=#0x10000003fcd' 2024-09-21T00:30:11.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35445 fp=#0x10000005f7c' 2024-09-21T00:30:11.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35477 fp=#0x10000005f9d' 2024-09-21T00:30:11.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35509 fp=#0x10000005f6c' 2024-09-21T00:30:11.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35605 fp=#0x10000005bd3' 2024-09-21T00:30:11.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35637 fp=#0x10000005b75' 2024-09-21T00:30:11.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35669 fp=#0x10000005bb2' 2024-09-21T00:30:11.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35701 fp=#0x1000000492d' 2024-09-21T00:30:11.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35733 fp=#0x10000005022' 2024-09-21T00:30:11.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35765 fp=#0x10000004fb8' 2024-09-21T00:30:11.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35797 fp=#0x10000004f48' 2024-09-21T00:30:11.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35829 fp=#0x1000000500c' 2024-09-21T00:30:11.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35861 fp=#0x10000004f69' 2024-09-21T00:30:11.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35893 fp=#0x1000000674b' 2024-09-21T00:30:11.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35925 fp=#0x1000000670b' 2024-09-21T00:30:11.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35957 fp=#0x10000006830' 2024-09-21T00:30:11.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35989 fp=#0x1000000683e' 2024-09-21T00:30:11.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36021 fp=#0x10000006853' 2024-09-21T00:30:11.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36053 fp=#0x10000005f57' 2024-09-21T00:30:11.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36117 fp=#0x10000005f44' 2024-09-21T00:30:11.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36181 fp=#0x100000068a4' 2024-09-21T00:30:11.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36213 fp=#0x100000068d8' 2024-09-21T00:30:11.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36245 fp=#0x100000068b8' 2024-09-21T00:30:11.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36277 fp=#0x10000006618' 2024-09-21T00:30:11.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36341 fp=#0x100000065f7' 2024-09-21T00:30:11.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36405 fp=#0x1000000614a' 2024-09-21T00:30:11.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36437 fp=#0x1000000401d' 2024-09-21T00:30:11.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36469 fp=#0x1000000614d' 2024-09-21T00:30:11.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36565 fp=#0x10000005acc' 2024-09-21T00:30:11.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36629 fp=#0x10000005a85' 2024-09-21T00:30:11.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36661 fp=#0x10000004929' 2024-09-21T00:30:11.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36693 fp=#0x10000004824' 2024-09-21T00:30:11.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36725 fp=#0x1000000492c' 2024-09-21T00:30:11.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36757 fp=#0x100000047eb' 2024-09-21T00:30:11.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36789 fp=#0x1000000480a' 2024-09-21T00:30:11.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36821 fp=#0x10000005b1a' 2024-09-21T00:30:11.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36853 fp=#0x10000005af4' 2024-09-21T00:30:11.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36917 fp=#0x10000005b05' 2024-09-21T00:30:11.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36949 fp=#0x10000005898' 2024-09-21T00:30:11.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37013 fp=#0x10000005283' 2024-09-21T00:30:11.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37045 fp=#0x100000051f8' 2024-09-21T00:30:11.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37077 fp=#0x10000005224' 2024-09-21T00:30:11.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37141 fp=#0x1000000544b' 2024-09-21T00:30:11.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37205 fp=#0x10000005fe2' 2024-09-21T00:30:11.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37269 fp=#0x10000005c2c' 2024-09-21T00:30:11.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37301 fp=#0x10000005c14' 2024-09-21T00:30:11.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37333 fp=#0x10000005c1f' 2024-09-21T00:30:11.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37397 fp=#0x10000005ef9' 2024-09-21T00:30:11.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37429 fp=#0x10000005ef3' 2024-09-21T00:30:11.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37461 fp=#0x10000004b18' 2024-09-21T00:30:11.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37493 fp=#0x10000004b2a' 2024-09-21T00:30:11.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37525 fp=#0x10000004a91' 2024-09-21T00:30:11.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37557 fp=#0x10000004d57' 2024-09-21T00:30:11.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37589 fp=#0x100000054e5' 2024-09-21T00:30:11.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37621 fp=#0x1000000553a' 2024-09-21T00:30:11.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37653 fp=#0x10000005548' 2024-09-21T00:30:11.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37685 fp=#0x1000000426c' 2024-09-21T00:30:11.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37717 fp=#0x100000042da' 2024-09-21T00:30:11.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37749 fp=#0x100000042ac' 2024-09-21T00:30:11.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37781 fp=#0x10000004279' 2024-09-21T00:30:11.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37813 fp=#0x10000004285' 2024-09-21T00:30:11.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37845 fp=#0x10000004303' 2024-09-21T00:30:11.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37877 fp=#0x10000005809' 2024-09-21T00:30:11.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37909 fp=#0x10000001f64' 2024-09-21T00:30:11.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37941 fp=#0x10000001d79' 2024-09-21T00:30:11.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37973 fp=#0x10000001d20' 2024-09-21T00:30:11.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38005 fp=#0x10000001d0b' 2024-09-21T00:30:11.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38069 fp=#0x10000003c69' 2024-09-21T00:30:11.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38101 fp=#0x10000003aa5' 2024-09-21T00:30:11.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38197 fp=#0x100000059af' 2024-09-21T00:30:11.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38261 fp=#0x1000000618e' 2024-09-21T00:30:11.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38293 fp=#0x100000058e8' 2024-09-21T00:30:11.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38325 fp=#0x100000058f1' 2024-09-21T00:30:11.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38357 fp=#0x1000000616a' 2024-09-21T00:30:11.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38389 fp=#0x10000006186' 2024-09-21T00:30:11.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38421 fp=#0x10000002a07' 2024-09-21T00:30:11.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38453 fp=#0x10000002b8b' 2024-09-21T00:30:11.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38485 fp=#0x10000002b25' 2024-09-21T00:30:11.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38517 fp=#0x100000045d5' 2024-09-21T00:30:11.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38549 fp=#0x10000002b53' 2024-09-21T00:30:11.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38581 fp=#0x100000045c5' 2024-09-21T00:30:11.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38613 fp=#0x10000002ad5' 2024-09-21T00:30:11.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38645 fp=#0x10000001077' 2024-09-21T00:30:11.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38709 fp=#0x100000010d8' 2024-09-21T00:30:11.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38741 fp=#0x100000012c4' 2024-09-21T00:30:11.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38837 fp=#0x100000010dd' 2024-09-21T00:30:11.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38869 fp=#0x10000003fe7' 2024-09-21T00:30:11.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38933 fp=#0x10000004031' 2024-09-21T00:30:11.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38997 fp=#0x10000004054' 2024-09-21T00:30:11.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39029 fp=#0x100000062ef' 2024-09-21T00:30:11.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39061 fp=#0x10000001d96' 2024-09-21T00:30:11.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39093 fp=#0x10000001d10' 2024-09-21T00:30:11.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39125 fp=#0x10000001854' 2024-09-21T00:30:11.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39157 fp=#0x10000001d52' 2024-09-21T00:30:11.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39189 fp=#0x100000044c2' 2024-09-21T00:30:11.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39221 fp=#0x100000044c4' 2024-09-21T00:30:11.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39253 fp=#0x100000044ee' 2024-09-21T00:30:11.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39285 fp=#0x10000004844' 2024-09-21T00:30:11.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39317 fp=#0x1000000483e' 2024-09-21T00:30:11.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39349 fp=#0x100000048d9' 2024-09-21T00:30:11.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39381 fp=#0x10000005af6' 2024-09-21T00:30:11.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39413 fp=#0x10000005a43' 2024-09-21T00:30:11.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39445 fp=#0x100000059df' 2024-09-21T00:30:11.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39477 fp=#0x100000059ec' 2024-09-21T00:30:11.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39509 fp=#0x100000059fa' 2024-09-21T00:30:11.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39573 fp=#0x100000050f6' 2024-09-21T00:30:11.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39605 fp=#0x100000050fa' 2024-09-21T00:30:11.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39669 fp=#0x10000005def' 2024-09-21T00:30:11.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39829 fp=#0x10000006361' 2024-09-21T00:30:11.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39861 fp=#0x10000006307' 2024-09-21T00:30:11.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39893 fp=#0x10000004ef1' 2024-09-21T00:30:11.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39925 fp=#0x10000004eb0' 2024-09-21T00:30:11.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39989 fp=#0x10000004ee6' 2024-09-21T00:30:11.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40021 fp=#0x10000006226' 2024-09-21T00:30:11.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40053 fp=#0x1000000627c' 2024-09-21T00:30:11.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40085 fp=#0x1000000622a' 2024-09-21T00:30:11.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40117 fp=#0x10000006201' 2024-09-21T00:30:11.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40149 fp=#0x1000000261b' 2024-09-21T00:30:11.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40181 fp=#0x10000002624' 2024-09-21T00:30:11.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40213 fp=#0x10000002619' 2024-09-21T00:30:11.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40245 fp=#0x10000000b32' 2024-09-21T00:30:11.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40277 fp=#0x100000042e2' 2024-09-21T00:30:11.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40309 fp=#0x100000009eb' 2024-09-21T00:30:11.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40373 fp=#0x10000000a19' 2024-09-21T00:30:11.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40405 fp=#0x10000000a45' 2024-09-21T00:30:11.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40437 fp=#0x1000000597f' 2024-09-21T00:30:11.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40501 fp=#0x10000005f2c' 2024-09-21T00:30:11.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40533 fp=#0x1000000289b' 2024-09-21T00:30:11.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40565 fp=#0x10000003809' 2024-09-21T00:30:11.204 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40661 fp=#0x10000005eb0' 2024-09-21T00:30:11.204 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40693 fp=#0x10000004874' 2024-09-21T00:30:11.204 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40757 fp=#0x10000004868' 2024-09-21T00:30:11.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40789 fp=#0x10000005ec7' 2024-09-21T00:30:11.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40821 fp=#0x10000005655' 2024-09-21T00:30:11.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40853 fp=#0x100000063dc' 2024-09-21T00:30:11.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40885 fp=#0x100000063e3' 2024-09-21T00:30:11.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40981 fp=#0x10000004a3c' 2024-09-21T00:30:11.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41013 fp=#0x1000000511e' 2024-09-21T00:30:11.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41045 fp=#0x10000005145' 2024-09-21T00:30:11.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41077 fp=#0x10000004999' 2024-09-21T00:30:11.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41109 fp=#0x10000004a03' 2024-09-21T00:30:11.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30614 fp=#0x100000001fb' 2024-09-21T00:30:11.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30774 fp=#0x10000005b84' 2024-09-21T00:30:11.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30806 fp=#0x10000003fe0' 2024-09-21T00:30:11.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30870 fp=#0x100000025f7' 2024-09-21T00:30:11.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32694 fp=#0x1000000575d' 2024-09-21T00:30:11.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32726 fp=#0x10000005736' 2024-09-21T00:30:11.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32758 fp=#0x10000005771' 2024-09-21T00:30:11.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32854 fp=#0x10000005033' 2024-09-21T00:30:11.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32918 fp=#0x10000005055' 2024-09-21T00:30:11.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32950 fp=#0x1000000507f' 2024-09-21T00:30:11.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32982 fp=#0x10000005cf6' 2024-09-21T00:30:11.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33014 fp=#0x10000005c39' 2024-09-21T00:30:11.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33046 fp=#0x10000005d2e' 2024-09-21T00:30:11.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33078 fp=#0x10000005caf' 2024-09-21T00:30:11.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33110 fp=#0x10000005cc0' 2024-09-21T00:30:11.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33206 fp=#0x1000000666b' 2024-09-21T00:30:11.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33238 fp=#0x10000006042' 2024-09-21T00:30:11.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33302 fp=#0x100000067e1' 2024-09-21T00:30:11.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33334 fp=#0x100000067f9' 2024-09-21T00:30:11.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33366 fp=#0x100000067e5' 2024-09-21T00:30:11.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33398 fp=#0x10000003e1b' 2024-09-21T00:30:11.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33430 fp=#0x10000003ec8' 2024-09-21T00:30:11.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33462 fp=#0x10000003de7' 2024-09-21T00:30:11.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33494 fp=#0x10000003e80' 2024-09-21T00:30:11.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33558 fp=#0x100000055af' 2024-09-21T00:30:11.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33686 fp=#0x1000000564e' 2024-09-21T00:30:11.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33718 fp=#0x10000006781' 2024-09-21T00:30:11.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33750 fp=#0x100000067a7' 2024-09-21T00:30:11.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33782 fp=#0x100000067b7' 2024-09-21T00:30:11.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33910 fp=#0x1000000641b' 2024-09-21T00:30:11.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33942 fp=#0x1000000477a' 2024-09-21T00:30:11.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33974 fp=#0x10000006427' 2024-09-21T00:30:11.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34006 fp=#0x10000004742' 2024-09-21T00:30:11.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34038 fp=#0x1000000642b' 2024-09-21T00:30:11.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34070 fp=#0x10000006420' 2024-09-21T00:30:11.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34102 fp=#0x1000000325a' 2024-09-21T00:30:11.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34134 fp=#0x100000062b4' 2024-09-21T00:30:11.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34166 fp=#0x10000006299' 2024-09-21T00:30:11.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34198 fp=#0x10000006222' 2024-09-21T00:30:11.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34230 fp=#0x1000000627a' 2024-09-21T00:30:11.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34294 fp=#0x10000002581' 2024-09-21T00:30:11.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34326 fp=#0x10000002599' 2024-09-21T00:30:11.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34358 fp=#0x10000002685' 2024-09-21T00:30:11.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34390 fp=#0x100000061ac' 2024-09-21T00:30:11.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34422 fp=#0x100000061a3' 2024-09-21T00:30:11.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34486 fp=#0x100000061d7' 2024-09-21T00:30:11.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34518 fp=#0x100000025b1' 2024-09-21T00:30:11.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34550 fp=#0x10000006389' 2024-09-21T00:30:11.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34582 fp=#0x1000000635b' 2024-09-21T00:30:11.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34614 fp=#0x10000006339' 2024-09-21T00:30:11.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34646 fp=#0x100000063a6' 2024-09-21T00:30:11.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34678 fp=#0x1000000638b' 2024-09-21T00:30:11.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34710 fp=#0x10000005f33' 2024-09-21T00:30:11.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34774 fp=#0x100000064e0' 2024-09-21T00:30:11.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34806 fp=#0x100000064d6' 2024-09-21T00:30:11.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34838 fp=#0x100000064dc' 2024-09-21T00:30:11.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34870 fp=#0x100000064e1' 2024-09-21T00:30:11.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34902 fp=#0x1000000696a' 2024-09-21T00:30:11.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34934 fp=#0x10000005322' 2024-09-21T00:30:11.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34966 fp=#0x10000006968' 2024-09-21T00:30:11.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34998 fp=#0x100000052fb' 2024-09-21T00:30:11.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35030 fp=#0x100000052d3' 2024-09-21T00:30:11.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35094 fp=#0x100000060c4' 2024-09-21T00:30:11.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35126 fp=#0x10000002d1a' 2024-09-21T00:30:11.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35190 fp=#0x100000060d5' 2024-09-21T00:30:11.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35222 fp=#0x10000002c1a' 2024-09-21T00:30:11.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35254 fp=#0x10000003fcb' 2024-09-21T00:30:11.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35286 fp=#0x10000003fda' 2024-09-21T00:30:11.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35414 fp=#0x1000000532a' 2024-09-21T00:30:11.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35446 fp=#0x10000005ff4' 2024-09-21T00:30:11.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35478 fp=#0x10000005f90' 2024-09-21T00:30:11.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35510 fp=#0x10000005f76' 2024-09-21T00:30:11.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35542 fp=#0x10000005fbc' 2024-09-21T00:30:11.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35574 fp=#0x10000005f84' 2024-09-21T00:30:11.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35638 fp=#0x10000005b6c' 2024-09-21T00:30:11.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35670 fp=#0x10000005b7d' 2024-09-21T00:30:11.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35702 fp=#0x10000004911' 2024-09-21T00:30:11.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35734 fp=#0x10000004fe4' 2024-09-21T00:30:11.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35766 fp=#0x10000004fa3' 2024-09-21T00:30:11.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35798 fp=#0x10000004f79' 2024-09-21T00:30:11.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35830 fp=#0x10000004fba' 2024-09-21T00:30:11.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35862 fp=#0x10000004f99' 2024-09-21T00:30:11.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35894 fp=#0x10000006747' 2024-09-21T00:30:11.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35926 fp=#0x1000000672f' 2024-09-21T00:30:11.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35958 fp=#0x10000006887' 2024-09-21T00:30:11.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35990 fp=#0x10000006843' 2024-09-21T00:30:11.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36022 fp=#0x10000006851' 2024-09-21T00:30:11.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36118 fp=#0x10000005f4b' 2024-09-21T00:30:11.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36150 fp=#0x10000005f3f' 2024-09-21T00:30:11.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36182 fp=#0x100000068c1' 2024-09-21T00:30:11.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36214 fp=#0x100000068dd' 2024-09-21T00:30:11.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36246 fp=#0x100000068b3' 2024-09-21T00:30:11.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36278 fp=#0x100000065a0' 2024-09-21T00:30:11.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36310 fp=#0x100000065aa' 2024-09-21T00:30:11.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36342 fp=#0x100000065a9' 2024-09-21T00:30:11.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36374 fp=#0x1000000660a' 2024-09-21T00:30:11.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36406 fp=#0x10000004021' 2024-09-21T00:30:11.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36438 fp=#0x1000000614c' 2024-09-21T00:30:11.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36470 fp=#0x10000004005' 2024-09-21T00:30:11.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36502 fp=#0x1000000604e' 2024-09-21T00:30:11.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36534 fp=#0x10000005ffb' 2024-09-21T00:30:11.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36566 fp=#0x100000059e2' 2024-09-21T00:30:11.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36598 fp=#0x10000005ad0' 2024-09-21T00:30:11.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36630 fp=#0x10000005a00' 2024-09-21T00:30:11.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36662 fp=#0x10000004883' 2024-09-21T00:30:11.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36694 fp=#0x10000004846' 2024-09-21T00:30:11.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36726 fp=#0x10000004915' 2024-09-21T00:30:11.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36758 fp=#0x100000047cc' 2024-09-21T00:30:11.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36790 fp=#0x100000048a3' 2024-09-21T00:30:11.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36822 fp=#0x10000005927' 2024-09-21T00:30:11.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36854 fp=#0x100000058e6' 2024-09-21T00:30:11.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36886 fp=#0x10000005b09' 2024-09-21T00:30:11.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36918 fp=#0x1000000588d' 2024-09-21T00:30:11.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36982 fp=#0x100000051db' 2024-09-21T00:30:11.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37014 fp=#0x10000005260' 2024-09-21T00:30:11.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37046 fp=#0x100000051c2' 2024-09-21T00:30:11.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37078 fp=#0x100000051bb' 2024-09-21T00:30:11.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37174 fp=#0x10000003d14' 2024-09-21T00:30:11.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37206 fp=#0x10000003d1c' 2024-09-21T00:30:11.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37238 fp=#0x10000005bd2' 2024-09-21T00:30:11.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37270 fp=#0x10000005c1a' 2024-09-21T00:30:11.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37302 fp=#0x10000005c55' 2024-09-21T00:30:11.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37334 fp=#0x10000005ba6' 2024-09-21T00:30:11.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37366 fp=#0x10000005bce' 2024-09-21T00:30:11.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37398 fp=#0x10000005ec8' 2024-09-21T00:30:11.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37430 fp=#0x10000005ed8' 2024-09-21T00:30:11.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37462 fp=#0x10000004a94' 2024-09-21T00:30:11.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37494 fp=#0x10000004b39' 2024-09-21T00:30:11.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37526 fp=#0x10000004a61' 2024-09-21T00:30:11.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37558 fp=#0x10000004a5e' 2024-09-21T00:30:11.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37590 fp=#0x100000054db' 2024-09-21T00:30:11.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37622 fp=#0x10000005474' 2024-09-21T00:30:11.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37654 fp=#0x1000000552c' 2024-09-21T00:30:11.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37686 fp=#0x10000004310' 2024-09-21T00:30:11.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37718 fp=#0x1000000428b' 2024-09-21T00:30:11.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37750 fp=#0x100000057f7' 2024-09-21T00:30:11.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37782 fp=#0x10000005811' 2024-09-21T00:30:11.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37814 fp=#0x10000004251' 2024-09-21T00:30:11.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37846 fp=#0x1000000581c' 2024-09-21T00:30:11.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37878 fp=#0x10000001f6a' 2024-09-21T00:30:11.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37942 fp=#0x10000001cbd' 2024-09-21T00:30:11.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37974 fp=#0x10000001c55' 2024-09-21T00:30:11.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38006 fp=#0x10000001b3f' 2024-09-21T00:30:11.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38038 fp=#0x10000003c87' 2024-09-21T00:30:11.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38070 fp=#0x10000003c89' 2024-09-21T00:30:11.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38230 fp=#0x10000006169' 2024-09-21T00:30:11.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38262 fp=#0x10000005857' 2024-09-21T00:30:11.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38294 fp=#0x1000000592e' 2024-09-21T00:30:11.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38358 fp=#0x10000006171' 2024-09-21T00:30:11.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38422 fp=#0x100000029c8' 2024-09-21T00:30:11.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38454 fp=#0x10000002b70' 2024-09-21T00:30:11.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38486 fp=#0x10000002b7c' 2024-09-21T00:30:11.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38518 fp=#0x10000002af8' 2024-09-21T00:30:11.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38550 fp=#0x100000045c3' 2024-09-21T00:30:11.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38742 fp=#0x10000001f7b' 2024-09-21T00:30:11.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38774 fp=#0x100000010b1' 2024-09-21T00:30:11.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38966 fp=#0x10000004041' 2024-09-21T00:30:11.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38998 fp=#0x1000000414a' 2024-09-21T00:30:11.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39030 fp=#0x10000003ff7' 2024-09-21T00:30:11.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39062 fp=#0x10000001f97' 2024-09-21T00:30:11.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39094 fp=#0x10000001c87' 2024-09-21T00:30:11.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39126 fp=#0x100000017e8' 2024-09-21T00:30:11.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39158 fp=#0x10000001c93' 2024-09-21T00:30:11.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39222 fp=#0x10000004498' 2024-09-21T00:30:11.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39286 fp=#0x100000048ee' 2024-09-21T00:30:11.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39318 fp=#0x100000048f9' 2024-09-21T00:30:11.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39350 fp=#0x10000004838' 2024-09-21T00:30:11.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39382 fp=#0x10000005a0f' 2024-09-21T00:30:11.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39414 fp=#0x10000005ab4' 2024-09-21T00:30:11.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39446 fp=#0x10000005af2' 2024-09-21T00:30:11.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39542 fp=#0x10000005a54' 2024-09-21T00:30:11.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39638 fp=#0x100000050b9' 2024-09-21T00:30:11.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39766 fp=#0x10000005b36' 2024-09-21T00:30:11.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39798 fp=#0x10000005b3c' 2024-09-21T00:30:11.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39830 fp=#0x1000000634c' 2024-09-21T00:30:11.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39862 fp=#0x100000062f0' 2024-09-21T00:30:11.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39894 fp=#0x10000004e9d' 2024-09-21T00:30:11.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39926 fp=#0x10000004ee5' 2024-09-21T00:30:11.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39958 fp=#0x10000004edc' 2024-09-21T00:30:11.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39990 fp=#0x10000004ea9' 2024-09-21T00:30:11.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40022 fp=#0x10000006250' 2024-09-21T00:30:11.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40054 fp=#0x100000061fc' 2024-09-21T00:30:11.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40086 fp=#0x1000000628e' 2024-09-21T00:30:11.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40118 fp=#0x100000061e0' 2024-09-21T00:30:11.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40150 fp=#0x10000002620' 2024-09-21T00:30:11.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40182 fp=#0x1000000268a' 2024-09-21T00:30:11.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40214 fp=#0x10000002601' 2024-09-21T00:30:11.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40246 fp=#0x1000000431f' 2024-09-21T00:30:11.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40278 fp=#0x10000000a6f' 2024-09-21T00:30:11.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40310 fp=#0x100000042a5' 2024-09-21T00:30:11.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40374 fp=#0x100000042d9' 2024-09-21T00:30:11.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40406 fp=#0x10000004324' 2024-09-21T00:30:11.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40662 fp=#0x10000004831' 2024-09-21T00:30:11.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40694 fp=#0x100000048e1' 2024-09-21T00:30:11.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40758 fp=#0x10000004855' 2024-09-21T00:30:11.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40790 fp=#0x10000005eb8' 2024-09-21T00:30:11.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40918 fp=#0x10000003038' 2024-09-21T00:30:11.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40950 fp=#0x10000003033' 2024-09-21T00:30:11.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40982 fp=#0x1000000498a' 2024-09-21T00:30:11.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41014 fp=#0x100000049f7' 2024-09-21T00:30:11.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41046 fp=#0x1000000513d' 2024-09-21T00:30:11.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41078 fp=#0x10000004a29' 2024-09-21T00:30:11.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41110 fp=#0x1000000497b' 2024-09-21T00:30:11.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30615 fp=#0x10000000005' 2024-09-21T00:30:11.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30775 fp=#0x10000005ebb' 2024-09-21T00:30:11.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30807 fp=#0x100000023e8' 2024-09-21T00:30:11.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30871 fp=#0x100000066fd' 2024-09-21T00:30:11.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32695 fp=#0x10000005782' 2024-09-21T00:30:11.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32759 fp=#0x10000005b0a' 2024-09-21T00:30:11.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32791 fp=#0x1000000504a' 2024-09-21T00:30:11.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32823 fp=#0x1000000506e' 2024-09-21T00:30:11.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32855 fp=#0x1000000505f' 2024-09-21T00:30:11.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32887 fp=#0x10000005090' 2024-09-21T00:30:11.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32983 fp=#0x10000005c86' 2024-09-21T00:30:11.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33015 fp=#0x10000005ccb' 2024-09-21T00:30:11.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33047 fp=#0x10000005cb9' 2024-09-21T00:30:11.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33079 fp=#0x10000005c65' 2024-09-21T00:30:11.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33111 fp=#0x10000005c85' 2024-09-21T00:30:11.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33143 fp=#0x10000006077' 2024-09-21T00:30:11.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33239 fp=#0x10000006690' 2024-09-21T00:30:11.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33303 fp=#0x100000067fc' 2024-09-21T00:30:11.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33335 fp=#0x1000000680c' 2024-09-21T00:30:11.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33367 fp=#0x10000006816' 2024-09-21T00:30:11.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33399 fp=#0x10000003e9c' 2024-09-21T00:30:11.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33431 fp=#0x10000003e6e' 2024-09-21T00:30:11.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33463 fp=#0x10000003dfb' 2024-09-21T00:30:11.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33495 fp=#0x10000003dcb' 2024-09-21T00:30:11.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33591 fp=#0x1000000558a' 2024-09-21T00:30:11.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33687 fp=#0x10000005a68' 2024-09-21T00:30:11.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33719 fp=#0x100000067af' 2024-09-21T00:30:11.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33751 fp=#0x10000006772' 2024-09-21T00:30:11.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33783 fp=#0x1000000675d' 2024-09-21T00:30:11.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33911 fp=#0x10000004751' 2024-09-21T00:30:11.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33943 fp=#0x1000000643c' 2024-09-21T00:30:11.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33975 fp=#0x10000004714' 2024-09-21T00:30:11.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34007 fp=#0x1000000642d' 2024-09-21T00:30:11.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34039 fp=#0x10000006432' 2024-09-21T00:30:11.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34103 fp=#0x10000003210' 2024-09-21T00:30:11.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34135 fp=#0x100000062b6' 2024-09-21T00:30:11.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34167 fp=#0x1000000628f' 2024-09-21T00:30:11.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34199 fp=#0x10000006465' 2024-09-21T00:30:11.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34231 fp=#0x100000061f0' 2024-09-21T00:30:11.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34263 fp=#0x10000006479' 2024-09-21T00:30:11.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34295 fp=#0x100000061d3' 2024-09-21T00:30:11.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34359 fp=#0x100000056ce' 2024-09-21T00:30:11.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34391 fp=#0x1000000251b' 2024-09-21T00:30:11.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34423 fp=#0x100000061d2' 2024-09-21T00:30:11.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34487 fp=#0x10000005684' 2024-09-21T00:30:11.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34519 fp=#0x100000061ba' 2024-09-21T00:30:11.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34551 fp=#0x100000063b0' 2024-09-21T00:30:11.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34583 fp=#0x1000000637b' 2024-09-21T00:30:11.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34615 fp=#0x10000006311' 2024-09-21T00:30:11.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34647 fp=#0x10000006377' 2024-09-21T00:30:11.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34679 fp=#0x100000062f4' 2024-09-21T00:30:11.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34775 fp=#0x100000064eb' 2024-09-21T00:30:11.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34807 fp=#0x100000057c3' 2024-09-21T00:30:11.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34839 fp=#0x100000064be' 2024-09-21T00:30:11.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34871 fp=#0x100000064db' 2024-09-21T00:30:11.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34903 fp=#0x1000000531e' 2024-09-21T00:30:11.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34935 fp=#0x10000005314' 2024-09-21T00:30:11.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34967 fp=#0x1000000695e' 2024-09-21T00:30:11.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34999 fp=#0x100000052b0' 2024-09-21T00:30:11.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35031 fp=#0x10000005303' 2024-09-21T00:30:11.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35063 fp=#0x100000060e9' 2024-09-21T00:30:11.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35095 fp=#0x100000060c6' 2024-09-21T00:30:11.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35127 fp=#0x100000060d9' 2024-09-21T00:30:11.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35223 fp=#0x100000060d6' 2024-09-21T00:30:11.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35255 fp=#0x10000003fde' 2024-09-21T00:30:11.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35287 fp=#0x10000003fc9' 2024-09-21T00:30:11.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35319 fp=#0x10000003fa3' 2024-09-21T00:30:11.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35447 fp=#0x10000005ff6' 2024-09-21T00:30:11.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35511 fp=#0x10000005ff1' 2024-09-21T00:30:11.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35543 fp=#0x10000005f89' 2024-09-21T00:30:11.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35575 fp=#0x10000005fc3' 2024-09-21T00:30:11.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35607 fp=#0x10000005b6f' 2024-09-21T00:30:11.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35639 fp=#0x10000005b83' 2024-09-21T00:30:11.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35671 fp=#0x10000005ba4' 2024-09-21T00:30:11.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35703 fp=#0x10000004946' 2024-09-21T00:30:11.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35735 fp=#0x10000005024' 2024-09-21T00:30:11.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35767 fp=#0x10000004fd6' 2024-09-21T00:30:11.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35799 fp=#0x10000004f4a' 2024-09-21T00:30:11.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35831 fp=#0x10000004fcc' 2024-09-21T00:30:11.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35863 fp=#0x10000004fc2' 2024-09-21T00:30:11.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35895 fp=#0x1000000670a' 2024-09-21T00:30:11.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35927 fp=#0x10000006743' 2024-09-21T00:30:11.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35959 fp=#0x10000006859' 2024-09-21T00:30:11.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35991 fp=#0x1000000685c' 2024-09-21T00:30:11.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36023 fp=#0x10000006854' 2024-09-21T00:30:11.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36055 fp=#0x10000005d69' 2024-09-21T00:30:11.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36087 fp=#0x10000005f54' 2024-09-21T00:30:11.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36151 fp=#0x10000005f46' 2024-09-21T00:30:11.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36183 fp=#0x100000068e4' 2024-09-21T00:30:11.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36215 fp=#0x1000000689f' 2024-09-21T00:30:11.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36247 fp=#0x100000068ce' 2024-09-21T00:30:11.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36279 fp=#0x10000006616' 2024-09-21T00:30:11.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36311 fp=#0x100000065a1' 2024-09-21T00:30:11.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36375 fp=#0x1000000659c' 2024-09-21T00:30:11.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36407 fp=#0x1000000402a' 2024-09-21T00:30:11.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36439 fp=#0x10000006160' 2024-09-21T00:30:11.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36471 fp=#0x10000006149' 2024-09-21T00:30:11.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36535 fp=#0x10000006030' 2024-09-21T00:30:11.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36567 fp=#0x10000005a8b' 2024-09-21T00:30:11.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36631 fp=#0x10000005a70' 2024-09-21T00:30:11.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36663 fp=#0x1000000492f' 2024-09-21T00:30:11.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36695 fp=#0x1000000485b' 2024-09-21T00:30:11.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36727 fp=#0x1000000482a' 2024-09-21T00:30:11.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36759 fp=#0x10000004779' 2024-09-21T00:30:11.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36791 fp=#0x10000004796' 2024-09-21T00:30:11.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36823 fp=#0x10000005951' 2024-09-21T00:30:11.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36855 fp=#0x10000005907' 2024-09-21T00:30:11.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36951 fp=#0x100000058df' 2024-09-21T00:30:11.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36983 fp=#0x10000005499' 2024-09-21T00:30:11.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37015 fp=#0x100000051c7' 2024-09-21T00:30:11.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37175 fp=#0x10000003cc9' 2024-09-21T00:30:11.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37207 fp=#0x10000003cfb' 2024-09-21T00:30:11.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37239 fp=#0x10000005c4f' 2024-09-21T00:30:11.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37271 fp=#0x10000005c5c' 2024-09-21T00:30:11.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37335 fp=#0x10000005bf0' 2024-09-21T00:30:11.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37367 fp=#0x10000005bd6' 2024-09-21T00:30:11.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37399 fp=#0x10000005f00' 2024-09-21T00:30:11.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37431 fp=#0x10000005ee1' 2024-09-21T00:30:11.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37463 fp=#0x10000004bbe' 2024-09-21T00:30:11.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37495 fp=#0x10000004b96' 2024-09-21T00:30:11.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37527 fp=#0x10000004ba0' 2024-09-21T00:30:11.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37559 fp=#0x10000004aaa' 2024-09-21T00:30:11.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37591 fp=#0x10000005493' 2024-09-21T00:30:11.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37623 fp=#0x100000054b4' 2024-09-21T00:30:11.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37655 fp=#0x1000000554e' 2024-09-21T00:30:11.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37687 fp=#0x10000005804' 2024-09-21T00:30:11.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37719 fp=#0x1000000431d' 2024-09-21T00:30:11.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37751 fp=#0x100000042c2' 2024-09-21T00:30:11.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37783 fp=#0x1000000582b' 2024-09-21T00:30:11.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37815 fp=#0x100000042ff' 2024-09-21T00:30:11.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37847 fp=#0x100000057fd' 2024-09-21T00:30:11.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37879 fp=#0x10000001d59' 2024-09-21T00:30:11.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37911 fp=#0x10000001b59' 2024-09-21T00:30:11.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37943 fp=#0x10000001ca1' 2024-09-21T00:30:11.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37975 fp=#0x10000001c42' 2024-09-21T00:30:11.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38007 fp=#0x10000001bd0' 2024-09-21T00:30:11.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38039 fp=#0x10000003c64' 2024-09-21T00:30:11.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38071 fp=#0x10000003c3a' 2024-09-21T00:30:11.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38199 fp=#0x10000005843' 2024-09-21T00:30:11.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38231 fp=#0x10000005846' 2024-09-21T00:30:11.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38263 fp=#0x10000005883' 2024-09-21T00:30:11.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38295 fp=#0x10000005901' 2024-09-21T00:30:11.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38327 fp=#0x1000000585a' 2024-09-21T00:30:11.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38359 fp=#0x1000000616b' 2024-09-21T00:30:11.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38391 fp=#0x10000006168' 2024-09-21T00:30:11.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38423 fp=#0x10000002aff' 2024-09-21T00:30:11.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38455 fp=#0x10000002b13' 2024-09-21T00:30:11.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38487 fp=#0x100000045f5' 2024-09-21T00:30:11.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38519 fp=#0x10000002b44' 2024-09-21T00:30:11.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38551 fp=#0x10000002b05' 2024-09-21T00:30:11.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38583 fp=#0x1000000460a' 2024-09-21T00:30:11.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38615 fp=#0x10000004606' 2024-09-21T00:30:11.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38743 fp=#0x100000048af' 2024-09-21T00:30:11.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38903 fp=#0x10000004146' 2024-09-21T00:30:11.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38999 fp=#0x10000003fe2' 2024-09-21T00:30:11.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39063 fp=#0x10000001d56' 2024-09-21T00:30:11.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39095 fp=#0x10000001fed' 2024-09-21T00:30:11.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39127 fp=#0x1000000185b' 2024-09-21T00:30:11.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39255 fp=#0x100000044b5' 2024-09-21T00:30:11.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39287 fp=#0x100000048ec' 2024-09-21T00:30:11.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39319 fp=#0x10000004803' 2024-09-21T00:30:11.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39351 fp=#0x10000004829' 2024-09-21T00:30:11.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39383 fp=#0x10000005ae3' 2024-09-21T00:30:11.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39447 fp=#0x10000005e22' 2024-09-21T00:30:11.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39479 fp=#0x10000005ab8' 2024-09-21T00:30:11.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39543 fp=#0x10000005e1f' 2024-09-21T00:30:11.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39575 fp=#0x100000050f5' 2024-09-21T00:30:11.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39639 fp=#0x100000050a4' 2024-09-21T00:30:11.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39671 fp=#0x10000005df1' 2024-09-21T00:30:11.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39735 fp=#0x10000005de1' 2024-09-21T00:30:11.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39767 fp=#0x10000005b40' 2024-09-21T00:30:11.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39799 fp=#0x10000005b3a' 2024-09-21T00:30:11.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39831 fp=#0x10000006317' 2024-09-21T00:30:11.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39863 fp=#0x10000006374' 2024-09-21T00:30:11.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39895 fp=#0x10000004f00' 2024-09-21T00:30:11.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39927 fp=#0x10000004ed3' 2024-09-21T00:30:11.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39959 fp=#0x10000004ea2' 2024-09-21T00:30:11.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39991 fp=#0x10000004ee0' 2024-09-21T00:30:11.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40023 fp=#0x100000062b7' 2024-09-21T00:30:11.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40055 fp=#0x100000061eb' 2024-09-21T00:30:11.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40087 fp=#0x1000000626b' 2024-09-21T00:30:11.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40119 fp=#0x100000061ec' 2024-09-21T00:30:11.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40151 fp=#0x10000002684' 2024-09-21T00:30:11.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40183 fp=#0x10000002644' 2024-09-21T00:30:11.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40247 fp=#0x100000008f0' 2024-09-21T00:30:11.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40279 fp=#0x100000042b1' 2024-09-21T00:30:11.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40311 fp=#0x100000042d5' 2024-09-21T00:30:11.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40343 fp=#0x10000000a32' 2024-09-21T00:30:11.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40375 fp=#0x1000000086b' 2024-09-21T00:30:11.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40407 fp=#0x10000004318' 2024-09-21T00:30:11.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40439 fp=#0x10000005978' 2024-09-21T00:30:11.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40471 fp=#0x1000000597e' 2024-09-21T00:30:11.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40663 fp=#0x10000004952' 2024-09-21T00:30:11.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40695 fp=#0x100000048eb' 2024-09-21T00:30:11.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40727 fp=#0x10000005ed3' 2024-09-21T00:30:11.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40759 fp=#0x10000004834' 2024-09-21T00:30:11.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40791 fp=#0x10000005eb3' 2024-09-21T00:30:11.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40855 fp=#0x10000003047' 2024-09-21T00:30:11.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40887 fp=#0x100000063ec' 2024-09-21T00:30:11.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40983 fp=#0x10000004a3a' 2024-09-21T00:30:11.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41015 fp=#0x10000005106' 2024-09-21T00:30:11.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41047 fp=#0x10000005121' 2024-09-21T00:30:11.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41079 fp=#0x100000049d1' 2024-09-21T00:30:11.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41111 fp=#0x100000049cc' 2024-09-21T00:30:11.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30616 fp=#0x10000000006' 2024-09-21T00:30:11.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30776 fp=#0x10000005026' 2024-09-21T00:30:11.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30808 fp=#0x10000001ee4' 2024-09-21T00:30:11.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30872 fp=#0x1000000682e' 2024-09-21T00:30:11.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32728 fp=#0x10000005740' 2024-09-21T00:30:11.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32760 fp=#0x10000005798' 2024-09-21T00:30:11.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32792 fp=#0x1000000508f' 2024-09-21T00:30:11.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32824 fp=#0x10000005082' 2024-09-21T00:30:11.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32856 fp=#0x10000005063' 2024-09-21T00:30:11.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32888 fp=#0x100000066cf' 2024-09-21T00:30:11.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32920 fp=#0x1000000502f' 2024-09-21T00:30:11.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32984 fp=#0x10000005cf4' 2024-09-21T00:30:11.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33016 fp=#0x10000005cf8' 2024-09-21T00:30:11.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33048 fp=#0x10000005cb1' 2024-09-21T00:30:11.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33080 fp=#0x10000005ca9' 2024-09-21T00:30:11.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33112 fp=#0x10000005cf3' 2024-09-21T00:30:11.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33208 fp=#0x10000006036' 2024-09-21T00:30:11.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33240 fp=#0x10000006031' 2024-09-21T00:30:11.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33304 fp=#0x1000000682a' 2024-09-21T00:30:11.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33336 fp=#0x100000067e3' 2024-09-21T00:30:11.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33368 fp=#0x100000067ff' 2024-09-21T00:30:11.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33400 fp=#0x10000003d87' 2024-09-21T00:30:11.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33432 fp=#0x10000003def' 2024-09-21T00:30:11.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33464 fp=#0x10000003d8a' 2024-09-21T00:30:11.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33496 fp=#0x10000003d7a' 2024-09-21T00:30:11.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33528 fp=#0x100000047e3' 2024-09-21T00:30:11.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33560 fp=#0x10000005669' 2024-09-21T00:30:11.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33592 fp=#0x10000005a5d' 2024-09-21T00:30:11.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33624 fp=#0x10000005a3f' 2024-09-21T00:30:11.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33688 fp=#0x100000059ff' 2024-09-21T00:30:11.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33720 fp=#0x10000006759' 2024-09-21T00:30:11.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33752 fp=#0x1000000677e' 2024-09-21T00:30:11.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33784 fp=#0x10000006791' 2024-09-21T00:30:11.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33912 fp=#0x1000000644e' 2024-09-21T00:30:11.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33944 fp=#0x1000000643b' 2024-09-21T00:30:11.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33976 fp=#0x1000000472b' 2024-09-21T00:30:11.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34008 fp=#0x1000000640f' 2024-09-21T00:30:11.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34040 fp=#0x1000000642c' 2024-09-21T00:30:11.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34072 fp=#0x10000003254' 2024-09-21T00:30:11.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34104 fp=#0x100000031d9' 2024-09-21T00:30:11.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34136 fp=#0x1000000620d' 2024-09-21T00:30:11.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34168 fp=#0x10000006287' 2024-09-21T00:30:11.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34200 fp=#0x100000064a7' 2024-09-21T00:30:11.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34232 fp=#0x10000006457' 2024-09-21T00:30:11.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34264 fp=#0x10000006456' 2024-09-21T00:30:11.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34296 fp=#0x100000026aa' 2024-09-21T00:30:11.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34328 fp=#0x10000002519' 2024-09-21T00:30:11.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34360 fp=#0x10000002657' 2024-09-21T00:30:11.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34392 fp=#0x1000000254a' 2024-09-21T00:30:11.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34424 fp=#0x10000002647' 2024-09-21T00:30:11.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34456 fp=#0x10000002514' 2024-09-21T00:30:11.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34488 fp=#0x10000002521' 2024-09-21T00:30:11.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34520 fp=#0x1000000267d' 2024-09-21T00:30:11.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34552 fp=#0x100000062eb' 2024-09-21T00:30:11.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34584 fp=#0x10000006382' 2024-09-21T00:30:11.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34616 fp=#0x10000006373' 2024-09-21T00:30:11.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34648 fp=#0x1000000631b' 2024-09-21T00:30:11.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34680 fp=#0x100000063a2' 2024-09-21T00:30:11.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34776 fp=#0x100000064d9' 2024-09-21T00:30:11.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34808 fp=#0x100000057cd' 2024-09-21T00:30:11.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34840 fp=#0x100000057bd' 2024-09-21T00:30:11.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34872 fp=#0x100000057db' 2024-09-21T00:30:11.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34904 fp=#0x100000052b6' 2024-09-21T00:30:11.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34936 fp=#0x1000000530d' 2024-09-21T00:30:11.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34968 fp=#0x100000052a9' 2024-09-21T00:30:11.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35000 fp=#0x100000052e8' 2024-09-21T00:30:11.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35032 fp=#0x10000005319' 2024-09-21T00:30:11.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35064 fp=#0x100000060e0' 2024-09-21T00:30:11.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35256 fp=#0x10000003f42' 2024-09-21T00:30:11.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35288 fp=#0x10000003f4f' 2024-09-21T00:30:11.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35320 fp=#0x10000003f39' 2024-09-21T00:30:11.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35448 fp=#0x10000005f83' 2024-09-21T00:30:11.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35608 fp=#0x10000005bf2' 2024-09-21T00:30:11.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35640 fp=#0x10000005b92' 2024-09-21T00:30:11.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35672 fp=#0x10000005b77' 2024-09-21T00:30:11.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35704 fp=#0x10000006506' 2024-09-21T00:30:11.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35736 fp=#0x10000004f28' 2024-09-21T00:30:11.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35768 fp=#0x10000006580' 2024-09-21T00:30:11.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35800 fp=#0x10000004fc4' 2024-09-21T00:30:11.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35832 fp=#0x10000006624' 2024-09-21T00:30:11.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35864 fp=#0x10000004fb0' 2024-09-21T00:30:11.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35896 fp=#0x10000006745' 2024-09-21T00:30:11.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35928 fp=#0x10000006713' 2024-09-21T00:30:11.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35960 fp=#0x10000006884' 2024-09-21T00:30:11.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35992 fp=#0x10000006844' 2024-09-21T00:30:11.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36024 fp=#0x1000000686c' 2024-09-21T00:30:11.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36056 fp=#0x10000005f45' 2024-09-21T00:30:11.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36088 fp=#0x10000005f51' 2024-09-21T00:30:11.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36152 fp=#0x10000005f40' 2024-09-21T00:30:11.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36184 fp=#0x100000068e6' 2024-09-21T00:30:11.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36216 fp=#0x100000068ae' 2024-09-21T00:30:11.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36248 fp=#0x100000068ca' 2024-09-21T00:30:11.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36312 fp=#0x100000065b2' 2024-09-21T00:30:11.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36408 fp=#0x10000004016' 2024-09-21T00:30:11.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36440 fp=#0x10000006158' 2024-09-21T00:30:11.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36472 fp=#0x10000006164' 2024-09-21T00:30:11.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36504 fp=#0x10000005ff9' 2024-09-21T00:30:11.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36568 fp=#0x10000005a6d' 2024-09-21T00:30:11.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36600 fp=#0x10000005ac3' 2024-09-21T00:30:11.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36664 fp=#0x1000000477f' 2024-09-21T00:30:11.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36696 fp=#0x100000047a9' 2024-09-21T00:30:11.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36728 fp=#0x100000047d5' 2024-09-21T00:30:11.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36760 fp=#0x100000048f7' 2024-09-21T00:30:11.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36856 fp=#0x100000058ea' 2024-09-21T00:30:11.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36888 fp=#0x10000005933' 2024-09-21T00:30:11.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36952 fp=#0x10000005af1' 2024-09-21T00:30:11.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36984 fp=#0x100000051b4' 2024-09-21T00:30:11.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37016 fp=#0x100000051d9' 2024-09-21T00:30:11.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37048 fp=#0x100000051cc' 2024-09-21T00:30:11.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37080 fp=#0x10000005268' 2024-09-21T00:30:11.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37112 fp=#0x10000005450' 2024-09-21T00:30:11.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37176 fp=#0x10000005fe5' 2024-09-21T00:30:11.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37208 fp=#0x10000003cbc' 2024-09-21T00:30:11.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37272 fp=#0x10000005c34' 2024-09-21T00:30:11.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37304 fp=#0x10000005bef' 2024-09-21T00:30:11.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37336 fp=#0x10000005bcc' 2024-09-21T00:30:11.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37400 fp=#0x10000005f04' 2024-09-21T00:30:11.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37432 fp=#0x10000005ede' 2024-09-21T00:30:11.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37464 fp=#0x10000004a71' 2024-09-21T00:30:11.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37496 fp=#0x10000004a81' 2024-09-21T00:30:11.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37528 fp=#0x10000004b15' 2024-09-21T00:30:11.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37560 fp=#0x10000004ab2' 2024-09-21T00:30:11.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37592 fp=#0x10000005556' 2024-09-21T00:30:11.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37624 fp=#0x100000054e0' 2024-09-21T00:30:11.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37656 fp=#0x100000054cf' 2024-09-21T00:30:11.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37688 fp=#0x100000053a6' 2024-09-21T00:30:11.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37720 fp=#0x10000004260' 2024-09-21T00:30:11.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37752 fp=#0x100000042b0' 2024-09-21T00:30:11.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37784 fp=#0x100000042b4' 2024-09-21T00:30:11.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37816 fp=#0x100000042d6' 2024-09-21T00:30:11.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37848 fp=#0x10000005806' 2024-09-21T00:30:11.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37880 fp=#0x10000001d3a' 2024-09-21T00:30:11.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37912 fp=#0x10000001ce9' 2024-09-21T00:30:11.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37944 fp=#0x10000001b82' 2024-09-21T00:30:11.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37976 fp=#0x10000003689' 2024-09-21T00:30:11.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38008 fp=#0x10000001bdf' 2024-09-21T00:30:11.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38136 fp=#0x10000003a3e' 2024-09-21T00:30:11.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38232 fp=#0x1000000594f' 2024-09-21T00:30:11.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38264 fp=#0x1000000588c' 2024-09-21T00:30:11.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38296 fp=#0x1000000593b' 2024-09-21T00:30:11.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38328 fp=#0x10000005842' 2024-09-21T00:30:11.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38360 fp=#0x10000005950' 2024-09-21T00:30:11.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38392 fp=#0x10000005872' 2024-09-21T00:30:11.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38424 fp=#0x10000002bcb' 2024-09-21T00:30:11.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38456 fp=#0x10000002a6b' 2024-09-21T00:30:11.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38488 fp=#0x100000029eb' 2024-09-21T00:30:11.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38520 fp=#0x100000045ff' 2024-09-21T00:30:11.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38552 fp=#0x10000002a04' 2024-09-21T00:30:11.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38584 fp=#0x10000002a13' 2024-09-21T00:30:11.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38616 fp=#0x100000045f7' 2024-09-21T00:30:11.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38648 fp=#0x1000000106e' 2024-09-21T00:30:11.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38680 fp=#0x1000000487f' 2024-09-21T00:30:11.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38712 fp=#0x1000000488c' 2024-09-21T00:30:11.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38872 fp=#0x100000041e2' 2024-09-21T00:30:11.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38904 fp=#0x10000004012' 2024-09-21T00:30:11.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38968 fp=#0x10000004194' 2024-09-21T00:30:11.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39000 fp=#0x10000004085' 2024-09-21T00:30:11.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39032 fp=#0x100000053a2' 2024-09-21T00:30:11.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39064 fp=#0x10000001c27' 2024-09-21T00:30:11.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39096 fp=#0x10000001fc8' 2024-09-21T00:30:11.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39128 fp=#0x10000001d1f' 2024-09-21T00:30:11.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39160 fp=#0x10000001d1a' 2024-09-21T00:30:11.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39224 fp=#0x100000044a1' 2024-09-21T00:30:11.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39256 fp=#0x100000044fe' 2024-09-21T00:30:11.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39288 fp=#0x10000004904' 2024-09-21T00:30:11.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39320 fp=#0x1000000484f' 2024-09-21T00:30:11.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39352 fp=#0x100000048c7' 2024-09-21T00:30:11.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39384 fp=#0x100000059ca' 2024-09-21T00:30:11.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39416 fp=#0x10000005abb' 2024-09-21T00:30:11.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39448 fp=#0x10000005aa1' 2024-09-21T00:30:11.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39480 fp=#0x10000005e00' 2024-09-21T00:30:11.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39512 fp=#0x100000059d2' 2024-09-21T00:30:11.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39544 fp=#0x10000005ac6' 2024-09-21T00:30:11.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39576 fp=#0x100000050fe' 2024-09-21T00:30:11.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39672 fp=#0x10000005de3' 2024-09-21T00:30:11.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39704 fp=#0x10000005df0' 2024-09-21T00:30:11.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39800 fp=#0x10000005b3d' 2024-09-21T00:30:11.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39832 fp=#0x1000000633c' 2024-09-21T00:30:11.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39864 fp=#0x10000006313' 2024-09-21T00:30:11.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39896 fp=#0x10000004ee1' 2024-09-21T00:30:11.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39928 fp=#0x10000004efc' 2024-09-21T00:30:11.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39960 fp=#0x10000004eed' 2024-09-21T00:30:11.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39992 fp=#0x10000004ed9' 2024-09-21T00:30:11.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40024 fp=#0x100000062c3' 2024-09-21T00:30:11.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40056 fp=#0x100000062ac' 2024-09-21T00:30:11.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40088 fp=#0x100000061e3' 2024-09-21T00:30:11.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40120 fp=#0x10000006240' 2024-09-21T00:30:11.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40152 fp=#0x10000002616' 2024-09-21T00:30:11.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40184 fp=#0x100000026a2' 2024-09-21T00:30:11.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40216 fp=#0x10000002686' 2024-09-21T00:30:11.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40248 fp=#0x1000000087c' 2024-09-21T00:30:11.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40280 fp=#0x100000009f9' 2024-09-21T00:30:11.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40312 fp=#0x100000042b9' 2024-09-21T00:30:11.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40344 fp=#0x100000042c3' 2024-09-21T00:30:11.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40376 fp=#0x100000042dd' 2024-09-21T00:30:11.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40408 fp=#0x10000004309' 2024-09-21T00:30:11.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40440 fp=#0x10000005976' 2024-09-21T00:30:11.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40472 fp=#0x10000005974' 2024-09-21T00:30:11.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40664 fp=#0x100000048f5' 2024-09-21T00:30:11.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40696 fp=#0x10000005eb6' 2024-09-21T00:30:11.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40728 fp=#0x10000005ec2' 2024-09-21T00:30:11.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40760 fp=#0x10000004930' 2024-09-21T00:30:11.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40792 fp=#0x1000000480e' 2024-09-21T00:30:11.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40856 fp=#0x100000063c5' 2024-09-21T00:30:11.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40952 fp=#0x10000003063' 2024-09-21T00:30:11.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40984 fp=#0x10000005141' 2024-09-21T00:30:11.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41016 fp=#0x10000004a3b' 2024-09-21T00:30:11.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41048 fp=#0x1000000513b' 2024-09-21T00:30:11.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41080 fp=#0x10000004a32' 2024-09-21T00:30:11.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41112 fp=#0x100000049e7' 2024-09-21T00:30:11.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30617 fp=#0x10000000201' 2024-09-21T00:30:11.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30777 fp=#0x10000005c30' 2024-09-21T00:30:11.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30809 fp=#0x10000001759' 2024-09-21T00:30:11.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30873 fp=#0x10000000853' 2024-09-21T00:30:11.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32761 fp=#0x1000000568f' 2024-09-21T00:30:11.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32793 fp=#0x10000005070' 2024-09-21T00:30:11.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32825 fp=#0x10000005062' 2024-09-21T00:30:11.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32921 fp=#0x1000000504b' 2024-09-21T00:30:11.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32985 fp=#0x10000005c5b' 2024-09-21T00:30:11.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33017 fp=#0x10000005c95' 2024-09-21T00:30:11.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33049 fp=#0x10000005c50' 2024-09-21T00:30:11.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33113 fp=#0x10000005c6c' 2024-09-21T00:30:11.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33273 fp=#0x10000006074' 2024-09-21T00:30:11.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33305 fp=#0x1000000682c' 2024-09-21T00:30:11.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33337 fp=#0x10000006814' 2024-09-21T00:30:11.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33369 fp=#0x100000067ef' 2024-09-21T00:30:11.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33401 fp=#0x10000003dcf' 2024-09-21T00:30:11.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33433 fp=#0x10000003e18' 2024-09-21T00:30:11.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33465 fp=#0x10000003e15' 2024-09-21T00:30:11.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33561 fp=#0x10000005a12' 2024-09-21T00:30:11.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33593 fp=#0x100000055a4' 2024-09-21T00:30:11.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33657 fp=#0x1000000564b' 2024-09-21T00:30:11.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33689 fp=#0x10000005a25' 2024-09-21T00:30:11.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33721 fp=#0x10000006778' 2024-09-21T00:30:11.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33753 fp=#0x10000006774' 2024-09-21T00:30:11.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33785 fp=#0x1000000677a' 2024-09-21T00:30:11.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33913 fp=#0x10000006436' 2024-09-21T00:30:11.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33945 fp=#0x10000005f14' 2024-09-21T00:30:11.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33977 fp=#0x100000047e1' 2024-09-21T00:30:11.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34009 fp=#0x1000000644b' 2024-09-21T00:30:11.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34041 fp=#0x100000063fd' 2024-09-21T00:30:11.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34073 fp=#0x1000000323e' 2024-09-21T00:30:11.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34105 fp=#0x1000000322d' 2024-09-21T00:30:11.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34137 fp=#0x1000000626e' 2024-09-21T00:30:11.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34169 fp=#0x1000000648b' 2024-09-21T00:30:11.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34201 fp=#0x100000061f9' 2024-09-21T00:30:11.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34233 fp=#0x10000006237' 2024-09-21T00:30:11.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34265 fp=#0x10000006463' 2024-09-21T00:30:11.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34297 fp=#0x10000002508' 2024-09-21T00:30:11.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34329 fp=#0x100000026d1' 2024-09-21T00:30:11.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34361 fp=#0x100000026cb' 2024-09-21T00:30:11.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34425 fp=#0x100000056a0' 2024-09-21T00:30:11.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34457 fp=#0x100000026a1' 2024-09-21T00:30:11.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34489 fp=#0x10000002689' 2024-09-21T00:30:11.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34521 fp=#0x1000000264c' 2024-09-21T00:30:11.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34553 fp=#0x10000006342' 2024-09-21T00:30:11.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34585 fp=#0x1000000630a' 2024-09-21T00:30:11.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34617 fp=#0x100000063af' 2024-09-21T00:30:11.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34649 fp=#0x10000006364' 2024-09-21T00:30:11.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34681 fp=#0x100000062dd' 2024-09-21T00:30:11.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34777 fp=#0x100000064b7' 2024-09-21T00:30:11.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34809 fp=#0x100000064e9' 2024-09-21T00:30:11.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34841 fp=#0x100000064fc' 2024-09-21T00:30:11.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34873 fp=#0x100000064b0' 2024-09-21T00:30:11.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34905 fp=#0x10000006946' 2024-09-21T00:30:11.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34937 fp=#0x10000006951' 2024-09-21T00:30:11.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34969 fp=#0x100000052b3' 2024-09-21T00:30:11.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35001 fp=#0x10000005326' 2024-09-21T00:30:11.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35033 fp=#0x10000006944' 2024-09-21T00:30:11.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35097 fp=#0x100000060a5' 2024-09-21T00:30:11.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35129 fp=#0x1000000609f' 2024-09-21T00:30:11.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35161 fp=#0x100000060ae' 2024-09-21T00:30:11.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35257 fp=#0x10000003fd4' 2024-09-21T00:30:11.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35289 fp=#0x10000003f08' 2024-09-21T00:30:11.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35321 fp=#0x10000003f5c' 2024-09-21T00:30:11.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35353 fp=#0x1000000532b' 2024-09-21T00:30:11.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35449 fp=#0x10000005fd7' 2024-09-21T00:30:11.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35481 fp=#0x10000005fb7' 2024-09-21T00:30:11.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35545 fp=#0x10000005f69' 2024-09-21T00:30:11.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35609 fp=#0x10000005ba5' 2024-09-21T00:30:11.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35641 fp=#0x10000005b9c' 2024-09-21T00:30:11.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35673 fp=#0x10000005bcf' 2024-09-21T00:30:11.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35705 fp=#0x100000053a9' 2024-09-21T00:30:11.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35737 fp=#0x10000004f89' 2024-09-21T00:30:11.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35769 fp=#0x10000004fdc' 2024-09-21T00:30:11.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35801 fp=#0x10000004fce' 2024-09-21T00:30:11.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35833 fp=#0x10000005012' 2024-09-21T00:30:11.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35865 fp=#0x10000004fe6' 2024-09-21T00:30:11.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35897 fp=#0x10000006728' 2024-09-21T00:30:11.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35929 fp=#0x1000000674a' 2024-09-21T00:30:11.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35961 fp=#0x10000006883' 2024-09-21T00:30:11.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35993 fp=#0x1000000688b' 2024-09-21T00:30:11.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36025 fp=#0x10000006877' 2024-09-21T00:30:11.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36057 fp=#0x10000005f4e' 2024-09-21T00:30:11.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36089 fp=#0x10000005f66' 2024-09-21T00:30:11.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36185 fp=#0x10000006891' 2024-09-21T00:30:11.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36217 fp=#0x100000068bb' 2024-09-21T00:30:11.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36249 fp=#0x100000068b0' 2024-09-21T00:30:11.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36281 fp=#0x10000006615' 2024-09-21T00:30:11.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36313 fp=#0x100000065b9' 2024-09-21T00:30:11.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36345 fp=#0x1000000658e' 2024-09-21T00:30:11.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36377 fp=#0x10000006597' 2024-09-21T00:30:11.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36473 fp=#0x10000006155' 2024-09-21T00:30:11.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36505 fp=#0x10000006003' 2024-09-21T00:30:11.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36569 fp=#0x10000005ab2' 2024-09-21T00:30:11.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36601 fp=#0x10000005a2a' 2024-09-21T00:30:11.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36633 fp=#0x100000059ed' 2024-09-21T00:30:11.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36665 fp=#0x10000004801' 2024-09-21T00:30:11.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36697 fp=#0x100000048dd' 2024-09-21T00:30:11.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36729 fp=#0x10000004767' 2024-09-21T00:30:11.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36761 fp=#0x100000047d3' 2024-09-21T00:30:11.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36793 fp=#0x10000005b07' 2024-09-21T00:30:11.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36825 fp=#0x10000005b13' 2024-09-21T00:30:11.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36857 fp=#0x1000000588b' 2024-09-21T00:30:11.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36889 fp=#0x100000058e4' 2024-09-21T00:30:11.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36921 fp=#0x10000005925' 2024-09-21T00:30:11.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36985 fp=#0x100000051ac' 2024-09-21T00:30:11.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37017 fp=#0x100000051ca' 2024-09-21T00:30:11.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37081 fp=#0x100000051c1' 2024-09-21T00:30:11.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37145 fp=#0x10000005451' 2024-09-21T00:30:11.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37209 fp=#0x10000005fdc' 2024-09-21T00:30:11.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37241 fp=#0x10000005bb3' 2024-09-21T00:30:11.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37273 fp=#0x10000005bc2' 2024-09-21T00:30:11.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37305 fp=#0x10000005bc0' 2024-09-21T00:30:11.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37369 fp=#0x10000005c2b' 2024-09-21T00:30:11.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37433 fp=#0x10000005ee2' 2024-09-21T00:30:11.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37465 fp=#0x10000004a75' 2024-09-21T00:30:11.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37497 fp=#0x10000004a95' 2024-09-21T00:30:11.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37529 fp=#0x10000004b01' 2024-09-21T00:30:11.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37561 fp=#0x10000004b14' 2024-09-21T00:30:11.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37593 fp=#0x10000005506' 2024-09-21T00:30:11.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37625 fp=#0x10000005500' 2024-09-21T00:30:11.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37657 fp=#0x10000005537' 2024-09-21T00:30:11.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37689 fp=#0x10000004245' 2024-09-21T00:30:11.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37721 fp=#0x10000005802' 2024-09-21T00:30:11.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37753 fp=#0x100000057f2' 2024-09-21T00:30:11.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37785 fp=#0x10000005833' 2024-09-21T00:30:11.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37817 fp=#0x10000005801' 2024-09-21T00:30:11.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37849 fp=#0x10000004291' 2024-09-21T00:30:11.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37881 fp=#0x10000001b3b' 2024-09-21T00:30:11.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37913 fp=#0x10000001d40' 2024-09-21T00:30:11.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37945 fp=#0x10000001c8a' 2024-09-21T00:30:11.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37977 fp=#0x10000001c1b' 2024-09-21T00:30:11.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38009 fp=#0x10000001c51' 2024-09-21T00:30:11.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38041 fp=#0x10000003c58' 2024-09-21T00:30:11.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38105 fp=#0x10000003aa2' 2024-09-21T00:30:11.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38201 fp=#0x10000005875' 2024-09-21T00:30:11.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38233 fp=#0x1000000598f' 2024-09-21T00:30:11.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38265 fp=#0x10000006184' 2024-09-21T00:30:11.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38297 fp=#0x10000006188' 2024-09-21T00:30:11.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38361 fp=#0x10000005942' 2024-09-21T00:30:11.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38393 fp=#0x10000005878' 2024-09-21T00:30:11.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38425 fp=#0x10000002a2d' 2024-09-21T00:30:11.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38457 fp=#0x10000002b50' 2024-09-21T00:30:11.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38489 fp=#0x100000045f2' 2024-09-21T00:30:11.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38521 fp=#0x10000002bd3' 2024-09-21T00:30:11.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38553 fp=#0x10000002ad8' 2024-09-21T00:30:11.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38585 fp=#0x10000002b79' 2024-09-21T00:30:11.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38617 fp=#0x100000029d1' 2024-09-21T00:30:11.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38681 fp=#0x1000000487a' 2024-09-21T00:30:11.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38745 fp=#0x10000001092' 2024-09-21T00:30:11.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38809 fp=#0x1000000109d' 2024-09-21T00:30:11.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38873 fp=#0x10000004204' 2024-09-21T00:30:11.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38937 fp=#0x10000003fea' 2024-09-21T00:30:11.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38969 fp=#0x10000004087' 2024-09-21T00:30:11.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39065 fp=#0x100000017f5' 2024-09-21T00:30:11.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39097 fp=#0x10000001d4d' 2024-09-21T00:30:11.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39129 fp=#0x10000001d2a' 2024-09-21T00:30:11.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39161 fp=#0x10000004505' 2024-09-21T00:30:11.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39193 fp=#0x100000044f9' 2024-09-21T00:30:11.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39289 fp=#0x10000004890' 2024-09-21T00:30:11.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39321 fp=#0x10000004806' 2024-09-21T00:30:11.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39353 fp=#0x1000000487e' 2024-09-21T00:30:11.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39385 fp=#0x100000059c4' 2024-09-21T00:30:11.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39417 fp=#0x10000005e27' 2024-09-21T00:30:11.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39481 fp=#0x10000005a07' 2024-09-21T00:30:11.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39513 fp=#0x10000005e1e' 2024-09-21T00:30:11.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39577 fp=#0x100000050b4' 2024-09-21T00:30:11.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39609 fp=#0x100000050a1' 2024-09-21T00:30:11.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39641 fp=#0x10000005097' 2024-09-21T00:30:11.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39673 fp=#0x10000005dea' 2024-09-21T00:30:11.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39801 fp=#0x10000005b55' 2024-09-21T00:30:11.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39833 fp=#0x10000006360' 2024-09-21T00:30:11.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39865 fp=#0x1000000637a' 2024-09-21T00:30:11.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39897 fp=#0x10000004f02' 2024-09-21T00:30:11.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39929 fp=#0x10000004eea' 2024-09-21T00:30:11.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39961 fp=#0x10000004eaa' 2024-09-21T00:30:11.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39993 fp=#0x10000004ea5' 2024-09-21T00:30:11.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40025 fp=#0x100000061f6' 2024-09-21T00:30:11.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40057 fp=#0x10000006247' 2024-09-21T00:30:11.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40089 fp=#0x10000006230' 2024-09-21T00:30:11.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40121 fp=#0x100000061ee' 2024-09-21T00:30:11.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40153 fp=#0x100000026c3' 2024-09-21T00:30:11.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40185 fp=#0x1000000264b' 2024-09-21T00:30:11.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40217 fp=#0x10000002699' 2024-09-21T00:30:11.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40249 fp=#0x10000000b17' 2024-09-21T00:30:11.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40281 fp=#0x10000000b0c' 2024-09-21T00:30:11.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40313 fp=#0x1000000432d' 2024-09-21T00:30:11.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40345 fp=#0x100000008c6' 2024-09-21T00:30:11.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40377 fp=#0x10000000900' 2024-09-21T00:30:11.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40409 fp=#0x100000063f8' 2024-09-21T00:30:11.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40473 fp=#0x100000058f6' 2024-09-21T00:30:11.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40537 fp=#0x10000002883' 2024-09-21T00:30:11.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40601 fp=#0x100000037fe' 2024-09-21T00:30:11.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40633 fp=#0x100000054cc' 2024-09-21T00:30:11.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40665 fp=#0x10000004959' 2024-09-21T00:30:11.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40697 fp=#0x10000005eb5' 2024-09-21T00:30:11.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40729 fp=#0x10000005ea2' 2024-09-21T00:30:11.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40793 fp=#0x1000000490c' 2024-09-21T00:30:11.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40825 fp=#0x100000031c4' 2024-09-21T00:30:11.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40857 fp=#0x100000063c6' 2024-09-21T00:30:11.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40953 fp=#0x100000063d6' 2024-09-21T00:30:11.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40985 fp=#0x10000005138' 2024-09-21T00:30:11.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41017 fp=#0x10000004a08' 2024-09-21T00:30:11.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41049 fp=#0x10000004993' 2024-09-21T00:30:11.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41081 fp=#0x1000000511a' 2024-09-21T00:30:11.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41113 fp=#0x1000000512e' 2024-09-21T00:30:11.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30778 fp=#0x10000004a4f' 2024-09-21T00:30:11.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30810 fp=#0x1000000529a' 2024-09-21T00:30:11.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30874 fp=#0x10000005cc1' 2024-09-21T00:30:11.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32698 fp=#0x100000057a6' 2024-09-21T00:30:11.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32730 fp=#0x10000005795' 2024-09-21T00:30:11.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32762 fp=#0x10000005712' 2024-09-21T00:30:11.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32794 fp=#0x10000005091' 2024-09-21T00:30:11.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32826 fp=#0x1000000507b' 2024-09-21T00:30:11.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32922 fp=#0x10000005084' 2024-09-21T00:30:11.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33018 fp=#0x10000005cec' 2024-09-21T00:30:11.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33050 fp=#0x10000005caa' 2024-09-21T00:30:11.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33082 fp=#0x10000005c63' 2024-09-21T00:30:11.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33114 fp=#0x10000005cc9' 2024-09-21T00:30:11.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33146 fp=#0x10000005dda' 2024-09-21T00:30:11.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33178 fp=#0x10000006673' 2024-09-21T00:30:11.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33210 fp=#0x1000000605a' 2024-09-21T00:30:11.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33274 fp=#0x1000000604d' 2024-09-21T00:30:11.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33306 fp=#0x100000067c9' 2024-09-21T00:30:11.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33338 fp=#0x10000006818' 2024-09-21T00:30:11.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33370 fp=#0x100000067f3' 2024-09-21T00:30:11.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33402 fp=#0x10000003dbf' 2024-09-21T00:30:11.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33434 fp=#0x10000003df7' 2024-09-21T00:30:11.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33466 fp=#0x10000003db1' 2024-09-21T00:30:11.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33498 fp=#0x10000003d45' 2024-09-21T00:30:11.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33530 fp=#0x100000047ea' 2024-09-21T00:30:11.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33562 fp=#0x10000005a1e' 2024-09-21T00:30:11.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33594 fp=#0x100000059fb' 2024-09-21T00:30:11.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33658 fp=#0x100000055a9' 2024-09-21T00:30:11.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33722 fp=#0x100000067bf' 2024-09-21T00:30:11.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33754 fp=#0x10000006786' 2024-09-21T00:30:11.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33786 fp=#0x1000000676c' 2024-09-21T00:30:11.630 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33914 fp=#0x10000004737' 2024-09-21T00:30:11.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33946 fp=#0x10000004778' 2024-09-21T00:30:11.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33978 fp=#0x10000006438' 2024-09-21T00:30:11.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34010 fp=#0x10000006403' 2024-09-21T00:30:11.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34042 fp=#0x1000000474e' 2024-09-21T00:30:11.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34074 fp=#0x100000031ea' 2024-09-21T00:30:11.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34106 fp=#0x100000031f4' 2024-09-21T00:30:11.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34138 fp=#0x100000062a0' 2024-09-21T00:30:11.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34170 fp=#0x1000000648a' 2024-09-21T00:30:11.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34202 fp=#0x1000000649c' 2024-09-21T00:30:11.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34234 fp=#0x10000006235' 2024-09-21T00:30:11.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34266 fp=#0x10000006242' 2024-09-21T00:30:11.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34298 fp=#0x100000061bb' 2024-09-21T00:30:11.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34394 fp=#0x1000000568a' 2024-09-21T00:30:11.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34426 fp=#0x100000026c2' 2024-09-21T00:30:11.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34458 fp=#0x10000002594' 2024-09-21T00:30:11.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34554 fp=#0x100000063a4' 2024-09-21T00:30:11.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34586 fp=#0x10000006398' 2024-09-21T00:30:11.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34618 fp=#0x100000063a7' 2024-09-21T00:30:11.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34650 fp=#0x1000000633d' 2024-09-21T00:30:11.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34682 fp=#0x100000062ba' 2024-09-21T00:30:11.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34778 fp=#0x100000057eb' 2024-09-21T00:30:11.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34810 fp=#0x100000064b6' 2024-09-21T00:30:11.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34842 fp=#0x100000064b2' 2024-09-21T00:30:11.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34874 fp=#0x100000064e2' 2024-09-21T00:30:11.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34906 fp=#0x100000052e6' 2024-09-21T00:30:11.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34938 fp=#0x10000006950' 2024-09-21T00:30:11.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35002 fp=#0x10000005308' 2024-09-21T00:30:11.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35034 fp=#0x1000000696e' 2024-09-21T00:30:11.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35066 fp=#0x100000060cf' 2024-09-21T00:30:11.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35162 fp=#0x100000060ce' 2024-09-21T00:30:11.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35194 fp=#0x100000060a2' 2024-09-21T00:30:11.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35258 fp=#0x10000003f14' 2024-09-21T00:30:11.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35386 fp=#0x10000005399' 2024-09-21T00:30:11.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35450 fp=#0x10000005f6e' 2024-09-21T00:30:11.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35482 fp=#0x10000005f8f' 2024-09-21T00:30:11.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35546 fp=#0x10000005fbe' 2024-09-21T00:30:11.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35610 fp=#0x10000005bf4' 2024-09-21T00:30:11.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35642 fp=#0x10000005bb4' 2024-09-21T00:30:11.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35674 fp=#0x10000005ba0' 2024-09-21T00:30:11.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35706 fp=#0x10000004916' 2024-09-21T00:30:11.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35738 fp=#0x1000000500e' 2024-09-21T00:30:11.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35770 fp=#0x10000004f44' 2024-09-21T00:30:11.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35802 fp=#0x10000005023' 2024-09-21T00:30:11.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35834 fp=#0x10000004fd2' 2024-09-21T00:30:11.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35866 fp=#0x10000004fc0' 2024-09-21T00:30:11.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35898 fp=#0x10000006736' 2024-09-21T00:30:11.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35930 fp=#0x10000006731' 2024-09-21T00:30:11.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35962 fp=#0x1000000688d' 2024-09-21T00:30:11.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35994 fp=#0x10000006881' 2024-09-21T00:30:11.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36026 fp=#0x10000006833' 2024-09-21T00:30:11.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36122 fp=#0x10000005f41' 2024-09-21T00:30:11.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36186 fp=#0x100000068c9' 2024-09-21T00:30:11.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36218 fp=#0x100000068e3' 2024-09-21T00:30:11.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36250 fp=#0x10000006897' 2024-09-21T00:30:11.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36282 fp=#0x1000000661e' 2024-09-21T00:30:11.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36314 fp=#0x100000065bd' 2024-09-21T00:30:11.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36346 fp=#0x10000006608' 2024-09-21T00:30:11.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36378 fp=#0x10000006601' 2024-09-21T00:30:11.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36410 fp=#0x1000000615c' 2024-09-21T00:30:11.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36442 fp=#0x10000006156' 2024-09-21T00:30:11.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36474 fp=#0x1000000615b' 2024-09-21T00:30:11.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36506 fp=#0x1000000600c' 2024-09-21T00:30:11.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36538 fp=#0x10000006005' 2024-09-21T00:30:11.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36570 fp=#0x10000005a30' 2024-09-21T00:30:11.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36602 fp=#0x100000059b0' 2024-09-21T00:30:11.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36634 fp=#0x10000005a9e' 2024-09-21T00:30:11.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36666 fp=#0x1000000490d' 2024-09-21T00:30:11.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36698 fp=#0x10000004821' 2024-09-21T00:30:11.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36730 fp=#0x1000000476e' 2024-09-21T00:30:11.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36762 fp=#0x1000000475e' 2024-09-21T00:30:11.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36794 fp=#0x1000000588f' 2024-09-21T00:30:11.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36858 fp=#0x1000000591c' 2024-09-21T00:30:11.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36890 fp=#0x100000058a3' 2024-09-21T00:30:11.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36922 fp=#0x10000005b08' 2024-09-21T00:30:11.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37018 fp=#0x100000051e6' 2024-09-21T00:30:11.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37050 fp=#0x1000000524a' 2024-09-21T00:30:11.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37114 fp=#0x10000005453' 2024-09-21T00:30:11.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37178 fp=#0x10000003d36' 2024-09-21T00:30:11.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37210 fp=#0x10000003cdb' 2024-09-21T00:30:11.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37242 fp=#0x10000005baa' 2024-09-21T00:30:11.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37274 fp=#0x10000005bd1' 2024-09-21T00:30:11.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37402 fp=#0x10000005ed6' 2024-09-21T00:30:11.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37434 fp=#0x10000005efa' 2024-09-21T00:30:11.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37466 fp=#0x10000004ae8' 2024-09-21T00:30:11.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37498 fp=#0x10000004abc' 2024-09-21T00:30:11.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37530 fp=#0x10000004a87' 2024-09-21T00:30:11.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37562 fp=#0x10000004b3e' 2024-09-21T00:30:11.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37594 fp=#0x10000005554' 2024-09-21T00:30:11.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37626 fp=#0x10000005561' 2024-09-21T00:30:11.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37658 fp=#0x10000005510' 2024-09-21T00:30:11.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37690 fp=#0x100000042c8' 2024-09-21T00:30:11.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37722 fp=#0x10000004266' 2024-09-21T00:30:11.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37754 fp=#0x10000005818' 2024-09-21T00:30:11.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37786 fp=#0x1000000427b' 2024-09-21T00:30:11.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37818 fp=#0x100000042cc' 2024-09-21T00:30:11.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37850 fp=#0x10000004262' 2024-09-21T00:30:11.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37882 fp=#0x10000001c3b' 2024-09-21T00:30:11.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37914 fp=#0x10000001c7a' 2024-09-21T00:30:11.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37946 fp=#0x10000001b7f' 2024-09-21T00:30:11.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37978 fp=#0x10000001d4c' 2024-09-21T00:30:11.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38010 fp=#0x10000001cb6' 2024-09-21T00:30:11.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38202 fp=#0x10000005997' 2024-09-21T00:30:11.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38266 fp=#0x10000005924' 2024-09-21T00:30:11.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38298 fp=#0x100000059ac' 2024-09-21T00:30:11.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38330 fp=#0x10000005971' 2024-09-21T00:30:11.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38362 fp=#0x10000005876' 2024-09-21T00:30:11.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38426 fp=#0x10000002bc7' 2024-09-21T00:30:11.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38458 fp=#0x10000002ad2' 2024-09-21T00:30:11.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38490 fp=#0x10000004608' 2024-09-21T00:30:11.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38522 fp=#0x100000045cd' 2024-09-21T00:30:11.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38586 fp=#0x10000002a53' 2024-09-21T00:30:11.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38618 fp=#0x10000002ba3' 2024-09-21T00:30:11.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38714 fp=#0x10000004882' 2024-09-21T00:30:11.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38810 fp=#0x10000001fde' 2024-09-21T00:30:11.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38842 fp=#0x10000001f84' 2024-09-21T00:30:11.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38906 fp=#0x10000003ffa' 2024-09-21T00:30:11.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38970 fp=#0x10000004115' 2024-09-21T00:30:11.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39066 fp=#0x100000017ee' 2024-09-21T00:30:11.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39098 fp=#0x10000001fbe' 2024-09-21T00:30:11.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39130 fp=#0x100000017f4' 2024-09-21T00:30:11.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39162 fp=#0x10000004500' 2024-09-21T00:30:11.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39194 fp=#0x100000044c1' 2024-09-21T00:30:11.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39258 fp=#0x100000044f7' 2024-09-21T00:30:11.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39290 fp=#0x1000000481e' 2024-09-21T00:30:11.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39322 fp=#0x100000048a5' 2024-09-21T00:30:11.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39354 fp=#0x100000048bf' 2024-09-21T00:30:11.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39418 fp=#0x10000005e25' 2024-09-21T00:30:11.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39450 fp=#0x10000005e09' 2024-09-21T00:30:11.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39482 fp=#0x10000005e17' 2024-09-21T00:30:11.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39514 fp=#0x10000005adc' 2024-09-21T00:30:11.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39578 fp=#0x100000050f9' 2024-09-21T00:30:11.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39642 fp=#0x100000050ea' 2024-09-21T00:30:11.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39674 fp=#0x10000005de8' 2024-09-21T00:30:11.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39706 fp=#0x10000005de6' 2024-09-21T00:30:11.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39802 fp=#0x10000005b64' 2024-09-21T00:30:11.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39834 fp=#0x10000006316' 2024-09-21T00:30:11.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39866 fp=#0x10000006323' 2024-09-21T00:30:11.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39898 fp=#0x10000004ea4' 2024-09-21T00:30:11.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39930 fp=#0x10000004e98' 2024-09-21T00:30:11.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39962 fp=#0x10000004ed7' 2024-09-21T00:30:11.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39994 fp=#0x10000004ef0' 2024-09-21T00:30:11.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40026 fp=#0x10000006261' 2024-09-21T00:30:11.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40058 fp=#0x10000006260' 2024-09-21T00:30:11.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40090 fp=#0x100000061dc' 2024-09-21T00:30:11.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40122 fp=#0x1000000623c' 2024-09-21T00:30:11.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40154 fp=#0x100000026b7' 2024-09-21T00:30:11.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40186 fp=#0x10000005235' 2024-09-21T00:30:11.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40218 fp=#0x1000000261f' 2024-09-21T00:30:11.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40282 fp=#0x10000000a8f' 2024-09-21T00:30:11.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40314 fp=#0x10000004321' 2024-09-21T00:30:11.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40346 fp=#0x100000042f2' 2024-09-21T00:30:11.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40378 fp=#0x100000008bb' 2024-09-21T00:30:11.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40410 fp=#0x10000000a88' 2024-09-21T00:30:11.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40442 fp=#0x10000005975' 2024-09-21T00:30:11.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40570 fp=#0x100000037f5' 2024-09-21T00:30:11.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40666 fp=#0x10000004845' 2024-09-21T00:30:11.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40698 fp=#0x10000005773' 2024-09-21T00:30:11.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40730 fp=#0x10000005ebe' 2024-09-21T00:30:11.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40794 fp=#0x10000004808' 2024-09-21T00:30:11.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40826 fp=#0x100000031b8' 2024-09-21T00:30:11.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40922 fp=#0x1000000303c' 2024-09-21T00:30:11.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40986 fp=#0x10000005127' 2024-09-21T00:30:11.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41018 fp=#0x10000004987' 2024-09-21T00:30:11.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41050 fp=#0x10000005143' 2024-09-21T00:30:11.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41082 fp=#0x1000000511c' 2024-09-21T00:30:11.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41114 fp=#0x100000049ee' 2024-09-21T00:30:11.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30619 fp=#0x10000000208' 2024-09-21T00:30:11.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30779 fp=#0x10000005d38' 2024-09-21T00:30:11.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30811 fp=#0x1000000330a' 2024-09-21T00:30:11.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30875 fp=#0x100000058e9' 2024-09-21T00:30:11.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32699 fp=#0x10000005792' 2024-09-21T00:30:11.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32731 fp=#0x10000005752' 2024-09-21T00:30:11.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32763 fp=#0x1000000576c' 2024-09-21T00:30:11.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32795 fp=#0x10000005051' 2024-09-21T00:30:11.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32827 fp=#0x10000005072' 2024-09-21T00:30:11.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32859 fp=#0x10000005076' 2024-09-21T00:30:11.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32891 fp=#0x10000005088' 2024-09-21T00:30:11.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32923 fp=#0x1000000505b' 2024-09-21T00:30:11.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32955 fp=#0x1000000503b' 2024-09-21T00:30:11.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32987 fp=#0x10000005c7d' 2024-09-21T00:30:11.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33019 fp=#0x10000005cd2' 2024-09-21T00:30:11.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33051 fp=#0x10000005c4e' 2024-09-21T00:30:11.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33083 fp=#0x10000005c35' 2024-09-21T00:30:11.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33115 fp=#0x10000005c40' 2024-09-21T00:30:11.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33179 fp=#0x10000006671' 2024-09-21T00:30:11.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33211 fp=#0x10000006046' 2024-09-21T00:30:11.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33275 fp=#0x10000006058' 2024-09-21T00:30:11.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33307 fp=#0x10000006805' 2024-09-21T00:30:11.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33339 fp=#0x100000067da' 2024-09-21T00:30:11.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33371 fp=#0x100000067ed' 2024-09-21T00:30:11.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33403 fp=#0x10000003d67' 2024-09-21T00:30:11.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33435 fp=#0x10000003e33' 2024-09-21T00:30:11.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33499 fp=#0x10000003dd9' 2024-09-21T00:30:11.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33531 fp=#0x1000000565a' 2024-09-21T00:30:11.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33563 fp=#0x10000005a64' 2024-09-21T00:30:11.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33595 fp=#0x10000005ab3' 2024-09-21T00:30:11.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33627 fp=#0x10000005aa5' 2024-09-21T00:30:11.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33659 fp=#0x10000005a0e' 2024-09-21T00:30:11.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33691 fp=#0x10000005a9d' 2024-09-21T00:30:11.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33723 fp=#0x1000000679d' 2024-09-21T00:30:11.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33755 fp=#0x1000000675a' 2024-09-21T00:30:11.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33787 fp=#0x100000067ad' 2024-09-21T00:30:11.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33915 fp=#0x100000047e8' 2024-09-21T00:30:11.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33947 fp=#0x10000006424' 2024-09-21T00:30:11.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33979 fp=#0x10000006407' 2024-09-21T00:30:11.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34011 fp=#0x1000000475c' 2024-09-21T00:30:11.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34043 fp=#0x10000006401' 2024-09-21T00:30:11.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34075 fp=#0x1000000323b' 2024-09-21T00:30:11.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34107 fp=#0x10000003230' 2024-09-21T00:30:11.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34139 fp=#0x100000061f3' 2024-09-21T00:30:11.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34171 fp=#0x10000006225' 2024-09-21T00:30:11.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34203 fp=#0x10000006221' 2024-09-21T00:30:11.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34235 fp=#0x1000000620b' 2024-09-21T00:30:11.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34267 fp=#0x10000006216' 2024-09-21T00:30:11.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34331 fp=#0x1000000619d' 2024-09-21T00:30:11.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34363 fp=#0x10000002696' 2024-09-21T00:30:11.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34395 fp=#0x100000026d4' 2024-09-21T00:30:11.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34459 fp=#0x1000000619c' 2024-09-21T00:30:11.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34491 fp=#0x1000000569a' 2024-09-21T00:30:11.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34555 fp=#0x100000062d1' 2024-09-21T00:30:11.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34587 fp=#0x100000062bf' 2024-09-21T00:30:11.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34619 fp=#0x10000006362' 2024-09-21T00:30:11.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34651 fp=#0x10000006329' 2024-09-21T00:30:11.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34683 fp=#0x1000000633f' 2024-09-21T00:30:11.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34779 fp=#0x100000057e7' 2024-09-21T00:30:11.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34811 fp=#0x10000006634' 2024-09-21T00:30:11.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34843 fp=#0x100000057e0' 2024-09-21T00:30:11.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34875 fp=#0x100000064c3' 2024-09-21T00:30:11.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34907 fp=#0x10000005352' 2024-09-21T00:30:11.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34939 fp=#0x100000052ae' 2024-09-21T00:30:11.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34971 fp=#0x10000006959' 2024-09-21T00:30:11.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35003 fp=#0x10000005331' 2024-09-21T00:30:11.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35035 fp=#0x10000006958' 2024-09-21T00:30:11.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35067 fp=#0x100000060a4' 2024-09-21T00:30:11.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35131 fp=#0x100000060b7' 2024-09-21T00:30:11.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35163 fp=#0x100000060a7' 2024-09-21T00:30:11.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35195 fp=#0x100000060af' 2024-09-21T00:30:11.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35259 fp=#0x10000003f9f' 2024-09-21T00:30:11.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35291 fp=#0x10000003f1a' 2024-09-21T00:30:11.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35323 fp=#0x10000005dec' 2024-09-21T00:30:11.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35451 fp=#0x10000005f6a' 2024-09-21T00:30:11.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35547 fp=#0x10000005f8c' 2024-09-21T00:30:11.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35579 fp=#0x10000005f6d' 2024-09-21T00:30:11.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35611 fp=#0x10000005b76' 2024-09-21T00:30:11.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35675 fp=#0x10000005b65' 2024-09-21T00:30:11.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35707 fp=#0x1000000496c' 2024-09-21T00:30:11.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35739 fp=#0x10000004f12' 2024-09-21T00:30:11.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35771 fp=#0x10000004ff8' 2024-09-21T00:30:11.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35803 fp=#0x10000004fbc' 2024-09-21T00:30:11.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35835 fp=#0x10000004f52' 2024-09-21T00:30:11.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35867 fp=#0x10000004f30' 2024-09-21T00:30:11.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35899 fp=#0x1000000672a' 2024-09-21T00:30:11.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35931 fp=#0x1000000671f' 2024-09-21T00:30:11.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35963 fp=#0x1000000686f' 2024-09-21T00:30:11.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35995 fp=#0x10000006873' 2024-09-21T00:30:11.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36027 fp=#0x1000000687b' 2024-09-21T00:30:11.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36091 fp=#0x10000005f3b' 2024-09-21T00:30:11.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36187 fp=#0x1000000689a' 2024-09-21T00:30:11.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36219 fp=#0x1000000689e' 2024-09-21T00:30:11.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36251 fp=#0x100000068b6' 2024-09-21T00:30:11.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36283 fp=#0x100000065b6' 2024-09-21T00:30:11.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36379 fp=#0x100000065b0' 2024-09-21T00:30:11.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36411 fp=#0x10000006153' 2024-09-21T00:30:11.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36443 fp=#0x1000000615e' 2024-09-21T00:30:11.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36475 fp=#0x10000006045' 2024-09-21T00:30:11.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36507 fp=#0x10000006010' 2024-09-21T00:30:11.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36539 fp=#0x10000006011' 2024-09-21T00:30:11.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36571 fp=#0x10000005a93' 2024-09-21T00:30:11.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36603 fp=#0x100000059ce' 2024-09-21T00:30:11.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36667 fp=#0x10000004762' 2024-09-21T00:30:11.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36699 fp=#0x100000048aa' 2024-09-21T00:30:11.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36731 fp=#0x100000047e2' 2024-09-21T00:30:11.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36763 fp=#0x100000047c7' 2024-09-21T00:30:11.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36795 fp=#0x10000005aff' 2024-09-21T00:30:11.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36891 fp=#0x100000058a8' 2024-09-21T00:30:11.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36955 fp=#0x100000058e1' 2024-09-21T00:30:11.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36987 fp=#0x10000005288' 2024-09-21T00:30:11.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37019 fp=#0x100000051a6' 2024-09-21T00:30:11.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37051 fp=#0x100000051d8' 2024-09-21T00:30:11.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37083 fp=#0x100000051b7' 2024-09-21T00:30:11.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37179 fp=#0x10000003d3c' 2024-09-21T00:30:11.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37211 fp=#0x10000003ce5' 2024-09-21T00:30:11.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37275 fp=#0x10000005bc6' 2024-09-21T00:30:11.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37307 fp=#0x10000005bc7' 2024-09-21T00:30:11.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37339 fp=#0x10000005bb8' 2024-09-21T00:30:11.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37371 fp=#0x10000005c1b' 2024-09-21T00:30:11.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37403 fp=#0x10000005ee5' 2024-09-21T00:30:11.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37435 fp=#0x10000005f05' 2024-09-21T00:30:11.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37467 fp=#0x10000004b97' 2024-09-21T00:30:11.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37499 fp=#0x10000004bc0' 2024-09-21T00:30:11.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37531 fp=#0x10000004ae1' 2024-09-21T00:30:11.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37563 fp=#0x10000004b2d' 2024-09-21T00:30:11.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37595 fp=#0x10000005481' 2024-09-21T00:30:11.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37627 fp=#0x100000054ad' 2024-09-21T00:30:11.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37659 fp=#0x100000054f1' 2024-09-21T00:30:11.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37691 fp=#0x10000005837' 2024-09-21T00:30:11.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37723 fp=#0x10000005805' 2024-09-21T00:30:11.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37755 fp=#0x100000042d2' 2024-09-21T00:30:11.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37787 fp=#0x1000000580f' 2024-09-21T00:30:11.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37819 fp=#0x10000004277' 2024-09-21T00:30:11.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37851 fp=#0x1000000582f' 2024-09-21T00:30:11.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37883 fp=#0x10000001d33' 2024-09-21T00:30:11.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37915 fp=#0x10000001d15' 2024-09-21T00:30:11.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37947 fp=#0x10000001b94' 2024-09-21T00:30:11.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37979 fp=#0x10000001ca8' 2024-09-21T00:30:11.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38011 fp=#0x10000001c82' 2024-09-21T00:30:11.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38043 fp=#0x10000003c55' 2024-09-21T00:30:11.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38107 fp=#0x10000003ad7' 2024-09-21T00:30:11.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38171 fp=#0x100000060f4' 2024-09-21T00:30:11.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38235 fp=#0x1000000588a' 2024-09-21T00:30:11.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38267 fp=#0x10000005961' 2024-09-21T00:30:11.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38299 fp=#0x1000000618c' 2024-09-21T00:30:11.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38331 fp=#0x1000000587e' 2024-09-21T00:30:11.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38363 fp=#0x10000005988' 2024-09-21T00:30:11.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38395 fp=#0x10000005959' 2024-09-21T00:30:11.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38427 fp=#0x100000045fe' 2024-09-21T00:30:11.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38459 fp=#0x10000002b59' 2024-09-21T00:30:11.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38523 fp=#0x10000004603' 2024-09-21T00:30:11.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38555 fp=#0x10000002a02' 2024-09-21T00:30:11.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38587 fp=#0x100000045de' 2024-09-21T00:30:11.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38619 fp=#0x100000045fd' 2024-09-21T00:30:11.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38715 fp=#0x100000048b9' 2024-09-21T00:30:11.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38811 fp=#0x100000010e3' 2024-09-21T00:30:11.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38843 fp=#0x1000000402c' 2024-09-21T00:30:11.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38875 fp=#0x10000004090' 2024-09-21T00:30:11.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38907 fp=#0x100000041e8' 2024-09-21T00:30:11.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38939 fp=#0x1000000402e' 2024-09-21T00:30:11.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39067 fp=#0x10000001fa1' 2024-09-21T00:30:11.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39099 fp=#0x10000001fdf' 2024-09-21T00:30:11.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39131 fp=#0x10000001dae' 2024-09-21T00:30:11.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39163 fp=#0x100000044b7' 2024-09-21T00:30:11.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39195 fp=#0x100000044f3' 2024-09-21T00:30:11.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39227 fp=#0x100000044ae' 2024-09-21T00:30:11.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39259 fp=#0x10000004494' 2024-09-21T00:30:11.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39291 fp=#0x100000048f6' 2024-09-21T00:30:11.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39323 fp=#0x100000048fe' 2024-09-21T00:30:11.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39355 fp=#0x100000048ae' 2024-09-21T00:30:11.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39387 fp=#0x10000005aee' 2024-09-21T00:30:11.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39419 fp=#0x10000005ad8' 2024-09-21T00:30:11.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39451 fp=#0x10000005dfa' 2024-09-21T00:30:11.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39483 fp=#0x100000059bb' 2024-09-21T00:30:11.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39515 fp=#0x100000059db' 2024-09-21T00:30:11.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39547 fp=#0x10000005e3b' 2024-09-21T00:30:11.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39579 fp=#0x100000050a5' 2024-09-21T00:30:11.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39643 fp=#0x1000000509a' 2024-09-21T00:30:11.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39675 fp=#0x10000005de7' 2024-09-21T00:30:11.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39707 fp=#0x10000005d8c' 2024-09-21T00:30:11.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39739 fp=#0x10000005b4f' 2024-09-21T00:30:11.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39771 fp=#0x10000005b45' 2024-09-21T00:30:11.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39803 fp=#0x10000005b4a' 2024-09-21T00:30:11.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39835 fp=#0x10000006351' 2024-09-21T00:30:11.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39867 fp=#0x100000062f5' 2024-09-21T00:30:11.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39899 fp=#0x10000004ef6' 2024-09-21T00:30:11.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39931 fp=#0x10000004ea3' 2024-09-21T00:30:11.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39963 fp=#0x10000004edb' 2024-09-21T00:30:11.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39995 fp=#0x10000004e8e' 2024-09-21T00:30:11.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40027 fp=#0x1000000629f' 2024-09-21T00:30:11.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40059 fp=#0x1000000624b' 2024-09-21T00:30:11.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40091 fp=#0x1000000622e' 2024-09-21T00:30:11.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40123 fp=#0x10000006241' 2024-09-21T00:30:11.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40155 fp=#0x10000002631' 2024-09-21T00:30:11.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40187 fp=#0x10000002648' 2024-09-21T00:30:11.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40219 fp=#0x10000000aeb' 2024-09-21T00:30:11.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40251 fp=#0x100000042c9' 2024-09-21T00:30:11.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40283 fp=#0x100000008a3' 2024-09-21T00:30:11.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40315 fp=#0x10000004d63' 2024-09-21T00:30:11.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40347 fp=#0x100000042b5' 2024-09-21T00:30:11.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40379 fp=#0x10000000891' 2024-09-21T00:30:11.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40411 fp=#0x1000000088e' 2024-09-21T00:30:11.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40443 fp=#0x10000005981' 2024-09-21T00:30:11.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40507 fp=#0x100000028ad' 2024-09-21T00:30:11.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40539 fp=#0x1000000288b' 2024-09-21T00:30:11.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40667 fp=#0x10000004932' 2024-09-21T00:30:11.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40699 fp=#0x10000004871' 2024-09-21T00:30:11.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40731 fp=#0x100000054a8' 2024-09-21T00:30:11.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40763 fp=#0x1000000486c' 2024-09-21T00:30:11.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40795 fp=#0x10000004816' 2024-09-21T00:30:11.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40827 fp=#0x100000063ea' 2024-09-21T00:30:11.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40891 fp=#0x100000063c7' 2024-09-21T00:30:11.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40923 fp=#0x100000031ae' 2024-09-21T00:30:11.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40987 fp=#0x10000005105' 2024-09-21T00:30:11.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41019 fp=#0x10000005107' 2024-09-21T00:30:11.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41051 fp=#0x1000000511f' 2024-09-21T00:30:11.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41083 fp=#0x100000049df' 2024-09-21T00:30:11.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41115 fp=#0x10000005149' 2024-09-21T00:30:11.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30780 fp=#0x100000067c8' 2024-09-21T00:30:11.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30812 fp=#0x100000017dc' 2024-09-21T00:30:11.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30876 fp=#0x100000010b2' 2024-09-21T00:30:11.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32700 fp=#0x1000000572e' 2024-09-21T00:30:11.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32732 fp=#0x1000000569c' 2024-09-21T00:30:11.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32796 fp=#0x10000005085' 2024-09-21T00:30:11.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32828 fp=#0x1000000505e' 2024-09-21T00:30:11.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32892 fp=#0x10000005060' 2024-09-21T00:30:11.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32924 fp=#0x10000005087' 2024-09-21T00:30:11.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32956 fp=#0x1000000506d' 2024-09-21T00:30:11.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32988 fp=#0x10000005d30' 2024-09-21T00:30:11.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33020 fp=#0x10000005cb2' 2024-09-21T00:30:11.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33052 fp=#0x10000005c58' 2024-09-21T00:30:11.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33084 fp=#0x10000005cf0' 2024-09-21T00:30:11.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33116 fp=#0x10000005c73' 2024-09-21T00:30:11.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33244 fp=#0x1000000604f' 2024-09-21T00:30:11.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33308 fp=#0x1000000681f' 2024-09-21T00:30:11.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33340 fp=#0x100000067ea' 2024-09-21T00:30:11.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33372 fp=#0x1000000680a' 2024-09-21T00:30:11.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33404 fp=#0x10000003ecf' 2024-09-21T00:30:11.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33468 fp=#0x10000003e77' 2024-09-21T00:30:11.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33500 fp=#0x10000003e6b' 2024-09-21T00:30:11.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33532 fp=#0x10000005a1a' 2024-09-21T00:30:11.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33564 fp=#0x10000005a7c' 2024-09-21T00:30:11.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33596 fp=#0x100000055b7' 2024-09-21T00:30:11.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33628 fp=#0x10000005670' 2024-09-21T00:30:11.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33660 fp=#0x1000000559a' 2024-09-21T00:30:11.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33692 fp=#0x10000005a87' 2024-09-21T00:30:11.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33724 fp=#0x100000067c1' 2024-09-21T00:30:11.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33756 fp=#0x1000000678d' 2024-09-21T00:30:11.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33788 fp=#0x1000000676f' 2024-09-21T00:30:11.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33916 fp=#0x100000047f2' 2024-09-21T00:30:11.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33948 fp=#0x10000006425' 2024-09-21T00:30:11.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33980 fp=#0x1000000641f' 2024-09-21T00:30:11.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34044 fp=#0x10000006433' 2024-09-21T00:30:11.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34076 fp=#0x1000000326c' 2024-09-21T00:30:11.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34108 fp=#0x100000031fd' 2024-09-21T00:30:11.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34140 fp=#0x1000000621b' 2024-09-21T00:30:11.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34172 fp=#0x10000006285' 2024-09-21T00:30:11.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34204 fp=#0x10000006229' 2024-09-21T00:30:11.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34236 fp=#0x10000006280' 2024-09-21T00:30:11.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34268 fp=#0x100000064a1' 2024-09-21T00:30:11.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34300 fp=#0x10000002567' 2024-09-21T00:30:11.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34332 fp=#0x100000061a1' 2024-09-21T00:30:11.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34364 fp=#0x1000000567a' 2024-09-21T00:30:11.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34396 fp=#0x100000061bd' 2024-09-21T00:30:11.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34428 fp=#0x100000025a6' 2024-09-21T00:30:11.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34460 fp=#0x10000002587' 2024-09-21T00:30:11.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34492 fp=#0x10000002589' 2024-09-21T00:30:11.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34524 fp=#0x100000061d5' 2024-09-21T00:30:11.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34556 fp=#0x100000062ff' 2024-09-21T00:30:11.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34588 fp=#0x10000006359' 2024-09-21T00:30:11.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34620 fp=#0x1000000632d' 2024-09-21T00:30:11.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34652 fp=#0x100000062e7' 2024-09-21T00:30:11.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34684 fp=#0x10000006349' 2024-09-21T00:30:11.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34748 fp=#0x100000057b3' 2024-09-21T00:30:11.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34780 fp=#0x100000064ed' 2024-09-21T00:30:11.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34812 fp=#0x100000064ce' 2024-09-21T00:30:11.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34844 fp=#0x100000064c4' 2024-09-21T00:30:11.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34876 fp=#0x100000064cc' 2024-09-21T00:30:11.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34908 fp=#0x10000005358' 2024-09-21T00:30:11.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34940 fp=#0x1000000530b' 2024-09-21T00:30:11.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34972 fp=#0x100000052e4' 2024-09-21T00:30:11.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35036 fp=#0x1000000529c' 2024-09-21T00:30:11.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35132 fp=#0x100000060a9' 2024-09-21T00:30:11.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35228 fp=#0x100000060f1' 2024-09-21T00:30:11.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35292 fp=#0x10000003fac' 2024-09-21T00:30:11.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35324 fp=#0x10000003f17' 2024-09-21T00:30:11.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35388 fp=#0x10000005391' 2024-09-21T00:30:11.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35452 fp=#0x10000005fee' 2024-09-21T00:30:11.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35612 fp=#0x10000005be1' 2024-09-21T00:30:11.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35644 fp=#0x10000005b69' 2024-09-21T00:30:11.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35676 fp=#0x10000005bb9' 2024-09-21T00:30:11.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35708 fp=#0x1000000495d' 2024-09-21T00:30:11.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35740 fp=#0x10000004f39' 2024-09-21T00:30:11.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35772 fp=#0x10000004fa1' 2024-09-21T00:30:11.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35804 fp=#0x10000005014' 2024-09-21T00:30:11.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35836 fp=#0x10000004faa' 2024-09-21T00:30:11.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35868 fp=#0x1000000500a' 2024-09-21T00:30:11.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35900 fp=#0x10000006740' 2024-09-21T00:30:11.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35932 fp=#0x10000006724' 2024-09-21T00:30:11.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35964 fp=#0x1000000684c' 2024-09-21T00:30:11.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35996 fp=#0x10000006856' 2024-09-21T00:30:11.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36028 fp=#0x10000006861' 2024-09-21T00:30:11.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36060 fp=#0x10000005f3e' 2024-09-21T00:30:11.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36156 fp=#0x10000005f47' 2024-09-21T00:30:11.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36188 fp=#0x100000068b2' 2024-09-21T00:30:11.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36220 fp=#0x1000000689d' 2024-09-21T00:30:11.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36252 fp=#0x100000068b4' 2024-09-21T00:30:11.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36316 fp=#0x10000006602' 2024-09-21T00:30:11.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36348 fp=#0x100000065b1' 2024-09-21T00:30:11.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36380 fp=#0x100000065fb' 2024-09-21T00:30:11.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36412 fp=#0x1000000613e' 2024-09-21T00:30:11.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36444 fp=#0x1000000615d' 2024-09-21T00:30:11.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36476 fp=#0x10000006038' 2024-09-21T00:30:11.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36572 fp=#0x10000005a7a' 2024-09-21T00:30:11.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36636 fp=#0x10000005a6c' 2024-09-21T00:30:11.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36668 fp=#0x10000004799' 2024-09-21T00:30:11.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36700 fp=#0x1000000478a' 2024-09-21T00:30:11.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36732 fp=#0x1000000479d' 2024-09-21T00:30:11.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36764 fp=#0x100000048b2' 2024-09-21T00:30:11.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36828 fp=#0x100000058fe' 2024-09-21T00:30:11.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36860 fp=#0x10000005930' 2024-09-21T00:30:11.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36892 fp=#0x10000005f39' 2024-09-21T00:30:11.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36956 fp=#0x1000000591f' 2024-09-21T00:30:11.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36988 fp=#0x100000051c9' 2024-09-21T00:30:11.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37020 fp=#0x100000051f6' 2024-09-21T00:30:11.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37052 fp=#0x100000051a8' 2024-09-21T00:30:11.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37084 fp=#0x1000000525b' 2024-09-21T00:30:11.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37116 fp=#0x10000005446' 2024-09-21T00:30:11.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37148 fp=#0x100000053c4' 2024-09-21T00:30:11.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37180 fp=#0x10000003cc2' 2024-09-21T00:30:11.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37212 fp=#0x10000003cd3' 2024-09-21T00:30:11.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37244 fp=#0x10000005c60' 2024-09-21T00:30:11.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37276 fp=#0x10000005bde' 2024-09-21T00:30:11.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37372 fp=#0x10000005ef4' 2024-09-21T00:30:11.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37404 fp=#0x10000005f0b' 2024-09-21T00:30:11.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37436 fp=#0x10000005eef' 2024-09-21T00:30:11.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37468 fp=#0x10000004a6a' 2024-09-21T00:30:11.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37500 fp=#0x10000004b43' 2024-09-21T00:30:11.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37532 fp=#0x10000004bad' 2024-09-21T00:30:11.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37564 fp=#0x10000004b8f' 2024-09-21T00:30:11.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37596 fp=#0x10000005551' 2024-09-21T00:30:11.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37628 fp=#0x10000005483' 2024-09-21T00:30:11.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37660 fp=#0x1000000548c' 2024-09-21T00:30:11.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37692 fp=#0x10000004305' 2024-09-21T00:30:11.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37724 fp=#0x100000042bc' 2024-09-21T00:30:11.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37756 fp=#0x100000057fc' 2024-09-21T00:30:11.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37788 fp=#0x10000005832' 2024-09-21T00:30:11.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37820 fp=#0x100000042b8' 2024-09-21T00:30:11.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37852 fp=#0x10000005827' 2024-09-21T00:30:11.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37884 fp=#0x10000001b9c' 2024-09-21T00:30:11.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37916 fp=#0x10000001cf6' 2024-09-21T00:30:11.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37948 fp=#0x10000001c0d' 2024-09-21T00:30:11.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37980 fp=#0x10000001cc6' 2024-09-21T00:30:11.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38012 fp=#0x10000001cf3' 2024-09-21T00:30:11.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38044 fp=#0x10000003c9e' 2024-09-21T00:30:11.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38108 fp=#0x10000003b38' 2024-09-21T00:30:11.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38204 fp=#0x1000000584c' 2024-09-21T00:30:11.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38236 fp=#0x1000000596d' 2024-09-21T00:30:11.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38268 fp=#0x1000000617f' 2024-09-21T00:30:11.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38300 fp=#0x10000005921' 2024-09-21T00:30:11.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38332 fp=#0x10000005879' 2024-09-21T00:30:11.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38364 fp=#0x10000006172' 2024-09-21T00:30:11.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38396 fp=#0x1000000584a' 2024-09-21T00:30:11.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38428 fp=#0x100000045f4' 2024-09-21T00:30:11.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38460 fp=#0x100000045f1' 2024-09-21T00:30:11.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38492 fp=#0x100000045d6' 2024-09-21T00:30:11.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38524 fp=#0x10000002b1f' 2024-09-21T00:30:11.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38588 fp=#0x100000045e6' 2024-09-21T00:30:11.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38716 fp=#0x10000001fa8' 2024-09-21T00:30:11.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38812 fp=#0x10000001f78' 2024-09-21T00:30:11.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38844 fp=#0x100000041d6' 2024-09-21T00:30:11.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38876 fp=#0x1000000404b' 2024-09-21T00:30:11.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38908 fp=#0x10000004176' 2024-09-21T00:30:11.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38940 fp=#0x10000004039' 2024-09-21T00:30:11.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38972 fp=#0x10000004047' 2024-09-21T00:30:11.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39004 fp=#0x10000004124' 2024-09-21T00:30:11.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39036 fp=#0x10000004076' 2024-09-21T00:30:11.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39068 fp=#0x10000001f9c' 2024-09-21T00:30:11.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39100 fp=#0x10000001d3b' 2024-09-21T00:30:11.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39132 fp=#0x10000001c56' 2024-09-21T00:30:11.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39164 fp=#0x100000044ff' 2024-09-21T00:30:11.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39228 fp=#0x100000044fb' 2024-09-21T00:30:11.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39260 fp=#0x100000044bd' 2024-09-21T00:30:11.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39292 fp=#0x1000000485f' 2024-09-21T00:30:11.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39324 fp=#0x100000048ea' 2024-09-21T00:30:11.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39356 fp=#0x10000004872' 2024-09-21T00:30:11.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39388 fp=#0x100000059e4' 2024-09-21T00:30:11.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39420 fp=#0x10000005a40' 2024-09-21T00:30:11.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39484 fp=#0x10000005e36' 2024-09-21T00:30:11.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39516 fp=#0x10000005e01' 2024-09-21T00:30:11.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39548 fp=#0x10000005e2c' 2024-09-21T00:30:11.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39580 fp=#0x100000050e3' 2024-09-21T00:30:11.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39644 fp=#0x10000005095' 2024-09-21T00:30:11.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39676 fp=#0x10000005df2' 2024-09-21T00:30:11.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39804 fp=#0x10000005b50' 2024-09-21T00:30:11.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39836 fp=#0x10000006384' 2024-09-21T00:30:11.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39868 fp=#0x100000062d8' 2024-09-21T00:30:11.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39900 fp=#0x10000004e8f' 2024-09-21T00:30:11.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39932 fp=#0x10000004e9a' 2024-09-21T00:30:11.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39996 fp=#0x10000004ede' 2024-09-21T00:30:11.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40028 fp=#0x100000061e4' 2024-09-21T00:30:11.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40060 fp=#0x100000061d9' 2024-09-21T00:30:11.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40092 fp=#0x100000061f4' 2024-09-21T00:30:11.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40124 fp=#0x10000006275' 2024-09-21T00:30:11.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40156 fp=#0x100000026b4' 2024-09-21T00:30:11.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40188 fp=#0x10000002610' 2024-09-21T00:30:11.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40220 fp=#0x100000042cd' 2024-09-21T00:30:11.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40252 fp=#0x100000042cf' 2024-09-21T00:30:11.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40284 fp=#0x1000000428f' 2024-09-21T00:30:11.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40316 fp=#0x100000042e7' 2024-09-21T00:30:11.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40348 fp=#0x10000000868' 2024-09-21T00:30:11.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40380 fp=#0x10000004300' 2024-09-21T00:30:11.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40412 fp=#0x10000000897' 2024-09-21T00:30:11.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40444 fp=#0x100000058ef' 2024-09-21T00:30:11.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40508 fp=#0x100000028b1' 2024-09-21T00:30:11.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40668 fp=#0x10000004802' 2024-09-21T00:30:11.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40700 fp=#0x10000005e9e' 2024-09-21T00:30:11.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40732 fp=#0x10000005ecf' 2024-09-21T00:30:11.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40764 fp=#0x10000004938' 2024-09-21T00:30:11.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40796 fp=#0x10000004865' 2024-09-21T00:30:11.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40828 fp=#0x100000063e0' 2024-09-21T00:30:11.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40860 fp=#0x100000031be' 2024-09-21T00:30:11.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40988 fp=#0x10000004a37' 2024-09-21T00:30:11.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41020 fp=#0x100000049ad' 2024-09-21T00:30:11.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41052 fp=#0x10000005142' 2024-09-21T00:30:11.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41084 fp=#0x1000000499c' 2024-09-21T00:30:11.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41116 fp=#0x1000000513a' 2024-09-21T00:30:11.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30781 fp=#0x10000002d80' 2024-09-21T00:30:11.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30813 fp=#0x10000001872' 2024-09-21T00:30:11.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30877 fp=#0x1000000688e' 2024-09-21T00:30:11.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32701 fp=#0x10000005742' 2024-09-21T00:30:11.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32733 fp=#0x10000005707' 2024-09-21T00:30:11.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32765 fp=#0x10000005693' 2024-09-21T00:30:11.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32797 fp=#0x10000005028' 2024-09-21T00:30:11.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32829 fp=#0x10000005069' 2024-09-21T00:30:11.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32861 fp=#0x1000000502d' 2024-09-21T00:30:11.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32893 fp=#0x10000005043' 2024-09-21T00:30:11.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32925 fp=#0x10000005067' 2024-09-21T00:30:11.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32989 fp=#0x10000005ccd' 2024-09-21T00:30:11.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33021 fp=#0x10000005c91' 2024-09-21T00:30:11.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33053 fp=#0x10000005c7c' 2024-09-21T00:30:11.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33085 fp=#0x10000005c8d' 2024-09-21T00:30:11.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33117 fp=#0x10000005ce5' 2024-09-21T00:30:11.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33213 fp=#0x10000006691' 2024-09-21T00:30:11.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33277 fp=#0x10000006687' 2024-09-21T00:30:11.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33309 fp=#0x100000067d2' 2024-09-21T00:30:11.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33341 fp=#0x100000067f5' 2024-09-21T00:30:11.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33373 fp=#0x10000006806' 2024-09-21T00:30:11.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33405 fp=#0x10000003df3' 2024-09-21T00:30:11.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33437 fp=#0x10000003dad' 2024-09-21T00:30:11.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33469 fp=#0x10000003e2e' 2024-09-21T00:30:11.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33501 fp=#0x10000003d4e' 2024-09-21T00:30:11.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33533 fp=#0x10000005650' 2024-09-21T00:30:11.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33597 fp=#0x100000055c3' 2024-09-21T00:30:11.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33629 fp=#0x10000005a29' 2024-09-21T00:30:11.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33661 fp=#0x10000005657' 2024-09-21T00:30:11.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33693 fp=#0x10000005a79' 2024-09-21T00:30:11.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33725 fp=#0x1000000677f' 2024-09-21T00:30:11.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33757 fp=#0x10000006794' 2024-09-21T00:30:11.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33789 fp=#0x10000006765' 2024-09-21T00:30:11.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33917 fp=#0x10000004749' 2024-09-21T00:30:11.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33981 fp=#0x10000006411' 2024-09-21T00:30:11.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34013 fp=#0x10000006415' 2024-09-21T00:30:11.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34045 fp=#0x10000006414' 2024-09-21T00:30:11.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34077 fp=#0x10000003273' 2024-09-21T00:30:11.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34109 fp=#0x100000031ef' 2024-09-21T00:30:11.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34141 fp=#0x10000006245' 2024-09-21T00:30:11.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34173 fp=#0x1000000646a' 2024-09-21T00:30:11.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34205 fp=#0x1000000646e' 2024-09-21T00:30:11.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34237 fp=#0x1000000645a' 2024-09-21T00:30:11.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34269 fp=#0x1000000648f' 2024-09-21T00:30:11.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34301 fp=#0x100000026d6' 2024-09-21T00:30:11.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34333 fp=#0x10000002570' 2024-09-21T00:30:11.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34365 fp=#0x10000002558' 2024-09-21T00:30:11.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34397 fp=#0x10000002654' 2024-09-21T00:30:11.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34429 fp=#0x10000003648' 2024-09-21T00:30:11.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34461 fp=#0x100000056bf' 2024-09-21T00:30:11.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34493 fp=#0x100000056b7' 2024-09-21T00:30:11.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34525 fp=#0x100000061c6' 2024-09-21T00:30:11.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34557 fp=#0x10000006326' 2024-09-21T00:30:11.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34589 fp=#0x10000006392' 2024-09-21T00:30:11.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34621 fp=#0x100000062d5' 2024-09-21T00:30:11.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34653 fp=#0x1000000639c' 2024-09-21T00:30:11.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34685 fp=#0x10000006390' 2024-09-21T00:30:11.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34749 fp=#0x100000064ca' 2024-09-21T00:30:11.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34781 fp=#0x100000064ec' 2024-09-21T00:30:11.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34813 fp=#0x100000064c0' 2024-09-21T00:30:11.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34845 fp=#0x100000057c0' 2024-09-21T00:30:11.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34877 fp=#0x100000057c4' 2024-09-21T00:30:11.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34909 fp=#0x100000052f3' 2024-09-21T00:30:11.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34973 fp=#0x10000006966' 2024-09-21T00:30:11.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35005 fp=#0x100000052bc' 2024-09-21T00:30:11.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35037 fp=#0x10000005324' 2024-09-21T00:30:11.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35069 fp=#0x1000000607f' 2024-09-21T00:30:11.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35197 fp=#0x1000000609d' 2024-09-21T00:30:11.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35229 fp=#0x100000060e2' 2024-09-21T00:30:11.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35261 fp=#0x10000003fba' 2024-09-21T00:30:11.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35293 fp=#0x10000003f33' 2024-09-21T00:30:11.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35325 fp=#0x10000003f2b' 2024-09-21T00:30:11.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35357 fp=#0x10000005325' 2024-09-21T00:30:11.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35453 fp=#0x10000005f78' 2024-09-21T00:30:11.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35485 fp=#0x10000005fa4' 2024-09-21T00:30:11.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35549 fp=#0x10000005f87' 2024-09-21T00:30:11.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35613 fp=#0x10000005bed' 2024-09-21T00:30:11.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35645 fp=#0x10000005bf1' 2024-09-21T00:30:11.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35677 fp=#0x10000005b62' 2024-09-21T00:30:11.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35709 fp=#0x10000004913' 2024-09-21T00:30:11.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35741 fp=#0x10000004f60' 2024-09-21T00:30:11.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35773 fp=#0x10000004fec' 2024-09-21T00:30:11.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35805 fp=#0x10000004fa8' 2024-09-21T00:30:11.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35837 fp=#0x10000004f7f' 2024-09-21T00:30:11.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35869 fp=#0x10000004f1c' 2024-09-21T00:30:11.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35901 fp=#0x10000006710' 2024-09-21T00:30:11.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35933 fp=#0x1000000671d' 2024-09-21T00:30:11.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35965 fp=#0x10000006888' 2024-09-21T00:30:11.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35997 fp=#0x1000000685b' 2024-09-21T00:30:11.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36029 fp=#0x10000006867' 2024-09-21T00:30:11.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36125 fp=#0x10000005f60' 2024-09-21T00:30:11.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36189 fp=#0x100000068ab' 2024-09-21T00:30:11.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36221 fp=#0x100000068a2' 2024-09-21T00:30:11.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36253 fp=#0x100000068b7' 2024-09-21T00:30:11.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36285 fp=#0x10000006593' 2024-09-21T00:30:11.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36349 fp=#0x100000065f2' 2024-09-21T00:30:11.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36381 fp=#0x100000065f4' 2024-09-21T00:30:11.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36413 fp=#0x1000000400e' 2024-09-21T00:30:11.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36445 fp=#0x10000004027' 2024-09-21T00:30:11.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36477 fp=#0x10000005ffc' 2024-09-21T00:30:11.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36509 fp=#0x1000000605f' 2024-09-21T00:30:11.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36541 fp=#0x1000000603d' 2024-09-21T00:30:11.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36573 fp=#0x10000005a1f' 2024-09-21T00:30:11.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36605 fp=#0x100000059e7' 2024-09-21T00:30:11.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36637 fp=#0x100000059a7' 2024-09-21T00:30:11.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36669 fp=#0x100000047c5' 2024-09-21T00:30:11.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36701 fp=#0x10000006639' 2024-09-21T00:30:11.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36733 fp=#0x10000004851' 2024-09-21T00:30:11.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36765 fp=#0x10000004764' 2024-09-21T00:30:11.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36797 fp=#0x1000000590e' 2024-09-21T00:30:11.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36829 fp=#0x10000005aed' 2024-09-21T00:30:11.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36861 fp=#0x1000000593a' 2024-09-21T00:30:11.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36893 fp=#0x100000058ee' 2024-09-21T00:30:11.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36925 fp=#0x10000005936' 2024-09-21T00:30:11.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36989 fp=#0x10000005280' 2024-09-21T00:30:11.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37021 fp=#0x100000051ff' 2024-09-21T00:30:11.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37053 fp=#0x10000005264' 2024-09-21T00:30:11.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37085 fp=#0x100000051e3' 2024-09-21T00:30:11.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37117 fp=#0x10000005442' 2024-09-21T00:30:11.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37181 fp=#0x10000003d0f' 2024-09-21T00:30:11.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37213 fp=#0x10000003ce7' 2024-09-21T00:30:11.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37245 fp=#0x10000005bc4' 2024-09-21T00:30:11.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37277 fp=#0x10000005ba1' 2024-09-21T00:30:11.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37341 fp=#0x10000005c42' 2024-09-21T00:30:11.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37373 fp=#0x10000005ec6' 2024-09-21T00:30:11.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37437 fp=#0x10000005ef5' 2024-09-21T00:30:11.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37469 fp=#0x10000004b8b' 2024-09-21T00:30:11.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37501 fp=#0x10000004a80' 2024-09-21T00:30:11.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37533 fp=#0x10000004b32' 2024-09-21T00:30:11.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37565 fp=#0x10000004b3b' 2024-09-21T00:30:11.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37597 fp=#0x10000005582' 2024-09-21T00:30:11.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37629 fp=#0x100000054ac' 2024-09-21T00:30:11.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37661 fp=#0x100000054f9' 2024-09-21T00:30:11.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37693 fp=#0x1000000427d' 2024-09-21T00:30:11.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37725 fp=#0x1000000581a' 2024-09-21T00:30:11.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37757 fp=#0x100000057f4' 2024-09-21T00:30:11.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37789 fp=#0x10000004326' 2024-09-21T00:30:11.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37821 fp=#0x1000000430e' 2024-09-21T00:30:11.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37853 fp=#0x10000005822' 2024-09-21T00:30:11.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37885 fp=#0x10000001c16' 2024-09-21T00:30:11.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37917 fp=#0x10000001ca5' 2024-09-21T00:30:11.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37949 fp=#0x100000028a5' 2024-09-21T00:30:11.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37981 fp=#0x10000001c5a' 2024-09-21T00:30:11.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38013 fp=#0x10000001cb3' 2024-09-21T00:30:11.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38077 fp=#0x10000003b42' 2024-09-21T00:30:11.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38205 fp=#0x10000005860' 2024-09-21T00:30:11.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38237 fp=#0x10000005957' 2024-09-21T00:30:11.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38269 fp=#0x10000005851' 2024-09-21T00:30:11.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38301 fp=#0x1000000599e' 2024-09-21T00:30:11.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38333 fp=#0x10000005853' 2024-09-21T00:30:11.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38365 fp=#0x1000000585f' 2024-09-21T00:30:11.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38397 fp=#0x10000005937' 2024-09-21T00:30:11.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38429 fp=#0x10000002ae8' 2024-09-21T00:30:11.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38461 fp=#0x100000045f0' 2024-09-21T00:30:11.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38525 fp=#0x10000002bc3' 2024-09-21T00:30:11.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38557 fp=#0x10000002b91' 2024-09-21T00:30:11.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38589 fp=#0x100000045df' 2024-09-21T00:30:11.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38621 fp=#0x100000045d1' 2024-09-21T00:30:11.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38653 fp=#0x100000048a1' 2024-09-21T00:30:11.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38685 fp=#0x10000001f99' 2024-09-21T00:30:11.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38717 fp=#0x100000012c9' 2024-09-21T00:30:11.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38749 fp=#0x10000001fcf' 2024-09-21T00:30:11.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38877 fp=#0x100000041f0' 2024-09-21T00:30:11.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38909 fp=#0x10000004064' 2024-09-21T00:30:11.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38973 fp=#0x10000004045' 2024-09-21T00:30:11.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39005 fp=#0x1000000407e' 2024-09-21T00:30:11.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39037 fp=#0x10000004082' 2024-09-21T00:30:11.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39069 fp=#0x100000017ec' 2024-09-21T00:30:11.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39101 fp=#0x10000001be4' 2024-09-21T00:30:11.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39133 fp=#0x10000001cbc' 2024-09-21T00:30:11.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39165 fp=#0x100000044a5' 2024-09-21T00:30:11.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39197 fp=#0x1000000450a' 2024-09-21T00:30:11.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39229 fp=#0x100000044bb' 2024-09-21T00:30:11.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39293 fp=#0x100000048ca' 2024-09-21T00:30:11.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39325 fp=#0x10000004899' 2024-09-21T00:30:11.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39357 fp=#0x100000048da' 2024-09-21T00:30:11.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39389 fp=#0x10000005aea' 2024-09-21T00:30:11.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39421 fp=#0x10000005e0e' 2024-09-21T00:30:11.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39453 fp=#0x10000005e37' 2024-09-21T00:30:11.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39485 fp=#0x10000005ada' 2024-09-21T00:30:11.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39517 fp=#0x10000005e3d' 2024-09-21T00:30:11.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39549 fp=#0x10000005ae1' 2024-09-21T00:30:11.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39581 fp=#0x100000050da' 2024-09-21T00:30:11.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39613 fp=#0x100000050fc' 2024-09-21T00:30:11.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39645 fp=#0x100000050e1' 2024-09-21T00:30:11.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39677 fp=#0x10000005d85' 2024-09-21T00:30:11.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39741 fp=#0x10000005b46' 2024-09-21T00:30:11.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39773 fp=#0x10000005b7b' 2024-09-21T00:30:11.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39837 fp=#0x1000000635d' 2024-09-21T00:30:11.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39869 fp=#0x10000006367' 2024-09-21T00:30:11.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39901 fp=#0x10000004e8b' 2024-09-21T00:30:11.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39933 fp=#0x10000004eab' 2024-09-21T00:30:11.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39965 fp=#0x10000004e89' 2024-09-21T00:30:11.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39997 fp=#0x10000004e93' 2024-09-21T00:30:11.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40029 fp=#0x1000000620c' 2024-09-21T00:30:11.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40061 fp=#0x10000006270' 2024-09-21T00:30:11.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40093 fp=#0x10000006290' 2024-09-21T00:30:11.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40125 fp=#0x1000000628a' 2024-09-21T00:30:11.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40157 fp=#0x10000002615' 2024-09-21T00:30:11.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40189 fp=#0x100000026ab' 2024-09-21T00:30:11.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40221 fp=#0x10000000ac5' 2024-09-21T00:30:11.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40253 fp=#0x100000008be' 2024-09-21T00:30:11.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40285 fp=#0x100000042b3' 2024-09-21T00:30:11.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40317 fp=#0x1000000431e' 2024-09-21T00:30:11.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40349 fp=#0x1000000432b' 2024-09-21T00:30:11.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40381 fp=#0x10000000aae' 2024-09-21T00:30:11.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40413 fp=#0x100000008ea' 2024-09-21T00:30:11.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40509 fp=#0x1000000639d' 2024-09-21T00:30:12.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40669 fp=#0x100000047fa' 2024-09-21T00:30:12.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40701 fp=#0x10000005e9f' 2024-09-21T00:30:12.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40765 fp=#0x100000048db' 2024-09-21T00:30:12.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40925 fp=#0x100000063c2' 2024-09-21T00:30:12.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40957 fp=#0x100000063f3' 2024-09-21T00:30:12.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40989 fp=#0x10000004a4b' 2024-09-21T00:30:12.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41021 fp=#0x1000000498d' 2024-09-21T00:30:12.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41053 fp=#0x10000004a46' 2024-09-21T00:30:12.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41085 fp=#0x10000004978' 2024-09-21T00:30:12.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41117 fp=#0x10000004a25' 2024-09-21T00:30:12.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30782 fp=#0x10000005473' 2024-09-21T00:30:12.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30814 fp=#0x1000000219b' 2024-09-21T00:30:12.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30878 fp=#0x10000002830' 2024-09-21T00:30:12.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32702 fp=#0x10000005732' 2024-09-21T00:30:12.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32734 fp=#0x10000005687' 2024-09-21T00:30:12.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32798 fp=#0x1000000503c' 2024-09-21T00:30:12.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32990 fp=#0x10000005d34' 2024-09-21T00:30:12.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33022 fp=#0x10000005cc2' 2024-09-21T00:30:12.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33054 fp=#0x10000005c5a' 2024-09-21T00:30:12.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33086 fp=#0x10000005c88' 2024-09-21T00:30:12.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33118 fp=#0x10000005c31' 2024-09-21T00:30:12.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33182 fp=#0x10000006062' 2024-09-21T00:30:12.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33214 fp=#0x10000006683' 2024-09-21T00:30:12.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33278 fp=#0x10000006677' 2024-09-21T00:30:12.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33310 fp=#0x100000067ec' 2024-09-21T00:30:12.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33342 fp=#0x10000006829' 2024-09-21T00:30:12.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33374 fp=#0x100000067eb' 2024-09-21T00:30:12.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33406 fp=#0x10000003ebf' 2024-09-21T00:30:12.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33438 fp=#0x10000003e4a' 2024-09-21T00:30:12.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33470 fp=#0x10000003dbb' 2024-09-21T00:30:12.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33502 fp=#0x10000003dd3' 2024-09-21T00:30:12.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33534 fp=#0x10000005593' 2024-09-21T00:30:12.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33598 fp=#0x10000005a9a' 2024-09-21T00:30:12.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33630 fp=#0x10000005aae' 2024-09-21T00:30:12.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33726 fp=#0x100000067b5' 2024-09-21T00:30:12.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33758 fp=#0x100000067a4' 2024-09-21T00:30:12.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33790 fp=#0x100000067a3' 2024-09-21T00:30:12.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33918 fp=#0x10000004792' 2024-09-21T00:30:12.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33950 fp=#0x1000000640d' 2024-09-21T00:30:12.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33982 fp=#0x1000000644c' 2024-09-21T00:30:12.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34014 fp=#0x1000000470d' 2024-09-21T00:30:12.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34046 fp=#0x1000000641d' 2024-09-21T00:30:12.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34078 fp=#0x100000031df' 2024-09-21T00:30:12.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34110 fp=#0x1000000324b' 2024-09-21T00:30:12.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34142 fp=#0x1000000623a' 2024-09-21T00:30:12.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34174 fp=#0x1000000646b' 2024-09-21T00:30:12.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34206 fp=#0x1000000649a' 2024-09-21T00:30:12.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34238 fp=#0x10000006274' 2024-09-21T00:30:12.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34270 fp=#0x10000006488' 2024-09-21T00:30:12.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34302 fp=#0x1000000267b' 2024-09-21T00:30:12.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34334 fp=#0x100000061b6' 2024-09-21T00:30:12.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34366 fp=#0x100000056a3' 2024-09-21T00:30:12.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34398 fp=#0x100000056c3' 2024-09-21T00:30:12.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34430 fp=#0x100000061b5' 2024-09-21T00:30:12.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34462 fp=#0x10000006512' 2024-09-21T00:30:12.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34494 fp=#0x100000061b0' 2024-09-21T00:30:12.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34526 fp=#0x100000026b3' 2024-09-21T00:30:12.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34558 fp=#0x10000006320' 2024-09-21T00:30:12.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34590 fp=#0x100000062f1' 2024-09-21T00:30:12.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34622 fp=#0x100000062db' 2024-09-21T00:30:12.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34654 fp=#0x1000000632c' 2024-09-21T00:30:12.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34686 fp=#0x100000062fb' 2024-09-21T00:30:12.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34782 fp=#0x100000057b7' 2024-09-21T00:30:12.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34814 fp=#0x100000064fd' 2024-09-21T00:30:12.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34846 fp=#0x100000057bf' 2024-09-21T00:30:12.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34878 fp=#0x100000064f7' 2024-09-21T00:30:12.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34910 fp=#0x10000005328' 2024-09-21T00:30:12.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34974 fp=#0x100000052b4' 2024-09-21T00:30:12.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35038 fp=#0x10000006961' 2024-09-21T00:30:12.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35070 fp=#0x10000002d26' 2024-09-21T00:30:12.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35134 fp=#0x100000060ed' 2024-09-21T00:30:12.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35166 fp=#0x100000060ec' 2024-09-21T00:30:12.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35198 fp=#0x100000060f3' 2024-09-21T00:30:12.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35262 fp=#0x10000003f55' 2024-09-21T00:30:12.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35326 fp=#0x10000003f94' 2024-09-21T00:30:12.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35390 fp=#0x10000005329' 2024-09-21T00:30:12.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35454 fp=#0x10000005fe6' 2024-09-21T00:30:12.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35614 fp=#0x10000005b6b' 2024-09-21T00:30:12.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35646 fp=#0x10000005b6e' 2024-09-21T00:30:12.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35678 fp=#0x10000005b66' 2024-09-21T00:30:12.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35710 fp=#0x1000000492a' 2024-09-21T00:30:12.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35742 fp=#0x10000004f58' 2024-09-21T00:30:12.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35774 fp=#0x10000004f2c' 2024-09-21T00:30:12.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35806 fp=#0x10000004f6d' 2024-09-21T00:30:12.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35838 fp=#0x10000004fb4' 2024-09-21T00:30:12.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35870 fp=#0x10000004f83' 2024-09-21T00:30:12.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35902 fp=#0x10000006748' 2024-09-21T00:30:12.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35934 fp=#0x1000000673d' 2024-09-21T00:30:12.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35966 fp=#0x10000006834' 2024-09-21T00:30:12.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35998 fp=#0x1000000683c' 2024-09-21T00:30:12.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36030 fp=#0x10000006838' 2024-09-21T00:30:12.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36094 fp=#0x10000005d6b' 2024-09-21T00:30:12.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36158 fp=#0x10000005f5a' 2024-09-21T00:30:12.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36190 fp=#0x10000006892' 2024-09-21T00:30:12.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36222 fp=#0x100000068c0' 2024-09-21T00:30:12.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36254 fp=#0x100000068d1' 2024-09-21T00:30:12.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36286 fp=#0x10000006619' 2024-09-21T00:30:12.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36350 fp=#0x100000065f6' 2024-09-21T00:30:12.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36382 fp=#0x100000065ac' 2024-09-21T00:30:12.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36414 fp=#0x1000000402f' 2024-09-21T00:30:12.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36446 fp=#0x10000006161' 2024-09-21T00:30:12.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36478 fp=#0x10000006033' 2024-09-21T00:30:12.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36542 fp=#0x10000005ffd' 2024-09-21T00:30:12.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36638 fp=#0x10000005a8e' 2024-09-21T00:30:12.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36670 fp=#0x100000047bd' 2024-09-21T00:30:12.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36702 fp=#0x10000004771' 2024-09-21T00:30:12.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36734 fp=#0x100000047f3' 2024-09-21T00:30:12.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36766 fp=#0x1000000480f' 2024-09-21T00:30:12.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36798 fp=#0x10000005afe' 2024-09-21T00:30:12.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36830 fp=#0x1000000589c' 2024-09-21T00:30:12.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36926 fp=#0x100000058fb' 2024-09-21T00:30:12.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36958 fp=#0x1000000593c' 2024-09-21T00:30:12.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37022 fp=#0x1000000522c' 2024-09-21T00:30:12.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37086 fp=#0x10000005250' 2024-09-21T00:30:12.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37182 fp=#0x10000003d2b' 2024-09-21T00:30:12.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37214 fp=#0x10000003d19' 2024-09-21T00:30:12.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37246 fp=#0x10000005c59' 2024-09-21T00:30:12.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37278 fp=#0x10000005bec' 2024-09-21T00:30:12.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37310 fp=#0x10000005c23' 2024-09-21T00:30:12.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37342 fp=#0x10000005bbe' 2024-09-21T00:30:12.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37374 fp=#0x10000005eeb' 2024-09-21T00:30:12.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37406 fp=#0x10000005eea' 2024-09-21T00:30:12.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37438 fp=#0x10000004a7c' 2024-09-21T00:30:12.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37470 fp=#0x10000004ba9' 2024-09-21T00:30:12.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37502 fp=#0x10000004a52' 2024-09-21T00:30:12.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37534 fp=#0x10000004b60' 2024-09-21T00:30:12.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37566 fp=#0x10000004a6d' 2024-09-21T00:30:12.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37598 fp=#0x1000000552d' 2024-09-21T00:30:12.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37630 fp=#0x100000054bc' 2024-09-21T00:30:12.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37662 fp=#0x10000005f27' 2024-09-21T00:30:12.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37694 fp=#0x1000000581f' 2024-09-21T00:30:12.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37726 fp=#0x10000005823' 2024-09-21T00:30:12.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37758 fp=#0x10000005838' 2024-09-21T00:30:12.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37790 fp=#0x10000005807' 2024-09-21T00:30:12.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37822 fp=#0x100000042d8' 2024-09-21T00:30:12.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37854 fp=#0x10000004241' 2024-09-21T00:30:12.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37886 fp=#0x10000001d7c' 2024-09-21T00:30:12.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37918 fp=#0x10000001c68' 2024-09-21T00:30:12.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37950 fp=#0x10000001cb9' 2024-09-21T00:30:12.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37982 fp=#0x10000003682' 2024-09-21T00:30:12.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38014 fp=#0x10000001b5d' 2024-09-21T00:30:12.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38046 fp=#0x10000003c46' 2024-09-21T00:30:12.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38142 fp=#0x10000003a48' 2024-09-21T00:30:12.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38206 fp=#0x1000000585c' 2024-09-21T00:30:12.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38238 fp=#0x1000000591b' 2024-09-21T00:30:12.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38270 fp=#0x1000000617c' 2024-09-21T00:30:12.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38302 fp=#0x100000058e3' 2024-09-21T00:30:12.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38334 fp=#0x1000000586f' 2024-09-21T00:30:12.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38366 fp=#0x1000000594d' 2024-09-21T00:30:12.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38398 fp=#0x100000029e4' 2024-09-21T00:30:12.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38430 fp=#0x100000045e1' 2024-09-21T00:30:12.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38462 fp=#0x100000029ee' 2024-09-21T00:30:12.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38494 fp=#0x10000002a0c' 2024-09-21T00:30:12.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38526 fp=#0x10000002acc' 2024-09-21T00:30:12.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38558 fp=#0x10000002a5f' 2024-09-21T00:30:12.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38590 fp=#0x10000002b62' 2024-09-21T00:30:12.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38622 fp=#0x100000045db' 2024-09-21T00:30:12.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38654 fp=#0x10000004889' 2024-09-21T00:30:12.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38686 fp=#0x10000001f9e' 2024-09-21T00:30:12.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38750 fp=#0x1000000486d' 2024-09-21T00:30:12.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38814 fp=#0x1000000107b' 2024-09-21T00:30:12.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38846 fp=#0x100000041c2' 2024-09-21T00:30:12.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38878 fp=#0x1000000401b' 2024-09-21T00:30:12.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38974 fp=#0x10000004009' 2024-09-21T00:30:12.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39006 fp=#0x100000041b1' 2024-09-21T00:30:12.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39038 fp=#0x1000000413e' 2024-09-21T00:30:12.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39070 fp=#0x10000001d30' 2024-09-21T00:30:12.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39102 fp=#0x10000001bf1' 2024-09-21T00:30:12.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39134 fp=#0x10000001c0f' 2024-09-21T00:30:12.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39166 fp=#0x1000000450e' 2024-09-21T00:30:12.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39198 fp=#0x100000044f8' 2024-09-21T00:30:12.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39262 fp=#0x100000044c8' 2024-09-21T00:30:12.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39294 fp=#0x100000048bd' 2024-09-21T00:30:12.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39326 fp=#0x1000000483b' 2024-09-21T00:30:12.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39358 fp=#0x10000004823' 2024-09-21T00:30:12.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39390 fp=#0x10000005e33' 2024-09-21T00:30:12.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39422 fp=#0x10000005e0f' 2024-09-21T00:30:12.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39454 fp=#0x10000005e2f' 2024-09-21T00:30:12.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39486 fp=#0x10000005a27' 2024-09-21T00:30:12.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39518 fp=#0x100000059cb' 2024-09-21T00:30:12.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39550 fp=#0x10000005e32' 2024-09-21T00:30:12.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39582 fp=#0x100000050ee' 2024-09-21T00:30:12.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39646 fp=#0x100000050ae' 2024-09-21T00:30:12.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39678 fp=#0x10000005deb' 2024-09-21T00:30:12.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39710 fp=#0x10000005d7b' 2024-09-21T00:30:12.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39742 fp=#0x10000005b78' 2024-09-21T00:30:12.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39774 fp=#0x10000005b5c' 2024-09-21T00:30:12.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39806 fp=#0x10000006378' 2024-09-21T00:30:12.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39838 fp=#0x100000062e4' 2024-09-21T00:30:12.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39870 fp=#0x100000062de' 2024-09-21T00:30:12.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39934 fp=#0x10000004eb2' 2024-09-21T00:30:12.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39998 fp=#0x10000004ee9' 2024-09-21T00:30:12.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40030 fp=#0x10000006239' 2024-09-21T00:30:12.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40062 fp=#0x10000006228' 2024-09-21T00:30:12.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40094 fp=#0x10000006277' 2024-09-21T00:30:12.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40126 fp=#0x100000061fe' 2024-09-21T00:30:12.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40158 fp=#0x10000002604' 2024-09-21T00:30:12.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40190 fp=#0x1000000269c' 2024-09-21T00:30:12.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40222 fp=#0x100000008d5' 2024-09-21T00:30:12.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40254 fp=#0x100000042fa' 2024-09-21T00:30:12.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40286 fp=#0x100000008ca' 2024-09-21T00:30:12.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40318 fp=#0x100000008b1' 2024-09-21T00:30:12.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40350 fp=#0x1000000429b' 2024-09-21T00:30:12.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40382 fp=#0x10000000c4e' 2024-09-21T00:30:12.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40414 fp=#0x100000009ce' 2024-09-21T00:30:12.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40446 fp=#0x10000005979' 2024-09-21T00:30:12.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40542 fp=#0x1000000288d' 2024-09-21T00:30:12.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40574 fp=#0x1000000380d' 2024-09-21T00:30:12.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40670 fp=#0x1000000494a' 2024-09-21T00:30:12.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40702 fp=#0x10000004900' 2024-09-21T00:30:12.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40766 fp=#0x10000005e9d' 2024-09-21T00:30:12.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40798 fp=#0x10000004887' 2024-09-21T00:30:12.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40830 fp=#0x100000063cf' 2024-09-21T00:30:12.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40862 fp=#0x100000063d0' 2024-09-21T00:30:12.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40894 fp=#0x100000063eb' 2024-09-21T00:30:12.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40926 fp=#0x100000063c4' 2024-09-21T00:30:12.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40958 fp=#0x100000063e2' 2024-09-21T00:30:12.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40990 fp=#0x1000000513c' 2024-09-21T00:30:12.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41022 fp=#0x10000005102' 2024-09-21T00:30:12.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41054 fp=#0x10000005117' 2024-09-21T00:30:12.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41086 fp=#0x1000000512d' 2024-09-21T00:30:12.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41118 fp=#0x10000005140' 2024-09-21T00:30:12.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30783 fp=#0x10000000880' 2024-09-21T00:30:12.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30815 fp=#0x10000002ccd' 2024-09-21T00:30:12.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30879 fp=#0x1000000658d' 2024-09-21T00:30:12.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32703 fp=#0x10000005678' 2024-09-21T00:30:12.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32735 fp=#0x10000005702' 2024-09-21T00:30:12.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32767 fp=#0x10000005720' 2024-09-21T00:30:12.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32799 fp=#0x10000005038' 2024-09-21T00:30:12.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32831 fp=#0x1000000506c' 2024-09-21T00:30:12.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32927 fp=#0x10000005035' 2024-09-21T00:30:12.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33023 fp=#0x10000005ca7' 2024-09-21T00:30:12.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33087 fp=#0x10000005c56' 2024-09-21T00:30:12.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33119 fp=#0x100000066d6' 2024-09-21T00:30:12.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33151 fp=#0x1000000668b' 2024-09-21T00:30:12.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33183 fp=#0x10000006063' 2024-09-21T00:30:12.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33215 fp=#0x10000006069' 2024-09-21T00:30:12.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33279 fp=#0x1000000666d' 2024-09-21T00:30:12.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33311 fp=#0x100000067e8' 2024-09-21T00:30:12.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33343 fp=#0x100000067d9' 2024-09-21T00:30:12.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33375 fp=#0x100000067cf' 2024-09-21T00:30:12.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33439 fp=#0x10000003e56' 2024-09-21T00:30:12.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33471 fp=#0x10000003db7' 2024-09-21T00:30:12.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33503 fp=#0x10000003ddf' 2024-09-21T00:30:12.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33535 fp=#0x10000005ab0' 2024-09-21T00:30:12.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33567 fp=#0x10000005a49' 2024-09-21T00:30:12.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33599 fp=#0x100000059ee' 2024-09-21T00:30:12.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33695 fp=#0x100000055a7' 2024-09-21T00:30:12.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33727 fp=#0x10000006764' 2024-09-21T00:30:12.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33759 fp=#0x10000006762' 2024-09-21T00:30:12.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33791 fp=#0x10000006784' 2024-09-21T00:30:12.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33919 fp=#0x10000004717' 2024-09-21T00:30:12.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33951 fp=#0x100000047ce' 2024-09-21T00:30:12.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33983 fp=#0x10000006444' 2024-09-21T00:30:12.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34015 fp=#0x1000000476a' 2024-09-21T00:30:12.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34047 fp=#0x10000004715' 2024-09-21T00:30:12.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34079 fp=#0x10000003250' 2024-09-21T00:30:12.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34111 fp=#0x10000003200' 2024-09-21T00:30:12.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34143 fp=#0x1000000620f' 2024-09-21T00:30:12.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34175 fp=#0x10000006210' 2024-09-21T00:30:12.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34207 fp=#0x1000000625d' 2024-09-21T00:30:12.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34239 fp=#0x10000006227' 2024-09-21T00:30:12.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34271 fp=#0x10000006204' 2024-09-21T00:30:12.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34303 fp=#0x100000026d9' 2024-09-21T00:30:12.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34335 fp=#0x100000061bf' 2024-09-21T00:30:12.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34367 fp=#0x10000006192' 2024-09-21T00:30:12.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34431 fp=#0x100000061bc' 2024-09-21T00:30:12.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34463 fp=#0x1000000256c' 2024-09-21T00:30:12.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34495 fp=#0x100000061b7' 2024-09-21T00:30:12.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34527 fp=#0x100000061cc' 2024-09-21T00:30:12.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34559 fp=#0x100000062ed' 2024-09-21T00:30:12.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34591 fp=#0x1000000636d' 2024-09-21T00:30:12.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34623 fp=#0x10000006338' 2024-09-21T00:30:12.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34655 fp=#0x100000062cb' 2024-09-21T00:30:12.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34687 fp=#0x1000000636f' 2024-09-21T00:30:12.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34751 fp=#0x100000057dc' 2024-09-21T00:30:12.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34783 fp=#0x100000057e6' 2024-09-21T00:30:12.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34815 fp=#0x100000064f5' 2024-09-21T00:30:12.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34847 fp=#0x100000057e3' 2024-09-21T00:30:12.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34879 fp=#0x100000064ef' 2024-09-21T00:30:12.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34911 fp=#0x100000052a1' 2024-09-21T00:30:12.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34943 fp=#0x10000005341' 2024-09-21T00:30:12.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35007 fp=#0x10000005312' 2024-09-21T00:30:12.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35071 fp=#0x100000060e4' 2024-09-21T00:30:12.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35103 fp=#0x10000006098' 2024-09-21T00:30:12.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35135 fp=#0x100000060e5' 2024-09-21T00:30:12.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35167 fp=#0x1000000609b' 2024-09-21T00:30:12.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35199 fp=#0x1000000607d' 2024-09-21T00:30:12.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35231 fp=#0x100000060e8' 2024-09-21T00:30:12.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35263 fp=#0x10000003faa' 2024-09-21T00:30:12.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35487 fp=#0x10000005feb' 2024-09-21T00:30:12.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35551 fp=#0x10000005f7d' 2024-09-21T00:30:12.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35583 fp=#0x10000005f72' 2024-09-21T00:30:12.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35615 fp=#0x10000005be9' 2024-09-21T00:30:12.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35647 fp=#0x10000006587' 2024-09-21T00:30:12.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35679 fp=#0x10000005b87' 2024-09-21T00:30:12.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35711 fp=#0x10000004944' 2024-09-21T00:30:12.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35743 fp=#0x10000004f2b' 2024-09-21T00:30:12.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35775 fp=#0x10000004fc8' 2024-09-21T00:30:12.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35807 fp=#0x10000004f15' 2024-09-21T00:30:12.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35839 fp=#0x10000004f66' 2024-09-21T00:30:12.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35871 fp=#0x10000004f8f' 2024-09-21T00:30:12.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35903 fp=#0x1000000674c' 2024-09-21T00:30:12.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35935 fp=#0x10000006738' 2024-09-21T00:30:12.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35967 fp=#0x10000006839' 2024-09-21T00:30:12.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35999 fp=#0x10000006862' 2024-09-21T00:30:12.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36095 fp=#0x10000005f4c' 2024-09-21T00:30:12.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36159 fp=#0x10000005f52' 2024-09-21T00:30:12.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36191 fp=#0x100000068e1' 2024-09-21T00:30:12.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36223 fp=#0x100000068a3' 2024-09-21T00:30:12.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36255 fp=#0x100000068de' 2024-09-21T00:30:12.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36287 fp=#0x100000065a7' 2024-09-21T00:30:12.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36319 fp=#0x10000006594' 2024-09-21T00:30:12.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36383 fp=#0x1000000660b' 2024-09-21T00:30:12.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36415 fp=#0x10000006157' 2024-09-21T00:30:12.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36447 fp=#0x1000000400b' 2024-09-21T00:30:12.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36479 fp=#0x10000006061' 2024-09-21T00:30:12.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36511 fp=#0x10000006064' 2024-09-21T00:30:12.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36607 fp=#0x10000005a96' 2024-09-21T00:30:12.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36671 fp=#0x10000004782' 2024-09-21T00:30:12.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36703 fp=#0x1000000491e' 2024-09-21T00:30:12.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36735 fp=#0x100000047b4' 2024-09-21T00:30:12.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36767 fp=#0x10000004879' 2024-09-21T00:30:12.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36831 fp=#0x10000005897' 2024-09-21T00:30:12.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36863 fp=#0x10000005891' 2024-09-21T00:30:12.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36959 fp=#0x10000005a21' 2024-09-21T00:30:12.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36991 fp=#0x1000000527c' 2024-09-21T00:30:12.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37023 fp=#0x1000000525c' 2024-09-21T00:30:12.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37055 fp=#0x100000051e4' 2024-09-21T00:30:12.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37087 fp=#0x10000005243' 2024-09-21T00:30:12.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37119 fp=#0x10000005454' 2024-09-21T00:30:12.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37183 fp=#0x10000003d24' 2024-09-21T00:30:12.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37215 fp=#0x10000003cfe' 2024-09-21T00:30:12.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37279 fp=#0x10000005bf7' 2024-09-21T00:30:12.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37311 fp=#0x10000005bd0' 2024-09-21T00:30:12.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37375 fp=#0x10000005f0c' 2024-09-21T00:30:12.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37439 fp=#0x10000004bb0' 2024-09-21T00:30:12.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37471 fp=#0x10000004b25' 2024-09-21T00:30:12.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37503 fp=#0x10000004a7f' 2024-09-21T00:30:12.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37535 fp=#0x10000004b16' 2024-09-21T00:30:12.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37567 fp=#0x10000004bab' 2024-09-21T00:30:12.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37599 fp=#0x100000054de' 2024-09-21T00:30:12.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37631 fp=#0x1000000550c' 2024-09-21T00:30:12.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37663 fp=#0x100000054e9' 2024-09-21T00:30:12.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37695 fp=#0x100000042b6' 2024-09-21T00:30:12.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37727 fp=#0x100000042f3' 2024-09-21T00:30:12.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37759 fp=#0x1000000427f' 2024-09-21T00:30:12.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37791 fp=#0x10000005836' 2024-09-21T00:30:12.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37823 fp=#0x10000004311' 2024-09-21T00:30:12.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37855 fp=#0x100000042a2' 2024-09-21T00:30:12.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37887 fp=#0x10000001cf0' 2024-09-21T00:30:12.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37919 fp=#0x10000001cd7' 2024-09-21T00:30:12.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37951 fp=#0x10000001c94' 2024-09-21T00:30:12.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37983 fp=#0x10000005778' 2024-09-21T00:30:12.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38015 fp=#0x10000001d46' 2024-09-21T00:30:12.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38047 fp=#0x10000003c5b' 2024-09-21T00:30:12.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38079 fp=#0x10000003b46' 2024-09-21T00:30:12.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38143 fp=#0x10000003a45' 2024-09-21T00:30:12.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38207 fp=#0x10000005844' 2024-09-21T00:30:12.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38239 fp=#0x10000005882' 2024-09-21T00:30:12.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38271 fp=#0x10000006190' 2024-09-21T00:30:12.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38303 fp=#0x10000005884' 2024-09-21T00:30:12.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38335 fp=#0x10000005995' 2024-09-21T00:30:12.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38367 fp=#0x10000005849' 2024-09-21T00:30:12.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38399 fp=#0x10000002bbc' 2024-09-21T00:30:12.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38431 fp=#0x10000002bc6' 2024-09-21T00:30:12.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38463 fp=#0x10000002b89' 2024-09-21T00:30:12.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38495 fp=#0x10000002a27' 2024-09-21T00:30:12.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38527 fp=#0x10000002a71' 2024-09-21T00:30:12.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38559 fp=#0x100000045c6' 2024-09-21T00:30:12.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38591 fp=#0x10000002b4d' 2024-09-21T00:30:12.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38623 fp=#0x100000045c0' 2024-09-21T00:30:12.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38655 fp=#0x10000001fc6' 2024-09-21T00:30:12.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38719 fp=#0x10000001073' 2024-09-21T00:30:12.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38751 fp=#0x10000001f66' 2024-09-21T00:30:12.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38783 fp=#0x10000001f88' 2024-09-21T00:30:12.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38847 fp=#0x10000004000' 2024-09-21T00:30:12.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38911 fp=#0x1000000407a' 2024-09-21T00:30:12.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38943 fp=#0x1000000406c' 2024-09-21T00:30:12.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38975 fp=#0x1000000419f' 2024-09-21T00:30:12.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39039 fp=#0x10000004020' 2024-09-21T00:30:12.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39071 fp=#0x10000001d37' 2024-09-21T00:30:12.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39103 fp=#0x10000001fdc' 2024-09-21T00:30:12.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39135 fp=#0x10000002008' 2024-09-21T00:30:12.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39167 fp=#0x100000044ef' 2024-09-21T00:30:12.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39199 fp=#0x1000000449b' 2024-09-21T00:30:12.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39263 fp=#0x100000044f0' 2024-09-21T00:30:12.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39295 fp=#0x100000048e0' 2024-09-21T00:30:12.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39327 fp=#0x1000000484b' 2024-09-21T00:30:12.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39359 fp=#0x100000048ba' 2024-09-21T00:30:12.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39391 fp=#0x10000005e2a' 2024-09-21T00:30:12.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39423 fp=#0x10000005acd' 2024-09-21T00:30:12.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39455 fp=#0x10000005e12' 2024-09-21T00:30:12.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39487 fp=#0x10000005aad' 2024-09-21T00:30:12.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39519 fp=#0x10000005ad1' 2024-09-21T00:30:12.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39551 fp=#0x10000005e0c' 2024-09-21T00:30:12.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39583 fp=#0x100000050ba' 2024-09-21T00:30:12.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39615 fp=#0x100000050f4' 2024-09-21T00:30:12.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39647 fp=#0x100000050db' 2024-09-21T00:30:12.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39743 fp=#0x10000005b7f' 2024-09-21T00:30:12.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39775 fp=#0x10000005b3f' 2024-09-21T00:30:12.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39807 fp=#0x1000000636e' 2024-09-21T00:30:12.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39839 fp=#0x100000062fc' 2024-09-21T00:30:12.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39871 fp=#0x100000062d4' 2024-09-21T00:30:12.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39903 fp=#0x10000004efd' 2024-09-21T00:30:12.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39935 fp=#0x10000004eb6' 2024-09-21T00:30:12.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39967 fp=#0x10000004e86' 2024-09-21T00:30:12.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39999 fp=#0x10000004e91' 2024-09-21T00:30:12.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40031 fp=#0x10000006233' 2024-09-21T00:30:12.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40063 fp=#0x1000000627f' 2024-09-21T00:30:12.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40095 fp=#0x10000006268' 2024-09-21T00:30:12.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40127 fp=#0x1000000629b' 2024-09-21T00:30:12.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40159 fp=#0x10000002670' 2024-09-21T00:30:12.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40191 fp=#0x10000002634' 2024-09-21T00:30:12.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40223 fp=#0x10000004330' 2024-09-21T00:30:12.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40255 fp=#0x1000000430b' 2024-09-21T00:30:12.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40287 fp=#0x100000008a9' 2024-09-21T00:30:12.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40319 fp=#0x1000000432a' 2024-09-21T00:30:12.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40351 fp=#0x10000000aa1' 2024-09-21T00:30:12.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40383 fp=#0x100000042ab' 2024-09-21T00:30:12.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40415 fp=#0x1000000090a' 2024-09-21T00:30:12.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40511 fp=#0x10000002831' 2024-09-21T00:30:12.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40543 fp=#0x1000000289d' 2024-09-21T00:30:12.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40671 fp=#0x10000004825' 2024-09-21T00:30:12.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40735 fp=#0x10000005ea0' 2024-09-21T00:30:12.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40767 fp=#0x1000000487d' 2024-09-21T00:30:12.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40799 fp=#0x10000005eaf' 2024-09-21T00:30:12.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40831 fp=#0x100000031b7' 2024-09-21T00:30:12.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40863 fp=#0x100000063f2' 2024-09-21T00:30:12.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40895 fp=#0x100000031d2' 2024-09-21T00:30:12.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40927 fp=#0x10000003058' 2024-09-21T00:30:12.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40991 fp=#0x100000049d4' 2024-09-21T00:30:12.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41023 fp=#0x10000005128' 2024-09-21T00:30:12.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41055 fp=#0x10000005146' 2024-09-21T00:30:12.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41087 fp=#0x10000004a16' 2024-09-21T00:30:12.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41119 fp=#0x1000000511b' 2024-09-21T00:30:12.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30784 fp=#0x100000018ef' 2024-09-21T00:30:12.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30816 fp=#0x10000002c11' 2024-09-21T00:30:12.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30880 fp=#0x100000009fd' 2024-09-21T00:30:12.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32704 fp=#0x10000005754' 2024-09-21T00:30:12.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32768 fp=#0x1000000578d' 2024-09-21T00:30:12.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32800 fp=#0x1000000508c' 2024-09-21T00:30:12.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32960 fp=#0x10000005040' 2024-09-21T00:30:12.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32992 fp=#0x10000005c36' 2024-09-21T00:30:12.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33024 fp=#0x10000005cc7' 2024-09-21T00:30:12.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33056 fp=#0x10000005ca6' 2024-09-21T00:30:12.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33088 fp=#0x10000005c7e' 2024-09-21T00:30:12.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33120 fp=#0x10000005c74' 2024-09-21T00:30:12.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33152 fp=#0x10000006679' 2024-09-21T00:30:12.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33216 fp=#0x10000006680' 2024-09-21T00:30:12.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33248 fp=#0x10000006052' 2024-09-21T00:30:12.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33312 fp=#0x100000067ca' 2024-09-21T00:30:12.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33344 fp=#0x100000067d8' 2024-09-21T00:30:12.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33376 fp=#0x100000067f0' 2024-09-21T00:30:12.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33408 fp=#0x10000003eb8' 2024-09-21T00:30:12.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33472 fp=#0x10000003d5f' 2024-09-21T00:30:12.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33504 fp=#0x10000003e36' 2024-09-21T00:30:12.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33536 fp=#0x1000000564f' 2024-09-21T00:30:12.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33632 fp=#0x1000000565f' 2024-09-21T00:30:12.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33664 fp=#0x100000055ac' 2024-09-21T00:30:12.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33728 fp=#0x10000006760' 2024-09-21T00:30:12.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33760 fp=#0x10000006771' 2024-09-21T00:30:12.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33792 fp=#0x1000000679b' 2024-09-21T00:30:12.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33920 fp=#0x1000000470f' 2024-09-21T00:30:12.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33984 fp=#0x10000006428' 2024-09-21T00:30:12.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34016 fp=#0x10000006418' 2024-09-21T00:30:12.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34080 fp=#0x10000003265' 2024-09-21T00:30:12.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34112 fp=#0x10000003257' 2024-09-21T00:30:12.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34144 fp=#0x100000062af' 2024-09-21T00:30:12.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34176 fp=#0x10000006202' 2024-09-21T00:30:12.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34208 fp=#0x100000064a5' 2024-09-21T00:30:12.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34240 fp=#0x10000006262' 2024-09-21T00:30:12.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34272 fp=#0x1000000624f' 2024-09-21T00:30:12.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34336 fp=#0x1000000268f' 2024-09-21T00:30:12.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34368 fp=#0x10000002578' 2024-09-21T00:30:12.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34400 fp=#0x100000061a4' 2024-09-21T00:30:12.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34432 fp=#0x1000000619e' 2024-09-21T00:30:12.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34464 fp=#0x100000026b6' 2024-09-21T00:30:12.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34496 fp=#0x100000061b1' 2024-09-21T00:30:12.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34560 fp=#0x100000063ab' 2024-09-21T00:30:12.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34592 fp=#0x100000062df' 2024-09-21T00:30:12.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34624 fp=#0x10000006300' 2024-09-21T00:30:12.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34656 fp=#0x100000062bb' 2024-09-21T00:30:12.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34752 fp=#0x100000064ff' 2024-09-21T00:30:12.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34784 fp=#0x100000064d3' 2024-09-21T00:30:12.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34816 fp=#0x100000057b4' 2024-09-21T00:30:12.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34848 fp=#0x100000064c7' 2024-09-21T00:30:12.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34880 fp=#0x100000064ea' 2024-09-21T00:30:12.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34912 fp=#0x100000052ca' 2024-09-21T00:30:12.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34944 fp=#0x100000052dc' 2024-09-21T00:30:12.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34976 fp=#0x10000006963' 2024-09-21T00:30:12.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35008 fp=#0x100000052ea' 2024-09-21T00:30:12.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35104 fp=#0x100000060a6' 2024-09-21T00:30:12.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35136 fp=#0x100000060c9' 2024-09-21T00:30:12.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35264 fp=#0x10000003f92' 2024-09-21T00:30:12.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35328 fp=#0x1000000538f' 2024-09-21T00:30:12.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35488 fp=#0x10000005fb3' 2024-09-21T00:30:12.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35552 fp=#0x10000005fd4' 2024-09-21T00:30:12.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35584 fp=#0x10000005fb0' 2024-09-21T00:30:12.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35616 fp=#0x10000005be7' 2024-09-21T00:30:12.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35648 fp=#0x10000005be6' 2024-09-21T00:30:12.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35680 fp=#0x10000005b9e' 2024-09-21T00:30:12.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35712 fp=#0x1000000494d' 2024-09-21T00:30:12.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35744 fp=#0x1000000501c' 2024-09-21T00:30:12.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35776 fp=#0x10000004f1e' 2024-09-21T00:30:12.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35808 fp=#0x10000004f1a' 2024-09-21T00:30:12.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35840 fp=#0x10000004f25' 2024-09-21T00:30:12.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35872 fp=#0x10000004fea' 2024-09-21T00:30:12.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35904 fp=#0x10000006709' 2024-09-21T00:30:12.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35936 fp=#0x1000000671a' 2024-09-21T00:30:12.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35968 fp=#0x10000006860' 2024-09-21T00:30:12.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36000 fp=#0x10000006847' 2024-09-21T00:30:12.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36064 fp=#0x10000005cca' 2024-09-21T00:30:12.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36160 fp=#0x10000005f4d' 2024-09-21T00:30:12.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36192 fp=#0x100000068bd' 2024-09-21T00:30:12.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36224 fp=#0x100000068e5' 2024-09-21T00:30:12.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36256 fp=#0x10000006895' 2024-09-21T00:30:12.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36288 fp=#0x10000006603' 2024-09-21T00:30:12.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36320 fp=#0x1000000659e' 2024-09-21T00:30:12.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36352 fp=#0x10000006591' 2024-09-21T00:30:12.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36384 fp=#0x10000006595' 2024-09-21T00:30:12.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36416 fp=#0x10000003fef' 2024-09-21T00:30:12.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36448 fp=#0x10000003ff9' 2024-09-21T00:30:12.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36512 fp=#0x10000006049' 2024-09-21T00:30:12.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36576 fp=#0x10000005a91' 2024-09-21T00:30:12.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36608 fp=#0x10000005a04' 2024-09-21T00:30:12.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36640 fp=#0x100000059a9' 2024-09-21T00:30:12.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36672 fp=#0x10000004922' 2024-09-21T00:30:12.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36704 fp=#0x100000048d4' 2024-09-21T00:30:12.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36736 fp=#0x100000048ef' 2024-09-21T00:30:12.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36768 fp=#0x10000004843' 2024-09-21T00:30:12.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36800 fp=#0x10000005956' 2024-09-21T00:30:12.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36864 fp=#0x10000005ae7' 2024-09-21T00:30:12.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36992 fp=#0x10000005297' 2024-09-21T00:30:12.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37024 fp=#0x10000005197' 2024-09-21T00:30:12.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37056 fp=#0x10000005240' 2024-09-21T00:30:12.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37088 fp=#0x100000051de' 2024-09-21T00:30:12.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37120 fp=#0x10000005448' 2024-09-21T00:30:12.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37184 fp=#0x10000003d20' 2024-09-21T00:30:12.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37216 fp=#0x10000003cb5' 2024-09-21T00:30:12.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37248 fp=#0x10000005c57' 2024-09-21T00:30:12.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37280 fp=#0x10000005c15' 2024-09-21T00:30:12.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37312 fp=#0x10000005ba3' 2024-09-21T00:30:12.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37344 fp=#0x10000005bb5' 2024-09-21T00:30:12.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37376 fp=#0x10000005f0e' 2024-09-21T00:30:12.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37408 fp=#0x10000005f0d' 2024-09-21T00:30:12.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37440 fp=#0x10000004ba7' 2024-09-21T00:30:12.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37472 fp=#0x10000004b9d' 2024-09-21T00:30:12.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37504 fp=#0x10000004a88' 2024-09-21T00:30:12.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37536 fp=#0x10000004bb2' 2024-09-21T00:30:12.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37568 fp=#0x10000004a57' 2024-09-21T00:30:12.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37600 fp=#0x10000005559' 2024-09-21T00:30:12.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37632 fp=#0x100000054c0' 2024-09-21T00:30:12.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37664 fp=#0x1000000547e' 2024-09-21T00:30:12.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37696 fp=#0x10000004323' 2024-09-21T00:30:12.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37728 fp=#0x1000000421c' 2024-09-21T00:30:12.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37760 fp=#0x100000042e0' 2024-09-21T00:30:12.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37792 fp=#0x10000004312' 2024-09-21T00:30:12.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37824 fp=#0x1000000580a' 2024-09-21T00:30:12.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37856 fp=#0x100000042f5' 2024-09-21T00:30:12.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37888 fp=#0x10000001d83' 2024-09-21T00:30:12.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37920 fp=#0x10000001ce2' 2024-09-21T00:30:12.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37952 fp=#0x10000001cca' 2024-09-21T00:30:12.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37984 fp=#0x10000001ba9' 2024-09-21T00:30:12.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38016 fp=#0x1000000213d' 2024-09-21T00:30:12.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38048 fp=#0x10000003c67' 2024-09-21T00:30:12.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38176 fp=#0x10000003ab3' 2024-09-21T00:30:12.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38208 fp=#0x1000000590c' 2024-09-21T00:30:12.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38240 fp=#0x10000005944' 2024-09-21T00:30:12.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38304 fp=#0x10000005867' 2024-09-21T00:30:12.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38336 fp=#0x1000000591e' 2024-09-21T00:30:12.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38368 fp=#0x10000006185' 2024-09-21T00:30:12.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38400 fp=#0x100000045ca' 2024-09-21T00:30:12.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38432 fp=#0x10000002bd7' 2024-09-21T00:30:12.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38464 fp=#0x10000002a68' 2024-09-21T00:30:12.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38528 fp=#0x10000002a24' 2024-09-21T00:30:12.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38560 fp=#0x10000002a50' 2024-09-21T00:30:12.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38592 fp=#0x10000002a47' 2024-09-21T00:30:12.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38624 fp=#0x10000002a3f' 2024-09-21T00:30:12.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38656 fp=#0x100000010ca' 2024-09-21T00:30:12.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38688 fp=#0x100000010ce' 2024-09-21T00:30:12.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38784 fp=#0x10000000ae0' 2024-09-21T00:30:12.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38816 fp=#0x1000000489c' 2024-09-21T00:30:12.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38848 fp=#0x10000004080' 2024-09-21T00:30:12.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38880 fp=#0x10000004023' 2024-09-21T00:30:12.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38912 fp=#0x100000062f2' 2024-09-21T00:30:12.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38976 fp=#0x1000000408c' 2024-09-21T00:30:12.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39072 fp=#0x10000001c67' 2024-09-21T00:30:12.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39104 fp=#0x1000000186a' 2024-09-21T00:30:12.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39136 fp=#0x100000017de' 2024-09-21T00:30:12.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39168 fp=#0x10000004510' 2024-09-21T00:30:12.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39200 fp=#0x100000044b0' 2024-09-21T00:30:12.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39296 fp=#0x1000000489e' 2024-09-21T00:30:12.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39328 fp=#0x100000048b6' 2024-09-21T00:30:12.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39360 fp=#0x10000004832' 2024-09-21T00:30:12.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39424 fp=#0x10000005aa4' 2024-09-21T00:30:12.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39456 fp=#0x10000005e2d' 2024-09-21T00:30:12.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39488 fp=#0x10000005ab6' 2024-09-21T00:30:12.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39520 fp=#0x10000005a24' 2024-09-21T00:30:12.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39552 fp=#0x10000005e13' 2024-09-21T00:30:12.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39584 fp=#0x100000050e7' 2024-09-21T00:30:12.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39616 fp=#0x100000050b8' 2024-09-21T00:30:12.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39744 fp=#0x10000005b4d' 2024-09-21T00:30:12.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39776 fp=#0x10000005b44' 2024-09-21T00:30:12.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39808 fp=#0x10000006303' 2024-09-21T00:30:12.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39840 fp=#0x100000062e8' 2024-09-21T00:30:12.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39872 fp=#0x10000006353' 2024-09-21T00:30:12.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39904 fp=#0x10000004e99' 2024-09-21T00:30:12.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39968 fp=#0x10000004eee' 2024-09-21T00:30:12.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40000 fp=#0x10000004e94' 2024-09-21T00:30:12.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40032 fp=#0x100000061fa' 2024-09-21T00:30:12.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40064 fp=#0x10000006288' 2024-09-21T00:30:12.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40096 fp=#0x1000000621e' 2024-09-21T00:30:12.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40128 fp=#0x100000061ea' 2024-09-21T00:30:12.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40160 fp=#0x1000000268d' 2024-09-21T00:30:12.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40192 fp=#0x1000000267e' 2024-09-21T00:30:12.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40224 fp=#0x10000000abd' 2024-09-21T00:30:12.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40256 fp=#0x10000000a80' 2024-09-21T00:30:12.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40288 fp=#0x100000042a9' 2024-09-21T00:30:12.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40320 fp=#0x1000000085e' 2024-09-21T00:30:12.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40352 fp=#0x100000008e4' 2024-09-21T00:30:12.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40384 fp=#0x100000042c5' 2024-09-21T00:30:12.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40416 fp=#0x10000004302' 2024-09-21T00:30:12.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40672 fp=#0x100000053ac' 2024-09-21T00:30:12.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40704 fp=#0x10000004947' 2024-09-21T00:30:12.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40736 fp=#0x100000048d7' 2024-09-21T00:30:12.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40768 fp=#0x10000004836' 2024-09-21T00:30:12.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40800 fp=#0x100000048f8' 2024-09-21T00:30:12.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40832 fp=#0x100000031d6' 2024-09-21T00:30:12.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40864 fp=#0x10000003050' 2024-09-21T00:30:12.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40896 fp=#0x100000063f0' 2024-09-21T00:30:12.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40960 fp=#0x100000063e9' 2024-09-21T00:30:12.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40992 fp=#0x10000004a3e' 2024-09-21T00:30:12.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41024 fp=#0x100000049c9' 2024-09-21T00:30:12.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41056 fp=#0x10000004a35' 2024-09-21T00:30:12.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41088 fp=#0x1000000497e' 2024-09-21T00:30:12.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41120 fp=#0x10000005122' 2024-09-21T00:30:12.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30785 fp=#0x1000000421a' 2024-09-21T00:30:12.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30817 fp=#0x10000004492' 2024-09-21T00:30:12.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30881 fp=#0x10000003fe9' 2024-09-21T00:30:12.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32737 fp=#0x10000005796' 2024-09-21T00:30:12.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32801 fp=#0x10000005046' 2024-09-21T00:30:12.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32865 fp=#0x1000000503a' 2024-09-21T00:30:12.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32897 fp=#0x10000005048' 2024-09-21T00:30:12.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32929 fp=#0x10000005061' 2024-09-21T00:30:12.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32961 fp=#0x10000005f35' 2024-09-21T00:30:12.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32993 fp=#0x10000005dd0' 2024-09-21T00:30:12.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33025 fp=#0x10000005c53' 2024-09-21T00:30:12.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33057 fp=#0x10000005cb8' 2024-09-21T00:30:12.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33089 fp=#0x10000005cfc' 2024-09-21T00:30:12.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33121 fp=#0x10000005c8c' 2024-09-21T00:30:12.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33185 fp=#0x1000000603f' 2024-09-21T00:30:12.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33217 fp=#0x1000000668f' 2024-09-21T00:30:12.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33249 fp=#0x10000006035' 2024-09-21T00:30:12.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33313 fp=#0x10000006827' 2024-09-21T00:30:12.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33345 fp=#0x100000067de' 2024-09-21T00:30:12.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33377 fp=#0x100000067ee' 2024-09-21T00:30:12.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33409 fp=#0x10000003ef9' 2024-09-21T00:30:12.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33441 fp=#0x10000003e64' 2024-09-21T00:30:12.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33473 fp=#0x10000003e7a' 2024-09-21T00:30:12.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33505 fp=#0x10000003e59' 2024-09-21T00:30:12.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33537 fp=#0x10000005a71' 2024-09-21T00:30:12.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33569 fp=#0x10000005652' 2024-09-21T00:30:12.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33665 fp=#0x1000000565d' 2024-09-21T00:30:12.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33697 fp=#0x100000055c6' 2024-09-21T00:30:12.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33729 fp=#0x100000067bc' 2024-09-21T00:30:12.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33761 fp=#0x100000067be' 2024-09-21T00:30:12.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33793 fp=#0x10000006780' 2024-09-21T00:30:12.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33921 fp=#0x100000047d7' 2024-09-21T00:30:12.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33953 fp=#0x1000000472d' 2024-09-21T00:30:12.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33985 fp=#0x10000006442' 2024-09-21T00:30:12.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34017 fp=#0x10000004756' 2024-09-21T00:30:12.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34049 fp=#0x10000006446' 2024-09-21T00:30:12.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34113 fp=#0x10000003213' 2024-09-21T00:30:12.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34145 fp=#0x10000006259' 2024-09-21T00:30:12.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34177 fp=#0x100000062ad' 2024-09-21T00:30:12.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34209 fp=#0x1000000622b' 2024-09-21T00:30:12.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34241 fp=#0x1000000629d' 2024-09-21T00:30:12.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34273 fp=#0x1000000658b' 2024-09-21T00:30:12.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34305 fp=#0x100000036b5' 2024-09-21T00:30:12.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34337 fp=#0x100000061af' 2024-09-21T00:30:12.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34369 fp=#0x10000005b81' 2024-09-21T00:30:12.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34401 fp=#0x10000006195' 2024-09-21T00:30:12.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34465 fp=#0x10000002645' 2024-09-21T00:30:12.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34497 fp=#0x1000000569f' 2024-09-21T00:30:12.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34529 fp=#0x100000061a7' 2024-09-21T00:30:12.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34561 fp=#0x10000006334' 2024-09-21T00:30:12.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34593 fp=#0x100000063aa' 2024-09-21T00:30:12.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34625 fp=#0x1000000637f' 2024-09-21T00:30:12.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34657 fp=#0x10000006327' 2024-09-21T00:30:12.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34753 fp=#0x100000057bb' 2024-09-21T00:30:12.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34785 fp=#0x100000064d4' 2024-09-21T00:30:12.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34817 fp=#0x100000057b9' 2024-09-21T00:30:12.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34849 fp=#0x100000057de' 2024-09-21T00:30:12.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34881 fp=#0x100000057c9' 2024-09-21T00:30:12.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34913 fp=#0x100000052c2' 2024-09-21T00:30:12.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34945 fp=#0x100000052f2' 2024-09-21T00:30:12.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34977 fp=#0x10000005355' 2024-09-21T00:30:12.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35009 fp=#0x10000005b1b' 2024-09-21T00:30:12.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35041 fp=#0x100000052ee' 2024-09-21T00:30:12.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35105 fp=#0x1000000608d' 2024-09-21T00:30:12.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35137 fp=#0x100000060e3' 2024-09-21T00:30:12.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35201 fp=#0x1000000608f' 2024-09-21T00:30:12.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35233 fp=#0x100000060bd' 2024-09-21T00:30:12.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35265 fp=#0x10000003f48' 2024-09-21T00:30:12.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35297 fp=#0x10000003fae' 2024-09-21T00:30:12.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35489 fp=#0x10000005f77' 2024-09-21T00:30:12.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35521 fp=#0x10000005f7b' 2024-09-21T00:30:12.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35553 fp=#0x10000005f8d' 2024-09-21T00:30:12.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35585 fp=#0x10000005f70' 2024-09-21T00:30:12.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35617 fp=#0x10000005bf5' 2024-09-21T00:30:12.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35649 fp=#0x10000005b94' 2024-09-21T00:30:12.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35681 fp=#0x10000005ba7' 2024-09-21T00:30:12.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35713 fp=#0x1000000493d' 2024-09-21T00:30:12.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35745 fp=#0x10000004f73' 2024-09-21T00:30:12.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35777 fp=#0x1000000501a' 2024-09-21T00:30:12.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35809 fp=#0x10000004f77' 2024-09-21T00:30:12.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35841 fp=#0x10000005000' 2024-09-21T00:30:12.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35873 fp=#0x10000004f35' 2024-09-21T00:30:12.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35905 fp=#0x10000006718' 2024-09-21T00:30:12.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35937 fp=#0x10000006721' 2024-09-21T00:30:12.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35969 fp=#0x10000006832' 2024-09-21T00:30:12.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36001 fp=#0x1000000686a' 2024-09-21T00:30:12.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36033 fp=#0x10000005f63' 2024-09-21T00:30:12.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36193 fp=#0x100000068df' 2024-09-21T00:30:12.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36225 fp=#0x100000068da' 2024-09-21T00:30:12.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36257 fp=#0x100000068c5' 2024-09-21T00:30:12.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36385 fp=#0x100000065f9' 2024-09-21T00:30:12.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36449 fp=#0x10000006143' 2024-09-21T00:30:12.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36481 fp=#0x10000006065' 2024-09-21T00:30:12.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36513 fp=#0x1000000600d' 2024-09-21T00:30:12.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36545 fp=#0x10000005abf' 2024-09-21T00:30:12.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36577 fp=#0x10000005a1c' 2024-09-21T00:30:12.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36641 fp=#0x10000005a0c' 2024-09-21T00:30:12.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36673 fp=#0x100000047de' 2024-09-21T00:30:12.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36705 fp=#0x100000047dc' 2024-09-21T00:30:12.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36737 fp=#0x10000004811' 2024-09-21T00:30:12.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36769 fp=#0x10000004804' 2024-09-21T00:30:12.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36801 fp=#0x100000058f3' 2024-09-21T00:30:12.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36833 fp=#0x10000005a19' 2024-09-21T00:30:12.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36897 fp=#0x10000005906' 2024-09-21T00:30:12.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36993 fp=#0x100000051ef' 2024-09-21T00:30:12.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37025 fp=#0x100000051ae' 2024-09-21T00:30:12.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37057 fp=#0x10000005248' 2024-09-21T00:30:12.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37089 fp=#0x1000000526a' 2024-09-21T00:30:12.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37185 fp=#0x10000003d3f' 2024-09-21T00:30:12.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37217 fp=#0x10000005fec' 2024-09-21T00:30:12.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37249 fp=#0x10000005c6e' 2024-09-21T00:30:12.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37281 fp=#0x10000005c2e' 2024-09-21T00:30:12.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37313 fp=#0x10000005c3f' 2024-09-21T00:30:12.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37345 fp=#0x10000005c2d' 2024-09-21T00:30:12.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37377 fp=#0x10000005ef2' 2024-09-21T00:30:12.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37409 fp=#0x10000005f01' 2024-09-21T00:30:12.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37441 fp=#0x10000004a5d' 2024-09-21T00:30:12.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37473 fp=#0x10000004b90' 2024-09-21T00:30:12.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37505 fp=#0x10000004ad5' 2024-09-21T00:30:12.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37537 fp=#0x10000004b74' 2024-09-21T00:30:12.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37569 fp=#0x10000004aec' 2024-09-21T00:30:12.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37601 fp=#0x1000000549c' 2024-09-21T00:30:12.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37633 fp=#0x10000005f26' 2024-09-21T00:30:12.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37665 fp=#0x10000005523' 2024-09-21T00:30:12.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37697 fp=#0x10000004329' 2024-09-21T00:30:12.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37729 fp=#0x10000005813' 2024-09-21T00:30:12.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37761 fp=#0x100000042e8' 2024-09-21T00:30:12.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37793 fp=#0x100000042e1' 2024-09-21T00:30:12.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37825 fp=#0x1000000565b' 2024-09-21T00:30:12.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37857 fp=#0x10000004249' 2024-09-21T00:30:12.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37889 fp=#0x10000001b4c' 2024-09-21T00:30:12.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37921 fp=#0x10000001b8c' 2024-09-21T00:30:12.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37953 fp=#0x10000001d7f' 2024-09-21T00:30:12.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37985 fp=#0x10000001c76' 2024-09-21T00:30:12.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38017 fp=#0x10000001c3e' 2024-09-21T00:30:12.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38049 fp=#0x10000003ca9' 2024-09-21T00:30:12.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38209 fp=#0x100000059a6' 2024-09-21T00:30:12.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38241 fp=#0x100000058eb' 2024-09-21T00:30:12.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38305 fp=#0x1000000616f' 2024-09-21T00:30:12.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38337 fp=#0x10000005939' 2024-09-21T00:30:12.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38369 fp=#0x1000000617d' 2024-09-21T00:30:12.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38433 fp=#0x100000045f9' 2024-09-21T00:30:12.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38465 fp=#0x100000045d3' 2024-09-21T00:30:12.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38497 fp=#0x10000002bce' 2024-09-21T00:30:12.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38529 fp=#0x100000045e3' 2024-09-21T00:30:12.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38561 fp=#0x100000029f0' 2024-09-21T00:30:12.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38593 fp=#0x10000002b97' 2024-09-21T00:30:12.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38625 fp=#0x10000002a4d' 2024-09-21T00:30:12.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38721 fp=#0x10000004870' 2024-09-21T00:30:12.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38785 fp=#0x10000001f94' 2024-09-21T00:30:12.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38817 fp=#0x10000004886' 2024-09-21T00:30:12.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38945 fp=#0x1000000403b' 2024-09-21T00:30:12.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39041 fp=#0x10000001859' 2024-09-21T00:30:12.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39073 fp=#0x10000001863' 2024-09-21T00:30:12.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39105 fp=#0x10000001d00' 2024-09-21T00:30:12.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39137 fp=#0x10000001c07' 2024-09-21T00:30:12.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39169 fp=#0x100000044b3' 2024-09-21T00:30:12.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39201 fp=#0x1000000449f' 2024-09-21T00:30:12.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39297 fp=#0x100000048cf' 2024-09-21T00:30:12.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39329 fp=#0x10000004867' 2024-09-21T00:30:12.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39361 fp=#0x100000048cb' 2024-09-21T00:30:12.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39393 fp=#0x10000005e18' 2024-09-21T00:30:12.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39425 fp=#0x10000005aec' 2024-09-21T00:30:12.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39457 fp=#0x100000059f0' 2024-09-21T00:30:12.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39489 fp=#0x10000005dff' 2024-09-21T00:30:12.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39521 fp=#0x10000005e15' 2024-09-21T00:30:12.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39553 fp=#0x10000005dfb' 2024-09-21T00:30:12.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39585 fp=#0x100000050de' 2024-09-21T00:30:12.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39617 fp=#0x1000000509b' 2024-09-21T00:30:12.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39649 fp=#0x100000050aa' 2024-09-21T00:30:12.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39809 fp=#0x1000000636b' 2024-09-21T00:30:12.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39841 fp=#0x1000000630d' 2024-09-21T00:30:12.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39873 fp=#0x10000006309' 2024-09-21T00:30:12.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39905 fp=#0x10000004efb' 2024-09-21T00:30:12.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39937 fp=#0x10000004ee7' 2024-09-21T00:30:12.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39969 fp=#0x10000004ead' 2024-09-21T00:30:12.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40001 fp=#0x10000004eac' 2024-09-21T00:30:12.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40033 fp=#0x100000062ae' 2024-09-21T00:30:12.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40065 fp=#0x100000061dd' 2024-09-21T00:30:12.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40097 fp=#0x10000006253' 2024-09-21T00:30:12.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40129 fp=#0x1000000625c' 2024-09-21T00:30:12.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40161 fp=#0x10000002675' 2024-09-21T00:30:12.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40193 fp=#0x100000026ae' 2024-09-21T00:30:12.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40225 fp=#0x10000000861' 2024-09-21T00:30:12.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40289 fp=#0x100000042f6' 2024-09-21T00:30:12.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40321 fp=#0x10000005f21' 2024-09-21T00:30:12.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40353 fp=#0x10000000a12' 2024-09-21T00:30:12.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40385 fp=#0x100000042a1' 2024-09-21T00:30:12.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40417 fp=#0x10000000a5e' 2024-09-21T00:30:12.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40513 fp=#0x100000028b3' 2024-09-21T00:30:12.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40577 fp=#0x100000037fa' 2024-09-21T00:30:12.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40673 fp=#0x10000005ecb' 2024-09-21T00:30:12.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40705 fp=#0x1000000490f' 2024-09-21T00:30:12.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40737 fp=#0x10000005ecd' 2024-09-21T00:30:12.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40833 fp=#0x100000063e4' 2024-09-21T00:30:12.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40865 fp=#0x10000003061' 2024-09-21T00:30:12.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40993 fp=#0x10000005112' 2024-09-21T00:30:12.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41025 fp=#0x1000000510c' 2024-09-21T00:30:12.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41057 fp=#0x100000049dd' 2024-09-21T00:30:12.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41089 fp=#0x10000005103' 2024-09-21T00:30:12.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41121 fp=#0x1000000510b' 2024-09-21T00:30:12.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30786 fp=#0x1000000249d' 2024-09-21T00:30:12.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30818 fp=#0x10000004800' 2024-09-21T00:30:12.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30882 fp=#0x100000036c1' 2024-09-21T00:30:12.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32706 fp=#0x10000005760' 2024-09-21T00:30:12.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32738 fp=#0x10000005757' 2024-09-21T00:30:12.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32770 fp=#0x10000005719' 2024-09-21T00:30:12.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32802 fp=#0x1000000508a' 2024-09-21T00:30:12.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32834 fp=#0x1000000502e' 2024-09-21T00:30:12.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32866 fp=#0x10000005044' 2024-09-21T00:30:12.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32898 fp=#0x10000005029' 2024-09-21T00:30:12.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32930 fp=#0x1000000504c' 2024-09-21T00:30:12.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32962 fp=#0x1000000503e' 2024-09-21T00:30:12.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32994 fp=#0x10000005cff' 2024-09-21T00:30:12.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33026 fp=#0x10000005ce9' 2024-09-21T00:30:12.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33058 fp=#0x10000005cad' 2024-09-21T00:30:12.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33090 fp=#0x10000005cb3' 2024-09-21T00:30:12.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33122 fp=#0x10000005cb7' 2024-09-21T00:30:12.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33154 fp=#0x1000000606f' 2024-09-21T00:30:12.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33282 fp=#0x10000005d45' 2024-09-21T00:30:12.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33314 fp=#0x100000067f7' 2024-09-21T00:30:12.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33346 fp=#0x100000067fb' 2024-09-21T00:30:12.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33378 fp=#0x100000067f2' 2024-09-21T00:30:12.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33474 fp=#0x10000003e39' 2024-09-21T00:30:12.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33602 fp=#0x10000005a8d' 2024-09-21T00:30:12.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33634 fp=#0x100000055ba' 2024-09-21T00:30:12.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33698 fp=#0x10000005a6f' 2024-09-21T00:30:12.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33730 fp=#0x10000006773' 2024-09-21T00:30:12.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33762 fp=#0x10000006777' 2024-09-21T00:30:12.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33794 fp=#0x100000067ae' 2024-09-21T00:30:12.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33922 fp=#0x1000000472f' 2024-09-21T00:30:12.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33954 fp=#0x10000006400' 2024-09-21T00:30:12.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33986 fp=#0x10000004736' 2024-09-21T00:30:12.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34018 fp=#0x10000006406' 2024-09-21T00:30:12.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34050 fp=#0x1000000643e' 2024-09-21T00:30:12.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34082 fp=#0x10000003260' 2024-09-21T00:30:12.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34114 fp=#0x10000003237' 2024-09-21T00:30:12.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34146 fp=#0x100000062ab' 2024-09-21T00:30:12.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34178 fp=#0x10000006258' 2024-09-21T00:30:12.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34210 fp=#0x1000000646c' 2024-09-21T00:30:12.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34242 fp=#0x10000006281' 2024-09-21T00:30:12.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34274 fp=#0x1000000624a' 2024-09-21T00:30:12.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34306 fp=#0x100000026b9' 2024-09-21T00:30:12.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34338 fp=#0x100000056c5' 2024-09-21T00:30:12.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34370 fp=#0x10000005696' 2024-09-21T00:30:12.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34434 fp=#0x10000002561' 2024-09-21T00:30:12.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34466 fp=#0x1000000265c' 2024-09-21T00:30:12.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34498 fp=#0x10000002678' 2024-09-21T00:30:12.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34530 fp=#0x1000000569d' 2024-09-21T00:30:12.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34562 fp=#0x100000063a9' 2024-09-21T00:30:12.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34594 fp=#0x10000006397' 2024-09-21T00:30:12.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34626 fp=#0x10000006340' 2024-09-21T00:30:12.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34658 fp=#0x10000006393' 2024-09-21T00:30:12.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34754 fp=#0x100000064e6' 2024-09-21T00:30:12.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34786 fp=#0x100000064bc' 2024-09-21T00:30:12.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34818 fp=#0x100000064d7' 2024-09-21T00:30:12.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34850 fp=#0x100000064b5' 2024-09-21T00:30:12.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34882 fp=#0x100000057c7' 2024-09-21T00:30:12.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34914 fp=#0x10000005345' 2024-09-21T00:30:12.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34946 fp=#0x100000052e0' 2024-09-21T00:30:12.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35010 fp=#0x100000052d0' 2024-09-21T00:30:12.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35042 fp=#0x100000052f8' 2024-09-21T00:30:12.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35074 fp=#0x100000060b1' 2024-09-21T00:30:12.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35106 fp=#0x100000060a1' 2024-09-21T00:30:12.630 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35138 fp=#0x100000060eb' 2024-09-21T00:30:12.630 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35202 fp=#0x1000000609c' 2024-09-21T00:30:12.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35234 fp=#0x10000006090' 2024-09-21T00:30:12.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35298 fp=#0x10000005786' 2024-09-21T00:30:12.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35362 fp=#0x10000005323' 2024-09-21T00:30:12.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35426 fp=#0x1000000531b' 2024-09-21T00:30:12.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35554 fp=#0x10000005fdd' 2024-09-21T00:30:12.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35586 fp=#0x10000005f73' 2024-09-21T00:30:12.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35618 fp=#0x10000005b8d' 2024-09-21T00:30:12.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35650 fp=#0x10000005b68' 2024-09-21T00:30:12.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35714 fp=#0x1000000491b' 2024-09-21T00:30:12.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35746 fp=#0x10000005018' 2024-09-21T00:30:12.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35778 fp=#0x10000004ff6' 2024-09-21T00:30:12.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35810 fp=#0x10000004f3b' 2024-09-21T00:30:12.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35842 fp=#0x10000004f6b' 2024-09-21T00:30:12.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35874 fp=#0x10000004fca' 2024-09-21T00:30:12.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35906 fp=#0x10000006726' 2024-09-21T00:30:12.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35938 fp=#0x1000000671e' 2024-09-21T00:30:12.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35970 fp=#0x1000000687a' 2024-09-21T00:30:12.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36002 fp=#0x1000000684a' 2024-09-21T00:30:12.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36194 fp=#0x100000068e7' 2024-09-21T00:30:12.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36226 fp=#0x100000068b9' 2024-09-21T00:30:12.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36258 fp=#0x100000068cc' 2024-09-21T00:30:12.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36290 fp=#0x100000065fa' 2024-09-21T00:30:12.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36322 fp=#0x1000000661a' 2024-09-21T00:30:12.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36386 fp=#0x1000000659a' 2024-09-21T00:30:12.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36418 fp=#0x1000000401a' 2024-09-21T00:30:12.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36450 fp=#0x1000000614b' 2024-09-21T00:30:12.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36482 fp=#0x10000005ffa' 2024-09-21T00:30:12.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36546 fp=#0x10000005aab' 2024-09-21T00:30:12.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36578 fp=#0x10000005a80' 2024-09-21T00:30:12.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36674 fp=#0x1000000491c' 2024-09-21T00:30:12.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36706 fp=#0x100000047f7' 2024-09-21T00:30:12.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36738 fp=#0x1000000483d' 2024-09-21T00:30:12.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36770 fp=#0x100000048fc' 2024-09-21T00:30:12.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36802 fp=#0x10000005b15' 2024-09-21T00:30:12.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36834 fp=#0x10000005b00' 2024-09-21T00:30:12.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36994 fp=#0x100000051ad' 2024-09-21T00:30:12.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37026 fp=#0x100000051c4' 2024-09-21T00:30:12.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37090 fp=#0x100000051b2' 2024-09-21T00:30:12.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37186 fp=#0x10000003ceb' 2024-09-21T00:30:12.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37250 fp=#0x10000005be5' 2024-09-21T00:30:12.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37282 fp=#0x10000005b96' 2024-09-21T00:30:12.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37346 fp=#0x10000005bda' 2024-09-21T00:30:12.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37378 fp=#0x10000005eda' 2024-09-21T00:30:12.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37410 fp=#0x10000005ee4' 2024-09-21T00:30:12.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37442 fp=#0x10000004afc' 2024-09-21T00:30:12.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37474 fp=#0x10000004b2f' 2024-09-21T00:30:12.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37506 fp=#0x10000004a8c' 2024-09-21T00:30:12.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37538 fp=#0x10000004a90' 2024-09-21T00:30:12.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37570 fp=#0x10000004aff' 2024-09-21T00:30:12.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37602 fp=#0x100000054a0' 2024-09-21T00:30:12.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37634 fp=#0x10000005574' 2024-09-21T00:30:12.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37666 fp=#0x10000005519' 2024-09-21T00:30:12.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37698 fp=#0x10000004255' 2024-09-21T00:30:12.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37730 fp=#0x1000000425c' 2024-09-21T00:30:12.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37762 fp=#0x1000000582e' 2024-09-21T00:30:12.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37794 fp=#0x100000042a8' 2024-09-21T00:30:12.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37826 fp=#0x1000000580c' 2024-09-21T00:30:12.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37858 fp=#0x1000000429c' 2024-09-21T00:30:12.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37890 fp=#0x10000005785' 2024-09-21T00:30:12.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37922 fp=#0x10000001d12' 2024-09-21T00:30:12.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37954 fp=#0x10000001cac' 2024-09-21T00:30:12.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37986 fp=#0x10000001c20' 2024-09-21T00:30:12.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38018 fp=#0x100000028a1' 2024-09-21T00:30:12.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38050 fp=#0x10000003cad' 2024-09-21T00:30:12.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38114 fp=#0x10000003aa7' 2024-09-21T00:30:12.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38210 fp=#0x1000000586a' 2024-09-21T00:30:12.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38242 fp=#0x1000000594b' 2024-09-21T00:30:12.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38274 fp=#0x1000000585e' 2024-09-21T00:30:12.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38306 fp=#0x10000006176' 2024-09-21T00:30:12.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38338 fp=#0x1000000587f' 2024-09-21T00:30:12.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38370 fp=#0x10000006178' 2024-09-21T00:30:12.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38402 fp=#0x10000002b9d' 2024-09-21T00:30:12.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38434 fp=#0x10000002a65' 2024-09-21T00:30:12.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38466 fp=#0x100000045d4' 2024-09-21T00:30:12.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38498 fp=#0x100000045ed' 2024-09-21T00:30:12.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38530 fp=#0x100000045ec' 2024-09-21T00:30:12.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38562 fp=#0x10000002a39' 2024-09-21T00:30:12.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38594 fp=#0x100000045e7' 2024-09-21T00:30:12.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38818 fp=#0x10000004873' 2024-09-21T00:30:12.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38850 fp=#0x100000041bb' 2024-09-21T00:30:12.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38882 fp=#0x10000004075' 2024-09-21T00:30:12.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38914 fp=#0x10000004088' 2024-09-21T00:30:12.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38978 fp=#0x10000004084' 2024-09-21T00:30:12.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39010 fp=#0x10000004050' 2024-09-21T00:30:12.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39042 fp=#0x10000001ce3' 2024-09-21T00:30:12.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39074 fp=#0x10000001be8' 2024-09-21T00:30:12.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39106 fp=#0x10000001ff1' 2024-09-21T00:30:12.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39138 fp=#0x10000001d5c' 2024-09-21T00:30:12.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39170 fp=#0x10000004504' 2024-09-21T00:30:12.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39202 fp=#0x100000044bc' 2024-09-21T00:30:12.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39234 fp=#0x100000044fc' 2024-09-21T00:30:12.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39266 fp=#0x10000004502' 2024-09-21T00:30:12.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39298 fp=#0x100000048c2' 2024-09-21T00:30:12.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39330 fp=#0x10000004810' 2024-09-21T00:30:12.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39362 fp=#0x1000000482c' 2024-09-21T00:30:12.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39394 fp=#0x10000005ae8' 2024-09-21T00:30:12.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39426 fp=#0x10000005ad7' 2024-09-21T00:30:12.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39458 fp=#0x10000005e35' 2024-09-21T00:30:12.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39522 fp=#0x10000005e1c' 2024-09-21T00:30:12.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39554 fp=#0x10000005a03' 2024-09-21T00:30:12.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39618 fp=#0x100000050a0' 2024-09-21T00:30:12.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39650 fp=#0x100000050eb' 2024-09-21T00:30:12.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39682 fp=#0x10000005d90' 2024-09-21T00:30:12.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39714 fp=#0x10000005d7f' 2024-09-21T00:30:12.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39746 fp=#0x10000005b38' 2024-09-21T00:30:12.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39778 fp=#0x10000005b4b' 2024-09-21T00:30:12.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39810 fp=#0x100000062ee' 2024-09-21T00:30:12.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39842 fp=#0x10000006319' 2024-09-21T00:30:12.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39874 fp=#0x10000006346' 2024-09-21T00:30:12.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39938 fp=#0x10000004e8d' 2024-09-21T00:30:12.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39970 fp=#0x10000004ea8' 2024-09-21T00:30:12.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40034 fp=#0x10000006248' 2024-09-21T00:30:12.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40066 fp=#0x10000006218' 2024-09-21T00:30:12.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40098 fp=#0x1000000629e' 2024-09-21T00:30:12.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40130 fp=#0x10000006266' 2024-09-21T00:30:12.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40162 fp=#0x100000026a5' 2024-09-21T00:30:12.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40194 fp=#0x10000002646' 2024-09-21T00:30:12.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40226 fp=#0x10000004304' 2024-09-21T00:30:12.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40258 fp=#0x100000042eb' 2024-09-21T00:30:12.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40290 fp=#0x100000008e8' 2024-09-21T00:30:12.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40322 fp=#0x100000042e3' 2024-09-21T00:30:12.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40354 fp=#0x10000000a2b' 2024-09-21T00:30:12.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40386 fp=#0x10000004338' 2024-09-21T00:30:12.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40418 fp=#0x100000009be' 2024-09-21T00:30:12.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40450 fp=#0x100000058fa' 2024-09-21T00:30:12.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40514 fp=#0x10000002890' 2024-09-21T00:30:12.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40610 fp=#0x10000003805' 2024-09-21T00:30:12.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40674 fp=#0x10000005eb9' 2024-09-21T00:30:12.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40706 fp=#0x10000004822' 2024-09-21T00:30:12.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40738 fp=#0x10000004955' 2024-09-21T00:30:12.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40770 fp=#0x1000000480b' 2024-09-21T00:30:12.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40802 fp=#0x10000005edb' 2024-09-21T00:30:12.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40834 fp=#0x10000003041' 2024-09-21T00:30:12.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40866 fp=#0x100000063e7' 2024-09-21T00:30:12.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40898 fp=#0x100000031b4' 2024-09-21T00:30:12.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40930 fp=#0x100000063d5' 2024-09-21T00:30:12.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40962 fp=#0x100000063c3' 2024-09-21T00:30:12.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40994 fp=#0x10000005115' 2024-09-21T00:30:12.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41026 fp=#0x10000005104' 2024-09-21T00:30:12.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41058 fp=#0x10000004981' 2024-09-21T00:30:12.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41090 fp=#0x10000005110' 2024-09-21T00:30:12.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41122 fp=#0x100000062a6' 2024-09-21T00:30:12.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30787 fp=#0x10000003d43' 2024-09-21T00:30:12.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30819 fp=#0x10000003efe' 2024-09-21T00:30:12.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30883 fp=#0x1000000128d' 2024-09-21T00:30:12.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32707 fp=#0x10000005768' 2024-09-21T00:30:12.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32739 fp=#0x1000000574b' 2024-09-21T00:30:12.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32771 fp=#0x10000005724' 2024-09-21T00:30:12.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32835 fp=#0x1000000507a' 2024-09-21T00:30:12.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32867 fp=#0x1000000508e' 2024-09-21T00:30:12.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32899 fp=#0x1000000506b' 2024-09-21T00:30:12.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32931 fp=#0x10000006583' 2024-09-21T00:30:12.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32963 fp=#0x10000005041' 2024-09-21T00:30:12.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32995 fp=#0x10000005c47' 2024-09-21T00:30:12.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33027 fp=#0x10000005c90' 2024-09-21T00:30:12.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33059 fp=#0x10000005cbd' 2024-09-21T00:30:12.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33091 fp=#0x10000005cbc' 2024-09-21T00:30:12.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33123 fp=#0x10000005c97' 2024-09-21T00:30:12.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33155 fp=#0x1000000603e' 2024-09-21T00:30:12.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33283 fp=#0x10000006076' 2024-09-21T00:30:12.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33315 fp=#0x10000006825' 2024-09-21T00:30:12.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33347 fp=#0x100000067e0' 2024-09-21T00:30:12.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33379 fp=#0x1000000680d' 2024-09-21T00:30:12.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33411 fp=#0x10000003e3f' 2024-09-21T00:30:12.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33443 fp=#0x10000003d98' 2024-09-21T00:30:12.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33475 fp=#0x10000003e28' 2024-09-21T00:30:12.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33539 fp=#0x1000000566d' 2024-09-21T00:30:12.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33571 fp=#0x100000055c0' 2024-09-21T00:30:12.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33603 fp=#0x10000005a84' 2024-09-21T00:30:12.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33635 fp=#0x10000005a61' 2024-09-21T00:30:12.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33667 fp=#0x10000005a35' 2024-09-21T00:30:12.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33731 fp=#0x100000067ba' 2024-09-21T00:30:12.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33763 fp=#0x10000006758' 2024-09-21T00:30:12.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33795 fp=#0x10000006763' 2024-09-21T00:30:12.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33923 fp=#0x100000047c0' 2024-09-21T00:30:12.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33955 fp=#0x1000000640e' 2024-09-21T00:30:12.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33987 fp=#0x1000000644a' 2024-09-21T00:30:12.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34019 fp=#0x1000000473c' 2024-09-21T00:30:12.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34051 fp=#0x10000006439' 2024-09-21T00:30:12.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34083 fp=#0x1000000325e' 2024-09-21T00:30:12.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34115 fp=#0x100000031f1' 2024-09-21T00:30:12.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34147 fp=#0x100000064a3' 2024-09-21T00:30:12.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34179 fp=#0x1000000626d' 2024-09-21T00:30:12.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34211 fp=#0x100000064a4' 2024-09-21T00:30:12.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34243 fp=#0x100000062a3' 2024-09-21T00:30:12.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34275 fp=#0x10000006251' 2024-09-21T00:30:12.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34339 fp=#0x10000002676' 2024-09-21T00:30:12.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34371 fp=#0x100000056c7' 2024-09-21T00:30:12.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34403 fp=#0x100000061d1' 2024-09-21T00:30:12.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34467 fp=#0x10000002596' 2024-09-21T00:30:12.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34499 fp=#0x10000002665' 2024-09-21T00:30:12.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34531 fp=#0x100000061ae' 2024-09-21T00:30:12.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34563 fp=#0x100000062d7' 2024-09-21T00:30:12.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34595 fp=#0x10000006331' 2024-09-21T00:30:12.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34627 fp=#0x10000006318' 2024-09-21T00:30:12.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34659 fp=#0x100000062d3' 2024-09-21T00:30:12.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34755 fp=#0x100000057d3' 2024-09-21T00:30:12.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34787 fp=#0x100000057ce' 2024-09-21T00:30:12.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34819 fp=#0x100000064f3' 2024-09-21T00:30:12.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34851 fp=#0x100000057b8' 2024-09-21T00:30:12.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34883 fp=#0x100000057ca' 2024-09-21T00:30:12.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34915 fp=#0x100000052df' 2024-09-21T00:30:12.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34979 fp=#0x10000006969' 2024-09-21T00:30:12.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35011 fp=#0x10000006943' 2024-09-21T00:30:12.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35043 fp=#0x1000000529f' 2024-09-21T00:30:12.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35075 fp=#0x100000060b4' 2024-09-21T00:30:12.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35107 fp=#0x1000000608b' 2024-09-21T00:30:12.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35203 fp=#0x100000060cc' 2024-09-21T00:30:12.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35235 fp=#0x100000060ca' 2024-09-21T00:30:12.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35299 fp=#0x10000003f2e' 2024-09-21T00:30:12.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35491 fp=#0x10000005f82' 2024-09-21T00:30:12.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35587 fp=#0x10000005f8b' 2024-09-21T00:30:12.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35683 fp=#0x10000005f23' 2024-09-21T00:30:12.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35715 fp=#0x10000004950' 2024-09-21T00:30:12.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35747 fp=#0x10000004f17' 2024-09-21T00:30:12.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35779 fp=#0x10000004f71' 2024-09-21T00:30:12.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35811 fp=#0x10000004fda' 2024-09-21T00:30:12.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35843 fp=#0x10000004f0e' 2024-09-21T00:30:12.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35875 fp=#0x10000006703' 2024-09-21T00:30:12.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35907 fp=#0x10000006751' 2024-09-21T00:30:12.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35939 fp=#0x10000006723' 2024-09-21T00:30:12.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35971 fp=#0x10000006875' 2024-09-21T00:30:12.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36003 fp=#0x10000006849' 2024-09-21T00:30:12.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36035 fp=#0x10000005f4a' 2024-09-21T00:30:12.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36099 fp=#0x10000005f61' 2024-09-21T00:30:12.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36195 fp=#0x100000068d4' 2024-09-21T00:30:12.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36227 fp=#0x100000068bf' 2024-09-21T00:30:12.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36259 fp=#0x10000006898' 2024-09-21T00:30:12.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36291 fp=#0x1000000660e' 2024-09-21T00:30:12.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36323 fp=#0x100000065ed' 2024-09-21T00:30:12.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36355 fp=#0x10000006590' 2024-09-21T00:30:12.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36387 fp=#0x10000006604' 2024-09-21T00:30:12.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36419 fp=#0x10000006145' 2024-09-21T00:30:12.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36451 fp=#0x1000000613b' 2024-09-21T00:30:12.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36483 fp=#0x10000006043' 2024-09-21T00:30:12.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36547 fp=#0x100000059f3' 2024-09-21T00:30:12.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36579 fp=#0x10000006626' 2024-09-21T00:30:12.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36643 fp=#0x10000005a38' 2024-09-21T00:30:12.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36675 fp=#0x10000004769' 2024-09-21T00:30:12.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36707 fp=#0x100000047df' 2024-09-21T00:30:12.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36739 fp=#0x1000000484a' 2024-09-21T00:30:12.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36771 fp=#0x10000004755' 2024-09-21T00:30:12.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36803 fp=#0x10000005a20' 2024-09-21T00:30:12.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36835 fp=#0x10000005888' 2024-09-21T00:30:12.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36867 fp=#0x100000058e5' 2024-09-21T00:30:12.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36899 fp=#0x10000005afa' 2024-09-21T00:30:12.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36963 fp=#0x10000005b01' 2024-09-21T00:30:12.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36995 fp=#0x1000000528c' 2024-09-21T00:30:12.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37027 fp=#0x1000000528e' 2024-09-21T00:30:12.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37059 fp=#0x100000051ab' 2024-09-21T00:30:12.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37091 fp=#0x1000000524d' 2024-09-21T00:30:12.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37187 fp=#0x10000003d2e' 2024-09-21T00:30:12.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37219 fp=#0x10000003cd6' 2024-09-21T00:30:12.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37251 fp=#0x10000005bae' 2024-09-21T00:30:12.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37283 fp=#0x10000005bb0' 2024-09-21T00:30:12.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37315 fp=#0x10000005bdc' 2024-09-21T00:30:12.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37347 fp=#0x10000005c27' 2024-09-21T00:30:12.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37379 fp=#0x10000005ed2' 2024-09-21T00:30:12.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37411 fp=#0x10000005ee9' 2024-09-21T00:30:12.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37443 fp=#0x10000004ba6' 2024-09-21T00:30:12.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37475 fp=#0x10000004b1d' 2024-09-21T00:30:12.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37507 fp=#0x10000004b41' 2024-09-21T00:30:12.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37539 fp=#0x10000004b23' 2024-09-21T00:30:12.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37571 fp=#0x10000004b92' 2024-09-21T00:30:12.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37603 fp=#0x10000005511' 2024-09-21T00:30:12.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37635 fp=#0x1000000554f' 2024-09-21T00:30:12.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37667 fp=#0x1000000553e' 2024-09-21T00:30:12.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37699 fp=#0x100000042c4' 2024-09-21T00:30:12.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37731 fp=#0x100000057f1' 2024-09-21T00:30:12.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37763 fp=#0x100000057f9' 2024-09-21T00:30:12.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37795 fp=#0x10000005819' 2024-09-21T00:30:12.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37827 fp=#0x100000042e4' 2024-09-21T00:30:12.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37859 fp=#0x100000042a4' 2024-09-21T00:30:12.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37891 fp=#0x10000001d53' 2024-09-21T00:30:12.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37923 fp=#0x10000004ac1' 2024-09-21T00:30:12.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37955 fp=#0x10000001d65' 2024-09-21T00:30:12.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37987 fp=#0x10000001c9e' 2024-09-21T00:30:12.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38019 fp=#0x10000001cf9' 2024-09-21T00:30:12.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38051 fp=#0x10000003c95' 2024-09-21T00:30:12.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38083 fp=#0x10000003a4a' 2024-09-21T00:30:12.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38115 fp=#0x10000003ab0' 2024-09-21T00:30:12.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38211 fp=#0x100000059a2' 2024-09-21T00:30:12.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38243 fp=#0x1000000617b' 2024-09-21T00:30:12.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38275 fp=#0x10000005868' 2024-09-21T00:30:12.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38307 fp=#0x100000058e0' 2024-09-21T00:30:12.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38339 fp=#0x1000000599b' 2024-09-21T00:30:12.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38403 fp=#0x10000002a49' 2024-09-21T00:30:12.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38467 fp=#0x10000002b76' 2024-09-21T00:30:12.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38499 fp=#0x10000002b39' 2024-09-21T00:30:12.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38531 fp=#0x100000045c8' 2024-09-21T00:30:12.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38563 fp=#0x10000002ba6' 2024-09-21T00:30:12.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38595 fp=#0x100000045c2' 2024-09-21T00:30:12.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38627 fp=#0x10000002b73' 2024-09-21T00:30:12.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38659 fp=#0x10000004894' 2024-09-21T00:30:12.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38723 fp=#0x100000010b7' 2024-09-21T00:30:12.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38755 fp=#0x100000010ea' 2024-09-21T00:30:12.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38787 fp=#0x10000001442' 2024-09-21T00:30:12.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38851 fp=#0x1000000403d' 2024-09-21T00:30:12.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38979 fp=#0x10000004037' 2024-09-21T00:30:12.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39011 fp=#0x1000000411a' 2024-09-21T00:30:12.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39043 fp=#0x10000001ff6' 2024-09-21T00:30:12.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39075 fp=#0x10000001c6e' 2024-09-21T00:30:12.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39107 fp=#0x10000001bde' 2024-09-21T00:30:12.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39139 fp=#0x10000001d9c' 2024-09-21T00:30:12.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39203 fp=#0x100000044c3' 2024-09-21T00:30:12.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39235 fp=#0x100000044be' 2024-09-21T00:30:12.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39267 fp=#0x100000044ac' 2024-09-21T00:30:12.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39299 fp=#0x10000004895' 2024-09-21T00:30:12.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39331 fp=#0x100000048d2' 2024-09-21T00:30:12.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39363 fp=#0x10000004835' 2024-09-21T00:30:12.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39395 fp=#0x10000005b03' 2024-09-21T00:30:12.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39427 fp=#0x100000059e1' 2024-09-21T00:30:12.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39491 fp=#0x100000059c2' 2024-09-21T00:30:12.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39523 fp=#0x10000005e16' 2024-09-21T00:30:12.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39555 fp=#0x10000005a1b' 2024-09-21T00:30:12.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39587 fp=#0x100000050b6' 2024-09-21T00:30:12.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39651 fp=#0x100000050d9' 2024-09-21T00:30:12.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39715 fp=#0x10000005de2' 2024-09-21T00:30:12.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39747 fp=#0x10000005b34' 2024-09-21T00:30:12.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39779 fp=#0x10000005b30' 2024-09-21T00:30:12.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39811 fp=#0x1000000638c' 2024-09-21T00:30:12.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39843 fp=#0x1000000632e' 2024-09-21T00:30:12.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39875 fp=#0x10000006301' 2024-09-21T00:30:12.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39907 fp=#0x10000004efa' 2024-09-21T00:30:12.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39939 fp=#0x10000004e97' 2024-09-21T00:30:12.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39971 fp=#0x10000004e9e' 2024-09-21T00:30:12.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40003 fp=#0x10000004eb7' 2024-09-21T00:30:12.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40035 fp=#0x100000062aa' 2024-09-21T00:30:12.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40067 fp=#0x10000006236' 2024-09-21T00:30:12.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40099 fp=#0x100000061da' 2024-09-21T00:30:12.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40131 fp=#0x10000006203' 2024-09-21T00:30:12.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40163 fp=#0x10000002640' 2024-09-21T00:30:12.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40195 fp=#0x100000026c0' 2024-09-21T00:30:12.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40227 fp=#0x100000008b4' 2024-09-21T00:30:12.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40291 fp=#0x100000042bd' 2024-09-21T00:30:12.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40323 fp=#0x1000000428c' 2024-09-21T00:30:12.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40355 fp=#0x100000042c1' 2024-09-21T00:30:12.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40387 fp=#0x1000000088b' 2024-09-21T00:30:12.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40419 fp=#0x10000004334' 2024-09-21T00:30:12.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40483 fp=#0x100000058ec' 2024-09-21T00:30:12.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40515 fp=#0x10000002834' 2024-09-21T00:30:12.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40675 fp=#0x10000005ea8' 2024-09-21T00:30:12.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40707 fp=#0x10000004842' 2024-09-21T00:30:12.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40739 fp=#0x10000005ed5' 2024-09-21T00:30:12.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40771 fp=#0x10000005eae' 2024-09-21T00:30:12.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40803 fp=#0x10000005ebc' 2024-09-21T00:30:12.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40963 fp=#0x100000063ca' 2024-09-21T00:30:12.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40995 fp=#0x1000000499f' 2024-09-21T00:30:12.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41027 fp=#0x10000005148' 2024-09-21T00:30:12.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41059 fp=#0x10000005129' 2024-09-21T00:30:12.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41091 fp=#0x10000004e8a' 2024-09-21T00:30:12.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41123 fp=#0x100000049bb' 2024-09-21T00:30:12.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30788 fp=#0x100000047db' 2024-09-21T00:30:12.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30820 fp=#0x100000059b4' 2024-09-21T00:30:12.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:30884 fp=#0x100000047f8' 2024-09-21T00:30:12.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32708 fp=#0x1000000568d' 2024-09-21T00:30:12.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32772 fp=#0x10000005755' 2024-09-21T00:30:12.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32836 fp=#0x1000000505d' 2024-09-21T00:30:12.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32964 fp=#0x10000005059' 2024-09-21T00:30:12.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:32996 fp=#0x10000005c6f' 2024-09-21T00:30:12.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33028 fp=#0x10000005cd4' 2024-09-21T00:30:12.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33060 fp=#0x10000005d32' 2024-09-21T00:30:12.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33092 fp=#0x10000005c8e' 2024-09-21T00:30:12.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33124 fp=#0x10000005ccf' 2024-09-21T00:30:12.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33220 fp=#0x10000006067' 2024-09-21T00:30:12.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33316 fp=#0x10000006824' 2024-09-21T00:30:12.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33348 fp=#0x1000000682b' 2024-09-21T00:30:12.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33380 fp=#0x10000006819' 2024-09-21T00:30:12.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33412 fp=#0x10000003dc3' 2024-09-21T00:30:12.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33476 fp=#0x10000003d9e' 2024-09-21T00:30:12.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33508 fp=#0x10000003d72' 2024-09-21T00:30:12.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33604 fp=#0x10000005ab7' 2024-09-21T00:30:12.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33636 fp=#0x10000005a73' 2024-09-21T00:30:12.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33668 fp=#0x10000005a39' 2024-09-21T00:30:12.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33732 fp=#0x100000067b9' 2024-09-21T00:30:12.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33764 fp=#0x100000067c0' 2024-09-21T00:30:12.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33796 fp=#0x10000006798' 2024-09-21T00:30:12.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33924 fp=#0x10000006448' 2024-09-21T00:30:12.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33956 fp=#0x10000004747' 2024-09-21T00:30:12.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:33988 fp=#0x10000006426' 2024-09-21T00:30:12.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34020 fp=#0x1000000478f' 2024-09-21T00:30:12.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34084 fp=#0x10000003274' 2024-09-21T00:30:12.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34116 fp=#0x100000031e4' 2024-09-21T00:30:12.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34148 fp=#0x10000006495' 2024-09-21T00:30:12.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34180 fp=#0x1000000625a' 2024-09-21T00:30:12.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34212 fp=#0x100000062b5' 2024-09-21T00:30:12.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34244 fp=#0x10000006467' 2024-09-21T00:30:12.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34276 fp=#0x1000000627e' 2024-09-21T00:30:12.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34308 fp=#0x1000000568b' 2024-09-21T00:30:12.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34340 fp=#0x100000026b0' 2024-09-21T00:30:12.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34372 fp=#0x10000005692' 2024-09-21T00:30:12.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34404 fp=#0x100000061c9' 2024-09-21T00:30:12.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34436 fp=#0x1000000266f' 2024-09-21T00:30:12.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34468 fp=#0x100000026bf' 2024-09-21T00:30:12.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34500 fp=#0x1000000257f' 2024-09-21T00:30:12.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34532 fp=#0x10000006196' 2024-09-21T00:30:12.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34564 fp=#0x100000063a8' 2024-09-21T00:30:12.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34596 fp=#0x10000006341' 2024-09-21T00:30:12.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34628 fp=#0x1000000639a' 2024-09-21T00:30:12.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34660 fp=#0x10000006622' 2024-09-21T00:30:12.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34756 fp=#0x100000057ef' 2024-09-21T00:30:12.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34788 fp=#0x100000064bd' 2024-09-21T00:30:12.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34820 fp=#0x100000057d2' 2024-09-21T00:30:12.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34852 fp=#0x100000057d4' 2024-09-21T00:30:12.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34884 fp=#0x100000057e2' 2024-09-21T00:30:12.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34916 fp=#0x1000000533d' 2024-09-21T00:30:12.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34948 fp=#0x10000005301' 2024-09-21T00:30:12.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:34980 fp=#0x10000005336' 2024-09-21T00:30:12.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35012 fp=#0x100000052d6' 2024-09-21T00:30:12.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35076 fp=#0x100000060d2' 2024-09-21T00:30:12.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35108 fp=#0x100000060d0' 2024-09-21T00:30:12.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35172 fp=#0x100000060ad' 2024-09-21T00:30:12.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35204 fp=#0x100000060d3' 2024-09-21T00:30:12.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35236 fp=#0x100000060aa' 2024-09-21T00:30:12.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35268 fp=#0x10000003fa7' 2024-09-21T00:30:12.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35300 fp=#0x10000003f0e' 2024-09-21T00:30:12.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35332 fp=#0x10000005398' 2024-09-21T00:30:12.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35428 fp=#0x1000000532d' 2024-09-21T00:30:12.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35460 fp=#0x10000005fe3' 2024-09-21T00:30:12.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35492 fp=#0x10000005f79' 2024-09-21T00:30:12.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35588 fp=#0x10000005f99' 2024-09-21T00:30:12.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35620 fp=#0x10000005bee' 2024-09-21T00:30:12.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35652 fp=#0x10000005b6d' 2024-09-21T00:30:12.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35684 fp=#0x10000005b73' 2024-09-21T00:30:12.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35716 fp=#0x10000004960' 2024-09-21T00:30:12.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35748 fp=#0x10000005016' 2024-09-21T00:30:12.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35780 fp=#0x10000004f87' 2024-09-21T00:30:12.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35812 fp=#0x10000006705' 2024-09-21T00:30:12.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35844 fp=#0x10000004f61' 2024-09-21T00:30:12.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35876 fp=#0x10000006739' 2024-09-21T00:30:12.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35908 fp=#0x10000006707' 2024-09-21T00:30:12.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35940 fp=#0x10000006741' 2024-09-21T00:30:12.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:35972 fp=#0x10000006871' 2024-09-21T00:30:12.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36004 fp=#0x1000000686d' 2024-09-21T00:30:12.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36100 fp=#0x10000005f58' 2024-09-21T00:30:12.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36164 fp=#0x10000005cd0' 2024-09-21T00:30:12.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36196 fp=#0x100000068ac' 2024-09-21T00:30:12.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36228 fp=#0x1000000689b' 2024-09-21T00:30:12.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36260 fp=#0x10000006612' 2024-09-21T00:30:12.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36388 fp=#0x10000006598' 2024-09-21T00:30:12.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36420 fp=#0x1000000614e' 2024-09-21T00:30:12.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36484 fp=#0x10000006057' 2024-09-21T00:30:12.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36548 fp=#0x10000005aa2' 2024-09-21T00:30:12.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36580 fp=#0x10000005ac8' 2024-09-21T00:30:12.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36644 fp=#0x10000005a77' 2024-09-21T00:30:12.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36676 fp=#0x1000000491a' 2024-09-21T00:30:12.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36708 fp=#0x10000004813' 2024-09-21T00:30:12.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36740 fp=#0x10000004761' 2024-09-21T00:30:12.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36772 fp=#0x100000047c4' 2024-09-21T00:30:12.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36804 fp=#0x10000005b19' 2024-09-21T00:30:12.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36836 fp=#0x10000005938' 2024-09-21T00:30:12.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36868 fp=#0x1000000595d' 2024-09-21T00:30:12.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36900 fp=#0x1000000596c' 2024-09-21T00:30:12.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36932 fp=#0x10000005afc' 2024-09-21T00:30:12.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36964 fp=#0x100000058a9' 2024-09-21T00:30:12.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:36996 fp=#0x100000051d6' 2024-09-21T00:30:12.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37028 fp=#0x10000005233' 2024-09-21T00:30:12.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37092 fp=#0x100000051bd' 2024-09-21T00:30:12.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37124 fp=#0x100000053c9' 2024-09-21T00:30:12.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37188 fp=#0x10000003cb8' 2024-09-21T00:30:12.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37220 fp=#0x10000003cc5' 2024-09-21T00:30:12.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37252 fp=#0x10000005c64' 2024-09-21T00:30:12.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37284 fp=#0x10000005bc1' 2024-09-21T00:30:12.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37316 fp=#0x10000005c1e' 2024-09-21T00:30:12.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37348 fp=#0x10000005c20' 2024-09-21T00:30:12.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37380 fp=#0x10000005f09' 2024-09-21T00:30:12.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37412 fp=#0x10000005ef0' 2024-09-21T00:30:12.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37444 fp=#0x10000004d5b' 2024-09-21T00:30:12.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37476 fp=#0x10000004b83' 2024-09-21T00:30:12.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37508 fp=#0x10000004b64' 2024-09-21T00:30:12.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37540 fp=#0x10000004ae4' 2024-09-21T00:30:12.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37572 fp=#0x10000004a74' 2024-09-21T00:30:12.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37604 fp=#0x1000000547b' 2024-09-21T00:30:12.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37636 fp=#0x10000005535' 2024-09-21T00:30:12.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37668 fp=#0x10000005475' 2024-09-21T00:30:12.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37700 fp=#0x1000000430f' 2024-09-21T00:30:12.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37732 fp=#0x1000000430a' 2024-09-21T00:30:12.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37764 fp=#0x100000042f1' 2024-09-21T00:30:12.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37796 fp=#0x10000005820' 2024-09-21T00:30:12.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37828 fp=#0x100000042ba' 2024-09-21T00:30:12.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37860 fp=#0x100000042d4' 2024-09-21T00:30:12.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37892 fp=#0x10000001b71' 2024-09-21T00:30:12.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37924 fp=#0x10000001cfd' 2024-09-21T00:30:12.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37956 fp=#0x10000001c70' 2024-09-21T00:30:12.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:37988 fp=#0x10000001bd5' 2024-09-21T00:30:12.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38020 fp=#0x10000003920' 2024-09-21T00:30:12.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38052 fp=#0x10000003c5e' 2024-09-21T00:30:12.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38212 fp=#0x10000006187' 2024-09-21T00:30:12.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38244 fp=#0x1000000617a' 2024-09-21T00:30:12.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38308 fp=#0x1000000586c' 2024-09-21T00:30:12.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38340 fp=#0x1000000593f' 2024-09-21T00:30:12.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38372 fp=#0x10000005864' 2024-09-21T00:30:12.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38404 fp=#0x10000004604' 2024-09-21T00:30:12.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38436 fp=#0x10000002a09' 2024-09-21T00:30:12.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38468 fp=#0x100000029cb' 2024-09-21T00:30:12.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38500 fp=#0x10000002b0e' 2024-09-21T00:30:12.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38564 fp=#0x10000002b1a' 2024-09-21T00:30:12.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38596 fp=#0x100000045cc' 2024-09-21T00:30:12.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38628 fp=#0x100000029da' 2024-09-21T00:30:12.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38692 fp=#0x10000000edc' 2024-09-21T00:30:12.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38724 fp=#0x10000001f7d' 2024-09-21T00:30:12.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38948 fp=#0x100000041fc' 2024-09-21T00:30:12.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:38980 fp=#0x10000004071' 2024-09-21T00:30:12.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39012 fp=#0x1000000401e' 2024-09-21T00:30:12.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39044 fp=#0x10000001dbf' 2024-09-21T00:30:12.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39076 fp=#0x10000001c50' 2024-09-21T00:30:12.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39108 fp=#0x1000000185d' 2024-09-21T00:30:12.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39140 fp=#0x10000001d60' 2024-09-21T00:30:12.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39172 fp=#0x1000000450b' 2024-09-21T00:30:12.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39204 fp=#0x100000044c7' 2024-09-21T00:30:12.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39236 fp=#0x100000044b9' 2024-09-21T00:30:12.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39268 fp=#0x100000044b8' 2024-09-21T00:30:12.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39300 fp=#0x100000048b0' 2024-09-21T00:30:12.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39332 fp=#0x10000004881' 2024-09-21T00:30:12.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39364 fp=#0x10000004888' 2024-09-21T00:30:12.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39396 fp=#0x10000005e2e' 2024-09-21T00:30:12.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39428 fp=#0x10000005a0b' 2024-09-21T00:30:12.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39460 fp=#0x10000005a9f' 2024-09-21T00:30:12.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39524 fp=#0x10000005ac1' 2024-09-21T00:30:12.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39556 fp=#0x10000005acb' 2024-09-21T00:30:12.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39588 fp=#0x100000050d5' 2024-09-21T00:30:12.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39620 fp=#0x100000050d7' 2024-09-21T00:30:12.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39652 fp=#0x10000005098' 2024-09-21T00:30:12.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39748 fp=#0x10000005b42' 2024-09-21T00:30:12.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39780 fp=#0x10000005b53' 2024-09-21T00:30:12.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39812 fp=#0x10000006348' 2024-09-21T00:30:12.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39844 fp=#0x10000006356' 2024-09-21T00:30:12.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39876 fp=#0x10000006369' 2024-09-21T00:30:12.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39908 fp=#0x10000004f03' 2024-09-21T00:30:12.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39940 fp=#0x10000004eff' 2024-09-21T00:30:12.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:39972 fp=#0x10000004ef5' 2024-09-21T00:30:12.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40004 fp=#0x10000004ee2' 2024-09-21T00:30:12.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40036 fp=#0x100000061e7' 2024-09-21T00:30:12.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40068 fp=#0x10000006246' 2024-09-21T00:30:12.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40100 fp=#0x10000006278' 2024-09-21T00:30:12.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40132 fp=#0x10000002625' 2024-09-21T00:30:12.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40164 fp=#0x100000026ba' 2024-09-21T00:30:12.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40196 fp=#0x10000002681' 2024-09-21T00:30:12.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40228 fp=#0x10000000b23' 2024-09-21T00:30:12.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40260 fp=#0x100000042a7' 2024-09-21T00:30:12.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40292 fp=#0x100000042ad' 2024-09-21T00:30:12.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40324 fp=#0x10000000a0c' 2024-09-21T00:30:12.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40388 fp=#0x10000000a75' 2024-09-21T00:30:12.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40420 fp=#0x1000000431c' 2024-09-21T00:30:12.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40676 fp=#0x1000000493e' 2024-09-21T00:30:12.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40708 fp=#0x10000004828' 2024-09-21T00:30:12.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40772 fp=#0x100000054a2' 2024-09-21T00:30:12.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40804 fp=#0x1000000492e' 2024-09-21T00:30:12.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40836 fp=#0x100000031ca' 2024-09-21T00:30:12.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40900 fp=#0x10000003054' 2024-09-21T00:30:12.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:40996 fp=#0x1000000512a' 2024-09-21T00:30:12.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41028 fp=#0x100000049da' 2024-09-21T00:30:12.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41060 fp=#0x10000005130' 2024-09-21T00:30:12.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41092 fp=#0x1000000514b' 2024-09-21T00:30:12.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 47099ce0-1: 'internal op quiesce_inode:mds.0:41124 fp=#0x100000049f4' 2024-09-21T00:30:12.895 INFO:tasks.quiescer.fs.[cephfs]:Pulled 10518 ops from rank 0 (b) into home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310/ops-47099ce0-1-mds.b.json 2024-09-21T00:30:12.895 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -- /var/run/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ops-47099ce0-1-mds.b.json 2024-09-21T00:30:12.897 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:08 smithi027 ceph-mon[30745]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T00:30:12.897 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:09 smithi027 ceph-mon[30745]: pgmap v648: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 981 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 106 KiB/s wr, 33 op/s 2024-09-21T00:30:12.897 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:11 smithi027 ceph-mon[30745]: pgmap v649: 129 pgs: 10 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 90 active+clean; 980 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 106 KiB/s wr, 33 op/s 2024-09-21T00:30:12.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:08 smithi098 ceph-mon[34725]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T00:30:12.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:09 smithi098 ceph-mon[34725]: pgmap v648: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 981 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 106 KiB/s wr, 33 op/s 2024-09-21T00:30:12.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:11 smithi098 ceph-mon[34725]: pgmap v649: 129 pgs: 10 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 90 active+clean; 980 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 106 KiB/s wr, 33 op/s 2024-09-21T00:30:12.898 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:08 smithi089 ceph-mon[34481]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T00:30:12.898 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:09 smithi089 ceph-mon[34481]: pgmap v648: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 981 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 106 KiB/s wr, 33 op/s 2024-09-21T00:30:12.898 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:11 smithi089 ceph-mon[34481]: pgmap v649: 129 pgs: 10 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 90 active+clean; 980 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 106 KiB/s wr, 33 op/s 2024-09-21T00:30:12.992 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:30:13.679 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:30:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.011+0000 7f1a97200640 1 -- 172.21.15.27:0/3567046512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a98102e80 msgr2=0x7f1a981073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.011+0000 7f1a97200640 1 --2- 172.21.15.27:0/3567046512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a98102e80 0x7f1a981073a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009900 tx=0x7f1a8402a420 comp rx=0 tx=0).stop 2024-09-21T00:30:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a97200640 1 -- 172.21.15.27:0/3567046512 shutdown_connections 2024-09-21T00:30:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a97200640 1 --2- 172.21.15.27:0/3567046512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a981079f0 0x7f1a98107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a97200640 1 --2- 172.21.15.27:0/3567046512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a98102e80 0x7f1a981073a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a97200640 1 --2- 172.21.15.27:0/3567046512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a98102530 0x7f1a98102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a97200640 1 -- 172.21.15.27:0/3567046512 >> 172.21.15.27:0/3567046512 conn(0x7f1a980fe210 msgr2=0x7f1a98100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a97200640 1 -- 172.21.15.27:0/3567046512 shutdown_connections 2024-09-21T00:30:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a97200640 1 -- 172.21.15.27:0/3567046512 wait complete. 2024-09-21T00:30:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a97200640 1 Processor -- start 2024-09-21T00:30:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a97200640 1 -- start start 2024-09-21T00:30:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a97200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a98102530 0x7f1a98116a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a97200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a98102e80 0x7f1a9810fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a97200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a981079f0 0x7f1a98110300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a97200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1a98119ee0 con 0x7f1a981079f0 2024-09-21T00:30:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a97200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1a98119d60 con 0x7f1a98102530 2024-09-21T00:30:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a97200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9811a060 con 0x7f1a98102e80 2024-09-21T00:30:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a96800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a981079f0 0x7f1a98110300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a96800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a981079f0 0x7f1a98110300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39374/0 (socket says 172.21.15.27:39374) 2024-09-21T00:30:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.017+0000 7f1a96800640 1 -- 172.21.15.27:0/2836079461 learned_addr learned my addr 172.21.15.27:0/2836079461 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.022+0000 7f1a96800640 1 -- 172.21.15.27:0/2836079461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a98102e80 msgr2=0x7f1a9810fd30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:30:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.022+0000 7f1a96800640 1 --2- 172.21.15.27:0/2836079461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a98102e80 0x7f1a9810fd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.022+0000 7f1a96800640 1 -- 172.21.15.27:0/2836079461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a98102530 msgr2=0x7f1a98116a70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:30:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.022+0000 7f1a96800640 1 --2- 172.21.15.27:0/2836079461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a98102530 0x7f1a98116a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.022+0000 7f1a96800640 1 -- 172.21.15.27:0/2836079461 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a98110b20 con 0x7f1a981079f0 2024-09-21T00:30:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.023+0000 7f1a96800640 1 --2- 172.21.15.27:0/2836079461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a981079f0 0x7f1a98110300 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000dac0 tx=0x7f1a9000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:14.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.023+0000 7f1a8ea00640 1 -- 172.21.15.27:0/2836079461 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90017070 con 0x7f1a981079f0 2024-09-21T00:30:14.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.023+0000 7f1a8ea00640 1 -- 172.21.15.27:0/2836079461 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9000be30 con 0x7f1a981079f0 2024-09-21T00:30:14.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.023+0000 7f1a8ea00640 1 -- 172.21.15.27:0/2836079461 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90010460 con 0x7f1a981079f0 2024-09-21T00:30:14.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.023+0000 7f1a97200640 1 -- 172.21.15.27:0/2836079461 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a981157f0 con 0x7f1a981079f0 2024-09-21T00:30:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.023+0000 7f1a97200640 1 -- 172.21.15.27:0/2836079461 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a98115cf0 con 0x7f1a981079f0 2024-09-21T00:30:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.034+0000 7f1a8ea00640 1 -- 172.21.15.27:0/2836079461 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1a900040b0 con 0x7f1a981079f0 2024-09-21T00:30:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1aa4294740 1 Processor -- start 2024-09-21T00:30:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1aa4294740 1 -- start start 2024-09-21T00:30:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1a8ea00640 1 --2- 172.21.15.27:0/2836079461 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a7c0b6840 0x7f1a7c0b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1a8ea00640 1 -- 172.21.15.27:0/2836079461 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9607+0+0 (secure 0 0 0) 0x7f1a900e7900 con 0x7f1a981079f0 2024-09-21T00:30:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1aa4294740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e991b0 0x561f2223de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1aa4294740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e702f0 0x561f2223e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1aa4294740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e784d0 0x561f2223e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1aa4294740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561f221b36b0 con 0x561f21e784d0 2024-09-21T00:30:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1aa4294740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561f221a2ae0 con 0x561f21e702f0 2024-09-21T00:30:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1aa4294740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561f221a68f0 con 0x561f21e991b0 2024-09-21T00:30:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1a96800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e784d0 0x561f2223e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1a96800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e784d0 0x561f2223e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39390/0 (socket says 172.21.15.27:39390) 2024-09-21T00:30:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1a96800640 1 -- 172.21.15.27:0/170284414 learned_addr learned my addr 172.21.15.27:0/170284414 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1a95e00640 1 --2- 172.21.15.27:0/170284414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e991b0 0x561f2223de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1a96800640 1 -- 172.21.15.27:0/170284414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e991b0 msgr2=0x561f2223de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1a96800640 1 --2- 172.21.15.27:0/170284414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e991b0 0x561f2223de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1a96800640 1 -- 172.21.15.27:0/170284414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e702f0 msgr2=0x561f2223e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1a95400640 1 --2- 172.21.15.27:0/170284414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e702f0 0x561f2223e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1a96800640 1 --2- 172.21.15.27:0/170284414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e702f0 0x561f2223e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.035+0000 7f1a96800640 1 -- 172.21.15.27:0/170284414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f2216f780 con 0x561f21e784d0 2024-09-21T00:30:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1a96800640 1 --2- 172.21.15.27:0/170284414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e784d0 0x561f2223e890 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1a900a9900 tx=0x7f1a900acf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6638aa908be4e4f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1a83400640 1 -- 172.21.15.27:0/170284414 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a900e5030 con 0x561f21e784d0 2024-09-21T00:30:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1a83400640 1 -- 172.21.15.27:0/170284414 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a900e5030 con 0x561f21e784d0 2024-09-21T00:30:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1a83400640 1 -- 172.21.15.27:0/170284414 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a900ac000 con 0x561f21e784d0 2024-09-21T00:30:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1aa4294740 1 -- 172.21.15.27:0/170284414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e784d0 msgr2=0x561f2223e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1aa4294740 1 --2- 172.21.15.27:0/170284414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e784d0 0x561f2223e890 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1a900a9900 tx=0x7f1a900acf90 comp rx=0 tx=0).stop 2024-09-21T00:30:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1aa4294740 1 -- 172.21.15.27:0/170284414 shutdown_connections 2024-09-21T00:30:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1aa4294740 1 --2- 172.21.15.27:0/170284414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e784d0 0x561f2223e890 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1aa4294740 1 --2- 172.21.15.27:0/170284414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e702f0 0x561f2223e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1aa4294740 1 --2- 172.21.15.27:0/170284414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e991b0 0x561f2223de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1aa4294740 1 -- 172.21.15.27:0/170284414 >> 172.21.15.27:0/170284414 conn(0x561f21e8d300 msgr2=0x561f2223bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1a95400640 1 --2- 172.21.15.27:0/170284414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e702f0 0x561f2223e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:30:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1a95400640 1 --2- 172.21.15.27:0/2836079461 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a7c0b6840 0x7f1a7c0b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1aa4294740 1 -- 172.21.15.27:0/170284414 shutdown_connections 2024-09-21T00:30:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1aa4294740 1 -- 172.21.15.27:0/170284414 wait complete. 2024-09-21T00:30:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1aa4294740 1 -- start start 2024-09-21T00:30:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1a95400640 1 --2- 172.21.15.27:0/2836079461 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a7c0b6840 0x7f1a7c0b8d50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009230 tx=0x7f1a84009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.041+0000 7f1aa4294740 1 Processor -- start 2024-09-21T00:30:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.046+0000 7f1aa4294740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e784d0 0x561f22249400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.046+0000 7f1aa4294740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e991b0 0x561f22249940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.046+0000 7f1aa4294740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e57d80 0x561f22249e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.046+0000 7f1aa4294740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561f220daab0 con 0x561f21e784d0 2024-09-21T00:30:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.046+0000 7f1aa4294740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561f22149570 con 0x561f21e991b0 2024-09-21T00:30:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.046+0000 7f1aa4294740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561f2224c590 con 0x561f21e57d80 2024-09-21T00:30:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.046+0000 7f1aa4294740 10 client.0 _ll_register_callbacks cb 0x561f22158100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.047+0000 7f1a95400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e991b0 0x561f22249940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.047+0000 7f1a95400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e991b0 0x561f22249940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49242/0 (socket says 172.21.15.27:49242) 2024-09-21T00:30:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.047+0000 7f1a95400640 1 -- 172.21.15.27:0/2134709961 learned_addr learned my addr 172.21.15.27:0/2134709961 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.047+0000 7f1a95e00640 1 --2- 172.21.15.27:0/2134709961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e784d0 0x561f22249400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.047+0000 7f1a95400640 1 -- 172.21.15.27:0/2134709961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e57d80 msgr2=0x561f22249e80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:30:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.047+0000 7f1a95400640 1 --2- 172.21.15.27:0/2134709961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e57d80 0x561f22249e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.047+0000 7f1a95400640 1 -- 172.21.15.27:0/2134709961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e784d0 msgr2=0x561f22249400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.047+0000 7f1a95400640 1 --2- 172.21.15.27:0/2134709961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e784d0 0x561f22249400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.047+0000 7f1a95400640 1 -- 172.21.15.27:0/2134709961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f21f04790 con 0x561f21e991b0 2024-09-21T00:30:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.047+0000 7f1a95400640 1 --2- 172.21.15.27:0/2134709961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e991b0 0x561f22249940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1a84033000 tx=0x7f1a84031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=345bec74ac92b02e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.047+0000 7f1a77e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:30:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.047+0000 7f1a77e00640 1 -- 172.21.15.27:0/2134709961 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84041070 con 0x561f21e991b0 2024-09-21T00:30:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.047+0000 7f1a77e00640 1 -- 172.21.15.27:0/2134709961 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9607+0+0 (secure 0 0 0) 0x7f1a840314a0 con 0x561f21e991b0 2024-09-21T00:30:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.052+0000 7f1aa4294740 1 -- 172.21.15.27:0/2134709961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a900e5030 con 0x561f21e991b0 2024-09-21T00:30:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.052+0000 7f1a77e00640 1 -- 172.21.15.27:0/2134709961 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1a84041070 con 0x561f21e991b0 2024-09-21T00:30:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.052+0000 7f1aa4294740 10 client.26333 fetch_fsmap learned FSMap version 38 2024-09-21T00:30:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.052+0000 7f1aa4294740 1 -- 172.21.15.27:0/2134709961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561f21f7d620 con 0x561f21e991b0 2024-09-21T00:30:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.052+0000 7f1a77e00640 1 -- 172.21.15.27:0/2134709961 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a900e5030 con 0x561f21e991b0 2024-09-21T00:30:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.052+0000 7f1a77e00640 1 -- 172.21.15.27:0/2134709961 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f1a84048600 con 0x561f21e991b0 2024-09-21T00:30:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.052+0000 7f1aa4294740 10 client.26333 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:30:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.052+0000 7f1aa4294740 10 client.26333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:30:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.052+0000 7f1aa4294740 20 client.26333 populate_metadata read hostname 'smithi027' 2024-09-21T00:30:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.052+0000 7f1aa4294740 1 --2- 172.21.15.27:0/2134709961 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561f21e2d1c0 0x561f2225b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.052+0000 7f1aa4294740 4 client.26333 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:30:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.052+0000 7f1aa4294740 1 -- 172.21.15.27:0/2134709961 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f221b5600 con 0x561f21e2d1c0 2024-09-21T00:30:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.057+0000 7f1a95e00640 1 --2- 172.21.15.27:0/2134709961 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561f21e2d1c0 0x561f2225b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.057+0000 7f1a95e00640 1 --2- 172.21.15.27:0/2134709961 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561f21e2d1c0 0x561f2225b550 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dc60c4233875bfb server_cookie=7d0621f7c729822d in_seq=0 out_seq=0 2024-09-21T00:30:14.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.057+0000 7f1a77e00640 10 client.26333 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:14.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.057+0000 7f1a77e00640 1 -- 172.21.15.27:0/2134709961 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561f221b5600 con 0x561f21e2d1c0 2024-09-21T00:30:14.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.057+0000 7f1a77e00640 10 client.26333 handle_command_reply: tid=0 2024-09-21T00:30:14.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.059+0000 7f1aa4294740 1 client.26333 shutdown 2024-09-21T00:30:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.059+0000 7f1aa4294740 1 -- 172.21.15.27:0/2134709961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e991b0 msgr2=0x561f22249940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.059+0000 7f1aa4294740 1 --2- 172.21.15.27:0/2134709961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e991b0 0x561f22249940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1a84033000 tx=0x7f1a84031400 comp rx=0 tx=0).stop 2024-09-21T00:30:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.059+0000 7f1aa4294740 1 -- 172.21.15.27:0/2134709961 shutdown_connections 2024-09-21T00:30:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.059+0000 7f1aa4294740 1 --2- 172.21.15.27:0/2134709961 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561f21e2d1c0 0x561f2225b550 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.059+0000 7f1a77e00640 0 client.26333 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.059+0000 7f1aa4294740 1 --2- 172.21.15.27:0/2134709961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e57d80 0x561f22249e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.059+0000 7f1aa4294740 1 --2- 172.21.15.27:0/2134709961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e991b0 0x561f22249940 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.059+0000 7f1aa4294740 1 --2- 172.21.15.27:0/2134709961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e784d0 0x561f22249400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.059+0000 7f1aa4294740 1 -- 172.21.15.27:0/2134709961 >> 172.21.15.27:0/2134709961 conn(0x561f21e8d300 msgr2=0x561f2223ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.059+0000 7f1aa4294740 1 -- 172.21.15.27:0/2134709961 shutdown_connections 2024-09-21T00:30:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.059+0000 7f1aa4294740 1 -- 172.21.15.27:0/2134709961 wait complete. 2024-09-21T00:30:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.059+0000 7f1aa4294740 20 client.26333 trim_cache size 0 max 16384 2024-09-21T00:30:14.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:13 smithi027 ceph-mon[30745]: pgmap v650: 129 pgs: 19 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 54 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 93 KiB/s wr, 30 op/s 2024-09-21T00:30:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.088+0000 7f1aa4294740 1 Processor -- start 2024-09-21T00:30:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.093+0000 7f1aa4294740 1 -- start start 2024-09-21T00:30:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.093+0000 7f1aa4294740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e2d1c0 0x561f2223f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.093+0000 7f1aa4294740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e57d80 0x561f222465d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.093+0000 7f1aa4294740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e784d0 0x561f22246b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.093+0000 7f1aa4294740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1a981c25f0 con 0x561f21e57d80 2024-09-21T00:30:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.093+0000 7f1aa4294740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1a981bfe60 con 0x561f21e2d1c0 2024-09-21T00:30:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.093+0000 7f1aa4294740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1a981bdcd0 con 0x561f21e784d0 2024-09-21T00:30:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.093+0000 7f1a95e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e2d1c0 0x561f2223f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.094+0000 7f1a95e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e2d1c0 0x561f2223f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49258/0 (socket says 172.21.15.27:49258) 2024-09-21T00:30:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.094+0000 7f1a95e00640 1 -- 172.21.15.27:0/635391685 learned_addr learned my addr 172.21.15.27:0/635391685 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.094+0000 7f1a95400640 1 --2- 172.21.15.27:0/635391685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e57d80 0x561f222465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.094+0000 7f1a95400640 1 -- 172.21.15.27:0/635391685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e784d0 msgr2=0x561f22246b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.094+0000 7f1a95400640 1 --2- 172.21.15.27:0/635391685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e784d0 0x561f22246b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.094+0000 7f1a95400640 1 -- 172.21.15.27:0/635391685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e2d1c0 msgr2=0x561f2223f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.094+0000 7f1a95400640 1 --2- 172.21.15.27:0/635391685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e2d1c0 0x561f2223f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.094+0000 7f1a95400640 1 -- 172.21.15.27:0/635391685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f22244f50 con 0x561f21e57d80 2024-09-21T00:30:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.094+0000 7f1a95400640 1 --2- 172.21.15.27:0/635391685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e57d80 0x561f222465d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1a8402a940 tx=0x7f1a84005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f64b44e1b62b93c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.094+0000 7f1a82a00640 1 -- 172.21.15.27:0/635391685 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84035050 con 0x561f21e57d80 2024-09-21T00:30:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.094+0000 7f1a82a00640 1 -- 172.21.15.27:0/635391685 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a8403f070 con 0x561f21e57d80 2024-09-21T00:30:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.094+0000 7f1a82a00640 1 -- 172.21.15.27:0/635391685 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84031c30 con 0x561f21e57d80 2024-09-21T00:30:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.094+0000 7f1aa4294740 1 -- 172.21.15.27:0/635391685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e57d80 msgr2=0x561f222465d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.100+0000 7f1aa4294740 1 --2- 172.21.15.27:0/635391685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e57d80 0x561f222465d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1a8402a940 tx=0x7f1a84005780 comp rx=0 tx=0).stop 2024-09-21T00:30:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.100+0000 7f1aa4294740 1 -- 172.21.15.27:0/635391685 shutdown_connections 2024-09-21T00:30:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.100+0000 7f1aa4294740 1 --2- 172.21.15.27:0/635391685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e784d0 0x561f22246b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.100+0000 7f1aa4294740 1 --2- 172.21.15.27:0/635391685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e57d80 0x561f222465d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.100+0000 7f1aa4294740 1 --2- 172.21.15.27:0/635391685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e2d1c0 0x561f2223f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.100+0000 7f1aa4294740 1 -- 172.21.15.27:0/635391685 >> 172.21.15.27:0/635391685 conn(0x561f21e8d300 msgr2=0x561f2223ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.100+0000 7f1aa4294740 1 -- 172.21.15.27:0/635391685 shutdown_connections 2024-09-21T00:30:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.100+0000 7f1aa4294740 1 -- 172.21.15.27:0/635391685 wait complete. 2024-09-21T00:30:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.100+0000 7f1aa4294740 1 -- start start 2024-09-21T00:30:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.100+0000 7f1aa4294740 1 Processor -- start 2024-09-21T00:30:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.105+0000 7f1aa4294740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e2d1c0 0x561f222bee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.105+0000 7f1aa4294740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e57d80 0x561f222bf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.105+0000 7f1aa4294740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e784d0 0x561f222bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a95400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e57d80 0x561f222bf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a95400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e57d80 0x561f222bf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38442/0 (socket says 172.21.15.27:38442) 2024-09-21T00:30:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a95400640 1 -- 172.21.15.27:0/1645271307 learned_addr learned my addr 172.21.15.27:0/1645271307 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1aa4294740 1 -- 172.21.15.27:0/1645271307 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1a981be320 con 0x561f21e2d1c0 2024-09-21T00:30:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1aa4294740 1 -- 172.21.15.27:0/1645271307 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1a981bf790 con 0x561f21e784d0 2024-09-21T00:30:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1aa4294740 1 -- 172.21.15.27:0/1645271307 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1a981be9f0 con 0x561f21e57d80 2024-09-21T00:30:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a96800640 1 --2- 172.21.15.27:0/1645271307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e784d0 0x561f222bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a95400640 1 -- 172.21.15.27:0/1645271307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e784d0 msgr2=0x561f222bf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a95400640 1 --2- 172.21.15.27:0/1645271307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e784d0 0x561f222bf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a95400640 1 -- 172.21.15.27:0/1645271307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e2d1c0 msgr2=0x561f222bee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a95400640 1 --2- 172.21.15.27:0/1645271307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e2d1c0 0x561f222bee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a95400640 1 -- 172.21.15.27:0/1645271307 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f22250e60 con 0x561f21e57d80 2024-09-21T00:30:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a96800640 1 --2- 172.21.15.27:0/1645271307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e784d0 0x561f222bf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:30:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1aa4294740 10 client.0 _ll_register_callbacks cb 0x7f1a84004280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1aa4294740 1 -- 172.21.15.27:0/1645271307 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a8403f070 con 0x561f21e57d80 2024-09-21T00:30:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a95400640 1 --2- 172.21.15.27:0/1645271307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e57d80 0x561f222bf390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1a84048a70 tx=0x7f1a840486a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d0d44a82ff97841 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a77e00640 10 client.26344 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:30:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a77e00640 1 -- 172.21.15.27:0/1645271307 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84035040 con 0x561f21e57d80 2024-09-21T00:30:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a77e00640 1 -- 172.21.15.27:0/1645271307 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9607+0+0 (secure 0 0 0) 0x7f1a840318b0 con 0x561f21e57d80 2024-09-21T00:30:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a77e00640 1 -- 172.21.15.27:0/1645271307 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1a84031af0 con 0x561f21e57d80 2024-09-21T00:30:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1aa4294740 10 client.26344 fetch_fsmap learned FSMap version 38 2024-09-21T00:30:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1aa4294740 1 -- 172.21.15.27:0/1645271307 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=38}) -- 0x561f222d7020 con 0x561f21e57d80 2024-09-21T00:30:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a77e00640 1 -- 172.21.15.27:0/1645271307 <== mon.1 v2:172.21.15.98:3300/0 4 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f1a8405a070 con 0x561f21e57d80 2024-09-21T00:30:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1aa4294740 10 client.26344 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:30:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1aa4294740 10 client.26344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:30:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1aa4294740 20 client.26344 populate_metadata read hostname 'smithi027' 2024-09-21T00:30:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1aa4294740 1 --2- 172.21.15.27:0/1645271307 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561f222d7580 0x561f222d99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1aa4294740 4 client.26344 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:30:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1aa4294740 1 -- 172.21.15.27:0/1645271307 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f221b10b0 con 0x561f222d7580 2024-09-21T00:30:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.106+0000 7f1a95e00640 1 --2- 172.21.15.27:0/1645271307 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561f222d7580 0x561f222d99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.112+0000 7f1a95e00640 1 --2- 172.21.15.27:0/1645271307 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561f222d7580 0x561f222d99a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a5466d4cd07226b server_cookie=6cf5a456321b3ade in_seq=0 out_seq=0 2024-09-21T00:30:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.112+0000 7f1a77e00640 10 client.26344 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.193+0000 7f1a77e00640 1 -- 172.21.15.27:0/1645271307 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f221b10b0 con 0x561f222d7580 2024-09-21T00:30:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.193+0000 7f1a77e00640 10 client.26344 handle_command_reply: tid=0 2024-09-21T00:30:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.193+0000 7f1aa4294740 1 client.26344 shutdown 2024-09-21T00:30:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.193+0000 7f1aa4294740 1 -- 172.21.15.27:0/1645271307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e57d80 msgr2=0x561f222bf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.193+0000 7f1aa4294740 1 --2- 172.21.15.27:0/1645271307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e57d80 0x561f222bf390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1a84048a70 tx=0x7f1a840486a0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.193+0000 7f1aa4294740 1 -- 172.21.15.27:0/1645271307 shutdown_connections 2024-09-21T00:30:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.193+0000 7f1aa4294740 1 --2- 172.21.15.27:0/1645271307 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561f222d7580 0x561f222d99a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.193+0000 7f1aa4294740 1 --2- 172.21.15.27:0/1645271307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561f21e784d0 0x561f222bf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.193+0000 7f1aa4294740 1 --2- 172.21.15.27:0/1645271307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f21e57d80 0x561f222bf390 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.193+0000 7f1aa4294740 1 --2- 172.21.15.27:0/1645271307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f21e2d1c0 0x561f222bee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.193+0000 7f1aa4294740 1 -- 172.21.15.27:0/1645271307 >> 172.21.15.27:0/1645271307 conn(0x561f21e8d300 msgr2=0x561f2223a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.193+0000 7f1a77e00640 0 client.26344 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.193+0000 7f1aa4294740 1 -- 172.21.15.27:0/1645271307 shutdown_connections 2024-09-21T00:30:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.193+0000 7f1aa4294740 1 -- 172.21.15.27:0/1645271307 wait complete. 2024-09-21T00:30:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.193+0000 7f1aa4294740 20 client.26344 trim_cache size 0 max 16384 2024-09-21T00:30:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:30:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:30:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:30:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:30:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.198+0000 7f1a97200640 1 -- 172.21.15.27:0/2836079461 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a7c0b6840 msgr2=0x7f1a7c0b8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.198+0000 7f1a97200640 1 --2- 172.21.15.27:0/2836079461 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a7c0b6840 0x7f1a7c0b8d50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009230 tx=0x7f1a84009260 comp rx=0 tx=0).stop 2024-09-21T00:30:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.198+0000 7f1a97200640 1 -- 172.21.15.27:0/2836079461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a981079f0 msgr2=0x7f1a98110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.198+0000 7f1a97200640 1 --2- 172.21.15.27:0/2836079461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a981079f0 0x7f1a98110300 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000dac0 tx=0x7f1a9000df80 comp rx=0 tx=0).stop 2024-09-21T00:30:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.198+0000 7f1a97200640 1 -- 172.21.15.27:0/2836079461 shutdown_connections 2024-09-21T00:30:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.198+0000 7f1a97200640 1 --2- 172.21.15.27:0/2836079461 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1a7c0b6840 0x7f1a7c0b8d50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.198+0000 7f1a97200640 1 --2- 172.21.15.27:0/2836079461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a981079f0 0x7f1a98110300 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.198+0000 7f1a97200640 1 --2- 172.21.15.27:0/2836079461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a98102e80 0x7f1a9810fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.198+0000 7f1a97200640 1 --2- 172.21.15.27:0/2836079461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a98102530 0x7f1a98116a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.198+0000 7f1a97200640 1 -- 172.21.15.27:0/2836079461 >> 172.21.15.27:0/2836079461 conn(0x7f1a980fe210 msgr2=0x7f1a98105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.198+0000 7f1a97200640 1 -- 172.21.15.27:0/2836079461 shutdown_connections 2024-09-21T00:30:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:14.198+0000 7f1a97200640 1 -- 172.21.15.27:0/2836079461 wait complete. 2024-09-21T00:30:14.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:13 smithi089 ceph-mon[34481]: pgmap v650: 129 pgs: 19 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 54 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 93 KiB/s wr, 30 op/s 2024-09-21T00:30:14.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:13 smithi098 ceph-mon[34725]: pgmap v650: 129 pgs: 19 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 54 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 93 KiB/s wr, 30 op/s 2024-09-21T00:30:16.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:15 smithi027 ceph-mon[30745]: pgmap v651: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 94 KiB/s wr, 30 op/s 2024-09-21T00:30:16.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:15 smithi089 ceph-mon[34481]: pgmap v651: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 94 KiB/s wr, 30 op/s 2024-09-21T00:30:16.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:15 smithi098 ceph-mon[34725]: pgmap v651: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 980 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 94 KiB/s wr, 30 op/s 2024-09-21T00:30:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:17 smithi027 ceph-mon[30745]: pgmap v652: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 976 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 10 op/s 2024-09-21T00:30:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:17 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:30:18.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:17 smithi089 ceph-mon[34481]: pgmap v652: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 976 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 10 op/s 2024-09-21T00:30:18.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:17 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:30:18.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:17 smithi098 ceph-mon[34725]: pgmap v652: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 976 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 10 op/s 2024-09-21T00:30:18.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:17 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:30:20.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:19 smithi027 ceph-mon[30745]: pgmap v653: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 976 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 10 op/s 2024-09-21T00:30:20.070 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:19 smithi098 ceph-mon[34725]: pgmap v653: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 976 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 10 op/s 2024-09-21T00:30:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:19 smithi089 ceph-mon[34481]: pgmap v653: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 976 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 10 op/s 2024-09-21T00:30:22.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:21 smithi027 ceph-mon[30745]: pgmap v654: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 976 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-09-21T00:30:22.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:21 smithi089 ceph-mon[34481]: pgmap v654: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 976 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-09-21T00:30:22.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:21 smithi098 ceph-mon[34725]: pgmap v654: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 976 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-09-21T00:30:24.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:23 smithi027 ceph-mon[30745]: pgmap v655: 129 pgs: 129 active+clean; 975 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-09-21T00:30:24.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:23 smithi089 ceph-mon[34481]: pgmap v655: 129 pgs: 129 active+clean; 975 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-09-21T00:30:24.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:23 smithi098 ceph-mon[34725]: pgmap v655: 129 pgs: 129 active+clean; 975 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-09-21T00:30:26.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:25 smithi089 ceph-mon[34481]: pgmap v656: 129 pgs: 129 active+clean; 975 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-09-21T00:30:26.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:25 smithi098 ceph-mon[34725]: pgmap v656: 129 pgs: 129 active+clean; 975 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-09-21T00:30:26.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:25 smithi027 ceph-mon[30745]: pgmap v656: 129 pgs: 129 active+clean; 975 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-09-21T00:30:27.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.283+0000 7f3c6fe00640 1 -- 172.21.15.27:0/1207297840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c7010be60 msgr2=0x7f3c70112690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:27.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.283+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/1207297840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c7010be60 0x7f3c70112690 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009ca0 tx=0x7f3c6802a8b0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.283+0000 7f3c6fe00640 1 -- 172.21.15.27:0/1207297840 shutdown_connections 2024-09-21T00:30:27.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.283+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/1207297840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c7010be60 0x7f3c70112690 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.288+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/1207297840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c7010b510 0x7f3c7010b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.288+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/1207297840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c70105bd0 0x7f3c70105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.288+0000 7f3c6fe00640 1 -- 172.21.15.27:0/1207297840 >> 172.21.15.27:0/1207297840 conn(0x7f3c70101620 msgr2=0x7f3c70103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:27.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.288+0000 7f3c6fe00640 1 -- 172.21.15.27:0/1207297840 shutdown_connections 2024-09-21T00:30:27.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.288+0000 7f3c6fe00640 1 -- 172.21.15.27:0/1207297840 wait complete. 2024-09-21T00:30:27.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6fe00640 1 Processor -- start 2024-09-21T00:30:27.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6fe00640 1 -- start start 2024-09-21T00:30:27.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c70105bd0 0x7f3c7010fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c7010b510 0x7f3c7010fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c7010be60 0x7f3c70110530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70115b70 con 0x7f3c7010b510 2024-09-21T00:30:27.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6fe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3c701159f0 con 0x7f3c70105bd0 2024-09-21T00:30:27.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6fe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70115cf0 con 0x7f3c7010be60 2024-09-21T00:30:27.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c7010b510 0x7f3c7010fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c7010b510 0x7f3c7010fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38532/0 (socket says 172.21.15.27:38532) 2024-09-21T00:30:27.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6f400640 1 --2- 172.21.15.27:0/2291169919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c7010be60 0x7f3c70110530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6ea00640 1 --2- 172.21.15.27:0/2291169919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c70105bd0 0x7f3c7010fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6e000640 1 -- 172.21.15.27:0/2291169919 learned_addr learned my addr 172.21.15.27:0/2291169919 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:27.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6f400640 1 -- 172.21.15.27:0/2291169919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c70105bd0 msgr2=0x7f3c7010fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:27.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6f400640 1 --2- 172.21.15.27:0/2291169919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c70105bd0 0x7f3c7010fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6f400640 1 -- 172.21.15.27:0/2291169919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c7010b510 msgr2=0x7f3c7010fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:27.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6f400640 1 --2- 172.21.15.27:0/2291169919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c7010b510 0x7f3c7010fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6f400640 1 -- 172.21.15.27:0/2291169919 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c701a2170 con 0x7f3c7010be60 2024-09-21T00:30:27.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6f400640 1 --2- 172.21.15.27:0/2291169919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c7010be60 0x7f3c70110530 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3c6802add0 tx=0x7f3c68030da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:27.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c67400640 1 -- 172.21.15.27:0/2291169919 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68004040 con 0x7f3c7010be60 2024-09-21T00:30:27.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c67400640 1 -- 172.21.15.27:0/2291169919 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c68040070 con 0x7f3c7010be60 2024-09-21T00:30:27.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6fe00640 1 -- 172.21.15.27:0/2291169919 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c701a2400 con 0x7f3c7010be60 2024-09-21T00:30:27.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c6fe00640 1 -- 172.21.15.27:0/2291169919 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c701a2990 con 0x7f3c7010be60 2024-09-21T00:30:27.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.290+0000 7f3c67400640 1 -- 172.21.15.27:0/2291169919 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68045b20 con 0x7f3c7010be60 2024-09-21T00:30:27.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.295+0000 7f3c6fe00640 1 -- 172.21.15.27:0/2291169919 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c70105fe0 con 0x7f3c7010be60 2024-09-21T00:30:27.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.295+0000 7f3c67400640 1 -- 172.21.15.27:0/2291169919 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3c680041e0 con 0x7f3c7010be60 2024-09-21T00:30:27.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.295+0000 7f3c67400640 1 --2- 172.21.15.27:0/2291169919 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3c540b6700 0x7f3c540b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.300+0000 7f3c67400640 1 -- 172.21.15.27:0/2291169919 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f3c68108fa0 con 0x7f3c7010be60 2024-09-21T00:30:27.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.300+0000 7f3c6ea00640 1 --2- 172.21.15.27:0/2291169919 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3c540b6700 0x7f3c540b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.300+0000 7f3c6ea00640 1 --2- 172.21.15.27:0/2291169919 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3c540b6700 0x7f3c540b8c10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3c60005fd0 tx=0x7f3c60005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:27.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.302+0000 7f3c67400640 1 -- 172.21.15.27:0/2291169919 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3c680cf0b0 con 0x7f3c7010be60 2024-09-21T00:30:27.481 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:27 smithi027 ceph-mon[30745]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T00:30:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.520+0000 7fda81a00640 1 -- 172.21.15.27:0/341668333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda7c10b470 msgr2=0x7fda7c10b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.520+0000 7fda81a00640 1 --2- 172.21.15.27:0/341668333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda7c10b470 0x7fda7c10b880 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fda68009860 tx=0x7fda6802a340 comp rx=0 tx=0).stop 2024-09-21T00:30:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.520+0000 7fda81a00640 1 -- 172.21.15.27:0/341668333 shutdown_connections 2024-09-21T00:30:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.520+0000 7fda81a00640 1 --2- 172.21.15.27:0/341668333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda7c10bdc0 0x7fda7c1125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.520+0000 7fda81a00640 1 --2- 172.21.15.27:0/341668333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda7c10b470 0x7fda7c10b880 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.520+0000 7fda81a00640 1 --2- 172.21.15.27:0/341668333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda7c105b40 0x7fda7c105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.520+0000 7fda81a00640 1 -- 172.21.15.27:0/341668333 >> 172.21.15.27:0/341668333 conn(0x7fda7c101560 msgr2=0x7fda7c1039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.520+0000 7fda81a00640 1 -- 172.21.15.27:0/341668333 shutdown_connections 2024-09-21T00:30:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.525+0000 7fda81a00640 1 -- 172.21.15.27:0/341668333 wait complete. 2024-09-21T00:30:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81a00640 1 Processor -- start 2024-09-21T00:30:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81a00640 1 -- start start 2024-09-21T00:30:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda7c105b40 0x7fda7c10fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda7c10b470 0x7fda7c10ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda7c10bdc0 0x7fda7c110510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c115a80 con 0x7fda7c10b470 2024-09-21T00:30:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda7b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda7c10b470 0x7fda7c10ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c115900 con 0x7fda7c10bdc0 2024-09-21T00:30:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c115c00 con 0x7fda7c105b40 2024-09-21T00:30:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda7b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda7c10b470 0x7fda7c10ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38554/0 (socket says 172.21.15.27:38554) 2024-09-21T00:30:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda7b400640 1 -- 172.21.15.27:0/85592274 learned_addr learned my addr 172.21.15.27:0/85592274 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda7c10bdc0 0x7fda7c110510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81000640 1 -- 172.21.15.27:0/85592274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda7c105b40 msgr2=0x7fda7c10fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:30:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81000640 1 --2- 172.21.15.27:0/85592274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda7c105b40 0x7fda7c10fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81000640 1 -- 172.21.15.27:0/85592274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda7c10b470 msgr2=0x7fda7c10ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81000640 1 --2- 172.21.15.27:0/85592274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda7c10b470 0x7fda7c10ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81000640 1 -- 172.21.15.27:0/85592274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7c1ac910 con 0x7fda7c10bdc0 2024-09-21T00:30:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81000640 1 --2- 172.21.15.27:0/85592274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda7c10bdc0 0x7fda7c110510 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fda7400ea80 tx=0x7fda7400ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda78c00640 1 -- 172.21.15.27:0/85592274 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7400cea0 con 0x7fda7c10bdc0 2024-09-21T00:30:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81a00640 1 -- 172.21.15.27:0/85592274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7c1acc00 con 0x7fda7c10bdc0 2024-09-21T00:30:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda78c00640 1 -- 172.21.15.27:0/85592274 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda74005020 con 0x7fda7c10bdc0 2024-09-21T00:30:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda78c00640 1 -- 172.21.15.27:0/85592274 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74010690 con 0x7fda7c10bdc0 2024-09-21T00:30:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.527+0000 7fda81a00640 1 -- 172.21.15.27:0/85592274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7c1ad120 con 0x7fda7c10bdc0 2024-09-21T00:30:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.532+0000 7fda8a1be740 1 Processor -- start 2024-09-21T00:30:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.532+0000 7fda78c00640 1 -- 172.21.15.27:0/85592274 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fda740054d0 con 0x7fda7c10bdc0 2024-09-21T00:30:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.532+0000 7fda8a1be740 1 -- start start 2024-09-21T00:30:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.532+0000 7fda8a1be740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 0x562991d70e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.532+0000 7fda8a1be740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d71350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.532+0000 7fda8a1be740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919ab4d0 0x562991d71890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.532+0000 7fda8a1be740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562991ce66b0 con 0x5629919cc1b0 2024-09-21T00:30:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.532+0000 7fda8a1be740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562991cd5ae0 con 0x5629919ab4d0 2024-09-21T00:30:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.532+0000 7fda8a1be740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562991cd98f0 con 0x5629919a32f0 2024-09-21T00:30:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.537+0000 7fda7b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 0x562991d70e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.537+0000 7fda7be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d71350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.537+0000 7fda7b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 0x562991d70e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38570/0 (socket says 172.21.15.27:38570) 2024-09-21T00:30:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.537+0000 7fda7b400640 1 -- 172.21.15.27:0/3153606441 learned_addr learned my addr 172.21.15.27:0/3153606441 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.537+0000 7fda7b400640 1 -- 172.21.15.27:0/3153606441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 msgr2=0x562991d71350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.537+0000 7fda7b400640 1 --2- 172.21.15.27:0/3153606441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d71350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.537+0000 7fda7b400640 1 -- 172.21.15.27:0/3153606441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919ab4d0 msgr2=0x562991d71890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:30:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.537+0000 7fda7b400640 1 --2- 172.21.15.27:0/3153606441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919ab4d0 0x562991d71890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.537+0000 7fda7b400640 1 -- 172.21.15.27:0/3153606441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562991ca2780 con 0x5629919cc1b0 2024-09-21T00:30:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.537+0000 7fda7be00640 1 --2- 172.21.15.27:0/3153606441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d71350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:30:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda7b400640 1 --2- 172.21.15.27:0/3153606441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 0x562991d70e10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fda68035040 tx=0x7fda68009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd5cdc6be1aafc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda78c00640 1 --2- 172.21.15.27:0/85592274 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda600b66f0 0x7fda600b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda71600640 1 -- 172.21.15.27:0/3153606441 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68037040 con 0x5629919cc1b0 2024-09-21T00:30:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda71600640 1 -- 172.21.15.27:0/3153606441 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda68030d50 con 0x5629919cc1b0 2024-09-21T00:30:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda71600640 1 -- 172.21.15.27:0/3153606441 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68037040 con 0x5629919cc1b0 2024-09-21T00:30:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda78c00640 1 -- 172.21.15.27:0/85592274 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7fda740e8280 con 0x7fda7c10bdc0 2024-09-21T00:30:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda8a1be740 1 -- 172.21.15.27:0/3153606441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 msgr2=0x562991d70e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda8a1be740 1 --2- 172.21.15.27:0/3153606441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 0x562991d70e10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fda68035040 tx=0x7fda68009580 comp rx=0 tx=0).stop 2024-09-21T00:30:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda7be00640 1 --2- 172.21.15.27:0/85592274 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda600b66f0 0x7fda600b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda8a1be740 1 -- 172.21.15.27:0/3153606441 shutdown_connections 2024-09-21T00:30:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda8a1be740 1 --2- 172.21.15.27:0/3153606441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919ab4d0 0x562991d71890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda8a1be740 1 --2- 172.21.15.27:0/3153606441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d71350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda8a1be740 1 --2- 172.21.15.27:0/3153606441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 0x562991d70e10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda8a1be740 1 -- 172.21.15.27:0/3153606441 >> 172.21.15.27:0/3153606441 conn(0x5629919c0300 msgr2=0x562991d6ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda8a1be740 1 -- 172.21.15.27:0/3153606441 shutdown_connections 2024-09-21T00:30:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda7be00640 1 --2- 172.21.15.27:0/85592274 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda600b66f0 0x7fda600b8c00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x562991d6fa90 tx=0x7fda6c009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda8a1be740 1 -- 172.21.15.27:0/3153606441 wait complete. 2024-09-21T00:30:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda8a1be740 1 -- start start 2024-09-21T00:30:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.538+0000 7fda8a1be740 1 Processor -- start 2024-09-21T00:30:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d7c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919ab4d0 0x562991d7c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 0x562991d7ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda7be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d7c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda7be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d7c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:34874/0 (socket says 172.21.15.27:34874) 2024-09-21T00:30:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda7be00640 1 -- 172.21.15.27:0/2946527572 learned_addr learned my addr 172.21.15.27:0/2946527572 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 1 -- 172.21.15.27:0/2946527572 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562991c0dab0 con 0x5629919cc1b0 2024-09-21T00:30:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 1 -- 172.21.15.27:0/2946527572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562991c7c570 con 0x5629919ab4d0 2024-09-21T00:30:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 1 -- 172.21.15.27:0/2946527572 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562991d90540 con 0x5629919a32f0 2024-09-21T00:30:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 10 client.0 _ll_register_callbacks cb 0x562991c8b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda81000640 1 --2- 172.21.15.27:0/2946527572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 0x562991d7ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda7be00640 1 -- 172.21.15.27:0/2946527572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919ab4d0 msgr2=0x562991d7c8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:30:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda7be00640 1 --2- 172.21.15.27:0/2946527572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919ab4d0 0x562991d7c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda7be00640 1 -- 172.21.15.27:0/2946527572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 msgr2=0x562991d7ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda7be00640 1 --2- 172.21.15.27:0/2946527572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 0x562991d7ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda7be00640 1 -- 172.21.15.27:0/2946527572 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562991a37790 con 0x5629919a32f0 2024-09-21T00:30:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda7be00640 1 --2- 172.21.15.27:0/2946527572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d7c3a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fda6c004620 tx=0x7fda6c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a542f06c8663bb92 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda66000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:30:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda66000640 1 -- 172.21.15.27:0/2946527572 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c01a870 con 0x5629919a32f0 2024-09-21T00:30:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda66000640 1 -- 172.21.15.27:0/2946527572 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7fda6c01aa10 con 0x5629919a32f0 2024-09-21T00:30:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 1 -- 172.21.15.27:0/2946527572 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda68030d50 con 0x5629919a32f0 2024-09-21T00:30:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda66000640 1 -- 172.21.15.27:0/2946527572 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fda6c01ac10 con 0x5629919a32f0 2024-09-21T00:30:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 10 client.26359 fetch_fsmap learned FSMap version 38 2024-09-21T00:30:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 1 -- 172.21.15.27:0/2946527572 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562991ab0620 con 0x5629919a32f0 2024-09-21T00:30:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda66000640 1 -- 172.21.15.27:0/2946527572 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda68030d50 con 0x5629919a32f0 2024-09-21T00:30:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda66000640 1 -- 172.21.15.27:0/2946527572 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fda6c023070 con 0x5629919a32f0 2024-09-21T00:30:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 10 client.26359 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:30:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 10 client.26359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:30:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 20 client.26359 populate_metadata read hostname 'smithi027' 2024-09-21T00:30:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 1 --2- 172.21.15.27:0/2946527572 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56299198ad80 0x562991d9f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 4 client.26359 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:30:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda8a1be740 1 -- 172.21.15.27:0/2946527572 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562991ce8600 con 0x56299198ad80 2024-09-21T00:30:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.544+0000 7fda7b400640 1 --2- 172.21.15.27:0/2946527572 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56299198ad80 0x562991d9f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.549+0000 7fda7b400640 1 --2- 172.21.15.27:0/2946527572 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56299198ad80 0x562991d9f5f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56e3b30b2271c1e7 server_cookie=e43bda941a2aa707 in_seq=0 out_seq=0 2024-09-21T00:30:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.549+0000 7fda66000640 10 client.26359 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.550+0000 7fda66000640 1 -- 172.21.15.27:0/2946527572 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562991ce8600 con 0x56299198ad80 2024-09-21T00:30:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.550+0000 7fda66000640 10 client.26359 handle_command_reply: tid=0 2024-09-21T00:30:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.550+0000 7fda8a1be740 1 client.26359 shutdown 2024-09-21T00:30:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.550+0000 7fda8a1be740 1 -- 172.21.15.27:0/2946527572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 msgr2=0x562991d7c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.550+0000 7fda8a1be740 1 --2- 172.21.15.27:0/2946527572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d7c3a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fda6c004620 tx=0x7fda6c004730 comp rx=0 tx=0).stop 2024-09-21T00:30:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.550+0000 7fda8a1be740 1 -- 172.21.15.27:0/2946527572 shutdown_connections 2024-09-21T00:30:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.550+0000 7fda8a1be740 1 --2- 172.21.15.27:0/2946527572 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56299198ad80 0x562991d9f5f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.550+0000 7fda8a1be740 1 --2- 172.21.15.27:0/2946527572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 0x562991d7ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.550+0000 7fda8a1be740 1 --2- 172.21.15.27:0/2946527572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919ab4d0 0x562991d7c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.550+0000 7fda8a1be740 1 --2- 172.21.15.27:0/2946527572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d7c3a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.550+0000 7fda66000640 0 client.26359 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.550+0000 7fda8a1be740 1 -- 172.21.15.27:0/2946527572 >> 172.21.15.27:0/2946527572 conn(0x5629919c0300 msgr2=0x562991d71e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.550+0000 7fda8a1be740 1 -- 172.21.15.27:0/2946527572 shutdown_connections 2024-09-21T00:30:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.550+0000 7fda8a1be740 1 -- 172.21.15.27:0/2946527572 wait complete. 2024-09-21T00:30:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.550+0000 7fda8a1be740 20 client.26359 trim_cache size 0 max 16384 2024-09-21T00:30:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 Processor -- start 2024-09-21T00:30:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 -- start start 2024-09-21T00:30:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d7dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919ab4d0 0x562991d79570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919cc1b0 0x562991d79ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c1b13f0 con 0x5629919ab4d0 2024-09-21T00:30:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c1aec60 con 0x5629919cc1b0 2024-09-21T00:30:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c1a2e40 con 0x5629919a32f0 2024-09-21T00:30:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda7be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d7dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda81000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919cc1b0 0x562991d79ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda7be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d7dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:34882/0 (socket says 172.21.15.27:34882) 2024-09-21T00:30:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda7be00640 1 -- 172.21.15.27:0/280875828 learned_addr learned my addr 172.21.15.27:0/280875828 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda7b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919ab4d0 0x562991d79570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda7be00640 1 -- 172.21.15.27:0/280875828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919cc1b0 msgr2=0x562991d79ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda7be00640 1 --2- 172.21.15.27:0/280875828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919cc1b0 0x562991d79ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda7be00640 1 -- 172.21.15.27:0/280875828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919ab4d0 msgr2=0x562991d79570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda7be00640 1 --2- 172.21.15.27:0/280875828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919ab4d0 0x562991d79570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda7be00640 1 -- 172.21.15.27:0/280875828 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562991d77ee0 con 0x5629919a32f0 2024-09-21T00:30:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda7b400640 1 --2- 172.21.15.27:0/280875828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919ab4d0 0x562991d79570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:30:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda7be00640 1 --2- 172.21.15.27:0/280875828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d7dfd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009bc0 tx=0x7fda6c004690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31de2317162ba019 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda70c00640 1 -- 172.21.15.27:0/280875828 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c01a670 con 0x5629919a32f0 2024-09-21T00:30:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda70c00640 1 -- 172.21.15.27:0/280875828 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda6c01bc60 con 0x5629919a32f0 2024-09-21T00:30:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda70c00640 1 -- 172.21.15.27:0/280875828 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c0336a0 con 0x5629919a32f0 2024-09-21T00:30:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 -- 172.21.15.27:0/280875828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 msgr2=0x562991d7dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 --2- 172.21.15.27:0/280875828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d7dfd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009bc0 tx=0x7fda6c004690 comp rx=0 tx=0).stop 2024-09-21T00:30:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 -- 172.21.15.27:0/280875828 shutdown_connections 2024-09-21T00:30:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 --2- 172.21.15.27:0/280875828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919cc1b0 0x562991d79ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 --2- 172.21.15.27:0/280875828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919ab4d0 0x562991d79570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 --2- 172.21.15.27:0/280875828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919a32f0 0x562991d7dfd0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 -- 172.21.15.27:0/280875828 >> 172.21.15.27:0/280875828 conn(0x5629919c0300 msgr2=0x562991d7d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 -- 172.21.15.27:0/280875828 shutdown_connections 2024-09-21T00:30:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 -- 172.21.15.27:0/280875828 wait complete. 2024-09-21T00:30:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 -- start start 2024-09-21T00:30:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.579+0000 7fda8a1be740 1 Processor -- start 2024-09-21T00:30:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.584+0000 7fda8a1be740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919a32f0 0x562991d95d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.584+0000 7fda8a1be740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919ab4d0 0x562991d93290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.584+0000 7fda8a1be740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 0x562991d937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.584+0000 7fda8a1be740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c1a3490 con 0x5629919cc1b0 2024-09-21T00:30:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.584+0000 7fda8a1be740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c1ae590 con 0x5629919a32f0 2024-09-21T00:30:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.584+0000 7fda8a1be740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c1ad7f0 con 0x5629919ab4d0 2024-09-21T00:30:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.584+0000 7fda8a1be740 10 client.0 _ll_register_callbacks cb 0x7fda6c009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.584+0000 7fda7be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919a32f0 0x562991d95d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.584+0000 7fda7be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919a32f0 0x562991d95d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:44116/0 (socket says 172.21.15.27:44116) 2024-09-21T00:30:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.584+0000 7fda7be00640 1 -- 172.21.15.27:0/2150760912 learned_addr learned my addr 172.21.15.27:0/2150760912 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.584+0000 7fda7b400640 1 --2- 172.21.15.27:0/2150760912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919ab4d0 0x562991d93290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.584+0000 7fda81000640 1 --2- 172.21.15.27:0/2150760912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 0x562991d937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda7b400640 1 -- 172.21.15.27:0/2150760912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919a32f0 msgr2=0x562991d95d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda7b400640 1 --2- 172.21.15.27:0/2150760912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919a32f0 0x562991d95d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda7b400640 1 -- 172.21.15.27:0/2150760912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 msgr2=0x562991d937d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda7b400640 1 --2- 172.21.15.27:0/2150760912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 0x562991d937d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda7b400640 1 -- 172.21.15.27:0/2150760912 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562991d93d40 con 0x5629919ab4d0 2024-09-21T00:30:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda81000640 1 --2- 172.21.15.27:0/2150760912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 0x562991d937d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:30:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda7b400640 1 --2- 172.21.15.27:0/2150760912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919ab4d0 0x562991d93290 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fda68009830 tx=0x7fda68009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af288fb4634566f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda66000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:30:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda66000640 1 -- 172.21.15.27:0/2150760912 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68037040 con 0x5629919ab4d0 2024-09-21T00:30:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda66000640 1 -- 172.21.15.27:0/2150760912 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7fda680305d0 con 0x5629919ab4d0 2024-09-21T00:30:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda8a1be740 1 -- 172.21.15.27:0/2150760912 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda6c01bc60 con 0x5629919ab4d0 2024-09-21T00:30:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda66000640 1 -- 172.21.15.27:0/2150760912 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fda68030d50 con 0x5629919ab4d0 2024-09-21T00:30:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda8a1be740 10 client.26365 fetch_fsmap learned FSMap version 38 2024-09-21T00:30:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda8a1be740 1 -- 172.21.15.27:0/2150760912 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562991e09d30 con 0x5629919ab4d0 2024-09-21T00:30:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda66000640 1 -- 172.21.15.27:0/2150760912 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda6c01bc60 con 0x5629919ab4d0 2024-09-21T00:30:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda66000640 1 -- 172.21.15.27:0/2150760912 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fda68003960 con 0x5629919ab4d0 2024-09-21T00:30:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda8a1be740 10 client.26365 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:30:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda8a1be740 10 client.26365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:30:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda8a1be740 20 client.26365 populate_metadata read hostname 'smithi027' 2024-09-21T00:30:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda8a1be740 1 --2- 172.21.15.27:0/2150760912 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5629919601c0 0x562991e0bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda8a1be740 4 client.26365 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:30:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda8a1be740 1 -- 172.21.15.27:0/2150760912 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562991ce40b0 con 0x5629919601c0 2024-09-21T00:30:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda7be00640 1 --2- 172.21.15.27:0/2150760912 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5629919601c0 0x562991e0bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda66000640 10 client.26365 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.586+0000 7fda7be00640 1 --2- 172.21.15.27:0/2150760912 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5629919601c0 0x562991e0bf80 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ce91e1025c6882c server_cookie=adca53d2c563b1cd in_seq=0 out_seq=0 2024-09-21T00:30:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.598+0000 7fda66000640 1 -- 172.21.15.27:0/2150760912 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562991ce40b0 con 0x5629919601c0 2024-09-21T00:30:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.598+0000 7fda66000640 10 client.26365 handle_command_reply: tid=0 2024-09-21T00:30:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.598+0000 7fda8a1be740 1 client.26365 shutdown 2024-09-21T00:30:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.598+0000 7fda8a1be740 1 -- 172.21.15.27:0/2150760912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919ab4d0 msgr2=0x562991d93290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.598+0000 7fda8a1be740 1 --2- 172.21.15.27:0/2150760912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919ab4d0 0x562991d93290 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fda68009830 tx=0x7fda68009450 comp rx=0 tx=0).stop 2024-09-21T00:30:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.598+0000 7fda8a1be740 1 -- 172.21.15.27:0/2150760912 shutdown_connections 2024-09-21T00:30:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.598+0000 7fda8a1be740 1 --2- 172.21.15.27:0/2150760912 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5629919601c0 0x562991e0bf80 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.598+0000 7fda8a1be740 1 --2- 172.21.15.27:0/2150760912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629919cc1b0 0x562991d937d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.598+0000 7fda8a1be740 1 --2- 172.21.15.27:0/2150760912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629919ab4d0 0x562991d93290 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.598+0000 7fda8a1be740 1 --2- 172.21.15.27:0/2150760912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5629919a32f0 0x562991d95d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.598+0000 7fda8a1be740 1 -- 172.21.15.27:0/2150760912 >> 172.21.15.27:0/2150760912 conn(0x5629919c0300 msgr2=0x562991d6d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.598+0000 7fda66000640 0 client.26365 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.603+0000 7fda8a1be740 1 -- 172.21.15.27:0/2150760912 shutdown_connections 2024-09-21T00:30:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.603+0000 7fda8a1be740 1 -- 172.21.15.27:0/2150760912 wait complete. 2024-09-21T00:30:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.603+0000 7fda8a1be740 20 client.26365 trim_cache size 0 max 16384 2024-09-21T00:30:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:30:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:30:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:30:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:30:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.603+0000 7fda81a00640 1 -- 172.21.15.27:0/85592274 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda600b66f0 msgr2=0x7fda600b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.603+0000 7fda81a00640 1 --2- 172.21.15.27:0/85592274 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda600b66f0 0x7fda600b8c00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x562991d6fa90 tx=0x7fda6c009260 comp rx=0 tx=0).stop 2024-09-21T00:30:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.603+0000 7fda81a00640 1 -- 172.21.15.27:0/85592274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda7c10bdc0 msgr2=0x7fda7c110510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.603+0000 7fda81a00640 1 --2- 172.21.15.27:0/85592274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda7c10bdc0 0x7fda7c110510 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fda7400ea80 tx=0x7fda7400ef40 comp rx=0 tx=0).stop 2024-09-21T00:30:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.603+0000 7fda81a00640 1 -- 172.21.15.27:0/85592274 shutdown_connections 2024-09-21T00:30:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.603+0000 7fda81a00640 1 --2- 172.21.15.27:0/85592274 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fda600b66f0 0x7fda600b8c00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.603+0000 7fda81a00640 1 --2- 172.21.15.27:0/85592274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda7c10bdc0 0x7fda7c110510 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.603+0000 7fda81a00640 1 --2- 172.21.15.27:0/85592274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda7c10b470 0x7fda7c10ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.603+0000 7fda81a00640 1 --2- 172.21.15.27:0/85592274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda7c105b40 0x7fda7c10fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.603+0000 7fda81a00640 1 -- 172.21.15.27:0/85592274 >> 172.21.15.27:0/85592274 conn(0x7fda7c101560 msgr2=0x7fda7c103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.603+0000 7fda81a00640 1 -- 172.21.15.27:0/85592274 shutdown_connections 2024-09-21T00:30:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:27.603+0000 7fda81a00640 1 -- 172.21.15.27:0/85592274 wait complete. 2024-09-21T00:30:27.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.662+0000 7f3c6fe00640 1 -- 172.21.15.27:0/2291169919 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c701a3460 con 0x7f3c7010be60 2024-09-21T00:30:27.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.667+0000 7f3c67400640 1 -- 172.21.15.27:0/2291169919 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11896 (secure 0 0 0) 0x7f3c680d3490 con 0x7f3c7010be60 2024-09-21T00:30:27.668 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:30:27.669 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"btime":"2024-09-21T00:29:52:686376+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:29:52.321069+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26057":{"gid":26057,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/4028763758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":4028763758},{"type":"v1","addr":"172.21.15.89:6837","nonce":4028763758}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24727,24751,24761]},"id":1}]} 2024-09-21T00:30:27.669 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 38 2024-09-21T00:30:27.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.669+0000 7f3c6fe00640 1 -- 172.21.15.27:0/2291169919 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3c540b6700 msgr2=0x7f3c540b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:27.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.669+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/2291169919 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3c540b6700 0x7f3c540b8c10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3c60005fd0 tx=0x7f3c60005850 comp rx=0 tx=0).stop 2024-09-21T00:30:27.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.669+0000 7f3c6fe00640 1 -- 172.21.15.27:0/2291169919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c7010be60 msgr2=0x7f3c70110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:27.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.669+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/2291169919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c7010be60 0x7f3c70110530 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3c6802add0 tx=0x7f3c68030da0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.669+0000 7f3c6fe00640 1 -- 172.21.15.27:0/2291169919 shutdown_connections 2024-09-21T00:30:27.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.669+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/2291169919 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3c540b6700 0x7f3c540b8c10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.669+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/2291169919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c7010be60 0x7f3c70110530 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.669+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/2291169919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c7010b510 0x7f3c7010fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.669+0000 7f3c6fe00640 1 --2- 172.21.15.27:0/2291169919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c70105bd0 0x7f3c7010fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.669+0000 7f3c6fe00640 1 -- 172.21.15.27:0/2291169919 >> 172.21.15.27:0/2291169919 conn(0x7f3c70101620 msgr2=0x7f3c70103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:27.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.669+0000 7f3c6fe00640 1 -- 172.21.15.27:0/2291169919 shutdown_connections 2024-09-21T00:30:27.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:27.669+0000 7f3c6fe00640 1 -- 172.21.15.27:0/2291169919 wait complete. 2024-09-21T00:30:27.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:27 smithi089 ceph-mon[34481]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T00:30:27.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:27 smithi098 ceph-mon[34725]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T00:30:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:28 smithi027 ceph-mon[30745]: pgmap v658: 129 pgs: 129 active+clean; 975 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 7 op/s 2024-09-21T00:30:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:28 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:30:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:28 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:30:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:28 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:30:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:28 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2291169919' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:30:28.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:28 smithi089 ceph-mon[34481]: pgmap v658: 129 pgs: 129 active+clean; 975 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 7 op/s 2024-09-21T00:30:28.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:28 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:30:28.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:28 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:30:28.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:28 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:30:28.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:28 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2291169919' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:30:28.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:28 smithi098 ceph-mon[34725]: pgmap v658: 129 pgs: 129 active+clean; 975 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 7 op/s 2024-09-21T00:30:28.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:28 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:30:28.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:28 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:30:28.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:28 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:30:28.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:28 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2291169919' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:30:28.776 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id 47099ce0 --await-for 45.0' 2024-09-21T00:30:28.776 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id 47099ce0 --await-for 45.0 2024-09-21T00:30:29.242 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:30:30.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.260+0000 7f200ac00640 1 -- 172.21.15.27:0/2874418146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2004105110 msgr2=0x7f200410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.260+0000 7f200ac00640 1 --2- 172.21.15.27:0/2874418146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2004105110 0x7f200410d660 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009900 tx=0x7f1ff002a3f0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.260+0000 7f200ac00640 1 -- 172.21.15.27:0/2874418146 shutdown_connections 2024-09-21T00:30:30.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.260+0000 7f200ac00640 1 --2- 172.21.15.27:0/2874418146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f200410dc20 0x7f2004110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.260+0000 7f200ac00640 1 --2- 172.21.15.27:0/2874418146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2004105110 0x7f200410d660 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.260+0000 7f200ac00640 1 --2- 172.21.15.27:0/2874418146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20041047c0 0x7f2004104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.260+0000 7f200ac00640 1 -- 172.21.15.27:0/2874418146 >> 172.21.15.27:0/2874418146 conn(0x7f20040fe2c0 msgr2=0x7f2004100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:30.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.260+0000 7f200ac00640 1 -- 172.21.15.27:0/2874418146 shutdown_connections 2024-09-21T00:30:30.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.260+0000 7f200ac00640 1 -- 172.21.15.27:0/2874418146 wait complete. 2024-09-21T00:30:30.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.260+0000 7f200ac00640 1 Processor -- start 2024-09-21T00:30:30.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200ac00640 1 -- start start 2024-09-21T00:30:30.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200ac00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20041047c0 0x7f200419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200ac00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2004105110 0x7f200419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200ac00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f200410dc20 0x7f20041a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200ac00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2004113880 con 0x7f20041047c0 2024-09-21T00:30:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200ac00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2004113700 con 0x7f200410dc20 2024-09-21T00:30:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200ac00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2004113a00 con 0x7f2004105110 2024-09-21T00:30:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f2009800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20041047c0 0x7f200419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f2009800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20041047c0 0x7f200419c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38618/0 (socket says 172.21.15.27:38618) 2024-09-21T00:30:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f2009800640 1 -- 172.21.15.27:0/2606030634 learned_addr learned my addr 172.21.15.27:0/2606030634 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f2008e00640 1 --2- 172.21.15.27:0/2606030634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2004105110 0x7f200419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200a200640 1 --2- 172.21.15.27:0/2606030634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f200410dc20 0x7f20041a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200a200640 1 -- 172.21.15.27:0/2606030634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2004105110 msgr2=0x7f200419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200a200640 1 --2- 172.21.15.27:0/2606030634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2004105110 0x7f200419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200a200640 1 -- 172.21.15.27:0/2606030634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20041047c0 msgr2=0x7f200419c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200a200640 1 --2- 172.21.15.27:0/2606030634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20041047c0 0x7f200419c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200a200640 1 -- 172.21.15.27:0/2606030634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20041a2440 con 0x7f200410dc20 2024-09-21T00:30:30.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f2008e00640 1 --2- 172.21.15.27:0/2606030634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2004105110 0x7f200419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:30:30.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200a200640 1 --2- 172.21.15.27:0/2606030634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f200410dc20 0x7f20041a1d10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009fd0 tx=0x7f1ffc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:30.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f2002000640 1 -- 172.21.15.27:0/2606030634 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc00ce20 con 0x7f200410dc20 2024-09-21T00:30:30.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200ac00640 1 -- 172.21.15.27:0/2606030634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20041a2730 con 0x7f200410dc20 2024-09-21T00:30:30.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f2002000640 1 -- 172.21.15.27:0/2606030634 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ffc005020 con 0x7f200410dc20 2024-09-21T00:30:30.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f2002000640 1 -- 172.21.15.27:0/2606030634 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc010690 con 0x7f200410dc20 2024-09-21T00:30:30.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.265+0000 7f200ac00640 1 -- 172.21.15.27:0/2606030634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20041a8e90 con 0x7f200410dc20 2024-09-21T00:30:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.271+0000 7f2002000640 1 -- 172.21.15.27:0/2606030634 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1ffc0054d0 con 0x7f200410dc20 2024-09-21T00:30:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 Processor -- start 2024-09-21T00:30:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 -- start start 2024-09-21T00:30:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f9bb520 0x55ec2fd61800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f992660 0x55ec2fd61d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f99a840 0x55ec2fd62280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ec2fd048d0 con 0x55ec2f99a840 2024-09-21T00:30:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ec2fcd90c0 con 0x55ec2f992660 2024-09-21T00:30:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ec2fcc84f0 con 0x55ec2f9bb520 2024-09-21T00:30:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2002000640 1 --2- 172.21.15.27:0/2606030634 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1fec0b6680 0x7f1fec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2009800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f992660 0x55ec2fd61d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2009800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f992660 0x55ec2fd61d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:44142/0 (socket says 172.21.15.27:44142) 2024-09-21T00:30:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2009800640 1 -- 172.21.15.27:0/2622123877 learned_addr learned my addr 172.21.15.27:0/2622123877 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2008e00640 1 --2- 172.21.15.27:0/2622123877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f9bb520 0x55ec2fd61800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2002000640 1 -- 172.21.15.27:0/2606030634 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f1ffc0e7e80 con 0x7f200410dc20 2024-09-21T00:30:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2009800640 1 -- 172.21.15.27:0/2622123877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f9bb520 msgr2=0x55ec2fd61800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2009800640 1 --2- 172.21.15.27:0/2622123877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f9bb520 0x55ec2fd61800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2009800640 1 -- 172.21.15.27:0/2622123877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f99a840 msgr2=0x55ec2fd62280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2009800640 1 --2- 172.21.15.27:0/2622123877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f99a840 0x55ec2fd62280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2009800640 1 -- 172.21.15.27:0/2622123877 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec2fa26b00 con 0x55ec2f992660 2024-09-21T00:30:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2008e00640 1 --2- 172.21.15.27:0/2622123877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f9bb520 0x55ec2fd61800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:30:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2009800640 1 --2- 172.21.15.27:0/2622123877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f992660 0x55ec2fd61d40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009e10 tx=0x7f1ff8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59948e5169c63a8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f200a200640 1 --2- 172.21.15.27:0/2606030634 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1fec0b6680 0x7f1fec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f1ff6a00640 1 -- 172.21.15.27:0/2622123877 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff801e070 con 0x55ec2f992660 2024-09-21T00:30:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f1ff6a00640 1 -- 172.21.15.27:0/2622123877 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff8019ae0 con 0x55ec2f992660 2024-09-21T00:30:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f1ff6a00640 1 -- 172.21.15.27:0/2622123877 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800fb00 con 0x55ec2f992660 2024-09-21T00:30:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 -- 172.21.15.27:0/2622123877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f992660 msgr2=0x55ec2fd61d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f200a200640 1 --2- 172.21.15.27:0/2606030634 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1fec0b6680 0x7f1fec0b8b90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc08bf10 tx=0x7f1ffc0057c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 --2- 172.21.15.27:0/2622123877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f992660 0x55ec2fd61d40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009e10 tx=0x7f1ff8011810 comp rx=0 tx=0).stop 2024-09-21T00:30:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 -- 172.21.15.27:0/2622123877 shutdown_connections 2024-09-21T00:30:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 --2- 172.21.15.27:0/2622123877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f99a840 0x55ec2fd62280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 --2- 172.21.15.27:0/2622123877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f992660 0x55ec2fd61d40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 --2- 172.21.15.27:0/2622123877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f9bb520 0x55ec2fd61800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 -- 172.21.15.27:0/2622123877 >> 172.21.15.27:0/2622123877 conn(0x55ec2f9af910 msgr2=0x55ec2fd5f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 -- 172.21.15.27:0/2622123877 shutdown_connections 2024-09-21T00:30:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 -- 172.21.15.27:0/2622123877 wait complete. 2024-09-21T00:30:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.272+0000 7f2012e90740 1 -- start start 2024-09-21T00:30:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2012e90740 1 Processor -- start 2024-09-21T00:30:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2012e90740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f992660 0x55ec2fd6d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2012e90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f99a840 0x55ec2fd6d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2012e90740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f9bb520 0x55ec2fd79180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2012e90740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ec2fc63500 con 0x55ec2f992660 2024-09-21T00:30:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2012e90740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ec2fccc300 con 0x55ec2f9bb520 2024-09-21T00:30:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2012e90740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ec2fb29bb0 con 0x55ec2f99a840 2024-09-21T00:30:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2012e90740 10 client.0 _ll_register_callbacks cb 0x55ec2f9d2d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2008e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f992660 0x55ec2fd6d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2008e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f992660 0x55ec2fd6d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38636/0 (socket says 172.21.15.27:38636) 2024-09-21T00:30:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2008e00640 1 -- 172.21.15.27:0/3950594042 learned_addr learned my addr 172.21.15.27:0/3950594042 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2009800640 1 --2- 172.21.15.27:0/3950594042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f99a840 0x55ec2fd6d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2008e00640 1 --2- 172.21.15.27:0/3950594042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f9bb520 0x55ec2fd79180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2009800640 1 -- 172.21.15.27:0/3950594042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f9bb520 msgr2=0x55ec2fd79180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2009800640 1 --2- 172.21.15.27:0/3950594042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f9bb520 0x55ec2fd79180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2009800640 1 -- 172.21.15.27:0/3950594042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f992660 msgr2=0x55ec2fd6d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2009800640 1 --2- 172.21.15.27:0/3950594042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f992660 0x55ec2fd6d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2009800640 1 -- 172.21.15.27:0/3950594042 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec2fd796c0 con 0x55ec2f99a840 2024-09-21T00:30:30.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.277+0000 7f2008e00640 1 --2- 172.21.15.27:0/3950594042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f9bb520 0x55ec2fd79180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:30.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f2009800640 1 --2- 172.21.15.27:0/3950594042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f99a840 0x55ec2fd6d880 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55ec2fd60480 tx=0x7f1ff8009aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95a102c424f77dae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:30.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f1feb400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:30:30.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f1feb400640 1 -- 172.21.15.27:0/3950594042 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8026020 con 0x55ec2f99a840 2024-09-21T00:30:30.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f2012e90740 1 -- 172.21.15.27:0/3950594042 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff8019ae0 con 0x55ec2f99a840 2024-09-21T00:30:30.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f1feb400640 1 -- 172.21.15.27:0/3950594042 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f1ff8004440 con 0x55ec2f99a840 2024-09-21T00:30:30.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f1feb400640 1 -- 172.21.15.27:0/3950594042 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1ff801e040 con 0x55ec2f99a840 2024-09-21T00:30:30.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f2012e90740 10 client.26374 fetch_fsmap learned FSMap version 38 2024-09-21T00:30:30.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f2012e90740 1 -- 172.21.15.27:0/3950594042 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ec2fd7bf60 con 0x55ec2f99a840 2024-09-21T00:30:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f1feb400640 1 -- 172.21.15.27:0/3950594042 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff8019ae0 con 0x55ec2f99a840 2024-09-21T00:30:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f1feb400640 1 -- 172.21.15.27:0/3950594042 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f1ff800f880 con 0x55ec2f99a840 2024-09-21T00:30:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f2012e90740 10 client.26374 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:30:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f2012e90740 10 client.26374 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:30:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f2012e90740 20 client.26374 populate_metadata read hostname 'smithi027' 2024-09-21T00:30:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f2012e90740 1 --2- 172.21.15.27:0/3950594042 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ec2f979ea0 0x55ec2fd7f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f2012e90740 4 client.26374 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:30:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f2012e90740 1 -- 172.21.15.27:0/3950594042 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec2fcdb010 con 0x55ec2f979ea0 2024-09-21T00:30:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f2008e00640 1 --2- 172.21.15.27:0/3950594042 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ec2f979ea0 0x55ec2fd7f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f2008e00640 1 --2- 172.21.15.27:0/3950594042 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ec2f979ea0 0x55ec2fd7f2c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e0d3a3af49f6688 server_cookie=227bcfa5d30c5d02 in_seq=0 out_seq=0 2024-09-21T00:30:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.289+0000 7f1feb400640 10 client.26374 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.294+0000 7f1feb400640 1 -- 172.21.15.27:0/3950594042 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ec2fcdb010 con 0x55ec2f979ea0 2024-09-21T00:30:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.294+0000 7f1feb400640 10 client.26374 handle_command_reply: tid=0 2024-09-21T00:30:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.294+0000 7f2012e90740 1 client.26374 shutdown 2024-09-21T00:30:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.294+0000 7f2012e90740 1 -- 172.21.15.27:0/3950594042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f99a840 msgr2=0x55ec2fd6d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.294+0000 7f2012e90740 1 --2- 172.21.15.27:0/3950594042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f99a840 0x55ec2fd6d880 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55ec2fd60480 tx=0x7f1ff8009aa0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.294+0000 7f2012e90740 1 -- 172.21.15.27:0/3950594042 shutdown_connections 2024-09-21T00:30:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.294+0000 7f2012e90740 1 --2- 172.21.15.27:0/3950594042 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ec2f979ea0 0x55ec2fd7f2c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.294+0000 7f2012e90740 1 --2- 172.21.15.27:0/3950594042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f9bb520 0x55ec2fd79180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.294+0000 7f2012e90740 1 --2- 172.21.15.27:0/3950594042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f99a840 0x55ec2fd6d880 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.294+0000 7f2012e90740 1 --2- 172.21.15.27:0/3950594042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f992660 0x55ec2fd6d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.294+0000 7f1feb400640 0 client.26374 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.294+0000 7f2012e90740 1 -- 172.21.15.27:0/3950594042 >> 172.21.15.27:0/3950594042 conn(0x55ec2f9af910 msgr2=0x55ec2fd64990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.294+0000 7f2012e90740 1 -- 172.21.15.27:0/3950594042 shutdown_connections 2024-09-21T00:30:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.294+0000 7f2012e90740 1 -- 172.21.15.27:0/3950594042 wait complete. 2024-09-21T00:30:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.294+0000 7f2012e90740 20 client.26374 trim_cache size 0 max 16384 2024-09-21T00:30:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 Processor -- start 2024-09-21T00:30:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 -- start start 2024-09-21T00:30:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f992660 0x55ec2fd6f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f99a840 0x55ec2fd6c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f9bb520 0x55ec2fd62f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f20041b58d0 con 0x55ec2f9bb520 2024-09-21T00:30:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f20041b3140 con 0x55ec2f99a840 2024-09-21T00:30:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f20041b0fb0 con 0x55ec2f992660 2024-09-21T00:30:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2008e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f9bb520 0x55ec2fd62f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2008e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f9bb520 0x55ec2fd62f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38638/0 (socket says 172.21.15.27:38638) 2024-09-21T00:30:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2008e00640 1 -- 172.21.15.27:0/1980723074 learned_addr learned my addr 172.21.15.27:0/1980723074 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2008e00640 1 --2- 172.21.15.27:0/1980723074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f992660 0x55ec2fd6f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2008e00640 1 -- 172.21.15.27:0/1980723074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f992660 msgr2=0x55ec2fd6f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2008e00640 1 --2- 172.21.15.27:0/1980723074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f992660 0x55ec2fd6f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2008e00640 1 -- 172.21.15.27:0/1980723074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f99a840 msgr2=0x55ec2fd6c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2008e00640 1 --2- 172.21.15.27:0/1980723074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f99a840 0x55ec2fd6c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2008e00640 1 -- 172.21.15.27:0/1980723074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec2fd794c0 con 0x55ec2f9bb520 2024-09-21T00:30:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2008e00640 1 --2- 172.21.15.27:0/1980723074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f9bb520 0x55ec2fd62f40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00096c0 tx=0x7f1ff0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ea55b42d9b3867 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f1ff6000640 1 -- 172.21.15.27:0/1980723074 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0035040 con 0x55ec2f9bb520 2024-09-21T00:30:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f1ff6000640 1 -- 172.21.15.27:0/1980723074 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff0004c20 con 0x55ec2f9bb520 2024-09-21T00:30:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f1ff6000640 1 -- 172.21.15.27:0/1980723074 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0005650 con 0x55ec2f9bb520 2024-09-21T00:30:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 -- 172.21.15.27:0/1980723074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f9bb520 msgr2=0x55ec2fd62f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 --2- 172.21.15.27:0/1980723074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f9bb520 0x55ec2fd62f40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00096c0 tx=0x7f1ff0009260 comp rx=0 tx=0).stop 2024-09-21T00:30:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 -- 172.21.15.27:0/1980723074 shutdown_connections 2024-09-21T00:30:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 --2- 172.21.15.27:0/1980723074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f9bb520 0x55ec2fd62f40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 --2- 172.21.15.27:0/1980723074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f99a840 0x55ec2fd6c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 --2- 172.21.15.27:0/1980723074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f992660 0x55ec2fd6f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 -- 172.21.15.27:0/1980723074 >> 172.21.15.27:0/1980723074 conn(0x55ec2f9af910 msgr2=0x55ec2fd62830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 -- 172.21.15.27:0/1980723074 shutdown_connections 2024-09-21T00:30:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.313+0000 7f2012e90740 1 -- 172.21.15.27:0/1980723074 wait complete. 2024-09-21T00:30:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 1 -- start start 2024-09-21T00:30:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 1 Processor -- start 2024-09-21T00:30:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f992660 0x55ec2fd75660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f99a840 0x55ec2fd71b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f9bb520 0x55ec2fd72060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f20041b1600 con 0x55ec2f992660 2024-09-21T00:30:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f20041b2a70 con 0x55ec2f99a840 2024-09-21T00:30:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f20041b1cd0 con 0x55ec2f9bb520 2024-09-21T00:30:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 10 client.0 _ll_register_callbacks cb 0x7f1ff801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2008e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f992660 0x55ec2fd75660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2008e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f992660 0x55ec2fd75660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38650/0 (socket says 172.21.15.27:38650) 2024-09-21T00:30:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2008e00640 1 -- 172.21.15.27:0/423587073 learned_addr learned my addr 172.21.15.27:0/423587073 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2008e00640 1 --2- 172.21.15.27:0/423587073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f9bb520 0x55ec2fd72060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2008e00640 1 -- 172.21.15.27:0/423587073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f9bb520 msgr2=0x55ec2fd72060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2008e00640 1 --2- 172.21.15.27:0/423587073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f9bb520 0x55ec2fd72060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2008e00640 1 -- 172.21.15.27:0/423587073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f99a840 msgr2=0x55ec2fd71b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:30:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2008e00640 1 --2- 172.21.15.27:0/423587073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f99a840 0x55ec2fd71b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2008e00640 1 -- 172.21.15.27:0/423587073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec2fd725a0 con 0x55ec2f992660 2024-09-21T00:30:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2008e00640 1 --2- 172.21.15.27:0/423587073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f992660 0x55ec2fd75660 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0032fd0 tx=0x7f1ff0005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca619f9e7523525 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f1feb400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:30:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f1feb400640 1 -- 172.21.15.27:0/423587073 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0035070 con 0x55ec2f992660 2024-09-21T00:30:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f1feb400640 1 -- 172.21.15.27:0/423587073 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f1ff0004770 con 0x55ec2f992660 2024-09-21T00:30:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 1 -- 172.21.15.27:0/423587073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff0004c20 con 0x55ec2f992660 2024-09-21T00:30:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f1feb400640 1 -- 172.21.15.27:0/423587073 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1ff0004970 con 0x55ec2f992660 2024-09-21T00:30:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 10 client.16374 fetch_fsmap learned FSMap version 38 2024-09-21T00:30:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 1 -- 172.21.15.27:0/423587073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ec2fdea830 con 0x55ec2f992660 2024-09-21T00:30:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f1feb400640 1 -- 172.21.15.27:0/423587073 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff0004c20 con 0x55ec2f992660 2024-09-21T00:30:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f1feb400640 1 -- 172.21.15.27:0/423587073 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f1ff00315c0 con 0x55ec2f992660 2024-09-21T00:30:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 10 client.16374 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:30:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 10 client.16374 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:30:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 20 client.16374 populate_metadata read hostname 'smithi027' 2024-09-21T00:30:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 1 --2- 172.21.15.27:0/423587073 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ec2f94f250 0x55ec2fdecee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 4 client.16374 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "47099ce0", "await_for": 45.0}] 2024-09-21T00:30:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2012e90740 1 -- 172.21.15.27:0/423587073 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "quiesce db", "set_id": "47099ce0", "await_for": 45.0}) -- 0x55ec2fcd6ac0 con 0x55ec2f94f250 2024-09-21T00:30:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.318+0000 7f2009800640 1 --2- 172.21.15.27:0/423587073 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ec2f94f250 0x55ec2fdecee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.323+0000 7f2009800640 1 --2- 172.21.15.27:0/423587073 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ec2f94f250 0x55ec2fdecee0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aa72d6f6093dfc8 server_cookie=248ab9720cff99bc in_seq=0 out_seq=0 2024-09-21T00:30:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.323+0000 7f1feb400640 10 client.16374 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.323+0000 7f1feb400640 1 -- 172.21.15.27:0/423587073 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: -1 ) ==== 8+0+594 (crc 0 0 0) 0x55ec2fcd6ac0 con 0x55ec2f94f250 2024-09-21T00:30:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.323+0000 7f1feb400640 10 client.16374 handle_command_reply: tid=0 2024-09-21T00:30:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.323+0000 7f2012e90740 1 client.16374 shutdown 2024-09-21T00:30:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.324+0000 7f2012e90740 1 -- 172.21.15.27:0/423587073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f992660 msgr2=0x55ec2fd75660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.324+0000 7f2012e90740 1 --2- 172.21.15.27:0/423587073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f992660 0x55ec2fd75660 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0032fd0 tx=0x7f1ff0005230 comp rx=0 tx=0).stop 2024-09-21T00:30:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.324+0000 7f2012e90740 1 -- 172.21.15.27:0/423587073 shutdown_connections 2024-09-21T00:30:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.324+0000 7f2012e90740 1 --2- 172.21.15.27:0/423587073 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55ec2f94f250 0x55ec2fdecee0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.324+0000 7f2012e90740 1 --2- 172.21.15.27:0/423587073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec2f9bb520 0x55ec2fd72060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.324+0000 7f2012e90740 1 --2- 172.21.15.27:0/423587073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec2f99a840 0x55ec2fd71b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.324+0000 7f2012e90740 1 --2- 172.21.15.27:0/423587073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec2f992660 0x55ec2fd75660 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.324+0000 7f2012e90740 1 -- 172.21.15.27:0/423587073 >> 172.21.15.27:0/423587073 conn(0x55ec2f9af910 msgr2=0x55ec2fd6ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:30.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.324+0000 7f1feb400640 0 client.16374 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:30.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.324+0000 7f2012e90740 1 -- 172.21.15.27:0/423587073 shutdown_connections 2024-09-21T00:30:30.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.324+0000 7f2012e90740 1 -- 172.21.15.27:0/423587073 wait complete. 2024-09-21T00:30:30.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:30:30.324+0000 7f2012e90740 20 client.16374 trim_cache size 0 max 16384 2024-09-21T00:30:30.379 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:30:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 38, 2024-09-21T00:30:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24761, 2024-09-21T00:30:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 26, 2024-09-21T00:30:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T00:30:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "47099ce0": { 2024-09-21T00:30:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 26, 2024-09-21T00:30:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 13.8, 2024-09-21T00:30:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:30:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "TIMEDOUT", 2024-09-21T00:30:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T00:30:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:30:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T00:30:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 139.2, 2024-09-21T00:30:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T00:30:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T00:30:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T00:30:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T00:30:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCING", 2024-09-21T00:30:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 90.0 2024-09-21T00:30:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:30:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:30:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:30:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:30:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:30:30.381 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:30:30.381 INFO:teuthology.orchestra.run.smithi027.stderr:Error EPERM: 2024-09-21T00:30:30.526 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:30 smithi027 ceph-mon[30745]: pgmap v659: 129 pgs: 129 active+clean; 975 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:30:30.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:30 smithi089 ceph-mon[34481]: pgmap v659: 129 pgs: 129 active+clean; 975 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:30:30.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:30 smithi098 ceph-mon[34725]: pgmap v659: 129 pgs: 129 active+clean; 975 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:30:31.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:30:31.324 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't quiesce root with rc: 1 (EPERM), stdout: { "epoch": 38, "leader": 24761, "set_version": 26, "sets": { "47099ce0": { "version": 26, "age_ref": 13.8, "state": { "name": "TIMEDOUT", "age": 0.0 }, "timeout": 90.0, "expiration": 139.2, "members": { "file:/": { "excluded": false, "state": { "name": "QUIESCING", "age": 90.0 } } } } } } 2024-09-21T00:30:31.324 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/quiescer.py", line 357, in _run self.do_quiesce(d) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/quiescer.py", line 313, in do_quiesce raise RuntimeError(f"Error quiescing set '{set_id}': {rcinfo(rc)}") RuntimeError: Error quiescing set '47099ce0': 1 (EPERM) 2024-09-21T00:30:32.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:32 smithi027 ceph-mon[30745]: pgmap v660: 129 pgs: 129 active+clean; 975 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 102 B/s wr, 5 op/s 2024-09-21T00:30:32.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:32 smithi089 ceph-mon[34481]: pgmap v660: 129 pgs: 129 active+clean; 975 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 102 B/s wr, 5 op/s 2024-09-21T00:30:32.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:32 smithi098 ceph-mon[34725]: pgmap v660: 129 pgs: 129 active+clean; 975 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 102 B/s wr, 5 op/s 2024-09-21T00:30:34.527 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:34 smithi027 ceph-mon[30745]: pgmap v661: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-09-21T00:30:34.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:34 smithi089 ceph-mon[34481]: pgmap v661: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-09-21T00:30:34.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:34 smithi098 ceph-mon[34725]: pgmap v661: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-09-21T00:30:36.494 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:36 smithi027 ceph-mon[30745]: pgmap v662: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-09-21T00:30:36.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:36 smithi089 ceph-mon[34481]: pgmap v662: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-09-21T00:30:36.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:36 smithi098 ceph-mon[34725]: pgmap v662: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-09-21T00:30:38.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:38 smithi027 ceph-mon[30745]: pgmap v663: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 5 op/s 2024-09-21T00:30:38.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:38 smithi089 ceph-mon[34481]: pgmap v663: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 5 op/s 2024-09-21T00:30:38.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:38 smithi098 ceph-mon[34725]: pgmap v663: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 5 op/s 2024-09-21T00:30:40.675 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:40 smithi027 ceph-mon[30745]: pgmap v664: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:40.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:40 smithi089 ceph-mon[34481]: pgmap v664: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:40.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:40 smithi098 ceph-mon[34725]: pgmap v664: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.863+0000 7f24d2c00640 1 -- 172.21.15.27:0/1712274206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24cc102530 msgr2=0x7f24cc102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.863+0000 7f24d2c00640 1 --2- 172.21.15.27:0/1712274206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24cc102530 0x7f24cc102940 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009900 tx=0x7f24c002a410 comp rx=0 tx=0).stop 2024-09-21T00:30:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.863+0000 7f24d2c00640 1 -- 172.21.15.27:0/1712274206 shutdown_connections 2024-09-21T00:30:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.863+0000 7f24d2c00640 1 --2- 172.21.15.27:0/1712274206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24cc1079f0 0x7f24cc107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.863+0000 7f24d2c00640 1 --2- 172.21.15.27:0/1712274206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24cc102e80 0x7f24cc1073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.863+0000 7f24d2c00640 1 --2- 172.21.15.27:0/1712274206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24cc102530 0x7f24cc102940 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.863+0000 7f24d2c00640 1 -- 172.21.15.27:0/1712274206 >> 172.21.15.27:0/1712274206 conn(0x7f24cc0fe210 msgr2=0x7f24cc100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.863+0000 7f24d2c00640 1 -- 172.21.15.27:0/1712274206 shutdown_connections 2024-09-21T00:30:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.863+0000 7f24d2c00640 1 -- 172.21.15.27:0/1712274206 wait complete. 2024-09-21T00:30:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.863+0000 7f24d2c00640 1 Processor -- start 2024-09-21T00:30:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.863+0000 7f24d2c00640 1 -- start start 2024-09-21T00:30:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d2c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24cc102530 0x7f24cc1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d2c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24cc102e80 0x7f24cc1a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d2c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24cc1079f0 0x7f24cc1a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d2c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc119e50 con 0x7f24cc102530 2024-09-21T00:30:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d2c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc119cd0 con 0x7f24cc1079f0 2024-09-21T00:30:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d2c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc119fd0 con 0x7f24cc102e80 2024-09-21T00:30:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d1800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24cc102530 0x7f24cc1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d1800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24cc102530 0x7f24cc1a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53374/0 (socket says 172.21.15.27:53374) 2024-09-21T00:30:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d1800640 1 -- 172.21.15.27:0/1239619408 learned_addr learned my addr 172.21.15.27:0/1239619408 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d0e00640 1 --2- 172.21.15.27:0/1239619408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24cc102e80 0x7f24cc1a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d2200640 1 --2- 172.21.15.27:0/1239619408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24cc1079f0 0x7f24cc1a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d1800640 1 -- 172.21.15.27:0/1239619408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24cc102e80 msgr2=0x7f24cc1a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d1800640 1 --2- 172.21.15.27:0/1239619408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24cc102e80 0x7f24cc1a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d1800640 1 -- 172.21.15.27:0/1239619408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24cc1079f0 msgr2=0x7f24cc1a60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d1800640 1 --2- 172.21.15.27:0/1239619408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24cc1079f0 0x7f24cc1a60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d1800640 1 -- 172.21.15.27:0/1239619408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24cc1a67e0 con 0x7f24cc102530 2024-09-21T00:30:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d1800640 1 --2- 172.21.15.27:0/1239619408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24cc102530 0x7f24cc1a08b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f24c0005940 tx=0x7f24c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24ca000640 1 -- 172.21.15.27:0/1239619408 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c003f030 con 0x7f24cc102530 2024-09-21T00:30:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24ca000640 1 -- 172.21.15.27:0/1239619408 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c0003aa0 con 0x7f24cc102530 2024-09-21T00:30:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d2c00640 1 -- 172.21.15.27:0/1239619408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24cc1a6a70 con 0x7f24cc102530 2024-09-21T00:30:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24ca000640 1 -- 172.21.15.27:0/1239619408 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0035070 con 0x7f24cc102530 2024-09-21T00:30:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.868+0000 7f24d2c00640 1 -- 172.21.15.27:0/1239619408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24cc1ad260 con 0x7f24cc102530 2024-09-21T00:30:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.873+0000 7f24ca000640 1 -- 172.21.15.27:0/1239619408 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f24c0042030 con 0x7f24cc102530 2024-09-21T00:30:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.873+0000 7f24db3d8740 1 Processor -- start 2024-09-21T00:30:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 -- start start 2024-09-21T00:30:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d411b0 0x55e95a0e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d182f0 0x55e95a0e6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d204d0 0x55e95a0e6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e95a05b6b0 con 0x55e959d411b0 2024-09-21T00:30:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e95a04aae0 con 0x55e959d182f0 2024-09-21T00:30:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e95a04e8f0 con 0x55e959d204d0 2024-09-21T00:30:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24d2200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d411b0 0x55e95a0e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24d2200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d411b0 0x55e95a0e5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53376/0 (socket says 172.21.15.27:53376) 2024-09-21T00:30:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24d2200640 1 -- 172.21.15.27:0/2563088189 learned_addr learned my addr 172.21.15.27:0/2563088189 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24d1800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d182f0 0x55e95a0e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24d0e00640 1 --2- 172.21.15.27:0/2563088189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d204d0 0x55e95a0e6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24d2200640 1 -- 172.21.15.27:0/2563088189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d204d0 msgr2=0x55e95a0e6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24d2200640 1 --2- 172.21.15.27:0/2563088189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d204d0 0x55e95a0e6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24d2200640 1 -- 172.21.15.27:0/2563088189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d182f0 msgr2=0x55e95a0e6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24d2200640 1 --2- 172.21.15.27:0/2563088189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d182f0 0x55e95a0e6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24d2200640 1 -- 172.21.15.27:0/2563088189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e95a017780 con 0x55e959d411b0 2024-09-21T00:30:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24d2200640 1 --2- 172.21.15.27:0/2563088189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d411b0 0x55e95a0e5e10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009f90 tx=0x7f24c40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e473b3f13e8a1322 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24ca000640 1 --2- 172.21.15.27:0/1239619408 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f24b40b66f0 0x7f24b40b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24bea00640 1 -- 172.21.15.27:0/2563088189 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4004d20 con 0x55e959d411b0 2024-09-21T00:30:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24bea00640 1 -- 172.21.15.27:0/2563088189 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c4004050 con 0x55e959d411b0 2024-09-21T00:30:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24bea00640 1 -- 172.21.15.27:0/2563088189 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4005240 con 0x55e959d411b0 2024-09-21T00:30:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24ca000640 1 -- 172.21.15.27:0/1239619408 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f24c0107ab0 con 0x7f24cc102530 2024-09-21T00:30:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24d1800640 1 --2- 172.21.15.27:0/1239619408 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f24b40b66f0 0x7f24b40b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 -- 172.21.15.27:0/2563088189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d411b0 msgr2=0x55e95a0e5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 --2- 172.21.15.27:0/2563088189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d411b0 0x55e95a0e5e10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009f90 tx=0x7f24c40118d0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24d1800640 1 --2- 172.21.15.27:0/1239619408 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f24b40b66f0 0x7f24b40b8c00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55e95a0e4a90 tx=0x7f24c010b560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 -- 172.21.15.27:0/2563088189 shutdown_connections 2024-09-21T00:30:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 --2- 172.21.15.27:0/2563088189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d204d0 0x55e95a0e6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 --2- 172.21.15.27:0/2563088189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d182f0 0x55e95a0e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 --2- 172.21.15.27:0/2563088189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d411b0 0x55e95a0e5e10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 -- 172.21.15.27:0/2563088189 >> 172.21.15.27:0/2563088189 conn(0x55e959d35300 msgr2=0x55e95a0e3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 -- 172.21.15.27:0/2563088189 shutdown_connections 2024-09-21T00:30:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 -- 172.21.15.27:0/2563088189 wait complete. 2024-09-21T00:30:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 -- start start 2024-09-21T00:30:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.874+0000 7f24db3d8740 1 Processor -- start 2024-09-21T00:30:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d182f0 0x55e95a0f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d204d0 0x55e95a0f1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d411b0 0x55e95a0f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e959f82ab0 con 0x55e959d204d0 2024-09-21T00:30:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e959ff1570 con 0x55e959d182f0 2024-09-21T00:30:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e95a0f4590 con 0x55e959d411b0 2024-09-21T00:30:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 10 client.0 _ll_register_callbacks cb 0x55e95a000100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24d0e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d204d0 0x55e95a0f1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24d2200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d182f0 0x55e95a0f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24d0e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d204d0 0x55e95a0f1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53392/0 (socket says 172.21.15.27:53392) 2024-09-21T00:30:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24d0e00640 1 -- 172.21.15.27:0/3332831657 learned_addr learned my addr 172.21.15.27:0/3332831657 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24d2200640 1 --2- 172.21.15.27:0/3332831657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d411b0 0x55e95a0f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24d0e00640 1 -- 172.21.15.27:0/3332831657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d411b0 msgr2=0x55e95a0f1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24d0e00640 1 --2- 172.21.15.27:0/3332831657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d411b0 0x55e95a0f1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24d0e00640 1 -- 172.21.15.27:0/3332831657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d182f0 msgr2=0x55e95a0f13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24d0e00640 1 --2- 172.21.15.27:0/3332831657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d182f0 0x55e95a0f13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24d0e00640 1 -- 172.21.15.27:0/3332831657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e959dac790 con 0x55e959d204d0 2024-09-21T00:30:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24d0e00640 1 --2- 172.21.15.27:0/3332831657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d204d0 0x55e95a0f1900 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f24b800ba20 tx=0x7f24b800bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507fd3be2e4ec817 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24b3400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:30:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24b3400640 1 -- 172.21.15.27:0/3332831657 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b801b440 con 0x55e959d204d0 2024-09-21T00:30:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24b3400640 1 -- 172.21.15.27:0/3332831657 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f24b801b5e0 con 0x55e959d204d0 2024-09-21T00:30:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 1 -- 172.21.15.27:0/3332831657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c4004050 con 0x55e959d204d0 2024-09-21T00:30:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24b3400640 1 -- 172.21.15.27:0/3332831657 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f24b8024050 con 0x55e959d204d0 2024-09-21T00:30:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 10 client.16386 fetch_fsmap learned FSMap version 38 2024-09-21T00:30:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 1 -- 172.21.15.27:0/3332831657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e959e25620 con 0x55e959d204d0 2024-09-21T00:30:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24b3400640 1 -- 172.21.15.27:0/3332831657 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c4004050 con 0x55e959d204d0 2024-09-21T00:30:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24b3400640 1 -- 172.21.15.27:0/3332831657 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f24b8019020 con 0x55e959d204d0 2024-09-21T00:30:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 10 client.16386 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:30:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 10 client.16386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:30:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 20 client.16386 populate_metadata read hostname 'smithi027' 2024-09-21T00:30:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 1 --2- 172.21.15.27:0/3332831657 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55e959cffd80 0x55e95a103610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 4 client.16386 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:30:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 1 -- 172.21.15.27:0/3332831657 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e95a05d600 con 0x55e959cffd80 2024-09-21T00:30:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24d2200640 1 --2- 172.21.15.27:0/3332831657 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55e959cffd80 0x55e95a103610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24d2200640 1 --2- 172.21.15.27:0/3332831657 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55e959cffd80 0x55e95a103610 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1502dfc68b8959f server_cookie=7b01f13b8d02f41e in_seq=0 out_seq=0 2024-09-21T00:30:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24b3400640 10 client.16386 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24b3400640 1 -- 172.21.15.27:0/3332831657 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e95a05d600 con 0x55e959cffd80 2024-09-21T00:30:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24b3400640 10 client.16386 handle_command_reply: tid=0 2024-09-21T00:30:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.879+0000 7f24db3d8740 1 client.16386 shutdown 2024-09-21T00:30:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.884+0000 7f24db3d8740 1 -- 172.21.15.27:0/3332831657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d204d0 msgr2=0x55e95a0f1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.884+0000 7f24db3d8740 1 --2- 172.21.15.27:0/3332831657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d204d0 0x55e95a0f1900 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f24b800ba20 tx=0x7f24b800bee0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.884+0000 7f24db3d8740 1 -- 172.21.15.27:0/3332831657 shutdown_connections 2024-09-21T00:30:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.884+0000 7f24db3d8740 1 --2- 172.21.15.27:0/3332831657 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55e959cffd80 0x55e95a103610 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.884+0000 7f24db3d8740 1 --2- 172.21.15.27:0/3332831657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d411b0 0x55e95a0f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.884+0000 7f24db3d8740 1 --2- 172.21.15.27:0/3332831657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d204d0 0x55e95a0f1900 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.884+0000 7f24db3d8740 1 --2- 172.21.15.27:0/3332831657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d182f0 0x55e95a0f13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.884+0000 7f24db3d8740 1 -- 172.21.15.27:0/3332831657 >> 172.21.15.27:0/3332831657 conn(0x55e959d35300 msgr2=0x55e95a0e6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.884+0000 7f24b3400640 0 client.16386 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.884+0000 7f24db3d8740 1 -- 172.21.15.27:0/3332831657 shutdown_connections 2024-09-21T00:30:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.884+0000 7f24db3d8740 1 -- 172.21.15.27:0/3332831657 wait complete. 2024-09-21T00:30:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.884+0000 7f24db3d8740 20 client.16386 trim_cache size 0 max 16384 2024-09-21T00:30:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.908+0000 7f24db3d8740 1 Processor -- start 2024-09-21T00:30:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.908+0000 7f24db3d8740 1 -- start start 2024-09-21T00:30:40.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d182f0 0x55e95a0e7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d204d0 0x55e95a0ee590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d411b0 0x55e95a0eead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc1b9c00 con 0x55e959d204d0 2024-09-21T00:30:40.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc1b7470 con 0x55e959d411b0 2024-09-21T00:30:40.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc1b52e0 con 0x55e959d182f0 2024-09-21T00:30:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d0e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d204d0 0x55e95a0ee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d182f0 0x55e95a0e7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d0e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d204d0 0x55e95a0ee590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53398/0 (socket says 172.21.15.27:53398) 2024-09-21T00:30:40.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d0e00640 1 -- 172.21.15.27:0/1388389992 learned_addr learned my addr 172.21.15.27:0/1388389992 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:40.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d182f0 0x55e95a0e7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:50392/0 (socket says 172.21.15.27:50392) 2024-09-21T00:30:40.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 --2- 172.21.15.27:0/1388389992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d411b0 0x55e95a0eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 -- 172.21.15.27:0/1388389992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d411b0 msgr2=0x55e95a0eead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:40.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 --2- 172.21.15.27:0/1388389992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d411b0 0x55e95a0eead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 -- 172.21.15.27:0/1388389992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d204d0 msgr2=0x55e95a0ee590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:40.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 --2- 172.21.15.27:0/1388389992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d204d0 0x55e95a0ee590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 -- 172.21.15.27:0/1388389992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e95a0e8c20 con 0x55e959d182f0 2024-09-21T00:30:40.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 --2- 172.21.15.27:0/1388389992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d182f0 0x55e95a0e7550 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55e95a0e8b20 tx=0x7f24c4009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1224ba710a060bed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:40.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24be000640 1 -- 172.21.15.27:0/1388389992 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4003c20 con 0x55e959d182f0 2024-09-21T00:30:40.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24be000640 1 -- 172.21.15.27:0/1388389992 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c4004e20 con 0x55e959d182f0 2024-09-21T00:30:40.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24be000640 1 -- 172.21.15.27:0/1388389992 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4005590 con 0x55e959d182f0 2024-09-21T00:30:40.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 -- 172.21.15.27:0/1388389992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d182f0 msgr2=0x55e95a0e7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:40.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 --2- 172.21.15.27:0/1388389992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d182f0 0x55e95a0e7550 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55e95a0e8b20 tx=0x7f24c4009500 comp rx=0 tx=0).stop 2024-09-21T00:30:40.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 -- 172.21.15.27:0/1388389992 shutdown_connections 2024-09-21T00:30:40.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 --2- 172.21.15.27:0/1388389992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d411b0 0x55e95a0eead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 --2- 172.21.15.27:0/1388389992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d204d0 0x55e95a0ee590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 --2- 172.21.15.27:0/1388389992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d182f0 0x55e95a0e7550 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 -- 172.21.15.27:0/1388389992 >> 172.21.15.27:0/1388389992 conn(0x55e959d35300 msgr2=0x55e95a0e6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 -- 172.21.15.27:0/1388389992 shutdown_connections 2024-09-21T00:30:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 -- 172.21.15.27:0/1388389992 wait complete. 2024-09-21T00:30:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 -- start start 2024-09-21T00:30:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 Processor -- start 2024-09-21T00:30:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d182f0 0x55e95a0f7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d204d0 0x55e95a0f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d411b0 0x55e95a16c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc1b5930 con 0x55e959d182f0 2024-09-21T00:30:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc1b6da0 con 0x55e959d204d0 2024-09-21T00:30:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc1b6000 con 0x55e959d411b0 2024-09-21T00:30:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24db3d8740 10 client.0 _ll_register_callbacks cb 0x7f24b8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d182f0 0x55e95a0f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d182f0 0x55e95a0f7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53400/0 (socket says 172.21.15.27:53400) 2024-09-21T00:30:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 -- 172.21.15.27:0/1630311741 learned_addr learned my addr 172.21.15.27:0/1630311741 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d0e00640 1 --2- 172.21.15.27:0/1630311741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d204d0 0x55e95a0f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 --2- 172.21.15.27:0/1630311741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d411b0 0x55e95a16c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 -- 172.21.15.27:0/1630311741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d411b0 msgr2=0x55e95a16c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 --2- 172.21.15.27:0/1630311741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d411b0 0x55e95a16c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 -- 172.21.15.27:0/1630311741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d204d0 msgr2=0x55e95a0f75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 --2- 172.21.15.27:0/1630311741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d204d0 0x55e95a0f75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 -- 172.21.15.27:0/1630311741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e95a16c660 con 0x55e959d182f0 2024-09-21T00:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24d2200640 1 --2- 172.21.15.27:0/1630311741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d182f0 0x55e95a0f7040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f24c4005c10 tx=0x7f24c4031040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4018512e789349a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.910+0000 7f24b3400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.915+0000 7f24b3400640 1 -- 172.21.15.27:0/1630311741 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c403b060 con 0x55e959d182f0 2024-09-21T00:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.915+0000 7f24b3400640 1 -- 172.21.15.27:0/1630311741 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f24c4005000 con 0x55e959d182f0 2024-09-21T00:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.915+0000 7f24db3d8740 1 -- 172.21.15.27:0/1630311741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c4004e20 con 0x55e959d182f0 2024-09-21T00:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.915+0000 7f24b3400640 1 -- 172.21.15.27:0/1630311741 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f24c4042050 con 0x55e959d182f0 2024-09-21T00:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.915+0000 7f24db3d8740 10 client.16392 fetch_fsmap learned FSMap version 38 2024-09-21T00:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.915+0000 7f24db3d8740 1 -- 172.21.15.27:0/1630311741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e95a16d8d0 con 0x55e959d182f0 2024-09-21T00:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.915+0000 7f24b3400640 1 -- 172.21.15.27:0/1630311741 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c4004e20 con 0x55e959d182f0 2024-09-21T00:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.915+0000 7f24b3400640 1 -- 172.21.15.27:0/1630311741 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f24c4025040 con 0x55e959d182f0 2024-09-21T00:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.915+0000 7f24db3d8740 10 client.16392 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.915+0000 7f24db3d8740 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.915+0000 7f24db3d8740 20 client.16392 populate_metadata read hostname 'smithi027' 2024-09-21T00:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.915+0000 7f24db3d8740 1 --2- 172.21.15.27:0/1630311741 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55e959cd51c0 0x55e95a16fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.915+0000 7f24db3d8740 4 client.16392 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:30:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.915+0000 7f24db3d8740 1 -- 172.21.15.27:0/1630311741 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e95a0590b0 con 0x55e959cd51c0 2024-09-21T00:30:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.915+0000 7f24d0e00640 1 --2- 172.21.15.27:0/1630311741 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55e959cd51c0 0x55e95a16fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.957+0000 7f24d0e00640 1 --2- 172.21.15.27:0/1630311741 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55e959cd51c0 0x55e95a16fdf0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f49c9a50d84be8f server_cookie=62ff1d5c29c63e62 in_seq=0 out_seq=0 2024-09-21T00:30:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.957+0000 7f24b3400640 10 client.16392 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.981+0000 7f24b3400640 1 -- 172.21.15.27:0/1630311741 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e95a0590b0 con 0x55e959cd51c0 2024-09-21T00:30:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.981+0000 7f24b3400640 10 client.16392 handle_command_reply: tid=0 2024-09-21T00:30:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.981+0000 7f24db3d8740 1 client.16392 shutdown 2024-09-21T00:30:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.981+0000 7f24db3d8740 1 -- 172.21.15.27:0/1630311741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d182f0 msgr2=0x55e95a0f7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.981+0000 7f24db3d8740 1 --2- 172.21.15.27:0/1630311741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d182f0 0x55e95a0f7040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f24c4005c10 tx=0x7f24c4031040 comp rx=0 tx=0).stop 2024-09-21T00:30:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24db3d8740 1 -- 172.21.15.27:0/1630311741 shutdown_connections 2024-09-21T00:30:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24db3d8740 1 --2- 172.21.15.27:0/1630311741 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55e959cd51c0 0x55e95a16fdf0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24db3d8740 1 --2- 172.21.15.27:0/1630311741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e959d411b0 0x55e95a16c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24db3d8740 1 --2- 172.21.15.27:0/1630311741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e959d204d0 0x55e95a0f75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24db3d8740 1 --2- 172.21.15.27:0/1630311741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e959d182f0 0x55e95a0f7040 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24db3d8740 1 -- 172.21.15.27:0/1630311741 >> 172.21.15.27:0/1630311741 conn(0x55e959d35300 msgr2=0x55e95a0e2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24b3400640 0 client.16392 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24db3d8740 1 -- 172.21.15.27:0/1630311741 shutdown_connections 2024-09-21T00:30:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24db3d8740 1 -- 172.21.15.27:0/1630311741 wait complete. 2024-09-21T00:30:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24db3d8740 20 client.16392 trim_cache size 0 max 16384 2024-09-21T00:30:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:30:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:30:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:30:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:30:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24d2c00640 1 -- 172.21.15.27:0/1239619408 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f24b40b66f0 msgr2=0x7f24b40b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24d2c00640 1 --2- 172.21.15.27:0/1239619408 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f24b40b66f0 0x7f24b40b8c00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55e95a0e4a90 tx=0x7f24c010b560 comp rx=0 tx=0).stop 2024-09-21T00:30:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24d2c00640 1 -- 172.21.15.27:0/1239619408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24cc102530 msgr2=0x7f24cc1a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24d2c00640 1 --2- 172.21.15.27:0/1239619408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24cc102530 0x7f24cc1a08b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f24c0005940 tx=0x7f24c0004270 comp rx=0 tx=0).stop 2024-09-21T00:30:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24d2c00640 1 -- 172.21.15.27:0/1239619408 shutdown_connections 2024-09-21T00:30:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24d2c00640 1 --2- 172.21.15.27:0/1239619408 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f24b40b66f0 0x7f24b40b8c00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24d2c00640 1 --2- 172.21.15.27:0/1239619408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24cc1079f0 0x7f24cc1a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24d2c00640 1 --2- 172.21.15.27:0/1239619408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24cc102e80 0x7f24cc1a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24d2c00640 1 --2- 172.21.15.27:0/1239619408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24cc102530 0x7f24cc1a08b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24d2c00640 1 -- 172.21.15.27:0/1239619408 >> 172.21.15.27:0/1239619408 conn(0x7f24cc0fe210 msgr2=0x7f24cc105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24d2c00640 1 -- 172.21.15.27:0/1239619408 shutdown_connections 2024-09-21T00:30:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:40.982+0000 7f24d2c00640 1 -- 172.21.15.27:0/1239619408 wait complete. 2024-09-21T00:30:42.385 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:42 smithi027 ceph-mon[30745]: pgmap v665: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:42.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:42 smithi089 ceph-mon[34481]: pgmap v665: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:42.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:42 smithi098 ceph-mon[34725]: pgmap v665: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:44.386 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:44 smithi027 ceph-mon[30745]: pgmap v666: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 36 KiB/s wr, 6 op/s 2024-09-21T00:30:44.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:44 smithi089 ceph-mon[34481]: pgmap v666: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 36 KiB/s wr, 6 op/s 2024-09-21T00:30:44.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:44 smithi098 ceph-mon[34725]: pgmap v666: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 36 KiB/s wr, 6 op/s 2024-09-21T00:30:46.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:46 smithi089 ceph-mon[34481]: pgmap v667: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 18 KiB/s wr, 5 op/s 2024-09-21T00:30:46.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:46 smithi098 ceph-mon[34725]: pgmap v667: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 18 KiB/s wr, 5 op/s 2024-09-21T00:30:46.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:46 smithi027 ceph-mon[30745]: pgmap v667: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 18 KiB/s wr, 5 op/s 2024-09-21T00:30:47.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:47 smithi089 ceph-mon[34481]: pgmap v668: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:47.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:47 smithi098 ceph-mon[34725]: pgmap v668: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:47.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:47 smithi027 ceph-mon[30745]: pgmap v668: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:49.907 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:49 smithi027 ceph-mon[30745]: pgmap v669: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:49.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:49 smithi089 ceph-mon[34481]: pgmap v669: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:49.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:49 smithi098 ceph-mon[34725]: pgmap v669: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:51.858 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:51 smithi027 ceph-mon[30745]: pgmap v670: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:51.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:51 smithi098 ceph-mon[34725]: pgmap v670: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:52.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:51 smithi089 ceph-mon[34481]: pgmap v670: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:53.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:53 smithi098 ceph-mon[34725]: pgmap v671: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:54.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:53 smithi027 ceph-mon[30745]: pgmap v671: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 -- 172.21.15.27:0/2049990467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab00109b90 msgr2=0x7fab001103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 --2- 172.21.15.27:0/2049990467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab00109b90 0x7fab001103c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009ca0 tx=0x7faaf802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 -- 172.21.15.27:0/2049990467 shutdown_connections 2024-09-21T00:30:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 --2- 172.21.15.27:0/2049990467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab00109b90 0x7fab001103c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 --2- 172.21.15.27:0/2049990467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab00109240 0x7fab00109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 --2- 172.21.15.27:0/2049990467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00103910 0x7fab00103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 -- 172.21.15.27:0/2049990467 >> 172.21.15.27:0/2049990467 conn(0x7fab000ff390 msgr2=0x7fab001017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 -- 172.21.15.27:0/2049990467 shutdown_connections 2024-09-21T00:30:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 -- 172.21.15.27:0/2049990467 wait complete. 2024-09-21T00:30:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 Processor -- start 2024-09-21T00:30:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 -- start start 2024-09-21T00:30:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00103910 0x7fab0019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab00109240 0x7fab0019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab00109b90 0x7fab0019fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab00113850 con 0x7fab00103910 2024-09-21T00:30:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fab001136d0 con 0x7fab00109b90 2024-09-21T00:30:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fab001139d0 con 0x7fab00109240 2024-09-21T00:30:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab05000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00103910 0x7fab0019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab05a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab00109b90 0x7fab0019fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab05000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00103910 0x7fab0019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43340/0 (socket says 172.21.15.27:43340) 2024-09-21T00:30:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab05000640 1 -- 172.21.15.27:0/3775766758 learned_addr learned my addr 172.21.15.27:0/3775766758 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7faaffe00640 1 --2- 172.21.15.27:0/3775766758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab00109240 0x7fab0019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab05a00640 1 -- 172.21.15.27:0/3775766758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab00109240 msgr2=0x7fab0019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab05a00640 1 --2- 172.21.15.27:0/3775766758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab00109240 0x7fab0019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab05a00640 1 -- 172.21.15.27:0/3775766758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00103910 msgr2=0x7fab0019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab05a00640 1 --2- 172.21.15.27:0/3775766758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00103910 0x7fab0019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab05a00640 1 -- 172.21.15.27:0/3775766758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab001a01c0 con 0x7fab00109b90 2024-09-21T00:30:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab05a00640 1 --2- 172.21.15.27:0/3775766758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab00109b90 0x7fab0019fac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faaf802ade0 tx=0x7faaf8031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7faafd600640 1 -- 172.21.15.27:0/3775766758 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8003b70 con 0x7fab00109b90 2024-09-21T00:30:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 -- 172.21.15.27:0/3775766758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab001a0450 con 0x7fab00109b90 2024-09-21T00:30:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7faafd600640 1 -- 172.21.15.27:0/3775766758 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaf8031910 con 0x7fab00109b90 2024-09-21T00:30:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7faafd600640 1 -- 172.21.15.27:0/3775766758 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf802f950 con 0x7fab00109b90 2024-09-21T00:30:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.178+0000 7fab06400640 1 -- 172.21.15.27:0/3775766758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab001a6c40 con 0x7fab00109b90 2024-09-21T00:30:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7fab0ec2c740 1 Processor -- start 2024-09-21T00:30:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faafd600640 1 -- 172.21.15.27:0/3775766758 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7faaf802faf0 con 0x7fab00109b90 2024-09-21T00:30:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7fab0ec2c740 1 -- start start 2024-09-21T00:30:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7fab0ec2c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7f61b0 0x55827cb9ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7fab0ec2c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7cd2f0 0x55827cb9b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7fab0ec2c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7d54d0 0x55827cb9b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7fab0ec2c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55827cb106b0 con 0x55827c7f61b0 2024-09-21T00:30:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7fab0ec2c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55827caffae0 con 0x55827c7cd2f0 2024-09-21T00:30:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7fab0ec2c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55827cb038f0 con 0x55827c7d54d0 2024-09-21T00:30:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faaffe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7f61b0 0x55827cb9ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faaffe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7f61b0 0x55827cb9ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43356/0 (socket says 172.21.15.27:43356) 2024-09-21T00:30:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faaffe00640 1 -- 172.21.15.27:0/1364736806 learned_addr learned my addr 172.21.15.27:0/1364736806 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faaffe00640 1 --2- 172.21.15.27:0/1364736806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7d54d0 0x55827cb9b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faaffe00640 1 -- 172.21.15.27:0/1364736806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7d54d0 msgr2=0x55827cb9b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faaffe00640 1 --2- 172.21.15.27:0/1364736806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7d54d0 0x55827cb9b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faaffe00640 1 -- 172.21.15.27:0/1364736806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7cd2f0 msgr2=0x55827cb9b350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:30:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faaffe00640 1 --2- 172.21.15.27:0/1364736806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7cd2f0 0x55827cb9b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faaffe00640 1 -- 172.21.15.27:0/1364736806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55827cacc780 con 0x55827c7f61b0 2024-09-21T00:30:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faaffe00640 1 --2- 172.21.15.27:0/1364736806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7f61b0 0x55827cb9ae10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7faaec009e50 tx=0x7faaec005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40b305c84feba3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faafd600640 1 --2- 172.21.15.27:0/3775766758 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7faae80b6670 0x7faae80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faaf2000640 1 -- 172.21.15.27:0/1364736806 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec004050 con 0x55827c7f61b0 2024-09-21T00:30:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faaf2000640 1 -- 172.21.15.27:0/1364736806 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaec0041f0 con 0x55827c7f61b0 2024-09-21T00:30:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faafd600640 1 -- 172.21.15.27:0/3775766758 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7faaf8108b80 con 0x7fab00109b90 2024-09-21T00:30:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7faaf2000640 1 -- 172.21.15.27:0/1364736806 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec010700 con 0x55827c7f61b0 2024-09-21T00:30:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7fab05000640 1 --2- 172.21.15.27:0/3775766758 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7faae80b6670 0x7faae80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.182+0000 7fab0ec2c740 1 -- 172.21.15.27:0/1364736806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7f61b0 msgr2=0x55827cb9ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/1364736806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7f61b0 0x55827cb9ae10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7faaec009e50 tx=0x7faaec005850 comp rx=0 tx=0).stop 2024-09-21T00:30:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 -- 172.21.15.27:0/1364736806 shutdown_connections 2024-09-21T00:30:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/1364736806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7d54d0 0x55827cb9b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/1364736806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7cd2f0 0x55827cb9b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/1364736806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7f61b0 0x55827cb9ae10 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 -- 172.21.15.27:0/1364736806 >> 172.21.15.27:0/1364736806 conn(0x55827c7ea300 msgr2=0x55827cb98d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab05000640 1 --2- 172.21.15.27:0/3775766758 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7faae80b6670 0x7faae80b8b80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faaf40095f0 tx=0x7faaf4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 -- 172.21.15.27:0/1364736806 shutdown_connections 2024-09-21T00:30:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 -- 172.21.15.27:0/1364736806 wait complete. 2024-09-21T00:30:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 -- start start 2024-09-21T00:30:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 Processor -- start 2024-09-21T00:30:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7cd2f0 0x55827cbaa770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7d54d0 0x55827cbaacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7f61b0 0x55827cbab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55827ca37ab0 con 0x55827c7d54d0 2024-09-21T00:30:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55827caa6570 con 0x55827c7f61b0 2024-09-21T00:30:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55827cbad9c0 con 0x55827c7cd2f0 2024-09-21T00:30:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 10 client.0 _ll_register_callbacks cb 0x55827cab5100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7faaffe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7d54d0 0x55827cbaacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab05000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7cd2f0 0x55827cbaa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab05a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7f61b0 0x55827cbab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7faaffe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7d54d0 0x55827cbaacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43360/0 (socket says 172.21.15.27:43360) 2024-09-21T00:30:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab05000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7cd2f0 0x55827cbaa770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:57706/0 (socket says 172.21.15.27:57706) 2024-09-21T00:30:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7faaffe00640 1 -- 172.21.15.27:0/3645121598 learned_addr learned my addr 172.21.15.27:0/3645121598 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7faaffe00640 1 -- 172.21.15.27:0/3645121598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7cd2f0 msgr2=0x55827cbaa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7faaffe00640 1 --2- 172.21.15.27:0/3645121598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7cd2f0 0x55827cbaa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7faaffe00640 1 -- 172.21.15.27:0/3645121598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7f61b0 msgr2=0x55827cbab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7faaffe00640 1 --2- 172.21.15.27:0/3645121598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7f61b0 0x55827cbab1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7faaffe00640 1 -- 172.21.15.27:0/3645121598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55827c861790 con 0x55827c7d54d0 2024-09-21T00:30:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7faaffe00640 1 --2- 172.21.15.27:0/3645121598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7d54d0 0x55827cbaacb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7faaec009a90 tx=0x7faaec009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa7a0b51bb1c25f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7faae6a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:30:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7faae6a00640 1 -- 172.21.15.27:0/3645121598 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec005050 con 0x55827c7d54d0 2024-09-21T00:30:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 -- 172.21.15.27:0/3645121598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaec0041f0 con 0x55827c7d54d0 2024-09-21T00:30:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7faae6a00640 1 -- 172.21.15.27:0/3645121598 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7faaec0051f0 con 0x55827c7d54d0 2024-09-21T00:30:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7faae6a00640 1 -- 172.21.15.27:0/3645121598 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7faaec0053f0 con 0x55827c7d54d0 2024-09-21T00:30:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 10 client.16404 fetch_fsmap learned FSMap version 38 2024-09-21T00:30:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 -- 172.21.15.27:0/3645121598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55827c8da620 con 0x55827c7d54d0 2024-09-21T00:30:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7faae6a00640 1 -- 172.21.15.27:0/3645121598 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaec0041f0 con 0x55827c7d54d0 2024-09-21T00:30:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7faae6a00640 1 -- 172.21.15.27:0/3645121598 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7faaec0105a0 con 0x55827c7d54d0 2024-09-21T00:30:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 10 client.16404 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:30:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:30:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 20 client.16404 populate_metadata read hostname 'smithi027' 2024-09-21T00:30:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/3645121598 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827c7b4d80 0x55827cbbc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 4 client.16404 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:30:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab0ec2c740 1 -- 172.21.15.27:0/3645121598 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55827cb12600 con 0x55827c7b4d80 2024-09-21T00:30:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.188+0000 7fab05000640 1 --2- 172.21.15.27:0/3645121598 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827c7b4d80 0x55827cbbc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.193+0000 7fab05000640 1 --2- 172.21.15.27:0/3645121598 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827c7b4d80 0x55827cbbc8a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e5f45062819c0f server_cookie=b56478bed8d9fc0 in_seq=0 out_seq=0 2024-09-21T00:30:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.193+0000 7faae6a00640 10 client.16404 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.193+0000 7faae6a00640 1 -- 172.21.15.27:0/3645121598 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55827cb12600 con 0x55827c7b4d80 2024-09-21T00:30:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.193+0000 7faae6a00640 10 client.16404 handle_command_reply: tid=0 2024-09-21T00:30:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.193+0000 7fab0ec2c740 1 client.16404 shutdown 2024-09-21T00:30:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.193+0000 7fab0ec2c740 1 -- 172.21.15.27:0/3645121598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7d54d0 msgr2=0x55827cbaacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.193+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/3645121598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7d54d0 0x55827cbaacb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7faaec009a90 tx=0x7faaec009ac0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.193+0000 7fab0ec2c740 1 -- 172.21.15.27:0/3645121598 shutdown_connections 2024-09-21T00:30:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.193+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/3645121598 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827c7b4d80 0x55827cbbc8a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.193+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/3645121598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7f61b0 0x55827cbab1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.193+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/3645121598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7d54d0 0x55827cbaacb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.193+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/3645121598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7cd2f0 0x55827cbaa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.193+0000 7faae6a00640 0 client.16404 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.193+0000 7fab0ec2c740 1 -- 172.21.15.27:0/3645121598 >> 172.21.15.27:0/3645121598 conn(0x55827c7ea300 msgr2=0x55827cb9dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.194+0000 7fab0ec2c740 1 -- 172.21.15.27:0/3645121598 shutdown_connections 2024-09-21T00:30:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.194+0000 7fab0ec2c740 1 -- 172.21.15.27:0/3645121598 wait complete. 2024-09-21T00:30:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.194+0000 7fab0ec2c740 20 client.16404 trim_cache size 0 max 16384 2024-09-21T00:30:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7fab0ec2c740 1 Processor -- start 2024-09-21T00:30:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7fab0ec2c740 1 -- start start 2024-09-21T00:30:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7fab0ec2c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7cd2f0 0x55827cba5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7fab0ec2c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7d54d0 0x55827cbaac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7fab0ec2c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7f61b0 0x55827cbab190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7fab0ec2c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab001b13d0 con 0x55827c7d54d0 2024-09-21T00:30:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7fab0ec2c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fab001aec40 con 0x55827c7f61b0 2024-09-21T00:30:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7fab0ec2c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fab001acab0 con 0x55827c7cd2f0 2024-09-21T00:30:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7faaffe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7d54d0 0x55827cbaac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7faaffe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7d54d0 0x55827cbaac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43368/0 (socket says 172.21.15.27:43368) 2024-09-21T00:30:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7faaffe00640 1 -- 172.21.15.27:0/4160646592 learned_addr learned my addr 172.21.15.27:0/4160646592 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7fab05000640 1 --2- 172.21.15.27:0/4160646592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7cd2f0 0x55827cba5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7fab05a00640 1 --2- 172.21.15.27:0/4160646592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7f61b0 0x55827cbab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7faaffe00640 1 -- 172.21.15.27:0/4160646592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7cd2f0 msgr2=0x55827cba5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7faaffe00640 1 --2- 172.21.15.27:0/4160646592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7cd2f0 0x55827cba5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7faaffe00640 1 -- 172.21.15.27:0/4160646592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7f61b0 msgr2=0x55827cbab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7faaffe00640 1 --2- 172.21.15.27:0/4160646592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7f61b0 0x55827cbab190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7faaffe00640 1 -- 172.21.15.27:0/4160646592 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55827cba8c50 con 0x55827c7d54d0 2024-09-21T00:30:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7fab05a00640 1 --2- 172.21.15.27:0/4160646592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7f61b0 0x55827cbab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7faaffe00640 1 --2- 172.21.15.27:0/4160646592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7d54d0 0x55827cbaac50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7faaec0059f0 tx=0x7faaec003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76b7e15269a8e1d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7faaf1600640 1 -- 172.21.15.27:0/4160646592 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec013c00 con 0x55827c7d54d0 2024-09-21T00:30:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7faaf1600640 1 -- 172.21.15.27:0/4160646592 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaec005000 con 0x55827c7d54d0 2024-09-21T00:30:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.206+0000 7faaf1600640 1 -- 172.21.15.27:0/4160646592 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec02a6c0 con 0x55827c7d54d0 2024-09-21T00:30:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 -- 172.21.15.27:0/4160646592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7d54d0 msgr2=0x55827cbaac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/4160646592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7d54d0 0x55827cbaac50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7faaec0059f0 tx=0x7faaec003e20 comp rx=0 tx=0).stop 2024-09-21T00:30:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 -- 172.21.15.27:0/4160646592 shutdown_connections 2024-09-21T00:30:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/4160646592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7f61b0 0x55827cbab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/4160646592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7d54d0 0x55827cbaac50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/4160646592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7cd2f0 0x55827cba5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 -- 172.21.15.27:0/4160646592 >> 172.21.15.27:0/4160646592 conn(0x55827c7ea300 msgr2=0x55827cba5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 -- 172.21.15.27:0/4160646592 shutdown_connections 2024-09-21T00:30:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 -- 172.21.15.27:0/4160646592 wait complete. 2024-09-21T00:30:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 -- start start 2024-09-21T00:30:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 Processor -- start 2024-09-21T00:30:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7cd2f0 0x55827cbb3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7d54d0 0x55827cbb3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7f61b0 0x55827cbae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab001ad100 con 0x55827c7f61b0 2024-09-21T00:30:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fab001ae570 con 0x55827c7cd2f0 2024-09-21T00:30:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fab001ad7d0 con 0x55827c7d54d0 2024-09-21T00:30:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 10 client.0 _ll_register_callbacks cb 0x7faaec009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab05a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7f61b0 0x55827cbae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7faaffe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7d54d0 0x55827cbb3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab05a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7f61b0 0x55827cbae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43382/0 (socket says 172.21.15.27:43382) 2024-09-21T00:30:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7faaffe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7d54d0 0x55827cbb3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:57716/0 (socket says 172.21.15.27:57716) 2024-09-21T00:30:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7faaffe00640 1 -- 172.21.15.27:0/2382316250 learned_addr learned my addr 172.21.15.27:0/2382316250 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:30:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab05000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7cd2f0 0x55827cbb3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab05a00640 1 -- 172.21.15.27:0/2382316250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7d54d0 msgr2=0x55827cbb3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab05a00640 1 --2- 172.21.15.27:0/2382316250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7d54d0 0x55827cbb3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab05a00640 1 -- 172.21.15.27:0/2382316250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7cd2f0 msgr2=0x55827cbb3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab05a00640 1 --2- 172.21.15.27:0/2382316250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7cd2f0 0x55827cbb3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab05a00640 1 -- 172.21.15.27:0/2382316250 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55827cbaede0 con 0x55827c7f61b0 2024-09-21T00:30:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7faaffe00640 1 --2- 172.21.15.27:0/2382316250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7d54d0 0x55827cbb3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:30:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab05000640 1 --2- 172.21.15.27:0/2382316250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7cd2f0 0x55827cbb3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:30:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab05a00640 1 --2- 172.21.15.27:0/2382316250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7f61b0 0x55827cbae650 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faaf8105e90 tx=0x7faaf8106390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704c0e7ee168e925 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7faae6a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:30:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7faae6a00640 1 -- 172.21.15.27:0/2382316250 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf80d7d20 con 0x55827c7f61b0 2024-09-21T00:30:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7faae6a00640 1 -- 172.21.15.27:0/2382316250 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7faaf80d7040 con 0x55827c7f61b0 2024-09-21T00:30:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 -- 172.21.15.27:0/2382316250 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaec005000 con 0x55827c7f61b0 2024-09-21T00:30:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7faae6a00640 1 -- 172.21.15.27:0/2382316250 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7faaf80cec90 con 0x55827c7f61b0 2024-09-21T00:30:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 10 client.16410 fetch_fsmap learned FSMap version 38 2024-09-21T00:30:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 -- 172.21.15.27:0/2382316250 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55827cc26cf0 con 0x55827c7f61b0 2024-09-21T00:30:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7faae6a00640 1 -- 172.21.15.27:0/2382316250 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaec005000 con 0x55827c7f61b0 2024-09-21T00:30:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7faae6a00640 1 -- 172.21.15.27:0/2382316250 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7faaf80e9be0 con 0x55827c7f61b0 2024-09-21T00:30:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 10 client.16410 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:30:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:30:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 20 client.16410 populate_metadata read hostname 'smithi027' 2024-09-21T00:30:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/2382316250 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827c78a1c0 0x55827cc29480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 4 client.16410 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:30:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab0ec2c740 1 -- 172.21.15.27:0/2382316250 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55827cb0e0b0 con 0x55827c78a1c0 2024-09-21T00:30:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab05000640 1 --2- 172.21.15.27:0/2382316250 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827c78a1c0 0x55827cc29480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7faae6a00640 10 client.16410 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.212+0000 7fab05000640 1 --2- 172.21.15.27:0/2382316250 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827c78a1c0 0x55827cc29480 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ceb6e9275a4a5d1 server_cookie=72358667e741cf75 in_seq=0 out_seq=0 2024-09-21T00:30:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7faae6a00640 1 -- 172.21.15.27:0/2382316250 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55827cb0e0b0 con 0x55827c78a1c0 2024-09-21T00:30:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7faae6a00640 10 client.16410 handle_command_reply: tid=0 2024-09-21T00:30:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab0ec2c740 1 client.16410 shutdown 2024-09-21T00:30:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab0ec2c740 1 -- 172.21.15.27:0/2382316250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7f61b0 msgr2=0x55827cbae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/2382316250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7f61b0 0x55827cbae650 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faaf8105e90 tx=0x7faaf8106390 comp rx=0 tx=0).stop 2024-09-21T00:30:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab0ec2c740 1 -- 172.21.15.27:0/2382316250 shutdown_connections 2024-09-21T00:30:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/2382316250 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55827c78a1c0 0x55827cc29480 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/2382316250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55827c7f61b0 0x55827cbae650 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/2382316250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55827c7d54d0 0x55827cbb3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab0ec2c740 1 --2- 172.21.15.27:0/2382316250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55827c7cd2f0 0x55827cbb3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7faae6a00640 0 client.16410 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:30:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab0ec2c740 1 -- 172.21.15.27:0/2382316250 >> 172.21.15.27:0/2382316250 conn(0x55827c7ea300 msgr2=0x55827cb97360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab0ec2c740 1 -- 172.21.15.27:0/2382316250 shutdown_connections 2024-09-21T00:30:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab0ec2c740 1 -- 172.21.15.27:0/2382316250 wait complete. 2024-09-21T00:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab0ec2c740 20 client.16410 trim_cache size 0 max 16384 2024-09-21T00:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab06400640 1 -- 172.21.15.27:0/3775766758 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7faae80b6670 msgr2=0x7faae80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab06400640 1 --2- 172.21.15.27:0/3775766758 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7faae80b6670 0x7faae80b8b80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faaf40095f0 tx=0x7faaf4005f90 comp rx=0 tx=0).stop 2024-09-21T00:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab06400640 1 -- 172.21.15.27:0/3775766758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab00109b90 msgr2=0x7fab0019fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab06400640 1 --2- 172.21.15.27:0/3775766758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab00109b90 0x7fab0019fac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faaf802ade0 tx=0x7faaf8031660 comp rx=0 tx=0).stop 2024-09-21T00:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab06400640 1 -- 172.21.15.27:0/3775766758 shutdown_connections 2024-09-21T00:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab06400640 1 --2- 172.21.15.27:0/3775766758 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7faae80b6670 0x7faae80b8b80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab06400640 1 --2- 172.21.15.27:0/3775766758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab00109b90 0x7fab0019fac0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab06400640 1 --2- 172.21.15.27:0/3775766758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab00109240 0x7fab0019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab06400640 1 --2- 172.21.15.27:0/3775766758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00103910 0x7fab0019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.224+0000 7fab06400640 1 -- 172.21.15.27:0/3775766758 >> 172.21.15.27:0/3775766758 conn(0x7fab000ff390 msgr2=0x7fab001088c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.229+0000 7fab06400640 1 -- 172.21.15.27:0/3775766758 shutdown_connections 2024-09-21T00:30:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:30:54.229+0000 7fab06400640 1 -- 172.21.15.27:0/3775766758 wait complete. 2024-09-21T00:30:54.232 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:53 smithi089 ceph-mon[34481]: pgmap v671: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:30:55.836 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:55 smithi027 ceph-mon[30745]: pgmap v672: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T00:30:55.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:55 smithi098 ceph-mon[34725]: pgmap v672: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T00:30:56.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:55 smithi089 ceph-mon[34481]: pgmap v672: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T00:30:57.834 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:57 smithi027 ceph-mon[30745]: pgmap v673: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:30:58.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:57 smithi089 ceph-mon[34481]: pgmap v673: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:30:58.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:57 smithi098 ceph-mon[34725]: pgmap v673: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:30:59.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:30:59 smithi027 ceph-mon[30745]: pgmap v674: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:00.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:30:59 smithi089 ceph-mon[34481]: pgmap v674: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:00.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:30:59 smithi098 ceph-mon[34725]: pgmap v674: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:01.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:01 smithi027 ceph-mon[30745]: pgmap v675: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:02.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:01 smithi089 ceph-mon[34481]: pgmap v675: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:02.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:01 smithi098 ceph-mon[34725]: pgmap v675: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:03.826 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:03 smithi027 ceph-mon[30745]: pgmap v676: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 7 op/s 2024-09-21T00:31:04.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:03 smithi089 ceph-mon[34481]: pgmap v676: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 7 op/s 2024-09-21T00:31:04.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:03 smithi098 ceph-mon[34725]: pgmap v676: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 7 op/s 2024-09-21T00:31:06.064 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:05 smithi027 ceph-mon[30745]: pgmap v677: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:06.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:05 smithi089 ceph-mon[34481]: pgmap v677: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:06.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:05 smithi098 ceph-mon[34725]: pgmap v677: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.423+0000 7f7f59e00640 1 -- 172.21.15.27:0/3791035093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54103910 msgr2=0x7f7f54103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.423+0000 7f7f59e00640 1 --2- 172.21.15.27:0/3791035093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54103910 0x7f7f54103d20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009900 tx=0x7f7f4802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.423+0000 7f7f59e00640 1 -- 172.21.15.27:0/3791035093 shutdown_connections 2024-09-21T00:31:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.423+0000 7f7f59e00640 1 --2- 172.21.15.27:0/3791035093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f54109b90 0x7f7f541103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.423+0000 7f7f59e00640 1 --2- 172.21.15.27:0/3791035093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f54109240 0x7f7f54109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.423+0000 7f7f59e00640 1 --2- 172.21.15.27:0/3791035093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54103910 0x7f7f54103d20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.423+0000 7f7f59e00640 1 -- 172.21.15.27:0/3791035093 >> 172.21.15.27:0/3791035093 conn(0x7f7f540ff390 msgr2=0x7f7f541017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.423+0000 7f7f59e00640 1 -- 172.21.15.27:0/3791035093 shutdown_connections 2024-09-21T00:31:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.423+0000 7f7f59e00640 1 -- 172.21.15.27:0/3791035093 wait complete. 2024-09-21T00:31:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.423+0000 7f7f59e00640 1 Processor -- start 2024-09-21T00:31:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.423+0000 7f7f59e00640 1 -- start start 2024-09-21T00:31:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f59e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54103910 0x7f7f54075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f59e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f54109240 0x7f7f54075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f59e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f54109b90 0x7f7f54076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f59e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7f541138a0 con 0x7f7f54103910 2024-09-21T00:31:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f59e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7f54113720 con 0x7f7f54109240 2024-09-21T00:31:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f59e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7f54113a20 con 0x7f7f54109b90 2024-09-21T00:31:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f58a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54103910 0x7f7f54075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f58a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54103910 0x7f7f54075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51180/0 (socket says 172.21.15.27:51180) 2024-09-21T00:31:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f58a00640 1 -- 172.21.15.27:0/784499029 learned_addr learned my addr 172.21.15.27:0/784499029 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f59400640 1 --2- 172.21.15.27:0/784499029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f54109b90 0x7f7f54076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f53e00640 1 --2- 172.21.15.27:0/784499029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f54109240 0x7f7f54075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f58a00640 1 -- 172.21.15.27:0/784499029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f54109b90 msgr2=0x7f7f54076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f58a00640 1 --2- 172.21.15.27:0/784499029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f54109b90 0x7f7f54076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f58a00640 1 -- 172.21.15.27:0/784499029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f54109240 msgr2=0x7f7f54075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f58a00640 1 --2- 172.21.15.27:0/784499029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f54109240 0x7f7f54075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f58a00640 1 -- 172.21.15.27:0/784499029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f541a8610 con 0x7f7f54103910 2024-09-21T00:31:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f58a00640 1 --2- 172.21.15.27:0/784499029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54103910 0x7f7f54075710 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7f48005ab0 tx=0x7f7f48003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f51600640 1 -- 172.21.15.27:0/784499029 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4803f070 con 0x7f7f54103910 2024-09-21T00:31:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f51600640 1 -- 172.21.15.27:0/784499029 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f48031d60 con 0x7f7f54103910 2024-09-21T00:31:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f59e00640 1 -- 172.21.15.27:0/784499029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f541a88a0 con 0x7f7f54103910 2024-09-21T00:31:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f51600640 1 -- 172.21.15.27:0/784499029 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48035030 con 0x7f7f54103910 2024-09-21T00:31:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f59e00640 1 -- 172.21.15.27:0/784499029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f541a8d90 con 0x7f7f54103910 2024-09-21T00:31:07.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f625e9740 1 Processor -- start 2024-09-21T00:31:07.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.424+0000 7f7f625e9740 1 -- start start 2024-09-21T00:31:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b97201b0 0x5589b9ac4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96f72f0 0x5589b9ac5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b96ff4d0 0x5589b9ac5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5589b9a3a6b0 con 0x5589b97201b0 2024-09-21T00:31:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5589b9a29ae0 con 0x5589b96ff4d0 2024-09-21T00:31:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5589b9a2d8f0 con 0x5589b96f72f0 2024-09-21T00:31:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f51600640 1 -- 172.21.15.27:0/784499029 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7f48042070 con 0x7f7f54103910 2024-09-21T00:31:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f59400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b97201b0 0x5589b9ac4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f59400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b97201b0 0x5589b9ac4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51190/0 (socket says 172.21.15.27:51190) 2024-09-21T00:31:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f59400640 1 -- 172.21.15.27:0/2039149597 learned_addr learned my addr 172.21.15.27:0/2039149597 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f58a00640 1 --2- 172.21.15.27:0/2039149597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96f72f0 0x5589b9ac5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f59400640 1 -- 172.21.15.27:0/2039149597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96f72f0 msgr2=0x5589b9ac5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f59400640 1 --2- 172.21.15.27:0/2039149597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96f72f0 0x5589b9ac5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f59400640 1 -- 172.21.15.27:0/2039149597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b96ff4d0 msgr2=0x5589b9ac5890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:31:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f59400640 1 --2- 172.21.15.27:0/2039149597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b96ff4d0 0x5589b9ac5890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f59400640 1 -- 172.21.15.27:0/2039149597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b99f6780 con 0x5589b97201b0 2024-09-21T00:31:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f59400640 1 --2- 172.21.15.27:0/2039149597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b97201b0 0x5589b9ac4e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009f90 tx=0x7f7f4c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f920666e97cbf91e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f45c00640 1 -- 172.21.15.27:0/2039149597 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c011700 con 0x5589b97201b0 2024-09-21T00:31:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f45c00640 1 -- 172.21.15.27:0/2039149597 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f4c011d50 con 0x5589b97201b0 2024-09-21T00:31:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f45c00640 1 -- 172.21.15.27:0/2039149597 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c019d00 con 0x5589b97201b0 2024-09-21T00:31:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 -- 172.21.15.27:0/2039149597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b97201b0 msgr2=0x5589b9ac4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 --2- 172.21.15.27:0/2039149597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b97201b0 0x5589b9ac4e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009f90 tx=0x7f7f4c010860 comp rx=0 tx=0).stop 2024-09-21T00:31:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 -- 172.21.15.27:0/2039149597 shutdown_connections 2024-09-21T00:31:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 --2- 172.21.15.27:0/2039149597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b96ff4d0 0x5589b9ac5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 --2- 172.21.15.27:0/2039149597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96f72f0 0x5589b9ac5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 --2- 172.21.15.27:0/2039149597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b97201b0 0x5589b9ac4e10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 -- 172.21.15.27:0/2039149597 >> 172.21.15.27:0/2039149597 conn(0x5589b9714300 msgr2=0x5589b9ac2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 -- 172.21.15.27:0/2039149597 shutdown_connections 2024-09-21T00:31:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 -- 172.21.15.27:0/2039149597 wait complete. 2024-09-21T00:31:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f51600640 1 --2- 172.21.15.27:0/784499029 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7f3c0b65b0 0x7f7f3c0b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f51600640 1 -- 172.21.15.27:0/784499029 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f7f48107f00 con 0x7f7f54103910 2024-09-21T00:31:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f53e00640 1 --2- 172.21.15.27:0/784499029 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7f3c0b65b0 0x7f7f3c0b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 -- start start 2024-09-21T00:31:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 Processor -- start 2024-09-21T00:31:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f53e00640 1 --2- 172.21.15.27:0/784499029 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7f3c0b65b0 0x7f7f3c0b8ac0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7f54108cc0 tx=0x7f7f40005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96f72f0 0x5589b9ad03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b96ff4d0 0x5589b9ad0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b97201b0 0x5589b9ad0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5589b9961ab0 con 0x5589b96ff4d0 2024-09-21T00:31:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5589b99d0570 con 0x5589b97201b0 2024-09-21T00:31:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5589b9ad3590 con 0x5589b96f72f0 2024-09-21T00:31:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f625e9740 10 client.0 _ll_register_callbacks cb 0x5589b99df100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:31:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.430+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b96ff4d0 0x5589b9ad0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b96ff4d0 0x5589b9ad0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51200/0 (socket says 172.21.15.27:51200) 2024-09-21T00:31:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f53e00640 1 -- 172.21.15.27:0/4103792576 learned_addr learned my addr 172.21.15.27:0/4103792576 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f58a00640 1 --2- 172.21.15.27:0/4103792576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96f72f0 0x5589b9ad03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f59400640 1 --2- 172.21.15.27:0/4103792576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b97201b0 0x5589b9ad0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f53e00640 1 -- 172.21.15.27:0/4103792576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96f72f0 msgr2=0x5589b9ad03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f53e00640 1 --2- 172.21.15.27:0/4103792576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96f72f0 0x5589b9ad03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f53e00640 1 -- 172.21.15.27:0/4103792576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b97201b0 msgr2=0x5589b9ad0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f53e00640 1 --2- 172.21.15.27:0/4103792576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b97201b0 0x5589b9ad0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f53e00640 1 -- 172.21.15.27:0/4103792576 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589b978b790 con 0x5589b96ff4d0 2024-09-21T00:31:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f53e00640 1 --2- 172.21.15.27:0/4103792576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b96ff4d0 0x5589b9ad0910 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7f40005bc0 tx=0x7f7f40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78c8c65fa7c978b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f3a000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:31:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f3a000640 1 -- 172.21.15.27:0/4103792576 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4001e830 con 0x5589b96ff4d0 2024-09-21T00:31:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 1 -- 172.21.15.27:0/4103792576 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f4c011d50 con 0x5589b96ff4d0 2024-09-21T00:31:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f3a000640 1 -- 172.21.15.27:0/4103792576 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f7f4001e9d0 con 0x5589b96ff4d0 2024-09-21T00:31:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f3a000640 1 -- 172.21.15.27:0/4103792576 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7f4002f050 con 0x5589b96ff4d0 2024-09-21T00:31:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 10 client.16422 fetch_fsmap learned FSMap version 38 2024-09-21T00:31:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 1 -- 172.21.15.27:0/4103792576 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5589b9804620 con 0x5589b96ff4d0 2024-09-21T00:31:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f3a000640 1 -- 172.21.15.27:0/4103792576 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f4c011d50 con 0x5589b96ff4d0 2024-09-21T00:31:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f3a000640 1 -- 172.21.15.27:0/4103792576 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f7f40026c50 con 0x5589b96ff4d0 2024-09-21T00:31:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 10 client.16422 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:31:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 10 client.16422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:31:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 20 client.16422 populate_metadata read hostname 'smithi027' 2024-09-21T00:31:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 1 --2- 172.21.15.27:0/4103792576 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5589b96ded80 0x5589b9ae2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 4 client.16422 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:31:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 1 -- 172.21.15.27:0/4103792576 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589b9a3c600 con 0x5589b96ded80 2024-09-21T00:31:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f58a00640 1 --2- 172.21.15.27:0/4103792576 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5589b96ded80 0x5589b9ae2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f58a00640 1 --2- 172.21.15.27:0/4103792576 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5589b96ded80 0x5589b9ae2570 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4613d9e66cb9ef server_cookie=837faf33d99251d0 in_seq=0 out_seq=0 2024-09-21T00:31:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f3a000640 10 client.16422 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f3a000640 1 -- 172.21.15.27:0/4103792576 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589b9a3c600 con 0x5589b96ded80 2024-09-21T00:31:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f3a000640 10 client.16422 handle_command_reply: tid=0 2024-09-21T00:31:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 1 client.16422 shutdown 2024-09-21T00:31:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 1 -- 172.21.15.27:0/4103792576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b96ff4d0 msgr2=0x5589b9ad0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 1 --2- 172.21.15.27:0/4103792576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b96ff4d0 0x5589b9ad0910 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7f40005bc0 tx=0x7f7f40004730 comp rx=0 tx=0).stop 2024-09-21T00:31:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 1 -- 172.21.15.27:0/4103792576 shutdown_connections 2024-09-21T00:31:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 1 --2- 172.21.15.27:0/4103792576 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5589b96ded80 0x5589b9ae2570 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 1 --2- 172.21.15.27:0/4103792576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b97201b0 0x5589b9ad0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 1 --2- 172.21.15.27:0/4103792576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b96ff4d0 0x5589b9ad0910 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f3a000640 0 client.16422 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 1 --2- 172.21.15.27:0/4103792576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96f72f0 0x5589b9ad03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 1 -- 172.21.15.27:0/4103792576 >> 172.21.15.27:0/4103792576 conn(0x5589b9714300 msgr2=0x5589b9ac5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 1 -- 172.21.15.27:0/4103792576 shutdown_connections 2024-09-21T00:31:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 1 -- 172.21.15.27:0/4103792576 wait complete. 2024-09-21T00:31:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.435+0000 7f7f625e9740 20 client.16422 trim_cache size 0 max 16384 2024-09-21T00:31:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 Processor -- start 2024-09-21T00:31:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 -- start start 2024-09-21T00:31:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b96f72f0 0x5589b9ad2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96ff4d0 0x5589b9acd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b97201b0 0x5589b9acdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7f541b5a40 con 0x5589b97201b0 2024-09-21T00:31:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7f541b32b0 con 0x5589b96f72f0 2024-09-21T00:31:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7f541b1120 con 0x5589b96ff4d0 2024-09-21T00:31:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f58a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b96f72f0 0x5589b9ad2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f58a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b96f72f0 0x5589b9ad2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:56186/0 (socket says 172.21.15.27:56186) 2024-09-21T00:31:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f59400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b97201b0 0x5589b9acdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f58a00640 1 -- 172.21.15.27:0/1367047165 learned_addr learned my addr 172.21.15.27:0/1367047165 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f53e00640 1 --2- 172.21.15.27:0/1367047165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96ff4d0 0x5589b9acd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f58a00640 1 -- 172.21.15.27:0/1367047165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96ff4d0 msgr2=0x5589b9acd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f58a00640 1 --2- 172.21.15.27:0/1367047165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96ff4d0 0x5589b9acd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f58a00640 1 -- 172.21.15.27:0/1367047165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b97201b0 msgr2=0x5589b9acdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f58a00640 1 --2- 172.21.15.27:0/1367047165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b97201b0 0x5589b9acdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f58a00640 1 -- 172.21.15.27:0/1367047165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b9ac7420 con 0x5589b96f72f0 2024-09-21T00:31:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f58a00640 1 --2- 172.21.15.27:0/1367047165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b96f72f0 0x5589b9ad2000 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7f480c9ff0 tx=0x7f7f4803b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efbaffe27d02a703 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f45200640 1 -- 172.21.15.27:0/1367047165 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48049040 con 0x5589b96f72f0 2024-09-21T00:31:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f45200640 1 -- 172.21.15.27:0/1367047165 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f480cd530 con 0x5589b96f72f0 2024-09-21T00:31:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f45200640 1 -- 172.21.15.27:0/1367047165 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f480d7390 con 0x5589b96f72f0 2024-09-21T00:31:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 -- 172.21.15.27:0/1367047165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b96f72f0 msgr2=0x5589b9ad2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 --2- 172.21.15.27:0/1367047165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b96f72f0 0x5589b9ad2000 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7f480c9ff0 tx=0x7f7f4803b460 comp rx=0 tx=0).stop 2024-09-21T00:31:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 -- 172.21.15.27:0/1367047165 shutdown_connections 2024-09-21T00:31:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 --2- 172.21.15.27:0/1367047165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b97201b0 0x5589b9acdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 --2- 172.21.15.27:0/1367047165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96ff4d0 0x5589b9acd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 --2- 172.21.15.27:0/1367047165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b96f72f0 0x5589b9ad2000 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 -- 172.21.15.27:0/1367047165 >> 172.21.15.27:0/1367047165 conn(0x5589b9714300 msgr2=0x5589b9ad18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 -- 172.21.15.27:0/1367047165 shutdown_connections 2024-09-21T00:31:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 -- 172.21.15.27:0/1367047165 wait complete. 2024-09-21T00:31:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 -- start start 2024-09-21T00:31:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 Processor -- start 2024-09-21T00:31:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96f72f0 0x5589b9ad8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b96ff4d0 0x5589b9ad6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b97201b0 0x5589b9ad69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7f541b1770 con 0x5589b96ff4d0 2024-09-21T00:31:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7f541b2be0 con 0x5589b97201b0 2024-09-21T00:31:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7f541b1e40 con 0x5589b96f72f0 2024-09-21T00:31:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f625e9740 10 client.0 _ll_register_callbacks cb 0x7f7f40009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:31:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.454+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b96ff4d0 0x5589b9ad6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f58a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96f72f0 0x5589b9ad8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b96ff4d0 0x5589b9ad6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51222/0 (socket says 172.21.15.27:51222) 2024-09-21T00:31:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f53e00640 1 -- 172.21.15.27:0/389801564 learned_addr learned my addr 172.21.15.27:0/389801564 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f53e00640 1 -- 172.21.15.27:0/389801564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96f72f0 msgr2=0x5589b9ad8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f53e00640 1 --2- 172.21.15.27:0/389801564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96f72f0 0x5589b9ad8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f53e00640 1 -- 172.21.15.27:0/389801564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b97201b0 msgr2=0x5589b9ad69a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:31:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f53e00640 1 --2- 172.21.15.27:0/389801564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b97201b0 0x5589b9ad69a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f53e00640 1 -- 172.21.15.27:0/389801564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589b9ad7060 con 0x5589b96ff4d0 2024-09-21T00:31:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f53e00640 1 --2- 172.21.15.27:0/389801564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b96ff4d0 0x5589b9ad6460 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7f4001c400 tx=0x7f7f400040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c3bb2edd60a5c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f3a000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:31:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f3a000640 1 -- 172.21.15.27:0/389801564 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40039070 con 0x5589b96ff4d0 2024-09-21T00:31:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f3a000640 1 -- 172.21.15.27:0/389801564 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f7f4001ca90 con 0x5589b96ff4d0 2024-09-21T00:31:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f625e9740 1 -- 172.21.15.27:0/389801564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f480cd530 con 0x5589b96ff4d0 2024-09-21T00:31:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f3a000640 1 -- 172.21.15.27:0/389801564 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7f40046050 con 0x5589b96ff4d0 2024-09-21T00:31:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f625e9740 10 client.16428 fetch_fsmap learned FSMap version 38 2024-09-21T00:31:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f625e9740 1 -- 172.21.15.27:0/389801564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5589b9b4cde0 con 0x5589b96ff4d0 2024-09-21T00:31:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f3a000640 1 -- 172.21.15.27:0/389801564 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f480cd530 con 0x5589b96ff4d0 2024-09-21T00:31:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f3a000640 1 -- 172.21.15.27:0/389801564 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f7f400048c0 con 0x5589b96ff4d0 2024-09-21T00:31:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f625e9740 10 client.16428 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:31:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f625e9740 10 client.16428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:31:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f625e9740 20 client.16428 populate_metadata read hostname 'smithi027' 2024-09-21T00:31:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f625e9740 1 --2- 172.21.15.27:0/389801564 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5589b96b41c0 0x5589b9b4f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f625e9740 4 client.16428 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:31:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f625e9740 1 -- 172.21.15.27:0/389801564 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589b9a380b0 con 0x5589b96b41c0 2024-09-21T00:31:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f58a00640 1 --2- 172.21.15.27:0/389801564 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5589b96b41c0 0x5589b9b4f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f58a00640 1 --2- 172.21.15.27:0/389801564 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5589b96b41c0 0x5589b9b4f540 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf13bbf854ea22b server_cookie=c4a1ef21b4309e5e in_seq=0 out_seq=0 2024-09-21T00:31:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.459+0000 7f7f3a000640 10 client.16428 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f3a000640 1 -- 172.21.15.27:0/389801564 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589b9a380b0 con 0x5589b96b41c0 2024-09-21T00:31:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f3a000640 10 client.16428 handle_command_reply: tid=0 2024-09-21T00:31:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f625e9740 1 client.16428 shutdown 2024-09-21T00:31:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f625e9740 1 -- 172.21.15.27:0/389801564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b96ff4d0 msgr2=0x5589b9ad6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f625e9740 1 --2- 172.21.15.27:0/389801564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b96ff4d0 0x5589b9ad6460 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7f4001c400 tx=0x7f7f400040e0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f625e9740 1 -- 172.21.15.27:0/389801564 shutdown_connections 2024-09-21T00:31:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f625e9740 1 --2- 172.21.15.27:0/389801564 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5589b96b41c0 0x5589b9b4f540 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f625e9740 1 --2- 172.21.15.27:0/389801564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589b97201b0 0x5589b9ad69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f3a000640 0 client.16428 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f625e9740 1 --2- 172.21.15.27:0/389801564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5589b96ff4d0 0x5589b9ad6460 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f625e9740 1 --2- 172.21.15.27:0/389801564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5589b96f72f0 0x5589b9ad8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f625e9740 1 -- 172.21.15.27:0/389801564 >> 172.21.15.27:0/389801564 conn(0x5589b9714300 msgr2=0x5589b9ac1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f625e9740 1 -- 172.21.15.27:0/389801564 shutdown_connections 2024-09-21T00:31:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f625e9740 1 -- 172.21.15.27:0/389801564 wait complete. 2024-09-21T00:31:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f625e9740 20 client.16428 trim_cache size 0 max 16384 2024-09-21T00:31:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:31:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:31:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:31:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:31:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f59e00640 1 -- 172.21.15.27:0/784499029 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7f3c0b65b0 msgr2=0x7f7f3c0b8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f59e00640 1 --2- 172.21.15.27:0/784499029 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7f3c0b65b0 0x7f7f3c0b8ac0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7f54108cc0 tx=0x7f7f40005f90 comp rx=0 tx=0).stop 2024-09-21T00:31:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f59e00640 1 -- 172.21.15.27:0/784499029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54103910 msgr2=0x7f7f54075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f59e00640 1 --2- 172.21.15.27:0/784499029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54103910 0x7f7f54075710 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7f48005ab0 tx=0x7f7f48003c80 comp rx=0 tx=0).stop 2024-09-21T00:31:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f59e00640 1 -- 172.21.15.27:0/784499029 shutdown_connections 2024-09-21T00:31:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f59e00640 1 --2- 172.21.15.27:0/784499029 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7f3c0b65b0 0x7f7f3c0b8ac0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f59e00640 1 --2- 172.21.15.27:0/784499029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f54109b90 0x7f7f54076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f59e00640 1 --2- 172.21.15.27:0/784499029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f54109240 0x7f7f54075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f59e00640 1 --2- 172.21.15.27:0/784499029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54103910 0x7f7f54075710 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f59e00640 1 -- 172.21.15.27:0/784499029 >> 172.21.15.27:0/784499029 conn(0x7f7f540ff390 msgr2=0x7f7f541017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f59e00640 1 -- 172.21.15.27:0/784499029 shutdown_connections 2024-09-21T00:31:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:07.477+0000 7f7f59e00640 1 -- 172.21.15.27:0/784499029 wait complete. 2024-09-21T00:31:08.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:07 smithi027 ceph-mon[30745]: pgmap v678: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 7 op/s 2024-09-21T00:31:08.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:07 smithi089 ceph-mon[34481]: pgmap v678: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 7 op/s 2024-09-21T00:31:08.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:07 smithi098 ceph-mon[34725]: pgmap v678: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 7 op/s 2024-09-21T00:31:10.045 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:09 smithi027 ceph-mon[30745]: pgmap v679: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 7 op/s 2024-09-21T00:31:10.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:09 smithi089 ceph-mon[34481]: pgmap v679: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 7 op/s 2024-09-21T00:31:10.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:09 smithi098 ceph-mon[34725]: pgmap v679: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 7 op/s 2024-09-21T00:31:12.017 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:11 smithi027 ceph-mon[30745]: pgmap v680: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-21T00:31:12.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:11 smithi089 ceph-mon[34481]: pgmap v680: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-21T00:31:12.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:11 smithi098 ceph-mon[34725]: pgmap v680: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-21T00:31:14.106 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:13 smithi027 ceph-mon[30745]: pgmap v681: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 7 op/s 2024-09-21T00:31:14.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:13 smithi089 ceph-mon[34481]: pgmap v681: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 7 op/s 2024-09-21T00:31:14.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:13 smithi098 ceph-mon[34725]: pgmap v681: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 7 op/s 2024-09-21T00:31:16.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:15 smithi027 ceph-mon[30745]: pgmap v682: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 5 op/s 2024-09-21T00:31:16.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:15 smithi089 ceph-mon[34481]: pgmap v682: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 5 op/s 2024-09-21T00:31:16.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:15 smithi098 ceph-mon[34725]: pgmap v682: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 5 op/s 2024-09-21T00:31:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:17 smithi027 ceph-mon[30745]: pgmap v683: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:18.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:17 smithi089 ceph-mon[34481]: pgmap v683: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:18.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:17 smithi098 ceph-mon[34725]: pgmap v683: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:19 smithi089 ceph-mon[34481]: pgmap v684: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:20.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:19 smithi098 ceph-mon[34725]: pgmap v684: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:20.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:19 smithi027 ceph-mon[30745]: pgmap v684: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.712+0000 7fbe21200640 1 -- 172.21.15.27:0/2777438101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c104740 msgr2=0x7fbe1c104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.712+0000 7fbe21200640 1 --2- 172.21.15.27:0/2777438101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c104740 0x7fbe1c104b50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009900 tx=0x7fbe0c02a410 comp rx=0 tx=0).stop 2024-09-21T00:31:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.712+0000 7fbe21200640 1 -- 172.21.15.27:0/2777438101 shutdown_connections 2024-09-21T00:31:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.712+0000 7fbe21200640 1 --2- 172.21.15.27:0/2777438101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe1c10dba0 0x7fbe1c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.712+0000 7fbe21200640 1 --2- 172.21.15.27:0/2777438101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe1c105090 0x7fbe1c10d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.712+0000 7fbe21200640 1 --2- 172.21.15.27:0/2777438101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c104740 0x7fbe1c104b50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.712+0000 7fbe21200640 1 -- 172.21.15.27:0/2777438101 >> 172.21.15.27:0/2777438101 conn(0x7fbe1c0fe210 msgr2=0x7fbe1c100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.712+0000 7fbe21200640 1 -- 172.21.15.27:0/2777438101 shutdown_connections 2024-09-21T00:31:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.712+0000 7fbe21200640 1 -- 172.21.15.27:0/2777438101 wait complete. 2024-09-21T00:31:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.712+0000 7fbe21200640 1 Processor -- start 2024-09-21T00:31:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe21200640 1 -- start start 2024-09-21T00:31:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe21200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe1c104740 0x7fbe1c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe21200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c105090 0x7fbe1c19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe21200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe1c10dba0 0x7fbe1c19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe21200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c113800 con 0x7fbe1c105090 2024-09-21T00:31:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe21200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c113680 con 0x7fbe1c104740 2024-09-21T00:31:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe21200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c113980 con 0x7fbe1c10dba0 2024-09-21T00:31:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe1aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c105090 0x7fbe1c19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe1aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c105090 0x7fbe1c19a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58174/0 (socket says 172.21.15.27:58174) 2024-09-21T00:31:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe1aa00640 1 -- 172.21.15.27:0/4018387933 learned_addr learned my addr 172.21.15.27:0/4018387933 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe1be00640 1 --2- 172.21.15.27:0/4018387933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe1c10dba0 0x7fbe1c19fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe1aa00640 1 -- 172.21.15.27:0/4018387933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe1c10dba0 msgr2=0x7fbe1c19fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe1aa00640 1 --2- 172.21.15.27:0/4018387933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe1c10dba0 0x7fbe1c19fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe1aa00640 1 -- 172.21.15.27:0/4018387933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe1c104740 msgr2=0x7fbe1c19a340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:31:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe1aa00640 1 --2- 172.21.15.27:0/4018387933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe1c104740 0x7fbe1c19a340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe1aa00640 1 -- 172.21.15.27:0/4018387933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1c1a0270 con 0x7fbe1c105090 2024-09-21T00:31:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe1aa00640 1 --2- 172.21.15.27:0/4018387933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c105090 0x7fbe1c19a8e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800e9f0 tx=0x7fbe0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe13e00640 1 -- 172.21.15.27:0/4018387933 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0800ccc0 con 0x7fbe1c105090 2024-09-21T00:31:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe13e00640 1 -- 172.21.15.27:0/4018387933 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe080108e0 con 0x7fbe1c105090 2024-09-21T00:31:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe13e00640 1 -- 172.21.15.27:0/4018387933 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08018860 con 0x7fbe1c105090 2024-09-21T00:31:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe21200640 1 -- 172.21.15.27:0/4018387933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1c1a0560 con 0x7fbe1c105090 2024-09-21T00:31:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe21200640 1 -- 172.21.15.27:0/4018387933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe1c1a6cf0 con 0x7fbe1c105090 2024-09-21T00:31:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe298bf740 1 Processor -- start 2024-09-21T00:31:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe298bf740 1 -- start start 2024-09-21T00:31:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe298bf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db7b1b0 0x560f4df1fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe298bf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db522f0 0x560f4df20350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe298bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db5a4d0 0x560f4df20890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe298bf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560f4de956b0 con 0x560f4db522f0 2024-09-21T00:31:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe298bf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560f4de84ae0 con 0x560f4db5a4d0 2024-09-21T00:31:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe298bf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560f4de888f0 con 0x560f4db7b1b0 2024-09-21T00:31:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe1b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db522f0 0x560f4df20350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.714+0000 7fbe1be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db7b1b0 0x560f4df1fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe1be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db7b1b0 0x560f4df1fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:45504/0 (socket says 172.21.15.27:45504) 2024-09-21T00:31:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe1be00640 1 -- 172.21.15.27:0/2953949240 learned_addr learned my addr 172.21.15.27:0/2953949240 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe1be00640 1 -- 172.21.15.27:0/2953949240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db5a4d0 msgr2=0x560f4df20890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:31:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe1be00640 1 --2- 172.21.15.27:0/2953949240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db5a4d0 0x560f4df20890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe1be00640 1 -- 172.21.15.27:0/2953949240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db522f0 msgr2=0x560f4df20350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe1be00640 1 --2- 172.21.15.27:0/2953949240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db522f0 0x560f4df20350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe1be00640 1 -- 172.21.15.27:0/2953949240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f4de51780 con 0x560f4db7b1b0 2024-09-21T00:31:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe1be00640 1 --2- 172.21.15.27:0/2953949240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db7b1b0 0x560f4df1fe10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009f80 tx=0x7fbe140118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=833f0223ae511aee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe13e00640 1 -- 172.21.15.27:0/4018387933 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fbe08010430 con 0x7fbe1c105090 2024-09-21T00:31:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe10c00640 1 -- 172.21.15.27:0/2953949240 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14004b20 con 0x560f4db7b1b0 2024-09-21T00:31:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe10c00640 1 -- 172.21.15.27:0/2953949240 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe14004cc0 con 0x560f4db7b1b0 2024-09-21T00:31:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe10c00640 1 -- 172.21.15.27:0/2953949240 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14005420 con 0x560f4db7b1b0 2024-09-21T00:31:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 -- 172.21.15.27:0/2953949240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db7b1b0 msgr2=0x560f4df1fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2953949240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db7b1b0 0x560f4df1fe10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009f80 tx=0x7fbe140118d0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 -- 172.21.15.27:0/2953949240 shutdown_connections 2024-09-21T00:31:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2953949240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db5a4d0 0x560f4df20890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2953949240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db522f0 0x560f4df20350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2953949240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db7b1b0 0x560f4df1fe10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 -- 172.21.15.27:0/2953949240 >> 172.21.15.27:0/2953949240 conn(0x560f4db6f300 msgr2=0x560f4df1dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 -- 172.21.15.27:0/2953949240 shutdown_connections 2024-09-21T00:31:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 -- 172.21.15.27:0/2953949240 wait complete. 2024-09-21T00:31:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 -- start start 2024-09-21T00:31:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe13e00640 1 --2- 172.21.15.27:0/4018387933 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbe000b6580 0x7fbe000b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 Processor -- start 2024-09-21T00:31:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe13e00640 1 -- 172.21.15.27:0/4018387933 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7fbe080e7e90 con 0x7fbe1c105090 2024-09-21T00:31:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe1be00640 1 --2- 172.21.15.27:0/4018387933 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbe000b6580 0x7fbe000b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db522f0 0x560f4df2b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db5a4d0 0x560f4df2b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db7b1b0 0x560f4df2be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560f4ddbcab0 con 0x560f4db522f0 2024-09-21T00:31:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560f4de2b570 con 0x560f4db5a4d0 2024-09-21T00:31:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560f4df2e590 con 0x560f4db7b1b0 2024-09-21T00:31:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe298bf740 10 client.0 _ll_register_callbacks cb 0x560f4de3a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:31:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe1b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db522f0 0x560f4df2b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe1b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db522f0 0x560f4df2b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58182/0 (socket says 172.21.15.27:58182) 2024-09-21T00:31:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe1b400640 1 -- 172.21.15.27:0/2465930521 learned_addr learned my addr 172.21.15.27:0/2465930521 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe1be00640 1 --2- 172.21.15.27:0/4018387933 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbe000b6580 0x7fbe000b8a90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400a010 tx=0x7fbe14005fd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.720+0000 7fbe1aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db5a4d0 0x560f4df2b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.725+0000 7fbe1b400640 1 -- 172.21.15.27:0/2465930521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db7b1b0 msgr2=0x560f4df2be10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:31:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.725+0000 7fbe1b400640 1 --2- 172.21.15.27:0/2465930521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db7b1b0 0x560f4df2be10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.725+0000 7fbe1b400640 1 -- 172.21.15.27:0/2465930521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db5a4d0 msgr2=0x560f4df2b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.725+0000 7fbe1b400640 1 --2- 172.21.15.27:0/2465930521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db5a4d0 0x560f4df2b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.725+0000 7fbe1b400640 1 -- 172.21.15.27:0/2465930521 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f4dbe6790 con 0x560f4db522f0 2024-09-21T00:31:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.725+0000 7fbe1aa00640 1 --2- 172.21.15.27:0/2465930521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db5a4d0 0x560f4df2b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:31:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.725+0000 7fbe1b400640 1 --2- 172.21.15.27:0/2465930521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db522f0 0x560f4df2b390 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009900 tx=0x7fbe0c032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3652eb9e22c9c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.725+0000 7fbe05600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:31:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.725+0000 7fbe05600640 1 -- 172.21.15.27:0/2465930521 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c040070 con 0x560f4db522f0 2024-09-21T00:31:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.725+0000 7fbe05600640 1 -- 172.21.15.27:0/2465930521 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7fbe0c0318b0 con 0x560f4db522f0 2024-09-21T00:31:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.725+0000 7fbe298bf740 1 -- 172.21.15.27:0/2465930521 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe14004cc0 con 0x560f4db522f0 2024-09-21T00:31:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.725+0000 7fbe05600640 1 -- 172.21.15.27:0/2465930521 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbe0c04e050 con 0x560f4db522f0 2024-09-21T00:31:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.725+0000 7fbe298bf740 10 client.16440 fetch_fsmap learned FSMap version 38 2024-09-21T00:31:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.725+0000 7fbe298bf740 1 -- 172.21.15.27:0/2465930521 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560f4dc5f620 con 0x560f4db522f0 2024-09-21T00:31:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe05600640 1 -- 172.21.15.27:0/2465930521 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe14004cc0 con 0x560f4db522f0 2024-09-21T00:31:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe05600640 1 -- 172.21.15.27:0/2465930521 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fbe0c0046c0 con 0x560f4db522f0 2024-09-21T00:31:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 10 client.16440 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:31:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:31:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 20 client.16440 populate_metadata read hostname 'smithi027' 2024-09-21T00:31:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2465930521 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560f4db39d80 0x560f4df3d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 4 client.16440 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:31:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 1 -- 172.21.15.27:0/2465930521 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f4de97600 con 0x560f4db39d80 2024-09-21T00:31:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe1aa00640 1 --2- 172.21.15.27:0/2465930521 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560f4db39d80 0x560f4df3d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe1aa00640 1 --2- 172.21.15.27:0/2465930521 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560f4db39d80 0x560f4df3d580 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=439c9041291441bd server_cookie=550ddfe6af9ae2c in_seq=0 out_seq=0 2024-09-21T00:31:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe05600640 10 client.16440 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe05600640 1 -- 172.21.15.27:0/2465930521 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560f4de97600 con 0x560f4db39d80 2024-09-21T00:31:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe05600640 10 client.16440 handle_command_reply: tid=0 2024-09-21T00:31:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 1 client.16440 shutdown 2024-09-21T00:31:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 1 -- 172.21.15.27:0/2465930521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db522f0 msgr2=0x560f4df2b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2465930521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db522f0 0x560f4df2b390 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009900 tx=0x7fbe0c032d50 comp rx=0 tx=0).stop 2024-09-21T00:31:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 1 -- 172.21.15.27:0/2465930521 shutdown_connections 2024-09-21T00:31:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2465930521 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560f4db39d80 0x560f4df3d580 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2465930521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db7b1b0 0x560f4df2be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2465930521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db5a4d0 0x560f4df2b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2465930521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db522f0 0x560f4df2b390 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe05600640 0 client.16440 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 1 -- 172.21.15.27:0/2465930521 >> 172.21.15.27:0/2465930521 conn(0x560f4db6f300 msgr2=0x560f4df20e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 1 -- 172.21.15.27:0/2465930521 shutdown_connections 2024-09-21T00:31:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 1 -- 172.21.15.27:0/2465930521 wait complete. 2024-09-21T00:31:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.727+0000 7fbe298bf740 20 client.16440 trim_cache size 0 max 16384 2024-09-21T00:31:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 Processor -- start 2024-09-21T00:31:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 -- start start 2024-09-21T00:31:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db522f0 0x560f4df1cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db5a4d0 0x560f4df21550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db7b1b0 0x560f4df28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1b3820 con 0x560f4db7b1b0 2024-09-21T00:31:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1b1090 con 0x560f4db5a4d0 2024-09-21T00:31:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1aef00 con 0x560f4db522f0 2024-09-21T00:31:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe1b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db522f0 0x560f4df1cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe1be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db7b1b0 0x560f4df28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe1b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db522f0 0x560f4df1cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:45514/0 (socket says 172.21.15.27:45514) 2024-09-21T00:31:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe1b400640 1 -- 172.21.15.27:0/3310418835 learned_addr learned my addr 172.21.15.27:0/3310418835 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe1aa00640 1 --2- 172.21.15.27:0/3310418835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db5a4d0 0x560f4df21550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe1b400640 1 -- 172.21.15.27:0/3310418835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db5a4d0 msgr2=0x560f4df21550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe1b400640 1 --2- 172.21.15.27:0/3310418835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db5a4d0 0x560f4df21550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe1b400640 1 -- 172.21.15.27:0/3310418835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db7b1b0 msgr2=0x560f4df28580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe1b400640 1 --2- 172.21.15.27:0/3310418835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db7b1b0 0x560f4df28580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe1b400640 1 -- 172.21.15.27:0/3310418835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f4df20bf0 con 0x560f4db522f0 2024-09-21T00:31:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe1b400640 1 --2- 172.21.15.27:0/3310418835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db522f0 0x560f4df1cd20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c006010 tx=0x7fbe0c02ab10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=997a579ff8369d39 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe07e00640 1 -- 172.21.15.27:0/3310418835 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c056020 con 0x560f4db522f0 2024-09-21T00:31:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe07e00640 1 -- 172.21.15.27:0/3310418835 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe0c032dc0 con 0x560f4db522f0 2024-09-21T00:31:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe07e00640 1 -- 172.21.15.27:0/3310418835 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c040040 con 0x560f4db522f0 2024-09-21T00:31:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 -- 172.21.15.27:0/3310418835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db522f0 msgr2=0x560f4df1cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 --2- 172.21.15.27:0/3310418835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db522f0 0x560f4df1cd20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c006010 tx=0x7fbe0c02ab10 comp rx=0 tx=0).stop 2024-09-21T00:31:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 -- 172.21.15.27:0/3310418835 shutdown_connections 2024-09-21T00:31:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 --2- 172.21.15.27:0/3310418835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db7b1b0 0x560f4df28580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 --2- 172.21.15.27:0/3310418835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db5a4d0 0x560f4df21550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 --2- 172.21.15.27:0/3310418835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db522f0 0x560f4df1cd20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 -- 172.21.15.27:0/3310418835 >> 172.21.15.27:0/3310418835 conn(0x560f4db6f300 msgr2=0x560f4df20e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 -- 172.21.15.27:0/3310418835 shutdown_connections 2024-09-21T00:31:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 -- 172.21.15.27:0/3310418835 wait complete. 2024-09-21T00:31:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 -- start start 2024-09-21T00:31:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.744+0000 7fbe298bf740 1 Processor -- start 2024-09-21T00:31:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe298bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db522f0 0x560f4df33d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe298bf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db5a4d0 0x560f4df31250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe298bf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db7b1b0 0x560f4df31790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe298bf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1af550 con 0x560f4db7b1b0 2024-09-21T00:31:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe298bf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1b09c0 con 0x560f4db522f0 2024-09-21T00:31:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe298bf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1afc20 con 0x560f4db5a4d0 2024-09-21T00:31:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe298bf740 10 client.0 _ll_register_callbacks cb 0x7fbe140093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:31:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe1b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db522f0 0x560f4df33d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe1aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db5a4d0 0x560f4df31250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe1b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db522f0 0x560f4df33d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45956/0 (socket says 172.21.15.27:45956) 2024-09-21T00:31:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe1aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db5a4d0 0x560f4df31250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:45528/0 (socket says 172.21.15.27:45528) 2024-09-21T00:31:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe1b400640 1 -- 172.21.15.27:0/2584621610 learned_addr learned my addr 172.21.15.27:0/2584621610 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe1be00640 1 --2- 172.21.15.27:0/2584621610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db7b1b0 0x560f4df31790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe1b400640 1 -- 172.21.15.27:0/2584621610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db5a4d0 msgr2=0x560f4df31250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe1b400640 1 --2- 172.21.15.27:0/2584621610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db5a4d0 0x560f4df31250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.749+0000 7fbe1b400640 1 -- 172.21.15.27:0/2584621610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db7b1b0 msgr2=0x560f4df31790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe1b400640 1 --2- 172.21.15.27:0/2584621610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db7b1b0 0x560f4df31790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe1b400640 1 -- 172.21.15.27:0/2584621610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f4df31cd0 con 0x560f4db522f0 2024-09-21T00:31:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe1b400640 1 --2- 172.21.15.27:0/2584621610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db522f0 0x560f4df33d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009900 tx=0x7fbe0c0316b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7a5adf13dea5a69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe05600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:31:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe05600640 1 -- 172.21.15.27:0/2584621610 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c056030 con 0x560f4db522f0 2024-09-21T00:31:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe298bf740 1 -- 172.21.15.27:0/2584621610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe0c032dc0 con 0x560f4db522f0 2024-09-21T00:31:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe05600640 1 -- 172.21.15.27:0/2584621610 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7fbe0c004cb0 con 0x560f4db522f0 2024-09-21T00:31:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe05600640 1 -- 172.21.15.27:0/2584621610 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbe0c06a050 con 0x560f4db522f0 2024-09-21T00:31:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe298bf740 10 client.26426 fetch_fsmap learned FSMap version 38 2024-09-21T00:31:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe298bf740 1 -- 172.21.15.27:0/2584621610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560f4dfac040 con 0x560f4db522f0 2024-09-21T00:31:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe05600640 1 -- 172.21.15.27:0/2584621610 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe0c032dc0 con 0x560f4db522f0 2024-09-21T00:31:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe05600640 1 -- 172.21.15.27:0/2584621610 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fbe0c045700 con 0x560f4db522f0 2024-09-21T00:31:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe298bf740 10 client.26426 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:31:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe298bf740 10 client.26426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:31:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe298bf740 20 client.26426 populate_metadata read hostname 'smithi027' 2024-09-21T00:31:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2584621610 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560f4db0f1c0 0x560f4dfae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe298bf740 4 client.26426 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:31:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe298bf740 1 -- 172.21.15.27:0/2584621610 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f4de930b0 con 0x560f4db0f1c0 2024-09-21T00:31:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe1aa00640 1 --2- 172.21.15.27:0/2584621610 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560f4db0f1c0 0x560f4dfae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe1aa00640 1 --2- 172.21.15.27:0/2584621610 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560f4db0f1c0 0x560f4dfae6e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ffcb3a46efa73b2 server_cookie=6c00c86e1ecbf0d3 in_seq=0 out_seq=0 2024-09-21T00:31:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.751+0000 7fbe05600640 10 client.26426 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe05600640 1 -- 172.21.15.27:0/2584621610 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f4de930b0 con 0x560f4db0f1c0 2024-09-21T00:31:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe05600640 10 client.26426 handle_command_reply: tid=0 2024-09-21T00:31:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe298bf740 1 client.26426 shutdown 2024-09-21T00:31:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe298bf740 1 -- 172.21.15.27:0/2584621610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db522f0 msgr2=0x560f4df33d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2584621610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db522f0 0x560f4df33d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009900 tx=0x7fbe0c0316b0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe298bf740 1 -- 172.21.15.27:0/2584621610 shutdown_connections 2024-09-21T00:31:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2584621610 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x560f4db0f1c0 0x560f4dfae6e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2584621610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560f4db7b1b0 0x560f4df31790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2584621610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f4db5a4d0 0x560f4df31250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe05600640 0 client.26426 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe298bf740 1 --2- 172.21.15.27:0/2584621610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4db522f0 0x560f4df33d90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe298bf740 1 -- 172.21.15.27:0/2584621610 >> 172.21.15.27:0/2584621610 conn(0x560f4db6f300 msgr2=0x560f4df1caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe298bf740 1 -- 172.21.15.27:0/2584621610 shutdown_connections 2024-09-21T00:31:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe298bf740 1 -- 172.21.15.27:0/2584621610 wait complete. 2024-09-21T00:31:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe298bf740 20 client.26426 trim_cache size 0 max 16384 2024-09-21T00:31:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:31:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:31:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:31:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:31:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe21200640 1 -- 172.21.15.27:0/4018387933 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbe000b6580 msgr2=0x7fbe000b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe21200640 1 --2- 172.21.15.27:0/4018387933 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbe000b6580 0x7fbe000b8a90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400a010 tx=0x7fbe14005fd0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe21200640 1 -- 172.21.15.27:0/4018387933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c105090 msgr2=0x7fbe1c19a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe21200640 1 --2- 172.21.15.27:0/4018387933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c105090 0x7fbe1c19a8e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800e9f0 tx=0x7fbe0800eeb0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe21200640 1 -- 172.21.15.27:0/4018387933 shutdown_connections 2024-09-21T00:31:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe21200640 1 --2- 172.21.15.27:0/4018387933 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbe000b6580 0x7fbe000b8a90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe21200640 1 --2- 172.21.15.27:0/4018387933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe1c10dba0 0x7fbe1c19fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe21200640 1 --2- 172.21.15.27:0/4018387933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c105090 0x7fbe1c19a8e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe21200640 1 --2- 172.21.15.27:0/4018387933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe1c104740 0x7fbe1c19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe21200640 1 -- 172.21.15.27:0/4018387933 >> 172.21.15.27:0/4018387933 conn(0x7fbe1c0fe210 msgr2=0x7fbe1c1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe21200640 1 -- 172.21.15.27:0/4018387933 shutdown_connections 2024-09-21T00:31:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:20.791+0000 7fbe21200640 1 -- 172.21.15.27:0/4018387933 wait complete. 2024-09-21T00:31:22.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:22 smithi027 ceph-mon[30745]: pgmap v685: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:22.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:22 smithi089 ceph-mon[34481]: pgmap v685: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:22.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:22 smithi098 ceph-mon[34725]: pgmap v685: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 6 op/s 2024-09-21T00:31:24.418 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:24 smithi027 ceph-mon[30745]: pgmap v686: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 6 op/s 2024-09-21T00:31:24.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:24 smithi089 ceph-mon[34481]: pgmap v686: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 6 op/s 2024-09-21T00:31:24.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:24 smithi098 ceph-mon[34725]: pgmap v686: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 6 op/s 2024-09-21T00:31:26.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:26 smithi027 ceph-mon[30745]: pgmap v687: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-09-21T00:31:26.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:26 smithi089 ceph-mon[34481]: pgmap v687: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-09-21T00:31:26.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:26 smithi098 ceph-mon[34725]: pgmap v687: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-09-21T00:31:28.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:28 smithi027 ceph-mon[30745]: pgmap v688: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-21T00:31:28.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:28 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:31:28.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:28 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:28.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:28 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:28 smithi089 ceph-mon[34481]: pgmap v688: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-21T00:31:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:28 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:31:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:28 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:28.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:28 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:28.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:28 smithi098 ceph-mon[34725]: pgmap v688: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-21T00:31:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:28 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:31:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:28 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:28.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:28 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:30.156 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:29 smithi027 ceph-mon[30745]: pgmap v689: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 25 op/s 2024-09-21T00:31:30.156 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:30.156 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:30.156 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:30.156 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:29 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:30.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:29 smithi089 ceph-mon[34481]: pgmap v689: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 25 op/s 2024-09-21T00:31:30.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:30.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:30.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:30.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:29 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:30.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:29 smithi098 ceph-mon[34725]: pgmap v689: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 25 op/s 2024-09-21T00:31:30.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:30.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:30.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:30.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:29 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:32.132 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:31 smithi027 ceph-mon[30745]: pgmap v690: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-09-21T00:31:32.132 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:31 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:32.132 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:31 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:32.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:31 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:31:32.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:31 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:31:32.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:31 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:32.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:31 smithi089 ceph-mon[34481]: pgmap v690: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-09-21T00:31:32.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:31 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:32.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:31 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:32.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:31 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:31:32.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:31 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:31:32.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:31 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:32.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:31 smithi098 ceph-mon[34725]: pgmap v690: 129 pgs: 129 active+clean; 976 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-09-21T00:31:32.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:31 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:32.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:31 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:32.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:31 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:31:32.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:31 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:31:32.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:31 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:31:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.002+0000 7f2d3c400640 1 -- 172.21.15.27:0/159789481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d34105d00 msgr2=0x7f2d34106110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.002+0000 7f2d3c400640 1 --2- 172.21.15.27:0/159789481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d34105d00 0x7f2d34106110 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009900 tx=0x7f2d2802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.002+0000 7f2d3c400640 1 -- 172.21.15.27:0/159789481 shutdown_connections 2024-09-21T00:31:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.002+0000 7f2d3c400640 1 --2- 172.21.15.27:0/159789481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d3410bf80 0x7f2d341127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.002+0000 7f2d3c400640 1 --2- 172.21.15.27:0/159789481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d3410b630 0x7f2d3410ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.002+0000 7f2d3c400640 1 --2- 172.21.15.27:0/159789481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d34105d00 0x7f2d34106110 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.002+0000 7f2d3c400640 1 -- 172.21.15.27:0/159789481 >> 172.21.15.27:0/159789481 conn(0x7f2d34101720 msgr2=0x7f2d34103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.003+0000 7f2d3c400640 1 -- 172.21.15.27:0/159789481 shutdown_connections 2024-09-21T00:31:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.003+0000 7f2d3c400640 1 -- 172.21.15.27:0/159789481 wait complete. 2024-09-21T00:31:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.004+0000 7f2d3c400640 1 Processor -- start 2024-09-21T00:31:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.004+0000 7f2d3c400640 1 -- start start 2024-09-21T00:31:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.004+0000 7f2d3c400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d34105d00 0x7f2d3419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.004+0000 7f2d3c400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d3410b630 0x7f2d3419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.004+0000 7f2d3c400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d3410bf80 0x7f2d341a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.004+0000 7f2d3c400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2d34115c90 con 0x7f2d3410bf80 2024-09-21T00:31:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.004+0000 7f2d3c400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2d34115b10 con 0x7f2d3410b630 2024-09-21T00:31:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.004+0000 7f2d3c400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2d34115e10 con 0x7f2d34105d00 2024-09-21T00:31:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.004+0000 7f2d39a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d34105d00 0x7f2d3419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.005+0000 7f2d3a400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d3410bf80 0x7f2d341a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.005+0000 7f2d39a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d34105d00 0x7f2d3419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:48732/0 (socket says 172.21.15.27:48732) 2024-09-21T00:31:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.005+0000 7f2d39a00640 1 -- 172.21.15.27:0/1065915391 learned_addr learned my addr 172.21.15.27:0/1065915391 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.005+0000 7f2d39000640 1 --2- 172.21.15.27:0/1065915391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d3410b630 0x7f2d3419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.005+0000 7f2d39a00640 1 -- 172.21.15.27:0/1065915391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d3410b630 msgr2=0x7f2d3419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.005+0000 7f2d39a00640 1 --2- 172.21.15.27:0/1065915391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d3410b630 0x7f2d3419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.005+0000 7f2d39a00640 1 -- 172.21.15.27:0/1065915391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d3410bf80 msgr2=0x7f2d341a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.005+0000 7f2d39a00640 1 --2- 172.21.15.27:0/1065915391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d3410bf80 0x7f2d341a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.005+0000 7f2d39a00640 1 -- 172.21.15.27:0/1065915391 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d341a2620 con 0x7f2d34105d00 2024-09-21T00:31:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.005+0000 7f2d39000640 1 --2- 172.21.15.27:0/1065915391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d3410b630 0x7f2d3419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:31:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.005+0000 7f2d3a400640 1 --2- 172.21.15.27:0/1065915391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d3410bf80 0x7f2d341a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:31:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.006+0000 7f2d39a00640 1 --2- 172.21.15.27:0/1065915391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d34105d00 0x7f2d3419c6f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2d28005ab0 tx=0x7f2d28003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.006+0000 7f2d32000640 1 -- 172.21.15.27:0/1065915391 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2803f070 con 0x7f2d34105d00 2024-09-21T00:31:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.006+0000 7f2d3c400640 1 -- 172.21.15.27:0/1065915391 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d341a28b0 con 0x7f2d34105d00 2024-09-21T00:31:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.006+0000 7f2d32000640 1 -- 172.21.15.27:0/1065915391 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d28031b60 con 0x7f2d34105d00 2024-09-21T00:31:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.006+0000 7f2d32000640 1 -- 172.21.15.27:0/1065915391 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28035030 con 0x7f2d34105d00 2024-09-21T00:31:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.006+0000 7f2d3c400640 1 -- 172.21.15.27:0/1065915391 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d341a90a0 con 0x7f2d34105d00 2024-09-21T00:31:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.010+0000 7f2d32000640 1 -- 172.21.15.27:0/1065915391 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2d28042070 con 0x7f2d34105d00 2024-09-21T00:31:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.011+0000 7f2d434b1740 1 Processor -- start 2024-09-21T00:31:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.011+0000 7f2d434b1740 1 -- start start 2024-09-21T00:31:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.011+0000 7f2d32000640 1 --2- 172.21.15.27:0/1065915391 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2d1c0b6680 0x7f2d1c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.011+0000 7f2d434b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab53c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.012+0000 7f2d434b1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7862f0 0x55fb6ab54170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.012+0000 7f2d434b1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab546b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.012+0000 7f2d434b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fb6aac96b0 con 0x55fb6a7af1b0 2024-09-21T00:31:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.012+0000 7f2d434b1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fb6aab8ae0 con 0x55fb6a78e4d0 2024-09-21T00:31:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.012+0000 7f2d434b1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fb6aabc8f0 con 0x55fb6a7862f0 2024-09-21T00:31:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.012+0000 7f2d32000640 1 -- 172.21.15.27:0/1065915391 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f2d28108290 con 0x7f2d34105d00 2024-09-21T00:31:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.012+0000 7f2d3a400640 1 --2- 172.21.15.27:0/1065915391 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2d1c0b6680 0x7f2d1c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.012+0000 7f2d39a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab53c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.012+0000 7f2d39a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab53c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53478/0 (socket says 172.21.15.27:53478) 2024-09-21T00:31:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.012+0000 7f2d39a00640 1 -- 172.21.15.27:0/75340718 learned_addr learned my addr 172.21.15.27:0/75340718 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.012+0000 7f2d39000640 1 --2- 172.21.15.27:0/75340718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7862f0 0x55fb6ab54170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.012+0000 7f2d3a400640 1 --2- 172.21.15.27:0/75340718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.012+0000 7f2d39a00640 1 -- 172.21.15.27:0/75340718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7862f0 msgr2=0x55fb6ab54170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.013+0000 7f2d39a00640 1 --2- 172.21.15.27:0/75340718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7862f0 0x55fb6ab54170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.013+0000 7f2d39a00640 1 -- 172.21.15.27:0/75340718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a78e4d0 msgr2=0x55fb6ab546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.013+0000 7f2d39a00640 1 --2- 172.21.15.27:0/75340718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.013+0000 7f2d39a00640 1 -- 172.21.15.27:0/75340718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb6aa85780 con 0x55fb6a7af1b0 2024-09-21T00:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.013+0000 7f2d3a400640 1 --2- 172.21.15.27:0/1065915391 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2d1c0b6680 0x7f2d1c0b8b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c005fd0 tx=0x7f2d2c013e50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.013+0000 7f2d39a00640 1 --2- 172.21.15.27:0/75340718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab53c30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2d2802a9b0 tx=0x7f2d28009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64ecbe585212fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.013+0000 7f2d26a00640 1 -- 172.21.15.27:0/75340718 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d280d2780 con 0x55fb6a7af1b0 2024-09-21T00:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.013+0000 7f2d26a00640 1 -- 172.21.15.27:0/75340718 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d280d76c0 con 0x55fb6a7af1b0 2024-09-21T00:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.013+0000 7f2d26a00640 1 -- 172.21.15.27:0/75340718 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28079910 con 0x55fb6a7af1b0 2024-09-21T00:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.014+0000 7f2d434b1740 1 -- 172.21.15.27:0/75340718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7af1b0 msgr2=0x55fb6ab53c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.014+0000 7f2d434b1740 1 --2- 172.21.15.27:0/75340718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab53c30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2d2802a9b0 tx=0x7f2d28009260 comp rx=0 tx=0).stop 2024-09-21T00:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.014+0000 7f2d434b1740 1 -- 172.21.15.27:0/75340718 shutdown_connections 2024-09-21T00:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.014+0000 7f2d434b1740 1 --2- 172.21.15.27:0/75340718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.014+0000 7f2d434b1740 1 --2- 172.21.15.27:0/75340718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7862f0 0x55fb6ab54170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.014+0000 7f2d434b1740 1 --2- 172.21.15.27:0/75340718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab53c30 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.014+0000 7f2d434b1740 1 -- 172.21.15.27:0/75340718 >> 172.21.15.27:0/75340718 conn(0x55fb6a7a3300 msgr2=0x55fb6ab51b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.014+0000 7f2d434b1740 1 -- 172.21.15.27:0/75340718 shutdown_connections 2024-09-21T00:31:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.014+0000 7f2d434b1740 1 -- 172.21.15.27:0/75340718 wait complete. 2024-09-21T00:31:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.015+0000 7f2d434b1740 1 -- start start 2024-09-21T00:31:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.015+0000 7f2d434b1740 1 Processor -- start 2024-09-21T00:31:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d434b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7862f0 0x55fb6ab5f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d434b1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab5f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d434b1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab6b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d434b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fb6a9f0ab0 con 0x55fb6a7862f0 2024-09-21T00:31:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d434b1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fb6aa5f570 con 0x55fb6a7af1b0 2024-09-21T00:31:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d434b1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fb6ab62170 con 0x55fb6a78e4d0 2024-09-21T00:31:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d434b1740 10 client.0 _ll_register_callbacks cb 0x55fb6aa6e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:31:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d39a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7862f0 0x55fb6ab5f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d3a400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab6b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d39000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab5f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d3a400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab6b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38218/0 (socket says 172.21.15.27:38218) 2024-09-21T00:31:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d39a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7862f0 0x55fb6ab5f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53486/0 (socket says 172.21.15.27:53486) 2024-09-21T00:31:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d3a400640 1 -- 172.21.15.27:0/1976373503 learned_addr learned my addr 172.21.15.27:0/1976373503 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d3a400640 1 -- 172.21.15.27:0/1976373503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a78e4d0 msgr2=0x55fb6ab5f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d3a400640 1 --2- 172.21.15.27:0/1976373503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab5f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d3a400640 1 -- 172.21.15.27:0/1976373503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7862f0 msgr2=0x55fb6ab5f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.016+0000 7f2d3a400640 1 --2- 172.21.15.27:0/1976373503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7862f0 0x55fb6ab5f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.017+0000 7f2d3a400640 1 -- 172.21.15.27:0/1976373503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb6a81a790 con 0x55fb6a7af1b0 2024-09-21T00:31:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.017+0000 7f2d3a400640 1 --2- 172.21.15.27:0/1976373503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab6b350 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00ff60 tx=0x7f2d2c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdb86038d08c5efc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.017+0000 7f2d1b400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:31:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.017+0000 7f2d1b400640 1 -- 172.21.15.27:0/1976373503 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c024070 con 0x55fb6a7af1b0 2024-09-21T00:31:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.017+0000 7f2d434b1740 1 -- 172.21.15.27:0/1976373503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d280d76c0 con 0x55fb6a7af1b0 2024-09-21T00:31:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.017+0000 7f2d1b400640 1 -- 172.21.15.27:0/1976373503 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f2d2c0047d0 con 0x55fb6a7af1b0 2024-09-21T00:31:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.018+0000 7f2d1b400640 1 -- 172.21.15.27:0/1976373503 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2d2c030050 con 0x55fb6a7af1b0 2024-09-21T00:31:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.018+0000 7f2d434b1740 10 client.26438 fetch_fsmap learned FSMap version 38 2024-09-21T00:31:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.018+0000 7f2d434b1740 1 -- 172.21.15.27:0/1976373503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fb6a893620 con 0x55fb6a7af1b0 2024-09-21T00:31:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.018+0000 7f2d1b400640 1 -- 172.21.15.27:0/1976373503 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d280d76c0 con 0x55fb6a7af1b0 2024-09-21T00:31:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.018+0000 7f2d1b400640 1 -- 172.21.15.27:0/1976373503 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f2d2c01f070 con 0x55fb6a7af1b0 2024-09-21T00:31:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.018+0000 7f2d434b1740 10 client.26438 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:31:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.018+0000 7f2d434b1740 10 client.26438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:31:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.018+0000 7f2d434b1740 20 client.26438 populate_metadata read hostname 'smithi027' 2024-09-21T00:31:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.018+0000 7f2d434b1740 1 --2- 172.21.15.27:0/1976373503 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55fb6a76dd80 0x55fb6ab71010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.018+0000 7f2d434b1740 4 client.26438 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:31:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.018+0000 7f2d434b1740 1 -- 172.21.15.27:0/1976373503 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb6aacb600 con 0x55fb6a76dd80 2024-09-21T00:31:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.019+0000 7f2d39a00640 1 --2- 172.21.15.27:0/1976373503 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55fb6a76dd80 0x55fb6ab71010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.019+0000 7f2d39a00640 1 --2- 172.21.15.27:0/1976373503 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55fb6a76dd80 0x55fb6ab71010 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8a5dadebef42c0e server_cookie=e0c11239c42669cc in_seq=0 out_seq=0 2024-09-21T00:31:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.019+0000 7f2d1b400640 10 client.26438 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.020+0000 7f2d1b400640 1 -- 172.21.15.27:0/1976373503 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fb6aacb600 con 0x55fb6a76dd80 2024-09-21T00:31:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.020+0000 7f2d1b400640 10 client.26438 handle_command_reply: tid=0 2024-09-21T00:31:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.020+0000 7f2d434b1740 1 client.26438 shutdown 2024-09-21T00:31:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.020+0000 7f2d434b1740 1 -- 172.21.15.27:0/1976373503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a7af1b0 msgr2=0x55fb6ab6b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.020+0000 7f2d434b1740 1 --2- 172.21.15.27:0/1976373503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab6b350 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00ff60 tx=0x7f2d2c004ab0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.020+0000 7f2d434b1740 1 -- 172.21.15.27:0/1976373503 shutdown_connections 2024-09-21T00:31:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.020+0000 7f2d434b1740 1 --2- 172.21.15.27:0/1976373503 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55fb6a76dd80 0x55fb6ab71010 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.020+0000 7f2d434b1740 1 --2- 172.21.15.27:0/1976373503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab6b350 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.020+0000 7f2d434b1740 1 --2- 172.21.15.27:0/1976373503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab5f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.020+0000 7f2d1b400640 0 client.26438 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.020+0000 7f2d434b1740 1 --2- 172.21.15.27:0/1976373503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7862f0 0x55fb6ab5f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.020+0000 7f2d434b1740 1 -- 172.21.15.27:0/1976373503 >> 172.21.15.27:0/1976373503 conn(0x55fb6a7a3300 msgr2=0x55fb6ab54c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.020+0000 7f2d434b1740 1 -- 172.21.15.27:0/1976373503 shutdown_connections 2024-09-21T00:31:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.021+0000 7f2d434b1740 1 -- 172.21.15.27:0/1976373503 wait complete. 2024-09-21T00:31:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.021+0000 7f2d434b1740 20 client.26438 trim_cache size 0 max 16384 2024-09-21T00:31:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.036+0000 7f2d434b1740 1 Processor -- start 2024-09-21T00:31:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.036+0000 7f2d434b1740 1 -- start start 2024-09-21T00:31:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.037+0000 7f2d434b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7862f0 0x55fb6ab61150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.037+0000 7f2d434b1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab5c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.037+0000 7f2d434b1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab5c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.037+0000 7f2d434b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341b5b90 con 0x55fb6a7862f0 2024-09-21T00:31:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.037+0000 7f2d434b1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341b3400 con 0x55fb6a78e4d0 2024-09-21T00:31:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.037+0000 7f2d434b1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341b1270 con 0x55fb6a7af1b0 2024-09-21T00:31:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.037+0000 7f2d39000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab5c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.037+0000 7f2d39a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7862f0 0x55fb6ab61150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.037+0000 7f2d39000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab5c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38220/0 (socket says 172.21.15.27:38220) 2024-09-21T00:31:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.037+0000 7f2d39000640 1 -- 172.21.15.27:0/2674101413 learned_addr learned my addr 172.21.15.27:0/2674101413 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.037+0000 7f2d3a400640 1 --2- 172.21.15.27:0/2674101413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab5c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.038+0000 7f2d39000640 1 -- 172.21.15.27:0/2674101413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7af1b0 msgr2=0x55fb6ab5c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.038+0000 7f2d39000640 1 --2- 172.21.15.27:0/2674101413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab5c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.038+0000 7f2d39000640 1 -- 172.21.15.27:0/2674101413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7862f0 msgr2=0x55fb6ab61150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.038+0000 7f2d39000640 1 --2- 172.21.15.27:0/2674101413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7862f0 0x55fb6ab61150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.038+0000 7f2d39000640 1 -- 172.21.15.27:0/2674101413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb6ab56a40 con 0x55fb6a78e4d0 2024-09-21T00:31:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.038+0000 7f2d3a400640 1 --2- 172.21.15.27:0/2674101413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab5c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:31:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.038+0000 7f2d39000640 1 --2- 172.21.15.27:0/2674101413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab5c440 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2d200127a0 tx=0x7f2d20005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9c7b692062adbab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.038+0000 7f2d26000640 1 -- 172.21.15.27:0/2674101413 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20004030 con 0x55fb6a78e4d0 2024-09-21T00:31:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.038+0000 7f2d26000640 1 -- 172.21.15.27:0/2674101413 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d2000f920 con 0x55fb6a78e4d0 2024-09-21T00:31:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.038+0000 7f2d26000640 1 -- 172.21.15.27:0/2674101413 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20003960 con 0x55fb6a78e4d0 2024-09-21T00:31:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.039+0000 7f2d434b1740 1 -- 172.21.15.27:0/2674101413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a78e4d0 msgr2=0x55fb6ab5c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.039+0000 7f2d434b1740 1 --2- 172.21.15.27:0/2674101413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab5c440 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2d200127a0 tx=0x7f2d20005f90 comp rx=0 tx=0).stop 2024-09-21T00:31:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.039+0000 7f2d434b1740 1 -- 172.21.15.27:0/2674101413 shutdown_connections 2024-09-21T00:31:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.039+0000 7f2d434b1740 1 --2- 172.21.15.27:0/2674101413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab5c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.039+0000 7f2d434b1740 1 --2- 172.21.15.27:0/2674101413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab5c440 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.039+0000 7f2d434b1740 1 --2- 172.21.15.27:0/2674101413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a7862f0 0x55fb6ab61150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.039+0000 7f2d434b1740 1 -- 172.21.15.27:0/2674101413 >> 172.21.15.27:0/2674101413 conn(0x55fb6a7a3300 msgr2=0x55fb6ab6a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.040+0000 7f2d434b1740 1 -- 172.21.15.27:0/2674101413 shutdown_connections 2024-09-21T00:31:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.040+0000 7f2d434b1740 1 -- 172.21.15.27:0/2674101413 wait complete. 2024-09-21T00:31:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.040+0000 7f2d434b1740 1 -- start start 2024-09-21T00:31:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.040+0000 7f2d434b1740 1 Processor -- start 2024-09-21T00:31:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.041+0000 7f2d434b1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7862f0 0x55fb6ab67970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.041+0000 7f2d434b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab64e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.041+0000 7f2d434b1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab65370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.041+0000 7f2d434b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341b18c0 con 0x55fb6a78e4d0 2024-09-21T00:31:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.041+0000 7f2d434b1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341b2d30 con 0x55fb6a7af1b0 2024-09-21T00:31:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.041+0000 7f2d434b1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341b1f90 con 0x55fb6a7862f0 2024-09-21T00:31:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.041+0000 7f2d434b1740 10 client.0 _ll_register_callbacks cb 0x7f2d2c013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:31:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.041+0000 7f2d3a400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab65370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.041+0000 7f2d3a400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab65370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38224/0 (socket says 172.21.15.27:38224) 2024-09-21T00:31:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.041+0000 7f2d39000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab64e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.041+0000 7f2d3a400640 1 -- 172.21.15.27:0/945751962 learned_addr learned my addr 172.21.15.27:0/945751962 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.041+0000 7f2d39a00640 1 --2- 172.21.15.27:0/945751962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7862f0 0x55fb6ab67970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.042+0000 7f2d3a400640 1 -- 172.21.15.27:0/945751962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7862f0 msgr2=0x55fb6ab67970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.042+0000 7f2d3a400640 1 --2- 172.21.15.27:0/945751962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7862f0 0x55fb6ab67970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.042+0000 7f2d3a400640 1 -- 172.21.15.27:0/945751962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a78e4d0 msgr2=0x55fb6ab64e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.042+0000 7f2d3a400640 1 --2- 172.21.15.27:0/945751962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab64e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.042+0000 7f2d3a400640 1 -- 172.21.15.27:0/945751962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb6ab659e0 con 0x55fb6a7af1b0 2024-09-21T00:31:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.042+0000 7f2d39a00640 1 --2- 172.21.15.27:0/945751962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7862f0 0x55fb6ab67970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:31:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.042+0000 7f2d3a400640 1 --2- 172.21.15.27:0/945751962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab65370 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c000f80 tx=0x7f2d2c004a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=707f55beadb10034 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.042+0000 7f2d1b400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:31:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.042+0000 7f2d1b400640 1 -- 172.21.15.27:0/945751962 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c039060 con 0x55fb6a7af1b0 2024-09-21T00:31:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.042+0000 7f2d1b400640 1 -- 172.21.15.27:0/945751962 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f2d2c029440 con 0x55fb6a7af1b0 2024-09-21T00:31:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.042+0000 7f2d434b1740 1 -- 172.21.15.27:0/945751962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d2000f920 con 0x55fb6a7af1b0 2024-09-21T00:31:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.043+0000 7f2d1b400640 1 -- 172.21.15.27:0/945751962 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2d2c024040 con 0x55fb6a7af1b0 2024-09-21T00:31:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.043+0000 7f2d434b1740 10 client.26444 fetch_fsmap learned FSMap version 38 2024-09-21T00:31:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.043+0000 7f2d434b1740 1 -- 172.21.15.27:0/945751962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fb6abdb880 con 0x55fb6a7af1b0 2024-09-21T00:31:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.043+0000 7f2d1b400640 1 -- 172.21.15.27:0/945751962 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d2000f920 con 0x55fb6a7af1b0 2024-09-21T00:31:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.043+0000 7f2d1b400640 1 -- 172.21.15.27:0/945751962 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f2d2c01f050 con 0x55fb6a7af1b0 2024-09-21T00:31:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.043+0000 7f2d434b1740 10 client.26444 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:31:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.043+0000 7f2d434b1740 10 client.26444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:31:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.043+0000 7f2d434b1740 20 client.26444 populate_metadata read hostname 'smithi027' 2024-09-21T00:31:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.043+0000 7f2d434b1740 1 --2- 172.21.15.27:0/945751962 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55fb6a7431c0 0x55fb6abde020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.043+0000 7f2d434b1740 4 client.26444 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:31:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.043+0000 7f2d434b1740 1 -- 172.21.15.27:0/945751962 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb6aac70b0 con 0x55fb6a7431c0 2024-09-21T00:31:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.044+0000 7f2d39a00640 1 --2- 172.21.15.27:0/945751962 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55fb6a7431c0 0x55fb6abde020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.044+0000 7f2d39a00640 1 --2- 172.21.15.27:0/945751962 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55fb6a7431c0 0x55fb6abde020 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b7f3482eaa58ae8 server_cookie=90c71c9b3d343780 in_seq=0 out_seq=0 2024-09-21T00:31:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.044+0000 7f2d1b400640 10 client.26444 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.060+0000 7f2d1b400640 1 -- 172.21.15.27:0/945751962 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb6aac70b0 con 0x55fb6a7431c0 2024-09-21T00:31:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.060+0000 7f2d1b400640 10 client.26444 handle_command_reply: tid=0 2024-09-21T00:31:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.060+0000 7f2d434b1740 1 client.26444 shutdown 2024-09-21T00:31:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.060+0000 7f2d434b1740 1 -- 172.21.15.27:0/945751962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a7af1b0 msgr2=0x55fb6ab65370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.060+0000 7f2d434b1740 1 --2- 172.21.15.27:0/945751962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab65370 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c000f80 tx=0x7f2d2c004a00 comp rx=0 tx=0).stop 2024-09-21T00:31:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.060+0000 7f2d434b1740 1 -- 172.21.15.27:0/945751962 shutdown_connections 2024-09-21T00:31:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.060+0000 7f2d434b1740 1 --2- 172.21.15.27:0/945751962 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55fb6a7431c0 0x55fb6abde020 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.060+0000 7f2d434b1740 1 --2- 172.21.15.27:0/945751962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fb6a7af1b0 0x55fb6ab65370 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.060+0000 7f2d434b1740 1 --2- 172.21.15.27:0/945751962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb6a78e4d0 0x55fb6ab64e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.060+0000 7f2d1b400640 0 client.26444 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.060+0000 7f2d434b1740 1 --2- 172.21.15.27:0/945751962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fb6a7862f0 0x55fb6ab67970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.060+0000 7f2d434b1740 1 -- 172.21.15.27:0/945751962 >> 172.21.15.27:0/945751962 conn(0x55fb6a7a3300 msgr2=0x55fb6ab60eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.060+0000 7f2d434b1740 1 -- 172.21.15.27:0/945751962 shutdown_connections 2024-09-21T00:31:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.060+0000 7f2d434b1740 1 -- 172.21.15.27:0/945751962 wait complete. 2024-09-21T00:31:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.060+0000 7f2d434b1740 20 client.26444 trim_cache size 0 max 16384 2024-09-21T00:31:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:31:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:31:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:31:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:31:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.061+0000 7f2d3c400640 1 -- 172.21.15.27:0/1065915391 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2d1c0b6680 msgr2=0x7f2d1c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.061+0000 7f2d3c400640 1 --2- 172.21.15.27:0/1065915391 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2d1c0b6680 0x7f2d1c0b8b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c005fd0 tx=0x7f2d2c013e50 comp rx=0 tx=0).stop 2024-09-21T00:31:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.061+0000 7f2d3c400640 1 -- 172.21.15.27:0/1065915391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d34105d00 msgr2=0x7f2d3419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.061+0000 7f2d3c400640 1 --2- 172.21.15.27:0/1065915391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d34105d00 0x7f2d3419c6f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2d28005ab0 tx=0x7f2d28003c10 comp rx=0 tx=0).stop 2024-09-21T00:31:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.061+0000 7f2d3c400640 1 -- 172.21.15.27:0/1065915391 shutdown_connections 2024-09-21T00:31:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.061+0000 7f2d3c400640 1 --2- 172.21.15.27:0/1065915391 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2d1c0b6680 0x7f2d1c0b8b90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.061+0000 7f2d3c400640 1 --2- 172.21.15.27:0/1065915391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d3410bf80 0x7f2d341a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.061+0000 7f2d3c400640 1 --2- 172.21.15.27:0/1065915391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d3410b630 0x7f2d3419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.061+0000 7f2d3c400640 1 --2- 172.21.15.27:0/1065915391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d34105d00 0x7f2d3419c6f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.061+0000 7f2d3c400640 1 -- 172.21.15.27:0/1065915391 >> 172.21.15.27:0/1065915391 conn(0x7f2d34101720 msgr2=0x7f2d34103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.061+0000 7f2d3c400640 1 -- 172.21.15.27:0/1065915391 shutdown_connections 2024-09-21T00:31:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:34.061+0000 7f2d3c400640 1 -- 172.21.15.27:0/1065915391 wait complete. 2024-09-21T00:31:34.096 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:33 smithi027 ceph-mon[30745]: pgmap v691: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 99 KiB/s wr, 26 op/s 2024-09-21T00:31:34.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:33 smithi089 ceph-mon[34481]: pgmap v691: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 99 KiB/s wr, 26 op/s 2024-09-21T00:31:34.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:33 smithi098 ceph-mon[34725]: pgmap v691: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 99 KiB/s wr, 26 op/s 2024-09-21T00:31:36.095 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:35 smithi027 ceph-mon[30745]: pgmap v692: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 KiB/s wr, 26 op/s 2024-09-21T00:31:36.095 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:35 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:31:36.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:35 smithi089 ceph-mon[34481]: pgmap v692: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 KiB/s wr, 26 op/s 2024-09-21T00:31:36.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:35 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:31:36.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:35 smithi098 ceph-mon[34725]: pgmap v692: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 KiB/s wr, 26 op/s 2024-09-21T00:31:36.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:35 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:31:38.105 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:37 smithi027 ceph-mon[30745]: pgmap v693: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 91 KiB/s wr, 27 op/s 2024-09-21T00:31:38.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:37 smithi089 ceph-mon[34481]: pgmap v693: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 91 KiB/s wr, 27 op/s 2024-09-21T00:31:38.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:37 smithi098 ceph-mon[34725]: pgmap v693: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 91 KiB/s wr, 27 op/s 2024-09-21T00:31:38.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:38 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:31:39.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:38 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:31:39.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:38 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:31:40.096 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:39 smithi027 ceph-mon[30745]: pgmap v694: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 119 KiB/s wr, 21 op/s 2024-09-21T00:31:40.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:39 smithi089 ceph-mon[34481]: pgmap v694: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 119 KiB/s wr, 21 op/s 2024-09-21T00:31:40.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:39 smithi098 ceph-mon[34725]: pgmap v694: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 119 KiB/s wr, 21 op/s 2024-09-21T00:31:42.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:41 smithi089 ceph-mon[34481]: pgmap v695: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 119 KiB/s wr, 13 op/s 2024-09-21T00:31:42.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:41 smithi098 ceph-mon[34725]: pgmap v695: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 119 KiB/s wr, 13 op/s 2024-09-21T00:31:42.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:41 smithi027 ceph-mon[30745]: pgmap v695: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 119 KiB/s wr, 13 op/s 2024-09-21T00:31:44.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:43 smithi089 ceph-mon[34481]: pgmap v696: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 113 KiB/s wr, 13 op/s 2024-09-21T00:31:44.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:43 smithi098 ceph-mon[34725]: pgmap v696: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 113 KiB/s wr, 13 op/s 2024-09-21T00:31:44.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:43 smithi027 ceph-mon[30745]: pgmap v696: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 113 KiB/s wr, 13 op/s 2024-09-21T00:31:46.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:45 smithi089 ceph-mon[34481]: pgmap v697: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T00:31:46.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:45 smithi098 ceph-mon[34725]: pgmap v697: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T00:31:46.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:45 smithi027 ceph-mon[30745]: pgmap v697: 129 pgs: 129 active+clean; 977 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T00:31:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.357+0000 7f1789c00640 1 -- 172.21.15.27:0/3566963202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1784105b40 msgr2=0x7f1784105f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.357+0000 7f1789c00640 1 --2- 172.21.15.27:0/3566963202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1784105b40 0x7f1784105f50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1774009900 tx=0x7f177402a410 comp rx=0 tx=0).stop 2024-09-21T00:31:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.357+0000 7f1789c00640 1 -- 172.21.15.27:0/3566963202 shutdown_connections 2024-09-21T00:31:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.357+0000 7f1789c00640 1 --2- 172.21.15.27:0/3566963202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f178410bdc0 0x7f17841125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.357+0000 7f1789c00640 1 --2- 172.21.15.27:0/3566963202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f178410b470 0x7f178410b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.357+0000 7f1789c00640 1 --2- 172.21.15.27:0/3566963202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1784105b40 0x7f1784105f50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.357+0000 7f1789c00640 1 -- 172.21.15.27:0/3566963202 >> 172.21.15.27:0/3566963202 conn(0x7f1784101580 msgr2=0x7f17841039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.357+0000 7f1789c00640 1 -- 172.21.15.27:0/3566963202 shutdown_connections 2024-09-21T00:31:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.358+0000 7f1789c00640 1 -- 172.21.15.27:0/3566963202 wait complete. 2024-09-21T00:31:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.359+0000 7f1789c00640 1 Processor -- start 2024-09-21T00:31:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.359+0000 7f1789c00640 1 -- start start 2024-09-21T00:31:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.359+0000 7f1789c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1784105b40 0x7f178419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.359+0000 7f1789c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f178410b470 0x7f178419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.359+0000 7f1789c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f178410bdc0 0x7f17841a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.359+0000 7f1789c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1784115ad0 con 0x7f178410bdc0 2024-09-21T00:31:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.359+0000 7f1789c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1784115950 con 0x7f178410b470 2024-09-21T00:31:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.359+0000 7f1789c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1784115c50 con 0x7f1784105b40 2024-09-21T00:31:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.360+0000 7f1783e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1784105b40 0x7f178419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.360+0000 7f1783e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1784105b40 0x7f178419c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:33442/0 (socket says 172.21.15.27:33442) 2024-09-21T00:31:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.360+0000 7f1783e00640 1 -- 172.21.15.27:0/415277920 learned_addr learned my addr 172.21.15.27:0/415277920 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:47.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.360+0000 7f1789200640 1 --2- 172.21.15.27:0/415277920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f178410bdc0 0x7f17841a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.360+0000 7f177be00640 1 --2- 172.21.15.27:0/415277920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f178410b470 0x7f178419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.360+0000 7f1783e00640 1 -- 172.21.15.27:0/415277920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f178410b470 msgr2=0x7f178419ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:47.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.360+0000 7f1783e00640 1 --2- 172.21.15.27:0/415277920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f178410b470 0x7f178419ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.360+0000 7f1783e00640 1 -- 172.21.15.27:0/415277920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f178410bdc0 msgr2=0x7f17841a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.360+0000 7f1783e00640 1 --2- 172.21.15.27:0/415277920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f178410bdc0 0x7f17841a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.360+0000 7f1783e00640 1 -- 172.21.15.27:0/415277920 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17841a23a0 con 0x7f1784105b40 2024-09-21T00:31:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.361+0000 7f1783e00640 1 --2- 172.21.15.27:0/415277920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1784105b40 0x7f178419c470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1774005ab0 tx=0x7f1774003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.361+0000 7f1781600640 1 -- 172.21.15.27:0/415277920 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177403f070 con 0x7f1784105b40 2024-09-21T00:31:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.361+0000 7f1789c00640 1 -- 172.21.15.27:0/415277920 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17841a2630 con 0x7f1784105b40 2024-09-21T00:31:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.361+0000 7f1781600640 1 -- 172.21.15.27:0/415277920 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1774031d60 con 0x7f1784105b40 2024-09-21T00:31:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.361+0000 7f1781600640 1 -- 172.21.15.27:0/415277920 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774035030 con 0x7f1784105b40 2024-09-21T00:31:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.361+0000 7f1789c00640 1 -- 172.21.15.27:0/415277920 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17841a8e20 con 0x7f1784105b40 2024-09-21T00:31:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.364+0000 7f17922ca740 1 Processor -- start 2024-09-21T00:31:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.364+0000 7f17922ca740 1 -- start start 2024-09-21T00:31:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.364+0000 7f17922ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0491b0 0x5650ab3ede10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.364+0000 7f17922ca740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0202f0 0x5650ab3ee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.365+0000 7f17922ca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0284d0 0x5650ab3ee890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.365+0000 7f17922ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5650ab3636b0 con 0x5650ab0491b0 2024-09-21T00:31:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.365+0000 7f17922ca740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5650ab352ae0 con 0x5650ab0202f0 2024-09-21T00:31:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.365+0000 7f17922ca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5650ab3568f0 con 0x5650ab0284d0 2024-09-21T00:31:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.365+0000 7f177be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0284d0 0x5650ab3ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.365+0000 7f1789200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0491b0 0x5650ab3ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.365+0000 7f1781600640 1 -- 172.21.15.27:0/415277920 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1774042070 con 0x7f1784105b40 2024-09-21T00:31:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.365+0000 7f1789200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0491b0 0x5650ab3ede10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44308/0 (socket says 172.21.15.27:44308) 2024-09-21T00:31:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.365+0000 7f1789200640 1 -- 172.21.15.27:0/3703981478 learned_addr learned my addr 172.21.15.27:0/3703981478 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.366+0000 7f1789200640 1 -- 172.21.15.27:0/3703981478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0284d0 msgr2=0x5650ab3ee890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.366+0000 7f1789200640 1 --2- 172.21.15.27:0/3703981478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0284d0 0x5650ab3ee890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.366+0000 7f1789200640 1 -- 172.21.15.27:0/3703981478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0202f0 msgr2=0x5650ab3ee350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:31:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.366+0000 7f1789200640 1 --2- 172.21.15.27:0/3703981478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0202f0 0x5650ab3ee350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.366+0000 7f1789200640 1 -- 172.21.15.27:0/3703981478 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650ab31f780 con 0x5650ab0491b0 2024-09-21T00:31:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.367+0000 7f1789200640 1 --2- 172.21.15.27:0/3703981478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0491b0 0x5650ab3ede10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f177c010370 tx=0x7f177c010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8782eaa12def20 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.368+0000 7f1779600640 1 -- 172.21.15.27:0/3703981478 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c016070 con 0x5650ab0491b0 2024-09-21T00:31:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.368+0000 7f1779600640 1 -- 172.21.15.27:0/3703981478 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f177c004d20 con 0x5650ab0491b0 2024-09-21T00:31:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.368+0000 7f1779600640 1 -- 172.21.15.27:0/3703981478 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c0053e0 con 0x5650ab0491b0 2024-09-21T00:31:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.369+0000 7f1781600640 1 --2- 172.21.15.27:0/415277920 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f17680b6670 0x7f17680b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.371+0000 7f17922ca740 1 -- 172.21.15.27:0/3703981478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0491b0 msgr2=0x5650ab3ede10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.371+0000 7f17922ca740 1 --2- 172.21.15.27:0/3703981478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0491b0 0x5650ab3ede10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f177c010370 tx=0x7f177c010830 comp rx=0 tx=0).stop 2024-09-21T00:31:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.371+0000 7f1781600640 1 -- 172.21.15.27:0/415277920 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f1774107fb0 con 0x7f1784105b40 2024-09-21T00:31:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.371+0000 7f17922ca740 1 -- 172.21.15.27:0/3703981478 shutdown_connections 2024-09-21T00:31:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.371+0000 7f17922ca740 1 --2- 172.21.15.27:0/3703981478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0284d0 0x5650ab3ee890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.371+0000 7f17922ca740 1 --2- 172.21.15.27:0/3703981478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0202f0 0x5650ab3ee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.371+0000 7f17922ca740 1 --2- 172.21.15.27:0/3703981478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0491b0 0x5650ab3ede10 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.371+0000 7f17922ca740 1 -- 172.21.15.27:0/3703981478 >> 172.21.15.27:0/3703981478 conn(0x5650ab03d300 msgr2=0x5650ab3ebd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.372+0000 7f1783e00640 1 --2- 172.21.15.27:0/415277920 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f17680b6670 0x7f17680b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.372+0000 7f17922ca740 1 -- 172.21.15.27:0/3703981478 shutdown_connections 2024-09-21T00:31:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.372+0000 7f17922ca740 1 -- 172.21.15.27:0/3703981478 wait complete. 2024-09-21T00:31:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.373+0000 7f17922ca740 1 -- start start 2024-09-21T00:31:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.374+0000 7f17922ca740 1 Processor -- start 2024-09-21T00:31:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.375+0000 7f17922ca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0202f0 0x5650ab3f9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.375+0000 7f17922ca740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 0x5650ab3f9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.375+0000 7f17922ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0491b0 0x5650ab4058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.375+0000 7f17922ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5650ab28aab0 con 0x5650ab0491b0 2024-09-21T00:31:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.375+0000 7f1789200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0202f0 0x5650ab3f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.375+0000 7f17922ca740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5650ab2f9570 con 0x5650ab0284d0 2024-09-21T00:31:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.375+0000 7f17922ca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5650ab3fc590 con 0x5650ab0202f0 2024-09-21T00:31:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.375+0000 7f1789200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0202f0 0x5650ab3f9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:33456/0 (socket says 172.21.15.27:33456) 2024-09-21T00:31:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.375+0000 7f1789200640 1 -- 172.21.15.27:0/3339323000 learned_addr learned my addr 172.21.15.27:0/3339323000 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.375+0000 7f17922ca740 10 client.0 _ll_register_callbacks cb 0x5650ab308100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:31:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.376+0000 7f177be00640 1 --2- 172.21.15.27:0/3339323000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 0x5650ab3f9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.377+0000 7f1789200640 1 --2- 172.21.15.27:0/3339323000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0491b0 0x5650ab4058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.378+0000 7f1783e00640 1 --2- 172.21.15.27:0/415277920 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f17680b6670 0x7f17680b8b80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x5650ab3eca90 tx=0x7f177402a410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.378+0000 7f1789200640 1 -- 172.21.15.27:0/3339323000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 msgr2=0x5650ab3f9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.378+0000 7f1789200640 1 --2- 172.21.15.27:0/3339323000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 0x5650ab3f9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.378+0000 7f1789200640 1 -- 172.21.15.27:0/3339323000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0491b0 msgr2=0x5650ab4058b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.378+0000 7f1789200640 1 --2- 172.21.15.27:0/3339323000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0491b0 0x5650ab4058b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.378+0000 7f1789200640 1 -- 172.21.15.27:0/3339323000 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650ab0b4790 con 0x5650ab0202f0 2024-09-21T00:31:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.378+0000 7f1789200640 1 --2- 172.21.15.27:0/3339323000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0202f0 0x5650ab3f9540 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f177c009bf0 tx=0x7f177c012f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26b91134654d81b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.378+0000 7f1772000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:31:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.379+0000 7f1772000640 1 -- 172.21.15.27:0/3339323000 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c016070 con 0x5650ab0202f0 2024-09-21T00:31:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.379+0000 7f1772000640 1 -- 172.21.15.27:0/3339323000 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f177c012720 con 0x5650ab0202f0 2024-09-21T00:31:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.379+0000 7f17922ca740 1 -- 172.21.15.27:0/3339323000 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f177c004d20 con 0x5650ab0202f0 2024-09-21T00:31:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.382+0000 7f1772000640 1 -- 172.21.15.27:0/3339323000 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f177c012920 con 0x5650ab0202f0 2024-09-21T00:31:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.382+0000 7f17922ca740 10 client.26476 fetch_fsmap learned FSMap version 40 2024-09-21T00:31:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.382+0000 7f17922ca740 1 -- 172.21.15.27:0/3339323000 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5650ab12d620 con 0x5650ab0202f0 2024-09-21T00:31:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.383+0000 7f1772000640 1 -- 172.21.15.27:0/3339323000 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f177c004d20 con 0x5650ab0202f0 2024-09-21T00:31:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.383+0000 7f1772000640 1 -- 172.21.15.27:0/3339323000 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f177c028040 con 0x5650ab0202f0 2024-09-21T00:31:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.383+0000 7f17922ca740 10 client.26476 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T00:31:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.383+0000 7f17922ca740 10 client.26476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:31:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.383+0000 7f17922ca740 20 client.26476 populate_metadata read hostname 'smithi027' 2024-09-21T00:31:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.383+0000 7f17922ca740 1 --2- 172.21.15.27:0/3339323000 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5650ab007d80 0x5650ab40b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.383+0000 7f17922ca740 4 client.26476 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:31:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.383+0000 7f17922ca740 1 -- 172.21.15.27:0/3339323000 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650ab365600 con 0x5650ab007d80 2024-09-21T00:31:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.385+0000 7f177be00640 1 --2- 172.21.15.27:0/3339323000 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5650ab007d80 0x5650ab40b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.386+0000 7f177be00640 1 --2- 172.21.15.27:0/3339323000 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5650ab007d80 0x5650ab40b5a0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f82b6eb5f64babb server_cookie=43102c68a9679b18 in_seq=0 out_seq=0 2024-09-21T00:31:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.386+0000 7f1772000640 10 client.26476 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.387+0000 7f1772000640 1 -- 172.21.15.27:0/3339323000 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5650ab365600 con 0x5650ab007d80 2024-09-21T00:31:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.387+0000 7f1772000640 10 client.26476 handle_command_reply: tid=0 2024-09-21T00:31:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.387+0000 7f17922ca740 1 client.26476 shutdown 2024-09-21T00:31:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.387+0000 7f17922ca740 1 -- 172.21.15.27:0/3339323000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0202f0 msgr2=0x5650ab3f9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.387+0000 7f17922ca740 1 --2- 172.21.15.27:0/3339323000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0202f0 0x5650ab3f9540 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f177c009bf0 tx=0x7f177c012f30 comp rx=0 tx=0).stop 2024-09-21T00:31:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.387+0000 7f17922ca740 1 -- 172.21.15.27:0/3339323000 shutdown_connections 2024-09-21T00:31:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.387+0000 7f17922ca740 1 --2- 172.21.15.27:0/3339323000 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5650ab007d80 0x5650ab40b5a0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.388+0000 7f17922ca740 1 --2- 172.21.15.27:0/3339323000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0491b0 0x5650ab4058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.388+0000 7f1772000640 0 client.26476 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.388+0000 7f17922ca740 1 --2- 172.21.15.27:0/3339323000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 0x5650ab3f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.388+0000 7f17922ca740 1 --2- 172.21.15.27:0/3339323000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0202f0 0x5650ab3f9540 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.388+0000 7f17922ca740 1 -- 172.21.15.27:0/3339323000 >> 172.21.15.27:0/3339323000 conn(0x5650ab03d300 msgr2=0x5650ab3eee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.388+0000 7f17922ca740 1 -- 172.21.15.27:0/3339323000 shutdown_connections 2024-09-21T00:31:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.388+0000 7f17922ca740 1 -- 172.21.15.27:0/3339323000 wait complete. 2024-09-21T00:31:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.388+0000 7f17922ca740 20 client.26476 trim_cache size 0 max 16384 2024-09-21T00:31:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.405+0000 7f17922ca740 1 Processor -- start 2024-09-21T00:31:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.405+0000 7f17922ca740 1 -- start start 2024-09-21T00:31:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.405+0000 7f17922ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0202f0 0x5650ab3ef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.405+0000 7f17922ca740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 0x5650ab3eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.405+0000 7f17922ca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0491b0 0x5650ab405820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.405+0000 7f17922ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f17841b5990 con 0x5650ab0202f0 2024-09-21T00:31:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.405+0000 7f17922ca740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f17841b3260 con 0x5650ab0284d0 2024-09-21T00:31:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.405+0000 7f17922ca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17841b10d0 con 0x5650ab0491b0 2024-09-21T00:31:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.406+0000 7f1789200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0202f0 0x5650ab3ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.406+0000 7f177be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 0x5650ab3eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.406+0000 7f177be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 0x5650ab3eaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:46616/0 (socket says 172.21.15.27:46616) 2024-09-21T00:31:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.406+0000 7f177be00640 1 -- 172.21.15.27:0/608687850 learned_addr learned my addr 172.21.15.27:0/608687850 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.406+0000 7f1789200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0202f0 0x5650ab3ef800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44324/0 (socket says 172.21.15.27:44324) 2024-09-21T00:31:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.406+0000 7f1789200640 1 --2- 172.21.15.27:0/608687850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0491b0 0x5650ab405820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.406+0000 7f177be00640 1 -- 172.21.15.27:0/608687850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0491b0 msgr2=0x5650ab405820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.406+0000 7f177be00640 1 --2- 172.21.15.27:0/608687850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0491b0 0x5650ab405820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.406+0000 7f177be00640 1 -- 172.21.15.27:0/608687850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0202f0 msgr2=0x5650ab3ef800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.406+0000 7f177be00640 1 --2- 172.21.15.27:0/608687850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0202f0 0x5650ab3ef800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.406+0000 7f177be00640 1 -- 172.21.15.27:0/608687850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650ab3f0900 con 0x5650ab0284d0 2024-09-21T00:31:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.406+0000 7f1789200640 1 --2- 172.21.15.27:0/608687850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0491b0 0x5650ab405820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:31:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.406+0000 7f177be00640 1 --2- 172.21.15.27:0/608687850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 0x5650ab3eaa70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5650ab28ab40 tx=0x7f176c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42df029572ca4942 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.406+0000 7f1778c00640 1 -- 172.21.15.27:0/608687850 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c005220 con 0x5650ab0284d0 2024-09-21T00:31:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.406+0000 7f1778c00640 1 -- 172.21.15.27:0/608687850 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176c004c50 con 0x5650ab0284d0 2024-09-21T00:31:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.407+0000 7f1778c00640 1 -- 172.21.15.27:0/608687850 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c01ea40 con 0x5650ab0284d0 2024-09-21T00:31:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.407+0000 7f17922ca740 1 -- 172.21.15.27:0/608687850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 msgr2=0x5650ab3eaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.407+0000 7f17922ca740 1 --2- 172.21.15.27:0/608687850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 0x5650ab3eaa70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5650ab28ab40 tx=0x7f176c005f90 comp rx=0 tx=0).stop 2024-09-21T00:31:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.408+0000 7f17922ca740 1 -- 172.21.15.27:0/608687850 shutdown_connections 2024-09-21T00:31:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.408+0000 7f17922ca740 1 --2- 172.21.15.27:0/608687850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0491b0 0x5650ab405820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.408+0000 7f17922ca740 1 --2- 172.21.15.27:0/608687850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 0x5650ab3eaa70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.408+0000 7f17922ca740 1 --2- 172.21.15.27:0/608687850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0202f0 0x5650ab3ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.408+0000 7f17922ca740 1 -- 172.21.15.27:0/608687850 >> 172.21.15.27:0/608687850 conn(0x5650ab03d300 msgr2=0x5650ab3ea360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.408+0000 7f17922ca740 1 -- 172.21.15.27:0/608687850 shutdown_connections 2024-09-21T00:31:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.408+0000 7f17922ca740 1 -- 172.21.15.27:0/608687850 wait complete. 2024-09-21T00:31:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.408+0000 7f17922ca740 1 -- start start 2024-09-21T00:31:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.408+0000 7f17922ca740 1 Processor -- start 2024-09-21T00:31:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.409+0000 7f17922ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0202f0 0x5650ab40ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.409+0000 7f17922ca740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 0x5650ab40c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.409+0000 7f17922ca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0491b0 0x5650ab40c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.409+0000 7f17922ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f17841b1720 con 0x5650ab0202f0 2024-09-21T00:31:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.409+0000 7f17922ca740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f17841b2b90 con 0x5650ab0284d0 2024-09-21T00:31:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.409+0000 7f17922ca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17841b1df0 con 0x5650ab0491b0 2024-09-21T00:31:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.409+0000 7f17922ca740 10 client.0 _ll_register_callbacks cb 0x7f176c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:31:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.409+0000 7f1789200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0202f0 0x5650ab40ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.409+0000 7f1789200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0202f0 0x5650ab40ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44330/0 (socket says 172.21.15.27:44330) 2024-09-21T00:31:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.409+0000 7f1789200640 1 -- 172.21.15.27:0/461717923 learned_addr learned my addr 172.21.15.27:0/461717923 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:31:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.409+0000 7f177be00640 1 --2- 172.21.15.27:0/461717923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 0x5650ab40c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.409+0000 7f1789200640 1 --2- 172.21.15.27:0/461717923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0491b0 0x5650ab40c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.410+0000 7f177be00640 1 -- 172.21.15.27:0/461717923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0491b0 msgr2=0x5650ab40c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.410+0000 7f177be00640 1 --2- 172.21.15.27:0/461717923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0491b0 0x5650ab40c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.410+0000 7f177be00640 1 -- 172.21.15.27:0/461717923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0202f0 msgr2=0x5650ab40ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:31:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.410+0000 7f177be00640 1 --2- 172.21.15.27:0/461717923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0202f0 0x5650ab40ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.410+0000 7f177be00640 1 -- 172.21.15.27:0/461717923 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650ab40cbf0 con 0x5650ab0284d0 2024-09-21T00:31:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.410+0000 7f1789200640 1 --2- 172.21.15.27:0/461717923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0202f0 0x5650ab40ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:31:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.410+0000 7f177be00640 1 --2- 172.21.15.27:0/461717923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 0x5650ab40c0c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5650ab3f0230 tx=0x7f176c004eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=889cf94550f0b49e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:31:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.410+0000 7f1772000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:31:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.410+0000 7f1772000640 1 -- 172.21.15.27:0/461717923 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c004800 con 0x5650ab0284d0 2024-09-21T00:31:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.410+0000 7f1772000640 1 -- 172.21.15.27:0/461717923 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f176c0049a0 con 0x5650ab0284d0 2024-09-21T00:31:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.410+0000 7f17922ca740 1 -- 172.21.15.27:0/461717923 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f176c004c50 con 0x5650ab0284d0 2024-09-21T00:31:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.410+0000 7f1772000640 1 -- 172.21.15.27:0/461717923 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f176c038050 con 0x5650ab0284d0 2024-09-21T00:31:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.410+0000 7f17922ca740 10 client.26459 fetch_fsmap learned FSMap version 40 2024-09-21T00:31:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.410+0000 7f17922ca740 1 -- 172.21.15.27:0/461717923 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5650ab475b00 con 0x5650ab0284d0 2024-09-21T00:31:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.411+0000 7f1772000640 1 -- 172.21.15.27:0/461717923 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176c004c50 con 0x5650ab0284d0 2024-09-21T00:31:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.411+0000 7f1772000640 1 -- 172.21.15.27:0/461717923 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f176c01ec90 con 0x5650ab0284d0 2024-09-21T00:31:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.411+0000 7f17922ca740 10 client.26459 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T00:31:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.411+0000 7f17922ca740 10 client.26459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:31:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.411+0000 7f17922ca740 20 client.26459 populate_metadata read hostname 'smithi027' 2024-09-21T00:31:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.411+0000 7f17922ca740 1 --2- 172.21.15.27:0/461717923 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5650aafdd1c0 0x5650ab478190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:31:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.411+0000 7f17922ca740 4 client.26459 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:31:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.411+0000 7f17922ca740 1 -- 172.21.15.27:0/461717923 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650ab3610b0 con 0x5650aafdd1c0 2024-09-21T00:31:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.411+0000 7f1789200640 1 --2- 172.21.15.27:0/461717923 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5650aafdd1c0 0x5650ab478190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:31:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.412+0000 7f1789200640 1 --2- 172.21.15.27:0/461717923 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5650aafdd1c0 0x5650ab478190 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f08243252e072088 server_cookie=995b1140ee29a8a3 in_seq=0 out_seq=0 2024-09-21T00:31:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.412+0000 7f1772000640 10 client.26459 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.794+0000 7f1772000640 1 -- 172.21.15.27:0/461717923 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5650ab3610b0 con 0x5650aafdd1c0 2024-09-21T00:31:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.794+0000 7f1772000640 10 client.26459 handle_command_reply: tid=0 2024-09-21T00:31:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.795+0000 7f17922ca740 1 client.26459 shutdown 2024-09-21T00:31:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.795+0000 7f17922ca740 1 -- 172.21.15.27:0/461717923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 msgr2=0x5650ab40c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.795+0000 7f17922ca740 1 --2- 172.21.15.27:0/461717923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 0x5650ab40c0c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5650ab3f0230 tx=0x7f176c004eb0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.795+0000 7f17922ca740 1 -- 172.21.15.27:0/461717923 shutdown_connections 2024-09-21T00:31:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.796+0000 7f17922ca740 1 --2- 172.21.15.27:0/461717923 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5650aafdd1c0 0x5650ab478190 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.796+0000 7f17922ca740 1 --2- 172.21.15.27:0/461717923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5650ab0491b0 0x5650ab40c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.796+0000 7f17922ca740 1 --2- 172.21.15.27:0/461717923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650ab0284d0 0x5650ab40c0c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.796+0000 7f17922ca740 1 --2- 172.21.15.27:0/461717923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5650ab0202f0 0x5650ab40ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.796+0000 7f17922ca740 1 -- 172.21.15.27:0/461717923 >> 172.21.15.27:0/461717923 conn(0x5650ab03d300 msgr2=0x5650ab3ef5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.796+0000 7f1772000640 0 client.26459 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:31:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.796+0000 7f17922ca740 1 -- 172.21.15.27:0/461717923 shutdown_connections 2024-09-21T00:31:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.796+0000 7f17922ca740 1 -- 172.21.15.27:0/461717923 wait complete. 2024-09-21T00:31:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.797+0000 7f17922ca740 20 client.26459 trim_cache size 0 max 16384 2024-09-21T00:31:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:31:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:31:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:31:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:31:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.800+0000 7f1789c00640 1 -- 172.21.15.27:0/415277920 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f17680b6670 msgr2=0x7f17680b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.800+0000 7f1789c00640 1 --2- 172.21.15.27:0/415277920 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f17680b6670 0x7f17680b8b80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x5650ab3eca90 tx=0x7f177402a410 comp rx=0 tx=0).stop 2024-09-21T00:31:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.800+0000 7f1789c00640 1 -- 172.21.15.27:0/415277920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1784105b40 msgr2=0x7f178419c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:31:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.800+0000 7f1789c00640 1 --2- 172.21.15.27:0/415277920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1784105b40 0x7f178419c470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1774005ab0 tx=0x7f1774003c80 comp rx=0 tx=0).stop 2024-09-21T00:31:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.800+0000 7f1789c00640 1 -- 172.21.15.27:0/415277920 shutdown_connections 2024-09-21T00:31:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.800+0000 7f1789c00640 1 --2- 172.21.15.27:0/415277920 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f17680b6670 0x7f17680b8b80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.800+0000 7f1789c00640 1 --2- 172.21.15.27:0/415277920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f178410bdc0 0x7f17841a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.800+0000 7f1789c00640 1 --2- 172.21.15.27:0/415277920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f178410b470 0x7f178419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.800+0000 7f1789c00640 1 --2- 172.21.15.27:0/415277920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1784105b40 0x7f178419c470 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:31:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.800+0000 7f1789c00640 1 -- 172.21.15.27:0/415277920 >> 172.21.15.27:0/415277920 conn(0x7f1784101580 msgr2=0x7f17841039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:31:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.800+0000 7f1789c00640 1 -- 172.21.15.27:0/415277920 shutdown_connections 2024-09-21T00:31:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:31:47.801+0000 7f1789c00640 1 -- 172.21.15.27:0/415277920 wait complete. 2024-09-21T00:31:48.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:47 smithi089 ceph-mon[34481]: pgmap v698: 129 pgs: 129 active+clean; 983 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 558 KiB/s wr, 60 op/s 2024-09-21T00:31:48.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:47 smithi098 ceph-mon[34725]: pgmap v698: 129 pgs: 129 active+clean; 983 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 558 KiB/s wr, 60 op/s 2024-09-21T00:31:48.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:47 smithi027 ceph-mon[30745]: pgmap v698: 129 pgs: 129 active+clean; 983 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 558 KiB/s wr, 60 op/s 2024-09-21T00:31:49.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:48 smithi089 ceph-mon[34481]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T00:31:49.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:48 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:31:49.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:48 smithi098 ceph-mon[34725]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T00:31:49.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:48 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:31:49.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:48 smithi027 ceph-mon[30745]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T00:31:49.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:48 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:31:50.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:49 smithi089 ceph-mon[34481]: pgmap v700: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 985 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 890 KiB/s wr, 151 op/s 2024-09-21T00:31:50.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:49 smithi098 ceph-mon[34725]: pgmap v700: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 985 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 890 KiB/s wr, 151 op/s 2024-09-21T00:31:50.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:49 smithi027 ceph-mon[30745]: pgmap v700: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 985 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 890 KiB/s wr, 151 op/s 2024-09-21T00:31:51.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:50 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:31:51.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:50 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:31:51.319 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:50 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:31:52.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:51 smithi089 ceph-mon[34481]: pgmap v701: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 987 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.1 MiB/s wr, 244 op/s 2024-09-21T00:31:52.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:51 smithi098 ceph-mon[34725]: pgmap v701: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 987 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.1 MiB/s wr, 244 op/s 2024-09-21T00:31:52.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:51 smithi027 ceph-mon[30745]: pgmap v701: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 987 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.1 MiB/s wr, 244 op/s 2024-09-21T00:31:54.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:53 smithi098 ceph-mon[34725]: pgmap v702: 129 pgs: 20 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 49 active+clean; 995 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 366 op/s 2024-09-21T00:31:54.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:53 smithi027 ceph-mon[30745]: pgmap v702: 129 pgs: 20 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 49 active+clean; 995 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 366 op/s 2024-09-21T00:31:54.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:53 smithi089 ceph-mon[34481]: pgmap v702: 129 pgs: 20 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 49 active+clean; 995 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 366 op/s 2024-09-21T00:31:56.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:55 smithi098 ceph-mon[34725]: pgmap v703: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 997 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 381 op/s 2024-09-21T00:31:56.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:55 smithi027 ceph-mon[30745]: pgmap v703: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 997 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 381 op/s 2024-09-21T00:31:56.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:55 smithi089 ceph-mon[34481]: pgmap v703: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 997 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 381 op/s 2024-09-21T00:31:58.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:57 smithi098 ceph-mon[34725]: pgmap v704: 129 pgs: 12 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 77 active+clean; 995 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 352 op/s 2024-09-21T00:31:58.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:57 smithi027 ceph-mon[30745]: pgmap v704: 129 pgs: 12 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 77 active+clean; 995 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 352 op/s 2024-09-21T00:31:58.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:57 smithi089 ceph-mon[34481]: pgmap v704: 129 pgs: 12 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 77 active+clean; 995 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 352 op/s 2024-09-21T00:32:00.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:31:59 smithi098 ceph-mon[34725]: pgmap v705: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1007 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 373 op/s 2024-09-21T00:32:00.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:31:59 smithi027 ceph-mon[30745]: pgmap v705: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1007 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 373 op/s 2024-09-21T00:32:00.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:31:59 smithi089 ceph-mon[34481]: pgmap v705: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1007 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 373 op/s 2024-09-21T00:32:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.134+0000 7f1219000640 1 -- 172.21.15.27:0/3044225963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1214108a70 msgr2=0x7f121410f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.134+0000 7f1219000640 1 --2- 172.21.15.27:0/3044225963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1214108a70 0x7f121410f2a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f120c009ca0 tx=0x7f120c02a8d0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.134+0000 7f1219000640 1 -- 172.21.15.27:0/3044225963 shutdown_connections 2024-09-21T00:32:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.134+0000 7f1219000640 1 --2- 172.21.15.27:0/3044225963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1214108a70 0x7f121410f2a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.134+0000 7f1219000640 1 --2- 172.21.15.27:0/3044225963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1214108120 0x7f1214108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.134+0000 7f1219000640 1 --2- 172.21.15.27:0/3044225963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12141027f0 0x7f1214102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.134+0000 7f1219000640 1 -- 172.21.15.27:0/3044225963 >> 172.21.15.27:0/3044225963 conn(0x7f12140fe210 msgr2=0x7f1214100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.134+0000 7f1219000640 1 -- 172.21.15.27:0/3044225963 shutdown_connections 2024-09-21T00:32:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.134+0000 7f1219000640 1 -- 172.21.15.27:0/3044225963 wait complete. 2024-09-21T00:32:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.136+0000 7f1219000640 1 Processor -- start 2024-09-21T00:32:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.136+0000 7f1219000640 1 -- start start 2024-09-21T00:32:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.137+0000 7f1219000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12141027f0 0x7f121410cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.137+0000 7f1219000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1214108120 0x7f1214109270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.137+0000 7f1219000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1214108a70 0x7f12141099a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.137+0000 7f1219000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1214112800 con 0x7f12141027f0 2024-09-21T00:32:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.138+0000 7f1212a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1214108120 0x7f1214109270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.138+0000 7f1219000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1214112680 con 0x7f1214108120 2024-09-21T00:32:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.138+0000 7f1212a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1214108120 0x7f1214109270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39386/0 (socket says 172.21.15.27:39386) 2024-09-21T00:32:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.138+0000 7f1219000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1214112980 con 0x7f1214108a70 2024-09-21T00:32:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.138+0000 7f1212a00640 1 -- 172.21.15.27:0/2830089344 learned_addr learned my addr 172.21.15.27:0/2830089344 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.138+0000 7f1213400640 1 --2- 172.21.15.27:0/2830089344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12141027f0 0x7f121410cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.138+0000 7f1213e00640 1 --2- 172.21.15.27:0/2830089344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1214108a70 0x7f12141099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.138+0000 7f1212a00640 1 -- 172.21.15.27:0/2830089344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1214108a70 msgr2=0x7f12141099a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.138+0000 7f1212a00640 1 --2- 172.21.15.27:0/2830089344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1214108a70 0x7f12141099a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.138+0000 7f1212a00640 1 -- 172.21.15.27:0/2830089344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12141027f0 msgr2=0x7f121410cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.139+0000 7f1212a00640 1 --2- 172.21.15.27:0/2830089344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12141027f0 0x7f121410cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.139+0000 7f1212a00640 1 -- 172.21.15.27:0/2830089344 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12141ac9e0 con 0x7f1214108120 2024-09-21T00:32:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.140+0000 7f1212a00640 1 --2- 172.21.15.27:0/2830089344 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1214108120 0x7f1214109270 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f120400bee0 tx=0x7f120400dca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.140+0000 7f120be00640 1 -- 172.21.15.27:0/2830089344 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204028070 con 0x7f1214108120 2024-09-21T00:32:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.140+0000 7f120be00640 1 -- 172.21.15.27:0/2830089344 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1204009c00 con 0x7f1214108120 2024-09-21T00:32:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.140+0000 7f120be00640 1 -- 172.21.15.27:0/2830089344 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204005640 con 0x7f1214108120 2024-09-21T00:32:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.141+0000 7f1219000640 1 -- 172.21.15.27:0/2830089344 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121419c1e0 con 0x7f1214108120 2024-09-21T00:32:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.141+0000 7f1219000640 1 -- 172.21.15.27:0/2830089344 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f121419c6e0 con 0x7f1214108120 2024-09-21T00:32:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.146+0000 7f12216c8740 1 Processor -- start 2024-09-21T00:32:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.149+0000 7f12216c8740 1 -- start start 2024-09-21T00:32:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.149+0000 7f12216c8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a7f1b0 0x565050e23e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.152+0000 7f12216c8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a562f0 0x565050e24350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.152+0000 7f12216c8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 0x565050e24890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.152+0000 7f12216c8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x565050d996b0 con 0x565050a5e4d0 2024-09-21T00:32:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.152+0000 7f12216c8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x565050d88ae0 con 0x565050a562f0 2024-09-21T00:32:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.152+0000 7f12216c8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x565050d8c8f0 con 0x565050a7f1b0 2024-09-21T00:32:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.152+0000 7f120be00640 1 -- 172.21.15.27:0/2830089344 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1204004020 con 0x7f1214108120 2024-09-21T00:32:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.152+0000 7f1213e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a7f1b0 0x565050e23e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.152+0000 7f1213e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a7f1b0 0x565050e23e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:48954/0 (socket says 172.21.15.27:48954) 2024-09-21T00:32:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.152+0000 7f1213e00640 1 -- 172.21.15.27:0/811523689 learned_addr learned my addr 172.21.15.27:0/811523689 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.152+0000 7f1213400640 1 --2- 172.21.15.27:0/811523689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a562f0 0x565050e24350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.153+0000 7f1213e00640 1 -- 172.21.15.27:0/811523689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a562f0 msgr2=0x565050e24350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.153+0000 7f1213e00640 1 --2- 172.21.15.27:0/811523689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a562f0 0x565050e24350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.153+0000 7f1213e00640 1 -- 172.21.15.27:0/811523689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 msgr2=0x565050e24890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:32:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.153+0000 7f1213e00640 1 --2- 172.21.15.27:0/811523689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 0x565050e24890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.153+0000 7f1213e00640 1 -- 172.21.15.27:0/811523689 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565050d55780 con 0x565050a7f1b0 2024-09-21T00:32:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.154+0000 7f1213e00640 1 --2- 172.21.15.27:0/811523689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a7f1b0 0x565050e23e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f120c035000 tx=0x7f120c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb97e74d9548e63a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.155+0000 7f1208c00640 1 -- 172.21.15.27:0/811523689 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c031d00 con 0x565050a7f1b0 2024-09-21T00:32:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.155+0000 7f1208c00640 1 -- 172.21.15.27:0/811523689 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f120c036070 con 0x565050a7f1b0 2024-09-21T00:32:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.155+0000 7f1208c00640 1 -- 172.21.15.27:0/811523689 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c0042a0 con 0x565050a7f1b0 2024-09-21T00:32:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.156+0000 7f120be00640 1 --2- 172.21.15.27:0/2830089344 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f11f80b6890 0x7f11f80b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.156+0000 7f120be00640 1 -- 172.21.15.27:0/2830089344 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9607+0+0 (secure 0 0 0) 0x7f12040f87f0 con 0x7f1214108120 2024-09-21T00:32:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.156+0000 7f12216c8740 1 -- 172.21.15.27:0/811523689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a7f1b0 msgr2=0x565050e23e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.156+0000 7f12216c8740 1 --2- 172.21.15.27:0/811523689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a7f1b0 0x565050e23e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f120c035000 tx=0x7f120c0099e0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.157+0000 7f12216c8740 1 -- 172.21.15.27:0/811523689 shutdown_connections 2024-09-21T00:32:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.157+0000 7f12216c8740 1 --2- 172.21.15.27:0/811523689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 0x565050e24890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.157+0000 7f12216c8740 1 --2- 172.21.15.27:0/811523689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a562f0 0x565050e24350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.157+0000 7f12216c8740 1 --2- 172.21.15.27:0/811523689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a7f1b0 0x565050e23e10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.157+0000 7f12216c8740 1 -- 172.21.15.27:0/811523689 >> 172.21.15.27:0/811523689 conn(0x565050a73300 msgr2=0x565050e21d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.157+0000 7f1213400640 1 --2- 172.21.15.27:0/2830089344 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f11f80b6890 0x7f11f80b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.157+0000 7f12216c8740 1 -- 172.21.15.27:0/811523689 shutdown_connections 2024-09-21T00:32:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.158+0000 7f12216c8740 1 -- 172.21.15.27:0/811523689 wait complete. 2024-09-21T00:32:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.158+0000 7f12216c8740 1 -- start start 2024-09-21T00:32:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.159+0000 7f12216c8740 1 Processor -- start 2024-09-21T00:32:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.159+0000 7f1213400640 1 --2- 172.21.15.27:0/2830089344 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f11f80b6890 0x7f11f80b8da0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x565050e22a90 tx=0x7f11fc008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.160+0000 7f12216c8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e2f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.160+0000 7f12216c8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a5e4d0 0x565050e2f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.160+0000 7f1213400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e2f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.160+0000 7f1213400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e2f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:48958/0 (socket says 172.21.15.27:48958) 2024-09-21T00:32:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.161+0000 7f1212a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a5e4d0 0x565050e2f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.161+0000 7f1212a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a5e4d0 0x565050e2f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39404/0 (socket says 172.21.15.27:39404) 2024-09-21T00:32:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.161+0000 7f1213400640 1 -- 172.21.15.27:0/1589442039 learned_addr learned my addr 172.21.15.27:0/1589442039 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.161+0000 7f12216c8740 1 --2- 172.21.15.27:0/1589442039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a7f1b0 0x565050e3b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.161+0000 7f12216c8740 1 -- 172.21.15.27:0/1589442039 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x565050cc0ab0 con 0x565050a7f1b0 2024-09-21T00:32:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.161+0000 7f12216c8740 1 -- 172.21.15.27:0/1589442039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x565050d2f570 con 0x565050a5e4d0 2024-09-21T00:32:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.161+0000 7f12216c8740 1 -- 172.21.15.27:0/1589442039 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x565050e32530 con 0x565050a562f0 2024-09-21T00:32:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.162+0000 7f1212a00640 1 -- 172.21.15.27:0/1589442039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 msgr2=0x565050e2f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.162+0000 7f1212a00640 1 --2- 172.21.15.27:0/1589442039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e2f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.162+0000 7f1212a00640 1 -- 172.21.15.27:0/1589442039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a7f1b0 msgr2=0x565050e3b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.162+0000 7f1212a00640 1 --2- 172.21.15.27:0/1589442039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a7f1b0 0x565050e3b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.162+0000 7f1212a00640 1 -- 172.21.15.27:0/1589442039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565050aea790 con 0x565050a5e4d0 2024-09-21T00:32:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.162+0000 7f1213400640 1 --2- 172.21.15.27:0/1589442039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e2f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T00:32:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.162+0000 7f12216c8740 10 client.0 _ll_register_callbacks cb 0x565050d3e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:32:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.162+0000 7f12216c8740 1 -- 172.21.15.27:0/1589442039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120c036070 con 0x565050a5e4d0 2024-09-21T00:32:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.162+0000 7f1212a00640 1 --2- 172.21.15.27:0/1589442039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a5e4d0 0x565050e2f9c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f12040f5b00 tx=0x7f12040ba960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a361001d5d1b9428 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.162+0000 7f1201600640 10 client.26471 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:32:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.163+0000 7f1201600640 1 -- 172.21.15.27:0/1589442039 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12040be900 con 0x565050a5e4d0 2024-09-21T00:32:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.163+0000 7f1201600640 1 -- 172.21.15.27:0/1589442039 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9607+0+0 (secure 0 0 0) 0x7f12040f8390 con 0x565050a5e4d0 2024-09-21T00:32:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.163+0000 7f1201600640 1 -- 172.21.15.27:0/1589442039 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f12040f8590 con 0x565050a5e4d0 2024-09-21T00:32:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.163+0000 7f12216c8740 10 client.26471 fetch_fsmap learned FSMap version 42 2024-09-21T00:32:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.163+0000 7f12216c8740 1 -- 172.21.15.27:0/1589442039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({fsmap=42}) -- 0x565050b63620 con 0x565050a5e4d0 2024-09-21T00:32:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.164+0000 7f1201600640 1 -- 172.21.15.27:0/1589442039 <== mon.2 v2:172.21.15.89:3300/0 4 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7f12040f8070 con 0x565050a5e4d0 2024-09-21T00:32:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.164+0000 7f12216c8740 10 client.26471 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T00:32:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.164+0000 7f12216c8740 10 client.26471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:32:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.164+0000 7f12216c8740 20 client.26471 populate_metadata read hostname 'smithi027' 2024-09-21T00:32:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.164+0000 7f12216c8740 1 --2- 172.21.15.27:0/1589442039 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x565050a3dd80 0x565050e414c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.164+0000 7f12216c8740 4 client.26471 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:32:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.164+0000 7f12216c8740 1 -- 172.21.15.27:0/1589442039 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565050d9b600 con 0x565050a3dd80 2024-09-21T00:32:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.165+0000 7f1213400640 1 --2- 172.21.15.27:0/1589442039 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x565050a3dd80 0x565050e414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.166+0000 7f1201600640 10 client.26471 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.166+0000 7f1213400640 1 --2- 172.21.15.27:0/1589442039 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x565050a3dd80 0x565050e414c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=847baa891405ba0a server_cookie=29a3854cb67b5b9f in_seq=0 out_seq=0 2024-09-21T00:32:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.169+0000 7f1201600640 1 -- 172.21.15.27:0/1589442039 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565050d9b600 con 0x565050a3dd80 2024-09-21T00:32:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.169+0000 7f1201600640 10 client.26471 handle_command_reply: tid=0 2024-09-21T00:32:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.169+0000 7f12216c8740 1 client.26471 shutdown 2024-09-21T00:32:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.169+0000 7f12216c8740 1 -- 172.21.15.27:0/1589442039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a5e4d0 msgr2=0x565050e2f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.169+0000 7f12216c8740 1 --2- 172.21.15.27:0/1589442039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a5e4d0 0x565050e2f9c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f12040f5b00 tx=0x7f12040ba960 comp rx=0 tx=0).stop 2024-09-21T00:32:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.170+0000 7f12216c8740 1 -- 172.21.15.27:0/1589442039 shutdown_connections 2024-09-21T00:32:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.170+0000 7f12216c8740 1 --2- 172.21.15.27:0/1589442039 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x565050a3dd80 0x565050e414c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.170+0000 7f12216c8740 1 --2- 172.21.15.27:0/1589442039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a7f1b0 0x565050e3b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.170+0000 7f12216c8740 1 --2- 172.21.15.27:0/1589442039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a5e4d0 0x565050e2f9c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.170+0000 7f12216c8740 1 --2- 172.21.15.27:0/1589442039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e2f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.170+0000 7f12216c8740 1 -- 172.21.15.27:0/1589442039 >> 172.21.15.27:0/1589442039 conn(0x565050a73300 msgr2=0x565050e24e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.170+0000 7f1201600640 0 client.26471 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.170+0000 7f12216c8740 1 -- 172.21.15.27:0/1589442039 shutdown_connections 2024-09-21T00:32:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.170+0000 7f12216c8740 1 -- 172.21.15.27:0/1589442039 wait complete. 2024-09-21T00:32:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.170+0000 7f12216c8740 20 client.26471 trim_cache size 0 max 16384 2024-09-21T00:32:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.207+0000 7f12216c8740 1 Processor -- start 2024-09-21T00:32:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.207+0000 7f12216c8740 1 -- start start 2024-09-21T00:32:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.208+0000 7f12216c8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e20d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.208+0000 7f12216c8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 0x565050e2a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.208+0000 7f12216c8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a7f1b0 0x565050e3b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.208+0000 7f12216c8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f12141b1550 con 0x565050a5e4d0 2024-09-21T00:32:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.208+0000 7f12216c8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f12141aedf0 con 0x565050a7f1b0 2024-09-21T00:32:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.208+0000 7f12216c8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f121410a570 con 0x565050a562f0 2024-09-21T00:32:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.209+0000 7f1213400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e20d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.209+0000 7f1213400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e20d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:48962/0 (socket says 172.21.15.27:48962) 2024-09-21T00:32:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.209+0000 7f1213400640 1 -- 172.21.15.27:0/2279125831 learned_addr learned my addr 172.21.15.27:0/2279125831 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.209+0000 7f1213e00640 1 --2- 172.21.15.27:0/2279125831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a7f1b0 0x565050e3b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.209+0000 7f1212a00640 1 --2- 172.21.15.27:0/2279125831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 0x565050e2a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.210+0000 7f1213400640 1 -- 172.21.15.27:0/2279125831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a7f1b0 msgr2=0x565050e3b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.210+0000 7f1213400640 1 --2- 172.21.15.27:0/2279125831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a7f1b0 0x565050e3b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.210+0000 7f1213400640 1 -- 172.21.15.27:0/2279125831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 msgr2=0x565050e2a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.210+0000 7f1213400640 1 --2- 172.21.15.27:0/2279125831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 0x565050e2a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.210+0000 7f1213400640 1 -- 172.21.15.27:0/2279125831 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565050a3dfb0 con 0x565050a562f0 2024-09-21T00:32:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.211+0000 7f1213400640 1 --2- 172.21.15.27:0/2279125831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e20d20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f11fc006fd0 tx=0x7f11fc012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca260849e53fa33c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.211+0000 7f1203e00640 1 -- 172.21.15.27:0/2279125831 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc01b040 con 0x565050a562f0 2024-09-21T00:32:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.212+0000 7f1203e00640 1 -- 172.21.15.27:0/2279125831 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11fc02e070 con 0x565050a562f0 2024-09-21T00:32:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.212+0000 7f1203e00640 1 -- 172.21.15.27:0/2279125831 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc01b040 con 0x565050a562f0 2024-09-21T00:32:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.214+0000 7f12216c8740 1 -- 172.21.15.27:0/2279125831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 msgr2=0x565050e20d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.214+0000 7f12216c8740 1 --2- 172.21.15.27:0/2279125831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e20d20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f11fc006fd0 tx=0x7f11fc012000 comp rx=0 tx=0).stop 2024-09-21T00:32:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.215+0000 7f12216c8740 1 -- 172.21.15.27:0/2279125831 shutdown_connections 2024-09-21T00:32:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.215+0000 7f12216c8740 1 --2- 172.21.15.27:0/2279125831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a7f1b0 0x565050e3b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.215+0000 7f12216c8740 1 --2- 172.21.15.27:0/2279125831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 0x565050e2a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.215+0000 7f12216c8740 1 --2- 172.21.15.27:0/2279125831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e20d20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.215+0000 7f12216c8740 1 -- 172.21.15.27:0/2279125831 >> 172.21.15.27:0/2279125831 conn(0x565050a73300 msgr2=0x565050e2a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.215+0000 7f12216c8740 1 -- 172.21.15.27:0/2279125831 shutdown_connections 2024-09-21T00:32:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.216+0000 7f12216c8740 1 -- 172.21.15.27:0/2279125831 wait complete. 2024-09-21T00:32:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.216+0000 7f12216c8740 1 -- start start 2024-09-21T00:32:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.217+0000 7f12216c8740 1 Processor -- start 2024-09-21T00:32:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.217+0000 7f12216c8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e32c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.217+0000 7f12216c8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 0x565050e33170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.217+0000 7f12216c8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a7f1b0 0x565050ea9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.217+0000 7f12216c8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f12141a47f0 con 0x565050a5e4d0 2024-09-21T00:32:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.217+0000 7f12216c8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f12141ae720 con 0x565050a7f1b0 2024-09-21T00:32:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.218+0000 7f12216c8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f12141ad980 con 0x565050a562f0 2024-09-21T00:32:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.218+0000 7f12216c8740 10 client.0 _ll_register_callbacks cb 0x7f12040046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:32:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.218+0000 7f1212a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 0x565050e33170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.218+0000 7f1213e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a7f1b0 0x565050ea9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.218+0000 7f1213e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a7f1b0 0x565050ea9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39436/0 (socket says 172.21.15.27:39436) 2024-09-21T00:32:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.218+0000 7f1213e00640 1 -- 172.21.15.27:0/2742988044 learned_addr learned my addr 172.21.15.27:0/2742988044 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.218+0000 7f1212a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 0x565050e33170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56722/0 (socket says 172.21.15.27:56722) 2024-09-21T00:32:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.218+0000 7f1213400640 1 --2- 172.21.15.27:0/2742988044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e32c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.219+0000 7f1213e00640 1 -- 172.21.15.27:0/2742988044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 msgr2=0x565050e32c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.219+0000 7f1213e00640 1 --2- 172.21.15.27:0/2742988044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e32c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.219+0000 7f1213e00640 1 -- 172.21.15.27:0/2742988044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 msgr2=0x565050e33170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.219+0000 7f1213e00640 1 --2- 172.21.15.27:0/2742988044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 0x565050e33170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.219+0000 7f1213e00640 1 -- 172.21.15.27:0/2742988044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565050e33a10 con 0x565050a7f1b0 2024-09-21T00:32:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.219+0000 7f1213400640 1 --2- 172.21.15.27:0/2742988044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e32c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:32:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.221+0000 7f1213e00640 1 --2- 172.21.15.27:0/2742988044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a7f1b0 0x565050ea9eb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f120c009a20 tx=0x7f120c031690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dc476f6a3afc65 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.221+0000 7f1201600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:32:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.222+0000 7f1212a00640 1 --2- 172.21.15.27:0/2742988044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 0x565050e33170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:32:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.222+0000 7f1201600640 1 -- 172.21.15.27:0/2742988044 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c003b70 con 0x565050a7f1b0 2024-09-21T00:32:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.222+0000 7f1201600640 1 -- 172.21.15.27:0/2742988044 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9607+0+0 (secure 0 0 0) 0x7f120c003d10 con 0x565050a7f1b0 2024-09-21T00:32:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.222+0000 7f12216c8740 1 -- 172.21.15.27:0/2742988044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11fc02e070 con 0x565050a7f1b0 2024-09-21T00:32:01.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.223+0000 7f1201600640 1 -- 172.21.15.27:0/2742988044 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f120c004040 con 0x565050a7f1b0 2024-09-21T00:32:01.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.223+0000 7f12216c8740 10 client.26477 fetch_fsmap learned FSMap version 42 2024-09-21T00:32:01.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.223+0000 7f12216c8740 1 -- 172.21.15.27:0/2742988044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x565050ebc460 con 0x565050a7f1b0 2024-09-21T00:32:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.224+0000 7f1201600640 1 -- 172.21.15.27:0/2742988044 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11fc02e070 con 0x565050a7f1b0 2024-09-21T00:32:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.224+0000 7f1201600640 1 -- 172.21.15.27:0/2742988044 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7f120c043020 con 0x565050a7f1b0 2024-09-21T00:32:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.225+0000 7f12216c8740 10 client.26477 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T00:32:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.225+0000 7f12216c8740 10 client.26477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:32:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.225+0000 7f12216c8740 20 client.26477 populate_metadata read hostname 'smithi027' 2024-09-21T00:32:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.225+0000 7f12216c8740 1 --2- 172.21.15.27:0/2742988044 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x565050a131c0 0x565050ebe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.225+0000 7f12216c8740 4 client.26477 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:32:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.225+0000 7f12216c8740 1 -- 172.21.15.27:0/2742988044 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565050d970b0 con 0x565050a131c0 2024-09-21T00:32:01.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.225+0000 7f1213400640 1 --2- 172.21.15.27:0/2742988044 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x565050a131c0 0x565050ebe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:01.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.226+0000 7f1213400640 1 --2- 172.21.15.27:0/2742988044 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x565050a131c0 0x565050ebe6a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e6e8d41fe524ea server_cookie=2308f4adb5cc29e3 in_seq=0 out_seq=0 2024-09-21T00:32:01.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:01.226+0000 7f1201600640 10 client.26477 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:02.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:02 smithi089 ceph-mon[34481]: pgmap v706: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1007 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 330 op/s 2024-09-21T00:32:02.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:02 smithi089 ceph-mon[34481]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T00:32:02.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:02 smithi027 ceph-mon[30745]: pgmap v706: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1007 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 330 op/s 2024-09-21T00:32:02.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:02 smithi027 ceph-mon[30745]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T00:32:02.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:02 smithi098 ceph-mon[34725]: pgmap v706: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1007 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 330 op/s 2024-09-21T00:32:02.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:02 smithi098 ceph-mon[34725]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T00:32:03.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:03 smithi089 ceph-mon[34481]: osdmap e102: 12 total, 12 up, 12 in 2024-09-21T00:32:03.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:03 smithi098 ceph-mon[34725]: osdmap e102: 12 total, 12 up, 12 in 2024-09-21T00:32:03.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:03 smithi027 ceph-mon[30745]: osdmap e102: 12 total, 12 up, 12 in 2024-09-21T00:32:04.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:04 smithi089 ceph-mon[34481]: pgmap v709: 129 pgs: 129 active+clean; 1012 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 332 op/s 2024-09-21T00:32:04.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:04 smithi098 ceph-mon[34725]: pgmap v709: 129 pgs: 129 active+clean; 1012 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 332 op/s 2024-09-21T00:32:04.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:04 smithi027 ceph-mon[30745]: pgmap v709: 129 pgs: 129 active+clean; 1012 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 332 op/s 2024-09-21T00:32:06.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:06 smithi089 ceph-mon[34481]: pgmap v710: 129 pgs: 38 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 79 active+clean; 1013 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 928 op/s 2024-09-21T00:32:06.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:06 smithi089 ceph-mon[34481]: osdmap e103: 12 total, 12 up, 12 in 2024-09-21T00:32:06.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:06 smithi098 ceph-mon[34725]: pgmap v710: 129 pgs: 38 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 79 active+clean; 1013 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 928 op/s 2024-09-21T00:32:06.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:06 smithi098 ceph-mon[34725]: osdmap e103: 12 total, 12 up, 12 in 2024-09-21T00:32:06.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:06 smithi027 ceph-mon[30745]: pgmap v710: 129 pgs: 38 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 79 active+clean; 1013 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 928 op/s 2024-09-21T00:32:06.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:06 smithi027 ceph-mon[30745]: osdmap e103: 12 total, 12 up, 12 in 2024-09-21T00:32:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.837+0000 7f1201600640 1 -- 172.21.15.27:0/2742988044 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565050d970b0 con 0x565050a131c0 2024-09-21T00:32:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.837+0000 7f1201600640 10 client.26477 handle_command_reply: tid=0 2024-09-21T00:32:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.837+0000 7f12216c8740 1 client.26477 shutdown 2024-09-21T00:32:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.838+0000 7f12216c8740 1 -- 172.21.15.27:0/2742988044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a7f1b0 msgr2=0x565050ea9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.838+0000 7f12216c8740 1 --2- 172.21.15.27:0/2742988044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a7f1b0 0x565050ea9eb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f120c009a20 tx=0x7f120c031690 comp rx=0 tx=0).stop 2024-09-21T00:32:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.838+0000 7f12216c8740 1 -- 172.21.15.27:0/2742988044 shutdown_connections 2024-09-21T00:32:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.838+0000 7f12216c8740 1 --2- 172.21.15.27:0/2742988044 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x565050a131c0 0x565050ebe6a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.838+0000 7f12216c8740 1 --2- 172.21.15.27:0/2742988044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565050a7f1b0 0x565050ea9eb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.838+0000 7f12216c8740 1 --2- 172.21.15.27:0/2742988044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565050a5e4d0 0x565050e33170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.838+0000 7f12216c8740 1 --2- 172.21.15.27:0/2742988044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565050a562f0 0x565050e32c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.838+0000 7f1201600640 0 client.26477 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.838+0000 7f12216c8740 1 -- 172.21.15.27:0/2742988044 >> 172.21.15.27:0/2742988044 conn(0x565050a73300 msgr2=0x565050e276c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.839+0000 7f12216c8740 1 -- 172.21.15.27:0/2742988044 shutdown_connections 2024-09-21T00:32:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.839+0000 7f12216c8740 1 -- 172.21.15.27:0/2742988044 wait complete. 2024-09-21T00:32:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.839+0000 7f12216c8740 20 client.26477 trim_cache size 0 max 16384 2024-09-21T00:32:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:32:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:32:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:32:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:32:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.841+0000 7f1219000640 1 -- 172.21.15.27:0/2830089344 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f11f80b6890 msgr2=0x7f11f80b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.841+0000 7f1219000640 1 --2- 172.21.15.27:0/2830089344 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f11f80b6890 0x7f11f80b8da0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x565050e22a90 tx=0x7f11fc008040 comp rx=0 tx=0).stop 2024-09-21T00:32:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.841+0000 7f1219000640 1 -- 172.21.15.27:0/2830089344 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1214108120 msgr2=0x7f1214109270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.841+0000 7f1219000640 1 --2- 172.21.15.27:0/2830089344 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1214108120 0x7f1214109270 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f120400bee0 tx=0x7f120400dca0 comp rx=0 tx=0).stop 2024-09-21T00:32:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.841+0000 7f1219000640 1 -- 172.21.15.27:0/2830089344 shutdown_connections 2024-09-21T00:32:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.841+0000 7f1219000640 1 --2- 172.21.15.27:0/2830089344 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f11f80b6890 0x7f11f80b8da0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.841+0000 7f1219000640 1 --2- 172.21.15.27:0/2830089344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1214108a70 0x7f12141099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.841+0000 7f1219000640 1 --2- 172.21.15.27:0/2830089344 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1214108120 0x7f1214109270 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.841+0000 7f1219000640 1 --2- 172.21.15.27:0/2830089344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12141027f0 0x7f121410cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.841+0000 7f1219000640 1 -- 172.21.15.27:0/2830089344 >> 172.21.15.27:0/2830089344 conn(0x7f12140fe210 msgr2=0x7f121406ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.843+0000 7f1219000640 1 -- 172.21.15.27:0/2830089344 shutdown_connections 2024-09-21T00:32:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:06.843+0000 7f1219000640 1 -- 172.21.15.27:0/2830089344 wait complete. 2024-09-21T00:32:08.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:08 smithi089 ceph-mon[34481]: pgmap v712: 129 pgs: 55 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 55 active+clean; 1014 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 1.72k op/s 2024-09-21T00:32:08.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:08 smithi098 ceph-mon[34725]: pgmap v712: 129 pgs: 55 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 55 active+clean; 1014 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 1.72k op/s 2024-09-21T00:32:08.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:08 smithi027 ceph-mon[30745]: pgmap v712: 129 pgs: 55 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 55 active+clean; 1014 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 1.72k op/s 2024-09-21T00:32:10.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:10 smithi089 ceph-mon[34481]: pgmap v713: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 1008 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 1.94k op/s 2024-09-21T00:32:10.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:10 smithi098 ceph-mon[34725]: pgmap v713: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 1008 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 1.94k op/s 2024-09-21T00:32:10.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:10 smithi027 ceph-mon[30745]: pgmap v713: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 1008 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 1.94k op/s 2024-09-21T00:32:11.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:11 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:32:11.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:11 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:32:11.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:11 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:32:12.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:12 smithi089 ceph-mon[34481]: pgmap v714: 129 pgs: 57 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 53 active+clean; 1006 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 1.2 MiB/s wr, 1.65k op/s 2024-09-21T00:32:12.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:12 smithi098 ceph-mon[34725]: pgmap v714: 129 pgs: 57 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 53 active+clean; 1006 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 1.2 MiB/s wr, 1.65k op/s 2024-09-21T00:32:12.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:12 smithi027 ceph-mon[30745]: pgmap v714: 129 pgs: 57 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 53 active+clean; 1006 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 1.2 MiB/s wr, 1.65k op/s 2024-09-21T00:32:14.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:14 smithi089 ceph-mon[34481]: pgmap v715: 129 pgs: 52 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 60 active+clean; 1009 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.4 MiB/s wr, 1.40k op/s 2024-09-21T00:32:14.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:14 smithi098 ceph-mon[34725]: pgmap v715: 129 pgs: 52 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 60 active+clean; 1009 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.4 MiB/s wr, 1.40k op/s 2024-09-21T00:32:14.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:14 smithi027 ceph-mon[30745]: pgmap v715: 129 pgs: 52 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 60 active+clean; 1009 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.4 MiB/s wr, 1.40k op/s 2024-09-21T00:32:16.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:16 smithi089 ceph-mon[34481]: pgmap v716: 129 pgs: 44 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 72 active+clean; 1011 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 929 op/s 2024-09-21T00:32:16.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:16 smithi098 ceph-mon[34725]: pgmap v716: 129 pgs: 44 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 72 active+clean; 1011 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 929 op/s 2024-09-21T00:32:16.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:16 smithi027 ceph-mon[30745]: pgmap v716: 129 pgs: 44 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 72 active+clean; 1011 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 929 op/s 2024-09-21T00:32:18.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:18 smithi089 ceph-mon[34481]: pgmap v717: 129 pgs: 36 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 81 active+clean; 1013 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 767 op/s 2024-09-21T00:32:18.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:18 smithi098 ceph-mon[34725]: pgmap v717: 129 pgs: 36 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 81 active+clean; 1013 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 767 op/s 2024-09-21T00:32:18.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:18 smithi027 ceph-mon[30745]: pgmap v717: 129 pgs: 36 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 81 active+clean; 1013 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 767 op/s 2024-09-21T00:32:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.168+0000 7f2543200640 1 -- 172.21.15.27:0/652903617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2544104740 msgr2=0x7f2544104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.168+0000 7f2543200640 1 --2- 172.21.15.27:0/652903617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2544104740 0x7f2544104b50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2534009ca0 tx=0x7f253402a8f0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.168+0000 7f2543200640 1 -- 172.21.15.27:0/652903617 shutdown_connections 2024-09-21T00:32:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.168+0000 7f2543200640 1 --2- 172.21.15.27:0/652903617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f254410dba0 0x7f2544110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.168+0000 7f2543200640 1 --2- 172.21.15.27:0/652903617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2544105090 0x7f254410d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.168+0000 7f2543200640 1 --2- 172.21.15.27:0/652903617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2544104740 0x7f2544104b50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.168+0000 7f2543200640 1 -- 172.21.15.27:0/652903617 >> 172.21.15.27:0/652903617 conn(0x7f25440fe210 msgr2=0x7f2544100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.168+0000 7f2543200640 1 -- 172.21.15.27:0/652903617 shutdown_connections 2024-09-21T00:32:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.168+0000 7f2543200640 1 -- 172.21.15.27:0/652903617 wait complete. 2024-09-21T00:32:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.170+0000 7f2543200640 1 Processor -- start 2024-09-21T00:32:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.170+0000 7f2543200640 1 -- start start 2024-09-21T00:32:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.171+0000 7f2543200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2544104740 0x7f2544103e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.171+0000 7f2543200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2544105090 0x7f2544102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.171+0000 7f2543200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f254410dba0 0x7f25441029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.171+0000 7f2543200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f25441138b0 con 0x7f2544105090 2024-09-21T00:32:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.171+0000 7f2543200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2544113730 con 0x7f254410dba0 2024-09-21T00:32:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.171+0000 7f2543200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2544113a30 con 0x7f2544104740 2024-09-21T00:32:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.172+0000 7f2541400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2544105090 0x7f2544102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.172+0000 7f2541400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2544105090 0x7f2544102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57922/0 (socket says 172.21.15.27:57922) 2024-09-21T00:32:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.172+0000 7f2541400640 1 -- 172.21.15.27:0/755818988 learned_addr learned my addr 172.21.15.27:0/755818988 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.173+0000 7f2542800640 1 --2- 172.21.15.27:0/755818988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f254410dba0 0x7f25441029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.173+0000 7f2541400640 1 -- 172.21.15.27:0/755818988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2544104740 msgr2=0x7f2544103e90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:32:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.173+0000 7f2541e00640 1 --2- 172.21.15.27:0/755818988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2544104740 0x7f2544103e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.173+0000 7f2541400640 1 --2- 172.21.15.27:0/755818988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2544104740 0x7f2544103e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.173+0000 7f2541400640 1 -- 172.21.15.27:0/755818988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f254410dba0 msgr2=0x7f25441029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.173+0000 7f2541400640 1 --2- 172.21.15.27:0/755818988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f254410dba0 0x7f25441029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.173+0000 7f2541400640 1 -- 172.21.15.27:0/755818988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25441031d0 con 0x7f2544105090 2024-09-21T00:32:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.173+0000 7f2542800640 1 --2- 172.21.15.27:0/755818988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f254410dba0 0x7f25441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:32:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.174+0000 7f2541400640 1 --2- 172.21.15.27:0/755818988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2544105090 0x7f2544102470 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f253c006fd0 tx=0x7f253c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.175+0000 7f253aa00640 1 -- 172.21.15.27:0/755818988 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c011830 con 0x7f2544105090 2024-09-21T00:32:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.175+0000 7f253aa00640 1 -- 172.21.15.27:0/755818988 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f253c016070 con 0x7f2544105090 2024-09-21T00:32:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.175+0000 7f2543200640 1 -- 172.21.15.27:0/755818988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254419e400 con 0x7f2544105090 2024-09-21T00:32:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.175+0000 7f253aa00640 1 -- 172.21.15.27:0/755818988 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c010e00 con 0x7f2544105090 2024-09-21T00:32:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.176+0000 7f2543200640 1 -- 172.21.15.27:0/755818988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254419ea20 con 0x7f2544105090 2024-09-21T00:32:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.190+0000 7f2550066740 1 Processor -- start 2024-09-21T00:32:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.190+0000 7f253aa00640 1 -- 172.21.15.27:0/755818988 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f253c01b460 con 0x7f2544105090 2024-09-21T00:32:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.192+0000 7f2550066740 1 -- start start 2024-09-21T00:32:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.192+0000 7f2550066740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670eb1b0 0x56456748fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.192+0000 7f2550066740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670c22f0 0x564567490350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.192+0000 7f2550066740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670ca4d0 0x564567490890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.192+0000 7f2550066740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5645674056b0 con 0x5645670ca4d0 2024-09-21T00:32:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.192+0000 7f2550066740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5645673f4ae0 con 0x5645670c22f0 2024-09-21T00:32:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.192+0000 7f2550066740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5645673f88f0 con 0x5645670eb1b0 2024-09-21T00:32:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.193+0000 7f2542800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670eb1b0 0x56456748fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.193+0000 7f2542800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670eb1b0 0x56456748fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38620/0 (socket says 172.21.15.27:38620) 2024-09-21T00:32:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.193+0000 7f2542800640 1 -- 172.21.15.27:0/122849453 learned_addr learned my addr 172.21.15.27:0/122849453 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.193+0000 7f253aa00640 1 --2- 172.21.15.27:0/755818988 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f25280b66f0 0x7f25280b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.193+0000 7f2541e00640 1 --2- 172.21.15.27:0/122849453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670c22f0 0x564567490350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.193+0000 7f253aa00640 1 -- 172.21.15.27:0/755818988 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9591+0+0 (secure 0 0 0) 0x7f253c0e70e0 con 0x7f2544105090 2024-09-21T00:32:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.194+0000 7f2542800640 1 -- 172.21.15.27:0/122849453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670c22f0 msgr2=0x564567490350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.194+0000 7f2542800640 1 --2- 172.21.15.27:0/122849453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670c22f0 0x564567490350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.194+0000 7f2542800640 1 -- 172.21.15.27:0/122849453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670ca4d0 msgr2=0x564567490890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.194+0000 7f2542800640 1 --2- 172.21.15.27:0/122849453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670ca4d0 0x564567490890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.194+0000 7f2542800640 1 -- 172.21.15.27:0/122849453 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645673c1780 con 0x5645670eb1b0 2024-09-21T00:32:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.194+0000 7f2542800640 1 --2- 172.21.15.27:0/755818988 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f25280b66f0 0x7f25280b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.195+0000 7f2542800640 1 --2- 172.21.15.27:0/122849453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670eb1b0 0x56456748fe10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f252c009e10 tx=0x7f252c0137d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d9eea6d41e6e0b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.196+0000 7f2533400640 1 -- 172.21.15.27:0/122849453 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c01c5c0 con 0x5645670eb1b0 2024-09-21T00:32:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.196+0000 7f2533400640 1 -- 172.21.15.27:0/122849453 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f252c01cc10 con 0x5645670eb1b0 2024-09-21T00:32:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.196+0000 7f2533400640 1 -- 172.21.15.27:0/122849453 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c01ba40 con 0x5645670eb1b0 2024-09-21T00:32:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.198+0000 7f2550066740 1 -- 172.21.15.27:0/122849453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670eb1b0 msgr2=0x56456748fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.198+0000 7f2550066740 1 --2- 172.21.15.27:0/122849453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670eb1b0 0x56456748fe10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f252c009e10 tx=0x7f252c0137d0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.198+0000 7f2542800640 1 --2- 172.21.15.27:0/755818988 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f25280b66f0 0x7f25280b8c00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f252c021000 tx=0x7f252c01e000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.200+0000 7f2550066740 1 -- 172.21.15.27:0/122849453 shutdown_connections 2024-09-21T00:32:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.200+0000 7f2550066740 1 --2- 172.21.15.27:0/122849453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670ca4d0 0x564567490890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.200+0000 7f2550066740 1 --2- 172.21.15.27:0/122849453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670c22f0 0x564567490350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.200+0000 7f2550066740 1 --2- 172.21.15.27:0/122849453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670eb1b0 0x56456748fe10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.200+0000 7f2550066740 1 -- 172.21.15.27:0/122849453 >> 172.21.15.27:0/122849453 conn(0x5645670df300 msgr2=0x56456748dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.201+0000 7f2550066740 1 -- 172.21.15.27:0/122849453 shutdown_connections 2024-09-21T00:32:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.201+0000 7f2550066740 1 -- 172.21.15.27:0/122849453 wait complete. 2024-09-21T00:32:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.202+0000 7f2550066740 1 -- start start 2024-09-21T00:32:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.202+0000 7f2550066740 1 Processor -- start 2024-09-21T00:32:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.203+0000 7f2550066740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670c22f0 0x56456749b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.203+0000 7f2550066740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670ca4d0 0x56456749b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.203+0000 7f2550066740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670eb1b0 0x56456749be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.203+0000 7f2550066740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56456732cab0 con 0x5645670c22f0 2024-09-21T00:32:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.203+0000 7f2550066740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56456739b570 con 0x5645670eb1b0 2024-09-21T00:32:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.203+0000 7f2550066740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56456749e590 con 0x5645670ca4d0 2024-09-21T00:32:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.203+0000 7f2550066740 10 client.0 _ll_register_callbacks cb 0x5645673aa100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:32:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.204+0000 7f2541400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670ca4d0 0x56456749b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.204+0000 7f2541400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670ca4d0 0x56456749b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38630/0 (socket says 172.21.15.27:38630) 2024-09-21T00:32:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.204+0000 7f2541400640 1 -- 172.21.15.27:0/2012351278 learned_addr learned my addr 172.21.15.27:0/2012351278 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.204+0000 7f2542800640 1 --2- 172.21.15.27:0/2012351278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670eb1b0 0x56456749be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.205+0000 7f2541e00640 1 --2- 172.21.15.27:0/2012351278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670c22f0 0x56456749b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.205+0000 7f2541400640 1 -- 172.21.15.27:0/2012351278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670eb1b0 msgr2=0x56456749be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.205+0000 7f2541400640 1 --2- 172.21.15.27:0/2012351278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670eb1b0 0x56456749be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.205+0000 7f2541400640 1 -- 172.21.15.27:0/2012351278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670c22f0 msgr2=0x56456749b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.205+0000 7f2541400640 1 --2- 172.21.15.27:0/2012351278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670c22f0 0x56456749b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.205+0000 7f2541400640 1 -- 172.21.15.27:0/2012351278 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564567156790 con 0x5645670ca4d0 2024-09-21T00:32:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.205+0000 7f2541400640 1 --2- 172.21.15.27:0/2012351278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670ca4d0 0x56456749b8e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f253c0a9d70 tx=0x7f253c010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37c6d73c5f9ee980 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.205+0000 7f2527e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:32:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.205+0000 7f2527e00640 1 -- 172.21.15.27:0/2012351278 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c0a8270 con 0x5645670ca4d0 2024-09-21T00:32:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.205+0000 7f2527e00640 1 -- 172.21.15.27:0/2012351278 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9591+0+0 (secure 0 0 0) 0x7f253c0e4b50 con 0x5645670ca4d0 2024-09-21T00:32:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.206+0000 7f2550066740 1 -- 172.21.15.27:0/2012351278 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f252c01cc10 con 0x5645670ca4d0 2024-09-21T00:32:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.207+0000 7f2527e00640 1 -- 172.21.15.27:0/2012351278 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f253c0ac610 con 0x5645670ca4d0 2024-09-21T00:32:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.207+0000 7f2550066740 10 client.26503 fetch_fsmap learned FSMap version 43 2024-09-21T00:32:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.207+0000 7f2550066740 1 -- 172.21.15.27:0/2012351278 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5645671cf620 con 0x5645670ca4d0 2024-09-21T00:32:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.207+0000 7f2527e00640 1 -- 172.21.15.27:0/2012351278 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f252c01cc10 con 0x5645670ca4d0 2024-09-21T00:32:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.208+0000 7f2527e00640 1 -- 172.21.15.27:0/2012351278 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f253c0a8450 con 0x5645670ca4d0 2024-09-21T00:32:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.209+0000 7f2550066740 10 client.26503 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T00:32:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.209+0000 7f2550066740 10 client.26503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:32:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.210+0000 7f2550066740 20 client.26503 populate_metadata read hostname 'smithi027' 2024-09-21T00:32:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.210+0000 7f2550066740 1 --2- 172.21.15.27:0/2012351278 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5645670a9d80 0x5645674ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.210+0000 7f2550066740 4 client.26503 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:32:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.210+0000 7f2550066740 1 -- 172.21.15.27:0/2012351278 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564567407600 con 0x5645670a9d80 2024-09-21T00:32:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.210+0000 7f2541e00640 1 --2- 172.21.15.27:0/2012351278 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5645670a9d80 0x5645674ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.211+0000 7f2541e00640 1 --2- 172.21.15.27:0/2012351278 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5645670a9d80 0x5645674ad4a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecf5a8d7b46cd9fd server_cookie=3385dbc06425bf4b in_seq=0 out_seq=0 2024-09-21T00:32:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.211+0000 7f2527e00640 10 client.26503 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.212+0000 7f2527e00640 1 -- 172.21.15.27:0/2012351278 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564567407600 con 0x5645670a9d80 2024-09-21T00:32:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.213+0000 7f2527e00640 10 client.26503 handle_command_reply: tid=0 2024-09-21T00:32:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.213+0000 7f2550066740 1 client.26503 shutdown 2024-09-21T00:32:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.214+0000 7f2550066740 1 -- 172.21.15.27:0/2012351278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670ca4d0 msgr2=0x56456749b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.214+0000 7f2550066740 1 --2- 172.21.15.27:0/2012351278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670ca4d0 0x56456749b8e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f253c0a9d70 tx=0x7f253c010000 comp rx=0 tx=0).stop 2024-09-21T00:32:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.214+0000 7f2550066740 1 -- 172.21.15.27:0/2012351278 shutdown_connections 2024-09-21T00:32:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.214+0000 7f2550066740 1 --2- 172.21.15.27:0/2012351278 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5645670a9d80 0x5645674ad4a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.214+0000 7f2550066740 1 --2- 172.21.15.27:0/2012351278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670eb1b0 0x56456749be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.214+0000 7f2550066740 1 --2- 172.21.15.27:0/2012351278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670ca4d0 0x56456749b8e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.214+0000 7f2550066740 1 --2- 172.21.15.27:0/2012351278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670c22f0 0x56456749b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.214+0000 7f2550066740 1 -- 172.21.15.27:0/2012351278 >> 172.21.15.27:0/2012351278 conn(0x5645670df300 msgr2=0x564567490e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.214+0000 7f2527e00640 0 client.26503 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.215+0000 7f2550066740 1 -- 172.21.15.27:0/2012351278 shutdown_connections 2024-09-21T00:32:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.215+0000 7f2550066740 1 -- 172.21.15.27:0/2012351278 wait complete. 2024-09-21T00:32:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.215+0000 7f2550066740 20 client.26503 trim_cache size 0 max 16384 2024-09-21T00:32:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.249+0000 7f2550066740 1 Processor -- start 2024-09-21T00:32:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.250+0000 7f2550066740 1 -- start start 2024-09-21T00:32:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.250+0000 7f2550066740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670c22f0 0x56456749cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.250+0000 7f2550066740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670ca4d0 0x564567498570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.250+0000 7f2550066740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670eb1b0 0x564567498ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.250+0000 7f2550066740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f25441c1f30 con 0x5645670ca4d0 2024-09-21T00:32:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.250+0000 7f2550066740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f25441c2600 con 0x5645670eb1b0 2024-09-21T00:32:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.250+0000 7f2550066740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f25441bfe70 con 0x5645670c22f0 2024-09-21T00:32:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.251+0000 7f2542800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670eb1b0 0x564567498ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.251+0000 7f2541e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670c22f0 0x56456749cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.251+0000 7f2542800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670eb1b0 0x564567498ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:56970/0 (socket says 172.21.15.27:56970) 2024-09-21T00:32:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.251+0000 7f2542800640 1 -- 172.21.15.27:0/438736006 learned_addr learned my addr 172.21.15.27:0/438736006 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.251+0000 7f2541400640 1 --2- 172.21.15.27:0/438736006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670ca4d0 0x564567498570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.252+0000 7f2542800640 1 -- 172.21.15.27:0/438736006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670c22f0 msgr2=0x56456749cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.252+0000 7f2542800640 1 --2- 172.21.15.27:0/438736006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670c22f0 0x56456749cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.252+0000 7f2542800640 1 -- 172.21.15.27:0/438736006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670ca4d0 msgr2=0x564567498570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.252+0000 7f2542800640 1 --2- 172.21.15.27:0/438736006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670ca4d0 0x564567498570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.252+0000 7f2542800640 1 -- 172.21.15.27:0/438736006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56456749e470 con 0x5645670eb1b0 2024-09-21T00:32:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.253+0000 7f2542800640 1 --2- 172.21.15.27:0/438736006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670eb1b0 0x564567498ab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f252c009de0 tx=0x7f252c012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b62981dfe580279 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.253+0000 7f2532a00640 1 -- 172.21.15.27:0/438736006 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c0130e0 con 0x5645670eb1b0 2024-09-21T00:32:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.254+0000 7f2532a00640 1 -- 172.21.15.27:0/438736006 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f252c01bcf0 con 0x5645670eb1b0 2024-09-21T00:32:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.254+0000 7f2532a00640 1 -- 172.21.15.27:0/438736006 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c0053d0 con 0x5645670eb1b0 2024-09-21T00:32:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.255+0000 7f2550066740 1 -- 172.21.15.27:0/438736006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670eb1b0 msgr2=0x564567498ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.255+0000 7f2550066740 1 --2- 172.21.15.27:0/438736006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670eb1b0 0x564567498ab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f252c009de0 tx=0x7f252c012000 comp rx=0 tx=0).stop 2024-09-21T00:32:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.256+0000 7f2550066740 1 -- 172.21.15.27:0/438736006 shutdown_connections 2024-09-21T00:32:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.256+0000 7f2550066740 1 --2- 172.21.15.27:0/438736006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670eb1b0 0x564567498ab0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.256+0000 7f2550066740 1 --2- 172.21.15.27:0/438736006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670ca4d0 0x564567498570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.256+0000 7f2550066740 1 --2- 172.21.15.27:0/438736006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670c22f0 0x56456749cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.256+0000 7f2550066740 1 -- 172.21.15.27:0/438736006 >> 172.21.15.27:0/438736006 conn(0x5645670df300 msgr2=0x56456749c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.257+0000 7f2550066740 1 -- 172.21.15.27:0/438736006 shutdown_connections 2024-09-21T00:32:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.257+0000 7f2550066740 1 -- 172.21.15.27:0/438736006 wait complete. 2024-09-21T00:32:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.258+0000 7f2550066740 1 -- start start 2024-09-21T00:32:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.258+0000 7f2550066740 1 Processor -- start 2024-09-21T00:32:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.261+0000 7f2550066740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670c22f0 0x5645674a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.261+0000 7f2550066740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670ca4d0 0x5645674a4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.261+0000 7f2550066740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670eb1b0 0x5645675105a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.261+0000 7f2550066740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f25441bf7a0 con 0x5645670eb1b0 2024-09-21T00:32:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.261+0000 7f2550066740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2544104b50 con 0x5645670c22f0 2024-09-21T00:32:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.261+0000 7f2550066740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f25441bea00 con 0x5645670ca4d0 2024-09-21T00:32:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.261+0000 7f2550066740 10 client.0 _ll_register_callbacks cb 0x7f253c010720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:32:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.261+0000 7f2541e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670c22f0 0x5645674a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.261+0000 7f2541e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670c22f0 0x5645674a3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:56986/0 (socket says 172.21.15.27:56986) 2024-09-21T00:32:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.261+0000 7f2541e00640 1 -- 172.21.15.27:0/2945798096 learned_addr learned my addr 172.21.15.27:0/2945798096 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.261+0000 7f2541400640 1 --2- 172.21.15.27:0/2945798096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670ca4d0 0x5645674a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.261+0000 7f2542800640 1 --2- 172.21.15.27:0/2945798096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670eb1b0 0x5645675105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.262+0000 7f2541e00640 1 -- 172.21.15.27:0/2945798096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670ca4d0 msgr2=0x5645674a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.262+0000 7f2541e00640 1 --2- 172.21.15.27:0/2945798096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670ca4d0 0x5645674a4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.262+0000 7f2541e00640 1 -- 172.21.15.27:0/2945798096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670eb1b0 msgr2=0x5645675105a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.262+0000 7f2541e00640 1 --2- 172.21.15.27:0/2945798096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670eb1b0 0x5645675105a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.262+0000 7f2541e00640 1 -- 172.21.15.27:0/2945798096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564567510d30 con 0x5645670c22f0 2024-09-21T00:32:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.262+0000 7f2541400640 1 --2- 172.21.15.27:0/2945798096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670ca4d0 0x5645674a4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:32:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.263+0000 7f2541e00640 1 --2- 172.21.15.27:0/2945798096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670c22f0 0x5645674a3a90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f253402aeb0 tx=0x7f2534031a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b689f8535e02d54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.263+0000 7f2527e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:32:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.263+0000 7f2527e00640 1 -- 172.21.15.27:0/2945798096 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534052060 con 0x5645670c22f0 2024-09-21T00:32:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.263+0000 7f2527e00640 1 -- 172.21.15.27:0/2945798096 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9591+0+0 (secure 0 0 0) 0x7f2534031680 con 0x5645670c22f0 2024-09-21T00:32:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.263+0000 7f2550066740 1 -- 172.21.15.27:0/2945798096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f252c01bcf0 con 0x5645670c22f0 2024-09-21T00:32:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.264+0000 7f2527e00640 1 -- 172.21.15.27:0/2945798096 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2534031880 con 0x5645670c22f0 2024-09-21T00:32:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.264+0000 7f2550066740 10 client.26495 fetch_fsmap learned FSMap version 43 2024-09-21T00:32:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.264+0000 7f2550066740 1 -- 172.21.15.27:0/2945798096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5645674a3270 con 0x5645670c22f0 2024-09-21T00:32:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.264+0000 7f2527e00640 1 -- 172.21.15.27:0/2945798096 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2534036040 con 0x5645670c22f0 2024-09-21T00:32:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.265+0000 7f2527e00640 1 -- 172.21.15.27:0/2945798096 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f2534004650 con 0x5645670c22f0 2024-09-21T00:32:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.266+0000 7f2550066740 10 client.26495 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T00:32:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.266+0000 7f2550066740 10 client.26495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:32:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.266+0000 7f2550066740 20 client.26495 populate_metadata read hostname 'smithi027' 2024-09-21T00:32:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.266+0000 7f2550066740 1 --2- 172.21.15.27:0/2945798096 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56456707f1c0 0x56456752afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.266+0000 7f2550066740 4 client.26495 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:32:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.266+0000 7f2550066740 1 -- 172.21.15.27:0/2945798096 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f252c01bcf0 con 0x56456707f1c0 2024-09-21T00:32:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.267+0000 7f2541400640 1 --2- 172.21.15.27:0/2945798096 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56456707f1c0 0x56456752afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.268+0000 7f2541400640 1 --2- 172.21.15.27:0/2945798096 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56456707f1c0 0x56456752afc0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b745702402779f96 server_cookie=ea5e1cc2e7efdbe8 in_seq=0 out_seq=0 2024-09-21T00:32:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.268+0000 7f2527e00640 10 client.26495 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.359+0000 7f2527e00640 1 -- 172.21.15.27:0/2945798096 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f252c01bcf0 con 0x56456707f1c0 2024-09-21T00:32:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.359+0000 7f2527e00640 10 client.26495 handle_command_reply: tid=0 2024-09-21T00:32:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.359+0000 7f2550066740 1 client.26495 shutdown 2024-09-21T00:32:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.360+0000 7f2550066740 1 -- 172.21.15.27:0/2945798096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670c22f0 msgr2=0x5645674a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.360+0000 7f2550066740 1 --2- 172.21.15.27:0/2945798096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670c22f0 0x5645674a3a90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f253402aeb0 tx=0x7f2534031a90 comp rx=0 tx=0).stop 2024-09-21T00:32:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.360+0000 7f2550066740 1 -- 172.21.15.27:0/2945798096 shutdown_connections 2024-09-21T00:32:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.360+0000 7f2550066740 1 --2- 172.21.15.27:0/2945798096 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56456707f1c0 0x56456752afc0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.360+0000 7f2550066740 1 --2- 172.21.15.27:0/2945798096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5645670eb1b0 0x5645675105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.360+0000 7f2550066740 1 --2- 172.21.15.27:0/2945798096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645670ca4d0 0x5645674a4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.360+0000 7f2550066740 1 --2- 172.21.15.27:0/2945798096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645670c22f0 0x5645674a3a90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.360+0000 7f2550066740 1 -- 172.21.15.27:0/2945798096 >> 172.21.15.27:0/2945798096 conn(0x5645670df300 msgr2=0x56456749c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.360+0000 7f2527e00640 0 client.26495 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.360+0000 7f2550066740 1 -- 172.21.15.27:0/2945798096 shutdown_connections 2024-09-21T00:32:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.361+0000 7f2550066740 1 -- 172.21.15.27:0/2945798096 wait complete. 2024-09-21T00:32:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.361+0000 7f2550066740 20 client.26495 trim_cache size 0 max 16384 2024-09-21T00:32:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:32:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:32:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:32:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:32:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.362+0000 7f2543200640 1 -- 172.21.15.27:0/755818988 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f25280b66f0 msgr2=0x7f25280b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.362+0000 7f2543200640 1 --2- 172.21.15.27:0/755818988 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f25280b66f0 0x7f25280b8c00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f252c021000 tx=0x7f252c01e000 comp rx=0 tx=0).stop 2024-09-21T00:32:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.362+0000 7f2543200640 1 -- 172.21.15.27:0/755818988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2544105090 msgr2=0x7f2544102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.362+0000 7f2543200640 1 --2- 172.21.15.27:0/755818988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2544105090 0x7f2544102470 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f253c006fd0 tx=0x7f253c00b8e0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.362+0000 7f2543200640 1 -- 172.21.15.27:0/755818988 shutdown_connections 2024-09-21T00:32:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.362+0000 7f2543200640 1 --2- 172.21.15.27:0/755818988 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f25280b66f0 0x7f25280b8c00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.362+0000 7f2543200640 1 --2- 172.21.15.27:0/755818988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f254410dba0 0x7f25441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.362+0000 7f2543200640 1 --2- 172.21.15.27:0/755818988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2544105090 0x7f2544102470 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.362+0000 7f2543200640 1 --2- 172.21.15.27:0/755818988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2544104740 0x7f2544103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.362+0000 7f2543200640 1 -- 172.21.15.27:0/755818988 >> 172.21.15.27:0/755818988 conn(0x7f25440fe210 msgr2=0x7f2544107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.362+0000 7f2543200640 1 -- 172.21.15.27:0/755818988 shutdown_connections 2024-09-21T00:32:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:20.362+0000 7f2543200640 1 -- 172.21.15.27:0/755818988 wait complete. 2024-09-21T00:32:20.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:20 smithi089 ceph-mon[34481]: pgmap v718: 129 pgs: 11 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 113 active+clean; 1011 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 494 op/s 2024-09-21T00:32:20.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:20 smithi098 ceph-mon[34725]: pgmap v718: 129 pgs: 11 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 113 active+clean; 1011 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 494 op/s 2024-09-21T00:32:20.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:20 smithi027 ceph-mon[30745]: pgmap v718: 129 pgs: 11 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 113 active+clean; 1011 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 494 op/s 2024-09-21T00:32:22.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:22 smithi027 ceph-mon[30745]: pgmap v719: 129 pgs: 11 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 114 active+clean; 1013 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.1 MiB/s wr, 193 op/s 2024-09-21T00:32:22.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:22 smithi027 ceph-mon[30745]: osdmap e104: 12 total, 12 up, 12 in 2024-09-21T00:32:22.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:22 smithi089 ceph-mon[34481]: pgmap v719: 129 pgs: 11 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 114 active+clean; 1013 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.1 MiB/s wr, 193 op/s 2024-09-21T00:32:22.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:22 smithi089 ceph-mon[34481]: osdmap e104: 12 total, 12 up, 12 in 2024-09-21T00:32:22.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:22 smithi098 ceph-mon[34725]: pgmap v719: 129 pgs: 11 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 114 active+clean; 1013 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.1 MiB/s wr, 193 op/s 2024-09-21T00:32:22.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:22 smithi098 ceph-mon[34725]: osdmap e104: 12 total, 12 up, 12 in 2024-09-21T00:32:24.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:24 smithi027 ceph-mon[30745]: pgmap v721: 129 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 122 active+clean; 1017 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2024-09-21T00:32:24.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:24 smithi089 ceph-mon[34481]: pgmap v721: 129 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 122 active+clean; 1017 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2024-09-21T00:32:24.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:24 smithi098 ceph-mon[34725]: pgmap v721: 129 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 122 active+clean; 1017 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2024-09-21T00:32:26.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:26 smithi089 ceph-mon[34481]: pgmap v722: 129 pgs: 129 active+clean; 1022 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 271 op/s 2024-09-21T00:32:26.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:26 smithi098 ceph-mon[34725]: pgmap v722: 129 pgs: 129 active+clean; 1022 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 271 op/s 2024-09-21T00:32:26.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:26 smithi027 ceph-mon[30745]: pgmap v722: 129 pgs: 129 active+clean; 1022 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 271 op/s 2024-09-21T00:32:27.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:27 smithi089 ceph-mon[34481]: osdmap e105: 12 total, 12 up, 12 in 2024-09-21T00:32:27.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:27 smithi098 ceph-mon[34725]: osdmap e105: 12 total, 12 up, 12 in 2024-09-21T00:32:27.819 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:27 smithi027 ceph-mon[30745]: osdmap e105: 12 total, 12 up, 12 in 2024-09-21T00:32:28.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:28 smithi089 ceph-mon[34481]: pgmap v724: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 296 op/s 2024-09-21T00:32:28.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:28 smithi098 ceph-mon[34725]: pgmap v724: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 296 op/s 2024-09-21T00:32:28.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:28 smithi027 ceph-mon[30745]: pgmap v724: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 296 op/s 2024-09-21T00:32:30.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:30 smithi089 ceph-mon[34481]: pgmap v725: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 366 op/s 2024-09-21T00:32:30.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:30 smithi098 ceph-mon[34725]: pgmap v725: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 366 op/s 2024-09-21T00:32:30.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:30 smithi027 ceph-mon[30745]: pgmap v725: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 366 op/s 2024-09-21T00:32:32.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:32 smithi089 ceph-mon[34481]: pgmap v726: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 354 op/s 2024-09-21T00:32:32.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:32 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:32:32.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:32 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:32:32.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:32 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:32:32.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:32 smithi098 ceph-mon[34725]: pgmap v726: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 354 op/s 2024-09-21T00:32:32.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:32 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:32:32.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:32 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:32:32.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:32 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:32:32.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:32 smithi027 ceph-mon[30745]: pgmap v726: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 354 op/s 2024-09-21T00:32:32.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:32 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:32:32.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:32 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:32:32.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:32 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:32:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.689+0000 7fee55000640 1 -- 172.21.15.27:0/2238476067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee50103ad0 msgr2=0x7fee50103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.689+0000 7fee55000640 1 --2- 172.21.15.27:0/2238476067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee50103ad0 0x7fee50103ee0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fee40009900 tx=0x7fee4002a410 comp rx=0 tx=0).stop 2024-09-21T00:32:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.689+0000 7fee55000640 1 -- 172.21.15.27:0/2238476067 shutdown_connections 2024-09-21T00:32:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.689+0000 7fee55000640 1 --2- 172.21.15.27:0/2238476067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee50109d50 0x7fee50110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.689+0000 7fee55000640 1 --2- 172.21.15.27:0/2238476067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee50109400 0x7fee50109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.689+0000 7fee55000640 1 --2- 172.21.15.27:0/2238476067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee50103ad0 0x7fee50103ee0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.689+0000 7fee55000640 1 -- 172.21.15.27:0/2238476067 >> 172.21.15.27:0/2238476067 conn(0x7fee500ff4d0 msgr2=0x7fee50101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.689+0000 7fee55000640 1 -- 172.21.15.27:0/2238476067 shutdown_connections 2024-09-21T00:32:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.689+0000 7fee55000640 1 -- 172.21.15.27:0/2238476067 wait complete. 2024-09-21T00:32:33.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.690+0000 7fee55000640 1 Processor -- start 2024-09-21T00:32:33.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.691+0000 7fee55000640 1 -- start start 2024-09-21T00:32:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.691+0000 7fee55000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee50103ad0 0x7fee5010da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.691+0000 7fee55000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee50109400 0x7fee5010df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.691+0000 7fee55000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee50109d50 0x7fee5010e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.691+0000 7fee55000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fee50077040 con 0x7fee50109d50 2024-09-21T00:32:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.691+0000 7fee55000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fee50076ec0 con 0x7fee50103ad0 2024-09-21T00:32:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.691+0000 7fee55000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fee500771c0 con 0x7fee50109400 2024-09-21T00:32:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.691+0000 7fee4de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee50103ad0 0x7fee5010da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.691+0000 7fee4de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee50103ad0 0x7fee5010da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45932/0 (socket says 172.21.15.27:45932) 2024-09-21T00:32:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.691+0000 7fee4de00640 1 -- 172.21.15.27:0/1935259536 learned_addr learned my addr 172.21.15.27:0/1935259536 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.692+0000 7fee4e800640 1 --2- 172.21.15.27:0/1935259536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee50109d50 0x7fee5010e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.692+0000 7fee47e00640 1 --2- 172.21.15.27:0/1935259536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee50109400 0x7fee5010df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.692+0000 7fee4de00640 1 -- 172.21.15.27:0/1935259536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee50109400 msgr2=0x7fee5010df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.692+0000 7fee4de00640 1 --2- 172.21.15.27:0/1935259536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee50109400 0x7fee5010df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.692+0000 7fee4de00640 1 -- 172.21.15.27:0/1935259536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee50109d50 msgr2=0x7fee5010e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.692+0000 7fee4de00640 1 --2- 172.21.15.27:0/1935259536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee50109d50 0x7fee5010e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.692+0000 7fee4de00640 1 -- 172.21.15.27:0/1935259536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee501acae0 con 0x7fee50103ad0 2024-09-21T00:32:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.692+0000 7fee4de00640 1 --2- 172.21.15.27:0/1935259536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee50103ad0 0x7fee5010da10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fee40005940 tx=0x7fee400042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.692+0000 7fee46a00640 1 -- 172.21.15.27:0/1935259536 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4003f030 con 0x7fee50103ad0 2024-09-21T00:32:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.692+0000 7fee46a00640 1 -- 172.21.15.27:0/1935259536 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee40003cb0 con 0x7fee50103ad0 2024-09-21T00:32:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.692+0000 7fee46a00640 1 -- 172.21.15.27:0/1935259536 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40035070 con 0x7fee50103ad0 2024-09-21T00:32:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.693+0000 7fee55000640 1 -- 172.21.15.27:0/1935259536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee501acd70 con 0x7fee50103ad0 2024-09-21T00:32:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.693+0000 7fee55000640 1 -- 172.21.15.27:0/1935259536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee501ad1b0 con 0x7fee50103ad0 2024-09-21T00:32:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.696+0000 7fee5c1bb740 1 Processor -- start 2024-09-21T00:32:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.696+0000 7fee5c1bb740 1 -- start start 2024-09-21T00:32:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.696+0000 7fee5c1bb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c344c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.696+0000 7fee5c1bb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf772f0 0x56386c345170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.696+0000 7fee5c1bb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf7f4d0 0x56386c3456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.696+0000 7fee5c1bb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56386c2ba6b0 con 0x56386bf772f0 2024-09-21T00:32:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.696+0000 7fee5c1bb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56386c2a9ae0 con 0x56386bfa01b0 2024-09-21T00:32:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.696+0000 7fee46a00640 1 -- 172.21.15.27:0/1935259536 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fee40042030 con 0x7fee50103ad0 2024-09-21T00:32:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.696+0000 7fee5c1bb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56386c2ad8f0 con 0x56386bf7f4d0 2024-09-21T00:32:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.697+0000 7fee4e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c344c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.697+0000 7fee4e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c344c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45936/0 (socket says 172.21.15.27:45936) 2024-09-21T00:32:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.697+0000 7fee4e800640 1 -- 172.21.15.27:0/3616948666 learned_addr learned my addr 172.21.15.27:0/3616948666 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.697+0000 7fee4de00640 1 --2- 172.21.15.27:0/3616948666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf772f0 0x56386c345170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.697+0000 7fee47e00640 1 --2- 172.21.15.27:0/3616948666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf7f4d0 0x56386c3456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.697+0000 7fee4e800640 1 -- 172.21.15.27:0/3616948666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf7f4d0 msgr2=0x56386c3456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.697+0000 7fee4e800640 1 --2- 172.21.15.27:0/3616948666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf7f4d0 0x56386c3456b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.697+0000 7fee4e800640 1 -- 172.21.15.27:0/3616948666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf772f0 msgr2=0x56386c345170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.697+0000 7fee4e800640 1 --2- 172.21.15.27:0/3616948666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf772f0 0x56386c345170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.697+0000 7fee4e800640 1 -- 172.21.15.27:0/3616948666 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56386c276780 con 0x56386bfa01b0 2024-09-21T00:32:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.698+0000 7fee4e800640 1 --2- 172.21.15.27:0/3616948666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c344c30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fee48009fa0 tx=0x7fee48010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9a32e5f09e9c640 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.698+0000 7fee3f400640 1 -- 172.21.15.27:0/3616948666 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48016070 con 0x56386bfa01b0 2024-09-21T00:32:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.698+0000 7fee3f400640 1 -- 172.21.15.27:0/3616948666 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee48004d20 con 0x56386bfa01b0 2024-09-21T00:32:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.698+0000 7fee3f400640 1 -- 172.21.15.27:0/3616948666 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48012870 con 0x56386bfa01b0 2024-09-21T00:32:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.699+0000 7fee46a00640 1 --2- 172.21.15.27:0/1935259536 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee340b6700 0x7fee340b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.699+0000 7fee5c1bb740 1 -- 172.21.15.27:0/3616948666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 msgr2=0x56386c344c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.699+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/3616948666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c344c30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fee48009fa0 tx=0x7fee48010800 comp rx=0 tx=0).stop 2024-09-21T00:32:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.699+0000 7fee5c1bb740 1 -- 172.21.15.27:0/3616948666 shutdown_connections 2024-09-21T00:32:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.699+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/3616948666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf7f4d0 0x56386c3456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.699+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/3616948666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf772f0 0x56386c345170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.699+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/3616948666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c344c30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.699+0000 7fee5c1bb740 1 -- 172.21.15.27:0/3616948666 >> 172.21.15.27:0/3616948666 conn(0x56386bf94300 msgr2=0x56386c342b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.699+0000 7fee4de00640 1 --2- 172.21.15.27:0/1935259536 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee340b6700 0x7fee340b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.699+0000 7fee46a00640 1 -- 172.21.15.27:0/1935259536 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9503+0+0 (secure 0 0 0) 0x7fee401082d0 con 0x7fee50103ad0 2024-09-21T00:32:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.699+0000 7fee5c1bb740 1 -- 172.21.15.27:0/3616948666 shutdown_connections 2024-09-21T00:32:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.699+0000 7fee5c1bb740 1 -- 172.21.15.27:0/3616948666 wait complete. 2024-09-21T00:32:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.700+0000 7fee4de00640 1 --2- 172.21.15.27:0/1935259536 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee340b6700 0x7fee340b8c10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fee4010bfd0 tx=0x7fee4010bd80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.700+0000 7fee5c1bb740 1 -- start start 2024-09-21T00:32:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.700+0000 7fee5c1bb740 1 Processor -- start 2024-09-21T00:32:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.700+0000 7fee5c1bb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf772f0 0x56386c350220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.700+0000 7fee5c1bb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf7f4d0 0x56386c350760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.700+0000 7fee5c1bb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c350ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.700+0000 7fee5c1bb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56386c1e1ab0 con 0x56386bf7f4d0 2024-09-21T00:32:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.700+0000 7fee5c1bb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56386c250570 con 0x56386bfa01b0 2024-09-21T00:32:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.700+0000 7fee5c1bb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56386c3643c0 con 0x56386bf772f0 2024-09-21T00:32:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.700+0000 7fee5c1bb740 10 client.0 _ll_register_callbacks cb 0x56386c25f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:32:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.701+0000 7fee4e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c350ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.701+0000 7fee47e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf7f4d0 0x56386c350760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.701+0000 7fee4e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c350ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45938/0 (socket says 172.21.15.27:45938) 2024-09-21T00:32:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.701+0000 7fee4e800640 1 -- 172.21.15.27:0/2342868264 learned_addr learned my addr 172.21.15.27:0/2342868264 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.701+0000 7fee4e800640 1 --2- 172.21.15.27:0/2342868264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf772f0 0x56386c350220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.701+0000 7fee47e00640 1 -- 172.21.15.27:0/2342868264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf772f0 msgr2=0x56386c350220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.701+0000 7fee47e00640 1 --2- 172.21.15.27:0/2342868264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf772f0 0x56386c350220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.701+0000 7fee47e00640 1 -- 172.21.15.27:0/2342868264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 msgr2=0x56386c350ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.701+0000 7fee47e00640 1 --2- 172.21.15.27:0/2342868264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c350ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.701+0000 7fee47e00640 1 -- 172.21.15.27:0/2342868264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56386c00b790 con 0x56386bf7f4d0 2024-09-21T00:32:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.701+0000 7fee4e800640 1 --2- 172.21.15.27:0/2342868264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c350ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:32:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.702+0000 7fee47e00640 1 --2- 172.21.15.27:0/2342868264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf7f4d0 0x56386c350760 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fee38005d00 tx=0x7fee380118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d838da6b98f82a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.702+0000 7fee33e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:32:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.702+0000 7fee33e00640 1 -- 172.21.15.27:0/2342868264 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee38004020 con 0x56386bf7f4d0 2024-09-21T00:32:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.702+0000 7fee33e00640 1 -- 172.21.15.27:0/2342868264 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9503+0+0 (secure 0 0 0) 0x7fee380041c0 con 0x56386bf7f4d0 2024-09-21T00:32:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.702+0000 7fee5c1bb740 1 -- 172.21.15.27:0/2342868264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee48004d20 con 0x56386bf7f4d0 2024-09-21T00:32:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.702+0000 7fee33e00640 1 -- 172.21.15.27:0/2342868264 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fee38024050 con 0x56386bf7f4d0 2024-09-21T00:32:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.702+0000 7fee5c1bb740 10 client.16515 fetch_fsmap learned FSMap version 43 2024-09-21T00:32:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.702+0000 7fee5c1bb740 1 -- 172.21.15.27:0/2342868264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56386c084620 con 0x56386bf7f4d0 2024-09-21T00:32:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.703+0000 7fee33e00640 1 -- 172.21.15.27:0/2342868264 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee48004d20 con 0x56386bf7f4d0 2024-09-21T00:32:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.703+0000 7fee33e00640 1 -- 172.21.15.27:0/2342868264 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7fee38003920 con 0x56386bf7f4d0 2024-09-21T00:32:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.703+0000 7fee5c1bb740 10 client.16515 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T00:32:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.703+0000 7fee5c1bb740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:32:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.703+0000 7fee5c1bb740 20 client.16515 populate_metadata read hostname 'smithi027' 2024-09-21T00:32:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.703+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/2342868264 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56386bf5ed80 0x56386c373450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.703+0000 7fee5c1bb740 4 client.16515 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:32:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.703+0000 7fee5c1bb740 1 -- 172.21.15.27:0/2342868264 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56386c2bc600 con 0x56386bf5ed80 2024-09-21T00:32:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.704+0000 7fee4e800640 1 --2- 172.21.15.27:0/2342868264 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56386bf5ed80 0x56386c373450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.704+0000 7fee4e800640 1 --2- 172.21.15.27:0/2342868264 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56386bf5ed80 0x56386c373450 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6549d0220d0c481 server_cookie=d987808e37f78450 in_seq=0 out_seq=0 2024-09-21T00:32:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.704+0000 7fee33e00640 10 client.16515 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.705+0000 7fee33e00640 1 -- 172.21.15.27:0/2342868264 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56386c2bc600 con 0x56386bf5ed80 2024-09-21T00:32:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.705+0000 7fee33e00640 10 client.16515 handle_command_reply: tid=0 2024-09-21T00:32:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.706+0000 7fee5c1bb740 1 client.16515 shutdown 2024-09-21T00:32:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.706+0000 7fee5c1bb740 1 -- 172.21.15.27:0/2342868264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf7f4d0 msgr2=0x56386c350760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.706+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/2342868264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf7f4d0 0x56386c350760 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fee38005d00 tx=0x7fee380118d0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.706+0000 7fee5c1bb740 1 -- 172.21.15.27:0/2342868264 shutdown_connections 2024-09-21T00:32:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.706+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/2342868264 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56386bf5ed80 0x56386c373450 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.706+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/2342868264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c350ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.706+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/2342868264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf7f4d0 0x56386c350760 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.706+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/2342868264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf772f0 0x56386c350220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.706+0000 7fee5c1bb740 1 -- 172.21.15.27:0/2342868264 >> 172.21.15.27:0/2342868264 conn(0x56386bf94300 msgr2=0x56386c345c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.706+0000 7fee33e00640 0 client.16515 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.706+0000 7fee5c1bb740 1 -- 172.21.15.27:0/2342868264 shutdown_connections 2024-09-21T00:32:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.706+0000 7fee5c1bb740 1 -- 172.21.15.27:0/2342868264 wait complete. 2024-09-21T00:32:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.706+0000 7fee5c1bb740 20 client.16515 trim_cache size 0 max 16384 2024-09-21T00:32:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.722+0000 7fee5c1bb740 1 Processor -- start 2024-09-21T00:32:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.722+0000 7fee5c1bb740 1 -- start start 2024-09-21T00:32:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.722+0000 7fee5c1bb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf772f0 0x56386c346620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.722+0000 7fee5c1bb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bfa01b0 0x56386c34f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.722+0000 7fee5c1bb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bf341c0 0x56386c378bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.722+0000 7fee5c1bb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fee501b1590 con 0x56386bf772f0 2024-09-21T00:32:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.722+0000 7fee5c1bb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fee501aee00 con 0x56386bf341c0 2024-09-21T00:32:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.722+0000 7fee5c1bb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fee5011e460 con 0x56386bfa01b0 2024-09-21T00:32:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.722+0000 7fee4e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf772f0 0x56386c346620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.723+0000 7fee4e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf772f0 0x56386c346620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56018/0 (socket says 172.21.15.27:56018) 2024-09-21T00:32:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.723+0000 7fee4e800640 1 -- 172.21.15.27:0/1375283312 learned_addr learned my addr 172.21.15.27:0/1375283312 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.723+0000 7fee4e800640 1 --2- 172.21.15.27:0/1375283312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bf341c0 0x56386c378bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.723+0000 7fee47e00640 1 --2- 172.21.15.27:0/1375283312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bfa01b0 0x56386c34f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.723+0000 7fee4e800640 1 -- 172.21.15.27:0/1375283312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bfa01b0 msgr2=0x56386c34f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.723+0000 7fee4e800640 1 --2- 172.21.15.27:0/1375283312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bfa01b0 0x56386c34f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.723+0000 7fee4e800640 1 -- 172.21.15.27:0/1375283312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bf341c0 msgr2=0x56386c378bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.723+0000 7fee4e800640 1 --2- 172.21.15.27:0/1375283312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bf341c0 0x56386c378bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.723+0000 7fee4e800640 1 -- 172.21.15.27:0/1375283312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56386c347440 con 0x56386bf772f0 2024-09-21T00:32:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.723+0000 7fee47e00640 1 --2- 172.21.15.27:0/1375283312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bfa01b0 0x56386c34f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:32:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.723+0000 7fee4e800640 1 --2- 172.21.15.27:0/1375283312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf772f0 0x56386c346620 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fee48000f80 tx=0x7fee48010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5809309c282390 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.723+0000 7fee3ea00640 1 -- 172.21.15.27:0/1375283312 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48016040 con 0x56386bf772f0 2024-09-21T00:32:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.724+0000 7fee3ea00640 1 -- 172.21.15.27:0/1375283312 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee48012c70 con 0x56386bf772f0 2024-09-21T00:32:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.724+0000 7fee3ea00640 1 -- 172.21.15.27:0/1375283312 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4802bc10 con 0x56386bf772f0 2024-09-21T00:32:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.724+0000 7fee5c1bb740 1 -- 172.21.15.27:0/1375283312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf772f0 msgr2=0x56386c346620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.724+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/1375283312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf772f0 0x56386c346620 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fee48000f80 tx=0x7fee48010800 comp rx=0 tx=0).stop 2024-09-21T00:32:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.724+0000 7fee5c1bb740 1 -- 172.21.15.27:0/1375283312 shutdown_connections 2024-09-21T00:32:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.724+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/1375283312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bf341c0 0x56386c378bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.724+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/1375283312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bfa01b0 0x56386c34f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.724+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/1375283312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf772f0 0x56386c346620 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.724+0000 7fee5c1bb740 1 -- 172.21.15.27:0/1375283312 >> 172.21.15.27:0/1375283312 conn(0x56386bf94300 msgr2=0x56386c34ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.725+0000 7fee5c1bb740 1 -- 172.21.15.27:0/1375283312 shutdown_connections 2024-09-21T00:32:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.725+0000 7fee5c1bb740 1 -- 172.21.15.27:0/1375283312 wait complete. 2024-09-21T00:32:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.725+0000 7fee5c1bb740 1 -- start start 2024-09-21T00:32:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.725+0000 7fee5c1bb740 1 Processor -- start 2024-09-21T00:32:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee5c1bb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf341c0 0x56386c3659b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee5c1bb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf772f0 0x56386c365ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee5c1bb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c3dcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee5c1bb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fee501ad3e0 con 0x56386bf341c0 2024-09-21T00:32:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee5c1bb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fee501ae730 con 0x56386bfa01b0 2024-09-21T00:32:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee5c1bb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fee501ad990 con 0x56386bf772f0 2024-09-21T00:32:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee5c1bb740 10 client.0 _ll_register_callbacks cb 0x7fee3800bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:32:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee4e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf341c0 0x56386c3659b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee4e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf341c0 0x56386c3659b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56030/0 (socket says 172.21.15.27:56030) 2024-09-21T00:32:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee4e800640 1 -- 172.21.15.27:0/1110590895 learned_addr learned my addr 172.21.15.27:0/1110590895 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee47e00640 1 --2- 172.21.15.27:0/1110590895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf772f0 0x56386c365ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee4e800640 1 --2- 172.21.15.27:0/1110590895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c3dcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee47e00640 1 -- 172.21.15.27:0/1110590895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 msgr2=0x56386c3dcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee47e00640 1 --2- 172.21.15.27:0/1110590895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c3dcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee47e00640 1 -- 172.21.15.27:0/1110590895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf341c0 msgr2=0x56386c3659b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee47e00640 1 --2- 172.21.15.27:0/1110590895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf341c0 0x56386c3659b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.726+0000 7fee47e00640 1 -- 172.21.15.27:0/1110590895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56386c3dd330 con 0x56386bf772f0 2024-09-21T00:32:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.727+0000 7fee47e00640 1 --2- 172.21.15.27:0/1110590895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf772f0 0x56386c365ef0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fee38003d90 tx=0x7fee3800b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20475a2457c11ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.727+0000 7fee33e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:32:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.727+0000 7fee33e00640 1 -- 172.21.15.27:0/1110590895 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee38004210 con 0x56386bf772f0 2024-09-21T00:32:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.727+0000 7fee33e00640 1 -- 172.21.15.27:0/1110590895 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9503+0+0 (secure 0 0 0) 0x7fee380043b0 con 0x56386bf772f0 2024-09-21T00:32:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.727+0000 7fee5c1bb740 1 -- 172.21.15.27:0/1110590895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee48012c70 con 0x56386bf772f0 2024-09-21T00:32:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.727+0000 7fee33e00640 1 -- 172.21.15.27:0/1110590895 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fee38039050 con 0x56386bf772f0 2024-09-21T00:32:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.727+0000 7fee5c1bb740 10 client.26527 fetch_fsmap learned FSMap version 43 2024-09-21T00:32:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.727+0000 7fee5c1bb740 1 -- 172.21.15.27:0/1110590895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56386c3de430 con 0x56386bf772f0 2024-09-21T00:32:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.728+0000 7fee33e00640 1 -- 172.21.15.27:0/1110590895 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee48012c70 con 0x56386bf772f0 2024-09-21T00:32:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.728+0000 7fee33e00640 1 -- 172.21.15.27:0/1110590895 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7fee380107c0 con 0x56386bf772f0 2024-09-21T00:32:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.728+0000 7fee5c1bb740 10 client.26527 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T00:32:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.728+0000 7fee5c1bb740 10 client.26527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:32:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.728+0000 7fee5c1bb740 20 client.26527 populate_metadata read hostname 'smithi027' 2024-09-21T00:32:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.728+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/1110590895 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56386c3deb20 0x56386c3e0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.728+0000 7fee5c1bb740 4 client.26527 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:32:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.728+0000 7fee5c1bb740 1 -- 172.21.15.27:0/1110590895 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56386c2b80b0 con 0x56386c3deb20 2024-09-21T00:32:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.728+0000 7fee4e800640 1 --2- 172.21.15.27:0/1110590895 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56386c3deb20 0x56386c3e0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.729+0000 7fee4e800640 1 --2- 172.21.15.27:0/1110590895 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56386c3deb20 0x56386c3e0f40 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7be80fd99132ea5c server_cookie=c0d5cef526bb9bf in_seq=0 out_seq=0 2024-09-21T00:32:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:33.729+0000 7fee33e00640 10 client.26527 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:34.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:34 smithi027 ceph-mon[30745]: pgmap v727: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 299 op/s 2024-09-21T00:32:34.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:34 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:32:34.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:34 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:32:34.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:34 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:32:34.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:34 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:32:34.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:34 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:32:34.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:34 smithi089 ceph-mon[34481]: pgmap v727: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 299 op/s 2024-09-21T00:32:34.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:34 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:32:34.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:34 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:32:34.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:34 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:32:34.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:34 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:32:34.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:34 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:32:34.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:34 smithi098 ceph-mon[34725]: pgmap v727: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 299 op/s 2024-09-21T00:32:34.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:34 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:32:34.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:34 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:32:34.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:34 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:32:34.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:34 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:32:34.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:34 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:32:36.701 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:36 smithi027 ceph-mon[30745]: pgmap v728: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 332 op/s 2024-09-21T00:32:36.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:36 smithi089 ceph-mon[34481]: pgmap v728: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 332 op/s 2024-09-21T00:32:36.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:36 smithi098 ceph-mon[34725]: pgmap v728: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 332 op/s 2024-09-21T00:32:38.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:38 smithi089 ceph-mon[34481]: pgmap v729: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 545 op/s 2024-09-21T00:32:38.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:38 smithi098 ceph-mon[34725]: pgmap v729: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 545 op/s 2024-09-21T00:32:38.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:38 smithi027 ceph-mon[30745]: pgmap v729: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 545 op/s 2024-09-21T00:32:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.245+0000 7fee33e00640 1 -- 172.21.15.27:0/1110590895 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56386c2b80b0 con 0x56386c3deb20 2024-09-21T00:32:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.245+0000 7fee33e00640 10 client.26527 handle_command_reply: tid=0 2024-09-21T00:32:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.246+0000 7fee5c1bb740 1 client.26527 shutdown 2024-09-21T00:32:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.246+0000 7fee5c1bb740 1 -- 172.21.15.27:0/1110590895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf772f0 msgr2=0x56386c365ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.246+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/1110590895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf772f0 0x56386c365ef0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fee38003d90 tx=0x7fee3800b480 comp rx=0 tx=0).stop 2024-09-21T00:32:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.247+0000 7fee5c1bb740 1 -- 172.21.15.27:0/1110590895 shutdown_connections 2024-09-21T00:32:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.247+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/1110590895 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x56386c3deb20 0x56386c3e0f40 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.247+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/1110590895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56386bfa01b0 0x56386c3dcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.247+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/1110590895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56386bf772f0 0x56386c365ef0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.247+0000 7fee5c1bb740 1 --2- 172.21.15.27:0/1110590895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56386bf341c0 0x56386c3659b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.247+0000 7fee5c1bb740 1 -- 172.21.15.27:0/1110590895 >> 172.21.15.27:0/1110590895 conn(0x56386bf94300 msgr2=0x56386c3463f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.247+0000 7fee33e00640 0 client.26527 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.248+0000 7fee5c1bb740 1 -- 172.21.15.27:0/1110590895 shutdown_connections 2024-09-21T00:32:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.248+0000 7fee5c1bb740 1 -- 172.21.15.27:0/1110590895 wait complete. 2024-09-21T00:32:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.248+0000 7fee5c1bb740 20 client.26527 trim_cache size 0 max 16384 2024-09-21T00:32:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:32:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:32:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:32:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:32:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.250+0000 7fee55000640 1 -- 172.21.15.27:0/1935259536 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee340b6700 msgr2=0x7fee340b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.250+0000 7fee55000640 1 --2- 172.21.15.27:0/1935259536 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee340b6700 0x7fee340b8c10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fee4010bfd0 tx=0x7fee4010bd80 comp rx=0 tx=0).stop 2024-09-21T00:32:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.250+0000 7fee55000640 1 -- 172.21.15.27:0/1935259536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee50103ad0 msgr2=0x7fee5010da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.250+0000 7fee55000640 1 --2- 172.21.15.27:0/1935259536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee50103ad0 0x7fee5010da10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fee40005940 tx=0x7fee400042f0 comp rx=0 tx=0).stop 2024-09-21T00:32:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.250+0000 7fee55000640 1 -- 172.21.15.27:0/1935259536 shutdown_connections 2024-09-21T00:32:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.250+0000 7fee55000640 1 --2- 172.21.15.27:0/1935259536 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fee340b6700 0x7fee340b8c10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.250+0000 7fee55000640 1 --2- 172.21.15.27:0/1935259536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee50109d50 0x7fee5010e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.250+0000 7fee55000640 1 --2- 172.21.15.27:0/1935259536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee50109400 0x7fee5010df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.250+0000 7fee55000640 1 --2- 172.21.15.27:0/1935259536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee50103ad0 0x7fee5010da10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.250+0000 7fee55000640 1 -- 172.21.15.27:0/1935259536 >> 172.21.15.27:0/1935259536 conn(0x7fee500ff4d0 msgr2=0x7fee50108b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.250+0000 7fee55000640 1 -- 172.21.15.27:0/1935259536 shutdown_connections 2024-09-21T00:32:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:40.250+0000 7fee55000640 1 -- 172.21.15.27:0/1935259536 wait complete. 2024-09-21T00:32:40.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:40 smithi089 ceph-mon[34481]: pgmap v730: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 984 op/s 2024-09-21T00:32:40.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:40 smithi098 ceph-mon[34725]: pgmap v730: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 984 op/s 2024-09-21T00:32:40.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:40 smithi027 ceph-mon[30745]: pgmap v730: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 984 op/s 2024-09-21T00:32:42.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:42 smithi089 ceph-mon[34481]: pgmap v731: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 1.41k op/s 2024-09-21T00:32:42.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:42 smithi098 ceph-mon[34725]: pgmap v731: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 1.41k op/s 2024-09-21T00:32:42.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:42 smithi027 ceph-mon[30745]: pgmap v731: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 1.41k op/s 2024-09-21T00:32:44.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:44 smithi089 ceph-mon[34481]: pgmap v732: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 1.61k op/s 2024-09-21T00:32:44.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:44 smithi098 ceph-mon[34725]: pgmap v732: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 1.61k op/s 2024-09-21T00:32:44.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:44 smithi027 ceph-mon[30745]: pgmap v732: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 1.61k op/s 2024-09-21T00:32:46.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:46 smithi089 ceph-mon[34481]: pgmap v733: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 1.84k op/s 2024-09-21T00:32:46.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:46 smithi098 ceph-mon[34725]: pgmap v733: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 1.84k op/s 2024-09-21T00:32:46.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:46 smithi027 ceph-mon[30745]: pgmap v733: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 1.84k op/s 2024-09-21T00:32:48.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:48 smithi089 ceph-mon[34481]: pgmap v734: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 1.5 MiB/s wr, 1.79k op/s 2024-09-21T00:32:48.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:48 smithi098 ceph-mon[34725]: pgmap v734: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 1.5 MiB/s wr, 1.79k op/s 2024-09-21T00:32:48.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:48 smithi027 ceph-mon[30745]: pgmap v734: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 1.5 MiB/s wr, 1.79k op/s 2024-09-21T00:32:50.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:50 smithi089 ceph-mon[34481]: pgmap v735: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 1.68k op/s 2024-09-21T00:32:50.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:50 smithi098 ceph-mon[34725]: pgmap v735: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 1.68k op/s 2024-09-21T00:32:50.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:50 smithi027 ceph-mon[30745]: pgmap v735: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 1.68k op/s 2024-09-21T00:32:52.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:52 smithi089 ceph-mon[34481]: pgmap v736: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 1.23k op/s 2024-09-21T00:32:52.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:52 smithi098 ceph-mon[34725]: pgmap v736: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 1.23k op/s 2024-09-21T00:32:52.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:52 smithi027 ceph-mon[30745]: pgmap v736: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 1.23k op/s 2024-09-21T00:32:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.462+0000 7f8afe800640 1 -- 172.21.15.27:0/2372876656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af8075dd0 msgr2=0x7f8af80761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.462+0000 7f8afe800640 1 --2- 172.21.15.27:0/2372876656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af8075dd0 0x7f8af80761e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009900 tx=0x7f8ae402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.462+0000 7f8afe800640 1 -- 172.21.15.27:0/2372876656 shutdown_connections 2024-09-21T00:32:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.462+0000 7f8afe800640 1 --2- 172.21.15.27:0/2372876656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8af8114390 0x7f8af8116860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.462+0000 7f8afe800640 1 --2- 172.21.15.27:0/2372876656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af8075dd0 0x7f8af80761e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.462+0000 7f8afe800640 1 --2- 172.21.15.27:0/2372876656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8af8077390 0x7f8af80757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.462+0000 7f8afe800640 1 -- 172.21.15.27:0/2372876656 >> 172.21.15.27:0/2372876656 conn(0x7f8af80ff4d0 msgr2=0x7f8af8101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.462+0000 7f8afe800640 1 -- 172.21.15.27:0/2372876656 shutdown_connections 2024-09-21T00:32:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.462+0000 7f8afe800640 1 -- 172.21.15.27:0/2372876656 wait complete. 2024-09-21T00:32:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.463+0000 7f8afe800640 1 Processor -- start 2024-09-21T00:32:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.463+0000 7f8afe800640 1 -- start start 2024-09-21T00:32:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.463+0000 7f8afe800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af8075dd0 0x7f8af81a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.463+0000 7f8afe800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8af8077390 0x7f8af81a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.463+0000 7f8afe800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8af8114390 0x7f8af81a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.463+0000 7f8afe800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8af8119ff0 con 0x7f8af8077390 2024-09-21T00:32:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.463+0000 7f8afe800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8af8119e70 con 0x7f8af8075dd0 2024-09-21T00:32:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.463+0000 7f8afe800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8af811a170 con 0x7f8af8114390 2024-09-21T00:32:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.464+0000 7f8af7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af8075dd0 0x7f8af81a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.464+0000 7f8af7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af8075dd0 0x7f8af81a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:33592/0 (socket says 172.21.15.27:33592) 2024-09-21T00:32:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.464+0000 7f8af7400640 1 -- 172.21.15.27:0/3500183177 learned_addr learned my addr 172.21.15.27:0/3500183177 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.464+0000 7f8af6a00640 1 --2- 172.21.15.27:0/3500183177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8af8077390 0x7f8af81a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.464+0000 7f8af7e00640 1 --2- 172.21.15.27:0/3500183177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8af8114390 0x7f8af81a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.464+0000 7f8af7400640 1 -- 172.21.15.27:0/3500183177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8af8114390 msgr2=0x7f8af81a6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.464+0000 7f8af7400640 1 --2- 172.21.15.27:0/3500183177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8af8114390 0x7f8af81a6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.464+0000 7f8af7400640 1 -- 172.21.15.27:0/3500183177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8af8077390 msgr2=0x7f8af81a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.464+0000 7f8af7400640 1 --2- 172.21.15.27:0/3500183177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8af8077390 0x7f8af81a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.464+0000 7f8af7400640 1 -- 172.21.15.27:0/3500183177 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af81a6980 con 0x7f8af8075dd0 2024-09-21T00:32:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.465+0000 7f8af7400640 1 --2- 172.21.15.27:0/3500183177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af8075dd0 0x7f8af81a0a50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800e9f0 tx=0x7f8ae800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.465+0000 7f8aefe00640 1 -- 172.21.15.27:0/3500183177 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae800cd80 con 0x7f8af8075dd0 2024-09-21T00:32:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.465+0000 7f8aefe00640 1 -- 172.21.15.27:0/3500183177 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ae80108e0 con 0x7f8af8075dd0 2024-09-21T00:32:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.465+0000 7f8aefe00640 1 -- 172.21.15.27:0/3500183177 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8018980 con 0x7f8af8075dd0 2024-09-21T00:32:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.465+0000 7f8afe800640 1 -- 172.21.15.27:0/3500183177 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af81a6c70 con 0x7f8af8075dd0 2024-09-21T00:32:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.465+0000 7f8afe800640 1 -- 172.21.15.27:0/3500183177 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af81ad400 con 0x7f8af8075dd0 2024-09-21T00:32:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.468+0000 7f8b057ed740 1 Processor -- start 2024-09-21T00:32:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.468+0000 7f8b057ed740 1 -- start start 2024-09-21T00:32:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8b057ed740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabc271b0 0x55daabfcbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8b057ed740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabbfe2f0 0x55daabfcc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8b057ed740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc064d0 0x55daabfcc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8b057ed740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55daabf416b0 con 0x55daabbfe2f0 2024-09-21T00:32:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8b057ed740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55daabf30ae0 con 0x55daabc271b0 2024-09-21T00:32:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8b057ed740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55daabf348f0 con 0x55daabc064d0 2024-09-21T00:32:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8af7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabc271b0 0x55daabfcbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8af7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabc271b0 0x55daabfcbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:33600/0 (socket says 172.21.15.27:33600) 2024-09-21T00:32:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8af7e00640 1 -- 172.21.15.27:0/113407888 learned_addr learned my addr 172.21.15.27:0/113407888 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8aefe00640 1 -- 172.21.15.27:0/3500183177 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8ae8010430 con 0x7f8af8075dd0 2024-09-21T00:32:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8af6a00640 1 --2- 172.21.15.27:0/113407888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc064d0 0x55daabfcc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8af7e00640 1 -- 172.21.15.27:0/113407888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc064d0 msgr2=0x55daabfcc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8af7e00640 1 --2- 172.21.15.27:0/113407888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc064d0 0x55daabfcc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8af7e00640 1 -- 172.21.15.27:0/113407888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabbfe2f0 msgr2=0x55daabfcc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8af7e00640 1 --2- 172.21.15.27:0/113407888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabbfe2f0 0x55daabfcc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.469+0000 7f8af7e00640 1 -- 172.21.15.27:0/113407888 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daabefd780 con 0x55daabc271b0 2024-09-21T00:32:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.470+0000 7f8af7e00640 1 --2- 172.21.15.27:0/113407888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabc271b0 0x55daabfcbc30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009f80 tx=0x7f8af00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=435113b8728826d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.470+0000 7f8aecc00640 1 -- 172.21.15.27:0/113407888 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0004d20 con 0x55daabc271b0 2024-09-21T00:32:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.470+0000 7f8aecc00640 1 -- 172.21.15.27:0/113407888 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8af0004050 con 0x55daabc271b0 2024-09-21T00:32:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.470+0000 7f8aecc00640 1 -- 172.21.15.27:0/113407888 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0005220 con 0x55daabc271b0 2024-09-21T00:32:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.471+0000 7f8aefe00640 1 --2- 172.21.15.27:0/3500183177 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8adc0b6700 0x7f8adc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.471+0000 7f8b057ed740 1 -- 172.21.15.27:0/113407888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabc271b0 msgr2=0x55daabfcbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.471+0000 7f8b057ed740 1 --2- 172.21.15.27:0/113407888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabc271b0 0x55daabfcbc30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009f80 tx=0x7f8af00118d0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.471+0000 7f8aefe00640 1 -- 172.21.15.27:0/3500183177 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9503+0+0 (secure 0 0 0) 0x7f8ae80e7e00 con 0x7f8af8075dd0 2024-09-21T00:32:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.471+0000 7f8b057ed740 1 -- 172.21.15.27:0/113407888 shutdown_connections 2024-09-21T00:32:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.471+0000 7f8b057ed740 1 --2- 172.21.15.27:0/113407888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc064d0 0x55daabfcc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.471+0000 7f8b057ed740 1 --2- 172.21.15.27:0/113407888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabbfe2f0 0x55daabfcc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.471+0000 7f8b057ed740 1 --2- 172.21.15.27:0/113407888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabc271b0 0x55daabfcbc30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.471+0000 7f8af7400640 1 --2- 172.21.15.27:0/3500183177 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8adc0b6700 0x7f8adc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.471+0000 7f8b057ed740 1 -- 172.21.15.27:0/113407888 >> 172.21.15.27:0/113407888 conn(0x55daabc1b300 msgr2=0x55daabfc9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.471+0000 7f8b057ed740 1 -- 172.21.15.27:0/113407888 shutdown_connections 2024-09-21T00:32:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.471+0000 7f8b057ed740 1 -- 172.21.15.27:0/113407888 wait complete. 2024-09-21T00:32:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.472+0000 7f8af7400640 1 --2- 172.21.15.27:0/3500183177 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8adc0b6700 0x7f8adc0b8c10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55daabfca8b0 tx=0x7f8ae801f000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.472+0000 7f8b057ed740 1 -- start start 2024-09-21T00:32:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.472+0000 7f8b057ed740 1 Processor -- start 2024-09-21T00:32:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.473+0000 7f8b057ed740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabbfe2f0 0x55daabfd7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.473+0000 7f8b057ed740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabc064d0 0x55daabfd7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.473+0000 7f8b057ed740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc271b0 0x55daabfd7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.473+0000 7f8b057ed740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55daabe68ab0 con 0x55daabbfe2f0 2024-09-21T00:32:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.473+0000 7f8b057ed740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55daabed7570 con 0x55daabc064d0 2024-09-21T00:32:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.473+0000 7f8b057ed740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55daabfda3b0 con 0x55daabc271b0 2024-09-21T00:32:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.473+0000 7f8b057ed740 10 client.0 _ll_register_callbacks cb 0x55daabee6100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:32:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.473+0000 7f8af7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabbfe2f0 0x55daabfd7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.473+0000 7f8af6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabc064d0 0x55daabfd7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.473+0000 7f8af7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabbfe2f0 0x55daabfd7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37180/0 (socket says 172.21.15.27:37180) 2024-09-21T00:32:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.473+0000 7f8af6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabc064d0 0x55daabfd7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:33612/0 (socket says 172.21.15.27:33612) 2024-09-21T00:32:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.473+0000 7f8af7e00640 1 -- 172.21.15.27:0/2715567411 learned_addr learned my addr 172.21.15.27:0/2715567411 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.474+0000 7f8af7e00640 1 --2- 172.21.15.27:0/2715567411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc271b0 0x55daabfd7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.474+0000 7f8af6a00640 1 -- 172.21.15.27:0/2715567411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc271b0 msgr2=0x55daabfd7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.474+0000 7f8af6a00640 1 --2- 172.21.15.27:0/2715567411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc271b0 0x55daabfd7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.474+0000 7f8af6a00640 1 -- 172.21.15.27:0/2715567411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabbfe2f0 msgr2=0x55daabfd7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.474+0000 7f8af6a00640 1 --2- 172.21.15.27:0/2715567411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabbfe2f0 0x55daabfd7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.474+0000 7f8af6a00640 1 -- 172.21.15.27:0/2715567411 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daabc92790 con 0x55daabc064d0 2024-09-21T00:32:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.474+0000 7f8af7e00640 1 --2- 172.21.15.27:0/2715567411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc271b0 0x55daabfd7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:32:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.474+0000 7f8af6a00640 1 --2- 172.21.15.27:0/2715567411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabc064d0 0x55daabfd7790 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8ae402a9e0 tx=0x7f8ae40042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15c8231bb5d39db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.474+0000 7f8ae1600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:32:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.474+0000 7f8ae1600640 1 -- 172.21.15.27:0/2715567411 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4035040 con 0x55daabc064d0 2024-09-21T00:32:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.474+0000 7f8ae1600640 1 -- 172.21.15.27:0/2715567411 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9503+0+0 (secure 0 0 0) 0x7f8ae4005450 con 0x55daabc064d0 2024-09-21T00:32:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.474+0000 7f8b057ed740 1 -- 172.21.15.27:0/2715567411 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8af0004050 con 0x55daabc064d0 2024-09-21T00:32:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.475+0000 7f8ae1600640 1 -- 172.21.15.27:0/2715567411 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8ae404e050 con 0x55daabc064d0 2024-09-21T00:32:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.475+0000 7f8b057ed740 10 client.26525 fetch_fsmap learned FSMap version 43 2024-09-21T00:32:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.475+0000 7f8b057ed740 1 -- 172.21.15.27:0/2715567411 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55daabd0b620 con 0x55daabc064d0 2024-09-21T00:32:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.475+0000 7f8ae1600640 1 -- 172.21.15.27:0/2715567411 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8af0004050 con 0x55daabc064d0 2024-09-21T00:32:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.475+0000 7f8ae1600640 1 -- 172.21.15.27:0/2715567411 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f8ae4003850 con 0x55daabc064d0 2024-09-21T00:32:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.475+0000 7f8b057ed740 10 client.26525 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T00:32:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.475+0000 7f8b057ed740 10 client.26525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:32:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.475+0000 7f8b057ed740 20 client.26525 populate_metadata read hostname 'smithi027' 2024-09-21T00:32:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.475+0000 7f8b057ed740 1 --2- 172.21.15.27:0/2715567411 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55daabbe5d80 0x55daabfe9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.475+0000 7f8b057ed740 4 client.26525 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:32:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.475+0000 7f8b057ed740 1 -- 172.21.15.27:0/2715567411 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daabf43600 con 0x55daabbe5d80 2024-09-21T00:32:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.476+0000 7f8af7e00640 1 --2- 172.21.15.27:0/2715567411 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55daabbe5d80 0x55daabfe9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.476+0000 7f8af7e00640 1 --2- 172.21.15.27:0/2715567411 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55daabbe5d80 0x55daabfe9370 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33380a1a7b5f32ff server_cookie=6a70588241c5d100 in_seq=0 out_seq=0 2024-09-21T00:32:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.476+0000 7f8ae1600640 10 client.26525 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.477+0000 7f8ae1600640 1 -- 172.21.15.27:0/2715567411 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55daabf43600 con 0x55daabbe5d80 2024-09-21T00:32:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.477+0000 7f8ae1600640 10 client.26525 handle_command_reply: tid=0 2024-09-21T00:32:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.477+0000 7f8b057ed740 1 client.26525 shutdown 2024-09-21T00:32:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.477+0000 7f8b057ed740 1 -- 172.21.15.27:0/2715567411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabc064d0 msgr2=0x55daabfd7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.477+0000 7f8b057ed740 1 --2- 172.21.15.27:0/2715567411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabc064d0 0x55daabfd7790 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8ae402a9e0 tx=0x7f8ae40042e0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.478+0000 7f8b057ed740 1 -- 172.21.15.27:0/2715567411 shutdown_connections 2024-09-21T00:32:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.478+0000 7f8b057ed740 1 --2- 172.21.15.27:0/2715567411 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55daabbe5d80 0x55daabfe9370 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.478+0000 7f8b057ed740 1 --2- 172.21.15.27:0/2715567411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc271b0 0x55daabfd7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.478+0000 7f8b057ed740 1 --2- 172.21.15.27:0/2715567411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabc064d0 0x55daabfd7790 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.478+0000 7f8b057ed740 1 --2- 172.21.15.27:0/2715567411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabbfe2f0 0x55daabfd7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.478+0000 7f8b057ed740 1 -- 172.21.15.27:0/2715567411 >> 172.21.15.27:0/2715567411 conn(0x55daabc1b300 msgr2=0x55daabfccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.478+0000 7f8ae1600640 0 client.26525 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.478+0000 7f8b057ed740 1 -- 172.21.15.27:0/2715567411 shutdown_connections 2024-09-21T00:32:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.478+0000 7f8b057ed740 1 -- 172.21.15.27:0/2715567411 wait complete. 2024-09-21T00:32:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.478+0000 7f8b057ed740 20 client.26525 trim_cache size 0 max 16384 2024-09-21T00:32:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.497+0000 7f8b057ed740 1 Processor -- start 2024-09-21T00:32:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.497+0000 7f8b057ed740 1 -- start start 2024-09-21T00:32:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.497+0000 7f8b057ed740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabbfe2f0 0x55daabfcd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.497+0000 7f8b057ed740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc064d0 0x55daabfd7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.497+0000 7f8b057ed740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabc271b0 0x55daabfd7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.497+0000 7f8b057ed740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8af81b9f60 con 0x55daabc271b0 2024-09-21T00:32:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.497+0000 7f8b057ed740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8af81b77d0 con 0x55daabbfe2f0 2024-09-21T00:32:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.497+0000 7f8b057ed740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8af81b5640 con 0x55daabc064d0 2024-09-21T00:32:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.498+0000 7f8af6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc064d0 0x55daabfd7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.498+0000 7f8af6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc064d0 0x55daabfd7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:46290/0 (socket says 172.21.15.27:46290) 2024-09-21T00:32:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.498+0000 7f8af7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabbfe2f0 0x55daabfcd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.498+0000 7f8af6a00640 1 -- 172.21.15.27:0/2045505477 learned_addr learned my addr 172.21.15.27:0/2045505477 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.498+0000 7f8af7e00640 1 --2- 172.21.15.27:0/2045505477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabc271b0 0x55daabfd7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.498+0000 7f8af6a00640 1 -- 172.21.15.27:0/2045505477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabbfe2f0 msgr2=0x55daabfcd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.498+0000 7f8af6a00640 1 --2- 172.21.15.27:0/2045505477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabbfe2f0 0x55daabfcd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.498+0000 7f8af6a00640 1 -- 172.21.15.27:0/2045505477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabc271b0 msgr2=0x55daabfd7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.498+0000 7f8af6a00640 1 --2- 172.21.15.27:0/2045505477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabc271b0 0x55daabfd7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.498+0000 7f8af6a00640 1 -- 172.21.15.27:0/2045505477 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daabfd9d90 con 0x55daabc064d0 2024-09-21T00:32:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.498+0000 7f8af6a00640 1 --2- 172.21.15.27:0/2045505477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc064d0 0x55daabfd7730 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8ae402a430 tx=0x7f8ae4001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de0386aafaf1e2a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.499+0000 7f8ae3e00640 1 -- 172.21.15.27:0/2045505477 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4057070 con 0x55daabc064d0 2024-09-21T00:32:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.499+0000 7f8ae3e00640 1 -- 172.21.15.27:0/2045505477 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ae4040070 con 0x55daabc064d0 2024-09-21T00:32:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.499+0000 7f8ae3e00640 1 -- 172.21.15.27:0/2045505477 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4035030 con 0x55daabc064d0 2024-09-21T00:32:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.500+0000 7f8b057ed740 1 -- 172.21.15.27:0/2045505477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc064d0 msgr2=0x55daabfd7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.500+0000 7f8b057ed740 1 --2- 172.21.15.27:0/2045505477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc064d0 0x55daabfd7730 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8ae402a430 tx=0x7f8ae4001e70 comp rx=0 tx=0).stop 2024-09-21T00:32:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.500+0000 7f8b057ed740 1 -- 172.21.15.27:0/2045505477 shutdown_connections 2024-09-21T00:32:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.500+0000 7f8b057ed740 1 --2- 172.21.15.27:0/2045505477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabc271b0 0x55daabfd7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.500+0000 7f8b057ed740 1 --2- 172.21.15.27:0/2045505477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc064d0 0x55daabfd7730 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.500+0000 7f8b057ed740 1 --2- 172.21.15.27:0/2045505477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabbfe2f0 0x55daabfcd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.500+0000 7f8b057ed740 1 -- 172.21.15.27:0/2045505477 >> 172.21.15.27:0/2045505477 conn(0x55daabc1b300 msgr2=0x55daabfccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.500+0000 7f8b057ed740 1 -- 172.21.15.27:0/2045505477 shutdown_connections 2024-09-21T00:32:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.501+0000 7f8b057ed740 1 -- 172.21.15.27:0/2045505477 wait complete. 2024-09-21T00:32:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.501+0000 7f8b057ed740 1 -- start start 2024-09-21T00:32:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.501+0000 7f8b057ed740 1 Processor -- start 2024-09-21T00:32:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.502+0000 7f8b057ed740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabbfe2f0 0x55daabfdfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.502+0000 7f8b057ed740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabc064d0 0x55daabfdb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.502+0000 7f8b057ed740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc271b0 0x55daabfdb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.502+0000 7f8b057ed740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8af81b5c90 con 0x55daabc064d0 2024-09-21T00:32:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.502+0000 7f8b057ed740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8af81b7100 con 0x55daabbfe2f0 2024-09-21T00:32:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.502+0000 7f8b057ed740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8af81b6360 con 0x55daabc271b0 2024-09-21T00:32:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.502+0000 7f8b057ed740 10 client.0 _ll_register_callbacks cb 0x7f8ae8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:32:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.502+0000 7f8af7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabbfe2f0 0x55daabfdfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.502+0000 7f8af7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabbfe2f0 0x55daabfdfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:33620/0 (socket says 172.21.15.27:33620) 2024-09-21T00:32:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.502+0000 7f8af7e00640 1 -- 172.21.15.27:0/911446045 learned_addr learned my addr 172.21.15.27:0/911446045 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:32:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.502+0000 7f8af7e00640 1 --2- 172.21.15.27:0/911446045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc271b0 0x55daabfdb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.502+0000 7f8af6a00640 1 --2- 172.21.15.27:0/911446045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabc064d0 0x55daabfdb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.503+0000 7f8af6a00640 1 -- 172.21.15.27:0/911446045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc271b0 msgr2=0x55daabfdb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.503+0000 7f8af6a00640 1 --2- 172.21.15.27:0/911446045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc271b0 0x55daabfdb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.503+0000 7f8af6a00640 1 -- 172.21.15.27:0/911446045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabbfe2f0 msgr2=0x55daabfdfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:32:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.503+0000 7f8af6a00640 1 --2- 172.21.15.27:0/911446045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabbfe2f0 0x55daabfdfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.503+0000 7f8af6a00640 1 -- 172.21.15.27:0/911446045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daabfdbb90 con 0x55daabc064d0 2024-09-21T00:32:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.503+0000 7f8af7e00640 1 --2- 172.21.15.27:0/911446045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabbfe2f0 0x55daabfdfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:32:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.504+0000 7f8af6a00640 1 --2- 172.21.15.27:0/911446045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabc064d0 0x55daabfdb0e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4003dc0 tx=0x7f8ae4031520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fc29976631b06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:32:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.504+0000 7f8ae1600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:32:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.504+0000 7f8ae1600640 1 -- 172.21.15.27:0/911446045 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4057070 con 0x55daabc064d0 2024-09-21T00:32:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.504+0000 7f8ae1600640 1 -- 172.21.15.27:0/911446045 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9503+0+0 (secure 0 0 0) 0x7f8ae40037a0 con 0x55daabc064d0 2024-09-21T00:32:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.504+0000 7f8b057ed740 1 -- 172.21.15.27:0/911446045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ae4040070 con 0x55daabc064d0 2024-09-21T00:32:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.505+0000 7f8ae1600640 1 -- 172.21.15.27:0/911446045 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8ae4035040 con 0x55daabc064d0 2024-09-21T00:32:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.505+0000 7f8b057ed740 10 client.16536 fetch_fsmap learned FSMap version 43 2024-09-21T00:32:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.505+0000 7f8b057ed740 1 -- 172.21.15.27:0/911446045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55daac053a50 con 0x55daabc064d0 2024-09-21T00:32:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.505+0000 7f8ae1600640 1 -- 172.21.15.27:0/911446045 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ae4040070 con 0x55daabc064d0 2024-09-21T00:32:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.505+0000 7f8ae1600640 1 -- 172.21.15.27:0/911446045 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f8ae4004b40 con 0x55daabc064d0 2024-09-21T00:32:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.505+0000 7f8b057ed740 10 client.16536 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T00:32:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.505+0000 7f8b057ed740 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:32:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.505+0000 7f8b057ed740 20 client.16536 populate_metadata read hostname 'smithi027' 2024-09-21T00:32:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.505+0000 7f8b057ed740 1 --2- 172.21.15.27:0/911446045 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55daabbbb1c0 0x55daac056070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:32:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.505+0000 7f8b057ed740 4 client.16536 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:32:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.505+0000 7f8b057ed740 1 -- 172.21.15.27:0/911446045 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55daabf3f0b0 con 0x55daabbbb1c0 2024-09-21T00:32:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.506+0000 7f8af7e00640 1 --2- 172.21.15.27:0/911446045 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55daabbbb1c0 0x55daac056070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:32:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.506+0000 7f8af7e00640 1 --2- 172.21.15.27:0/911446045 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55daabbbb1c0 0x55daac056070 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c097b00d4b3fd0 server_cookie=386531691589c07f in_seq=0 out_seq=0 2024-09-21T00:32:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:53.506+0000 7f8ae1600640 10 client.16536 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.124+0000 7f8ae1600640 1 -- 172.21.15.27:0/911446045 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55daabf3f0b0 con 0x55daabbbb1c0 2024-09-21T00:32:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.124+0000 7f8ae1600640 10 client.16536 handle_command_reply: tid=0 2024-09-21T00:32:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.125+0000 7f8b057ed740 1 client.16536 shutdown 2024-09-21T00:32:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.126+0000 7f8b057ed740 1 -- 172.21.15.27:0/911446045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabc064d0 msgr2=0x55daabfdb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.126+0000 7f8b057ed740 1 --2- 172.21.15.27:0/911446045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabc064d0 0x55daabfdb0e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4003dc0 tx=0x7f8ae4031520 comp rx=0 tx=0).stop 2024-09-21T00:32:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.126+0000 7f8b057ed740 1 -- 172.21.15.27:0/911446045 shutdown_connections 2024-09-21T00:32:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.126+0000 7f8b057ed740 1 --2- 172.21.15.27:0/911446045 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55daabbbb1c0 0x55daac056070 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.126+0000 7f8b057ed740 1 --2- 172.21.15.27:0/911446045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55daabc271b0 0x55daabfdb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.126+0000 7f8b057ed740 1 --2- 172.21.15.27:0/911446045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daabc064d0 0x55daabfdb0e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.126+0000 7f8b057ed740 1 --2- 172.21.15.27:0/911446045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daabbfe2f0 0x55daabfdfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.126+0000 7f8b057ed740 1 -- 172.21.15.27:0/911446045 >> 172.21.15.27:0/911446045 conn(0x55daabc1b300 msgr2=0x55daabfd90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.126+0000 7f8ae1600640 0 client.16536 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:32:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.127+0000 7f8b057ed740 1 -- 172.21.15.27:0/911446045 shutdown_connections 2024-09-21T00:32:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.127+0000 7f8b057ed740 1 -- 172.21.15.27:0/911446045 wait complete. 2024-09-21T00:32:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.127+0000 7f8b057ed740 20 client.16536 trim_cache size 0 max 16384 2024-09-21T00:32:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:32:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:32:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:32:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:32:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.128+0000 7f8afe800640 1 -- 172.21.15.27:0/3500183177 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8adc0b6700 msgr2=0x7f8adc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.128+0000 7f8afe800640 1 --2- 172.21.15.27:0/3500183177 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8adc0b6700 0x7f8adc0b8c10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55daabfca8b0 tx=0x7f8ae801f000 comp rx=0 tx=0).stop 2024-09-21T00:32:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.128+0000 7f8afe800640 1 -- 172.21.15.27:0/3500183177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af8075dd0 msgr2=0x7f8af81a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:32:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.128+0000 7f8afe800640 1 --2- 172.21.15.27:0/3500183177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af8075dd0 0x7f8af81a0a50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800e9f0 tx=0x7f8ae800eeb0 comp rx=0 tx=0).stop 2024-09-21T00:32:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.129+0000 7f8afe800640 1 -- 172.21.15.27:0/3500183177 shutdown_connections 2024-09-21T00:32:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.129+0000 7f8afe800640 1 --2- 172.21.15.27:0/3500183177 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8adc0b6700 0x7f8adc0b8c10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.129+0000 7f8afe800640 1 --2- 172.21.15.27:0/3500183177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8af8114390 0x7f8af81a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.129+0000 7f8afe800640 1 --2- 172.21.15.27:0/3500183177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8af8077390 0x7f8af81a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.129+0000 7f8afe800640 1 --2- 172.21.15.27:0/3500183177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af8075dd0 0x7f8af81a0a50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:32:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.129+0000 7f8afe800640 1 -- 172.21.15.27:0/3500183177 >> 172.21.15.27:0/3500183177 conn(0x7f8af80ff4d0 msgr2=0x7f8af8110290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:32:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.129+0000 7f8afe800640 1 -- 172.21.15.27:0/3500183177 shutdown_connections 2024-09-21T00:32:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:32:54.129+0000 7f8afe800640 1 -- 172.21.15.27:0/3500183177 wait complete. 2024-09-21T00:32:54.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:54 smithi089 ceph-mon[34481]: pgmap v737: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 780 op/s 2024-09-21T00:32:54.729 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:32:54 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h[62840]: ignoring --setuser ceph since I am not root 2024-09-21T00:32:54.729 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:32:54 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h[62840]: ignoring --setgroup ceph since I am not root 2024-09-21T00:32:54.729 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:32:54 smithi089 ceph-mds[62844]: --2- 172.21.15.89:0/736058341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e7acd1a80 0x560e7acb6b00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x560e7ace2b70 tx=0x560e7ac09320 comp rx=0 tx=0).write_message sending message m=0x560e7ab968c0 seq=3 auth(proto 2 2 bytes epoch 0) 2024-09-21T00:32:54.729 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:32:54 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h[62840]: starting mds.h at 2024-09-21T00:32:54.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:54 smithi098 ceph-mon[34725]: pgmap v737: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 780 op/s 2024-09-21T00:32:54.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:54 smithi027 ceph-mon[30745]: pgmap v737: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 780 op/s 2024-09-21T00:32:55.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:55 smithi089 ceph-mon[34481]: daemon mds.h restarted 2024-09-21T00:32:55.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:55 smithi098 ceph-mon[34725]: daemon mds.h restarted 2024-09-21T00:32:55.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:55 smithi027 ceph-mon[30745]: daemon mds.h restarted 2024-09-21T00:32:56.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:56 smithi089 ceph-mon[34481]: pgmap v738: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 688 op/s 2024-09-21T00:32:56.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:56 smithi089 ceph-mon[34481]: osdmap e106: 12 total, 12 up, 12 in 2024-09-21T00:32:56.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:56 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.89:6836/2873147127,v1:172.21.15.89:6837/2873147127] up:boot 2024-09-21T00:32:56.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:56 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T00:32:56.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:56 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-21T00:32:56.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:56 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:32:56.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:56 smithi098 ceph-mon[34725]: pgmap v738: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 688 op/s 2024-09-21T00:32:56.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:56 smithi098 ceph-mon[34725]: osdmap e106: 12 total, 12 up, 12 in 2024-09-21T00:32:56.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:56 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.89:6836/2873147127,v1:172.21.15.89:6837/2873147127] up:boot 2024-09-21T00:32:56.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:56 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T00:32:56.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:56 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-21T00:32:56.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:56 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:32:56.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:56 smithi027 ceph-mon[30745]: pgmap v738: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 688 op/s 2024-09-21T00:32:56.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:56 smithi027 ceph-mon[30745]: osdmap e106: 12 total, 12 up, 12 in 2024-09-21T00:32:56.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:56 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.89:6836/2873147127,v1:172.21.15.89:6837/2873147127] up:boot 2024-09-21T00:32:56.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:56 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T00:32:56.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:56 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-21T00:32:56.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:56 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:32:58.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:58 smithi089 ceph-mon[34481]: pgmap v740: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 538 op/s 2024-09-21T00:32:58.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:58 smithi098 ceph-mon[34725]: pgmap v740: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 538 op/s 2024-09-21T00:32:58.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:58 smithi027 ceph-mon[30745]: pgmap v740: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 538 op/s 2024-09-21T00:32:59.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:32:59 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:32:59.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:32:59 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:32:59.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:32:59 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:33:00.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:00 smithi089 ceph-mon[34481]: pgmap v741: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 586 op/s 2024-09-21T00:33:00.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:00 smithi098 ceph-mon[34725]: pgmap v741: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 586 op/s 2024-09-21T00:33:00.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:00 smithi027 ceph-mon[30745]: pgmap v741: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 586 op/s 2024-09-21T00:33:02.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:02 smithi089 ceph-mon[34481]: pgmap v742: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 587 op/s 2024-09-21T00:33:02.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:02 smithi098 ceph-mon[34725]: pgmap v742: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 587 op/s 2024-09-21T00:33:02.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:02 smithi027 ceph-mon[30745]: pgmap v742: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 587 op/s 2024-09-21T00:33:03.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:03 smithi089 ceph-mon[34481]: pgmap v743: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 573 op/s 2024-09-21T00:33:03.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:03 smithi089 ceph-mon[34481]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:33:03.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:03 smithi098 ceph-mon[34725]: pgmap v743: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 573 op/s 2024-09-21T00:33:03.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:03 smithi098 ceph-mon[34725]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:33:03.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:03 smithi027 ceph-mon[30745]: pgmap v743: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 573 op/s 2024-09-21T00:33:03.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:03 smithi027 ceph-mon[30745]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T00:33:06.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:05 smithi027 ceph-mon[30745]: pgmap v744: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 488 op/s 2024-09-21T00:33:06.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:05 smithi089 ceph-mon[34481]: pgmap v744: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 488 op/s 2024-09-21T00:33:06.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:05 smithi098 ceph-mon[34725]: pgmap v744: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 488 op/s 2024-09-21T00:33:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.460+0000 7fe61da00640 1 -- 172.21.15.27:0/1378471302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61810b450 msgr2=0x7fe61810b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.460+0000 7fe61da00640 1 --2- 172.21.15.27:0/1378471302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61810b450 0x7fe61810b860 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe604009860 tx=0x7fe60402a340 comp rx=0 tx=0).stop 2024-09-21T00:33:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.461+0000 7fe61da00640 1 -- 172.21.15.27:0/1378471302 shutdown_connections 2024-09-21T00:33:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.461+0000 7fe61da00640 1 --2- 172.21.15.27:0/1378471302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe61810bda0 0x7fe6181125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.461+0000 7fe61da00640 1 --2- 172.21.15.27:0/1378471302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61810b450 0x7fe61810b860 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.461+0000 7fe61da00640 1 --2- 172.21.15.27:0/1378471302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe618105b20 0x7fe618105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.461+0000 7fe61da00640 1 -- 172.21.15.27:0/1378471302 >> 172.21.15.27:0/1378471302 conn(0x7fe618101560 msgr2=0x7fe6181039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.461+0000 7fe61da00640 1 -- 172.21.15.27:0/1378471302 shutdown_connections 2024-09-21T00:33:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.461+0000 7fe61da00640 1 -- 172.21.15.27:0/1378471302 wait complete. 2024-09-21T00:33:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.463+0000 7fe61da00640 1 Processor -- start 2024-09-21T00:33:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.463+0000 7fe61da00640 1 -- start start 2024-09-21T00:33:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.464+0000 7fe61da00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe618105b20 0x7fe61819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.464+0000 7fe61da00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe61810b450 0x7fe61819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.464+0000 7fe61da00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61810bda0 0x7fe6181a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.464+0000 7fe61da00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe618115ab0 con 0x7fe618105b20 2024-09-21T00:33:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.464+0000 7fe61da00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe618115930 con 0x7fe61810b450 2024-09-21T00:33:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.464+0000 7fe61da00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe618115c30 con 0x7fe61810bda0 2024-09-21T00:33:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.464+0000 7fe61d000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61810bda0 0x7fe6181a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.466+0000 7fe61d000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61810bda0 0x7fe6181a1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:43326/0 (socket says 172.21.15.27:43326) 2024-09-21T00:33:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.466+0000 7fe61d000640 1 -- 172.21.15.27:0/1087250240 learned_addr learned my addr 172.21.15.27:0/1087250240 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.464+0000 7fe617e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe618105b20 0x7fe61819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.464+0000 7fe617400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe61810b450 0x7fe61819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.466+0000 7fe61d000640 1 -- 172.21.15.27:0/1087250240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe61810b450 msgr2=0x7fe61819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.466+0000 7fe61d000640 1 --2- 172.21.15.27:0/1087250240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe61810b450 0x7fe61819ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.466+0000 7fe61d000640 1 -- 172.21.15.27:0/1087250240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe618105b20 msgr2=0x7fe61819c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.466+0000 7fe61d000640 1 --2- 172.21.15.27:0/1087250240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe618105b20 0x7fe61819c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.466+0000 7fe61d000640 1 -- 172.21.15.27:0/1087250240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6181a2410 con 0x7fe61810bda0 2024-09-21T00:33:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.467+0000 7fe61d000640 1 --2- 172.21.15.27:0/1087250240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61810bda0 0x7fe6181a1d10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe61000eaa0 tx=0x7fe61000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.467+0000 7fe614c00640 1 -- 172.21.15.27:0/1087250240 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61000cdb0 con 0x7fe61810bda0 2024-09-21T00:33:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.468+0000 7fe614c00640 1 -- 172.21.15.27:0/1087250240 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe610005020 con 0x7fe61810bda0 2024-09-21T00:33:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.468+0000 7fe614c00640 1 -- 172.21.15.27:0/1087250240 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6100116d0 con 0x7fe61810bda0 2024-09-21T00:33:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.469+0000 7fe61da00640 1 -- 172.21.15.27:0/1087250240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6181a2700 con 0x7fe61810bda0 2024-09-21T00:33:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.469+0000 7fe61da00640 1 -- 172.21.15.27:0/1087250240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe618077800 con 0x7fe61810bda0 2024-09-21T00:33:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.478+0000 7fe626052740 1 Processor -- start 2024-09-21T00:33:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.480+0000 7fe614c00640 1 -- 172.21.15.27:0/1087250240 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe6100051d0 con 0x7fe61810bda0 2024-09-21T00:33:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.480+0000 7fe626052740 1 -- start start 2024-09-21T00:33:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.483+0000 7fe614c00640 1 --2- 172.21.15.27:0/1087250240 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe5fc0b66f0 0x7fe5fc0b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.483+0000 7fe626052740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e311971b0 0x557e3153be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.483+0000 7fe626052740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e3116e2f0 0x557e3153c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.483+0000 7fe626052740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e311764d0 0x557e3153c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.484+0000 7fe617400640 1 --2- 172.21.15.27:0/1087250240 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe5fc0b66f0 0x7fe5fc0b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.484+0000 7fe626052740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557e314b16b0 con 0x557e311764d0 2024-09-21T00:33:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.484+0000 7fe626052740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557e314a0ae0 con 0x557e311971b0 2024-09-21T00:33:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.484+0000 7fe626052740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557e314a48f0 con 0x557e3116e2f0 2024-09-21T00:33:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.484+0000 7fe617e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e311971b0 0x557e3153be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.484+0000 7fe617e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e311971b0 0x557e3153be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:51126/0 (socket says 172.21.15.27:51126) 2024-09-21T00:33:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.484+0000 7fe617e00640 1 -- 172.21.15.27:0/1963959887 learned_addr learned my addr 172.21.15.27:0/1963959887 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.484+0000 7fe61d000640 1 --2- 172.21.15.27:0/1963959887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e311764d0 0x557e3153c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.485+0000 7fe617e00640 1 -- 172.21.15.27:0/1963959887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e3116e2f0 msgr2=0x557e3153c350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:33:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.485+0000 7fe617e00640 1 --2- 172.21.15.27:0/1963959887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e3116e2f0 0x557e3153c350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.485+0000 7fe617e00640 1 -- 172.21.15.27:0/1963959887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e311764d0 msgr2=0x557e3153c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.485+0000 7fe617e00640 1 --2- 172.21.15.27:0/1963959887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e311764d0 0x557e3153c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.485+0000 7fe617e00640 1 -- 172.21.15.27:0/1963959887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e3146d780 con 0x557e311971b0 2024-09-21T00:33:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.484+0000 7fe614c00640 1 -- 172.21.15.27:0/1087250240 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9431+0+0 (secure 0 0 0) 0x7fe6100e7b90 con 0x7fe61810bda0 2024-09-21T00:33:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.485+0000 7fe617e00640 1 --2- 172.21.15.27:0/1963959887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e311971b0 0x557e3153be10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe608009b80 tx=0x7fe608010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17deea48f8536e3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.486+0000 7fe60d600640 1 -- 172.21.15.27:0/1963959887 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60801b480 con 0x557e311971b0 2024-09-21T00:33:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.486+0000 7fe60d600640 1 -- 172.21.15.27:0/1963959887 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe60801bad0 con 0x557e311971b0 2024-09-21T00:33:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.487+0000 7fe60d600640 1 -- 172.21.15.27:0/1963959887 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608012a30 con 0x557e311971b0 2024-09-21T00:33:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.488+0000 7fe626052740 1 -- 172.21.15.27:0/1963959887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e311971b0 msgr2=0x557e3153be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.488+0000 7fe617400640 1 --2- 172.21.15.27:0/1087250240 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe5fc0b66f0 0x7fe5fc0b8c00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe604035040 tx=0x7fe604030340 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.488+0000 7fe626052740 1 --2- 172.21.15.27:0/1963959887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e311971b0 0x557e3153be10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe608009b80 tx=0x7fe608010740 comp rx=0 tx=0).stop 2024-09-21T00:33:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.491+0000 7fe626052740 1 -- 172.21.15.27:0/1963959887 shutdown_connections 2024-09-21T00:33:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.491+0000 7fe626052740 1 --2- 172.21.15.27:0/1963959887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e311764d0 0x557e3153c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.491+0000 7fe626052740 1 --2- 172.21.15.27:0/1963959887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e3116e2f0 0x557e3153c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.491+0000 7fe626052740 1 --2- 172.21.15.27:0/1963959887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e311971b0 0x557e3153be10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.491+0000 7fe626052740 1 -- 172.21.15.27:0/1963959887 >> 172.21.15.27:0/1963959887 conn(0x557e3118b300 msgr2=0x557e31539d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.491+0000 7fe626052740 1 -- 172.21.15.27:0/1963959887 shutdown_connections 2024-09-21T00:33:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.491+0000 7fe626052740 1 -- 172.21.15.27:0/1963959887 wait complete. 2024-09-21T00:33:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.492+0000 7fe626052740 1 -- start start 2024-09-21T00:33:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.492+0000 7fe626052740 1 Processor -- start 2024-09-21T00:33:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.493+0000 7fe626052740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e3116e2f0 0x557e31547350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.493+0000 7fe626052740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e311764d0 0x557e31547890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.494+0000 7fe626052740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e31547dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.494+0000 7fe626052740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557e313d8ab0 con 0x557e311764d0 2024-09-21T00:33:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.494+0000 7fe626052740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557e31447570 con 0x557e3116e2f0 2024-09-21T00:33:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.494+0000 7fe626052740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557e3154a630 con 0x557e311971b0 2024-09-21T00:33:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.494+0000 7fe626052740 10 client.0 _ll_register_callbacks cb 0x557e31456100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:33:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.494+0000 7fe61d000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e31547dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.494+0000 7fe61d000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e31547dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:43334/0 (socket says 172.21.15.27:43334) 2024-09-21T00:33:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.494+0000 7fe61d000640 1 -- 172.21.15.27:0/4043921112 learned_addr learned my addr 172.21.15.27:0/4043921112 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.494+0000 7fe617e00640 1 --2- 172.21.15.27:0/4043921112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e3116e2f0 0x557e31547350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.495+0000 7fe617400640 1 --2- 172.21.15.27:0/4043921112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e311764d0 0x557e31547890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.495+0000 7fe61d000640 1 -- 172.21.15.27:0/4043921112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e3116e2f0 msgr2=0x557e31547350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.495+0000 7fe61d000640 1 --2- 172.21.15.27:0/4043921112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e3116e2f0 0x557e31547350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.495+0000 7fe61d000640 1 -- 172.21.15.27:0/4043921112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e311764d0 msgr2=0x557e31547890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.495+0000 7fe61d000640 1 --2- 172.21.15.27:0/4043921112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e311764d0 0x557e31547890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.495+0000 7fe61d000640 1 -- 172.21.15.27:0/4043921112 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e31202790 con 0x557e311971b0 2024-09-21T00:33:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.495+0000 7fe617400640 1 --2- 172.21.15.27:0/4043921112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e311764d0 0x557e31547890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:33:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.496+0000 7fe61d000640 1 --2- 172.21.15.27:0/4043921112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e31547dd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe61000efd0 tx=0x7fe6100057e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b6080150b708546 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.496+0000 7fe602000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:33:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.497+0000 7fe602000640 1 -- 172.21.15.27:0/4043921112 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61001f040 con 0x557e311971b0 2024-09-21T00:33:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.497+0000 7fe602000640 1 -- 172.21.15.27:0/4043921112 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9431+0+0 (secure 0 0 0) 0x7fe6100ac320 con 0x557e311971b0 2024-09-21T00:33:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.499+0000 7fe626052740 1 -- 172.21.15.27:0/4043921112 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe60801bad0 con 0x557e311971b0 2024-09-21T00:33:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.501+0000 7fe602000640 1 -- 172.21.15.27:0/4043921112 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe6100add50 con 0x557e311971b0 2024-09-21T00:33:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.501+0000 7fe626052740 10 client.26557 fetch_fsmap learned FSMap version 47 2024-09-21T00:33:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.501+0000 7fe626052740 1 -- 172.21.15.27:0/4043921112 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557e3127b620 con 0x557e311971b0 2024-09-21T00:33:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.501+0000 7fe602000640 1 -- 172.21.15.27:0/4043921112 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe60801bad0 con 0x557e311971b0 2024-09-21T00:33:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.501+0000 7fe602000640 1 -- 172.21.15.27:0/4043921112 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fe61000f040 con 0x557e311971b0 2024-09-21T00:33:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.503+0000 7fe626052740 10 client.26557 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:33:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.503+0000 7fe626052740 10 client.26557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:33:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.503+0000 7fe626052740 20 client.26557 populate_metadata read hostname 'smithi027' 2024-09-21T00:33:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.503+0000 7fe626052740 1 --2- 172.21.15.27:0/4043921112 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557e31155d80 0x557e31559560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.503+0000 7fe626052740 4 client.26557 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:33:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.503+0000 7fe626052740 1 -- 172.21.15.27:0/4043921112 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e314b3600 con 0x557e31155d80 2024-09-21T00:33:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.504+0000 7fe617e00640 1 --2- 172.21.15.27:0/4043921112 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557e31155d80 0x557e31559560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.506+0000 7fe617e00640 1 --2- 172.21.15.27:0/4043921112 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557e31155d80 0x557e31559560 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=869caae794c0330a server_cookie=350b87a9bbc1eeda in_seq=0 out_seq=0 2024-09-21T00:33:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.506+0000 7fe602000640 10 client.26557 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.511+0000 7fe602000640 1 -- 172.21.15.27:0/4043921112 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557e314b3600 con 0x557e31155d80 2024-09-21T00:33:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.511+0000 7fe602000640 10 client.26557 handle_command_reply: tid=0 2024-09-21T00:33:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.512+0000 7fe626052740 1 client.26557 shutdown 2024-09-21T00:33:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.512+0000 7fe626052740 1 -- 172.21.15.27:0/4043921112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 msgr2=0x557e31547dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.512+0000 7fe626052740 1 --2- 172.21.15.27:0/4043921112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e31547dd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe61000efd0 tx=0x7fe6100057e0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.513+0000 7fe626052740 1 -- 172.21.15.27:0/4043921112 shutdown_connections 2024-09-21T00:33:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.513+0000 7fe626052740 1 --2- 172.21.15.27:0/4043921112 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557e31155d80 0x557e31559560 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.513+0000 7fe626052740 1 --2- 172.21.15.27:0/4043921112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e31547dd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.513+0000 7fe626052740 1 --2- 172.21.15.27:0/4043921112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e311764d0 0x557e31547890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.513+0000 7fe626052740 1 --2- 172.21.15.27:0/4043921112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e3116e2f0 0x557e31547350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.513+0000 7fe602000640 0 client.26557 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.513+0000 7fe626052740 1 -- 172.21.15.27:0/4043921112 >> 172.21.15.27:0/4043921112 conn(0x557e3118b300 msgr2=0x557e3153ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.513+0000 7fe626052740 1 -- 172.21.15.27:0/4043921112 shutdown_connections 2024-09-21T00:33:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.513+0000 7fe626052740 1 -- 172.21.15.27:0/4043921112 wait complete. 2024-09-21T00:33:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.515+0000 7fe626052740 20 client.26557 trim_cache size 0 max 16384 2024-09-21T00:33:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.555+0000 7fe626052740 1 Processor -- start 2024-09-21T00:33:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.555+0000 7fe626052740 1 -- start start 2024-09-21T00:33:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.556+0000 7fe626052740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e3116e2f0 0x557e3153d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.556+0000 7fe626052740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e311764d0 0x557e31544520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.556+0000 7fe626052740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e31544a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.556+0000 7fe626052740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe6181b15e0 con 0x557e3116e2f0 2024-09-21T00:33:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.556+0000 7fe626052740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe6181aedf0 con 0x557e311764d0 2024-09-21T00:33:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.556+0000 7fe626052740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe6181acc40 con 0x557e311971b0 2024-09-21T00:33:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.557+0000 7fe617400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e311764d0 0x557e31544520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.557+0000 7fe617400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e311764d0 0x557e31544520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:51134/0 (socket says 172.21.15.27:51134) 2024-09-21T00:33:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.557+0000 7fe617400640 1 -- 172.21.15.27:0/923369053 learned_addr learned my addr 172.21.15.27:0/923369053 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.557+0000 7fe61d000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e31544a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.558+0000 7fe617e00640 1 --2- 172.21.15.27:0/923369053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e3116e2f0 0x557e3153d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.559+0000 7fe617400640 1 -- 172.21.15.27:0/923369053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 msgr2=0x557e31544a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.559+0000 7fe617400640 1 --2- 172.21.15.27:0/923369053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e31544a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.559+0000 7fe617400640 1 -- 172.21.15.27:0/923369053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e3116e2f0 msgr2=0x557e3153d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.559+0000 7fe617400640 1 --2- 172.21.15.27:0/923369053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e3116e2f0 0x557e3153d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.559+0000 7fe617400640 1 -- 172.21.15.27:0/923369053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e3154a470 con 0x557e311764d0 2024-09-21T00:33:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.559+0000 7fe61d000640 1 --2- 172.21.15.27:0/923369053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e31544a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:33:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.560+0000 7fe617400640 1 --2- 172.21.15.27:0/923369053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e311764d0 0x557e31544520 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe604009860 tx=0x7fe604043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aa3736c775091e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.561+0000 7fe60cc00640 1 -- 172.21.15.27:0/923369053 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe604037040 con 0x557e311764d0 2024-09-21T00:33:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.561+0000 7fe60cc00640 1 -- 172.21.15.27:0/923369053 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe604030450 con 0x557e311764d0 2024-09-21T00:33:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.561+0000 7fe60cc00640 1 -- 172.21.15.27:0/923369053 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe604003970 con 0x557e311764d0 2024-09-21T00:33:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.562+0000 7fe626052740 1 -- 172.21.15.27:0/923369053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e311764d0 msgr2=0x557e31544520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.562+0000 7fe626052740 1 --2- 172.21.15.27:0/923369053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e311764d0 0x557e31544520 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe604009860 tx=0x7fe604043000 comp rx=0 tx=0).stop 2024-09-21T00:33:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.562+0000 7fe626052740 1 -- 172.21.15.27:0/923369053 shutdown_connections 2024-09-21T00:33:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.562+0000 7fe626052740 1 --2- 172.21.15.27:0/923369053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e31544a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.562+0000 7fe626052740 1 --2- 172.21.15.27:0/923369053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e311764d0 0x557e31544520 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.562+0000 7fe626052740 1 --2- 172.21.15.27:0/923369053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e3116e2f0 0x557e3153d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.562+0000 7fe626052740 1 -- 172.21.15.27:0/923369053 >> 172.21.15.27:0/923369053 conn(0x557e3118b300 msgr2=0x557e3153ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.562+0000 7fe626052740 1 -- 172.21.15.27:0/923369053 shutdown_connections 2024-09-21T00:33:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.563+0000 7fe626052740 1 -- 172.21.15.27:0/923369053 wait complete. 2024-09-21T00:33:07.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.563+0000 7fe626052740 1 -- start start 2024-09-21T00:33:07.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.564+0000 7fe626052740 1 Processor -- start 2024-09-21T00:33:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.564+0000 7fe626052740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e3116e2f0 0x557e31550260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.564+0000 7fe626052740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e3154d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.565+0000 7fe626052740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e3112b1c0 0x557e3154de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.565+0000 7fe626052740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe6181ad2b0 con 0x557e3116e2f0 2024-09-21T00:33:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.565+0000 7fe626052740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe6181ae720 con 0x557e3112b1c0 2024-09-21T00:33:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.565+0000 7fe626052740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe6181ad980 con 0x557e311971b0 2024-09-21T00:33:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.565+0000 7fe626052740 10 client.0 _ll_register_callbacks cb 0x7fe6100ad420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:33:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.565+0000 7fe617400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e3154d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.565+0000 7fe617400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e3154d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:43356/0 (socket says 172.21.15.27:43356) 2024-09-21T00:33:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.565+0000 7fe617400640 1 -- 172.21.15.27:0/4239387263 learned_addr learned my addr 172.21.15.27:0/4239387263 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.565+0000 7fe61d000640 1 --2- 172.21.15.27:0/4239387263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e3112b1c0 0x557e3154de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.566+0000 7fe617400640 1 -- 172.21.15.27:0/4239387263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e3112b1c0 msgr2=0x557e3154de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.566+0000 7fe617e00640 1 --2- 172.21.15.27:0/4239387263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e3116e2f0 0x557e31550260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.569+0000 7fe617400640 1 --2- 172.21.15.27:0/4239387263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e3112b1c0 0x557e3154de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.569+0000 7fe617400640 1 -- 172.21.15.27:0/4239387263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e3116e2f0 msgr2=0x557e31550260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.569+0000 7fe617400640 1 --2- 172.21.15.27:0/4239387263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e3116e2f0 0x557e31550260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.569+0000 7fe617400640 1 -- 172.21.15.27:0/4239387263 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e315c2f30 con 0x557e311971b0 2024-09-21T00:33:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.569+0000 7fe61d000640 1 --2- 172.21.15.27:0/4239387263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e3112b1c0 0x557e3154de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:33:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.570+0000 7fe617400640 1 --2- 172.21.15.27:0/4239387263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e3154d720 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe604004650 tx=0x7fe604041da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37e4765510987033 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.570+0000 7fe602000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:33:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.570+0000 7fe602000640 1 -- 172.21.15.27:0/4239387263 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe604037040 con 0x557e311971b0 2024-09-21T00:33:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.570+0000 7fe626052740 1 -- 172.21.15.27:0/4239387263 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe604030450 con 0x557e311971b0 2024-09-21T00:33:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.570+0000 7fe602000640 1 -- 172.21.15.27:0/4239387263 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9431+0+0 (secure 0 0 0) 0x7fe604004040 con 0x557e311971b0 2024-09-21T00:33:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.571+0000 7fe617e00640 1 --2- 172.21.15.27:0/4239387263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e3116e2f0 0x557e31550260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:33:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.571+0000 7fe602000640 1 -- 172.21.15.27:0/4239387263 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe604004240 con 0x557e311971b0 2024-09-21T00:33:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.571+0000 7fe626052740 10 client.26563 fetch_fsmap learned FSMap version 47 2024-09-21T00:33:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.571+0000 7fe626052740 1 -- 172.21.15.27:0/4239387263 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557e315c4030 con 0x557e311971b0 2024-09-21T00:33:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.572+0000 7fe602000640 1 -- 172.21.15.27:0/4239387263 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe604030450 con 0x557e311971b0 2024-09-21T00:33:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.572+0000 7fe602000640 1 -- 172.21.15.27:0/4239387263 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fe6040675a0 con 0x557e311971b0 2024-09-21T00:33:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.572+0000 7fe626052740 10 client.26563 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:33:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.572+0000 7fe626052740 10 client.26563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:33:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.572+0000 7fe626052740 20 client.26563 populate_metadata read hostname 'smithi027' 2024-09-21T00:33:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.572+0000 7fe626052740 1 --2- 172.21.15.27:0/4239387263 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557e315c47c0 0x557e315c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.572+0000 7fe626052740 4 client.26563 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:33:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.572+0000 7fe626052740 1 -- 172.21.15.27:0/4239387263 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e314af0b0 con 0x557e315c47c0 2024-09-21T00:33:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.573+0000 7fe617e00640 1 --2- 172.21.15.27:0/4239387263 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557e315c47c0 0x557e315c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.574+0000 7fe602000640 10 client.26563 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:07.575+0000 7fe617e00640 1 --2- 172.21.15.27:0/4239387263 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557e315c47c0 0x557e315c6be0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91566f223ddd2819 server_cookie=f93ea7c91bfc6e8c in_seq=0 out_seq=0 2024-09-21T00:33:08.047 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:07 smithi098 ceph-mon[34725]: pgmap v745: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 416 op/s 2024-09-21T00:33:08.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:07 smithi027 ceph-mon[30745]: pgmap v745: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 416 op/s 2024-09-21T00:33:08.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:07 smithi089 ceph-mon[34481]: pgmap v745: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 416 op/s 2024-09-21T00:33:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.345+0000 7fe602000640 1 -- 172.21.15.27:0/4239387263 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e314af0b0 con 0x557e315c47c0 2024-09-21T00:33:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.345+0000 7fe602000640 10 client.26563 handle_command_reply: tid=0 2024-09-21T00:33:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.345+0000 7fe626052740 1 client.26563 shutdown 2024-09-21T00:33:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.346+0000 7fe626052740 1 -- 172.21.15.27:0/4239387263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 msgr2=0x557e3154d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.346+0000 7fe626052740 1 --2- 172.21.15.27:0/4239387263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e3154d720 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe604004650 tx=0x7fe604041da0 comp rx=0 tx=0).stop 2024-09-21T00:33:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.346+0000 7fe626052740 1 -- 172.21.15.27:0/4239387263 shutdown_connections 2024-09-21T00:33:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.346+0000 7fe626052740 1 --2- 172.21.15.27:0/4239387263 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x557e315c47c0 0x557e315c6be0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.346+0000 7fe626052740 1 --2- 172.21.15.27:0/4239387263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e3112b1c0 0x557e3154de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.346+0000 7fe626052740 1 --2- 172.21.15.27:0/4239387263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e311971b0 0x557e3154d720 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.346+0000 7fe626052740 1 --2- 172.21.15.27:0/4239387263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e3116e2f0 0x557e31550260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.346+0000 7fe626052740 1 -- 172.21.15.27:0/4239387263 >> 172.21.15.27:0/4239387263 conn(0x557e3118b300 msgr2=0x557e3153ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.346+0000 7fe602000640 0 client.26563 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.347+0000 7fe626052740 1 -- 172.21.15.27:0/4239387263 shutdown_connections 2024-09-21T00:33:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.347+0000 7fe626052740 1 -- 172.21.15.27:0/4239387263 wait complete. 2024-09-21T00:33:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.348+0000 7fe626052740 20 client.26563 trim_cache size 0 max 16384 2024-09-21T00:33:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:33:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:33:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:33:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:33:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.350+0000 7fe61da00640 1 -- 172.21.15.27:0/1087250240 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe5fc0b66f0 msgr2=0x7fe5fc0b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.350+0000 7fe61da00640 1 --2- 172.21.15.27:0/1087250240 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe5fc0b66f0 0x7fe5fc0b8c00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe604035040 tx=0x7fe604030340 comp rx=0 tx=0).stop 2024-09-21T00:33:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.350+0000 7fe61da00640 1 -- 172.21.15.27:0/1087250240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61810bda0 msgr2=0x7fe6181a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.350+0000 7fe61da00640 1 --2- 172.21.15.27:0/1087250240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61810bda0 0x7fe6181a1d10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe61000eaa0 tx=0x7fe61000ef60 comp rx=0 tx=0).stop 2024-09-21T00:33:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.350+0000 7fe61da00640 1 -- 172.21.15.27:0/1087250240 shutdown_connections 2024-09-21T00:33:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.350+0000 7fe61da00640 1 --2- 172.21.15.27:0/1087250240 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe5fc0b66f0 0x7fe5fc0b8c00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.350+0000 7fe61da00640 1 --2- 172.21.15.27:0/1087250240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61810bda0 0x7fe6181a1d10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.350+0000 7fe61da00640 1 --2- 172.21.15.27:0/1087250240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe61810b450 0x7fe61819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.350+0000 7fe61da00640 1 --2- 172.21.15.27:0/1087250240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe618105b20 0x7fe61819c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.350+0000 7fe61da00640 1 -- 172.21.15.27:0/1087250240 >> 172.21.15.27:0/1087250240 conn(0x7fe618101560 msgr2=0x7fe618103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.350+0000 7fe61da00640 1 -- 172.21.15.27:0/1087250240 shutdown_connections 2024-09-21T00:33:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:08.351+0000 7fe61da00640 1 -- 172.21.15.27:0/1087250240 wait complete. 2024-09-21T00:33:09.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:08 smithi027 ceph-mon[30745]: osdmap e107: 12 total, 12 up, 12 in 2024-09-21T00:33:09.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:08 smithi089 ceph-mon[34481]: osdmap e107: 12 total, 12 up, 12 in 2024-09-21T00:33:09.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:08 smithi098 ceph-mon[34725]: osdmap e107: 12 total, 12 up, 12 in 2024-09-21T00:33:10.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:09 smithi027 ceph-mon[30745]: pgmap v747: 129 pgs: 21 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 102 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 405 op/s 2024-09-21T00:33:10.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:09 smithi089 ceph-mon[34481]: pgmap v747: 129 pgs: 21 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 102 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 405 op/s 2024-09-21T00:33:10.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:09 smithi098 ceph-mon[34725]: pgmap v747: 129 pgs: 21 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 102 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 405 op/s 2024-09-21T00:33:12.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:11 smithi027 ceph-mon[30745]: pgmap v748: 129 pgs: 56 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 55 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 440 op/s 2024-09-21T00:33:12.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:11 smithi089 ceph-mon[34481]: pgmap v748: 129 pgs: 56 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 55 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 440 op/s 2024-09-21T00:33:12.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:11 smithi098 ceph-mon[34725]: pgmap v748: 129 pgs: 56 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 55 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 440 op/s 2024-09-21T00:33:14.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:13 smithi027 ceph-mon[30745]: pgmap v749: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 475 op/s 2024-09-21T00:33:14.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:13 smithi089 ceph-mon[34481]: pgmap v749: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 475 op/s 2024-09-21T00:33:14.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:13 smithi098 ceph-mon[34725]: pgmap v749: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 475 op/s 2024-09-21T00:33:16.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:15 smithi027 ceph-mon[30745]: pgmap v750: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 482 op/s 2024-09-21T00:33:16.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:15 smithi089 ceph-mon[34481]: pgmap v750: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 482 op/s 2024-09-21T00:33:16.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:15 smithi098 ceph-mon[34725]: pgmap v750: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 482 op/s 2024-09-21T00:33:18.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:17 smithi027 ceph-mon[30745]: pgmap v751: 129 pgs: 67 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 39 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 480 op/s 2024-09-21T00:33:18.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:17 smithi089 ceph-mon[34481]: pgmap v751: 129 pgs: 67 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 39 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 480 op/s 2024-09-21T00:33:18.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:17 smithi098 ceph-mon[34725]: pgmap v751: 129 pgs: 67 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 39 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 480 op/s 2024-09-21T00:33:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:19 smithi089 ceph-mon[34481]: pgmap v752: 129 pgs: 67 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 39 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 435 op/s 2024-09-21T00:33:20.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:19 smithi098 ceph-mon[34725]: pgmap v752: 129 pgs: 67 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 39 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 435 op/s 2024-09-21T00:33:20.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:19 smithi027 ceph-mon[30745]: pgmap v752: 129 pgs: 67 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 39 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 435 op/s 2024-09-21T00:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.681+0000 7f4800200640 1 -- 172.21.15.27:0/3618135167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f810a4a0 msgr2=0x7f47f810a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.681+0000 7f4800200640 1 --2- 172.21.15.27:0/3618135167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f810a4a0 0x7f47f810a8b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009900 tx=0x7f47ec02a410 comp rx=0 tx=0).stop 2024-09-21T00:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.681+0000 7f4800200640 1 -- 172.21.15.27:0/3618135167 shutdown_connections 2024-09-21T00:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.681+0000 7f4800200640 1 --2- 172.21.15.27:0/3618135167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f810d140 0x7f47f8116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.681+0000 7f4800200640 1 --2- 172.21.15.27:0/3618135167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47f810c7f0 0x7f47f810cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.681+0000 7f4800200640 1 --2- 172.21.15.27:0/3618135167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f810a4a0 0x7f47f810a8b0 secure :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009900 tx=0x7f47ec02a410 comp rx=0 tx=0).stop 2024-09-21T00:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.681+0000 7f4800200640 1 -- 172.21.15.27:0/3618135167 >> 172.21.15.27:0/3618135167 conn(0x7f47f8079520 msgr2=0x7f47f807b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.682+0000 7f4800200640 1 -- 172.21.15.27:0/3618135167 shutdown_connections 2024-09-21T00:33:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.682+0000 7f4800200640 1 -- 172.21.15.27:0/3618135167 wait complete. 2024-09-21T00:33:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.683+0000 7f4800200640 1 Processor -- start 2024-09-21T00:33:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.684+0000 7f4800200640 1 -- start start 2024-09-21T00:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.685+0000 7f4800200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f810c7f0 0x7f47f81a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.685+0000 7f4800200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47f810d140 0x7f47f81a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.685+0000 7f4800200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f81a6450 0x7f47f81a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.685+0000 7f4800200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47f811a140 con 0x7f47f810c7f0 2024-09-21T00:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.685+0000 7f4800200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f47f8119fc0 con 0x7f47f81a6450 2024-09-21T00:33:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.685+0000 7f4800200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f47f811a2c0 con 0x7f47f810d140 2024-09-21T00:33:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.685+0000 7f47fce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47f810d140 0x7f47f81a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.686+0000 7f47fe200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f81a6450 0x7f47f81a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.686+0000 7f47fce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47f810d140 0x7f47f81a1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55306/0 (socket says 172.21.15.27:55306) 2024-09-21T00:33:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.686+0000 7f47fce00640 1 -- 172.21.15.27:0/1157389507 learned_addr learned my addr 172.21.15.27:0/1157389507 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.686+0000 7f47fd800640 1 --2- 172.21.15.27:0/1157389507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f810c7f0 0x7f47f81a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.686+0000 7f47fce00640 1 -- 172.21.15.27:0/1157389507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f81a6450 msgr2=0x7f47f81a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.686+0000 7f47fce00640 1 --2- 172.21.15.27:0/1157389507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f81a6450 0x7f47f81a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.686+0000 7f47fce00640 1 -- 172.21.15.27:0/1157389507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f810c7f0 msgr2=0x7f47f81a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.686+0000 7f47fce00640 1 --2- 172.21.15.27:0/1157389507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f810c7f0 0x7f47f81a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.686+0000 7f47fce00640 1 -- 172.21.15.27:0/1157389507 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f81a7030 con 0x7f47f810d140 2024-09-21T00:33:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.686+0000 7f47fd800640 1 --2- 172.21.15.27:0/1157389507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f810c7f0 0x7f47f81a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:33:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.688+0000 7f47fce00640 1 --2- 172.21.15.27:0/1157389507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47f810d140 0x7f47f81a1280 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f47e400b450 tx=0x7f47e400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.688+0000 7f47f6000640 1 -- 172.21.15.27:0/1157389507 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4015020 con 0x7f47f810d140 2024-09-21T00:33:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.688+0000 7f47f6000640 1 -- 172.21.15.27:0/1157389507 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47e4011d40 con 0x7f47f810d140 2024-09-21T00:33:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.688+0000 7f4800200640 1 -- 172.21.15.27:0/1157389507 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f81ad5d0 con 0x7f47f810d140 2024-09-21T00:33:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.688+0000 7f4800200640 1 -- 172.21.15.27:0/1157389507 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f81ada60 con 0x7f47f810d140 2024-09-21T00:33:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.688+0000 7f47f6000640 1 -- 172.21.15.27:0/1157389507 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e400f340 con 0x7f47f810d140 2024-09-21T00:33:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.693+0000 7f48071ea740 1 Processor -- start 2024-09-21T00:33:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.694+0000 7f48071ea740 1 -- start start 2024-09-21T00:33:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.694+0000 7f48071ea740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd89c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.694+0000 7f48071ea740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd8a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.694+0000 7f48071ea740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9c44d0 0x561fefd8a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.694+0000 7f48071ea740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561fefcff6b0 con 0x561fef9e51b0 2024-09-21T00:33:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.694+0000 7f48071ea740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561fefceeae0 con 0x561fef9bc2f0 2024-09-21T00:33:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.695+0000 7f48071ea740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561fefcf28f0 con 0x561fef9c44d0 2024-09-21T00:33:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.695+0000 7f47fd800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd8a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.695+0000 7f47fd800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd8a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54278/0 (socket says 172.21.15.27:54278) 2024-09-21T00:33:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.695+0000 7f47fd800640 1 -- 172.21.15.27:0/67244519 learned_addr learned my addr 172.21.15.27:0/67244519 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.695+0000 7f47fe200640 1 --2- 172.21.15.27:0/67244519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd89c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.695+0000 7f47f6000640 1 -- 172.21.15.27:0/1157389507 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f47e4018020 con 0x7f47f810d140 2024-09-21T00:33:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.695+0000 7f47fce00640 1 --2- 172.21.15.27:0/67244519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9c44d0 0x561fefd8a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.696+0000 7f47fd800640 1 -- 172.21.15.27:0/67244519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9c44d0 msgr2=0x561fefd8a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.696+0000 7f47fd800640 1 --2- 172.21.15.27:0/67244519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9c44d0 0x561fefd8a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.696+0000 7f47fd800640 1 -- 172.21.15.27:0/67244519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 msgr2=0x561fefd89c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.696+0000 7f47fd800640 1 --2- 172.21.15.27:0/67244519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd89c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.696+0000 7f47fd800640 1 -- 172.21.15.27:0/67244519 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fefcbb780 con 0x561fef9bc2f0 2024-09-21T00:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.696+0000 7f47fce00640 1 --2- 172.21.15.27:0/67244519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9c44d0 0x561fefd8a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.696+0000 7f47fe200640 1 --2- 172.21.15.27:0/67244519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd89c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.697+0000 7f47fd800640 1 --2- 172.21.15.27:0/67244519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd8a170 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009900 tx=0x7f47ec004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a23b3eea3654a13d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.698+0000 7f47eaa00640 1 -- 172.21.15.27:0/67244519 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec03f070 con 0x561fef9bc2f0 2024-09-21T00:33:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.698+0000 7f47eaa00640 1 -- 172.21.15.27:0/67244519 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ec005de0 con 0x561fef9bc2f0 2024-09-21T00:33:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.698+0000 7f47f6000640 1 --2- 172.21.15.27:0/1157389507 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f47e00b6680 0x7f47e00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.698+0000 7f47eaa00640 1 -- 172.21.15.27:0/67244519 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec035070 con 0x561fef9bc2f0 2024-09-21T00:33:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.698+0000 7f47f6000640 1 -- 172.21.15.27:0/1157389507 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9607+0+0 (secure 0 0 0) 0x7f47e40e7610 con 0x7f47f810d140 2024-09-21T00:33:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.699+0000 7f48071ea740 1 -- 172.21.15.27:0/67244519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 msgr2=0x561fefd8a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.699+0000 7f48071ea740 1 --2- 172.21.15.27:0/67244519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd8a170 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009900 tx=0x7f47ec004390 comp rx=0 tx=0).stop 2024-09-21T00:33:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.701+0000 7f47fe200640 1 --2- 172.21.15.27:0/1157389507 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f47e00b6680 0x7f47e00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.704+0000 7f48071ea740 1 -- 172.21.15.27:0/67244519 shutdown_connections 2024-09-21T00:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.704+0000 7f48071ea740 1 --2- 172.21.15.27:0/67244519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9c44d0 0x561fefd8a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.704+0000 7f48071ea740 1 --2- 172.21.15.27:0/67244519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd8a170 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.704+0000 7f48071ea740 1 --2- 172.21.15.27:0/67244519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd89c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.704+0000 7f48071ea740 1 -- 172.21.15.27:0/67244519 >> 172.21.15.27:0/67244519 conn(0x561fef9d9300 msgr2=0x561fefd87b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.704+0000 7f47fe200640 1 --2- 172.21.15.27:0/1157389507 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f47e00b6680 0x7f47e00b8b90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f47f000b6a0 tx=0x7f47f000e040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.705+0000 7f48071ea740 1 -- 172.21.15.27:0/67244519 shutdown_connections 2024-09-21T00:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.705+0000 7f48071ea740 1 -- 172.21.15.27:0/67244519 wait complete. 2024-09-21T00:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.705+0000 7f48071ea740 1 -- start start 2024-09-21T00:33:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.706+0000 7f48071ea740 1 Processor -- start 2024-09-21T00:33:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.707+0000 7f48071ea740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9bc2f0 0x561fefd952b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.707+0000 7f48071ea740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9c44d0 0x561fefd957f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.707+0000 7f48071ea740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd95d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.707+0000 7f47fce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9c44d0 0x561fefd957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.707+0000 7f47fd800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9bc2f0 0x561fefd952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.707+0000 7f47fce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9c44d0 0x561fefd957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54282/0 (socket says 172.21.15.27:54282) 2024-09-21T00:33:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.707+0000 7f47fd800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9bc2f0 0x561fefd952b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55332/0 (socket says 172.21.15.27:55332) 2024-09-21T00:33:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.708+0000 7f47fce00640 1 -- 172.21.15.27:0/1431981663 learned_addr learned my addr 172.21.15.27:0/1431981663 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.708+0000 7f48071ea740 1 -- 172.21.15.27:0/1431981663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561fefc26ab0 con 0x561fef9e51b0 2024-09-21T00:33:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.708+0000 7f48071ea740 1 -- 172.21.15.27:0/1431981663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561fefc95570 con 0x561fef9c44d0 2024-09-21T00:33:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.708+0000 7f48071ea740 1 -- 172.21.15.27:0/1431981663 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561fefd98350 con 0x561fef9bc2f0 2024-09-21T00:33:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.708+0000 7f47fe200640 1 --2- 172.21.15.27:0/1431981663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd95d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.708+0000 7f48071ea740 10 client.0 _ll_register_callbacks cb 0x561fefca4100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.708+0000 7f47fce00640 1 -- 172.21.15.27:0/1431981663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9bc2f0 msgr2=0x561fefd952b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.708+0000 7f47fce00640 1 --2- 172.21.15.27:0/1431981663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9bc2f0 0x561fefd952b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.708+0000 7f47fce00640 1 -- 172.21.15.27:0/1431981663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 msgr2=0x561fefd95d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.708+0000 7f47fce00640 1 --2- 172.21.15.27:0/1431981663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd95d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.708+0000 7f47fce00640 1 -- 172.21.15.27:0/1431981663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fefa50790 con 0x561fef9c44d0 2024-09-21T00:33:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.708+0000 7f47fe200640 1 --2- 172.21.15.27:0/1431981663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd95d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:33:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.709+0000 7f47fce00640 1 --2- 172.21.15.27:0/1431981663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9c44d0 0x561fefd957f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f47e40aa290 tx=0x7f47e4010730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fb63f778d293348 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.709+0000 7f47df400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:33:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.709+0000 7f47df400640 1 -- 172.21.15.27:0/1431981663 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e40ac790 con 0x561fef9c44d0 2024-09-21T00:33:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.710+0000 7f47df400640 1 -- 172.21.15.27:0/1431981663 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9607+0+0 (secure 0 0 0) 0x7f47e40a9290 con 0x561fef9c44d0 2024-09-21T00:33:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.710+0000 7f48071ea740 1 -- 172.21.15.27:0/1431981663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561fefd01600 con 0x561fef9c44d0 2024-09-21T00:33:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.710+0000 7f47df400640 1 -- 172.21.15.27:0/1431981663 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f47e40acb30 con 0x561fef9c44d0 2024-09-21T00:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.710+0000 7f48071ea740 10 client.26564 fetch_fsmap learned FSMap version 47 2024-09-21T00:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.710+0000 7f48071ea740 1 -- 172.21.15.27:0/1431981663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561fefac9620 con 0x561fef9c44d0 2024-09-21T00:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.711+0000 7f47df400640 1 -- 172.21.15.27:0/1431981663 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561fefd01600 con 0x561fef9c44d0 2024-09-21T00:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.711+0000 7f47df400640 1 -- 172.21.15.27:0/1431981663 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f47e40e7290 con 0x561fef9c44d0 2024-09-21T00:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.712+0000 7f48071ea740 10 client.26564 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.712+0000 7f48071ea740 10 client.26564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.712+0000 7f48071ea740 20 client.26564 populate_metadata read hostname 'smithi027' 2024-09-21T00:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.712+0000 7f48071ea740 1 --2- 172.21.15.27:0/1431981663 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fef9a3d80 0x561fefda7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.712+0000 7f48071ea740 4 client.26564 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.712+0000 7f48071ea740 1 -- 172.21.15.27:0/1431981663 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fefcfd0b0 con 0x561fef9a3d80 2024-09-21T00:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.713+0000 7f47fd800640 1 --2- 172.21.15.27:0/1431981663 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fef9a3d80 0x561fefda7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.716+0000 7f47df400640 10 client.26564 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.716+0000 7f47fd800640 1 --2- 172.21.15.27:0/1431981663 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fef9a3d80 0x561fefda7330 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7906e712043ec597 server_cookie=f028f872e2fe72b9 in_seq=0 out_seq=0 2024-09-21T00:33:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.716+0000 7f47df400640 1 -- 172.21.15.27:0/1431981663 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561fefcfd0b0 con 0x561fef9a3d80 2024-09-21T00:33:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.716+0000 7f47df400640 10 client.26564 handle_command_reply: tid=0 2024-09-21T00:33:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.717+0000 7f48071ea740 1 client.26564 shutdown 2024-09-21T00:33:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.720+0000 7f48071ea740 1 -- 172.21.15.27:0/1431981663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9c44d0 msgr2=0x561fefd957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.720+0000 7f48071ea740 1 --2- 172.21.15.27:0/1431981663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9c44d0 0x561fefd957f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f47e40aa290 tx=0x7f47e4010730 comp rx=0 tx=0).stop 2024-09-21T00:33:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.720+0000 7f48071ea740 1 -- 172.21.15.27:0/1431981663 shutdown_connections 2024-09-21T00:33:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.720+0000 7f48071ea740 1 --2- 172.21.15.27:0/1431981663 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fef9a3d80 0x561fefda7330 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.720+0000 7f48071ea740 1 --2- 172.21.15.27:0/1431981663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd95d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.720+0000 7f48071ea740 1 --2- 172.21.15.27:0/1431981663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9c44d0 0x561fefd957f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.720+0000 7f48071ea740 1 --2- 172.21.15.27:0/1431981663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9bc2f0 0x561fefd952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.720+0000 7f48071ea740 1 -- 172.21.15.27:0/1431981663 >> 172.21.15.27:0/1431981663 conn(0x561fef9d9300 msgr2=0x561fefd8ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.720+0000 7f47df400640 0 client.26564 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.721+0000 7f48071ea740 1 -- 172.21.15.27:0/1431981663 shutdown_connections 2024-09-21T00:33:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.721+0000 7f48071ea740 1 -- 172.21.15.27:0/1431981663 wait complete. 2024-09-21T00:33:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.721+0000 7f48071ea740 20 client.26564 trim_cache size 0 max 16384 2024-09-21T00:33:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.758+0000 7f48071ea740 1 Processor -- start 2024-09-21T00:33:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.759+0000 7f48071ea740 1 -- start start 2024-09-21T00:33:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.759+0000 7f48071ea740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd86aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.759+0000 7f48071ea740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9c44d0 0x561fefd96ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.759+0000 7f48071ea740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9e51b0 0x561fefdacaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.759+0000 7f48071ea740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47f81ba830 con 0x561fef9c44d0 2024-09-21T00:33:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.759+0000 7f48071ea740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f47f81b8040 con 0x561fef9bc2f0 2024-09-21T00:33:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.759+0000 7f48071ea740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f47f81b5eb0 con 0x561fef9e51b0 2024-09-21T00:33:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.760+0000 7f47fe200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9e51b0 0x561fefdacaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.760+0000 7f47fd800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd86aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.760+0000 7f47fd800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd86aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54284/0 (socket says 172.21.15.27:54284) 2024-09-21T00:33:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.760+0000 7f47fd800640 1 -- 172.21.15.27:0/2056339538 learned_addr learned my addr 172.21.15.27:0/2056339538 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.761+0000 7f47fce00640 1 --2- 172.21.15.27:0/2056339538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9c44d0 0x561fefd96ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.762+0000 7f47fd800640 1 -- 172.21.15.27:0/2056339538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9e51b0 msgr2=0x561fefdacaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.762+0000 7f47fd800640 1 --2- 172.21.15.27:0/2056339538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9e51b0 0x561fefdacaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.762+0000 7f47fd800640 1 -- 172.21.15.27:0/2056339538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9c44d0 msgr2=0x561fefd96ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.762+0000 7f47fd800640 1 --2- 172.21.15.27:0/2056339538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9c44d0 0x561fefd96ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.762+0000 7f47fd800640 1 -- 172.21.15.27:0/2056339538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fefd8bf60 con 0x561fef9bc2f0 2024-09-21T00:33:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.765+0000 7f47fd800640 1 --2- 172.21.15.27:0/2056339538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd86aa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x561fefd888b0 tx=0x7f47ec02aa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7126fabd2794edf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.768+0000 7f47ea000640 1 -- 172.21.15.27:0/2056339538 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec000e60 con 0x561fef9bc2f0 2024-09-21T00:33:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.768+0000 7f47ea000640 1 -- 172.21.15.27:0/2056339538 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ec005020 con 0x561fef9bc2f0 2024-09-21T00:33:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.768+0000 7f47ea000640 1 -- 172.21.15.27:0/2056339538 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec035040 con 0x561fef9bc2f0 2024-09-21T00:33:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.769+0000 7f48071ea740 1 -- 172.21.15.27:0/2056339538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 msgr2=0x561fefd86aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.769+0000 7f48071ea740 1 --2- 172.21.15.27:0/2056339538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd86aa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x561fefd888b0 tx=0x7f47ec02aa90 comp rx=0 tx=0).stop 2024-09-21T00:33:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.770+0000 7f48071ea740 1 -- 172.21.15.27:0/2056339538 shutdown_connections 2024-09-21T00:33:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.770+0000 7f48071ea740 1 --2- 172.21.15.27:0/2056339538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9e51b0 0x561fefdacaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.771+0000 7f48071ea740 1 --2- 172.21.15.27:0/2056339538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9c44d0 0x561fefd96ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.771+0000 7f48071ea740 1 --2- 172.21.15.27:0/2056339538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd86aa0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.771+0000 7f48071ea740 1 -- 172.21.15.27:0/2056339538 >> 172.21.15.27:0/2056339538 conn(0x561fef9d9300 msgr2=0x561fefd967d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.771+0000 7f48071ea740 1 -- 172.21.15.27:0/2056339538 shutdown_connections 2024-09-21T00:33:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.771+0000 7f48071ea740 1 -- 172.21.15.27:0/2056339538 wait complete. 2024-09-21T00:33:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.772+0000 7f48071ea740 1 -- start start 2024-09-21T00:33:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.773+0000 7f48071ea740 1 Processor -- start 2024-09-21T00:33:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.774+0000 7f48071ea740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd9dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.775+0000 7f48071ea740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9c44d0 0x561fefd9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.775+0000 7f48071ea740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd9bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.775+0000 7f47fd800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd9dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.776+0000 7f47fd800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd9dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54296/0 (socket says 172.21.15.27:54296) 2024-09-21T00:33:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.776+0000 7f48071ea740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47f81b6500 con 0x561fef9e51b0 2024-09-21T00:33:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.776+0000 7f48071ea740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f47f81b7970 con 0x561fef9bc2f0 2024-09-21T00:33:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.776+0000 7f48071ea740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f47f81b6bd0 con 0x561fef9c44d0 2024-09-21T00:33:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.776+0000 7f48071ea740 10 client.0 _ll_register_callbacks cb 0x7f47e40107d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:33:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.777+0000 7f47fe200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd9bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.777+0000 7f47fe200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd9bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43800/0 (socket says 172.21.15.27:43800) 2024-09-21T00:33:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.777+0000 7f47fe200640 1 -- 172.21.15.27:0/1758350900 learned_addr learned my addr 172.21.15.27:0/1758350900 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.780+0000 7f47fce00640 1 --2- 172.21.15.27:0/1758350900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9c44d0 0x561fefd9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.780+0000 7f47fd800640 1 -- 172.21.15.27:0/1758350900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9c44d0 msgr2=0x561fefd9b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.780+0000 7f47fd800640 1 --2- 172.21.15.27:0/1758350900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9c44d0 0x561fefd9b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.780+0000 7f47fd800640 1 -- 172.21.15.27:0/1758350900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 msgr2=0x561fefd9bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.780+0000 7f47fd800640 1 --2- 172.21.15.27:0/1758350900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd9bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.781+0000 7f47fd800640 1 -- 172.21.15.27:0/1758350900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fefe10d00 con 0x561fef9bc2f0 2024-09-21T00:33:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.781+0000 7f47fe200640 1 --2- 172.21.15.27:0/1758350900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd9bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:33:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.781+0000 7f47fd800640 1 --2- 172.21.15.27:0/1758350900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd9dfd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f47ec052040 tx=0x7f47ec0315c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=129bae8b8b522b90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.781+0000 7f47df400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:33:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.781+0000 7f47df400640 1 -- 172.21.15.27:0/1758350900 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec035070 con 0x561fef9bc2f0 2024-09-21T00:33:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.781+0000 7f47df400640 1 -- 172.21.15.27:0/1758350900 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9607+0+0 (secure 0 0 0) 0x7f47ec056040 con 0x561fef9bc2f0 2024-09-21T00:33:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.782+0000 7f48071ea740 1 -- 172.21.15.27:0/1758350900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47ec005020 con 0x561fef9bc2f0 2024-09-21T00:33:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.782+0000 7f47df400640 1 -- 172.21.15.27:0/1758350900 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f47ec03f040 con 0x561fef9bc2f0 2024-09-21T00:33:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.782+0000 7f48071ea740 10 client.26570 fetch_fsmap learned FSMap version 47 2024-09-21T00:33:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.782+0000 7f48071ea740 1 -- 172.21.15.27:0/1758350900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561fefe11e40 con 0x561fef9bc2f0 2024-09-21T00:33:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.783+0000 7f47df400640 1 -- 172.21.15.27:0/1758350900 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ec005020 con 0x561fef9bc2f0 2024-09-21T00:33:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.783+0000 7f47df400640 1 -- 172.21.15.27:0/1758350900 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f47ec0049d0 con 0x561fef9bc2f0 2024-09-21T00:33:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.783+0000 7f48071ea740 10 client.26570 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:33:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.783+0000 7f48071ea740 10 client.26570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:33:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.783+0000 7f48071ea740 20 client.26570 populate_metadata read hostname 'smithi027' 2024-09-21T00:33:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.783+0000 7f48071ea740 1 --2- 172.21.15.27:0/1758350900 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fef9791c0 0x561fefe14090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.783+0000 7f48071ea740 4 client.26570 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:33:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.783+0000 7f48071ea740 1 -- 172.21.15.27:0/1758350900 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561fefcfa4e0 con 0x561fef9791c0 2024-09-21T00:33:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.784+0000 7f47fce00640 1 --2- 172.21.15.27:0/1758350900 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fef9791c0 0x561fefe14090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.784+0000 7f47df400640 10 client.26570 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.787+0000 7f47fce00640 1 --2- 172.21.15.27:0/1758350900 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fef9791c0 0x561fefe14090 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2db39f1d2058878f server_cookie=98dc746b7f4f11ea in_seq=0 out_seq=0 2024-09-21T00:33:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.998+0000 7f47df400640 1 -- 172.21.15.27:0/1758350900 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561fefcfa4e0 con 0x561fef9791c0 2024-09-21T00:33:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.998+0000 7f47df400640 10 client.26570 handle_command_reply: tid=0 2024-09-21T00:33:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.998+0000 7f48071ea740 1 client.26570 shutdown 2024-09-21T00:33:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.998+0000 7f48071ea740 1 -- 172.21.15.27:0/1758350900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 msgr2=0x561fefd9dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.998+0000 7f48071ea740 1 --2- 172.21.15.27:0/1758350900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd9dfd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f47ec052040 tx=0x7f47ec0315c0 comp rx=0 tx=0).stop 2024-09-21T00:33:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.999+0000 7f48071ea740 1 -- 172.21.15.27:0/1758350900 shutdown_connections 2024-09-21T00:33:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.999+0000 7f48071ea740 1 --2- 172.21.15.27:0/1758350900 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561fef9791c0 0x561fefe14090 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.999+0000 7f48071ea740 1 --2- 172.21.15.27:0/1758350900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561fef9e51b0 0x561fefd9bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.999+0000 7f48071ea740 1 --2- 172.21.15.27:0/1758350900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fef9c44d0 0x561fefd9b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.999+0000 7f47df400640 0 client.26570 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.999+0000 7f48071ea740 1 --2- 172.21.15.27:0/1758350900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fef9bc2f0 0x561fefd9dfd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.999+0000 7f48071ea740 1 -- 172.21.15.27:0/1758350900 >> 172.21.15.27:0/1758350900 conn(0x561fef9d9300 msgr2=0x561fefd86870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.999+0000 7f48071ea740 1 -- 172.21.15.27:0/1758350900 shutdown_connections 2024-09-21T00:33:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:21.999+0000 7f48071ea740 1 -- 172.21.15.27:0/1758350900 wait complete. 2024-09-21T00:33:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:22.000+0000 7f48071ea740 20 client.26570 trim_cache size 0 max 16384 2024-09-21T00:33:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:33:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:33:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:33:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:33:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:22.001+0000 7f4800200640 1 -- 172.21.15.27:0/1157389507 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f47e00b6680 msgr2=0x7f47e00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:22.001+0000 7f4800200640 1 --2- 172.21.15.27:0/1157389507 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f47e00b6680 0x7f47e00b8b90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f47f000b6a0 tx=0x7f47f000e040 comp rx=0 tx=0).stop 2024-09-21T00:33:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:22.001+0000 7f4800200640 1 -- 172.21.15.27:0/1157389507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47f810d140 msgr2=0x7f47f81a1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:22.001+0000 7f4800200640 1 --2- 172.21.15.27:0/1157389507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47f810d140 0x7f47f81a1280 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f47e400b450 tx=0x7f47e400b910 comp rx=0 tx=0).stop 2024-09-21T00:33:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:22.001+0000 7f4800200640 1 -- 172.21.15.27:0/1157389507 shutdown_connections 2024-09-21T00:33:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:22.001+0000 7f4800200640 1 --2- 172.21.15.27:0/1157389507 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f47e00b6680 0x7f47e00b8b90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:22.001+0000 7f4800200640 1 --2- 172.21.15.27:0/1157389507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f81a6450 0x7f47f81a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:22.001+0000 7f4800200640 1 --2- 172.21.15.27:0/1157389507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47f810d140 0x7f47f81a1280 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:22.001+0000 7f4800200640 1 --2- 172.21.15.27:0/1157389507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f810c7f0 0x7f47f81a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:22.001+0000 7f4800200640 1 -- 172.21.15.27:0/1157389507 >> 172.21.15.27:0/1157389507 conn(0x7f47f8079520 msgr2=0x7f47f810f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:22.002+0000 7f4800200640 1 -- 172.21.15.27:0/1157389507 shutdown_connections 2024-09-21T00:33:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:22.002+0000 7f4800200640 1 -- 172.21.15.27:0/1157389507 wait complete. 2024-09-21T00:33:22.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:22 smithi089 ceph-mon[34481]: pgmap v753: 129 pgs: 57 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 51 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 375 op/s 2024-09-21T00:33:22.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:22 smithi098 ceph-mon[34725]: pgmap v753: 129 pgs: 57 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 51 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 375 op/s 2024-09-21T00:33:22.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:22 smithi027 ceph-mon[30745]: pgmap v753: 129 pgs: 57 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 51 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 375 op/s 2024-09-21T00:33:24.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:24 smithi089 ceph-mon[34481]: pgmap v754: 129 pgs: 57 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 52 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-09-21T00:33:24.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:24 smithi098 ceph-mon[34725]: pgmap v754: 129 pgs: 57 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 52 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-09-21T00:33:24.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:24 smithi027 ceph-mon[30745]: pgmap v754: 129 pgs: 57 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 52 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-09-21T00:33:26.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:26 smithi089 ceph-mon[34481]: pgmap v755: 129 pgs: 56 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 55 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 260 op/s 2024-09-21T00:33:26.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:26 smithi098 ceph-mon[34725]: pgmap v755: 129 pgs: 56 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 55 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 260 op/s 2024-09-21T00:33:26.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:26 smithi027 ceph-mon[30745]: pgmap v755: 129 pgs: 56 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 55 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 260 op/s 2024-09-21T00:33:28.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:28 smithi089 ceph-mon[34481]: pgmap v756: 129 pgs: 56 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 56 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 771 KiB/s wr, 216 op/s 2024-09-21T00:33:28.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:28 smithi098 ceph-mon[34725]: pgmap v756: 129 pgs: 56 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 56 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 771 KiB/s wr, 216 op/s 2024-09-21T00:33:28.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:28 smithi027 ceph-mon[30745]: pgmap v756: 129 pgs: 56 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 56 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 771 KiB/s wr, 216 op/s 2024-09-21T00:33:30.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:30 smithi089 ceph-mon[34481]: pgmap v757: 129 pgs: 48 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 65 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 777 KiB/s wr, 222 op/s 2024-09-21T00:33:30.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:30 smithi098 ceph-mon[34725]: pgmap v757: 129 pgs: 48 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 65 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 777 KiB/s wr, 222 op/s 2024-09-21T00:33:30.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:30 smithi027 ceph-mon[30745]: pgmap v757: 129 pgs: 48 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 65 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 777 KiB/s wr, 222 op/s 2024-09-21T00:33:32.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:32 smithi089 ceph-mon[34481]: pgmap v758: 129 pgs: 48 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 66 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 228 op/s 2024-09-21T00:33:32.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:32 smithi098 ceph-mon[34725]: pgmap v758: 129 pgs: 48 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 66 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 228 op/s 2024-09-21T00:33:32.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:32 smithi027 ceph-mon[30745]: pgmap v758: 129 pgs: 48 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 66 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 228 op/s 2024-09-21T00:33:34.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:34 smithi089 ceph-mon[34481]: pgmap v759: 129 pgs: 48 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 66 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 763 KiB/s wr, 203 op/s 2024-09-21T00:33:34.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:34 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:33:34.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:34 smithi098 ceph-mon[34725]: pgmap v759: 129 pgs: 48 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 66 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 763 KiB/s wr, 203 op/s 2024-09-21T00:33:34.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:34 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:33:34.834 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:34 smithi027 ceph-mon[30745]: pgmap v759: 129 pgs: 48 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 66 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 763 KiB/s wr, 203 op/s 2024-09-21T00:33:34.834 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:34 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:33:36.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:36 smithi089 ceph-mon[34481]: pgmap v760: 129 pgs: 42 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 73 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 235 op/s 2024-09-21T00:33:36.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:36 smithi098 ceph-mon[34725]: pgmap v760: 129 pgs: 42 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 73 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 235 op/s 2024-09-21T00:33:36.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:36 smithi027 ceph-mon[30745]: pgmap v760: 129 pgs: 42 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 73 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 235 op/s 2024-09-21T00:33:38.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:38 smithi089 ceph-mon[34481]: pgmap v761: 129 pgs: 26 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 92 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-09-21T00:33:38.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:38 smithi098 ceph-mon[34725]: pgmap v761: 129 pgs: 26 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 92 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-09-21T00:33:39.641 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:38 smithi027 ceph-mon[30745]: pgmap v761: 129 pgs: 26 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 92 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-09-21T00:33:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.938+0000 7fb471600640 1 -- 172.21.15.27:0/441591765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c102e80 msgr2=0x7fb46c1073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.938+0000 7fb471600640 1 --2- 172.21.15.27:0/441591765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c102e80 0x7fb46c1073a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb458009900 tx=0x7fb45802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.938+0000 7fb471600640 1 -- 172.21.15.27:0/441591765 shutdown_connections 2024-09-21T00:33:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.938+0000 7fb471600640 1 --2- 172.21.15.27:0/441591765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb46c1079f0 0x7fb46c107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.938+0000 7fb471600640 1 --2- 172.21.15.27:0/441591765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c102e80 0x7fb46c1073a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.938+0000 7fb471600640 1 --2- 172.21.15.27:0/441591765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb46c102530 0x7fb46c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.938+0000 7fb471600640 1 -- 172.21.15.27:0/441591765 >> 172.21.15.27:0/441591765 conn(0x7fb46c0fe210 msgr2=0x7fb46c100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.939+0000 7fb471600640 1 -- 172.21.15.27:0/441591765 shutdown_connections 2024-09-21T00:33:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.939+0000 7fb471600640 1 -- 172.21.15.27:0/441591765 wait complete. 2024-09-21T00:33:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.941+0000 7fb471600640 1 Processor -- start 2024-09-21T00:33:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.942+0000 7fb471600640 1 -- start start 2024-09-21T00:33:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.943+0000 7fb471600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb46c102530 0x7fb46c1a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.943+0000 7fb471600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb46c102e80 0x7fb46c1a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.943+0000 7fb471600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c1079f0 0x7fb46c1a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.944+0000 7fb471600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c119e30 con 0x7fb46c102530 2024-09-21T00:33:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.944+0000 7fb471600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c119cb0 con 0x7fb46c1079f0 2024-09-21T00:33:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.944+0000 7fb471600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c119fb0 con 0x7fb46c102e80 2024-09-21T00:33:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.944+0000 7fb46b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb46c102e80 0x7fb46c1a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.944+0000 7fb46b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb46c102e80 0x7fb46c1a0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:50104/0 (socket says 172.21.15.27:50104) 2024-09-21T00:33:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.944+0000 7fb46b400640 1 -- 172.21.15.27:0/677413361 learned_addr learned my addr 172.21.15.27:0/677413361 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.944+0000 7fb46be00640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb46c102530 0x7fb46c1a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.944+0000 7fb470c00640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c1079f0 0x7fb46c1a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.945+0000 7fb46be00640 1 -- 172.21.15.27:0/677413361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb46c102e80 msgr2=0x7fb46c1a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.945+0000 7fb46be00640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb46c102e80 0x7fb46c1a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.945+0000 7fb46be00640 1 -- 172.21.15.27:0/677413361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c1079f0 msgr2=0x7fb46c1a6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.945+0000 7fb46be00640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c1079f0 0x7fb46c1a6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.945+0000 7fb46be00640 1 -- 172.21.15.27:0/677413361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c1a6770 con 0x7fb46c102530 2024-09-21T00:33:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.945+0000 7fb470c00640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c1079f0 0x7fb46c1a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:33:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.945+0000 7fb46b400640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb46c102e80 0x7fb46c1a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:33:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.946+0000 7fb46be00640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb46c102530 0x7fb46c1a08d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00c970 tx=0x7fb45c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.948+0000 7fb468c00640 1 -- 172.21.15.27:0/677413361 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c007bb0 con 0x7fb46c102530 2024-09-21T00:33:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.948+0000 7fb468c00640 1 -- 172.21.15.27:0/677413361 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb45c007d50 con 0x7fb46c102530 2024-09-21T00:33:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.948+0000 7fb468c00640 1 -- 172.21.15.27:0/677413361 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c018780 con 0x7fb46c102530 2024-09-21T00:33:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.948+0000 7fb471600640 1 -- 172.21.15.27:0/677413361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46c1a6a60 con 0x7fb46c102530 2024-09-21T00:33:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.948+0000 7fb471600640 1 -- 172.21.15.27:0/677413361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46c1ad280 con 0x7fb46c102530 2024-09-21T00:33:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.956+0000 7fb468c00640 1 -- 172.21.15.27:0/677413361 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb45c00f450 con 0x7fb46c102530 2024-09-21T00:33:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.958+0000 7fb468c00640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb4500b67c0 0x7fb4500b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.959+0000 7fb468c00640 1 -- 172.21.15.27:0/677413361 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9607+0+0 (secure 0 0 0) 0x7fb45c0e7af0 con 0x7fb46c102530 2024-09-21T00:33:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.959+0000 7fb46b400640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb4500b67c0 0x7fb4500b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.961+0000 7fb46b400640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb4500b67c0 0x7fb4500b8cd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0ff6a0 tx=0x7fb458032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.967+0000 7fb479c76740 1 Processor -- start 2024-09-21T00:33:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.968+0000 7fb479c76740 1 -- start start 2024-09-21T00:33:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.968+0000 7fb479c76740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf08011b0 0x561cf0ba5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.968+0000 7fb479c76740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07d82f0 0x561cf0ba6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.968+0000 7fb479c76740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07e04d0 0x561cf0ba6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.968+0000 7fb479c76740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561cf0b1b6b0 con 0x561cf08011b0 2024-09-21T00:33:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.968+0000 7fb479c76740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561cf0b0aae0 con 0x561cf07e04d0 2024-09-21T00:33:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.968+0000 7fb479c76740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561cf0b0e8f0 con 0x561cf07d82f0 2024-09-21T00:33:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.969+0000 7fb46b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07d82f0 0x561cf0ba6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.969+0000 7fb470c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07e04d0 0x561cf0ba6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.969+0000 7fb470c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07e04d0 0x561cf0ba6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40580/0 (socket says 172.21.15.27:40580) 2024-09-21T00:33:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.969+0000 7fb470c00640 1 -- 172.21.15.27:0/1984843715 learned_addr learned my addr 172.21.15.27:0/1984843715 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.969+0000 7fb470c00640 1 -- 172.21.15.27:0/1984843715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07d82f0 msgr2=0x561cf0ba6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.969+0000 7fb470c00640 1 --2- 172.21.15.27:0/1984843715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07d82f0 0x561cf0ba6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.969+0000 7fb470c00640 1 -- 172.21.15.27:0/1984843715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf08011b0 msgr2=0x561cf0ba5e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:33:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.970+0000 7fb46be00640 1 --2- 172.21.15.27:0/1984843715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf08011b0 0x561cf0ba5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.971+0000 7fb470c00640 1 --2- 172.21.15.27:0/1984843715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf08011b0 0x561cf0ba5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.971+0000 7fb470c00640 1 -- 172.21.15.27:0/1984843715 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf0ad7780 con 0x561cf07e04d0 2024-09-21T00:33:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.971+0000 7fb470c00640 1 --2- 172.21.15.27:0/1984843715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07e04d0 0x561cf0ba6890 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb4640106f0 tx=0x7fb464010bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=825c3998e02400d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.971+0000 7fb461200640 1 -- 172.21.15.27:0/1984843715 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46400e8d0 con 0x561cf07e04d0 2024-09-21T00:33:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.971+0000 7fb461200640 1 -- 172.21.15.27:0/1984843715 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb464012450 con 0x561cf07e04d0 2024-09-21T00:33:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.971+0000 7fb461200640 1 -- 172.21.15.27:0/1984843715 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46401c670 con 0x561cf07e04d0 2024-09-21T00:33:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.973+0000 7fb479c76740 1 -- 172.21.15.27:0/1984843715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07e04d0 msgr2=0x561cf0ba6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.973+0000 7fb479c76740 1 --2- 172.21.15.27:0/1984843715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07e04d0 0x561cf0ba6890 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb4640106f0 tx=0x7fb464010bb0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.974+0000 7fb479c76740 1 -- 172.21.15.27:0/1984843715 shutdown_connections 2024-09-21T00:33:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.974+0000 7fb479c76740 1 --2- 172.21.15.27:0/1984843715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07e04d0 0x561cf0ba6890 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.974+0000 7fb479c76740 1 --2- 172.21.15.27:0/1984843715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07d82f0 0x561cf0ba6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.974+0000 7fb479c76740 1 --2- 172.21.15.27:0/1984843715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf08011b0 0x561cf0ba5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.974+0000 7fb479c76740 1 -- 172.21.15.27:0/1984843715 >> 172.21.15.27:0/1984843715 conn(0x561cf07f5300 msgr2=0x561cf0ba3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.975+0000 7fb479c76740 1 -- 172.21.15.27:0/1984843715 shutdown_connections 2024-09-21T00:33:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.975+0000 7fb479c76740 1 -- 172.21.15.27:0/1984843715 wait complete. 2024-09-21T00:33:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.976+0000 7fb479c76740 1 -- start start 2024-09-21T00:33:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.978+0000 7fb479c76740 1 Processor -- start 2024-09-21T00:33:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.979+0000 7fb479c76740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07d82f0 0x561cf0bb1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.979+0000 7fb479c76740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf07e04d0 0x561cf0bb19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.979+0000 7fb479c76740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf08011b0 0x561cf0bbd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.979+0000 7fb479c76740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561cf0a42ab0 con 0x561cf07e04d0 2024-09-21T00:33:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.979+0000 7fb479c76740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561cf0ab1570 con 0x561cf07d82f0 2024-09-21T00:33:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.979+0000 7fb479c76740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561cf0bb4590 con 0x561cf08011b0 2024-09-21T00:33:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.979+0000 7fb479c76740 10 client.0 _ll_register_callbacks cb 0x561cf0ac0100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:33:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.980+0000 7fb46be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07d82f0 0x561cf0bb1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.980+0000 7fb46b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf07e04d0 0x561cf0bb19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.980+0000 7fb46b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf07e04d0 0x561cf0bb19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52700/0 (socket says 172.21.15.27:52700) 2024-09-21T00:33:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.980+0000 7fb46b400640 1 -- 172.21.15.27:0/1100683605 learned_addr learned my addr 172.21.15.27:0/1100683605 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.980+0000 7fb470c00640 1 --2- 172.21.15.27:0/1100683605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf08011b0 0x561cf0bbd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.981+0000 7fb46b400640 1 -- 172.21.15.27:0/1100683605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf08011b0 msgr2=0x561cf0bbd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.981+0000 7fb46b400640 1 --2- 172.21.15.27:0/1100683605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf08011b0 0x561cf0bbd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.981+0000 7fb46b400640 1 -- 172.21.15.27:0/1100683605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07d82f0 msgr2=0x561cf0bb1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.981+0000 7fb46b400640 1 --2- 172.21.15.27:0/1100683605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07d82f0 0x561cf0bb1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.981+0000 7fb46b400640 1 -- 172.21.15.27:0/1100683605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf086c790 con 0x561cf07e04d0 2024-09-21T00:33:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.982+0000 7fb46b400640 1 --2- 172.21.15.27:0/1100683605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf07e04d0 0x561cf0bb19a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb458006010 tx=0x7fb458009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705238060a70ff31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.982+0000 7fb455600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:33:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.983+0000 7fb455600640 1 -- 172.21.15.27:0/1100683605 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458035040 con 0x561cf07e04d0 2024-09-21T00:33:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.983+0000 7fb455600640 1 -- 172.21.15.27:0/1100683605 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9607+0+0 (secure 0 0 0) 0x7fb458003ac0 con 0x561cf07e04d0 2024-09-21T00:33:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.983+0000 7fb479c76740 1 -- 172.21.15.27:0/1100683605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb464012450 con 0x561cf07e04d0 2024-09-21T00:33:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.984+0000 7fb455600640 1 -- 172.21.15.27:0/1100683605 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb458003cc0 con 0x561cf07e04d0 2024-09-21T00:33:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.984+0000 7fb479c76740 10 client.16572 fetch_fsmap learned FSMap version 47 2024-09-21T00:33:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.984+0000 7fb479c76740 1 -- 172.21.15.27:0/1100683605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561cf08e5620 con 0x561cf07e04d0 2024-09-21T00:33:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.984+0000 7fb455600640 1 -- 172.21.15.27:0/1100683605 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb464012450 con 0x561cf07e04d0 2024-09-21T00:33:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.986+0000 7fb455600640 1 -- 172.21.15.27:0/1100683605 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fb458004c50 con 0x561cf07e04d0 2024-09-21T00:33:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.986+0000 7fb479c76740 10 client.16572 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:33:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.986+0000 7fb479c76740 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:33:39.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.986+0000 7fb479c76740 20 client.16572 populate_metadata read hostname 'smithi027' 2024-09-21T00:33:39.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.987+0000 7fb479c76740 1 --2- 172.21.15.27:0/1100683605 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561cf07bfd80 0x561cf0bc35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:39.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.987+0000 7fb479c76740 4 client.16572 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:33:39.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.987+0000 7fb479c76740 1 -- 172.21.15.27:0/1100683605 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cf0b1d600 con 0x561cf07bfd80 2024-09-21T00:33:39.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.988+0000 7fb46be00640 1 --2- 172.21.15.27:0/1100683605 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561cf07bfd80 0x561cf0bc35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:39.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.991+0000 7fb46be00640 1 --2- 172.21.15.27:0/1100683605 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561cf07bfd80 0x561cf0bc35f0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96087145b069f5ad server_cookie=fff2f6fd50a2f674 in_seq=0 out_seq=0 2024-09-21T00:33:39.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.991+0000 7fb455600640 10 client.16572 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:39.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.993+0000 7fb455600640 1 -- 172.21.15.27:0/1100683605 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561cf0b1d600 con 0x561cf07bfd80 2024-09-21T00:33:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.993+0000 7fb455600640 10 client.16572 handle_command_reply: tid=0 2024-09-21T00:33:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.996+0000 7fb479c76740 1 client.16572 shutdown 2024-09-21T00:33:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.997+0000 7fb479c76740 1 -- 172.21.15.27:0/1100683605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf07e04d0 msgr2=0x561cf0bb19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.997+0000 7fb479c76740 1 --2- 172.21.15.27:0/1100683605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf07e04d0 0x561cf0bb19a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb458006010 tx=0x7fb458009260 comp rx=0 tx=0).stop 2024-09-21T00:33:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.997+0000 7fb479c76740 1 -- 172.21.15.27:0/1100683605 shutdown_connections 2024-09-21T00:33:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.997+0000 7fb479c76740 1 --2- 172.21.15.27:0/1100683605 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561cf07bfd80 0x561cf0bc35f0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.997+0000 7fb479c76740 1 --2- 172.21.15.27:0/1100683605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf08011b0 0x561cf0bbd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.997+0000 7fb479c76740 1 --2- 172.21.15.27:0/1100683605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf07e04d0 0x561cf0bb19a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.997+0000 7fb479c76740 1 --2- 172.21.15.27:0/1100683605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07d82f0 0x561cf0bb1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.997+0000 7fb455600640 0 client.16572 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.997+0000 7fb479c76740 1 -- 172.21.15.27:0/1100683605 >> 172.21.15.27:0/1100683605 conn(0x561cf07f5300 msgr2=0x561cf0ba6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.998+0000 7fb479c76740 1 -- 172.21.15.27:0/1100683605 shutdown_connections 2024-09-21T00:33:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.998+0000 7fb479c76740 1 -- 172.21.15.27:0/1100683605 wait complete. 2024-09-21T00:33:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:39.999+0000 7fb479c76740 20 client.16572 trim_cache size 0 max 16384 2024-09-21T00:33:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.048+0000 7fb479c76740 1 Processor -- start 2024-09-21T00:33:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.049+0000 7fb479c76740 1 -- start start 2024-09-21T00:33:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.050+0000 7fb479c76740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07d82f0 0x561cf0ba2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:40.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.050+0000 7fb479c76740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07e04d0 0x561cf0ba7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.053+0000 7fb479c76740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf08011b0 0x561cf0bae630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.053+0000 7fb479c76740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1b9e10 con 0x561cf08011b0 2024-09-21T00:33:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.053+0000 7fb479c76740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1b7620 con 0x561cf07d82f0 2024-09-21T00:33:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.053+0000 7fb479c76740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1b5490 con 0x561cf07e04d0 2024-09-21T00:33:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.053+0000 7fb46b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07e04d0 0x561cf0ba7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.053+0000 7fb46b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07e04d0 0x561cf0ba7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:50120/0 (socket says 172.21.15.27:50120) 2024-09-21T00:33:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.053+0000 7fb46b400640 1 -- 172.21.15.27:0/2508146006 learned_addr learned my addr 172.21.15.27:0/2508146006 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.054+0000 7fb46be00640 1 --2- 172.21.15.27:0/2508146006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07d82f0 0x561cf0ba2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.054+0000 7fb46be00640 1 -- 172.21.15.27:0/2508146006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07e04d0 msgr2=0x561cf0ba7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.054+0000 7fb46be00640 1 --2- 172.21.15.27:0/2508146006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07e04d0 0x561cf0ba7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.054+0000 7fb46be00640 1 -- 172.21.15.27:0/2508146006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf08011b0 msgr2=0x561cf0bae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.055+0000 7fb470c00640 1 --2- 172.21.15.27:0/2508146006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf08011b0 0x561cf0bae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.055+0000 7fb46be00640 1 --2- 172.21.15.27:0/2508146006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf08011b0 0x561cf0bae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.055+0000 7fb46be00640 1 -- 172.21.15.27:0/2508146006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf0ba8110 con 0x561cf07d82f0 2024-09-21T00:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.055+0000 7fb46b400640 1 --2- 172.21.15.27:0/2508146006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07e04d0 0x561cf0ba7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:33:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.055+0000 7fb46be00640 1 --2- 172.21.15.27:0/2508146006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07d82f0 0x561cf0ba2d20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0e4da0 tx=0x7fb45c00fdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc2152440ebdf588 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.056+0000 7fb457e00640 1 -- 172.21.15.27:0/2508146006 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0b6ec0 con 0x561cf07d82f0 2024-09-21T00:33:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.056+0000 7fb457e00640 1 -- 172.21.15.27:0/2508146006 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb45c00f740 con 0x561cf07d82f0 2024-09-21T00:33:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.056+0000 7fb457e00640 1 -- 172.21.15.27:0/2508146006 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0a7e00 con 0x561cf07d82f0 2024-09-21T00:33:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.057+0000 7fb479c76740 1 -- 172.21.15.27:0/2508146006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07d82f0 msgr2=0x561cf0ba2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.057+0000 7fb479c76740 1 --2- 172.21.15.27:0/2508146006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07d82f0 0x561cf0ba2d20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0e4da0 tx=0x7fb45c00fdf0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.058+0000 7fb479c76740 1 -- 172.21.15.27:0/2508146006 shutdown_connections 2024-09-21T00:33:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.058+0000 7fb479c76740 1 --2- 172.21.15.27:0/2508146006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf08011b0 0x561cf0bae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.059+0000 7fb479c76740 1 --2- 172.21.15.27:0/2508146006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07e04d0 0x561cf0ba7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.059+0000 7fb479c76740 1 --2- 172.21.15.27:0/2508146006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07d82f0 0x561cf0ba2d20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.059+0000 7fb479c76740 1 -- 172.21.15.27:0/2508146006 >> 172.21.15.27:0/2508146006 conn(0x561cf07f5300 msgr2=0x561cf0ba6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.061+0000 7fb479c76740 1 -- 172.21.15.27:0/2508146006 shutdown_connections 2024-09-21T00:33:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.061+0000 7fb479c76740 1 -- 172.21.15.27:0/2508146006 wait complete. 2024-09-21T00:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.061+0000 7fb479c76740 1 -- start start 2024-09-21T00:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.062+0000 7fb479c76740 1 Processor -- start 2024-09-21T00:33:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.062+0000 7fb479c76740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07d82f0 0x561cf0bb7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.063+0000 7fb479c76740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07e04d0 0x561cf0bb78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.063+0000 7fb479c76740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf08011b0 0x561cf0c24fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.063+0000 7fb479c76740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1b5ae0 con 0x561cf08011b0 2024-09-21T00:33:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.063+0000 7fb479c76740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1b6f50 con 0x561cf07e04d0 2024-09-21T00:33:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.063+0000 7fb479c76740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1b61b0 con 0x561cf07d82f0 2024-09-21T00:33:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.063+0000 7fb479c76740 10 client.0 _ll_register_callbacks cb 0x7fb4580316d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:33:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.063+0000 7fb46b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07e04d0 0x561cf0bb78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.063+0000 7fb46b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07e04d0 0x561cf0bb78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40590/0 (socket says 172.21.15.27:40590) 2024-09-21T00:33:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.063+0000 7fb46b400640 1 -- 172.21.15.27:0/3670049084 learned_addr learned my addr 172.21.15.27:0/3670049084 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.064+0000 7fb46b400640 1 -- 172.21.15.27:0/3670049084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07d82f0 msgr2=0x561cf0bb7310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:33:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.064+0000 7fb46b400640 1 --2- 172.21.15.27:0/3670049084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07d82f0 0x561cf0bb7310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.065+0000 7fb46b400640 1 -- 172.21.15.27:0/3670049084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf08011b0 msgr2=0x561cf0c24fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.065+0000 7fb46b400640 1 --2- 172.21.15.27:0/3670049084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf08011b0 0x561cf0c24fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.065+0000 7fb46b400640 1 -- 172.21.15.27:0/3670049084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf0c25650 con 0x561cf07e04d0 2024-09-21T00:33:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.066+0000 7fb46b400640 1 --2- 172.21.15.27:0/3670049084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07e04d0 0x561cf0bb78b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb458003f00 tx=0x7fb458031880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=952c5cea52613fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.066+0000 7fb455600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:33:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.067+0000 7fb455600640 1 -- 172.21.15.27:0/3670049084 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458035040 con 0x561cf07e04d0 2024-09-21T00:33:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.067+0000 7fb455600640 1 -- 172.21.15.27:0/3670049084 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9607+0+0 (secure 0 0 0) 0x7fb458004bd0 con 0x561cf07e04d0 2024-09-21T00:33:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.067+0000 7fb479c76740 1 -- 172.21.15.27:0/3670049084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb45c00f740 con 0x561cf07e04d0 2024-09-21T00:33:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.069+0000 7fb455600640 1 -- 172.21.15.27:0/3670049084 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb45806b050 con 0x561cf07e04d0 2024-09-21T00:33:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.069+0000 7fb479c76740 10 client.26588 fetch_fsmap learned FSMap version 47 2024-09-21T00:33:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.069+0000 7fb479c76740 1 -- 172.21.15.27:0/3670049084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561cf0c2db80 con 0x561cf07e04d0 2024-09-21T00:33:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.069+0000 7fb455600640 1 -- 172.21.15.27:0/3670049084 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb45c00f740 con 0x561cf07e04d0 2024-09-21T00:33:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.069+0000 7fb455600640 1 -- 172.21.15.27:0/3670049084 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fb458064070 con 0x561cf07e04d0 2024-09-21T00:33:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.070+0000 7fb479c76740 10 client.26588 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:33:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.070+0000 7fb479c76740 10 client.26588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:33:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.070+0000 7fb479c76740 20 client.26588 populate_metadata read hostname 'smithi027' 2024-09-21T00:33:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.070+0000 7fb479c76740 1 --2- 172.21.15.27:0/3670049084 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561cf07951c0 0x561cf0c30430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.070+0000 7fb479c76740 4 client.26588 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:33:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.070+0000 7fb479c76740 1 -- 172.21.15.27:0/3670049084 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561cf0b190b0 con 0x561cf07951c0 2024-09-21T00:33:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.070+0000 7fb46be00640 1 --2- 172.21.15.27:0/3670049084 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561cf07951c0 0x561cf0c30430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.071+0000 7fb455600640 10 client.26588 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.071+0000 7fb46be00640 1 --2- 172.21.15.27:0/3670049084 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561cf07951c0 0x561cf0c30430 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7f4b4dbe54cac12 server_cookie=6773e5c93617f976 in_seq=0 out_seq=0 2024-09-21T00:33:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.270+0000 7fb455600640 1 -- 172.21.15.27:0/3670049084 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561cf0b190b0 con 0x561cf07951c0 2024-09-21T00:33:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.270+0000 7fb455600640 10 client.26588 handle_command_reply: tid=0 2024-09-21T00:33:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.270+0000 7fb479c76740 1 client.26588 shutdown 2024-09-21T00:33:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.270+0000 7fb479c76740 1 -- 172.21.15.27:0/3670049084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07e04d0 msgr2=0x561cf0bb78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.271+0000 7fb479c76740 1 --2- 172.21.15.27:0/3670049084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07e04d0 0x561cf0bb78b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb458003f00 tx=0x7fb458031880 comp rx=0 tx=0).stop 2024-09-21T00:33:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.271+0000 7fb479c76740 1 -- 172.21.15.27:0/3670049084 shutdown_connections 2024-09-21T00:33:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.271+0000 7fb479c76740 1 --2- 172.21.15.27:0/3670049084 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561cf07951c0 0x561cf0c30430 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.271+0000 7fb479c76740 1 --2- 172.21.15.27:0/3670049084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561cf08011b0 0x561cf0c24fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.274+0000 7fb479c76740 1 --2- 172.21.15.27:0/3670049084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf07e04d0 0x561cf0bb78b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.274+0000 7fb479c76740 1 --2- 172.21.15.27:0/3670049084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf07d82f0 0x561cf0bb7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.274+0000 7fb479c76740 1 -- 172.21.15.27:0/3670049084 >> 172.21.15.27:0/3670049084 conn(0x561cf07f5300 msgr2=0x561cf0ba2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.274+0000 7fb455600640 0 client.26588 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.274+0000 7fb479c76740 1 -- 172.21.15.27:0/3670049084 shutdown_connections 2024-09-21T00:33:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.274+0000 7fb479c76740 1 -- 172.21.15.27:0/3670049084 wait complete. 2024-09-21T00:33:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.275+0000 7fb479c76740 20 client.26588 trim_cache size 0 max 16384 2024-09-21T00:33:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:33:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:33:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:33:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:33:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.279+0000 7fb471600640 1 -- 172.21.15.27:0/677413361 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb4500b67c0 msgr2=0x7fb4500b8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.279+0000 7fb471600640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb4500b67c0 0x7fb4500b8cd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0ff6a0 tx=0x7fb458032ed0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.279+0000 7fb471600640 1 -- 172.21.15.27:0/677413361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb46c102530 msgr2=0x7fb46c1a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.279+0000 7fb471600640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb46c102530 0x7fb46c1a08d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00c970 tx=0x7fb45c00ce30 comp rx=0 tx=0).stop 2024-09-21T00:33:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.279+0000 7fb471600640 1 -- 172.21.15.27:0/677413361 shutdown_connections 2024-09-21T00:33:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.279+0000 7fb471600640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb4500b67c0 0x7fb4500b8cd0 secure :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0ff6a0 tx=0x7fb458032ed0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.279+0000 7fb471600640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c1079f0 0x7fb46c1a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.279+0000 7fb471600640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb46c102e80 0x7fb46c1a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.279+0000 7fb471600640 1 --2- 172.21.15.27:0/677413361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb46c102530 0x7fb46c1a08d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.279+0000 7fb471600640 1 -- 172.21.15.27:0/677413361 >> 172.21.15.27:0/677413361 conn(0x7fb46c0fe210 msgr2=0x7fb46c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.283+0000 7fb471600640 1 -- 172.21.15.27:0/677413361 shutdown_connections 2024-09-21T00:33:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:40.283+0000 7fb471600640 1 -- 172.21.15.27:0/677413361 wait complete. 2024-09-21T00:33:40.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:40 smithi089 ceph-mon[34481]: pgmap v762: 129 pgs: 19 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 102 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-09-21T00:33:40.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:40 smithi098 ceph-mon[34725]: pgmap v762: 129 pgs: 19 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 102 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-09-21T00:33:40.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:40 smithi027 ceph-mon[30745]: pgmap v762: 129 pgs: 19 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 102 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-09-21T00:33:41.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:41 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:33:41.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:41 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:33:41.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:41 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:33:41.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:41 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:33:41.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:41 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:33:41.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:41 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:33:41.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:41 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:33:41.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:41 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:33:41.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:41 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:33:42.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:42 smithi089 ceph-mon[34481]: pgmap v763: 129 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 111 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 218 op/s 2024-09-21T00:33:42.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:42 smithi098 ceph-mon[34725]: pgmap v763: 129 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 111 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 218 op/s 2024-09-21T00:33:42.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:42 smithi027 ceph-mon[30745]: pgmap v763: 129 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 111 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 218 op/s 2024-09-21T00:33:44.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:44 smithi089 ceph-mon[34481]: pgmap v764: 129 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 112 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-09-21T00:33:44.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:44 smithi098 ceph-mon[34725]: pgmap v764: 129 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 112 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-09-21T00:33:44.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:44 smithi027 ceph-mon[30745]: pgmap v764: 129 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 112 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-09-21T00:33:46.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:46 smithi089 ceph-mon[34481]: pgmap v765: 129 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 112 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 264 op/s 2024-09-21T00:33:46.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:46 smithi098 ceph-mon[34725]: pgmap v765: 129 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 112 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 264 op/s 2024-09-21T00:33:46.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:46 smithi027 ceph-mon[30745]: pgmap v765: 129 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 112 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 264 op/s 2024-09-21T00:33:48.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:48 smithi089 ceph-mon[34481]: pgmap v766: 129 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 112 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 272 op/s 2024-09-21T00:33:48.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:48 smithi098 ceph-mon[34725]: pgmap v766: 129 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 112 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 272 op/s 2024-09-21T00:33:48.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:48 smithi027 ceph-mon[30745]: pgmap v766: 129 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 112 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 272 op/s 2024-09-21T00:33:50.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:50 smithi089 ceph-mon[34481]: pgmap v767: 129 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 119 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 303 op/s 2024-09-21T00:33:50.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:50 smithi098 ceph-mon[34725]: pgmap v767: 129 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 119 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 303 op/s 2024-09-21T00:33:50.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:50 smithi027 ceph-mon[30745]: pgmap v767: 129 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 119 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 303 op/s 2024-09-21T00:33:51.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:51 smithi098 ceph-mon[34725]: pgmap v768: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 366 op/s 2024-09-21T00:33:51.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:51 smithi027 ceph-mon[30745]: pgmap v768: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 366 op/s 2024-09-21T00:33:51.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:51 smithi089 ceph-mon[34481]: pgmap v768: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 366 op/s 2024-09-21T00:33:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.923+0000 7f376fe00640 1 -- 172.21.15.27:0/2704692670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3770076080 msgr2=0x7f377010f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.923+0000 7f376fe00640 1 --2- 172.21.15.27:0/2704692670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3770076080 0x7f377010f970 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f375c009900 tx=0x7f375c02a410 comp rx=0 tx=0).stop 2024-09-21T00:33:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.923+0000 7f376fe00640 1 -- 172.21.15.27:0/2704692670 shutdown_connections 2024-09-21T00:33:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.923+0000 7f376fe00640 1 --2- 172.21.15.27:0/2704692670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f377010feb0 0x7f37701122d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.923+0000 7f376fe00640 1 --2- 172.21.15.27:0/2704692670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3770076080 0x7f377010f970 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.923+0000 7f376fe00640 1 --2- 172.21.15.27:0/2704692670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3770075730 0x7f3770075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.923+0000 7f376fe00640 1 -- 172.21.15.27:0/2704692670 >> 172.21.15.27:0/2704692670 conn(0x7f37700ff2e0 msgr2=0x7f3770101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.923+0000 7f376fe00640 1 -- 172.21.15.27:0/2704692670 shutdown_connections 2024-09-21T00:33:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.923+0000 7f376fe00640 1 -- 172.21.15.27:0/2704692670 wait complete. 2024-09-21T00:33:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.924+0000 7f376fe00640 1 Processor -- start 2024-09-21T00:33:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.924+0000 7f376fe00640 1 -- start start 2024-09-21T00:33:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.925+0000 7f376fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3770075730 0x7f377019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.925+0000 7f376fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3770076080 0x7f377019c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.925+0000 7f376fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f377010feb0 0x7f37701a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.925+0000 7f376fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37701159b0 con 0x7f3770075730 2024-09-21T00:33:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.925+0000 7f376fe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3770115830 con 0x7f377010feb0 2024-09-21T00:33:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.925+0000 7f376fe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3770115b30 con 0x7f3770076080 2024-09-21T00:33:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.925+0000 7f376ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3770075730 0x7f377019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.925+0000 7f376f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f377010feb0 0x7f37701a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.925+0000 7f376e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3770076080 0x7f377019c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.925+0000 7f376ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3770075730 0x7f377019c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42420/0 (socket says 172.21.15.27:42420) 2024-09-21T00:33:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.925+0000 7f376f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f377010feb0 0x7f37701a1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:47082/0 (socket says 172.21.15.27:47082) 2024-09-21T00:33:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.925+0000 7f376ea00640 1 -- 172.21.15.27:0/138850772 learned_addr learned my addr 172.21.15.27:0/138850772 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.925+0000 7f376f400640 1 -- 172.21.15.27:0/138850772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3770076080 msgr2=0x7f377019c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.925+0000 7f376f400640 1 --2- 172.21.15.27:0/138850772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3770076080 0x7f377019c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.926+0000 7f376f400640 1 -- 172.21.15.27:0/138850772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3770075730 msgr2=0x7f377019c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.926+0000 7f376f400640 1 --2- 172.21.15.27:0/138850772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3770075730 0x7f377019c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.926+0000 7f376f400640 1 -- 172.21.15.27:0/138850772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37701a2360 con 0x7f377010feb0 2024-09-21T00:33:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.926+0000 7f376ea00640 1 --2- 172.21.15.27:0/138850772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3770075730 0x7f377019c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:33:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.926+0000 7f376e000640 1 --2- 172.21.15.27:0/138850772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3770076080 0x7f377019c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:33:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.926+0000 7f376f400640 1 --2- 172.21.15.27:0/138850772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f377010feb0 0x7f37701a1c60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3768009fd0 tx=0x7f376800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.926+0000 7f3767400640 1 -- 172.21.15.27:0/138850772 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376800ce20 con 0x7f377010feb0 2024-09-21T00:33:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.926+0000 7f3767400640 1 -- 172.21.15.27:0/138850772 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3768005020 con 0x7f377010feb0 2024-09-21T00:33:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.926+0000 7f3767400640 1 -- 172.21.15.27:0/138850772 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768010690 con 0x7f377010feb0 2024-09-21T00:33:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.926+0000 7f376fe00640 1 -- 172.21.15.27:0/138850772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37701a2650 con 0x7f377010feb0 2024-09-21T00:33:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.926+0000 7f376fe00640 1 -- 172.21.15.27:0/138850772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37701a8de0 con 0x7f377010feb0 2024-09-21T00:33:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.930+0000 7f3767400640 1 -- 172.21.15.27:0/138850772 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f37680054d0 con 0x7f377010feb0 2024-09-21T00:33:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.931+0000 7f3767400640 1 --2- 172.21.15.27:0/138850772 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f37540b6670 0x7f37540b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.932+0000 7f3767400640 1 -- 172.21.15.27:0/138850772 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9607+0+0 (secure 0 0 0) 0x7f37680e8270 con 0x7f377010feb0 2024-09-21T00:33:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.932+0000 7f376ea00640 1 --2- 172.21.15.27:0/138850772 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f37540b6670 0x7f37540b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.933+0000 7f376ea00640 1 --2- 172.21.15.27:0/138850772 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f37540b6670 0x7f37540b8b80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3760011550 tx=0x7f3760005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.933+0000 7f377c68e740 1 Processor -- start 2024-09-21T00:33:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.933+0000 7f377c68e740 1 -- start start 2024-09-21T00:33:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.933+0000 7f377c68e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 0x561be49a8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.933+0000 7f377c68e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45db2f0 0x561be49a9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.933+0000 7f377c68e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45e34d0 0x561be49a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.933+0000 7f377c68e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561be491e6b0 con 0x561be45e34d0 2024-09-21T00:33:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.933+0000 7f377c68e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561be490dae0 con 0x561be45db2f0 2024-09-21T00:33:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.933+0000 7f377c68e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561be49118f0 con 0x561be46041b0 2024-09-21T00:33:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.933+0000 7f376e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45db2f0 0x561be49a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.933+0000 7f376e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45db2f0 0x561be49a9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:47092/0 (socket says 172.21.15.27:47092) 2024-09-21T00:33:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.933+0000 7f376e000640 1 -- 172.21.15.27:0/2372637383 learned_addr learned my addr 172.21.15.27:0/2372637383 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.934+0000 7f376f400640 1 --2- 172.21.15.27:0/2372637383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45e34d0 0x561be49a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.934+0000 7f376ea00640 1 --2- 172.21.15.27:0/2372637383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 0x561be49a8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.934+0000 7f376e000640 1 -- 172.21.15.27:0/2372637383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 msgr2=0x561be49a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.934+0000 7f376e000640 1 --2- 172.21.15.27:0/2372637383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 0x561be49a8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.934+0000 7f376e000640 1 -- 172.21.15.27:0/2372637383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45e34d0 msgr2=0x561be49a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.934+0000 7f376e000640 1 --2- 172.21.15.27:0/2372637383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45e34d0 0x561be49a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.934+0000 7f376e000640 1 -- 172.21.15.27:0/2372637383 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be48da780 con 0x561be45db2f0 2024-09-21T00:33:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.934+0000 7f376e000640 1 --2- 172.21.15.27:0/2372637383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45db2f0 0x561be49a9350 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f375c033000 tx=0x7f375c005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f64e98a36e9003 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.934+0000 7f375b400640 1 -- 172.21.15.27:0/2372637383 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c035040 con 0x561be45db2f0 2024-09-21T00:33:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.935+0000 7f375b400640 1 -- 172.21.15.27:0/2372637383 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f375c031720 con 0x561be45db2f0 2024-09-21T00:33:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.935+0000 7f375b400640 1 -- 172.21.15.27:0/2372637383 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0049d0 con 0x561be45db2f0 2024-09-21T00:33:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.935+0000 7f377c68e740 1 -- 172.21.15.27:0/2372637383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45db2f0 msgr2=0x561be49a9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.935+0000 7f377c68e740 1 --2- 172.21.15.27:0/2372637383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45db2f0 0x561be49a9350 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f375c033000 tx=0x7f375c005e20 comp rx=0 tx=0).stop 2024-09-21T00:33:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.936+0000 7f377c68e740 1 -- 172.21.15.27:0/2372637383 shutdown_connections 2024-09-21T00:33:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.936+0000 7f377c68e740 1 --2- 172.21.15.27:0/2372637383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45e34d0 0x561be49a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.936+0000 7f377c68e740 1 --2- 172.21.15.27:0/2372637383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45db2f0 0x561be49a9350 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.936+0000 7f377c68e740 1 --2- 172.21.15.27:0/2372637383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 0x561be49a8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.936+0000 7f377c68e740 1 -- 172.21.15.27:0/2372637383 >> 172.21.15.27:0/2372637383 conn(0x561be45f8300 msgr2=0x561be49a6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.936+0000 7f377c68e740 1 -- 172.21.15.27:0/2372637383 shutdown_connections 2024-09-21T00:33:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.936+0000 7f377c68e740 1 -- 172.21.15.27:0/2372637383 wait complete. 2024-09-21T00:33:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.936+0000 7f377c68e740 1 -- start start 2024-09-21T00:33:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.936+0000 7f377c68e740 1 Processor -- start 2024-09-21T00:33:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.937+0000 7f377c68e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45db2f0 0x561be49b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.937+0000 7f377c68e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be45e34d0 0x561be49b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.937+0000 7f377c68e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be46041b0 0x561be49c0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.937+0000 7f377c68e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561be4845ab0 con 0x561be45db2f0 2024-09-21T00:33:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.937+0000 7f377c68e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561be48b4570 con 0x561be46041b0 2024-09-21T00:33:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.937+0000 7f377c68e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561be49b7530 con 0x561be45e34d0 2024-09-21T00:33:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.937+0000 7f377c68e740 10 client.0 _ll_register_callbacks cb 0x561be48c3100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:33:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.937+0000 7f376ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45db2f0 0x561be49b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.937+0000 7f376f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be46041b0 0x561be49c0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.937+0000 7f376e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be45e34d0 0x561be49b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.937+0000 7f376f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be46041b0 0x561be49c0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:47102/0 (socket says 172.21.15.27:47102) 2024-09-21T00:33:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.937+0000 7f376f400640 1 -- 172.21.15.27:0/1980059183 learned_addr learned my addr 172.21.15.27:0/1980059183 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.937+0000 7f376ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45db2f0 0x561be49b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42452/0 (socket says 172.21.15.27:42452) 2024-09-21T00:33:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.938+0000 7f376f400640 1 -- 172.21.15.27:0/1980059183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be45e34d0 msgr2=0x561be49b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.938+0000 7f376f400640 1 --2- 172.21.15.27:0/1980059183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be45e34d0 0x561be49b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.938+0000 7f376f400640 1 -- 172.21.15.27:0/1980059183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45db2f0 msgr2=0x561be49b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.938+0000 7f376f400640 1 --2- 172.21.15.27:0/1980059183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45db2f0 0x561be49b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.938+0000 7f376f400640 1 -- 172.21.15.27:0/1980059183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be466f790 con 0x561be46041b0 2024-09-21T00:33:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.938+0000 7f376f400640 1 --2- 172.21.15.27:0/1980059183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be46041b0 0x561be49c0820 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f37680e5580 tx=0x7f37680ad390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf7e5e2124b57ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.938+0000 7f374fe00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:33:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.938+0000 7f374fe00640 1 -- 172.21.15.27:0/1980059183 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37680b73b0 con 0x561be46041b0 2024-09-21T00:33:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.938+0000 7f374fe00640 1 -- 172.21.15.27:0/1980059183 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9607+0+0 (secure 0 0 0) 0x7f37680b2240 con 0x561be46041b0 2024-09-21T00:33:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.938+0000 7f377c68e740 1 -- 172.21.15.27:0/1980059183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375c031720 con 0x561be46041b0 2024-09-21T00:33:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.938+0000 7f374fe00640 1 -- 172.21.15.27:0/1980059183 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f37680b2700 con 0x561be46041b0 2024-09-21T00:33:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.938+0000 7f377c68e740 10 client.26600 fetch_fsmap learned FSMap version 47 2024-09-21T00:33:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.938+0000 7f377c68e740 1 -- 172.21.15.27:0/1980059183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561be46e8620 con 0x561be46041b0 2024-09-21T00:33:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.939+0000 7f374fe00640 1 -- 172.21.15.27:0/1980059183 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f375c031720 con 0x561be46041b0 2024-09-21T00:33:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.939+0000 7f374fe00640 1 -- 172.21.15.27:0/1980059183 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f37680a9570 con 0x561be46041b0 2024-09-21T00:33:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.939+0000 7f377c68e740 10 client.26600 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:33:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.939+0000 7f377c68e740 10 client.26600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:33:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.939+0000 7f377c68e740 20 client.26600 populate_metadata read hostname 'smithi027' 2024-09-21T00:33:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.939+0000 7f377c68e740 1 --2- 172.21.15.27:0/1980059183 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561be45c2d80 0x561be49c6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.939+0000 7f377c68e740 4 client.26600 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:33:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.939+0000 7f377c68e740 1 -- 172.21.15.27:0/1980059183 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561be4920600 con 0x561be45c2d80 2024-09-21T00:33:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.939+0000 7f376ea00640 1 --2- 172.21.15.27:0/1980059183 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561be45c2d80 0x561be49c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.940+0000 7f376ea00640 1 --2- 172.21.15.27:0/1980059183 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561be45c2d80 0x561be49c6590 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c66f396446881ef0 server_cookie=3246633d0d219d80 in_seq=0 out_seq=0 2024-09-21T00:33:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.940+0000 7f374fe00640 10 client.26600 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.942+0000 7f374fe00640 1 -- 172.21.15.27:0/1980059183 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561be4920600 con 0x561be45c2d80 2024-09-21T00:33:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.942+0000 7f374fe00640 10 client.26600 handle_command_reply: tid=0 2024-09-21T00:33:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.942+0000 7f377c68e740 1 client.26600 shutdown 2024-09-21T00:33:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.942+0000 7f377c68e740 1 -- 172.21.15.27:0/1980059183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be46041b0 msgr2=0x561be49c0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.942+0000 7f377c68e740 1 --2- 172.21.15.27:0/1980059183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be46041b0 0x561be49c0820 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f37680e5580 tx=0x7f37680ad390 comp rx=0 tx=0).stop 2024-09-21T00:33:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.942+0000 7f377c68e740 1 -- 172.21.15.27:0/1980059183 shutdown_connections 2024-09-21T00:33:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.942+0000 7f377c68e740 1 --2- 172.21.15.27:0/1980059183 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561be45c2d80 0x561be49c6590 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.942+0000 7f377c68e740 1 --2- 172.21.15.27:0/1980059183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be46041b0 0x561be49c0820 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.942+0000 7f377c68e740 1 --2- 172.21.15.27:0/1980059183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be45e34d0 0x561be49b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.942+0000 7f377c68e740 1 --2- 172.21.15.27:0/1980059183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45db2f0 0x561be49b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.942+0000 7f377c68e740 1 -- 172.21.15.27:0/1980059183 >> 172.21.15.27:0/1980059183 conn(0x561be45f8300 msgr2=0x561be49a9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.942+0000 7f374fe00640 0 client.26600 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.942+0000 7f377c68e740 1 -- 172.21.15.27:0/1980059183 shutdown_connections 2024-09-21T00:33:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.942+0000 7f377c68e740 1 -- 172.21.15.27:0/1980059183 wait complete. 2024-09-21T00:33:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.942+0000 7f377c68e740 20 client.26600 trim_cache size 0 max 16384 2024-09-21T00:33:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.958+0000 7f377c68e740 1 Processor -- start 2024-09-21T00:33:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.959+0000 7f377c68e740 1 -- start start 2024-09-21T00:33:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.959+0000 7f377c68e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45db2f0 0x561be49a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.959+0000 7f377c68e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45e34d0 0x561be49b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.959+0000 7f377c68e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 0x561be49aa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.959+0000 7f377c68e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37701b5970 con 0x561be45e34d0 2024-09-21T00:33:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.959+0000 7f377c68e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f37701b3180 con 0x561be45db2f0 2024-09-21T00:33:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.959+0000 7f377c68e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37701b0ff0 con 0x561be46041b0 2024-09-21T00:33:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.959+0000 7f376ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45db2f0 0x561be49a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.959+0000 7f376e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45e34d0 0x561be49b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.959+0000 7f376f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 0x561be49aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.959+0000 7f376f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 0x561be49aa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55370/0 (socket says 172.21.15.27:55370) 2024-09-21T00:33:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.959+0000 7f376f400640 1 -- 172.21.15.27:0/3059602890 learned_addr learned my addr 172.21.15.27:0/3059602890 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.959+0000 7f376e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45e34d0 0x561be49b6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42464/0 (socket says 172.21.15.27:42464) 2024-09-21T00:33:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.959+0000 7f376ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45db2f0 0x561be49a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:47116/0 (socket says 172.21.15.27:47116) 2024-09-21T00:33:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.960+0000 7f376f400640 1 -- 172.21.15.27:0/3059602890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45db2f0 msgr2=0x561be49a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.960+0000 7f376f400640 1 --2- 172.21.15.27:0/3059602890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45db2f0 0x561be49a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.960+0000 7f376f400640 1 -- 172.21.15.27:0/3059602890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45e34d0 msgr2=0x561be49b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.960+0000 7f376f400640 1 --2- 172.21.15.27:0/3059602890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45e34d0 0x561be49b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.960+0000 7f376f400640 1 -- 172.21.15.27:0/3059602890 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be49ab3b0 con 0x561be46041b0 2024-09-21T00:33:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.960+0000 7f376f400640 1 --2- 172.21.15.27:0/3059602890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 0x561be49aa500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f37680adbb0 tx=0x7f37680ae440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d98aef36fa95ba88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.960+0000 7f375aa00640 1 -- 172.21.15.27:0/3059602890 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37680a9770 con 0x561be46041b0 2024-09-21T00:33:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.960+0000 7f375aa00640 1 -- 172.21.15.27:0/3059602890 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37680b7640 con 0x561be46041b0 2024-09-21T00:33:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.960+0000 7f375aa00640 1 -- 172.21.15.27:0/3059602890 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37680ace80 con 0x561be46041b0 2024-09-21T00:33:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.961+0000 7f377c68e740 1 -- 172.21.15.27:0/3059602890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 msgr2=0x561be49aa500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.961+0000 7f377c68e740 1 --2- 172.21.15.27:0/3059602890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 0x561be49aa500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f37680adbb0 tx=0x7f37680ae440 comp rx=0 tx=0).stop 2024-09-21T00:33:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.961+0000 7f377c68e740 1 -- 172.21.15.27:0/3059602890 shutdown_connections 2024-09-21T00:33:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.961+0000 7f377c68e740 1 --2- 172.21.15.27:0/3059602890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 0x561be49aa500 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.961+0000 7f377c68e740 1 --2- 172.21.15.27:0/3059602890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45e34d0 0x561be49b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.961+0000 7f377c68e740 1 --2- 172.21.15.27:0/3059602890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45db2f0 0x561be49a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.961+0000 7f377c68e740 1 -- 172.21.15.27:0/3059602890 >> 172.21.15.27:0/3059602890 conn(0x561be45f8300 msgr2=0x561be49a9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.961+0000 7f377c68e740 1 -- 172.21.15.27:0/3059602890 shutdown_connections 2024-09-21T00:33:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.961+0000 7f377c68e740 1 -- 172.21.15.27:0/3059602890 wait complete. 2024-09-21T00:33:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.961+0000 7f377c68e740 1 -- start start 2024-09-21T00:33:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.962+0000 7f377c68e740 1 Processor -- start 2024-09-21T00:33:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.962+0000 7f377c68e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45db2f0 0x561be49bd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.962+0000 7f377c68e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45e34d0 0x561be49ba670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.962+0000 7f377c68e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 0x561be49bad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.962+0000 7f377c68e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37701b1640 con 0x561be45db2f0 2024-09-21T00:33:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.962+0000 7f377c68e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f37701b2ab0 con 0x561be45e34d0 2024-09-21T00:33:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.962+0000 7f377c68e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37701b1d10 con 0x561be46041b0 2024-09-21T00:33:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.962+0000 7f377c68e740 10 client.0 _ll_register_callbacks cb 0x7f37680adab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:33:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.962+0000 7f376e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45e34d0 0x561be49ba670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.962+0000 7f376ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45db2f0 0x561be49bd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.962+0000 7f376e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45e34d0 0x561be49ba670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:47120/0 (socket says 172.21.15.27:47120) 2024-09-21T00:33:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.962+0000 7f376e000640 1 -- 172.21.15.27:0/329048133 learned_addr learned my addr 172.21.15.27:0/329048133 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:33:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.963+0000 7f376f400640 1 --2- 172.21.15.27:0/329048133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 0x561be49bad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.963+0000 7f376e000640 1 -- 172.21.15.27:0/329048133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 msgr2=0x561be49bad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.963+0000 7f376e000640 1 --2- 172.21.15.27:0/329048133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 0x561be49bad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.963+0000 7f376e000640 1 -- 172.21.15.27:0/329048133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45db2f0 msgr2=0x561be49bd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:33:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.963+0000 7f376e000640 1 --2- 172.21.15.27:0/329048133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45db2f0 0x561be49bd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.963+0000 7f376e000640 1 -- 172.21.15.27:0/329048133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be4a2ff60 con 0x561be45e34d0 2024-09-21T00:33:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.963+0000 7f376e000640 1 --2- 172.21.15.27:0/329048133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45e34d0 0x561be49ba670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f375c005510 tx=0x7f375c0054a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f57edd83e20a621c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:33:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.963+0000 7f374fe00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:33:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.963+0000 7f374fe00640 1 -- 172.21.15.27:0/329048133 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c035040 con 0x561be45e34d0 2024-09-21T00:33:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.963+0000 7f377c68e740 1 -- 172.21.15.27:0/329048133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37680b7640 con 0x561be45e34d0 2024-09-21T00:33:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.963+0000 7f374fe00640 1 -- 172.21.15.27:0/329048133 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9607+0+0 (secure 0 0 0) 0x7f375c031720 con 0x561be45e34d0 2024-09-21T00:33:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.964+0000 7f374fe00640 1 -- 172.21.15.27:0/329048133 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f375c05c050 con 0x561be45e34d0 2024-09-21T00:33:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.964+0000 7f377c68e740 10 client.26606 fetch_fsmap learned FSMap version 47 2024-09-21T00:33:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.964+0000 7f377c68e740 1 -- 172.21.15.27:0/329048133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561be4a310c0 con 0x561be45e34d0 2024-09-21T00:33:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.964+0000 7f374fe00640 1 -- 172.21.15.27:0/329048133 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37680b7640 con 0x561be45e34d0 2024-09-21T00:33:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.964+0000 7f374fe00640 1 -- 172.21.15.27:0/329048133 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f375c004b40 con 0x561be45e34d0 2024-09-21T00:33:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.964+0000 7f377c68e740 10 client.26606 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:33:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.964+0000 7f377c68e740 10 client.26606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:33:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.964+0000 7f377c68e740 20 client.26606 populate_metadata read hostname 'smithi027' 2024-09-21T00:33:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.964+0000 7f377c68e740 1 --2- 172.21.15.27:0/329048133 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561be45981c0 0x561be4a334d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:33:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.964+0000 7f377c68e740 4 client.26606 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:33:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.964+0000 7f377c68e740 1 -- 172.21.15.27:0/329048133 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561be49b6dd0 con 0x561be45981c0 2024-09-21T00:33:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.964+0000 7f376ea00640 1 --2- 172.21.15.27:0/329048133 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561be45981c0 0x561be4a334d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:33:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.965+0000 7f374fe00640 10 client.26606 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:53.965+0000 7f376ea00640 1 --2- 172.21.15.27:0/329048133 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561be45981c0 0x561be4a334d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61d08735fbb086a server_cookie=bfa6b67cb59588e9 in_seq=0 out_seq=0 2024-09-21T00:33:54.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:53 smithi027 ceph-mon[30745]: pgmap v769: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 379 op/s 2024-09-21T00:33:54.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:53 smithi089 ceph-mon[34481]: pgmap v769: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 379 op/s 2024-09-21T00:33:54.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:53 smithi098 ceph-mon[34725]: pgmap v769: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 379 op/s 2024-09-21T00:33:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.613+0000 7f374fe00640 1 -- 172.21.15.27:0/329048133 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561be49b6dd0 con 0x561be45981c0 2024-09-21T00:33:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.613+0000 7f374fe00640 10 client.26606 handle_command_reply: tid=0 2024-09-21T00:33:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.613+0000 7f377c68e740 1 client.26606 shutdown 2024-09-21T00:33:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.613+0000 7f377c68e740 1 -- 172.21.15.27:0/329048133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45e34d0 msgr2=0x561be49ba670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.613+0000 7f377c68e740 1 --2- 172.21.15.27:0/329048133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45e34d0 0x561be49ba670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f375c005510 tx=0x7f375c0054a0 comp rx=0 tx=0).stop 2024-09-21T00:33:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.613+0000 7f377c68e740 1 -- 172.21.15.27:0/329048133 shutdown_connections 2024-09-21T00:33:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.613+0000 7f377c68e740 1 --2- 172.21.15.27:0/329048133 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561be45981c0 0x561be4a334d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.614+0000 7f377c68e740 1 --2- 172.21.15.27:0/329048133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be46041b0 0x561be49bad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.614+0000 7f377c68e740 1 --2- 172.21.15.27:0/329048133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561be45e34d0 0x561be49ba670 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.614+0000 7f377c68e740 1 --2- 172.21.15.27:0/329048133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561be45db2f0 0x561be49bd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.614+0000 7f374fe00640 0 client.26606 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:33:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.614+0000 7f377c68e740 1 -- 172.21.15.27:0/329048133 >> 172.21.15.27:0/329048133 conn(0x561be45f8300 msgr2=0x561be49aa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.614+0000 7f377c68e740 1 -- 172.21.15.27:0/329048133 shutdown_connections 2024-09-21T00:33:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.614+0000 7f377c68e740 1 -- 172.21.15.27:0/329048133 wait complete. 2024-09-21T00:33:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.614+0000 7f377c68e740 20 client.26606 trim_cache size 0 max 16384 2024-09-21T00:33:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:33:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:33:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:33:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:33:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.615+0000 7f376fe00640 1 -- 172.21.15.27:0/138850772 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f37540b6670 msgr2=0x7f37540b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.615+0000 7f376fe00640 1 --2- 172.21.15.27:0/138850772 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f37540b6670 0x7f37540b8b80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3760011550 tx=0x7f3760005f90 comp rx=0 tx=0).stop 2024-09-21T00:33:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.615+0000 7f376fe00640 1 -- 172.21.15.27:0/138850772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f377010feb0 msgr2=0x7f37701a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:33:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.615+0000 7f376fe00640 1 --2- 172.21.15.27:0/138850772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f377010feb0 0x7f37701a1c60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3768009fd0 tx=0x7f376800ef10 comp rx=0 tx=0).stop 2024-09-21T00:33:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.615+0000 7f376fe00640 1 -- 172.21.15.27:0/138850772 shutdown_connections 2024-09-21T00:33:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.615+0000 7f376fe00640 1 --2- 172.21.15.27:0/138850772 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f37540b6670 0x7f37540b8b80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.615+0000 7f376fe00640 1 --2- 172.21.15.27:0/138850772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f377010feb0 0x7f37701a1c60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.615+0000 7f376fe00640 1 --2- 172.21.15.27:0/138850772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3770076080 0x7f377019c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.615+0000 7f376fe00640 1 --2- 172.21.15.27:0/138850772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3770075730 0x7f377019c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:33:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.615+0000 7f376fe00640 1 -- 172.21.15.27:0/138850772 >> 172.21.15.27:0/138850772 conn(0x7f37700ff2e0 msgr2=0x7f3770109a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:33:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.615+0000 7f376fe00640 1 -- 172.21.15.27:0/138850772 shutdown_connections 2024-09-21T00:33:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:33:54.615+0000 7f376fe00640 1 -- 172.21.15.27:0/138850772 wait complete. 2024-09-21T00:33:56.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:55 smithi027 ceph-mon[30745]: pgmap v770: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 424 op/s 2024-09-21T00:33:56.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:55 smithi089 ceph-mon[34481]: pgmap v770: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 424 op/s 2024-09-21T00:33:56.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:55 smithi098 ceph-mon[34725]: pgmap v770: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 424 op/s 2024-09-21T00:33:57.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:57 smithi089 ceph-mon[34481]: osdmap e108: 12 total, 12 up, 12 in 2024-09-21T00:33:57.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:57 smithi098 ceph-mon[34725]: osdmap e108: 12 total, 12 up, 12 in 2024-09-21T00:33:57.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:57 smithi027 ceph-mon[30745]: osdmap e108: 12 total, 12 up, 12 in 2024-09-21T00:33:58.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:33:58 smithi089 ceph-mon[34481]: pgmap v772: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 451 op/s 2024-09-21T00:33:58.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:33:58 smithi098 ceph-mon[34725]: pgmap v772: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 451 op/s 2024-09-21T00:33:58.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:33:58 smithi027 ceph-mon[30745]: pgmap v772: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 451 op/s 2024-09-21T00:34:00.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:00 smithi089 ceph-mon[34481]: pgmap v773: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 539 op/s 2024-09-21T00:34:00.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:00 smithi098 ceph-mon[34725]: pgmap v773: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 539 op/s 2024-09-21T00:34:00.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:00 smithi027 ceph-mon[30745]: pgmap v773: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 539 op/s 2024-09-21T00:34:02.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:02 smithi089 ceph-mon[34481]: pgmap v774: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 593 op/s 2024-09-21T00:34:02.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:02 smithi098 ceph-mon[34725]: pgmap v774: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 593 op/s 2024-09-21T00:34:02.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:02 smithi027 ceph-mon[30745]: pgmap v774: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 593 op/s 2024-09-21T00:34:04.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:04 smithi089 ceph-mon[34481]: pgmap v775: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 601 op/s 2024-09-21T00:34:04.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:04 smithi098 ceph-mon[34725]: pgmap v775: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 601 op/s 2024-09-21T00:34:04.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:04 smithi027 ceph-mon[30745]: pgmap v775: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 601 op/s 2024-09-21T00:34:06.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:06 smithi027 ceph-mon[30745]: pgmap v776: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 559 op/s 2024-09-21T00:34:06.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:06 smithi089 ceph-mon[34481]: pgmap v776: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 559 op/s 2024-09-21T00:34:06.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:06 smithi098 ceph-mon[34725]: pgmap v776: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 559 op/s 2024-09-21T00:34:08.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:08 smithi089 ceph-mon[34481]: pgmap v777: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 521 op/s 2024-09-21T00:34:08.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:08 smithi098 ceph-mon[34725]: pgmap v777: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 521 op/s 2024-09-21T00:34:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.810+0000 7fad55e00640 1 -- 172.21.15.27:0/1404684999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad501017d0 msgr2=0x7fad50101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.810+0000 7fad55e00640 1 --2- 172.21.15.27:0/1404684999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad501017d0 0x7fad50101be0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fad38009900 tx=0x7fad3802a400 comp rx=0 tx=0).stop 2024-09-21T00:34:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.810+0000 7fad55e00640 1 -- 172.21.15.27:0/1404684999 shutdown_connections 2024-09-21T00:34:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.810+0000 7fad55e00640 1 --2- 172.21.15.27:0/1404684999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad501144f0 0x7fad50116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.810+0000 7fad55e00640 1 --2- 172.21.15.27:0/1404684999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad50102120 0x7fad50113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.810+0000 7fad55e00640 1 --2- 172.21.15.27:0/1404684999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad501017d0 0x7fad50101be0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.811+0000 7fad55e00640 1 -- 172.21.15.27:0/1404684999 >> 172.21.15.27:0/1404684999 conn(0x7fad50077730 msgr2=0x7fad50077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.811+0000 7fad55e00640 1 -- 172.21.15.27:0/1404684999 shutdown_connections 2024-09-21T00:34:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.811+0000 7fad55e00640 1 -- 172.21.15.27:0/1404684999 wait complete. 2024-09-21T00:34:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.814+0000 7fad55e00640 1 Processor -- start 2024-09-21T00:34:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.814+0000 7fad55e00640 1 -- start start 2024-09-21T00:34:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.815+0000 7fad55e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad501017d0 0x7fad50074e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.815+0000 7fad55e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad50102120 0x7fad5006e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.815+0000 7fad55e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad501144f0 0x7fad5006e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.816+0000 7fad4e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad50102120 0x7fad5006e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.816+0000 7fad4e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad50102120 0x7fad5006e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49020/0 (socket says 172.21.15.27:49020) 2024-09-21T00:34:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.816+0000 7fad4e000640 1 -- 172.21.15.27:0/1917282770 learned_addr learned my addr 172.21.15.27:0/1917282770 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.816+0000 7fad4ea00640 1 --2- 172.21.15.27:0/1917282770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad501017d0 0x7fad50074e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:08 smithi027 ceph-mon[30745]: pgmap v777: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 521 op/s 2024-09-21T00:34:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.816+0000 7fad55e00640 1 -- 172.21.15.27:0/1917282770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fad5011a0d0 con 0x7fad501144f0 2024-09-21T00:34:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.816+0000 7fad55e00640 1 -- 172.21.15.27:0/1917282770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fad50119f50 con 0x7fad501017d0 2024-09-21T00:34:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.816+0000 7fad55e00640 1 -- 172.21.15.27:0/1917282770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fad5011a250 con 0x7fad50102120 2024-09-21T00:34:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.816+0000 7fad4f400640 1 --2- 172.21.15.27:0/1917282770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad501144f0 0x7fad5006e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.818+0000 7fad4f400640 1 -- 172.21.15.27:0/1917282770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad50102120 msgr2=0x7fad5006e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.818+0000 7fad4f400640 1 --2- 172.21.15.27:0/1917282770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad50102120 0x7fad5006e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.818+0000 7fad4f400640 1 -- 172.21.15.27:0/1917282770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad501017d0 msgr2=0x7fad50074e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.818+0000 7fad4f400640 1 --2- 172.21.15.27:0/1917282770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad501017d0 0x7fad50074e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.818+0000 7fad4f400640 1 -- 172.21.15.27:0/1917282770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5006eec0 con 0x7fad501144f0 2024-09-21T00:34:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.818+0000 7fad4f400640 1 --2- 172.21.15.27:0/1917282770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad501144f0 0x7fad5006e6a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fad4800be20 tx=0x7fad4800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.819+0000 7fad54a00640 1 -- 172.21.15.27:0/1917282770 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48019070 con 0x7fad501144f0 2024-09-21T00:34:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.820+0000 7fad54a00640 1 -- 172.21.15.27:0/1917282770 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad480148f0 con 0x7fad501144f0 2024-09-21T00:34:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.820+0000 7fad55e00640 1 -- 172.21.15.27:0/1917282770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad50073b80 con 0x7fad501144f0 2024-09-21T00:34:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.820+0000 7fad55e00640 1 -- 172.21.15.27:0/1917282770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad50074150 con 0x7fad501144f0 2024-09-21T00:34:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.820+0000 7fad54a00640 1 -- 172.21.15.27:0/1917282770 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48012840 con 0x7fad501144f0 2024-09-21T00:34:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.825+0000 7fad5cea5740 1 Processor -- start 2024-09-21T00:34:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.829+0000 7fad54a00640 1 -- 172.21.15.27:0/1917282770 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fad4801c070 con 0x7fad501144f0 2024-09-21T00:34:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.831+0000 7fad54a00640 1 --2- 172.21.15.27:0/1917282770 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fad340b6700 0x7fad340b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.832+0000 7fad5cea5740 1 -- start start 2024-09-21T00:34:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.832+0000 7fad54a00640 1 -- 172.21.15.27:0/1917282770 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9607+0+0 (secure 0 0 0) 0x7fad480a8e00 con 0x7fad501144f0 2024-09-21T00:34:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.832+0000 7fad4e000640 1 --2- 172.21.15.27:0/1917282770 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fad340b6700 0x7fad340b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.833+0000 7fad5cea5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a6341b0 0x558c6a9d8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.833+0000 7fad5cea5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a60b2f0 0x558c6a9d9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.833+0000 7fad5cea5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6134d0 0x558c6a9d96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.833+0000 7fad5cea5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558c6a94e6b0 con 0x558c6a6341b0 2024-09-21T00:34:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.833+0000 7fad5cea5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558c6a93dae0 con 0x558c6a60b2f0 2024-09-21T00:34:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.833+0000 7fad5cea5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558c6a9418f0 con 0x558c6a6134d0 2024-09-21T00:34:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.833+0000 7fad4f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6134d0 0x558c6a9d96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.833+0000 7fad4f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6134d0 0x558c6a9d96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49026/0 (socket says 172.21.15.27:49026) 2024-09-21T00:34:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.833+0000 7fad4f400640 1 -- 172.21.15.27:0/2301200883 learned_addr learned my addr 172.21.15.27:0/2301200883 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.834+0000 7fad4e000640 1 --2- 172.21.15.27:0/2301200883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a60b2f0 0x558c6a9d9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.834+0000 7fad4e000640 1 -- 172.21.15.27:0/2301200883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6134d0 msgr2=0x558c6a9d96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.834+0000 7fad4e000640 1 --2- 172.21.15.27:0/2301200883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6134d0 0x558c6a9d96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.834+0000 7fad4e000640 1 -- 172.21.15.27:0/2301200883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a6341b0 msgr2=0x558c6a9d8c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:34:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.835+0000 7fad4e000640 1 --2- 172.21.15.27:0/2301200883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a6341b0 0x558c6a9d8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.835+0000 7fad4e000640 1 -- 172.21.15.27:0/2301200883 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c6a90a780 con 0x558c6a60b2f0 2024-09-21T00:34:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.835+0000 7fad4e000640 1 --2- 172.21.15.27:0/2301200883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a60b2f0 0x558c6a9d9170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fad4000f720 tx=0x7fad40019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdc014f2a4413712 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.836+0000 7fad4e000640 1 --2- 172.21.15.27:0/1917282770 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fad340b6700 0x7fad340b8c10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fad50077d80 tx=0x7fad400138e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.836+0000 7fad3fe00640 1 -- 172.21.15.27:0/2301200883 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4000fca0 con 0x558c6a60b2f0 2024-09-21T00:34:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.837+0000 7fad3fe00640 1 -- 172.21.15.27:0/2301200883 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad4000fe40 con 0x558c6a60b2f0 2024-09-21T00:34:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.837+0000 7fad3fe00640 1 -- 172.21.15.27:0/2301200883 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad400209f0 con 0x558c6a60b2f0 2024-09-21T00:34:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.839+0000 7fad5cea5740 1 -- 172.21.15.27:0/2301200883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a60b2f0 msgr2=0x558c6a9d9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.839+0000 7fad5cea5740 1 --2- 172.21.15.27:0/2301200883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a60b2f0 0x558c6a9d9170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fad4000f720 tx=0x7fad40019000 comp rx=0 tx=0).stop 2024-09-21T00:34:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.839+0000 7fad5cea5740 1 -- 172.21.15.27:0/2301200883 shutdown_connections 2024-09-21T00:34:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.839+0000 7fad5cea5740 1 --2- 172.21.15.27:0/2301200883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6134d0 0x558c6a9d96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.839+0000 7fad5cea5740 1 --2- 172.21.15.27:0/2301200883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a60b2f0 0x558c6a9d9170 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.839+0000 7fad5cea5740 1 --2- 172.21.15.27:0/2301200883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a6341b0 0x558c6a9d8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.839+0000 7fad5cea5740 1 -- 172.21.15.27:0/2301200883 >> 172.21.15.27:0/2301200883 conn(0x558c6a628300 msgr2=0x558c6a9d6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.840+0000 7fad5cea5740 1 -- 172.21.15.27:0/2301200883 shutdown_connections 2024-09-21T00:34:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.841+0000 7fad5cea5740 1 -- 172.21.15.27:0/2301200883 wait complete. 2024-09-21T00:34:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.842+0000 7fad5cea5740 1 -- start start 2024-09-21T00:34:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.842+0000 7fad5cea5740 1 Processor -- start 2024-09-21T00:34:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.845+0000 7fad5cea5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a60b2f0 0x558c6a9e42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.846+0000 7fad5cea5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6134d0 0x558c6a9e47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.846+0000 7fad5cea5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6341b0 0x558c6a9f0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.846+0000 7fad4ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a60b2f0 0x558c6a9e42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.846+0000 7fad4e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6134d0 0x558c6a9e47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.846+0000 7fad4e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6134d0 0x558c6a9e47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49028/0 (socket says 172.21.15.27:49028) 2024-09-21T00:34:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.846+0000 7fad4ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a60b2f0 0x558c6a9e42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39818/0 (socket says 172.21.15.27:39818) 2024-09-21T00:34:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.847+0000 7fad5cea5740 1 -- 172.21.15.27:0/2889380635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558c6a875ab0 con 0x558c6a60b2f0 2024-09-21T00:34:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.847+0000 7fad5cea5740 1 -- 172.21.15.27:0/2889380635 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558c6a8e4570 con 0x558c6a6341b0 2024-09-21T00:34:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.847+0000 7fad4e000640 1 -- 172.21.15.27:0/2889380635 learned_addr learned my addr 172.21.15.27:0/2889380635 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.847+0000 7fad5cea5740 1 -- 172.21.15.27:0/2889380635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558c6a9e7350 con 0x558c6a6134d0 2024-09-21T00:34:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.847+0000 7fad4f400640 1 --2- 172.21.15.27:0/2889380635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6341b0 0x558c6a9f0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.847+0000 7fad5cea5740 10 client.0 _ll_register_callbacks cb 0x558c6a8f3100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:34:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.847+0000 7fad4e000640 1 -- 172.21.15.27:0/2889380635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6341b0 msgr2=0x558c6a9f0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.847+0000 7fad4e000640 1 --2- 172.21.15.27:0/2889380635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6341b0 0x558c6a9f0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.847+0000 7fad4e000640 1 -- 172.21.15.27:0/2889380635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a60b2f0 msgr2=0x558c6a9e42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.847+0000 7fad4e000640 1 --2- 172.21.15.27:0/2889380635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a60b2f0 0x558c6a9e42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.848+0000 7fad4e000640 1 -- 172.21.15.27:0/2889380635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c6a69f790 con 0x558c6a6134d0 2024-09-21T00:34:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.848+0000 7fad4ea00640 1 --2- 172.21.15.27:0/2889380635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a60b2f0 0x558c6a9e42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:34:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.848+0000 7fad4e000640 1 --2- 172.21.15.27:0/2889380635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6134d0 0x558c6a9e47f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fad4000bae0 tx=0x7fad4000f4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8115129a189e7e77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.848+0000 7fad5cea5740 1 -- 172.21.15.27:0/2889380635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558c6a950600 con 0x558c6a6134d0 2024-09-21T00:34:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.848+0000 7fad3cc00640 10 client.26623 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:34:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.849+0000 7fad3cc00640 1 -- 172.21.15.27:0/2889380635 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4002d030 con 0x558c6a6134d0 2024-09-21T00:34:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.849+0000 7fad3cc00640 1 -- 172.21.15.27:0/2889380635 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9607+0+0 (secure 0 0 0) 0x7fad4002e070 con 0x558c6a6134d0 2024-09-21T00:34:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.849+0000 7fad3cc00640 1 -- 172.21.15.27:0/2889380635 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fad4002d030 con 0x558c6a6134d0 2024-09-21T00:34:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.849+0000 7fad5cea5740 10 client.26623 fetch_fsmap learned FSMap version 47 2024-09-21T00:34:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.849+0000 7fad5cea5740 1 -- 172.21.15.27:0/2889380635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=47}) -- 0x558c6a718620 con 0x558c6a6134d0 2024-09-21T00:34:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.850+0000 7fad3cc00640 1 -- 172.21.15.27:0/2889380635 <== mon.1 v2:172.21.15.98:3300/0 4 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fad40020cb0 con 0x558c6a6134d0 2024-09-21T00:34:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.850+0000 7fad5cea5740 10 client.26623 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:34:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.850+0000 7fad5cea5740 10 client.26623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:34:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.850+0000 7fad5cea5740 20 client.26623 populate_metadata read hostname 'smithi027' 2024-09-21T00:34:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.850+0000 7fad5cea5740 1 --2- 172.21.15.27:0/2889380635 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x558c6a5f2d80 0x558c6a9f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.851+0000 7fad5cea5740 4 client.26623 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:34:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.851+0000 7fad5cea5740 1 -- 172.21.15.27:0/2889380635 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c6a94c0b0 con 0x558c6a5f2d80 2024-09-21T00:34:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.854+0000 7fad4ea00640 1 --2- 172.21.15.27:0/2889380635 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x558c6a5f2d80 0x558c6a9f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.855+0000 7fad4ea00640 1 --2- 172.21.15.27:0/2889380635 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x558c6a5f2d80 0x558c6a9f6090 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e24ff68cd64ae463 server_cookie=ddf66b59c22d8a2f in_seq=0 out_seq=0 2024-09-21T00:34:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.855+0000 7fad3cc00640 10 client.26623 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.858+0000 7fad3cc00640 1 -- 172.21.15.27:0/2889380635 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558c6a94c0b0 con 0x558c6a5f2d80 2024-09-21T00:34:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.858+0000 7fad3cc00640 10 client.26623 handle_command_reply: tid=0 2024-09-21T00:34:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.858+0000 7fad5cea5740 1 client.26623 shutdown 2024-09-21T00:34:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.858+0000 7fad5cea5740 1 -- 172.21.15.27:0/2889380635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6134d0 msgr2=0x558c6a9e47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.858+0000 7fad5cea5740 1 --2- 172.21.15.27:0/2889380635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6134d0 0x558c6a9e47f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fad4000bae0 tx=0x7fad4000f4e0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.860+0000 7fad5cea5740 1 -- 172.21.15.27:0/2889380635 shutdown_connections 2024-09-21T00:34:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.860+0000 7fad5cea5740 1 --2- 172.21.15.27:0/2889380635 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x558c6a5f2d80 0x558c6a9f6090 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.860+0000 7fad5cea5740 1 --2- 172.21.15.27:0/2889380635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6341b0 0x558c6a9f0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.860+0000 7fad5cea5740 1 --2- 172.21.15.27:0/2889380635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6134d0 0x558c6a9e47f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.860+0000 7fad3cc00640 0 client.26623 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.860+0000 7fad5cea5740 1 --2- 172.21.15.27:0/2889380635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a60b2f0 0x558c6a9e42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.860+0000 7fad5cea5740 1 -- 172.21.15.27:0/2889380635 >> 172.21.15.27:0/2889380635 conn(0x558c6a628300 msgr2=0x558c6a9d9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.860+0000 7fad5cea5740 1 -- 172.21.15.27:0/2889380635 shutdown_connections 2024-09-21T00:34:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.862+0000 7fad5cea5740 1 -- 172.21.15.27:0/2889380635 wait complete. 2024-09-21T00:34:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.862+0000 7fad5cea5740 20 client.26623 trim_cache size 0 max 16384 2024-09-21T00:34:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.909+0000 7fad5cea5740 1 Processor -- start 2024-09-21T00:34:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.909+0000 7fad5cea5740 1 -- start start 2024-09-21T00:34:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.910+0000 7fad5cea5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a60b2f0 0x558c6a9d5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.910+0000 7fad5cea5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6134d0 0x558c6a9e5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.910+0000 7fad5cea5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6341b0 0x558c6a9e1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.910+0000 7fad5cea5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c012b00 con 0x558c6a60b2f0 2024-09-21T00:34:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.910+0000 7fad5cea5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c010310 con 0x558c6a6134d0 2024-09-21T00:34:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.910+0000 7fad5cea5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c00e180 con 0x558c6a6341b0 2024-09-21T00:34:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.910+0000 7fad4e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6134d0 0x558c6a9e5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.911+0000 7fad4e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6134d0 0x558c6a9e5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:46420/0 (socket says 172.21.15.27:46420) 2024-09-21T00:34:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.911+0000 7fad4e000640 1 -- 172.21.15.27:0/2977375998 learned_addr learned my addr 172.21.15.27:0/2977375998 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.911+0000 7fad4ea00640 1 --2- 172.21.15.27:0/2977375998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a60b2f0 0x558c6a9d5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.911+0000 7fad4f400640 1 --2- 172.21.15.27:0/2977375998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6341b0 0x558c6a9e1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.911+0000 7fad4ea00640 1 -- 172.21.15.27:0/2977375998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6341b0 msgr2=0x558c6a9e1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.911+0000 7fad4ea00640 1 --2- 172.21.15.27:0/2977375998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6341b0 0x558c6a9e1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.911+0000 7fad4ea00640 1 -- 172.21.15.27:0/2977375998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6134d0 msgr2=0x558c6a9e5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.911+0000 7fad4ea00640 1 --2- 172.21.15.27:0/2977375998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6134d0 0x558c6a9e5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.911+0000 7fad4ea00640 1 -- 172.21.15.27:0/2977375998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c6a9d7650 con 0x558c6a60b2f0 2024-09-21T00:34:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.912+0000 7fad4ea00640 1 --2- 172.21.15.27:0/2977375998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a60b2f0 0x558c6a9d5aa0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fad38009230 tx=0x7fad38000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176a9290cdceea96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.912+0000 7fad3f400640 1 -- 172.21.15.27:0/2977375998 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad38035040 con 0x558c6a60b2f0 2024-09-21T00:34:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.913+0000 7fad3f400640 1 -- 172.21.15.27:0/2977375998 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad380054f0 con 0x558c6a60b2f0 2024-09-21T00:34:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.913+0000 7fad3f400640 1 -- 172.21.15.27:0/2977375998 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad38035040 con 0x558c6a60b2f0 2024-09-21T00:34:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.914+0000 7fad5cea5740 1 -- 172.21.15.27:0/2977375998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a60b2f0 msgr2=0x558c6a9d5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.914+0000 7fad5cea5740 1 --2- 172.21.15.27:0/2977375998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a60b2f0 0x558c6a9d5aa0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fad38009230 tx=0x7fad38000bc0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.914+0000 7fad5cea5740 1 -- 172.21.15.27:0/2977375998 shutdown_connections 2024-09-21T00:34:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.914+0000 7fad5cea5740 1 --2- 172.21.15.27:0/2977375998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a6341b0 0x558c6a9e1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.914+0000 7fad5cea5740 1 --2- 172.21.15.27:0/2977375998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6134d0 0x558c6a9e5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.914+0000 7fad5cea5740 1 --2- 172.21.15.27:0/2977375998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a60b2f0 0x558c6a9d5aa0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.914+0000 7fad5cea5740 1 -- 172.21.15.27:0/2977375998 >> 172.21.15.27:0/2977375998 conn(0x558c6a628300 msgr2=0x558c6a9e57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.915+0000 7fad5cea5740 1 -- 172.21.15.27:0/2977375998 shutdown_connections 2024-09-21T00:34:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.915+0000 7fad5cea5740 1 -- 172.21.15.27:0/2977375998 wait complete. 2024-09-21T00:34:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.915+0000 7fad5cea5740 1 -- start start 2024-09-21T00:34:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.916+0000 7fad5cea5740 1 Processor -- start 2024-09-21T00:34:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.917+0000 7fad5cea5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a60b2f0 0x558c6a9ea5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.917+0000 7fad5cea5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6134d0 0x558c6a9eaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.917+0000 7fad5cea5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a6341b0 0x558c6a9eb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.917+0000 7fad4ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a60b2f0 0x558c6a9ea5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.917+0000 7fad4ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a60b2f0 0x558c6a9ea5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49046/0 (socket says 172.21.15.27:49046) 2024-09-21T00:34:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.917+0000 7fad4e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6134d0 0x558c6a9eaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.918+0000 7fad4e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6134d0 0x558c6a9eaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:46422/0 (socket says 172.21.15.27:46422) 2024-09-21T00:34:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.918+0000 7fad5cea5740 1 -- 172.21.15.27:0/375516401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c00e7d0 con 0x558c6a6341b0 2024-09-21T00:34:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.918+0000 7fad5cea5740 1 -- 172.21.15.27:0/375516401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c00fc40 con 0x558c6a6134d0 2024-09-21T00:34:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.918+0000 7fad5cea5740 1 -- 172.21.15.27:0/375516401 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c00eea0 con 0x558c6a60b2f0 2024-09-21T00:34:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.918+0000 7fad5cea5740 10 client.0 _ll_register_callbacks cb 0x7fad48021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:34:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.918+0000 7fad4e000640 1 -- 172.21.15.27:0/375516401 learned_addr learned my addr 172.21.15.27:0/375516401 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.918+0000 7fad4f400640 1 --2- 172.21.15.27:0/375516401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a6341b0 0x558c6a9eb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.919+0000 7fad4e000640 1 -- 172.21.15.27:0/375516401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a60b2f0 msgr2=0x558c6a9ea5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.919+0000 7fad4e000640 1 --2- 172.21.15.27:0/375516401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a60b2f0 0x558c6a9ea5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.919+0000 7fad4e000640 1 -- 172.21.15.27:0/375516401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a6341b0 msgr2=0x558c6a9eb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.919+0000 7fad4e000640 1 --2- 172.21.15.27:0/375516401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a6341b0 0x558c6a9eb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.919+0000 7fad4e000640 1 -- 172.21.15.27:0/375516401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c6aa5f870 con 0x558c6a6134d0 2024-09-21T00:34:08.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.921+0000 7fad4e000640 1 --2- 172.21.15.27:0/375516401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6134d0 0x558c6a9eaaf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fad40005270 tx=0x7fad4001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=181a6c257cfacdb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:08.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.921+0000 7fad3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:34:08.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.922+0000 7fad3cc00640 1 -- 172.21.15.27:0/375516401 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4000bae0 con 0x558c6a6134d0 2024-09-21T00:34:08.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.922+0000 7fad3cc00640 1 -- 172.21.15.27:0/375516401 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9607+0+0 (secure 0 0 0) 0x7fad40020480 con 0x558c6a6134d0 2024-09-21T00:34:08.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.922+0000 7fad5cea5740 1 -- 172.21.15.27:0/375516401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad380054f0 con 0x558c6a6134d0 2024-09-21T00:34:08.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.923+0000 7fad3cc00640 1 -- 172.21.15.27:0/375516401 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fad4000b430 con 0x558c6a6134d0 2024-09-21T00:34:08.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.923+0000 7fad5cea5740 10 client.26624 fetch_fsmap learned FSMap version 47 2024-09-21T00:34:08.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.923+0000 7fad5cea5740 1 -- 172.21.15.27:0/375516401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558c6aa60a90 con 0x558c6a6134d0 2024-09-21T00:34:08.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.924+0000 7fad3cc00640 1 -- 172.21.15.27:0/375516401 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad380054f0 con 0x558c6a6134d0 2024-09-21T00:34:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.924+0000 7fad3cc00640 1 -- 172.21.15.27:0/375516401 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fad40045060 con 0x558c6a6134d0 2024-09-21T00:34:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.925+0000 7fad5cea5740 10 client.26624 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:34:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.925+0000 7fad5cea5740 10 client.26624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:34:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.925+0000 7fad5cea5740 20 client.26624 populate_metadata read hostname 'smithi027' 2024-09-21T00:34:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.925+0000 7fad5cea5740 1 --2- 172.21.15.27:0/375516401 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x558c6a5c81c0 0x558c6aa63080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.925+0000 7fad5cea5740 4 client.26624 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:34:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.925+0000 7fad5cea5740 1 -- 172.21.15.27:0/375516401 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c6a9494e0 con 0x558c6a5c81c0 2024-09-21T00:34:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.926+0000 7fad4ea00640 1 --2- 172.21.15.27:0/375516401 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x558c6a5c81c0 0x558c6aa63080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.927+0000 7fad4ea00640 1 --2- 172.21.15.27:0/375516401 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x558c6a5c81c0 0x558c6aa63080 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a9bf9bda1e3dee0 server_cookie=7ea52a14a048b339 in_seq=0 out_seq=0 2024-09-21T00:34:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:08.927+0000 7fad3cc00640 10 client.26624 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:09.721 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:09 smithi027 ceph-mon[30745]: osdmap e109: 12 total, 12 up, 12 in 2024-09-21T00:34:09.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:09 smithi089 ceph-mon[34481]: osdmap e109: 12 total, 12 up, 12 in 2024-09-21T00:34:09.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:09 smithi098 ceph-mon[34725]: osdmap e109: 12 total, 12 up, 12 in 2024-09-21T00:34:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.839+0000 7fad3cc00640 1 -- 172.21.15.27:0/375516401 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c6a9494e0 con 0x558c6a5c81c0 2024-09-21T00:34:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.839+0000 7fad3cc00640 10 client.26624 handle_command_reply: tid=0 2024-09-21T00:34:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.839+0000 7fad5cea5740 1 client.26624 shutdown 2024-09-21T00:34:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.840+0000 7fad5cea5740 1 -- 172.21.15.27:0/375516401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6134d0 msgr2=0x558c6a9eaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.840+0000 7fad5cea5740 1 --2- 172.21.15.27:0/375516401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6134d0 0x558c6a9eaaf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fad40005270 tx=0x7fad4001c000 comp rx=0 tx=0).stop 2024-09-21T00:34:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.840+0000 7fad5cea5740 1 -- 172.21.15.27:0/375516401 shutdown_connections 2024-09-21T00:34:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.840+0000 7fad5cea5740 1 --2- 172.21.15.27:0/375516401 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x558c6a5c81c0 0x558c6aa63080 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.840+0000 7fad5cea5740 1 --2- 172.21.15.27:0/375516401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c6a6341b0 0x558c6a9eb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.840+0000 7fad5cea5740 1 --2- 172.21.15.27:0/375516401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c6a6134d0 0x558c6a9eaaf0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.840+0000 7fad5cea5740 1 --2- 172.21.15.27:0/375516401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c6a60b2f0 0x558c6a9ea5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.840+0000 7fad5cea5740 1 -- 172.21.15.27:0/375516401 >> 172.21.15.27:0/375516401 conn(0x558c6a628300 msgr2=0x558c6a9da620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.840+0000 7fad3cc00640 0 client.26624 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.841+0000 7fad5cea5740 1 -- 172.21.15.27:0/375516401 shutdown_connections 2024-09-21T00:34:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.841+0000 7fad5cea5740 1 -- 172.21.15.27:0/375516401 wait complete. 2024-09-21T00:34:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.841+0000 7fad5cea5740 20 client.26624 trim_cache size 0 max 16384 2024-09-21T00:34:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:34:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:34:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:34:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:34:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.842+0000 7fad55e00640 1 -- 172.21.15.27:0/1917282770 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fad340b6700 msgr2=0x7fad340b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.842+0000 7fad55e00640 1 --2- 172.21.15.27:0/1917282770 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fad340b6700 0x7fad340b8c10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fad50077d80 tx=0x7fad400138e0 comp rx=0 tx=0).stop 2024-09-21T00:34:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.842+0000 7fad55e00640 1 -- 172.21.15.27:0/1917282770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad501144f0 msgr2=0x7fad5006e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.842+0000 7fad55e00640 1 --2- 172.21.15.27:0/1917282770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad501144f0 0x7fad5006e6a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fad4800be20 tx=0x7fad4800bf10 comp rx=0 tx=0).stop 2024-09-21T00:34:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.843+0000 7fad55e00640 1 -- 172.21.15.27:0/1917282770 shutdown_connections 2024-09-21T00:34:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.843+0000 7fad55e00640 1 --2- 172.21.15.27:0/1917282770 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fad340b6700 0x7fad340b8c10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.843+0000 7fad55e00640 1 --2- 172.21.15.27:0/1917282770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad501144f0 0x7fad5006e6a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.843+0000 7fad55e00640 1 --2- 172.21.15.27:0/1917282770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad50102120 0x7fad5006e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.843+0000 7fad55e00640 1 --2- 172.21.15.27:0/1917282770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad501017d0 0x7fad50074e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.843+0000 7fad55e00640 1 -- 172.21.15.27:0/1917282770 >> 172.21.15.27:0/1917282770 conn(0x7fad50077730 msgr2=0x7fad501031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.843+0000 7fad55e00640 1 -- 172.21.15.27:0/1917282770 shutdown_connections 2024-09-21T00:34:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:09.843+0000 7fad55e00640 1 -- 172.21.15.27:0/1917282770 wait complete. 2024-09-21T00:34:10.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:10 smithi089 ceph-mon[34481]: pgmap v779: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 462 op/s 2024-09-21T00:34:10.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:10 smithi098 ceph-mon[34725]: pgmap v779: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 462 op/s 2024-09-21T00:34:10.774 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:10 smithi027 ceph-mon[30745]: pgmap v779: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 462 op/s 2024-09-21T00:34:12.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:12 smithi027 ceph-mon[30745]: pgmap v780: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 448 op/s 2024-09-21T00:34:12.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:12 smithi089 ceph-mon[34481]: pgmap v780: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 448 op/s 2024-09-21T00:34:12.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:12 smithi098 ceph-mon[34725]: pgmap v780: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 448 op/s 2024-09-21T00:34:14.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:14 smithi089 ceph-mon[34481]: pgmap v781: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 469 op/s 2024-09-21T00:34:14.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:14 smithi098 ceph-mon[34725]: pgmap v781: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 469 op/s 2024-09-21T00:34:14.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:14 smithi027 ceph-mon[30745]: pgmap v781: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 469 op/s 2024-09-21T00:34:16.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:16 smithi089 ceph-mon[34481]: pgmap v782: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 473 op/s 2024-09-21T00:34:16.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:16 smithi098 ceph-mon[34725]: pgmap v782: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 473 op/s 2024-09-21T00:34:16.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:16 smithi027 ceph-mon[30745]: pgmap v782: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 473 op/s 2024-09-21T00:34:18.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:18 smithi089 ceph-mon[34481]: pgmap v783: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 447 op/s 2024-09-21T00:34:18.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:18 smithi098 ceph-mon[34725]: pgmap v783: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 447 op/s 2024-09-21T00:34:18.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:18 smithi027 ceph-mon[30745]: pgmap v783: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 447 op/s 2024-09-21T00:34:19.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:19 smithi089 ceph-mon[34481]: pgmap v784: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 407 op/s 2024-09-21T00:34:19.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:19 smithi098 ceph-mon[34725]: pgmap v784: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 407 op/s 2024-09-21T00:34:19.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:19 smithi027 ceph-mon[30745]: pgmap v784: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 407 op/s 2024-09-21T00:34:22.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:21 smithi027 ceph-mon[30745]: pgmap v785: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 387 op/s 2024-09-21T00:34:22.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:21 smithi089 ceph-mon[34481]: pgmap v785: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 387 op/s 2024-09-21T00:34:22.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:21 smithi098 ceph-mon[34725]: pgmap v785: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 387 op/s 2024-09-21T00:34:24.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:23 smithi027 ceph-mon[30745]: pgmap v786: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-09-21T00:34:24.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:23 smithi089 ceph-mon[34481]: pgmap v786: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-09-21T00:34:24.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:23 smithi098 ceph-mon[34725]: pgmap v786: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-09-21T00:34:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.397+0000 7f27d2600640 1 -- 172.21.15.27:0/2742024862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc10a3c0 msgr2=0x7f27cc10a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.397+0000 7f27d2600640 1 --2- 172.21.15.27:0/2742024862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc10a3c0 0x7f27cc10a7d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009900 tx=0x7f27c002a3f0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.398+0000 7f27d2600640 1 -- 172.21.15.27:0/2742024862 shutdown_connections 2024-09-21T00:34:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.398+0000 7f27d2600640 1 --2- 172.21.15.27:0/2742024862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27cc116510 0x7f27cc118930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.398+0000 7f27d2600640 1 --2- 172.21.15.27:0/2742024862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27cc10ad10 0x7f27cc115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.398+0000 7f27d2600640 1 --2- 172.21.15.27:0/2742024862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc10a3c0 0x7f27cc10a7d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.398+0000 7f27d2600640 1 -- 172.21.15.27:0/2742024862 >> 172.21.15.27:0/2742024862 conn(0x7f27cc06e8f0 msgr2=0x7f27cc06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.398+0000 7f27d2600640 1 -- 172.21.15.27:0/2742024862 shutdown_connections 2024-09-21T00:34:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.398+0000 7f27d2600640 1 -- 172.21.15.27:0/2742024862 wait complete. 2024-09-21T00:34:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.400+0000 7f27d2600640 1 Processor -- start 2024-09-21T00:34:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.401+0000 7f27d2600640 1 -- start start 2024-09-21T00:34:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.401+0000 7f27d2600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27cc10a3c0 0x7f27cc1a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.401+0000 7f27d2600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27cc10ad10 0x7f27cc1a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.401+0000 7f27d2600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc116510 0x7f27cc1aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.401+0000 7f27d2600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc11c080 con 0x7f27cc116510 2024-09-21T00:34:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.401+0000 7f27d2600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc11bf00 con 0x7f27cc10ad10 2024-09-21T00:34:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.401+0000 7f27d2600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc11c200 con 0x7f27cc10a3c0 2024-09-21T00:34:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.402+0000 7f27d1200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27cc10a3c0 0x7f27cc1a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.402+0000 7f27cbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27cc10ad10 0x7f27cc1a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.402+0000 7f27d1200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27cc10a3c0 0x7f27cc1a4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38296/0 (socket says 172.21.15.27:38296) 2024-09-21T00:34:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.402+0000 7f27d1200640 1 -- 172.21.15.27:0/3388063636 learned_addr learned my addr 172.21.15.27:0/3388063636 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.402+0000 7f27d1c00640 1 --2- 172.21.15.27:0/3388063636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc116510 0x7f27cc1aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.402+0000 7f27d1200640 1 -- 172.21.15.27:0/3388063636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27cc10ad10 msgr2=0x7f27cc1a5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.402+0000 7f27d1200640 1 --2- 172.21.15.27:0/3388063636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27cc10ad10 0x7f27cc1a5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.402+0000 7f27d1200640 1 -- 172.21.15.27:0/3388063636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc116510 msgr2=0x7f27cc1aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.402+0000 7f27d1200640 1 --2- 172.21.15.27:0/3388063636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc116510 0x7f27cc1aa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.402+0000 7f27d1200640 1 -- 172.21.15.27:0/3388063636 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27cc1aab90 con 0x7f27cc10a3c0 2024-09-21T00:34:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.403+0000 7f27d1200640 1 --2- 172.21.15.27:0/3388063636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27cc10a3c0 0x7f27cc1a4c60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f27c0005ab0 tx=0x7f27c0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.403+0000 7f27c9600640 1 -- 172.21.15.27:0/3388063636 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c003f070 con 0x7f27cc10a3c0 2024-09-21T00:34:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.403+0000 7f27c9600640 1 -- 172.21.15.27:0/3388063636 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c0031b60 con 0x7f27cc10a3c0 2024-09-21T00:34:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.403+0000 7f27d2600640 1 -- 172.21.15.27:0/3388063636 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27cc1aadc0 con 0x7f27cc10a3c0 2024-09-21T00:34:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.403+0000 7f27d2600640 1 -- 172.21.15.27:0/3388063636 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27cc1ab2a0 con 0x7f27cc10a3c0 2024-09-21T00:34:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.404+0000 7f27c9600640 1 -- 172.21.15.27:0/3388063636 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0035030 con 0x7f27cc10a3c0 2024-09-21T00:34:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.411+0000 7f27dacd3740 1 Processor -- start 2024-09-21T00:34:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.412+0000 7f27c9600640 1 -- 172.21.15.27:0/3388063636 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f27c0042070 con 0x7f27cc10a3c0 2024-09-21T00:34:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.412+0000 7f27dacd3740 1 -- start start 2024-09-21T00:34:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.413+0000 7f27dacd3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862f81b0 0x55d58669ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.413+0000 7f27dacd3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862cf2f0 0x55d58669d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.413+0000 7f27dacd3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 0x55d58669d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.413+0000 7f27dacd3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d5866126b0 con 0x55d5862f81b0 2024-09-21T00:34:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.413+0000 7f27dacd3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d586601ae0 con 0x55d5862cf2f0 2024-09-21T00:34:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.413+0000 7f27dacd3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d5866058f0 con 0x55d5862d74d0 2024-09-21T00:34:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.414+0000 7f27cbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 0x55d58669d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.414+0000 7f27cbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 0x55d58669d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38310/0 (socket says 172.21.15.27:38310) 2024-09-21T00:34:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.414+0000 7f27cbe00640 1 -- 172.21.15.27:0/1893565078 learned_addr learned my addr 172.21.15.27:0/1893565078 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.414+0000 7f27d1c00640 1 --2- 172.21.15.27:0/1893565078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862f81b0 0x55d58669ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.415+0000 7f27cbe00640 1 -- 172.21.15.27:0/1893565078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862cf2f0 msgr2=0x55d58669d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.415+0000 7f27cbe00640 1 --2- 172.21.15.27:0/1893565078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862cf2f0 0x55d58669d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.415+0000 7f27cbe00640 1 -- 172.21.15.27:0/1893565078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862f81b0 msgr2=0x55d58669ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.415+0000 7f27cbe00640 1 --2- 172.21.15.27:0/1893565078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862f81b0 0x55d58669ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.415+0000 7f27cbe00640 1 -- 172.21.15.27:0/1893565078 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5865ce780 con 0x55d5862d74d0 2024-09-21T00:34:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.415+0000 7f27d1c00640 1 --2- 172.21.15.27:0/1893565078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862f81b0 0x55d58669ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:34:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.415+0000 7f27c9600640 1 --2- 172.21.15.27:0/3388063636 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f27b40b6770 0x7f27b40b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.415+0000 7f27cbe00640 1 --2- 172.21.15.27:0/1893565078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 0x55d58669d8b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009c50 tx=0x7f27b8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=582897b1c15d507b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.415+0000 7f27c9600640 1 -- 172.21.15.27:0/3388063636 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9607+0+0 (secure 0 0 0) 0x7f27c0108370 con 0x7f27cc10a3c0 2024-09-21T00:34:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.416+0000 7f27d1c00640 1 --2- 172.21.15.27:0/3388063636 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f27b40b6770 0x7f27b40b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.416+0000 7f27be000640 1 -- 172.21.15.27:0/1893565078 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8004230 con 0x55d5862d74d0 2024-09-21T00:34:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.416+0000 7f27be000640 1 -- 172.21.15.27:0/1893565078 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b80043d0 con 0x55d5862d74d0 2024-09-21T00:34:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.416+0000 7f27be000640 1 -- 172.21.15.27:0/1893565078 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8004cd0 con 0x55d5862d74d0 2024-09-21T00:34:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.418+0000 7f27dacd3740 1 -- 172.21.15.27:0/1893565078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 msgr2=0x55d58669d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.418+0000 7f27d1c00640 1 --2- 172.21.15.27:0/3388063636 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f27b40b6770 0x7f27b40b8c80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f27c4005fd0 tx=0x7f27c4009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.418+0000 7f27dacd3740 1 --2- 172.21.15.27:0/1893565078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 0x55d58669d8b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009c50 tx=0x7f27b8011810 comp rx=0 tx=0).stop 2024-09-21T00:34:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.420+0000 7f27dacd3740 1 -- 172.21.15.27:0/1893565078 shutdown_connections 2024-09-21T00:34:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.420+0000 7f27dacd3740 1 --2- 172.21.15.27:0/1893565078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 0x55d58669d8b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.420+0000 7f27dacd3740 1 --2- 172.21.15.27:0/1893565078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862cf2f0 0x55d58669d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.420+0000 7f27dacd3740 1 --2- 172.21.15.27:0/1893565078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862f81b0 0x55d58669ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.420+0000 7f27dacd3740 1 -- 172.21.15.27:0/1893565078 >> 172.21.15.27:0/1893565078 conn(0x55d5862ec300 msgr2=0x55d58669ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.421+0000 7f27dacd3740 1 -- 172.21.15.27:0/1893565078 shutdown_connections 2024-09-21T00:34:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.421+0000 7f27dacd3740 1 -- 172.21.15.27:0/1893565078 wait complete. 2024-09-21T00:34:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.422+0000 7f27dacd3740 1 -- start start 2024-09-21T00:34:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.422+0000 7f27dacd3740 1 Processor -- start 2024-09-21T00:34:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.426+0000 7f27dacd3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862cf2f0 0x55d5866a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.426+0000 7f27dacd3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862d74d0 0x55d5866a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.426+0000 7f27d1200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862cf2f0 0x55d5866a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.427+0000 7f27dacd3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862f81b0 0x55d5866b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.427+0000 7f27cbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862d74d0 0x55d5866a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.427+0000 7f27d1200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862cf2f0 0x55d5866a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49576/0 (socket says 172.21.15.27:49576) 2024-09-21T00:34:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.427+0000 7f27d1200640 1 -- 172.21.15.27:0/2799030204 learned_addr learned my addr 172.21.15.27:0/2799030204 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.427+0000 7f27d1c00640 1 --2- 172.21.15.27:0/2799030204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862f81b0 0x55d5866b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.427+0000 7f27dacd3740 1 -- 172.21.15.27:0/2799030204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d586539ab0 con 0x55d5862cf2f0 2024-09-21T00:34:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.427+0000 7f27dacd3740 1 -- 172.21.15.27:0/2799030204 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d5865a8570 con 0x55d5862d74d0 2024-09-21T00:34:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.427+0000 7f27dacd3740 1 -- 172.21.15.27:0/2799030204 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d5866ab5b0 con 0x55d5862f81b0 2024-09-21T00:34:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.427+0000 7f27dacd3740 10 client.0 _ll_register_callbacks cb 0x55d5865b7100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:34:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.428+0000 7f27d1c00640 1 -- 172.21.15.27:0/2799030204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862d74d0 msgr2=0x55d5866a8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.428+0000 7f27d1c00640 1 --2- 172.21.15.27:0/2799030204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862d74d0 0x55d5866a8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.429+0000 7f27d1c00640 1 -- 172.21.15.27:0/2799030204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862cf2f0 msgr2=0x55d5866a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.429+0000 7f27d1c00640 1 --2- 172.21.15.27:0/2799030204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862cf2f0 0x55d5866a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.429+0000 7f27d1c00640 1 -- 172.21.15.27:0/2799030204 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d586363790 con 0x55d5862f81b0 2024-09-21T00:34:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.429+0000 7f27d1200640 1 --2- 172.21.15.27:0/2799030204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862cf2f0 0x55d5866a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:34:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.429+0000 7f27d1c00640 1 --2- 172.21.15.27:0/2799030204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862f81b0 0x55d5866b48a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f27c4011010 tx=0x7f27c4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b9ee6ae2c16be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.429+0000 7f27b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:34:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.430+0000 7f27b2a00640 1 -- 172.21.15.27:0/2799030204 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c4023070 con 0x55d5862f81b0 2024-09-21T00:34:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.430+0000 7f27b2a00640 1 -- 172.21.15.27:0/2799030204 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9607+0+0 (secure 0 0 0) 0x7f27c4011930 con 0x55d5862f81b0 2024-09-21T00:34:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.430+0000 7f27dacd3740 1 -- 172.21.15.27:0/2799030204 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b80043d0 con 0x55d5862f81b0 2024-09-21T00:34:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.430+0000 7f27b2a00640 1 -- 172.21.15.27:0/2799030204 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f27c402f050 con 0x55d5862f81b0 2024-09-21T00:34:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.431+0000 7f27dacd3740 10 client.26641 fetch_fsmap learned FSMap version 47 2024-09-21T00:34:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.431+0000 7f27dacd3740 1 -- 172.21.15.27:0/2799030204 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d5863dc620 con 0x55d5862f81b0 2024-09-21T00:34:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.437+0000 7f27b2a00640 1 -- 172.21.15.27:0/2799030204 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b80043d0 con 0x55d5862f81b0 2024-09-21T00:34:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.437+0000 7f27b2a00640 1 -- 172.21.15.27:0/2799030204 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f27c400f040 con 0x55d5862f81b0 2024-09-21T00:34:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.437+0000 7f27dacd3740 10 client.26641 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:34:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.437+0000 7f27dacd3740 10 client.26641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:34:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.437+0000 7f27dacd3740 20 client.26641 populate_metadata read hostname 'smithi027' 2024-09-21T00:34:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.437+0000 7f27dacd3740 1 --2- 172.21.15.27:0/2799030204 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d5862b6d80 0x55d5866ba6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.437+0000 7f27dacd3740 4 client.26641 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:34:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.437+0000 7f27dacd3740 1 -- 172.21.15.27:0/2799030204 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d586614600 con 0x55d5862b6d80 2024-09-21T00:34:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.437+0000 7f27d1200640 1 --2- 172.21.15.27:0/2799030204 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d5862b6d80 0x55d5866ba6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.438+0000 7f27b2a00640 10 client.26641 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:24.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.442+0000 7f27d1200640 1 --2- 172.21.15.27:0/2799030204 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d5862b6d80 0x55d5866ba6f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a37b8c8afaa47610 server_cookie=98c6929fc670aa81 in_seq=0 out_seq=0 2024-09-21T00:34:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.443+0000 7f27b2a00640 1 -- 172.21.15.27:0/2799030204 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d586614600 con 0x55d5862b6d80 2024-09-21T00:34:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.443+0000 7f27b2a00640 10 client.26641 handle_command_reply: tid=0 2024-09-21T00:34:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.443+0000 7f27dacd3740 1 client.26641 shutdown 2024-09-21T00:34:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.445+0000 7f27dacd3740 1 -- 172.21.15.27:0/2799030204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862f81b0 msgr2=0x55d5866b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.445+0000 7f27dacd3740 1 --2- 172.21.15.27:0/2799030204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862f81b0 0x55d5866b48a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f27c4011010 tx=0x7f27c4004ab0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.445+0000 7f27dacd3740 1 -- 172.21.15.27:0/2799030204 shutdown_connections 2024-09-21T00:34:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.445+0000 7f27dacd3740 1 --2- 172.21.15.27:0/2799030204 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d5862b6d80 0x55d5866ba6f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.445+0000 7f27dacd3740 1 --2- 172.21.15.27:0/2799030204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862f81b0 0x55d5866b48a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.445+0000 7f27dacd3740 1 --2- 172.21.15.27:0/2799030204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862d74d0 0x55d5866a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.445+0000 7f27dacd3740 1 --2- 172.21.15.27:0/2799030204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862cf2f0 0x55d5866a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.445+0000 7f27dacd3740 1 -- 172.21.15.27:0/2799030204 >> 172.21.15.27:0/2799030204 conn(0x55d5862ec300 msgr2=0x55d58669de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.446+0000 7f27b2a00640 0 client.26641 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.447+0000 7f27dacd3740 1 -- 172.21.15.27:0/2799030204 shutdown_connections 2024-09-21T00:34:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.447+0000 7f27dacd3740 1 -- 172.21.15.27:0/2799030204 wait complete. 2024-09-21T00:34:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.447+0000 7f27dacd3740 20 client.26641 trim_cache size 0 max 16384 2024-09-21T00:34:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.480+0000 7f27dacd3740 1 Processor -- start 2024-09-21T00:34:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.480+0000 7f27dacd3740 1 -- start start 2024-09-21T00:34:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.481+0000 7f27dacd3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862cf2f0 0x55d5866aa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.481+0000 7f27dacd3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 0x55d5866b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.482+0000 7f27dacd3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862f81b0 0x55d5866b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.482+0000 7f27dacd3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1c2550 con 0x55d5862cf2f0 2024-09-21T00:34:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.482+0000 7f27dacd3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1bfd60 con 0x55d5862f81b0 2024-09-21T00:34:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.482+0000 7f27dacd3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1bdc70 con 0x55d5862d74d0 2024-09-21T00:34:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.482+0000 7f27cbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 0x55d5866b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.482+0000 7f27d1200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862cf2f0 0x55d5866aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.482+0000 7f27cbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 0x55d5866b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38330/0 (socket says 172.21.15.27:38330) 2024-09-21T00:34:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.482+0000 7f27cbe00640 1 -- 172.21.15.27:0/1178056003 learned_addr learned my addr 172.21.15.27:0/1178056003 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.482+0000 7f27d1c00640 1 --2- 172.21.15.27:0/1178056003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862f81b0 0x55d5866b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.482+0000 7f27cbe00640 1 -- 172.21.15.27:0/1178056003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862f81b0 msgr2=0x55d5866b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.482+0000 7f27cbe00640 1 --2- 172.21.15.27:0/1178056003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862f81b0 0x55d5866b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.482+0000 7f27cbe00640 1 -- 172.21.15.27:0/1178056003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862cf2f0 msgr2=0x55d5866aa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.482+0000 7f27cbe00640 1 --2- 172.21.15.27:0/1178056003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862cf2f0 0x55d5866aa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.482+0000 7f27cbe00640 1 -- 172.21.15.27:0/1178056003 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5866a8e60 con 0x55d5862d74d0 2024-09-21T00:34:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.486+0000 7f27cbe00640 1 --2- 172.21.15.27:0/1178056003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 0x55d5866b47a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f27b80058c0 tx=0x7f27b8009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=426b666328718c58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.486+0000 7f27bd600640 1 -- 172.21.15.27:0/1178056003 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8028070 con 0x55d5862d74d0 2024-09-21T00:34:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.486+0000 7f27bd600640 1 -- 172.21.15.27:0/1178056003 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b801f040 con 0x55d5862d74d0 2024-09-21T00:34:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.486+0000 7f27bd600640 1 -- 172.21.15.27:0/1178056003 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8019810 con 0x55d5862d74d0 2024-09-21T00:34:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.488+0000 7f27dacd3740 1 -- 172.21.15.27:0/1178056003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 msgr2=0x55d5866b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.488+0000 7f27dacd3740 1 --2- 172.21.15.27:0/1178056003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 0x55d5866b47a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f27b80058c0 tx=0x7f27b8009f20 comp rx=0 tx=0).stop 2024-09-21T00:34:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.489+0000 7f27dacd3740 1 -- 172.21.15.27:0/1178056003 shutdown_connections 2024-09-21T00:34:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.489+0000 7f27dacd3740 1 --2- 172.21.15.27:0/1178056003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862f81b0 0x55d5866b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.489+0000 7f27dacd3740 1 --2- 172.21.15.27:0/1178056003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 0x55d5866b47a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.489+0000 7f27dacd3740 1 --2- 172.21.15.27:0/1178056003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862cf2f0 0x55d5866aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.489+0000 7f27dacd3740 1 -- 172.21.15.27:0/1178056003 >> 172.21.15.27:0/1178056003 conn(0x55d5862ec300 msgr2=0x55d5866a9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.489+0000 7f27dacd3740 1 -- 172.21.15.27:0/1178056003 shutdown_connections 2024-09-21T00:34:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.491+0000 7f27dacd3740 1 -- 172.21.15.27:0/1178056003 wait complete. 2024-09-21T00:34:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.491+0000 7f27dacd3740 1 -- start start 2024-09-21T00:34:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.492+0000 7f27dacd3740 1 Processor -- start 2024-09-21T00:34:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.493+0000 7f27dacd3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862cf2f0 0x55d5866adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.493+0000 7f27dacd3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 0x55d5866ae010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.493+0000 7f27dacd3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862f81b0 0x55d5867230f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.493+0000 7f27dacd3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1be2c0 con 0x55d5862f81b0 2024-09-21T00:34:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.493+0000 7f27dacd3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1bf690 con 0x55d5862cf2f0 2024-09-21T00:34:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.493+0000 7f27dacd3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1be910 con 0x55d5862d74d0 2024-09-21T00:34:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.493+0000 7f27dacd3740 10 client.0 _ll_register_callbacks cb 0x7f27c4016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:34:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.494+0000 7f27d1200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862cf2f0 0x55d5866adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.494+0000 7f27d1200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862cf2f0 0x55d5866adad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45748/0 (socket says 172.21.15.27:45748) 2024-09-21T00:34:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.494+0000 7f27d1200640 1 -- 172.21.15.27:0/2398535161 learned_addr learned my addr 172.21.15.27:0/2398535161 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.494+0000 7f27d1c00640 1 --2- 172.21.15.27:0/2398535161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862f81b0 0x55d5867230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.494+0000 7f27d1200640 1 -- 172.21.15.27:0/2398535161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 msgr2=0x55d5866ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.494+0000 7f27d1200640 1 --2- 172.21.15.27:0/2398535161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 0x55d5866ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.494+0000 7f27d1200640 1 -- 172.21.15.27:0/2398535161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862f81b0 msgr2=0x55d5867230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.494+0000 7f27d1200640 1 --2- 172.21.15.27:0/2398535161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862f81b0 0x55d5867230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.494+0000 7f27d1200640 1 -- 172.21.15.27:0/2398535161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5866ae890 con 0x55d5862cf2f0 2024-09-21T00:34:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.495+0000 7f27d1200640 1 --2- 172.21.15.27:0/2398535161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862cf2f0 0x55d5866adad0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f27c00cbe80 tx=0x7f27c002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=415d5dadabee264c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.495+0000 7f27b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:34:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.495+0000 7f27b2a00640 1 -- 172.21.15.27:0/2398535161 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0049040 con 0x55d5862cf2f0 2024-09-21T00:34:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.495+0000 7f27dacd3740 1 -- 172.21.15.27:0/2398535161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b801f040 con 0x55d5862cf2f0 2024-09-21T00:34:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.496+0000 7f27b2a00640 1 -- 172.21.15.27:0/2398535161 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9607+0+0 (secure 0 0 0) 0x7f27c00cae80 con 0x55d5862cf2f0 2024-09-21T00:34:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.496+0000 7f27b2a00640 1 -- 172.21.15.27:0/2398535161 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f27c00ce720 con 0x55d5862cf2f0 2024-09-21T00:34:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.496+0000 7f27dacd3740 10 client.26639 fetch_fsmap learned FSMap version 47 2024-09-21T00:34:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.496+0000 7f27dacd3740 1 -- 172.21.15.27:0/2398535161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d5867246e0 con 0x55d5862cf2f0 2024-09-21T00:34:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.497+0000 7f27b2a00640 1 -- 172.21.15.27:0/2398535161 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b801f040 con 0x55d5862cf2f0 2024-09-21T00:34:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.497+0000 7f27b2a00640 1 -- 172.21.15.27:0/2398535161 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f27c003b460 con 0x55d5862cf2f0 2024-09-21T00:34:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.497+0000 7f27dacd3740 10 client.26639 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:34:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.498+0000 7f27dacd3740 10 client.26639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:34:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.498+0000 7f27dacd3740 20 client.26639 populate_metadata read hostname 'smithi027' 2024-09-21T00:34:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.498+0000 7f27dacd3740 1 --2- 172.21.15.27:0/2398535161 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d58628c1c0 0x55d586726e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.498+0000 7f27dacd3740 4 client.26639 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:34:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.498+0000 7f27dacd3740 1 -- 172.21.15.27:0/2398535161 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5866100b0 con 0x55d58628c1c0 2024-09-21T00:34:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.498+0000 7f27cbe00640 1 --2- 172.21.15.27:0/2398535161 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d58628c1c0 0x55d586726e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.499+0000 7f27cbe00640 1 --2- 172.21.15.27:0/2398535161 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d58628c1c0 0x55d586726e60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a3883a2bfb43bbb server_cookie=c128cec0379b5a1f in_seq=0 out_seq=0 2024-09-21T00:34:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:24.499+0000 7f27b2a00640 10 client.26639 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.156+0000 7f27b2a00640 1 -- 172.21.15.27:0/2398535161 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d5866100b0 con 0x55d58628c1c0 2024-09-21T00:34:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.157+0000 7f27b2a00640 10 client.26639 handle_command_reply: tid=0 2024-09-21T00:34:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.157+0000 7f27dacd3740 1 client.26639 shutdown 2024-09-21T00:34:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.158+0000 7f27dacd3740 1 -- 172.21.15.27:0/2398535161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862cf2f0 msgr2=0x55d5866adad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.158+0000 7f27dacd3740 1 --2- 172.21.15.27:0/2398535161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862cf2f0 0x55d5866adad0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f27c00cbe80 tx=0x7f27c002a3f0 comp rx=0 tx=0).stop 2024-09-21T00:34:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.158+0000 7f27dacd3740 1 -- 172.21.15.27:0/2398535161 shutdown_connections 2024-09-21T00:34:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.158+0000 7f27dacd3740 1 --2- 172.21.15.27:0/2398535161 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d58628c1c0 0x55d586726e60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.158+0000 7f27dacd3740 1 --2- 172.21.15.27:0/2398535161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d5862f81b0 0x55d5867230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.158+0000 7f27b2a00640 0 client.26639 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.158+0000 7f27dacd3740 1 --2- 172.21.15.27:0/2398535161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5862d74d0 0x55d5866ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.159+0000 7f27dacd3740 1 --2- 172.21.15.27:0/2398535161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5862cf2f0 0x55d5866adad0 secure :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f27c00cbe80 tx=0x7f27c002a3f0 comp rx=0 tx=0).stop 2024-09-21T00:34:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.159+0000 7f27dacd3740 1 -- 172.21.15.27:0/2398535161 >> 172.21.15.27:0/2398535161 conn(0x55d5862ec300 msgr2=0x55d58669dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.159+0000 7f27dacd3740 1 -- 172.21.15.27:0/2398535161 shutdown_connections 2024-09-21T00:34:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.159+0000 7f27dacd3740 1 -- 172.21.15.27:0/2398535161 wait complete. 2024-09-21T00:34:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.159+0000 7f27dacd3740 20 client.26639 trim_cache size 0 max 16384 2024-09-21T00:34:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:34:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:34:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:34:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:34:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.160+0000 7f27d2600640 1 -- 172.21.15.27:0/3388063636 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f27b40b6770 msgr2=0x7f27b40b8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.160+0000 7f27d2600640 1 --2- 172.21.15.27:0/3388063636 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f27b40b6770 0x7f27b40b8c80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f27c4005fd0 tx=0x7f27c4009500 comp rx=0 tx=0).stop 2024-09-21T00:34:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.160+0000 7f27d2600640 1 -- 172.21.15.27:0/3388063636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27cc10a3c0 msgr2=0x7f27cc1a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.160+0000 7f27d2600640 1 --2- 172.21.15.27:0/3388063636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27cc10a3c0 0x7f27cc1a4c60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f27c0005ab0 tx=0x7f27c0003c10 comp rx=0 tx=0).stop 2024-09-21T00:34:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.160+0000 7f27d2600640 1 -- 172.21.15.27:0/3388063636 shutdown_connections 2024-09-21T00:34:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.160+0000 7f27d2600640 1 --2- 172.21.15.27:0/3388063636 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f27b40b6770 0x7f27b40b8c80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.160+0000 7f27d2600640 1 --2- 172.21.15.27:0/3388063636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc116510 0x7f27cc1aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.160+0000 7f27d2600640 1 --2- 172.21.15.27:0/3388063636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27cc10ad10 0x7f27cc1a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.160+0000 7f27d2600640 1 --2- 172.21.15.27:0/3388063636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27cc10a3c0 0x7f27cc1a4c60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.160+0000 7f27d2600640 1 -- 172.21.15.27:0/3388063636 >> 172.21.15.27:0/3388063636 conn(0x7f27cc06e8f0 msgr2=0x7f27cc112380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.161+0000 7f27d2600640 1 -- 172.21.15.27:0/3388063636 shutdown_connections 2024-09-21T00:34:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:25.161+0000 7f27d2600640 1 -- 172.21.15.27:0/3388063636 wait complete. 2024-09-21T00:34:26.067 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:34:25 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:34:25.895+0000 7fbf41e00640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-21T00:34:26.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:25 smithi027 ceph-mon[30745]: pgmap v787: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-09-21T00:34:26.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:25 smithi089 ceph-mon[34481]: pgmap v787: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-09-21T00:34:26.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:25 smithi098 ceph-mon[34725]: pgmap v787: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-09-21T00:34:28.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:27 smithi089 ceph-mon[34481]: pgmap v788: 129 pgs: 20 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 42 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 262 op/s 2024-09-21T00:34:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:27 smithi098 ceph-mon[34725]: pgmap v788: 129 pgs: 20 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 42 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 262 op/s 2024-09-21T00:34:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:27 smithi027 ceph-mon[30745]: pgmap v788: 129 pgs: 20 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 42 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 262 op/s 2024-09-21T00:34:30.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:29 smithi089 ceph-mon[34481]: pgmap v789: 129 pgs: 20 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 42 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 285 op/s 2024-09-21T00:34:30.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:29 smithi098 ceph-mon[34725]: pgmap v789: 129 pgs: 20 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 42 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 285 op/s 2024-09-21T00:34:30.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:29 smithi027 ceph-mon[30745]: pgmap v789: 129 pgs: 20 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 42 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 285 op/s 2024-09-21T00:34:32.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:31 smithi089 ceph-mon[34481]: pgmap v790: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 927 KiB/s wr, 313 op/s 2024-09-21T00:34:32.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:31 smithi098 ceph-mon[34725]: pgmap v790: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 927 KiB/s wr, 313 op/s 2024-09-21T00:34:32.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:31 smithi027 ceph-mon[30745]: pgmap v790: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 927 KiB/s wr, 313 op/s 2024-09-21T00:34:34.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:33 smithi089 ceph-mon[34481]: pgmap v791: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.0 MiB/s wr, 283 op/s 2024-09-21T00:34:34.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:33 smithi098 ceph-mon[34725]: pgmap v791: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.0 MiB/s wr, 283 op/s 2024-09-21T00:34:34.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:33 smithi027 ceph-mon[30745]: pgmap v791: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.0 MiB/s wr, 283 op/s 2024-09-21T00:34:36.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:35 smithi089 ceph-mon[34481]: pgmap v792: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 285 op/s 2024-09-21T00:34:36.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:35 smithi098 ceph-mon[34725]: pgmap v792: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 285 op/s 2024-09-21T00:34:36.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:35 smithi027 ceph-mon[30745]: pgmap v792: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 285 op/s 2024-09-21T00:34:38.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:37 smithi089 ceph-mon[34481]: pgmap v793: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 278 op/s 2024-09-21T00:34:38.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:37 smithi098 ceph-mon[34725]: pgmap v793: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 278 op/s 2024-09-21T00:34:38.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:37 smithi027 ceph-mon[30745]: pgmap v793: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 278 op/s 2024-09-21T00:34:38.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.574+0000 7f8c2d400640 1 -- 172.21.15.27:0/1243685405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c28113c40 msgr2=0x7f8c28114050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:38.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.574+0000 7f8c2d400640 1 --2- 172.21.15.27:0/1243685405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c28113c40 0x7f8c28114050 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009900 tx=0x7f8c1802a410 comp rx=0 tx=0).stop 2024-09-21T00:34:38.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.574+0000 7f8c2d400640 1 -- 172.21.15.27:0/1243685405 shutdown_connections 2024-09-21T00:34:38.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.575+0000 7f8c2d400640 1 --2- 172.21.15.27:0/1243685405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c28114590 0x7f8c2811adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.575+0000 7f8c2d400640 1 --2- 172.21.15.27:0/1243685405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c28113c40 0x7f8c28114050 secure :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009900 tx=0x7f8c1802a410 comp rx=0 tx=0).stop 2024-09-21T00:34:38.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.575+0000 7f8c2d400640 1 --2- 172.21.15.27:0/1243685405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2810e310 0x7f8c2810e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.575+0000 7f8c2d400640 1 -- 172.21.15.27:0/1243685405 >> 172.21.15.27:0/1243685405 conn(0x7f8c2806e610 msgr2=0x7f8c2810c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.576+0000 7f8c2d400640 1 -- 172.21.15.27:0/1243685405 shutdown_connections 2024-09-21T00:34:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.576+0000 7f8c2d400640 1 -- 172.21.15.27:0/1243685405 wait complete. 2024-09-21T00:34:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.577+0000 7f8c2d400640 1 Processor -- start 2024-09-21T00:34:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.578+0000 7f8c2d400640 1 -- start start 2024-09-21T00:34:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.579+0000 7f8c2d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2810e310 0x7f8c28119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.579+0000 7f8c2d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c28114590 0x7f8c2811a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.579+0000 7f8c2d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c2811e1f0 0x7f8c2811e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.579+0000 7f8c2d400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2806fd50 con 0x7f8c2810e310 2024-09-21T00:34:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.579+0000 7f8c2d400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2806fbd0 con 0x7f8c2811e1f0 2024-09-21T00:34:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.579+0000 7f8c2d400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2806fed0 con 0x7f8c28114590 2024-09-21T00:34:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.579+0000 7f8c27400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c28114590 0x7f8c2811a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.580+0000 7f8c27e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2810e310 0x7f8c28119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.580+0000 7f8c27400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c28114590 0x7f8c2811a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55588/0 (socket says 172.21.15.27:55588) 2024-09-21T00:34:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.580+0000 7f8c27e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2810e310 0x7f8c28119f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56634/0 (socket says 172.21.15.27:56634) 2024-09-21T00:34:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.580+0000 7f8c27400640 1 -- 172.21.15.27:0/1809527374 learned_addr learned my addr 172.21.15.27:0/1809527374 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.580+0000 7f8c27e00640 1 -- 172.21.15.27:0/1809527374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c28114590 msgr2=0x7f8c2811a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.580+0000 7f8c27e00640 1 --2- 172.21.15.27:0/1809527374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c28114590 0x7f8c2811a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.580+0000 7f8c27e00640 1 -- 172.21.15.27:0/1809527374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c2811e1f0 msgr2=0x7f8c2811e600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.580+0000 7f8c27e00640 1 --2- 172.21.15.27:0/1809527374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c2811e1f0 0x7f8c2811e600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.580+0000 7f8c27e00640 1 -- 172.21.15.27:0/1809527374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2811ed50 con 0x7f8c2810e310 2024-09-21T00:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.581+0000 7f8c27e00640 1 --2- 172.21.15.27:0/1809527374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2810e310 0x7f8c28119f90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000d6d0 tx=0x7f8c1000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.581+0000 7f8c24c00640 1 -- 172.21.15.27:0/1809527374 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c10017070 con 0x7f8c2810e310 2024-09-21T00:34:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.581+0000 7f8c2d400640 1 -- 172.21.15.27:0/1809527374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c281245d0 con 0x7f8c2810e310 2024-09-21T00:34:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.582+0000 7f8c24c00640 1 -- 172.21.15.27:0/1809527374 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c10009820 con 0x7f8c2810e310 2024-09-21T00:34:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.582+0000 7f8c24c00640 1 -- 172.21.15.27:0/1809527374 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c10020060 con 0x7f8c2810e310 2024-09-21T00:34:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.582+0000 7f8c2d400640 1 -- 172.21.15.27:0/1809527374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c28124a30 con 0x7f8c2810e310 2024-09-21T00:34:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.589+0000 7f8c24c00640 1 -- 172.21.15.27:0/1809527374 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8c10009c80 con 0x7f8c2810e310 2024-09-21T00:34:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.593+0000 7f8c24c00640 1 --2- 172.21.15.27:0/1809527374 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8c0c0b68f0 0x7f8c0c0b8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.593+0000 7f8c27400640 1 --2- 172.21.15.27:0/1809527374 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8c0c0b68f0 0x7f8c0c0b8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.593+0000 7f8c24c00640 1 -- 172.21.15.27:0/1809527374 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9607+0+0 (secure 0 0 0) 0x7f8c100e7240 con 0x7f8c2810e310 2024-09-21T00:34:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.594+0000 7f8c359ff740 1 Processor -- start 2024-09-21T00:34:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.594+0000 7f8c359ff740 1 -- start start 2024-09-21T00:34:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.595+0000 7f8c27400640 1 --2- 172.21.15.27:0/1809527374 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8c0c0b68f0 0x7f8c0c0b8e00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8c18005ab0 tx=0x7f8c18005a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.596+0000 7f8c359ff740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3fbf1b0 0x55bfc4363e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.596+0000 7f8c359ff740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f962f0 0x55bfc4364350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.596+0000 7f8c359ff740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f9e4d0 0x55bfc4364890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.596+0000 7f8c359ff740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bfc42d96b0 con 0x55bfc3f9e4d0 2024-09-21T00:34:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.596+0000 7f8c359ff740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bfc42c8ae0 con 0x55bfc3fbf1b0 2024-09-21T00:34:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.596+0000 7f8c359ff740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bfc42cc8f0 con 0x55bfc3f962f0 2024-09-21T00:34:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.596+0000 7f8c27e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3fbf1b0 0x55bfc4363e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.596+0000 7f8c27e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3fbf1b0 0x55bfc4363e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54628/0 (socket says 172.21.15.27:54628) 2024-09-21T00:34:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.596+0000 7f8c27e00640 1 -- 172.21.15.27:0/3983166600 learned_addr learned my addr 172.21.15.27:0/3983166600 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.596+0000 7f8c27e00640 1 -- 172.21.15.27:0/3983166600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f962f0 msgr2=0x55bfc4364350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:34:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.596+0000 7f8c27e00640 1 --2- 172.21.15.27:0/3983166600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f962f0 0x55bfc4364350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.596+0000 7f8c27e00640 1 -- 172.21.15.27:0/3983166600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f9e4d0 msgr2=0x55bfc4364890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.596+0000 7f8c27e00640 1 --2- 172.21.15.27:0/3983166600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f9e4d0 0x55bfc4364890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.596+0000 7f8c27e00640 1 -- 172.21.15.27:0/3983166600 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfc4295780 con 0x55bfc3fbf1b0 2024-09-21T00:34:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.597+0000 7f8c27e00640 1 --2- 172.21.15.27:0/3983166600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3fbf1b0 0x55bfc4363e10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8c100e4b50 tx=0x7f8c100a83c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=990aafacb4fe75f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.597+0000 7f8c1d200640 1 -- 172.21.15.27:0/3983166600 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c10017070 con 0x55bfc3fbf1b0 2024-09-21T00:34:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.597+0000 7f8c1d200640 1 -- 172.21.15.27:0/3983166600 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c100ad350 con 0x55bfc3fbf1b0 2024-09-21T00:34:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.597+0000 7f8c1d200640 1 -- 172.21.15.27:0/3983166600 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c100d02d0 con 0x55bfc3fbf1b0 2024-09-21T00:34:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.598+0000 7f8c359ff740 1 -- 172.21.15.27:0/3983166600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3fbf1b0 msgr2=0x55bfc4363e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.598+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3983166600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3fbf1b0 0x55bfc4363e10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8c100e4b50 tx=0x7f8c100a83c0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.600+0000 7f8c359ff740 1 -- 172.21.15.27:0/3983166600 shutdown_connections 2024-09-21T00:34:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.600+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3983166600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f9e4d0 0x55bfc4364890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.600+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3983166600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f962f0 0x55bfc4364350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.600+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3983166600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3fbf1b0 0x55bfc4363e10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.600+0000 7f8c359ff740 1 -- 172.21.15.27:0/3983166600 >> 172.21.15.27:0/3983166600 conn(0x55bfc3fb3300 msgr2=0x55bfc4361d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.603+0000 7f8c359ff740 1 -- 172.21.15.27:0/3983166600 shutdown_connections 2024-09-21T00:34:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.603+0000 7f8c359ff740 1 -- 172.21.15.27:0/3983166600 wait complete. 2024-09-21T00:34:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.604+0000 7f8c359ff740 1 -- start start 2024-09-21T00:34:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.605+0000 7f8c359ff740 1 Processor -- start 2024-09-21T00:34:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.607+0000 7f8c359ff740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f962f0 0x55bfc436f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.607+0000 7f8c359ff740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3f9e4d0 0x55bfc436f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.607+0000 7f8c359ff740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3fbf1b0 0x55bfc437b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.607+0000 7f8c359ff740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bfc4200ab0 con 0x55bfc3f962f0 2024-09-21T00:34:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.607+0000 7f8c359ff740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bfc426f570 con 0x55bfc3f9e4d0 2024-09-21T00:34:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.607+0000 7f8c359ff740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bfc4372590 con 0x55bfc3fbf1b0 2024-09-21T00:34:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.607+0000 7f8c359ff740 10 client.0 _ll_register_callbacks cb 0x55bfc427e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:34:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.608+0000 7f8c2ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3fbf1b0 0x55bfc437b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.608+0000 7f8c2ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3fbf1b0 0x55bfc437b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55594/0 (socket says 172.21.15.27:55594) 2024-09-21T00:34:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.608+0000 7f8c2ca00640 1 -- 172.21.15.27:0/3373740899 learned_addr learned my addr 172.21.15.27:0/3373740899 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.608+0000 7f8c27e00640 1 --2- 172.21.15.27:0/3373740899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f962f0 0x55bfc436f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.608+0000 7f8c27400640 1 --2- 172.21.15.27:0/3373740899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3f9e4d0 0x55bfc436f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.608+0000 7f8c27e00640 1 -- 172.21.15.27:0/3373740899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3fbf1b0 msgr2=0x55bfc437b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.608+0000 7f8c27e00640 1 --2- 172.21.15.27:0/3373740899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3fbf1b0 0x55bfc437b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.608+0000 7f8c27e00640 1 -- 172.21.15.27:0/3373740899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3f9e4d0 msgr2=0x55bfc436f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.608+0000 7f8c27e00640 1 --2- 172.21.15.27:0/3373740899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3f9e4d0 0x55bfc436f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.608+0000 7f8c27e00640 1 -- 172.21.15.27:0/3373740899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfc402a790 con 0x55bfc3f962f0 2024-09-21T00:34:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.609+0000 7f8c2ca00640 1 --2- 172.21.15.27:0/3373740899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3fbf1b0 0x55bfc437b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:34:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.609+0000 7f8c15600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T00:34:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.609+0000 7f8c27e00640 1 --2- 172.21.15.27:0/3373740899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f962f0 0x55bfc436f460 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8c100e4b50 tx=0x7f8c10000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2997a990a1d75cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.609+0000 7f8c15600640 1 -- 172.21.15.27:0/3373740899 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c10017050 con 0x55bfc3f962f0 2024-09-21T00:34:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.609+0000 7f8c15600640 1 -- 172.21.15.27:0/3373740899 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9607+0+0 (secure 0 0 0) 0x7f8c10005220 con 0x55bfc3f962f0 2024-09-21T00:34:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.610+0000 7f8c359ff740 1 -- 172.21.15.27:0/3373740899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c100ad350 con 0x55bfc3f962f0 2024-09-21T00:34:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.612+0000 7f8c15600640 1 -- 172.21.15.27:0/3373740899 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8c10017050 con 0x55bfc3f962f0 2024-09-21T00:34:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.612+0000 7f8c359ff740 10 client.16623 fetch_fsmap learned FSMap version 47 2024-09-21T00:34:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.612+0000 7f8c359ff740 1 -- 172.21.15.27:0/3373740899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55bfc40a3620 con 0x55bfc3f962f0 2024-09-21T00:34:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.612+0000 7f8c15600640 1 -- 172.21.15.27:0/3373740899 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c100b6670 con 0x55bfc3f962f0 2024-09-21T00:34:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.615+0000 7f8c15600640 1 -- 172.21.15.27:0/3373740899 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f8c100b12f0 con 0x55bfc3f962f0 2024-09-21T00:34:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.615+0000 7f8c359ff740 10 client.16623 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:34:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.615+0000 7f8c359ff740 10 client.16623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:34:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.615+0000 7f8c359ff740 20 client.16623 populate_metadata read hostname 'smithi027' 2024-09-21T00:34:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.615+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3373740899 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55bfc3f7dd80 0x55bfc43813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.615+0000 7f8c359ff740 4 client.16623 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:34:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.615+0000 7f8c359ff740 1 -- 172.21.15.27:0/3373740899 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8c100ad350 con 0x55bfc3f7dd80 2024-09-21T00:34:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.616+0000 7f8c27400640 1 --2- 172.21.15.27:0/3373740899 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55bfc3f7dd80 0x55bfc43813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.617+0000 7f8c15600640 10 client.16623 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.617+0000 7f8c27400640 1 --2- 172.21.15.27:0/3373740899 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55bfc3f7dd80 0x55bfc43813d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d8495a741043c2 server_cookie=674a693c6f5e132f in_seq=0 out_seq=0 2024-09-21T00:34:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.618+0000 7f8c15600640 1 -- 172.21.15.27:0/3373740899 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f8c100ad350 con 0x55bfc3f7dd80 2024-09-21T00:34:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.618+0000 7f8c15600640 10 client.16623 handle_command_reply: tid=0 2024-09-21T00:34:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.618+0000 7f8c359ff740 1 client.16623 shutdown 2024-09-21T00:34:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.618+0000 7f8c359ff740 1 -- 172.21.15.27:0/3373740899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f962f0 msgr2=0x55bfc436f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.618+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3373740899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f962f0 0x55bfc436f460 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8c100e4b50 tx=0x7f8c10000bc0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.621+0000 7f8c359ff740 1 -- 172.21.15.27:0/3373740899 shutdown_connections 2024-09-21T00:34:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.621+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3373740899 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55bfc3f7dd80 0x55bfc43813d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.621+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3373740899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3fbf1b0 0x55bfc437b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.621+0000 7f8c15600640 0 client.16623 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.621+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3373740899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3f9e4d0 0x55bfc436f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.621+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3373740899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f962f0 0x55bfc436f460 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.621+0000 7f8c359ff740 1 -- 172.21.15.27:0/3373740899 >> 172.21.15.27:0/3373740899 conn(0x55bfc3fb3300 msgr2=0x55bfc4364e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.621+0000 7f8c359ff740 1 -- 172.21.15.27:0/3373740899 shutdown_connections 2024-09-21T00:34:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.621+0000 7f8c359ff740 1 -- 172.21.15.27:0/3373740899 wait complete. 2024-09-21T00:34:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.621+0000 7f8c359ff740 20 client.16623 trim_cache size 0 max 16384 2024-09-21T00:34:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.652+0000 7f8c359ff740 1 Processor -- start 2024-09-21T00:34:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.652+0000 7f8c359ff740 1 -- start start 2024-09-21T00:34:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.652+0000 7f8c359ff740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f7dd80 0x55bfc436f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.653+0000 7f8c359ff740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f962f0 0x55bfc436cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.653+0000 7f8c359ff740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3f9e4d0 0x55bfc4361c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.653+0000 7f8c359ff740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281cb390 con 0x55bfc3f7dd80 2024-09-21T00:34:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.653+0000 7f8c359ff740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281c8ba0 con 0x55bfc3f9e4d0 2024-09-21T00:34:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.653+0000 7f8c359ff740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281c6aa0 con 0x55bfc3f962f0 2024-09-21T00:34:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.653+0000 7f8c27400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f962f0 0x55bfc436cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.653+0000 7f8c27400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f962f0 0x55bfc436cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55596/0 (socket says 172.21.15.27:55596) 2024-09-21T00:34:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.653+0000 7f8c27400640 1 -- 172.21.15.27:0/1459854040 learned_addr learned my addr 172.21.15.27:0/1459854040 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.654+0000 7f8c27400640 1 -- 172.21.15.27:0/1459854040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3f9e4d0 msgr2=0x55bfc4361c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.654+0000 7f8c27400640 1 --2- 172.21.15.27:0/1459854040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3f9e4d0 0x55bfc4361c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.654+0000 7f8c27400640 1 -- 172.21.15.27:0/1459854040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f7dd80 msgr2=0x55bfc436f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.654+0000 7f8c27400640 1 --2- 172.21.15.27:0/1459854040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f7dd80 0x55bfc436f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.654+0000 7f8c27400640 1 -- 172.21.15.27:0/1459854040 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfc436d960 con 0x55bfc3f962f0 2024-09-21T00:34:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.655+0000 7f8c27400640 1 --2- 172.21.15.27:0/1459854040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f962f0 0x55bfc436cf00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8c180096c0 tx=0x7f8c18040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98f974965ec71f76 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.655+0000 7f8c17e00640 1 -- 172.21.15.27:0/1459854040 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18055070 con 0x55bfc3f962f0 2024-09-21T00:34:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.655+0000 7f8c17e00640 1 -- 172.21.15.27:0/1459854040 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c18003b60 con 0x55bfc3f962f0 2024-09-21T00:34:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.657+0000 7f8c359ff740 1 -- 172.21.15.27:0/1459854040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f962f0 msgr2=0x55bfc436cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.657+0000 7f8c359ff740 1 --2- 172.21.15.27:0/1459854040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f962f0 0x55bfc436cf00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8c180096c0 tx=0x7f8c18040000 comp rx=0 tx=0).stop 2024-09-21T00:34:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.657+0000 7f8c17e00640 1 -- 172.21.15.27:0/1459854040 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18035040 con 0x55bfc3f962f0 2024-09-21T00:34:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.660+0000 7f8c359ff740 1 -- 172.21.15.27:0/1459854040 shutdown_connections 2024-09-21T00:34:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.660+0000 7f8c359ff740 1 --2- 172.21.15.27:0/1459854040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3f9e4d0 0x55bfc4361c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.660+0000 7f8c359ff740 1 --2- 172.21.15.27:0/1459854040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f962f0 0x55bfc436cf00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.660+0000 7f8c359ff740 1 --2- 172.21.15.27:0/1459854040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f7dd80 0x55bfc436f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.660+0000 7f8c359ff740 1 -- 172.21.15.27:0/1459854040 >> 172.21.15.27:0/1459854040 conn(0x55bfc3fb3300 msgr2=0x55bfc4361540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.661+0000 7f8c359ff740 1 -- 172.21.15.27:0/1459854040 shutdown_connections 2024-09-21T00:34:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.663+0000 7f8c359ff740 1 -- 172.21.15.27:0/1459854040 wait complete. 2024-09-21T00:34:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.663+0000 7f8c359ff740 1 -- start start 2024-09-21T00:34:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.664+0000 7f8c359ff740 1 Processor -- start 2024-09-21T00:34:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.666+0000 7f8c359ff740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f7dd80 0x55bfc436c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.667+0000 7f8c359ff740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3f962f0 0x55bfc436c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.667+0000 7f8c359ff740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f9e4d0 0x55bfc43e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.667+0000 7f8c27e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f7dd80 0x55bfc436c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.667+0000 7f8c359ff740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281c7060 con 0x55bfc3f7dd80 2024-09-21T00:34:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.667+0000 7f8c27e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f7dd80 0x55bfc436c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56654/0 (socket says 172.21.15.27:56654) 2024-09-21T00:34:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.667+0000 7f8c27e00640 1 -- 172.21.15.27:0/3689997514 learned_addr learned my addr 172.21.15.27:0/3689997514 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.667+0000 7f8c359ff740 1 -- 172.21.15.27:0/3689997514 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281c84d0 con 0x55bfc3f962f0 2024-09-21T00:34:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.667+0000 7f8c359ff740 1 -- 172.21.15.27:0/3689997514 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281c7730 con 0x55bfc3f9e4d0 2024-09-21T00:34:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.667+0000 7f8c2ca00640 1 --2- 172.21.15.27:0/3689997514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f9e4d0 0x55bfc43e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.667+0000 7f8c359ff740 10 client.0 _ll_register_callbacks cb 0x7f8c100acaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:34:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.668+0000 7f8c2ca00640 1 -- 172.21.15.27:0/3689997514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3f962f0 msgr2=0x55bfc436c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.668+0000 7f8c2ca00640 1 --2- 172.21.15.27:0/3689997514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3f962f0 0x55bfc436c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.668+0000 7f8c2ca00640 1 -- 172.21.15.27:0/3689997514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f7dd80 msgr2=0x55bfc436c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.668+0000 7f8c2ca00640 1 --2- 172.21.15.27:0/3689997514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f7dd80 0x55bfc436c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.668+0000 7f8c2ca00640 1 -- 172.21.15.27:0/3689997514 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfc43e23d0 con 0x55bfc3f9e4d0 2024-09-21T00:34:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.670+0000 7f8c2ca00640 1 --2- 172.21.15.27:0/3689997514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f9e4d0 0x55bfc43e1d30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8c20005c40 tx=0x7f8c20005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdba01b706a7f3b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.670+0000 7f8c15600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:34:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.672+0000 7f8c15600640 1 -- 172.21.15.27:0/3689997514 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20009e60 con 0x55bfc3f9e4d0 2024-09-21T00:34:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.672+0000 7f8c15600640 1 -- 172.21.15.27:0/3689997514 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9607+0+0 (secure 0 0 0) 0x7f8c20011640 con 0x55bfc3f9e4d0 2024-09-21T00:34:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.672+0000 7f8c359ff740 1 -- 172.21.15.27:0/3689997514 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c18003b60 con 0x55bfc3f9e4d0 2024-09-21T00:34:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.673+0000 7f8c15600640 1 -- 172.21.15.27:0/3689997514 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8c20011840 con 0x55bfc3f9e4d0 2024-09-21T00:34:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.673+0000 7f8c359ff740 10 client.26659 fetch_fsmap learned FSMap version 47 2024-09-21T00:34:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.673+0000 7f8c359ff740 1 -- 172.21.15.27:0/3689997514 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55bfc43eba40 con 0x55bfc3f9e4d0 2024-09-21T00:34:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.679+0000 7f8c15600640 1 -- 172.21.15.27:0/3689997514 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c18003b60 con 0x55bfc3f9e4d0 2024-09-21T00:34:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.679+0000 7f8c15600640 1 -- 172.21.15.27:0/3689997514 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f8c20023c30 con 0x55bfc3f9e4d0 2024-09-21T00:34:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.679+0000 7f8c359ff740 10 client.26659 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:34:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.679+0000 7f8c359ff740 10 client.26659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:34:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.679+0000 7f8c359ff740 20 client.26659 populate_metadata read hostname 'smithi027' 2024-09-21T00:34:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.679+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3689997514 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55bfc3fbf1b0 0x55bfc43ee290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.679+0000 7f8c359ff740 4 client.26659 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:34:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.679+0000 7f8c359ff740 1 -- 172.21.15.27:0/3689997514 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bfc42db600 con 0x55bfc3fbf1b0 2024-09-21T00:34:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.681+0000 7f8c27e00640 1 --2- 172.21.15.27:0/3689997514 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55bfc3fbf1b0 0x55bfc43ee290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.682+0000 7f8c27e00640 1 --2- 172.21.15.27:0/3689997514 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55bfc3fbf1b0 0x55bfc43ee290 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61a77a256debea1b server_cookie=76de9130c0a3cd57 in_seq=0 out_seq=0 2024-09-21T00:34:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:38.682+0000 7f8c15600640 10 client.26659 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.116+0000 7f8c15600640 1 -- 172.21.15.27:0/3689997514 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bfc42db600 con 0x55bfc3fbf1b0 2024-09-21T00:34:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.117+0000 7f8c15600640 10 client.26659 handle_command_reply: tid=0 2024-09-21T00:34:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.117+0000 7f8c359ff740 1 client.26659 shutdown 2024-09-21T00:34:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.117+0000 7f8c359ff740 1 -- 172.21.15.27:0/3689997514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f9e4d0 msgr2=0x55bfc43e1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.117+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3689997514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f9e4d0 0x55bfc43e1d30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8c20005c40 tx=0x7f8c20005a40 comp rx=0 tx=0).stop 2024-09-21T00:34:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.118+0000 7f8c359ff740 1 -- 172.21.15.27:0/3689997514 shutdown_connections 2024-09-21T00:34:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.118+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3689997514 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55bfc3fbf1b0 0x55bfc43ee290 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.118+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3689997514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bfc3f9e4d0 0x55bfc43e1d30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.118+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3689997514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfc3f962f0 0x55bfc436c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.118+0000 7f8c359ff740 1 --2- 172.21.15.27:0/3689997514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfc3f7dd80 0x55bfc436c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.118+0000 7f8c359ff740 1 -- 172.21.15.27:0/3689997514 >> 172.21.15.27:0/3689997514 conn(0x55bfc3fb3300 msgr2=0x55bfc43707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.118+0000 7f8c15600640 0 client.26659 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.118+0000 7f8c359ff740 1 -- 172.21.15.27:0/3689997514 shutdown_connections 2024-09-21T00:34:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.118+0000 7f8c359ff740 1 -- 172.21.15.27:0/3689997514 wait complete. 2024-09-21T00:34:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.118+0000 7f8c359ff740 20 client.26659 trim_cache size 0 max 16384 2024-09-21T00:34:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:34:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:34:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:34:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:34:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.119+0000 7f8c2d400640 1 -- 172.21.15.27:0/1809527374 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8c0c0b68f0 msgr2=0x7f8c0c0b8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.119+0000 7f8c2d400640 1 --2- 172.21.15.27:0/1809527374 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8c0c0b68f0 0x7f8c0c0b8e00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8c18005ab0 tx=0x7f8c18005a40 comp rx=0 tx=0).stop 2024-09-21T00:34:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.119+0000 7f8c2d400640 1 -- 172.21.15.27:0/1809527374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2810e310 msgr2=0x7f8c28119f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.119+0000 7f8c2d400640 1 --2- 172.21.15.27:0/1809527374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2810e310 0x7f8c28119f90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000d6d0 tx=0x7f8c1000db90 comp rx=0 tx=0).stop 2024-09-21T00:34:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.120+0000 7f8c2d400640 1 -- 172.21.15.27:0/1809527374 shutdown_connections 2024-09-21T00:34:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.120+0000 7f8c2d400640 1 --2- 172.21.15.27:0/1809527374 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f8c0c0b68f0 0x7f8c0c0b8e00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.120+0000 7f8c2d400640 1 --2- 172.21.15.27:0/1809527374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c2811e1f0 0x7f8c2811e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.120+0000 7f8c2d400640 1 --2- 172.21.15.27:0/1809527374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c28114590 0x7f8c2811a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.120+0000 7f8c2d400640 1 --2- 172.21.15.27:0/1809527374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2810e310 0x7f8c28119f90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.120+0000 7f8c2d400640 1 -- 172.21.15.27:0/1809527374 >> 172.21.15.27:0/1809527374 conn(0x7f8c2806e610 msgr2=0x7f8c28072520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.120+0000 7f8c2d400640 1 -- 172.21.15.27:0/1809527374 shutdown_connections 2024-09-21T00:34:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:39.120+0000 7f8c2d400640 1 -- 172.21.15.27:0/1809527374 wait complete. 2024-09-21T00:34:40.119 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:39 smithi027 ceph-mon[30745]: pgmap v794: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 252 op/s 2024-09-21T00:34:40.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:39 smithi089 ceph-mon[34481]: pgmap v794: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 252 op/s 2024-09-21T00:34:40.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:39 smithi098 ceph-mon[34725]: pgmap v794: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 252 op/s 2024-09-21T00:34:41.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:40 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:34:41.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:40 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:34:41.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:40 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:34:42.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:41 smithi089 ceph-mon[34481]: pgmap v795: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 243 op/s 2024-09-21T00:34:42.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:41 smithi098 ceph-mon[34725]: pgmap v795: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 243 op/s 2024-09-21T00:34:42.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:41 smithi027 ceph-mon[30745]: pgmap v795: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 243 op/s 2024-09-21T00:34:43.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:42 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:34:43.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:42 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:34:43.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:42 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:34:43.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:42 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:34:43.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:42 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:34:43.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:42 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:34:43.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:42 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:34:43.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:42 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:34:43.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:42 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:34:44.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:44 smithi089 ceph-mon[34481]: pgmap v796: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 196 op/s 2024-09-21T00:34:44.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:44 smithi098 ceph-mon[34725]: pgmap v796: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 196 op/s 2024-09-21T00:34:44.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:44 smithi027 ceph-mon[30745]: pgmap v796: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 196 op/s 2024-09-21T00:34:46.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:46 smithi089 ceph-mon[34481]: pgmap v797: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 194 op/s 2024-09-21T00:34:46.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:46 smithi098 ceph-mon[34725]: pgmap v797: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 194 op/s 2024-09-21T00:34:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:46 smithi027 ceph-mon[30745]: pgmap v797: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 194 op/s 2024-09-21T00:34:48.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:48 smithi089 ceph-mon[34481]: pgmap v798: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 202 op/s 2024-09-21T00:34:48.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:48 smithi098 ceph-mon[34725]: pgmap v798: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 202 op/s 2024-09-21T00:34:48.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:48 smithi027 ceph-mon[30745]: pgmap v798: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 202 op/s 2024-09-21T00:34:50.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:50 smithi089 ceph-mon[34481]: pgmap v799: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 198 op/s 2024-09-21T00:34:50.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:50 smithi098 ceph-mon[34725]: pgmap v799: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 198 op/s 2024-09-21T00:34:50.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:50 smithi027 ceph-mon[30745]: pgmap v799: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 198 op/s 2024-09-21T00:34:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.652+0000 7f5f87200640 1 -- 172.21.15.27:0/2672082110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f80114a10 msgr2=0x7f5f80116f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.652+0000 7f5f87200640 1 --2- 172.21.15.27:0/2672082110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f80114a10 0x7f5f80116f00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009ca0 tx=0x7f5f7c02a8d0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.653+0000 7f5f87200640 1 -- 172.21.15.27:0/2672082110 shutdown_connections 2024-09-21T00:34:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.653+0000 7f5f87200640 1 --2- 172.21.15.27:0/2672082110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f80114a10 0x7f5f80116f00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.653+0000 7f5f87200640 1 --2- 172.21.15.27:0/2672082110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f8010dc10 0x7f5f8010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.653+0000 7f5f87200640 1 --2- 172.21.15.27:0/2672082110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f8010a3c0 0x7f5f8010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.653+0000 7f5f87200640 1 -- 172.21.15.27:0/2672082110 >> 172.21.15.27:0/2672082110 conn(0x7f5f8006e8f0 msgr2=0x7f5f8006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.654+0000 7f5f87200640 1 -- 172.21.15.27:0/2672082110 shutdown_connections 2024-09-21T00:34:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.654+0000 7f5f87200640 1 -- 172.21.15.27:0/2672082110 wait complete. 2024-09-21T00:34:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.656+0000 7f5f87200640 1 Processor -- start 2024-09-21T00:34:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.656+0000 7f5f87200640 1 -- start start 2024-09-21T00:34:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.656+0000 7f5f87200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f8010a3c0 0x7f5f8011a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.657+0000 7f5f87200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f8010dc10 0x7f5f801189d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.657+0000 7f5f87200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f80114a10 0x7f5f80118f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.657+0000 7f5f87200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8011c060 con 0x7f5f8010a3c0 2024-09-21T00:34:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.657+0000 7f5f87200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8011bee0 con 0x7f5f80114a10 2024-09-21T00:34:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.657+0000 7f5f87200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8011c1e0 con 0x7f5f8010dc10 2024-09-21T00:34:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.657+0000 7f5f85400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f8010dc10 0x7f5f801189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.657+0000 7f5f86800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f80114a10 0x7f5f80118f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.657+0000 7f5f85400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f8010dc10 0x7f5f801189d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:60064/0 (socket says 172.21.15.27:60064) 2024-09-21T00:34:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.657+0000 7f5f86800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f80114a10 0x7f5f80118f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:52002/0 (socket says 172.21.15.27:52002) 2024-09-21T00:34:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.657+0000 7f5f85400640 1 -- 172.21.15.27:0/2939683854 learned_addr learned my addr 172.21.15.27:0/2939683854 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.657+0000 7f5f85400640 1 -- 172.21.15.27:0/2939683854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f80114a10 msgr2=0x7f5f80118f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.658+0000 7f5f85400640 1 --2- 172.21.15.27:0/2939683854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f80114a10 0x7f5f80118f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.658+0000 7f5f85e00640 1 --2- 172.21.15.27:0/2939683854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f8010a3c0 0x7f5f8011a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.658+0000 7f5f85400640 1 -- 172.21.15.27:0/2939683854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f8010a3c0 msgr2=0x7f5f8011a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.658+0000 7f5f85400640 1 --2- 172.21.15.27:0/2939683854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f8010a3c0 0x7f5f8011a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.658+0000 7f5f85400640 1 -- 172.21.15.27:0/2939683854 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f80119730 con 0x7f5f8010dc10 2024-09-21T00:34:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.658+0000 7f5f85e00640 1 --2- 172.21.15.27:0/2939683854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f8010a3c0 0x7f5f8011a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:34:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.658+0000 7f5f85400640 1 --2- 172.21.15.27:0/2939683854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f8010dc10 0x7f5f801189d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000b3f0 tx=0x7f5f7000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.658+0000 7f5f7aa00640 1 -- 172.21.15.27:0/2939683854 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70015020 con 0x7f5f8010dc10 2024-09-21T00:34:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.658+0000 7f5f7aa00640 1 -- 172.21.15.27:0/2939683854 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f70011ce0 con 0x7f5f8010dc10 2024-09-21T00:34:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.659+0000 7f5f87200640 1 -- 172.21.15.27:0/2939683854 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f801b53c0 con 0x7f5f8010dc10 2024-09-21T00:34:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.659+0000 7f5f7aa00640 1 -- 172.21.15.27:0/2939683854 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70010da0 con 0x7f5f8010dc10 2024-09-21T00:34:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.659+0000 7f5f87200640 1 -- 172.21.15.27:0/2939683854 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f801b5990 con 0x7f5f8010dc10 2024-09-21T00:34:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.666+0000 7f5f8f811740 1 Processor -- start 2024-09-21T00:34:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.666+0000 7f5f8f811740 1 -- start start 2024-09-21T00:34:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.667+0000 7f5f8f811740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565fb1b0 0x55d75699fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.667+0000 7f5f8f811740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 0x55d7569a0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.667+0000 7f5f8f811740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565da4d0 0x55d7569a0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.667+0000 7f5f8f811740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d7569156b0 con 0x55d7565d22f0 2024-09-21T00:34:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.667+0000 7f5f8f811740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d756904ae0 con 0x55d7565da4d0 2024-09-21T00:34:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.667+0000 7f5f8f811740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d7569088f0 con 0x55d7565fb1b0 2024-09-21T00:34:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.667+0000 7f5f85e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 0x55d7569a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.667+0000 7f5f85e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 0x55d7569a0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55872/0 (socket says 172.21.15.27:55872) 2024-09-21T00:34:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.667+0000 7f5f85e00640 1 -- 172.21.15.27:0/3635059897 learned_addr learned my addr 172.21.15.27:0/3635059897 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.668+0000 7f5f86800640 1 --2- 172.21.15.27:0/3635059897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565fb1b0 0x55d75699fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.669+0000 7f5f86800640 1 -- 172.21.15.27:0/3635059897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565da4d0 msgr2=0x55d7569a0890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:34:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.669+0000 7f5f86800640 1 --2- 172.21.15.27:0/3635059897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565da4d0 0x55d7569a0890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.669+0000 7f5f86800640 1 -- 172.21.15.27:0/3635059897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 msgr2=0x55d7569a0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.669+0000 7f5f86800640 1 --2- 172.21.15.27:0/3635059897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 0x55d7569a0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.669+0000 7f5f86800640 1 -- 172.21.15.27:0/3635059897 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7568d1780 con 0x55d7565fb1b0 2024-09-21T00:34:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.669+0000 7f5f85e00640 1 --2- 172.21.15.27:0/3635059897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 0x55d7569a0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:34:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.673+0000 7f5f86800640 1 --2- 172.21.15.27:0/3635059897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565fb1b0 0x55d75699fe10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009bf0 tx=0x7f5f7c0098b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2262100e1ec85c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.673+0000 7f5f6f000640 1 -- 172.21.15.27:0/3635059897 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c031d00 con 0x55d7565fb1b0 2024-09-21T00:34:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.675+0000 7f5f6f000640 1 -- 172.21.15.27:0/3635059897 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f7c036070 con 0x55d7565fb1b0 2024-09-21T00:34:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.675+0000 7f5f6f000640 1 -- 172.21.15.27:0/3635059897 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0042a0 con 0x55d7565fb1b0 2024-09-21T00:34:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.675+0000 7f5f7aa00640 1 -- 172.21.15.27:0/2939683854 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5f70018020 con 0x7f5f8010dc10 2024-09-21T00:34:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.676+0000 7f5f8f811740 1 -- 172.21.15.27:0/3635059897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565fb1b0 msgr2=0x55d75699fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.676+0000 7f5f8f811740 1 --2- 172.21.15.27:0/3635059897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565fb1b0 0x55d75699fe10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009bf0 tx=0x7f5f7c0098b0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.680+0000 7f5f7aa00640 1 --2- 172.21.15.27:0/2939683854 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5f680b6730 0x7f5f680b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.681+0000 7f5f7aa00640 1 -- 172.21.15.27:0/2939683854 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9607+0+0 (secure 0 0 0) 0x7f5f700e7270 con 0x7f5f8010dc10 2024-09-21T00:34:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.681+0000 7f5f86800640 1 --2- 172.21.15.27:0/2939683854 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5f680b6730 0x7f5f680b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.682+0000 7f5f8f811740 1 -- 172.21.15.27:0/3635059897 shutdown_connections 2024-09-21T00:34:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.682+0000 7f5f8f811740 1 --2- 172.21.15.27:0/3635059897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565da4d0 0x55d7569a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.682+0000 7f5f8f811740 1 --2- 172.21.15.27:0/3635059897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 0x55d7569a0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.682+0000 7f5f8f811740 1 --2- 172.21.15.27:0/3635059897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565fb1b0 0x55d75699fe10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.682+0000 7f5f8f811740 1 -- 172.21.15.27:0/3635059897 >> 172.21.15.27:0/3635059897 conn(0x55d7565ef300 msgr2=0x55d75699dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.682+0000 7f5f8f811740 1 -- 172.21.15.27:0/3635059897 shutdown_connections 2024-09-21T00:34:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.682+0000 7f5f86800640 1 --2- 172.21.15.27:0/2939683854 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5f680b6730 0x7f5f680b8c40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c02ae90 tx=0x7f5f7c003f00 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.682+0000 7f5f8f811740 1 -- 172.21.15.27:0/3635059897 wait complete. 2024-09-21T00:34:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.683+0000 7f5f8f811740 1 -- start start 2024-09-21T00:34:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.684+0000 7f5f8f811740 1 Processor -- start 2024-09-21T00:34:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.685+0000 7f5f8f811740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 0x55d7569ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.685+0000 7f5f8f811740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565da4d0 0x55d7569ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.685+0000 7f5f8f811740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565fb1b0 0x55d7569b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.685+0000 7f5f8f811740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d75683cab0 con 0x55d7565d22f0 2024-09-21T00:34:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.685+0000 7f5f8f811740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d7568ab570 con 0x55d7565fb1b0 2024-09-21T00:34:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.685+0000 7f5f8f811740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d7569ae530 con 0x55d7565da4d0 2024-09-21T00:34:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.685+0000 7f5f8f811740 10 client.0 _ll_register_callbacks cb 0x55d7568ba100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:34:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.685+0000 7f5f85400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565da4d0 0x55d7569ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.685+0000 7f5f85400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565da4d0 0x55d7569ab9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:60086/0 (socket says 172.21.15.27:60086) 2024-09-21T00:34:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.686+0000 7f5f85400640 1 -- 172.21.15.27:0/1248212302 learned_addr learned my addr 172.21.15.27:0/1248212302 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.686+0000 7f5f86800640 1 --2- 172.21.15.27:0/1248212302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565fb1b0 0x55d7569b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.686+0000 7f5f85400640 1 -- 172.21.15.27:0/1248212302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565fb1b0 msgr2=0x55d7569b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.686+0000 7f5f85400640 1 --2- 172.21.15.27:0/1248212302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565fb1b0 0x55d7569b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.686+0000 7f5f85400640 1 -- 172.21.15.27:0/1248212302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 msgr2=0x55d7569ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.687+0000 7f5f85e00640 1 --2- 172.21.15.27:0/1248212302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 0x55d7569ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.687+0000 7f5f85400640 1 --2- 172.21.15.27:0/1248212302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 0x55d7569ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.688+0000 7f5f85400640 1 -- 172.21.15.27:0/1248212302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d756666790 con 0x55d7565da4d0 2024-09-21T00:34:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.688+0000 7f5f86800640 1 --2- 172.21.15.27:0/1248212302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565fb1b0 0x55d7569b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:34:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.688+0000 7f5f85400640 1 --2- 172.21.15.27:0/1248212302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565da4d0 0x55d7569ab9f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5f700ac980 tx=0x7f5f700acb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=538ef0ef4c9d900a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.688+0000 7f5f63400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:34:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.688+0000 7f5f63400640 1 -- 172.21.15.27:0/1248212302 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f700d02b0 con 0x55d7565da4d0 2024-09-21T00:34:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.688+0000 7f5f63400640 1 -- 172.21.15.27:0/1248212302 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9607+0+0 (secure 0 0 0) 0x7f5f700abee0 con 0x55d7565da4d0 2024-09-21T00:34:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.689+0000 7f5f8f811740 1 -- 172.21.15.27:0/1248212302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f7c036070 con 0x55d7565da4d0 2024-09-21T00:34:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.689+0000 7f5f63400640 1 -- 172.21.15.27:0/1248212302 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5f700ac790 con 0x55d7565da4d0 2024-09-21T00:34:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.689+0000 7f5f8f811740 10 client.26671 fetch_fsmap learned FSMap version 47 2024-09-21T00:34:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.689+0000 7f5f8f811740 1 -- 172.21.15.27:0/1248212302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d7566df620 con 0x55d7565da4d0 2024-09-21T00:34:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.689+0000 7f5f63400640 1 -- 172.21.15.27:0/1248212302 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f7c036070 con 0x55d7565da4d0 2024-09-21T00:34:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.692+0000 7f5f63400640 1 -- 172.21.15.27:0/1248212302 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f5f700d4020 con 0x55d7565da4d0 2024-09-21T00:34:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.693+0000 7f5f8f811740 10 client.26671 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:34:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.693+0000 7f5f8f811740 10 client.26671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:34:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.693+0000 7f5f8f811740 20 client.26671 populate_metadata read hostname 'smithi027' 2024-09-21T00:34:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.693+0000 7f5f8f811740 1 --2- 172.21.15.27:0/1248212302 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d7565b9d80 0x55d7569bd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.693+0000 7f5f8f811740 4 client.26671 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:34:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.693+0000 7f5f8f811740 1 -- 172.21.15.27:0/1248212302 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d756917600 con 0x55d7565b9d80 2024-09-21T00:34:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.693+0000 7f5f85e00640 1 --2- 172.21.15.27:0/1248212302 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d7565b9d80 0x55d7569bd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.694+0000 7f5f85e00640 1 --2- 172.21.15.27:0/1248212302 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d7565b9d80 0x55d7569bd510 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=370a1ff797499fcc server_cookie=10ee2c9ac5ac88d3 in_seq=0 out_seq=0 2024-09-21T00:34:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.694+0000 7f5f63400640 10 client.26671 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.694+0000 7f5f63400640 1 -- 172.21.15.27:0/1248212302 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d756917600 con 0x55d7565b9d80 2024-09-21T00:34:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.695+0000 7f5f63400640 10 client.26671 handle_command_reply: tid=0 2024-09-21T00:34:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.695+0000 7f5f8f811740 1 client.26671 shutdown 2024-09-21T00:34:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.695+0000 7f5f8f811740 1 -- 172.21.15.27:0/1248212302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565da4d0 msgr2=0x55d7569ab9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.695+0000 7f5f8f811740 1 --2- 172.21.15.27:0/1248212302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565da4d0 0x55d7569ab9f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5f700ac980 tx=0x7f5f700acb80 comp rx=0 tx=0).stop 2024-09-21T00:34:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.695+0000 7f5f8f811740 1 -- 172.21.15.27:0/1248212302 shutdown_connections 2024-09-21T00:34:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.695+0000 7f5f8f811740 1 --2- 172.21.15.27:0/1248212302 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d7565b9d80 0x55d7569bd510 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.695+0000 7f5f8f811740 1 --2- 172.21.15.27:0/1248212302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565fb1b0 0x55d7569b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.695+0000 7f5f8f811740 1 --2- 172.21.15.27:0/1248212302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565da4d0 0x55d7569ab9f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.695+0000 7f5f63400640 0 client.26671 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.695+0000 7f5f8f811740 1 --2- 172.21.15.27:0/1248212302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 0x55d7569ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.695+0000 7f5f8f811740 1 -- 172.21.15.27:0/1248212302 >> 172.21.15.27:0/1248212302 conn(0x55d7565ef300 msgr2=0x55d7569a0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.696+0000 7f5f8f811740 1 -- 172.21.15.27:0/1248212302 shutdown_connections 2024-09-21T00:34:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.696+0000 7f5f8f811740 1 -- 172.21.15.27:0/1248212302 wait complete. 2024-09-21T00:34:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.696+0000 7f5f8f811740 20 client.26671 trim_cache size 0 max 16384 2024-09-21T00:34:52.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:52 smithi089 ceph-mon[34481]: pgmap v800: 129 pgs: 17 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 56 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 206 op/s 2024-09-21T00:34:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.726+0000 7f5f8f811740 1 Processor -- start 2024-09-21T00:34:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.726+0000 7f5f8f811740 1 -- start start 2024-09-21T00:34:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.727+0000 7f5f8f811740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 0x55d7569a8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.727+0000 7f5f8f811740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565da4d0 0x55d75699dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.727+0000 7f5f8f811740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565fb1b0 0x55d7569b7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.727+0000 7f5f8f811740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f801c25b0 con 0x55d7565d22f0 2024-09-21T00:34:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.727+0000 7f5f8f811740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5f801bfdc0 con 0x55d7565fb1b0 2024-09-21T00:34:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.727+0000 7f5f8f811740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f801bdc30 con 0x55d7565da4d0 2024-09-21T00:34:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.728+0000 7f5f85400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565da4d0 0x55d75699dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.728+0000 7f5f85400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565da4d0 0x55d75699dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:60102/0 (socket says 172.21.15.27:60102) 2024-09-21T00:34:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.728+0000 7f5f85400640 1 -- 172.21.15.27:0/3860413092 learned_addr learned my addr 172.21.15.27:0/3860413092 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.728+0000 7f5f86800640 1 --2- 172.21.15.27:0/3860413092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565fb1b0 0x55d7569b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.728+0000 7f5f85e00640 1 --2- 172.21.15.27:0/3860413092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 0x55d7569a8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.728+0000 7f5f85400640 1 -- 172.21.15.27:0/3860413092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565fb1b0 msgr2=0x55d7569b7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.728+0000 7f5f85400640 1 --2- 172.21.15.27:0/3860413092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565fb1b0 0x55d7569b7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.728+0000 7f5f85400640 1 -- 172.21.15.27:0/3860413092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 msgr2=0x55d7569a8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.729+0000 7f5f85400640 1 --2- 172.21.15.27:0/3860413092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 0x55d7569a8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.729+0000 7f5f85400640 1 -- 172.21.15.27:0/3860413092 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7569a29b0 con 0x55d7565da4d0 2024-09-21T00:34:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.729+0000 7f5f85400640 1 --2- 172.21.15.27:0/3860413092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565da4d0 0x55d75699dca0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5f700acc10 tx=0x7f5f700b5af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62cea1e34c93a236 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.729+0000 7f5f6e600640 1 -- 172.21.15.27:0/3860413092 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f700d02f0 con 0x55d7565da4d0 2024-09-21T00:34:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.729+0000 7f5f6e600640 1 -- 172.21.15.27:0/3860413092 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f700b66a0 con 0x55d7565da4d0 2024-09-21T00:34:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.730+0000 7f5f6e600640 1 -- 172.21.15.27:0/3860413092 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f700a8830 con 0x55d7565da4d0 2024-09-21T00:34:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.731+0000 7f5f8f811740 1 -- 172.21.15.27:0/3860413092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565da4d0 msgr2=0x55d75699dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.731+0000 7f5f8f811740 1 --2- 172.21.15.27:0/3860413092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565da4d0 0x55d75699dca0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5f700acc10 tx=0x7f5f700b5af0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.731+0000 7f5f8f811740 1 -- 172.21.15.27:0/3860413092 shutdown_connections 2024-09-21T00:34:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.731+0000 7f5f8f811740 1 --2- 172.21.15.27:0/3860413092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565fb1b0 0x55d7569b7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.731+0000 7f5f8f811740 1 --2- 172.21.15.27:0/3860413092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565da4d0 0x55d75699dca0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.731+0000 7f5f8f811740 1 --2- 172.21.15.27:0/3860413092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565d22f0 0x55d7569a8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.731+0000 7f5f8f811740 1 -- 172.21.15.27:0/3860413092 >> 172.21.15.27:0/3860413092 conn(0x55d7565ef300 msgr2=0x55d75699d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.731+0000 7f5f8f811740 1 -- 172.21.15.27:0/3860413092 shutdown_connections 2024-09-21T00:34:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.731+0000 7f5f8f811740 1 -- 172.21.15.27:0/3860413092 wait complete. 2024-09-21T00:34:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.732+0000 7f5f8f811740 1 -- start start 2024-09-21T00:34:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.732+0000 7f5f8f811740 1 Processor -- start 2024-09-21T00:34:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.733+0000 7f5f8f811740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565d22f0 0x55d7569a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.733+0000 7f5f8f811740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565da4d0 0x55d7569a6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.733+0000 7f5f8f811740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565fb1b0 0x55d7569a6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.734+0000 7f5f85400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565da4d0 0x55d7569a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.734+0000 7f5f85400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565da4d0 0x55d7569a6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:52038/0 (socket says 172.21.15.27:52038) 2024-09-21T00:34:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.734+0000 7f5f8f811740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f801be280 con 0x55d7565fb1b0 2024-09-21T00:34:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.734+0000 7f5f8f811740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5f801bf6f0 con 0x55d7565da4d0 2024-09-21T00:34:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.734+0000 7f5f8f811740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f801be950 con 0x55d7565d22f0 2024-09-21T00:34:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.734+0000 7f5f8f811740 10 client.0 _ll_register_callbacks cb 0x7f5f70010fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:34:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.734+0000 7f5f85e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565d22f0 0x55d7569a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.734+0000 7f5f85e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565d22f0 0x55d7569a8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:60116/0 (socket says 172.21.15.27:60116) 2024-09-21T00:34:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.734+0000 7f5f85e00640 1 -- 172.21.15.27:0/352537704 learned_addr learned my addr 172.21.15.27:0/352537704 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:34:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.734+0000 7f5f86800640 1 --2- 172.21.15.27:0/352537704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565fb1b0 0x55d7569a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.735+0000 7f5f85e00640 1 -- 172.21.15.27:0/352537704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565da4d0 msgr2=0x55d7569a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.735+0000 7f5f85e00640 1 --2- 172.21.15.27:0/352537704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565da4d0 0x55d7569a6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.735+0000 7f5f85e00640 1 -- 172.21.15.27:0/352537704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565fb1b0 msgr2=0x55d7569a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:34:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.735+0000 7f5f85e00640 1 --2- 172.21.15.27:0/352537704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565fb1b0 0x55d7569a6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.735+0000 7f5f85e00640 1 -- 172.21.15.27:0/352537704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7569a6ec0 con 0x55d7565d22f0 2024-09-21T00:34:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.735+0000 7f5f86800640 1 --2- 172.21.15.27:0/352537704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565fb1b0 0x55d7569a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:34:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.736+0000 7f5f85400640 1 --2- 172.21.15.27:0/352537704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565da4d0 0x55d7569a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T00:34:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.736+0000 7f5f63400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:34:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.736+0000 7f5f85e00640 1 --2- 172.21.15.27:0/352537704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565d22f0 0x55d7569a8d80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5f74009b10 tx=0x7f5f740127c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d902c0a82365594c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:34:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.737+0000 7f5f63400640 1 -- 172.21.15.27:0/352537704 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74005220 con 0x55d7565d22f0 2024-09-21T00:34:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.737+0000 7f5f63400640 1 -- 172.21.15.27:0/352537704 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9607+0+0 (secure 0 0 0) 0x7f5f740053c0 con 0x55d7565d22f0 2024-09-21T00:34:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.737+0000 7f5f8f811740 1 -- 172.21.15.27:0/352537704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f700b66a0 con 0x55d7565d22f0 2024-09-21T00:34:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.738+0000 7f5f63400640 1 -- 172.21.15.27:0/352537704 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5f740055c0 con 0x55d7565d22f0 2024-09-21T00:34:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.738+0000 7f5f8f811740 10 client.26677 fetch_fsmap learned FSMap version 47 2024-09-21T00:34:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.738+0000 7f5f8f811740 1 -- 172.21.15.27:0/352537704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d756a27e90 con 0x55d7565d22f0 2024-09-21T00:34:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.739+0000 7f5f63400640 1 -- 172.21.15.27:0/352537704 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f700b66a0 con 0x55d7565d22f0 2024-09-21T00:34:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.740+0000 7f5f63400640 1 -- 172.21.15.27:0/352537704 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f5f7400f040 con 0x55d7565d22f0 2024-09-21T00:34:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.740+0000 7f5f8f811740 10 client.26677 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:34:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.740+0000 7f5f8f811740 10 client.26677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:34:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.740+0000 7f5f8f811740 20 client.26677 populate_metadata read hostname 'smithi027' 2024-09-21T00:34:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.741+0000 7f5f8f811740 1 --2- 172.21.15.27:0/352537704 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d75658f1c0 0x55d756a2a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:34:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.741+0000 7f5f8f811740 4 client.26677 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:34:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.741+0000 7f5f8f811740 1 -- 172.21.15.27:0/352537704 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7569130b0 con 0x55d75658f1c0 2024-09-21T00:34:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.741+0000 7f5f85400640 1 --2- 172.21.15.27:0/352537704 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d75658f1c0 0x55d756a2a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:34:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.742+0000 7f5f85400640 1 --2- 172.21.15.27:0/352537704 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d75658f1c0 0x55d756a2a600 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ba4afcb6391a9fc server_cookie=dfcfb69dfc8119bc in_seq=0 out_seq=0 2024-09-21T00:34:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.742+0000 7f5f63400640 10 client.26677 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:52.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:52 smithi098 ceph-mon[34725]: pgmap v800: 129 pgs: 17 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 56 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 206 op/s 2024-09-21T00:34:52.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:52 smithi027 ceph-mon[30745]: pgmap v800: 129 pgs: 17 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 56 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 206 op/s 2024-09-21T00:34:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.954+0000 7f5f63400640 1 -- 172.21.15.27:0/352537704 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7569130b0 con 0x55d75658f1c0 2024-09-21T00:34:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.954+0000 7f5f63400640 10 client.26677 handle_command_reply: tid=0 2024-09-21T00:34:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.954+0000 7f5f8f811740 1 client.26677 shutdown 2024-09-21T00:34:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.955+0000 7f5f8f811740 1 -- 172.21.15.27:0/352537704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565d22f0 msgr2=0x55d7569a8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.955+0000 7f5f8f811740 1 --2- 172.21.15.27:0/352537704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565d22f0 0x55d7569a8d80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5f74009b10 tx=0x7f5f740127c0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.955+0000 7f5f8f811740 1 -- 172.21.15.27:0/352537704 shutdown_connections 2024-09-21T00:34:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.955+0000 7f5f8f811740 1 --2- 172.21.15.27:0/352537704 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55d75658f1c0 0x55d756a2a600 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.955+0000 7f5f8f811740 1 --2- 172.21.15.27:0/352537704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7565fb1b0 0x55d7569a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.955+0000 7f5f8f811740 1 --2- 172.21.15.27:0/352537704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d7565da4d0 0x55d7569a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.955+0000 7f5f8f811740 1 --2- 172.21.15.27:0/352537704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7565d22f0 0x55d7569a8d80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.955+0000 7f5f8f811740 1 -- 172.21.15.27:0/352537704 >> 172.21.15.27:0/352537704 conn(0x55d7565ef300 msgr2=0x55d7569ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.955+0000 7f5f63400640 0 client.26677 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:34:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.955+0000 7f5f8f811740 1 -- 172.21.15.27:0/352537704 shutdown_connections 2024-09-21T00:34:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.957+0000 7f5f8f811740 1 -- 172.21.15.27:0/352537704 wait complete. 2024-09-21T00:34:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.957+0000 7f5f8f811740 20 client.26677 trim_cache size 0 max 16384 2024-09-21T00:34:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:34:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:34:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:34:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:34:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.958+0000 7f5f87200640 1 -- 172.21.15.27:0/2939683854 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5f680b6730 msgr2=0x7f5f680b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.958+0000 7f5f87200640 1 --2- 172.21.15.27:0/2939683854 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5f680b6730 0x7f5f680b8c40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c02ae90 tx=0x7f5f7c003f00 comp rx=0 tx=0).stop 2024-09-21T00:34:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.958+0000 7f5f87200640 1 -- 172.21.15.27:0/2939683854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f8010dc10 msgr2=0x7f5f801189d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:34:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.958+0000 7f5f87200640 1 --2- 172.21.15.27:0/2939683854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f8010dc10 0x7f5f801189d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000b3f0 tx=0x7f5f7000b8b0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.958+0000 7f5f87200640 1 -- 172.21.15.27:0/2939683854 shutdown_connections 2024-09-21T00:34:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.958+0000 7f5f87200640 1 --2- 172.21.15.27:0/2939683854 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5f680b6730 0x7f5f680b8c40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.958+0000 7f5f87200640 1 --2- 172.21.15.27:0/2939683854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f80114a10 0x7f5f80118f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.958+0000 7f5f87200640 1 --2- 172.21.15.27:0/2939683854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f8010dc10 0x7f5f801189d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.958+0000 7f5f87200640 1 --2- 172.21.15.27:0/2939683854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f8010a3c0 0x7f5f8011a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:34:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.958+0000 7f5f87200640 1 -- 172.21.15.27:0/2939683854 >> 172.21.15.27:0/2939683854 conn(0x7f5f8006e8f0 msgr2=0x7f5f8010b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:34:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.959+0000 7f5f87200640 1 -- 172.21.15.27:0/2939683854 shutdown_connections 2024-09-21T00:34:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:34:52.959+0000 7f5f87200640 1 -- 172.21.15.27:0/2939683854 wait complete. 2024-09-21T00:34:54.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:54 smithi089 ceph-mon[34481]: pgmap v801: 129 pgs: 17 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 56 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-09-21T00:34:54.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:54 smithi098 ceph-mon[34725]: pgmap v801: 129 pgs: 17 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 56 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-09-21T00:34:54.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:54 smithi027 ceph-mon[30745]: pgmap v801: 129 pgs: 17 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 56 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-09-21T00:34:56.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:56 smithi089 ceph-mon[34481]: pgmap v802: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 193 op/s 2024-09-21T00:34:56.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:56 smithi098 ceph-mon[34725]: pgmap v802: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 193 op/s 2024-09-21T00:34:56.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:56 smithi027 ceph-mon[30745]: pgmap v802: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 193 op/s 2024-09-21T00:34:58.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:34:58 smithi089 ceph-mon[34481]: pgmap v803: 129 pgs: 13 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 76 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2024-09-21T00:34:58.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:34:58 smithi098 ceph-mon[34725]: pgmap v803: 129 pgs: 13 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 76 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2024-09-21T00:34:58.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:34:58 smithi027 ceph-mon[30745]: pgmap v803: 129 pgs: 13 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 76 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2024-09-21T00:35:00.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:00 smithi089 ceph-mon[34481]: pgmap v804: 129 pgs: 11 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 91 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-09-21T00:35:00.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:00 smithi098 ceph-mon[34725]: pgmap v804: 129 pgs: 11 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 91 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-09-21T00:35:00.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:00 smithi027 ceph-mon[30745]: pgmap v804: 129 pgs: 11 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 91 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-09-21T00:35:02.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:02 smithi098 ceph-mon[34725]: pgmap v805: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 216 op/s 2024-09-21T00:35:02.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:02 smithi027 ceph-mon[30745]: pgmap v805: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 216 op/s 2024-09-21T00:35:02.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:02 smithi089 ceph-mon[34481]: pgmap v805: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 216 op/s 2024-09-21T00:35:03.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:03 smithi098 ceph-mon[34725]: pgmap v806: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 220 op/s 2024-09-21T00:35:03.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:03 smithi027 ceph-mon[30745]: pgmap v806: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 220 op/s 2024-09-21T00:35:03.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:03 smithi089 ceph-mon[34481]: pgmap v806: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 220 op/s 2024-09-21T00:35:06.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:05 smithi027 ceph-mon[30745]: pgmap v807: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 982 KiB/s wr, 223 op/s 2024-09-21T00:35:06.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:05 smithi089 ceph-mon[34481]: pgmap v807: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 982 KiB/s wr, 223 op/s 2024-09-21T00:35:06.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:05 smithi098 ceph-mon[34725]: pgmap v807: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 982 KiB/s wr, 223 op/s 2024-09-21T00:35:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.298+0000 7fb419000640 1 -- 172.21.15.27:0/58527456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb414105090 msgr2=0x7fb41410d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.298+0000 7fb419000640 1 --2- 172.21.15.27:0/58527456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb414105090 0x7fb41410d5d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb400009900 tx=0x7fb40002a410 comp rx=0 tx=0).stop 2024-09-21T00:35:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.298+0000 7fb419000640 1 -- 172.21.15.27:0/58527456 shutdown_connections 2024-09-21T00:35:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.298+0000 7fb419000640 1 --2- 172.21.15.27:0/58527456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb41410dba0 0x7fb414110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.298+0000 7fb419000640 1 --2- 172.21.15.27:0/58527456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb414105090 0x7fb41410d5d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.298+0000 7fb419000640 1 --2- 172.21.15.27:0/58527456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb414104740 0x7fb414104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.298+0000 7fb419000640 1 -- 172.21.15.27:0/58527456 >> 172.21.15.27:0/58527456 conn(0x7fb4140fe210 msgr2=0x7fb414100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.298+0000 7fb419000640 1 -- 172.21.15.27:0/58527456 shutdown_connections 2024-09-21T00:35:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.299+0000 7fb419000640 1 -- 172.21.15.27:0/58527456 wait complete. 2024-09-21T00:35:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.300+0000 7fb419000640 1 Processor -- start 2024-09-21T00:35:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.301+0000 7fb419000640 1 -- start start 2024-09-21T00:35:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.302+0000 7fb419000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb414104740 0x7fb41419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.302+0000 7fb419000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb414105090 0x7fb41419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.302+0000 7fb412a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb414105090 0x7fb41419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.302+0000 7fb412a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb414105090 0x7fb41419ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:56522/0 (socket says 172.21.15.27:56522) 2024-09-21T00:35:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.302+0000 7fb413400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb414104740 0x7fb41419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.302+0000 7fb419000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb41410dba0 0x7fb4141a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.302+0000 7fb419000640 1 -- 172.21.15.27:0/1101906181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb414113810 con 0x7fb41410dba0 2024-09-21T00:35:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.302+0000 7fb412a00640 1 -- 172.21.15.27:0/1101906181 learned_addr learned my addr 172.21.15.27:0/1101906181 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.302+0000 7fb419000640 1 -- 172.21.15.27:0/1101906181 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb414113690 con 0x7fb414105090 2024-09-21T00:35:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.302+0000 7fb419000640 1 -- 172.21.15.27:0/1101906181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb414113990 con 0x7fb414104740 2024-09-21T00:35:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.303+0000 7fb413400640 1 -- 172.21.15.27:0/1101906181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb414105090 msgr2=0x7fb41419ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.303+0000 7fb413400640 1 --2- 172.21.15.27:0/1101906181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb414105090 0x7fb41419ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.303+0000 7fb413400640 1 -- 172.21.15.27:0/1101906181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb41410dba0 msgr2=0x7fb4141a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.303+0000 7fb413400640 1 --2- 172.21.15.27:0/1101906181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb41410dba0 0x7fb4141a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.303+0000 7fb413400640 1 -- 172.21.15.27:0/1101906181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4141a23a0 con 0x7fb414104740 2024-09-21T00:35:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.304+0000 7fb413400640 1 --2- 172.21.15.27:0/1101906181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb414104740 0x7fb41419c470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb40400e9f0 tx=0x7fb40400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.304+0000 7fb40be00640 1 -- 172.21.15.27:0/1101906181 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40400ccc0 con 0x7fb414104740 2024-09-21T00:35:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.305+0000 7fb40be00640 1 -- 172.21.15.27:0/1101906181 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4040108e0 con 0x7fb414104740 2024-09-21T00:35:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.305+0000 7fb419000640 1 -- 172.21.15.27:0/1101906181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4141a2690 con 0x7fb414104740 2024-09-21T00:35:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.305+0000 7fb419000640 1 -- 172.21.15.27:0/1101906181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4141a8e20 con 0x7fb414104740 2024-09-21T00:35:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.305+0000 7fb40be00640 1 -- 172.21.15.27:0/1101906181 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404018880 con 0x7fb414104740 2024-09-21T00:35:06.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.311+0000 7fb40be00640 1 -- 172.21.15.27:0/1101906181 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb404018a20 con 0x7fb414104740 2024-09-21T00:35:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.313+0000 7fb42162b740 1 Processor -- start 2024-09-21T00:35:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.313+0000 7fb42162b740 1 -- start start 2024-09-21T00:35:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.314+0000 7fb42162b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e2fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.314+0000 7fb42162b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a622f0 0x561e32e30350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.314+0000 7fb42162b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a6a4d0 0x561e32e30890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.314+0000 7fb42162b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561e32da56b0 con 0x561e32a6a4d0 2024-09-21T00:35:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.314+0000 7fb42162b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561e32d94ae0 con 0x561e32a622f0 2024-09-21T00:35:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.314+0000 7fb42162b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561e32d988f0 con 0x561e32a8b1b0 2024-09-21T00:35:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.314+0000 7fb413e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e2fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.314+0000 7fb413e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e2fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56652/0 (socket says 172.21.15.27:56652) 2024-09-21T00:35:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.314+0000 7fb413e00640 1 -- 172.21.15.27:0/3794210028 learned_addr learned my addr 172.21.15.27:0/3794210028 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.315+0000 7fb412a00640 1 --2- 172.21.15.27:0/3794210028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a6a4d0 0x561e32e30890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.315+0000 7fb40be00640 1 --2- 172.21.15.27:0/1101906181 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb3f80b66f0 0x7fb3f80b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.315+0000 7fb413400640 1 --2- 172.21.15.27:0/3794210028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a622f0 0x561e32e30350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.315+0000 7fb40be00640 1 -- 172.21.15.27:0/1101906181 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9607+0+0 (secure 0 0 0) 0x7fb4040e7d30 con 0x7fb414104740 2024-09-21T00:35:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.316+0000 7fb413e00640 1 -- 172.21.15.27:0/3794210028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a622f0 msgr2=0x561e32e30350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.316+0000 7fb413e00640 1 --2- 172.21.15.27:0/3794210028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a622f0 0x561e32e30350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.316+0000 7fb413e00640 1 -- 172.21.15.27:0/3794210028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a6a4d0 msgr2=0x561e32e30890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.316+0000 7fb413e00640 1 --2- 172.21.15.27:0/3794210028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a6a4d0 0x561e32e30890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.316+0000 7fb413e00640 1 -- 172.21.15.27:0/3794210028 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e32d61780 con 0x561e32a8b1b0 2024-09-21T00:35:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.317+0000 7fb412a00640 1 --2- 172.21.15.27:0/3794210028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a6a4d0 0x561e32e30890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:35:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.317+0000 7fb413400640 1 --2- 172.21.15.27:0/3794210028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a622f0 0x561e32e30350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:35:06.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.318+0000 7fb413e00640 1 --2- 172.21.15.27:0/3794210028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e2fe10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009ea0 tx=0x7fb40c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb6947328be1401e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:06.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.318+0000 7fb413e00640 1 --2- 172.21.15.27:0/1101906181 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb3f80b66f0 0x7fb3f80b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.318+0000 7fb408c00640 1 -- 172.21.15.27:0/3794210028 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c004d60 con 0x561e32a8b1b0 2024-09-21T00:35:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.319+0000 7fb408c00640 1 -- 172.21.15.27:0/3794210028 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb40c0054f0 con 0x561e32a8b1b0 2024-09-21T00:35:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.320+0000 7fb42162b740 1 -- 172.21.15.27:0/3794210028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 msgr2=0x561e32e2fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.320+0000 7fb42162b740 1 --2- 172.21.15.27:0/3794210028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e2fe10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009ea0 tx=0x7fb40c009f90 comp rx=0 tx=0).stop 2024-09-21T00:35:06.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.321+0000 7fb42162b740 1 -- 172.21.15.27:0/3794210028 shutdown_connections 2024-09-21T00:35:06.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.321+0000 7fb42162b740 1 --2- 172.21.15.27:0/3794210028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a6a4d0 0x561e32e30890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.321+0000 7fb42162b740 1 --2- 172.21.15.27:0/3794210028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a622f0 0x561e32e30350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.322+0000 7fb42162b740 1 --2- 172.21.15.27:0/3794210028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e2fe10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.322+0000 7fb42162b740 1 -- 172.21.15.27:0/3794210028 >> 172.21.15.27:0/3794210028 conn(0x561e32a7f300 msgr2=0x561e32e2dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:06.324 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.323+0000 7fb42162b740 1 -- 172.21.15.27:0/3794210028 shutdown_connections 2024-09-21T00:35:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.323+0000 7fb42162b740 1 -- 172.21.15.27:0/3794210028 wait complete. 2024-09-21T00:35:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.324+0000 7fb42162b740 1 -- start start 2024-09-21T00:35:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.324+0000 7fb42162b740 1 Processor -- start 2024-09-21T00:35:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.329+0000 7fb42162b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a622f0 0x561e32e3bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.329+0000 7fb42162b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a6a4d0 0x561e32e463b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.329+0000 7fb42162b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e3c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.329+0000 7fb413400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a622f0 0x561e32e3bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.329+0000 7fb413400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a622f0 0x561e32e3bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:56538/0 (socket says 172.21.15.27:56538) 2024-09-21T00:35:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.330+0000 7fb412a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a6a4d0 0x561e32e463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.330+0000 7fb412a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a6a4d0 0x561e32e463b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42542/0 (socket says 172.21.15.27:42542) 2024-09-21T00:35:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.330+0000 7fb413e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e3c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.330+0000 7fb413e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e3c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56654/0 (socket says 172.21.15.27:56654) 2024-09-21T00:35:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.330+0000 7fb413400640 1 -- 172.21.15.27:0/2944001442 learned_addr learned my addr 172.21.15.27:0/2944001442 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.330+0000 7fb42162b740 1 -- 172.21.15.27:0/2944001442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561e32cccab0 con 0x561e32a6a4d0 2024-09-21T00:35:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.330+0000 7fb42162b740 1 -- 172.21.15.27:0/2944001442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561e32d3b570 con 0x561e32a622f0 2024-09-21T00:35:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.330+0000 7fb42162b740 1 -- 172.21.15.27:0/2944001442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561e32e36590 con 0x561e32a8b1b0 2024-09-21T00:35:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.330+0000 7fb42162b740 10 client.0 _ll_register_callbacks cb 0x561e32d4a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:35:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.331+0000 7fb413400640 1 -- 172.21.15.27:0/2944001442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 msgr2=0x561e32e3c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.331+0000 7fb413400640 1 --2- 172.21.15.27:0/2944001442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e3c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.331+0000 7fb413400640 1 -- 172.21.15.27:0/2944001442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a6a4d0 msgr2=0x561e32e463b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.331+0000 7fb413400640 1 --2- 172.21.15.27:0/2944001442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a6a4d0 0x561e32e463b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.331+0000 7fb413400640 1 -- 172.21.15.27:0/2944001442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e32af6790 con 0x561e32a622f0 2024-09-21T00:35:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.331+0000 7fb413400640 1 --2- 172.21.15.27:0/2944001442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a622f0 0x561e32e3bc30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb404000c00 tx=0x7fb404010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1e9df9e9b0b8300 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.332+0000 7fb3fd600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:35:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.332+0000 7fb3fd600640 1 -- 172.21.15.27:0/2944001442 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4040aceb0 con 0x561e32a622f0 2024-09-21T00:35:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.332+0000 7fb413e00640 1 --2- 172.21.15.27:0/1101906181 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb3f80b66f0 0x7fb3f80b8c00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb40c010620 tx=0x7fb40c009f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.332+0000 7fb3fd600640 1 -- 172.21.15.27:0/2944001442 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9607+0+0 (secure 0 0 0) 0x7fb4040a9090 con 0x561e32a622f0 2024-09-21T00:35:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.332+0000 7fb42162b740 1 -- 172.21.15.27:0/2944001442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb40c0054f0 con 0x561e32a622f0 2024-09-21T00:35:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.333+0000 7fb3fd600640 1 -- 172.21.15.27:0/2944001442 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb4040aceb0 con 0x561e32a622f0 2024-09-21T00:35:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.334+0000 7fb42162b740 10 client.26666 fetch_fsmap learned FSMap version 47 2024-09-21T00:35:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.334+0000 7fb42162b740 1 -- 172.21.15.27:0/2944001442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561e32b6f620 con 0x561e32a622f0 2024-09-21T00:35:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.334+0000 7fb3fd600640 1 -- 172.21.15.27:0/2944001442 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb40c0054f0 con 0x561e32a622f0 2024-09-21T00:35:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.336+0000 7fb3fd600640 1 -- 172.21.15.27:0/2944001442 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fb4040ac410 con 0x561e32a622f0 2024-09-21T00:35:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.337+0000 7fb42162b740 10 client.26666 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:35:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.337+0000 7fb42162b740 10 client.26666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:35:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.337+0000 7fb42162b740 20 client.26666 populate_metadata read hostname 'smithi027' 2024-09-21T00:35:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.337+0000 7fb42162b740 1 --2- 172.21.15.27:0/2944001442 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561e32a49d80 0x561e32e5e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.337+0000 7fb42162b740 4 client.26666 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:35:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.337+0000 7fb42162b740 1 -- 172.21.15.27:0/2944001442 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e32da7600 con 0x561e32a49d80 2024-09-21T00:35:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.337+0000 7fb412a00640 1 --2- 172.21.15.27:0/2944001442 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561e32a49d80 0x561e32e5e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.340+0000 7fb412a00640 1 --2- 172.21.15.27:0/2944001442 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561e32a49d80 0x561e32e5e4c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=693d0367246264f7 server_cookie=db798b28a35ba678 in_seq=0 out_seq=0 2024-09-21T00:35:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.340+0000 7fb3fd600640 10 client.26666 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.344+0000 7fb3fd600640 1 -- 172.21.15.27:0/2944001442 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561e32da7600 con 0x561e32a49d80 2024-09-21T00:35:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.344+0000 7fb3fd600640 10 client.26666 handle_command_reply: tid=0 2024-09-21T00:35:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.344+0000 7fb42162b740 1 client.26666 shutdown 2024-09-21T00:35:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.344+0000 7fb42162b740 1 -- 172.21.15.27:0/2944001442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a622f0 msgr2=0x561e32e3bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.344+0000 7fb42162b740 1 --2- 172.21.15.27:0/2944001442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a622f0 0x561e32e3bc30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb404000c00 tx=0x7fb404010800 comp rx=0 tx=0).stop 2024-09-21T00:35:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.344+0000 7fb42162b740 1 -- 172.21.15.27:0/2944001442 shutdown_connections 2024-09-21T00:35:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.344+0000 7fb42162b740 1 --2- 172.21.15.27:0/2944001442 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561e32a49d80 0x561e32e5e4c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.345+0000 7fb42162b740 1 --2- 172.21.15.27:0/2944001442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e3c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.345+0000 7fb3fd600640 0 client.26666 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.345+0000 7fb42162b740 1 --2- 172.21.15.27:0/2944001442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a6a4d0 0x561e32e463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.345+0000 7fb42162b740 1 --2- 172.21.15.27:0/2944001442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a622f0 0x561e32e3bc30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.345+0000 7fb42162b740 1 -- 172.21.15.27:0/2944001442 >> 172.21.15.27:0/2944001442 conn(0x561e32a7f300 msgr2=0x561e32e2c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.345+0000 7fb42162b740 1 -- 172.21.15.27:0/2944001442 shutdown_connections 2024-09-21T00:35:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.345+0000 7fb42162b740 1 -- 172.21.15.27:0/2944001442 wait complete. 2024-09-21T00:35:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.345+0000 7fb42162b740 20 client.26666 trim_cache size 0 max 16384 2024-09-21T00:35:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.386+0000 7fb42162b740 1 Processor -- start 2024-09-21T00:35:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.386+0000 7fb42162b740 1 -- start start 2024-09-21T00:35:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.387+0000 7fb42162b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a49d80 0x561e32e3b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.387+0000 7fb42162b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a622f0 0x561e32e2c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.387+0000 7fb42162b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a6a4d0 0x561e32e36890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.387+0000 7fb42162b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f4012c60 con 0x561e32a622f0 2024-09-21T00:35:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.387+0000 7fb42162b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f4010470 con 0x561e32a6a4d0 2024-09-21T00:35:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.387+0000 7fb42162b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f400e280 con 0x561e32a49d80 2024-09-21T00:35:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.388+0000 7fb413400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a49d80 0x561e32e3b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.388+0000 7fb412a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a622f0 0x561e32e2c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.388+0000 7fb413e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a6a4d0 0x561e32e36890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.388+0000 7fb413e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a6a4d0 0x561e32e36890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:56542/0 (socket says 172.21.15.27:56542) 2024-09-21T00:35:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.388+0000 7fb413e00640 1 -- 172.21.15.27:0/172249343 learned_addr learned my addr 172.21.15.27:0/172249343 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.389+0000 7fb413e00640 1 -- 172.21.15.27:0/172249343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a49d80 msgr2=0x561e32e3b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.390+0000 7fb413e00640 1 --2- 172.21.15.27:0/172249343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a49d80 0x561e32e3b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.390+0000 7fb413e00640 1 -- 172.21.15.27:0/172249343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a622f0 msgr2=0x561e32e2c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.390+0000 7fb413e00640 1 --2- 172.21.15.27:0/172249343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a622f0 0x561e32e2c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.390+0000 7fb413e00640 1 -- 172.21.15.27:0/172249343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e32e2dd70 con 0x561e32a6a4d0 2024-09-21T00:35:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.390+0000 7fb413400640 1 --2- 172.21.15.27:0/172249343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a49d80 0x561e32e3b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:35:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.390+0000 7fb412a00640 1 --2- 172.21.15.27:0/172249343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a622f0 0x561e32e2c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:35:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.392+0000 7fb413e00640 1 --2- 172.21.15.27:0/172249343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a6a4d0 0x561e32e36890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009e70 tx=0x7fb40c025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15ba3b641439aed8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.393+0000 7fb3ffe00640 1 -- 172.21.15.27:0/172249343 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c009b60 con 0x561e32a6a4d0 2024-09-21T00:35:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.393+0000 7fb3ffe00640 1 -- 172.21.15.27:0/172249343 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb40c036070 con 0x561e32a6a4d0 2024-09-21T00:35:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.394+0000 7fb42162b740 1 -- 172.21.15.27:0/172249343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a6a4d0 msgr2=0x561e32e36890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.394+0000 7fb42162b740 1 --2- 172.21.15.27:0/172249343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a6a4d0 0x561e32e36890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009e70 tx=0x7fb40c025000 comp rx=0 tx=0).stop 2024-09-21T00:35:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.395+0000 7fb42162b740 1 -- 172.21.15.27:0/172249343 shutdown_connections 2024-09-21T00:35:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.395+0000 7fb42162b740 1 --2- 172.21.15.27:0/172249343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a6a4d0 0x561e32e36890 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.395+0000 7fb42162b740 1 --2- 172.21.15.27:0/172249343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a622f0 0x561e32e2c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.395+0000 7fb42162b740 1 --2- 172.21.15.27:0/172249343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a49d80 0x561e32e3b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.395+0000 7fb42162b740 1 -- 172.21.15.27:0/172249343 >> 172.21.15.27:0/172249343 conn(0x561e32a7f300 msgr2=0x561e32e2d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.393+0000 7fb3ffe00640 1 -- 172.21.15.27:0/172249343 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c025480 con 0x561e32a6a4d0 2024-09-21T00:35:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.396+0000 7fb42162b740 1 -- 172.21.15.27:0/172249343 shutdown_connections 2024-09-21T00:35:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.396+0000 7fb42162b740 1 -- 172.21.15.27:0/172249343 wait complete. 2024-09-21T00:35:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.397+0000 7fb42162b740 1 -- start start 2024-09-21T00:35:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.397+0000 7fb42162b740 1 Processor -- start 2024-09-21T00:35:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.398+0000 7fb42162b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a49d80 0x561e32e6fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.402+0000 7fb42162b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a622f0 0x561e32e70300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.402+0000 7fb42162b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e6b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.402+0000 7fb413400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a49d80 0x561e32e6fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.403+0000 7fb413400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a49d80 0x561e32e6fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:56556/0 (socket says 172.21.15.27:56556) 2024-09-21T00:35:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.403+0000 7fb413400640 1 -- 172.21.15.27:0/2045123805 learned_addr learned my addr 172.21.15.27:0/2045123805 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.403+0000 7fb42162b740 1 -- 172.21.15.27:0/2045123805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f400e930 con 0x561e32a622f0 2024-09-21T00:35:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.403+0000 7fb42162b740 1 -- 172.21.15.27:0/2045123805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f400fda0 con 0x561e32a49d80 2024-09-21T00:35:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.403+0000 7fb42162b740 1 -- 172.21.15.27:0/2045123805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f400f000 con 0x561e32a8b1b0 2024-09-21T00:35:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.403+0000 7fb413e00640 1 --2- 172.21.15.27:0/2045123805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e6b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.403+0000 7fb42162b740 10 client.0 _ll_register_callbacks cb 0x7fb40400c7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:35:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.403+0000 7fb413e00640 1 -- 172.21.15.27:0/2045123805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a49d80 msgr2=0x561e32e6fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.403+0000 7fb413e00640 1 --2- 172.21.15.27:0/2045123805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a49d80 0x561e32e6fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.403+0000 7fb413e00640 1 -- 172.21.15.27:0/2045123805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a622f0 msgr2=0x561e32e70300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.403+0000 7fb413e00640 1 --2- 172.21.15.27:0/2045123805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a622f0 0x561e32e70300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.403+0000 7fb413e00640 1 -- 172.21.15.27:0/2045123805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e32e6bab0 con 0x561e32a8b1b0 2024-09-21T00:35:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.404+0000 7fb413e00640 1 --2- 172.21.15.27:0/2045123805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e6b340 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb40c01a040 tx=0x7fb40c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=372ee25e716486b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.404+0000 7fb3fd600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:35:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.405+0000 7fb3fd600640 1 -- 172.21.15.27:0/2045123805 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c03e070 con 0x561e32a8b1b0 2024-09-21T00:35:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.405+0000 7fb42162b740 1 -- 172.21.15.27:0/2045123805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb40c036070 con 0x561e32a8b1b0 2024-09-21T00:35:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.405+0000 7fb3fd600640 1 -- 172.21.15.27:0/2045123805 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9607+0+0 (secure 0 0 0) 0x7fb40c004800 con 0x561e32a8b1b0 2024-09-21T00:35:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.405+0000 7fb3fd600640 1 -- 172.21.15.27:0/2045123805 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb40c049050 con 0x561e32a8b1b0 2024-09-21T00:35:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.405+0000 7fb42162b740 10 client.26695 fetch_fsmap learned FSMap version 47 2024-09-21T00:35:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.405+0000 7fb42162b740 1 -- 172.21.15.27:0/2045123805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561e32ec88d0 con 0x561e32a8b1b0 2024-09-21T00:35:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.406+0000 7fb3fd600640 1 -- 172.21.15.27:0/2045123805 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb40c036070 con 0x561e32a8b1b0 2024-09-21T00:35:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.406+0000 7fb3fd600640 1 -- 172.21.15.27:0/2045123805 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fb40c025b70 con 0x561e32a8b1b0 2024-09-21T00:35:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.407+0000 7fb42162b740 10 client.26695 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:35:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.407+0000 7fb42162b740 10 client.26695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:35:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.407+0000 7fb42162b740 20 client.26695 populate_metadata read hostname 'smithi027' 2024-09-21T00:35:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.407+0000 7fb42162b740 1 --2- 172.21.15.27:0/2045123805 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561e32a1f1c0 0x561e32ecaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.407+0000 7fb42162b740 4 client.26695 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:35:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.407+0000 7fb42162b740 1 -- 172.21.15.27:0/2045123805 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e32da30b0 con 0x561e32a1f1c0 2024-09-21T00:35:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.407+0000 7fb413400640 1 --2- 172.21.15.27:0/2045123805 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561e32a1f1c0 0x561e32ecaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.410+0000 7fb413400640 1 --2- 172.21.15.27:0/2045123805 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561e32a1f1c0 0x561e32ecaee0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=975f02f45ffc5efd server_cookie=e6f26074496efc8f in_seq=0 out_seq=0 2024-09-21T00:35:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:06.410+0000 7fb3fd600640 10 client.26695 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.162+0000 7fb3fd600640 1 -- 172.21.15.27:0/2045123805 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e32da30b0 con 0x561e32a1f1c0 2024-09-21T00:35:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.162+0000 7fb3fd600640 10 client.26695 handle_command_reply: tid=0 2024-09-21T00:35:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.162+0000 7fb42162b740 1 client.26695 shutdown 2024-09-21T00:35:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.163+0000 7fb42162b740 1 -- 172.21.15.27:0/2045123805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 msgr2=0x561e32e6b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.163+0000 7fb42162b740 1 --2- 172.21.15.27:0/2045123805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e6b340 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb40c01a040 tx=0x7fb40c00ee60 comp rx=0 tx=0).stop 2024-09-21T00:35:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.163+0000 7fb42162b740 1 -- 172.21.15.27:0/2045123805 shutdown_connections 2024-09-21T00:35:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.163+0000 7fb42162b740 1 --2- 172.21.15.27:0/2045123805 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x561e32a1f1c0 0x561e32ecaee0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.163+0000 7fb42162b740 1 --2- 172.21.15.27:0/2045123805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e32a8b1b0 0x561e32e6b340 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.163+0000 7fb42162b740 1 --2- 172.21.15.27:0/2045123805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e32a622f0 0x561e32e70300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.163+0000 7fb42162b740 1 --2- 172.21.15.27:0/2045123805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e32a49d80 0x561e32e6fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.163+0000 7fb42162b740 1 -- 172.21.15.27:0/2045123805 >> 172.21.15.27:0/2045123805 conn(0x561e32a7f300 msgr2=0x561e32e2d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.163+0000 7fb3fd600640 0 client.26695 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.163+0000 7fb42162b740 1 -- 172.21.15.27:0/2045123805 shutdown_connections 2024-09-21T00:35:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.163+0000 7fb42162b740 1 -- 172.21.15.27:0/2045123805 wait complete. 2024-09-21T00:35:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.164+0000 7fb42162b740 20 client.26695 trim_cache size 0 max 16384 2024-09-21T00:35:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:35:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:35:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:35:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:35:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.165+0000 7fb419000640 1 -- 172.21.15.27:0/1101906181 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb3f80b66f0 msgr2=0x7fb3f80b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.165+0000 7fb419000640 1 --2- 172.21.15.27:0/1101906181 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb3f80b66f0 0x7fb3f80b8c00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb40c010620 tx=0x7fb40c009f90 comp rx=0 tx=0).stop 2024-09-21T00:35:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.165+0000 7fb419000640 1 -- 172.21.15.27:0/1101906181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb414104740 msgr2=0x7fb41419c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.165+0000 7fb419000640 1 --2- 172.21.15.27:0/1101906181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb414104740 0x7fb41419c470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb40400e9f0 tx=0x7fb40400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:35:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.165+0000 7fb419000640 1 -- 172.21.15.27:0/1101906181 shutdown_connections 2024-09-21T00:35:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.165+0000 7fb419000640 1 --2- 172.21.15.27:0/1101906181 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fb3f80b66f0 0x7fb3f80b8c00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.165+0000 7fb419000640 1 --2- 172.21.15.27:0/1101906181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb41410dba0 0x7fb4141a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.165+0000 7fb419000640 1 --2- 172.21.15.27:0/1101906181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb414105090 0x7fb41419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.165+0000 7fb419000640 1 --2- 172.21.15.27:0/1101906181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb414104740 0x7fb41419c470 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.165+0000 7fb419000640 1 -- 172.21.15.27:0/1101906181 >> 172.21.15.27:0/1101906181 conn(0x7fb4140fe210 msgr2=0x7fb414102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.167+0000 7fb419000640 1 -- 172.21.15.27:0/1101906181 shutdown_connections 2024-09-21T00:35:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:07.167+0000 7fb419000640 1 -- 172.21.15.27:0/1101906181 wait complete. 2024-09-21T00:35:08.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:07 smithi089 ceph-mon[34481]: pgmap v808: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 269 op/s 2024-09-21T00:35:08.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:07 smithi098 ceph-mon[34725]: pgmap v808: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 269 op/s 2024-09-21T00:35:08.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:07 smithi027 ceph-mon[30745]: pgmap v808: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 269 op/s 2024-09-21T00:35:09.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:08 smithi089 ceph-mon[34481]: osdmap e110: 12 total, 12 up, 12 in 2024-09-21T00:35:09.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:08 smithi098 ceph-mon[34725]: osdmap e110: 12 total, 12 up, 12 in 2024-09-21T00:35:09.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:08 smithi027 ceph-mon[30745]: osdmap e110: 12 total, 12 up, 12 in 2024-09-21T00:35:10.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:09 smithi027 ceph-mon[30745]: pgmap v810: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 999 KiB/s wr, 332 op/s 2024-09-21T00:35:10.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:09 smithi089 ceph-mon[34481]: pgmap v810: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 999 KiB/s wr, 332 op/s 2024-09-21T00:35:10.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:09 smithi098 ceph-mon[34725]: pgmap v810: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 999 KiB/s wr, 332 op/s 2024-09-21T00:35:12.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:11 smithi089 ceph-mon[34481]: pgmap v811: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 399 op/s 2024-09-21T00:35:12.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:11 smithi098 ceph-mon[34725]: pgmap v811: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 399 op/s 2024-09-21T00:35:12.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:11 smithi027 ceph-mon[30745]: pgmap v811: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 399 op/s 2024-09-21T00:35:14.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:13 smithi089 ceph-mon[34481]: pgmap v812: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-09-21T00:35:14.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:13 smithi098 ceph-mon[34725]: pgmap v812: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-09-21T00:35:14.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:13 smithi027 ceph-mon[30745]: pgmap v812: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-09-21T00:35:16.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:15 smithi089 ceph-mon[34481]: pgmap v813: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 453 op/s 2024-09-21T00:35:16.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:15 smithi098 ceph-mon[34725]: pgmap v813: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 453 op/s 2024-09-21T00:35:16.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:15 smithi027 ceph-mon[30745]: pgmap v813: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 453 op/s 2024-09-21T00:35:18.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:17 smithi089 ceph-mon[34481]: pgmap v814: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 437 op/s 2024-09-21T00:35:18.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:17 smithi098 ceph-mon[34725]: pgmap v814: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 437 op/s 2024-09-21T00:35:18.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:17 smithi027 ceph-mon[30745]: pgmap v814: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 437 op/s 2024-09-21T00:35:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:19 smithi089 ceph-mon[34481]: pgmap v815: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 409 op/s 2024-09-21T00:35:20.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:19 smithi098 ceph-mon[34725]: pgmap v815: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 409 op/s 2024-09-21T00:35:20.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:19 smithi027 ceph-mon[30745]: pgmap v815: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 409 op/s 2024-09-21T00:35:20.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.519+0000 7f7e7fe00640 1 -- 172.21.15.27:0/1600408340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e80113c40 msgr2=0x7f7e80114050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:20.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.519+0000 7f7e7fe00640 1 --2- 172.21.15.27:0/1600408340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e80113c40 0x7f7e80114050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009900 tx=0x7f7e6802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.519+0000 7f7e7fe00640 1 -- 172.21.15.27:0/1600408340 shutdown_connections 2024-09-21T00:35:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.519+0000 7f7e7fe00640 1 --2- 172.21.15.27:0/1600408340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e80114590 0x7f7e8011adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.519+0000 7f7e7fe00640 1 --2- 172.21.15.27:0/1600408340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e80113c40 0x7f7e80114050 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.519+0000 7f7e7fe00640 1 --2- 172.21.15.27:0/1600408340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e8010e310 0x7f7e8010e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.519+0000 7f7e7fe00640 1 -- 172.21.15.27:0/1600408340 >> 172.21.15.27:0/1600408340 conn(0x7f7e8006e610 msgr2=0x7f7e8010c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.521+0000 7f7e7fe00640 1 -- 172.21.15.27:0/1600408340 shutdown_connections 2024-09-21T00:35:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.521+0000 7f7e7fe00640 1 -- 172.21.15.27:0/1600408340 wait complete. 2024-09-21T00:35:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.522+0000 7f7e7fe00640 1 Processor -- start 2024-09-21T00:35:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.523+0000 7f7e7fe00640 1 -- start start 2024-09-21T00:35:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.523+0000 7f7e7fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e8010e310 0x7f7e8011c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.523+0000 7f7e7fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e80113c40 0x7f7e8011cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.523+0000 7f7e7fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e80114590 0x7f7e80123660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.523+0000 7f7e7fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e800707a0 con 0x7f7e80113c40 2024-09-21T00:35:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.523+0000 7f7e7fe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7e80070620 con 0x7f7e8010e310 2024-09-21T00:35:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.523+0000 7f7e7fe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7e80070920 con 0x7f7e80114590 2024-09-21T00:35:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.523+0000 7f7e7ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e8010e310 0x7f7e8011c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.524+0000 7f7e7ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e8010e310 0x7f7e8011c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54074/0 (socket says 172.21.15.27:54074) 2024-09-21T00:35:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.524+0000 7f7e7ea00640 1 -- 172.21.15.27:0/3888757392 learned_addr learned my addr 172.21.15.27:0/3888757392 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.524+0000 7f7e7f400640 1 --2- 172.21.15.27:0/3888757392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e80114590 0x7f7e80123660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.524+0000 7f7e7e000640 1 --2- 172.21.15.27:0/3888757392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e80113c40 0x7f7e8011cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.525+0000 7f7e7ea00640 1 -- 172.21.15.27:0/3888757392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e80114590 msgr2=0x7f7e80123660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.525+0000 7f7e7ea00640 1 --2- 172.21.15.27:0/3888757392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e80114590 0x7f7e80123660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.525+0000 7f7e7ea00640 1 -- 172.21.15.27:0/3888757392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e80113c40 msgr2=0x7f7e8011cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.525+0000 7f7e7ea00640 1 --2- 172.21.15.27:0/3888757392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e80113c40 0x7f7e8011cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.525+0000 7f7e7ea00640 1 -- 172.21.15.27:0/3888757392 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e80123c70 con 0x7f7e8010e310 2024-09-21T00:35:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.525+0000 7f7e7e000640 1 --2- 172.21.15.27:0/3888757392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e80113c40 0x7f7e8011cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:35:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.526+0000 7f7e7ea00640 1 --2- 172.21.15.27:0/3888757392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e8010e310 0x7f7e8011c7e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800ba50 tx=0x7f7e7800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.526+0000 7f7e77400640 1 -- 172.21.15.27:0/3888757392 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78019070 con 0x7f7e8010e310 2024-09-21T00:35:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.527+0000 7f7e77400640 1 -- 172.21.15.27:0/3888757392 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e7800cbe0 con 0x7f7e8010e310 2024-09-21T00:35:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.527+0000 7f7e7fe00640 1 -- 172.21.15.27:0/3888757392 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e80123f60 con 0x7f7e8010e310 2024-09-21T00:35:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.528+0000 7f7e7fe00640 1 -- 172.21.15.27:0/3888757392 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e80124550 con 0x7f7e8010e310 2024-09-21T00:35:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.528+0000 7f7e77400640 1 -- 172.21.15.27:0/3888757392 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e780075d0 con 0x7f7e8010e310 2024-09-21T00:35:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.532+0000 7f7e8cc09740 1 Processor -- start 2024-09-21T00:35:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.532+0000 7f7e8cc09740 1 -- start start 2024-09-21T00:35:20.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.533+0000 7f7e8cc09740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07451b0 0x5573b0ae9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.533+0000 7f7e8cc09740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b071c2f0 0x5573b0aea350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.533+0000 7f7e8cc09740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b07244d0 0x5573b0aea890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.533+0000 7f7e8cc09740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5573b0a5f6b0 con 0x5573b07244d0 2024-09-21T00:35:20.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.533+0000 7f7e8cc09740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5573b0a4eae0 con 0x5573b07451b0 2024-09-21T00:35:20.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.533+0000 7f7e8cc09740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5573b0a528f0 con 0x5573b071c2f0 2024-09-21T00:35:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.535+0000 7f7e7f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07451b0 0x5573b0ae9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.536+0000 7f7e7f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07451b0 0x5573b0ae9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54078/0 (socket says 172.21.15.27:54078) 2024-09-21T00:35:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.536+0000 7f7e7f400640 1 -- 172.21.15.27:0/95597936 learned_addr learned my addr 172.21.15.27:0/95597936 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.538+0000 7f7e7f400640 1 -- 172.21.15.27:0/95597936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b071c2f0 msgr2=0x5573b0aea350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:35:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.538+0000 7f7e7f400640 1 --2- 172.21.15.27:0/95597936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b071c2f0 0x5573b0aea350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.538+0000 7f7e7f400640 1 -- 172.21.15.27:0/95597936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b07244d0 msgr2=0x5573b0aea890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.538+0000 7f7e7f400640 1 --2- 172.21.15.27:0/95597936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b07244d0 0x5573b0aea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.538+0000 7f7e7f400640 1 -- 172.21.15.27:0/95597936 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b0a1b780 con 0x5573b07451b0 2024-09-21T00:35:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.540+0000 7f7e77400640 1 -- 172.21.15.27:0/3888757392 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7e7801c070 con 0x7f7e8010e310 2024-09-21T00:35:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.540+0000 7f7e7f400640 1 --2- 172.21.15.27:0/95597936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07451b0 0x5573b0ae9e10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009c30 tx=0x7f7e700117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23766a8a58c01190 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.541+0000 7f7e6fe00640 1 -- 172.21.15.27:0/95597936 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7001b470 con 0x5573b07451b0 2024-09-21T00:35:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.541+0000 7f7e6fe00640 1 -- 172.21.15.27:0/95597936 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e70016070 con 0x5573b07451b0 2024-09-21T00:35:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.541+0000 7f7e6fe00640 1 -- 172.21.15.27:0/95597936 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7000f800 con 0x5573b07451b0 2024-09-21T00:35:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.544+0000 7f7e8cc09740 1 -- 172.21.15.27:0/95597936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07451b0 msgr2=0x5573b0ae9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.544+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/95597936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07451b0 0x5573b0ae9e10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009c30 tx=0x7f7e700117b0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.544+0000 7f7e8cc09740 1 -- 172.21.15.27:0/95597936 shutdown_connections 2024-09-21T00:35:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.544+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/95597936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b07244d0 0x5573b0aea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.544+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/95597936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b071c2f0 0x5573b0aea350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.544+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/95597936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07451b0 0x5573b0ae9e10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.544+0000 7f7e8cc09740 1 -- 172.21.15.27:0/95597936 >> 172.21.15.27:0/95597936 conn(0x5573b0739300 msgr2=0x5573b0ae7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.547+0000 7f7e8cc09740 1 -- 172.21.15.27:0/95597936 shutdown_connections 2024-09-21T00:35:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.547+0000 7f7e77400640 1 --2- 172.21.15.27:0/3888757392 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7e640b6600 0x7f7e640b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.547+0000 7f7e8cc09740 1 -- 172.21.15.27:0/95597936 wait complete. 2024-09-21T00:35:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.547+0000 7f7e77400640 1 -- 172.21.15.27:0/3888757392 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7f7e780e6dd0 con 0x7f7e8010e310 2024-09-21T00:35:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.547+0000 7f7e7e000640 1 --2- 172.21.15.27:0/3888757392 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7e640b6600 0x7f7e640b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.549+0000 7f7e8cc09740 1 -- start start 2024-09-21T00:35:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.549+0000 7f7e7e000640 1 --2- 172.21.15.27:0/3888757392 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7e640b6600 0x7f7e640b8b10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009230 tx=0x7f7e68000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.550+0000 7f7e8cc09740 1 Processor -- start 2024-09-21T00:35:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.552+0000 7f7e8cc09740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b071c2f0 0x5573b0af5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.552+0000 7f7e8cc09740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07244d0 0x5573b0af58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.552+0000 7f7e8cc09740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b07451b0 0x5573b0af5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.552+0000 7f7e8cc09740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5573b0986ab0 con 0x5573b071c2f0 2024-09-21T00:35:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.552+0000 7f7e8cc09740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5573b09f5570 con 0x5573b07244d0 2024-09-21T00:35:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.552+0000 7f7e8cc09740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5573b0af8590 con 0x5573b07451b0 2024-09-21T00:35:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.552+0000 7f7e8cc09740 10 client.0 _ll_register_callbacks cb 0x5573b0a04100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:35:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.552+0000 7f7e7e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07244d0 0x5573b0af58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.553+0000 7f7e7e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07244d0 0x5573b0af58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54084/0 (socket says 172.21.15.27:54084) 2024-09-21T00:35:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.553+0000 7f7e7e000640 1 -- 172.21.15.27:0/1404423514 learned_addr learned my addr 172.21.15.27:0/1404423514 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.553+0000 7f7e7ea00640 1 --2- 172.21.15.27:0/1404423514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b071c2f0 0x5573b0af5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.553+0000 7f7e7f400640 1 --2- 172.21.15.27:0/1404423514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b07451b0 0x5573b0af5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.554+0000 7f7e7f400640 1 -- 172.21.15.27:0/1404423514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07244d0 msgr2=0x5573b0af58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.554+0000 7f7e7f400640 1 --2- 172.21.15.27:0/1404423514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07244d0 0x5573b0af58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.554+0000 7f7e7f400640 1 -- 172.21.15.27:0/1404423514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b071c2f0 msgr2=0x5573b0af5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.554+0000 7f7e7f400640 1 --2- 172.21.15.27:0/1404423514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b071c2f0 0x5573b0af5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.554+0000 7f7e7f400640 1 -- 172.21.15.27:0/1404423514 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b07b0790 con 0x5573b07451b0 2024-09-21T00:35:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.554+0000 7f7e7f400640 1 --2- 172.21.15.27:0/1404423514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b07451b0 0x5573b0af5df0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009390 tx=0x7f7e700052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c379bfbfce53c4da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.554+0000 7f7e6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:35:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.555+0000 7f7e6cc00640 1 -- 172.21.15.27:0/1404423514 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70012650 con 0x5573b07451b0 2024-09-21T00:35:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.555+0000 7f7e6cc00640 1 -- 172.21.15.27:0/1404423514 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7f7e700127f0 con 0x5573b07451b0 2024-09-21T00:35:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.555+0000 7f7e8cc09740 1 -- 172.21.15.27:0/1404423514 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e70016070 con 0x5573b07451b0 2024-09-21T00:35:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.558+0000 7f7e6cc00640 1 -- 172.21.15.27:0/1404423514 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f7e70012a70 con 0x5573b07451b0 2024-09-21T00:35:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.558+0000 7f7e8cc09740 10 client.26704 fetch_fsmap learned FSMap version 47 2024-09-21T00:35:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.558+0000 7f7e8cc09740 1 -- 172.21.15.27:0/1404423514 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5573b0829620 con 0x5573b07451b0 2024-09-21T00:35:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.562+0000 7f7e6cc00640 1 -- 172.21.15.27:0/1404423514 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e70016070 con 0x5573b07451b0 2024-09-21T00:35:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.562+0000 7f7e6cc00640 1 -- 172.21.15.27:0/1404423514 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f7e7000f5d0 con 0x5573b07451b0 2024-09-21T00:35:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.562+0000 7f7e8cc09740 10 client.26704 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:35:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.562+0000 7f7e8cc09740 10 client.26704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:35:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.562+0000 7f7e8cc09740 20 client.26704 populate_metadata read hostname 'smithi027' 2024-09-21T00:35:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.562+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/1404423514 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5573b0703d80 0x5573b0b078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.562+0000 7f7e8cc09740 4 client.26704 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:35:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.562+0000 7f7e8cc09740 1 -- 172.21.15.27:0/1404423514 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573b0a61600 con 0x5573b0703d80 2024-09-21T00:35:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.563+0000 7f7e7ea00640 1 --2- 172.21.15.27:0/1404423514 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5573b0703d80 0x5573b0b078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.564+0000 7f7e7ea00640 1 --2- 172.21.15.27:0/1404423514 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5573b0703d80 0x5573b0b078d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77d15999c35f77aa server_cookie=d1a117b687b6a5c5 in_seq=0 out_seq=0 2024-09-21T00:35:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.564+0000 7f7e6cc00640 10 client.26704 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.567+0000 7f7e6cc00640 1 -- 172.21.15.27:0/1404423514 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5573b0a61600 con 0x5573b0703d80 2024-09-21T00:35:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.567+0000 7f7e6cc00640 10 client.26704 handle_command_reply: tid=0 2024-09-21T00:35:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.567+0000 7f7e8cc09740 1 client.26704 shutdown 2024-09-21T00:35:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.567+0000 7f7e8cc09740 1 -- 172.21.15.27:0/1404423514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b07451b0 msgr2=0x5573b0af5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.567+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/1404423514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b07451b0 0x5573b0af5df0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009390 tx=0x7f7e700052f0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.568+0000 7f7e8cc09740 1 -- 172.21.15.27:0/1404423514 shutdown_connections 2024-09-21T00:35:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.568+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/1404423514 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5573b0703d80 0x5573b0b078d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.568+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/1404423514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b07451b0 0x5573b0af5df0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.568+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/1404423514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07244d0 0x5573b0af58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.568+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/1404423514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b071c2f0 0x5573b0af5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.568+0000 7f7e8cc09740 1 -- 172.21.15.27:0/1404423514 >> 172.21.15.27:0/1404423514 conn(0x5573b0739300 msgr2=0x5573b0aeae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.568+0000 7f7e6cc00640 0 client.26704 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.568+0000 7f7e8cc09740 1 -- 172.21.15.27:0/1404423514 shutdown_connections 2024-09-21T00:35:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.569+0000 7f7e8cc09740 1 -- 172.21.15.27:0/1404423514 wait complete. 2024-09-21T00:35:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.569+0000 7f7e8cc09740 20 client.26704 trim_cache size 0 max 16384 2024-09-21T00:35:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.623+0000 7f7e8cc09740 1 Processor -- start 2024-09-21T00:35:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.625+0000 7f7e8cc09740 1 -- start start 2024-09-21T00:35:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.625+0000 7f7e8cc09740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b071c2f0 0x5573b0af4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.625+0000 7f7e8cc09740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b07244d0 0x5573b0af2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.625+0000 7f7e8cc09740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07451b0 0x5573b0aeb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.625+0000 7f7e8cc09740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c012b90 con 0x5573b07244d0 2024-09-21T00:35:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.625+0000 7f7e8cc09740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c0103a0 con 0x5573b07451b0 2024-09-21T00:35:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.625+0000 7f7e8cc09740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c00e1d0 con 0x5573b071c2f0 2024-09-21T00:35:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.625+0000 7f7e7ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b071c2f0 0x5573b0af4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.626+0000 7f7e7ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b071c2f0 0x5573b0af4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:43324/0 (socket says 172.21.15.27:43324) 2024-09-21T00:35:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.626+0000 7f7e7ea00640 1 -- 172.21.15.27:0/463375964 learned_addr learned my addr 172.21.15.27:0/463375964 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.626+0000 7f7e7f400640 1 --2- 172.21.15.27:0/463375964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07451b0 0x5573b0aeb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.626+0000 7f7e7e000640 1 --2- 172.21.15.27:0/463375964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b07244d0 0x5573b0af2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.626+0000 7f7e7ea00640 1 -- 172.21.15.27:0/463375964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07451b0 msgr2=0x5573b0aeb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.626+0000 7f7e7ea00640 1 --2- 172.21.15.27:0/463375964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07451b0 0x5573b0aeb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.626+0000 7f7e7ea00640 1 -- 172.21.15.27:0/463375964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b07244d0 msgr2=0x5573b0af2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.626+0000 7f7e7ea00640 1 --2- 172.21.15.27:0/463375964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b07244d0 0x5573b0af2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.626+0000 7f7e7ea00640 1 -- 172.21.15.27:0/463375964 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b0af6630 con 0x5573b071c2f0 2024-09-21T00:35:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.626+0000 7f7e7e000640 1 --2- 172.21.15.27:0/463375964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b07244d0 0x5573b0af2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:35:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.626+0000 7f7e7ea00640 1 --2- 172.21.15.27:0/463375964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b071c2f0 0x5573b0af4260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7e780a80d0 tx=0x7f7e78004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb32f7db51941739 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.627+0000 7f7e6f400640 1 -- 172.21.15.27:0/463375964 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e780b6160 con 0x5573b071c2f0 2024-09-21T00:35:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.627+0000 7f7e6f400640 1 -- 172.21.15.27:0/463375964 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e780a8a60 con 0x5573b071c2f0 2024-09-21T00:35:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.627+0000 7f7e6f400640 1 -- 172.21.15.27:0/463375964 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e780ab4c0 con 0x5573b071c2f0 2024-09-21T00:35:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.628+0000 7f7e8cc09740 1 -- 172.21.15.27:0/463375964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b071c2f0 msgr2=0x5573b0af4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.628+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/463375964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b071c2f0 0x5573b0af4260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7e780a80d0 tx=0x7f7e78004a40 comp rx=0 tx=0).stop 2024-09-21T00:35:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.629+0000 7f7e8cc09740 1 -- 172.21.15.27:0/463375964 shutdown_connections 2024-09-21T00:35:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.629+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/463375964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b07451b0 0x5573b0aeb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.629+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/463375964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b07244d0 0x5573b0af2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.629+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/463375964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b071c2f0 0x5573b0af4260 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.629+0000 7f7e8cc09740 1 -- 172.21.15.27:0/463375964 >> 172.21.15.27:0/463375964 conn(0x5573b0739300 msgr2=0x5573b0aeae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.630+0000 7f7e8cc09740 1 -- 172.21.15.27:0/463375964 shutdown_connections 2024-09-21T00:35:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.630+0000 7f7e8cc09740 1 -- 172.21.15.27:0/463375964 wait complete. 2024-09-21T00:35:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.630+0000 7f7e8cc09740 1 -- start start 2024-09-21T00:35:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.631+0000 7f7e8cc09740 1 Processor -- start 2024-09-21T00:35:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.631+0000 7f7e8cc09740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b071c2f0 0x5573b0af3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.631+0000 7f7e8cc09740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b07244d0 0x5573b0aef530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.631+0000 7f7e8cc09740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b07451b0 0x5573b0aefc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.631+0000 7f7e8cc09740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c00e860 con 0x5573b07451b0 2024-09-21T00:35:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.631+0000 7f7e8cc09740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c00fcd0 con 0x5573b071c2f0 2024-09-21T00:35:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.631+0000 7f7e8cc09740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c00ef30 con 0x5573b07244d0 2024-09-21T00:35:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.631+0000 7f7e8cc09740 10 client.0 _ll_register_callbacks cb 0x7f7e70009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:35:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.632+0000 7f7e7ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b071c2f0 0x5573b0af3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.633+0000 7f7e7e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b07244d0 0x5573b0aef530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.633+0000 7f7e7e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b07244d0 0x5573b0aef530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:43338/0 (socket says 172.21.15.27:43338) 2024-09-21T00:35:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.633+0000 7f7e7e000640 1 -- 172.21.15.27:0/3908525040 learned_addr learned my addr 172.21.15.27:0/3908525040 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.633+0000 7f7e7f400640 1 --2- 172.21.15.27:0/3908525040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b07451b0 0x5573b0aefc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.634+0000 7f7e7e000640 1 -- 172.21.15.27:0/3908525040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b071c2f0 msgr2=0x5573b0af3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.634+0000 7f7e7e000640 1 --2- 172.21.15.27:0/3908525040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b071c2f0 0x5573b0af3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.634+0000 7f7e7e000640 1 -- 172.21.15.27:0/3908525040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b07451b0 msgr2=0x5573b0aefc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.634+0000 7f7e7e000640 1 --2- 172.21.15.27:0/3908525040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b07451b0 0x5573b0aefc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.634+0000 7f7e7e000640 1 -- 172.21.15.27:0/3908525040 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b0b702d0 con 0x5573b07244d0 2024-09-21T00:35:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.638+0000 7f7e7e000640 1 --2- 172.21.15.27:0/3908525040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b07244d0 0x5573b0aef530 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7e68033000 tx=0x7f7e6802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb82d2d1a5b5343c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.638+0000 7f7e6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:35:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.640+0000 7f7e6cc00640 1 -- 172.21.15.27:0/3908525040 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68035040 con 0x5573b07244d0 2024-09-21T00:35:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.640+0000 7f7e6cc00640 1 -- 172.21.15.27:0/3908525040 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7f7e68005510 con 0x5573b07244d0 2024-09-21T00:35:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.640+0000 7f7e8cc09740 1 -- 172.21.15.27:0/3908525040 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e780a8a60 con 0x5573b07244d0 2024-09-21T00:35:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.641+0000 7f7e6cc00640 1 -- 172.21.15.27:0/3908525040 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f7e68004770 con 0x5573b07244d0 2024-09-21T00:35:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.641+0000 7f7e8cc09740 10 client.26710 fetch_fsmap learned FSMap version 47 2024-09-21T00:35:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.641+0000 7f7e8cc09740 1 -- 172.21.15.27:0/3908525040 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5573b0b713d0 con 0x5573b07244d0 2024-09-21T00:35:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.641+0000 7f7e6cc00640 1 -- 172.21.15.27:0/3908525040 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e780a8a60 con 0x5573b07244d0 2024-09-21T00:35:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.642+0000 7f7e6cc00640 1 -- 172.21.15.27:0/3908525040 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f7e68003bc0 con 0x5573b07244d0 2024-09-21T00:35:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.642+0000 7f7e8cc09740 10 client.26710 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:35:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.642+0000 7f7e8cc09740 10 client.26710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:35:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.642+0000 7f7e8cc09740 20 client.26710 populate_metadata read hostname 'smithi027' 2024-09-21T00:35:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.642+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/3908525040 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5573b06d91c0 0x5573b0b73bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.642+0000 7f7e8cc09740 4 client.26710 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:35:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.642+0000 7f7e8cc09740 1 -- 172.21.15.27:0/3908525040 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5573b0a5d0b0 con 0x5573b06d91c0 2024-09-21T00:35:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.643+0000 7f7e7ea00640 1 --2- 172.21.15.27:0/3908525040 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5573b06d91c0 0x5573b0b73bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.644+0000 7f7e6cc00640 10 client.26710 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:20.644+0000 7f7e7ea00640 1 --2- 172.21.15.27:0/3908525040 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5573b06d91c0 0x5573b0b73bd0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c9aa7d48897f892 server_cookie=12ed39205cce79f6 in_seq=0 out_seq=0 2024-09-21T00:35:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.463+0000 7f7e6cc00640 1 -- 172.21.15.27:0/3908525040 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5573b0a5d0b0 con 0x5573b06d91c0 2024-09-21T00:35:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.463+0000 7f7e6cc00640 10 client.26710 handle_command_reply: tid=0 2024-09-21T00:35:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.463+0000 7f7e8cc09740 1 client.26710 shutdown 2024-09-21T00:35:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.464+0000 7f7e8cc09740 1 -- 172.21.15.27:0/3908525040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b07244d0 msgr2=0x5573b0aef530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.464+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/3908525040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b07244d0 0x5573b0aef530 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7e68033000 tx=0x7f7e6802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:35:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.464+0000 7f7e8cc09740 1 -- 172.21.15.27:0/3908525040 shutdown_connections 2024-09-21T00:35:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.464+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/3908525040 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x5573b06d91c0 0x5573b0b73bd0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.464+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/3908525040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573b07451b0 0x5573b0aefc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.464+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/3908525040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5573b07244d0 0x5573b0aef530 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.464+0000 7f7e8cc09740 1 --2- 172.21.15.27:0/3908525040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573b071c2f0 0x5573b0af3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.464+0000 7f7e8cc09740 1 -- 172.21.15.27:0/3908525040 >> 172.21.15.27:0/3908525040 conn(0x5573b0739300 msgr2=0x5573b0ae7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.464+0000 7f7e6cc00640 0 client.26710 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.465+0000 7f7e8cc09740 1 -- 172.21.15.27:0/3908525040 shutdown_connections 2024-09-21T00:35:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.465+0000 7f7e8cc09740 1 -- 172.21.15.27:0/3908525040 wait complete. 2024-09-21T00:35:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.465+0000 7f7e8cc09740 20 client.26710 trim_cache size 0 max 16384 2024-09-21T00:35:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:35:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:35:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:35:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:35:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.467+0000 7f7e7fe00640 1 -- 172.21.15.27:0/3888757392 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7e640b6600 msgr2=0x7f7e640b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.467+0000 7f7e7fe00640 1 --2- 172.21.15.27:0/3888757392 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7e640b6600 0x7f7e640b8b10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009230 tx=0x7f7e68000bc0 comp rx=0 tx=0).stop 2024-09-21T00:35:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.467+0000 7f7e7fe00640 1 -- 172.21.15.27:0/3888757392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e8010e310 msgr2=0x7f7e8011c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.467+0000 7f7e7fe00640 1 --2- 172.21.15.27:0/3888757392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e8010e310 0x7f7e8011c7e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800ba50 tx=0x7f7e7800bf10 comp rx=0 tx=0).stop 2024-09-21T00:35:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.467+0000 7f7e7fe00640 1 -- 172.21.15.27:0/3888757392 shutdown_connections 2024-09-21T00:35:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.467+0000 7f7e7fe00640 1 --2- 172.21.15.27:0/3888757392 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f7e640b6600 0x7f7e640b8b10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.467+0000 7f7e7fe00640 1 --2- 172.21.15.27:0/3888757392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e80114590 0x7f7e80123660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.467+0000 7f7e7fe00640 1 --2- 172.21.15.27:0/3888757392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e80113c40 0x7f7e8011cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.467+0000 7f7e7fe00640 1 --2- 172.21.15.27:0/3888757392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e8010e310 0x7f7e8011c7e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.467+0000 7f7e7fe00640 1 -- 172.21.15.27:0/3888757392 >> 172.21.15.27:0/3888757392 conn(0x7f7e8006e610 msgr2=0x7f7e8010a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.468+0000 7f7e7fe00640 1 -- 172.21.15.27:0/3888757392 shutdown_connections 2024-09-21T00:35:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:21.468+0000 7f7e7fe00640 1 -- 172.21.15.27:0/3888757392 wait complete. 2024-09-21T00:35:22.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:21 smithi089 ceph-mon[34481]: pgmap v816: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 397 op/s 2024-09-21T00:35:22.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:21 smithi098 ceph-mon[34725]: pgmap v816: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 397 op/s 2024-09-21T00:35:22.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:21 smithi027 ceph-mon[30745]: pgmap v816: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 397 op/s 2024-09-21T00:35:24.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:23 smithi089 ceph-mon[34481]: pgmap v817: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 390 op/s 2024-09-21T00:35:24.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:23 smithi098 ceph-mon[34725]: pgmap v817: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 390 op/s 2024-09-21T00:35:24.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:23 smithi027 ceph-mon[30745]: pgmap v817: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 390 op/s 2024-09-21T00:35:26.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:25 smithi089 ceph-mon[34481]: pgmap v818: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 363 op/s 2024-09-21T00:35:26.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:25 smithi098 ceph-mon[34725]: pgmap v818: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 363 op/s 2024-09-21T00:35:26.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:25 smithi027 ceph-mon[30745]: pgmap v818: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 363 op/s 2024-09-21T00:35:26.317 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:35:25 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:35:25.935+0000 7fbf41e00640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-21T00:35:26.317 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:35:25 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:35:25.935+0000 7fbf41e00640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-21T00:35:26.828 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-09-21T00:35:26.829 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T00:35:28.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:27 smithi089 ceph-mon[34481]: pgmap v819: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.1 MiB/s wr, 367 op/s 2024-09-21T00:35:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:27 smithi098 ceph-mon[34725]: pgmap v819: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.1 MiB/s wr, 367 op/s 2024-09-21T00:35:28.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:27 smithi027 ceph-mon[30745]: pgmap v819: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.1 MiB/s wr, 367 op/s 2024-09-21T00:35:29.580 DEBUG:teuthology.parallel:result is None 2024-09-21T00:35:29.581 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T00:35:29.707 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T00:35:29.707 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-21T00:35:29.716 INFO:tasks.quiescer:joining Quiescers 2024-09-21T00:35:29.716 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-21T00:35:29.717 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:35:29.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:29 smithi027 ceph-mon[30745]: pgmap v820: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-09-21T00:35:30.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:29 smithi089 ceph-mon[34481]: pgmap v820: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-09-21T00:35:30.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:29 smithi098 ceph-mon[34725]: pgmap v820: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-09-21T00:35:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:35:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.473+0000 7f0796c00640 1 -- 172.21.15.27:0/2655554008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0790109c20 msgr2=0x7f0790110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.473+0000 7f0796c00640 1 --2- 172.21.15.27:0/2655554008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0790109c20 0x7f0790110450 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0788009ca0 tx=0x7f078802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:35:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.480+0000 7f0796c00640 1 -- 172.21.15.27:0/2655554008 shutdown_connections 2024-09-21T00:35:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.480+0000 7f0796c00640 1 --2- 172.21.15.27:0/2655554008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0790109c20 0x7f0790110450 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.480+0000 7f0796c00640 1 --2- 172.21.15.27:0/2655554008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07901092d0 0x7f07901096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.480+0000 7f0796c00640 1 --2- 172.21.15.27:0/2655554008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0790103990 0x7f0790103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.480+0000 7f0796c00640 1 -- 172.21.15.27:0/2655554008 >> 172.21.15.27:0/2655554008 conn(0x7f07900ff3e0 msgr2=0x7f0790101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.480+0000 7f0796c00640 1 -- 172.21.15.27:0/2655554008 shutdown_connections 2024-09-21T00:35:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.480+0000 7f0796c00640 1 -- 172.21.15.27:0/2655554008 wait complete. 2024-09-21T00:35:31.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.485+0000 7f0796c00640 1 Processor -- start 2024-09-21T00:35:31.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.485+0000 7f0796c00640 1 -- start start 2024-09-21T00:35:31.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.486+0000 7f0796c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0790103990 0x7f079010d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:31.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.486+0000 7f0796c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07901092d0 0x7f079010de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:31.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.486+0000 7f0796c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0790109c20 0x7f079010e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:31.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.486+0000 7f0796c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0790115b30 con 0x7f0790103990 2024-09-21T00:35:31.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.486+0000 7f0796c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f07901159b0 con 0x7f07901092d0 2024-09-21T00:35:31.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.486+0000 7f0796c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0790115cb0 con 0x7f0790109c20 2024-09-21T00:35:31.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.486+0000 7f0796200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0790109c20 0x7f079010e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:31.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.486+0000 7f0796200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0790109c20 0x7f079010e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:47920/0 (socket says 172.21.15.27:47920) 2024-09-21T00:35:31.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.486+0000 7f0796200640 1 -- 172.21.15.27:0/1142277896 learned_addr learned my addr 172.21.15.27:0/1142277896 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:31.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.486+0000 7f0795800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0790103990 0x7f079010d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.486+0000 7f0794e00640 1 --2- 172.21.15.27:0/1142277896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07901092d0 0x7f079010de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.487+0000 7f0796200640 1 -- 172.21.15.27:0/1142277896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07901092d0 msgr2=0x7f079010de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.487+0000 7f0796200640 1 --2- 172.21.15.27:0/1142277896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07901092d0 0x7f079010de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.487+0000 7f0796200640 1 -- 172.21.15.27:0/1142277896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0790103990 msgr2=0x7f079010d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.487+0000 7f0796200640 1 --2- 172.21.15.27:0/1142277896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0790103990 0x7f079010d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.487+0000 7f0796200640 1 -- 172.21.15.27:0/1142277896 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07901ac9b0 con 0x7f0790109c20 2024-09-21T00:35:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.487+0000 7f0795800640 1 --2- 172.21.15.27:0/1142277896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0790103990 0x7f079010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:35:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.487+0000 7f0794e00640 1 --2- 172.21.15.27:0/1142277896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07901092d0 0x7f079010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:35:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.488+0000 7f0796200640 1 --2- 172.21.15.27:0/1142277896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0790109c20 0x7f079010e350 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0788009430 tx=0x7f07880317c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.488+0000 7f078e000640 1 -- 172.21.15.27:0/1142277896 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0788003b70 con 0x7f0790109c20 2024-09-21T00:35:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.488+0000 7f078e000640 1 -- 172.21.15.27:0/1142277896 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0788003d10 con 0x7f0790109c20 2024-09-21T00:35:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.488+0000 7f078e000640 1 -- 172.21.15.27:0/1142277896 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078802fa30 con 0x7f0790109c20 2024-09-21T00:35:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.489+0000 7f0796c00640 1 -- 172.21.15.27:0/1142277896 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07901acc40 con 0x7f0790109c20 2024-09-21T00:35:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.489+0000 7f0796c00640 1 -- 172.21.15.27:0/1142277896 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07901ad130 con 0x7f0790109c20 2024-09-21T00:35:31.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.492+0000 7f0796c00640 1 -- 172.21.15.27:0/1142277896 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07700052c0 con 0x7f0790109c20 2024-09-21T00:35:31.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.499+0000 7f078e000640 1 -- 172.21.15.27:0/1142277896 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f078802fbd0 con 0x7f0790109c20 2024-09-21T00:35:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.504+0000 7f078e000640 1 --2- 172.21.15.27:0/1142277896 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f07780b6780 0x7f07780b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.505+0000 7f078e000640 1 -- 172.21.15.27:0/1142277896 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7f0788108f10 con 0x7f0790109c20 2024-09-21T00:35:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.507+0000 7f0795800640 1 --2- 172.21.15.27:0/1142277896 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f07780b6780 0x7f07780b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.508+0000 7f0795800640 1 --2- 172.21.15.27:0/1142277896 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f07780b6780 0x7f07780b8c90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0790110330 tx=0x7f078400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:31.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.512+0000 7f078e000640 1 -- 172.21.15.27:0/1142277896 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f07880cef80 con 0x7f0790109c20 2024-09-21T00:35:31.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.979+0000 7f0796c00640 1 -- 172.21.15.27:0/1142277896 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0770005880 con 0x7f0790109c20 2024-09-21T00:35:31.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.980+0000 7f078e000640 1 -- 172.21.15.27:0/1142277896 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11902 (secure 0 0 0) 0x7f07880d3360 con 0x7f0790109c20 2024-09-21T00:35:31.981 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:35:31.982 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":47,"btime":"2024-09-21T00:33:03:420304+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:33:02.469717+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26548":{"gid":26548,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2873147127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2873147127},{"type":"v1","addr":"172.21.15.89:6837","nonce":2873147127}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24751,24727]},"id":1}]} 2024-09-21T00:35:31.982 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 47 2024-09-21T00:35:31.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.984+0000 7f0796c00640 1 -- 172.21.15.27:0/1142277896 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f07780b6780 msgr2=0x7f07780b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:31.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.984+0000 7f0796c00640 1 --2- 172.21.15.27:0/1142277896 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f07780b6780 0x7f07780b8c90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0790110330 tx=0x7f078400b390 comp rx=0 tx=0).stop 2024-09-21T00:35:31.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.984+0000 7f0796c00640 1 -- 172.21.15.27:0/1142277896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0790109c20 msgr2=0x7f079010e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:31.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.984+0000 7f0796c00640 1 --2- 172.21.15.27:0/1142277896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0790109c20 0x7f079010e350 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0788009430 tx=0x7f07880317c0 comp rx=0 tx=0).stop 2024-09-21T00:35:31.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.984+0000 7f0796c00640 1 -- 172.21.15.27:0/1142277896 shutdown_connections 2024-09-21T00:35:31.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.984+0000 7f0796c00640 1 --2- 172.21.15.27:0/1142277896 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f07780b6780 0x7f07780b8c90 secure :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0790110330 tx=0x7f078400b390 comp rx=0 tx=0).stop 2024-09-21T00:35:31.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.984+0000 7f0796c00640 1 --2- 172.21.15.27:0/1142277896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0790109c20 0x7f079010e350 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:31.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.984+0000 7f0796c00640 1 --2- 172.21.15.27:0/1142277896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07901092d0 0x7f079010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:31.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.984+0000 7f0796c00640 1 --2- 172.21.15.27:0/1142277896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0790103990 0x7f079010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:31.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.984+0000 7f0796c00640 1 -- 172.21.15.27:0/1142277896 >> 172.21.15.27:0/1142277896 conn(0x7f07900ff3e0 msgr2=0x7f0790101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:31.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.986+0000 7f0796c00640 1 -- 172.21.15.27:0/1142277896 shutdown_connections 2024-09-21T00:35:31.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:31.986+0000 7f0796c00640 1 -- 172.21.15.27:0/1142277896 wait complete. 2024-09-21T00:35:32.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:31 smithi089 ceph-mon[34481]: pgmap v821: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 974 KiB/s wr, 321 op/s 2024-09-21T00:35:32.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:31 smithi098 ceph-mon[34725]: pgmap v821: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 974 KiB/s wr, 321 op/s 2024-09-21T00:35:32.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:31 smithi027 ceph-mon[30745]: pgmap v821: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 974 KiB/s wr, 321 op/s 2024-09-21T00:35:33.035 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --cancel --all' 2024-09-21T00:35:33.036 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --cancel --all 2024-09-21T00:35:33.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:32 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1142277896' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:35:33.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:32 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1142277896' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:35:33.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:32 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1142277896' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:35:33.766 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:35:34.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:33 smithi027 ceph-mon[30745]: pgmap v822: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 977 KiB/s wr, 324 op/s 2024-09-21T00:35:34.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:33 smithi089 ceph-mon[34481]: pgmap v822: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 977 KiB/s wr, 324 op/s 2024-09-21T00:35:34.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:33 smithi098 ceph-mon[34725]: pgmap v822: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 977 KiB/s wr, 324 op/s 2024-09-21T00:35:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.802+0000 7f2c09400640 1 -- 172.21.15.27:0/3046666049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bfc0a6a30 msgr2=0x7f2bfc0a8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.802+0000 7f2c09400640 1 --2- 172.21.15.27:0/3046666049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bfc0a6a30 0x7f2bfc0a8f00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2c04066480 tx=0x7f2c04098110 comp rx=0 tx=0).stop 2024-09-21T00:35:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.803+0000 7f2c09400640 1 -- 172.21.15.27:0/3046666049 shutdown_connections 2024-09-21T00:35:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.803+0000 7f2c09400640 1 --2- 172.21.15.27:0/3046666049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bfc0a6a30 0x7f2bfc0a8f00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.803+0000 7f2c09400640 1 --2- 172.21.15.27:0/3046666049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bfc0073a0 0x7f2bfc0077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.803+0000 7f2c09400640 1 --2- 172.21.15.27:0/3046666049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc008960 0x7f2bfc006d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.803+0000 7f2c09400640 1 -- 172.21.15.27:0/3046666049 >> 172.21.15.27:0/3046666049 conn(0x7f2bfc091b70 msgr2=0x7f2bfc093fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.804+0000 7f2c09400640 1 -- 172.21.15.27:0/3046666049 shutdown_connections 2024-09-21T00:35:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.804+0000 7f2c09400640 1 -- 172.21.15.27:0/3046666049 wait complete. 2024-09-21T00:35:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.805+0000 7f2c09400640 1 Processor -- start 2024-09-21T00:35:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.806+0000 7f2c09400640 1 -- start start 2024-09-21T00:35:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.806+0000 7f2c09400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bfc0073a0 0x7f2bfc144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.809+0000 7f2c09400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bfc008960 0x7f2bfc1445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.809+0000 7f2c09400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc0a6a30 0x7f2bfc149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.809+0000 7f2c09400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc0bd540 con 0x7f2bfc008960 2024-09-21T00:35:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.809+0000 7f2c09400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc0bd3c0 con 0x7f2bfc0a6a30 2024-09-21T00:35:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.809+0000 7f2c09400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc0bd6c0 con 0x7f2bfc0073a0 2024-09-21T00:35:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.809+0000 7f2c03e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bfc0073a0 0x7f2bfc144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.809+0000 7f2c03e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bfc0073a0 0x7f2bfc144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55834/0 (socket says 172.21.15.27:55834) 2024-09-21T00:35:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.809+0000 7f2c03e00640 1 -- 172.21.15.27:0/2665529842 learned_addr learned my addr 172.21.15.27:0/2665529842 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.810+0000 7f2c03400640 1 --2- 172.21.15.27:0/2665529842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bfc008960 0x7f2bfc1445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.810+0000 7f2c08a00640 1 --2- 172.21.15.27:0/2665529842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc0a6a30 0x7f2bfc149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.810+0000 7f2c03e00640 1 -- 172.21.15.27:0/2665529842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc0a6a30 msgr2=0x7f2bfc149830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.810+0000 7f2c03e00640 1 --2- 172.21.15.27:0/2665529842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc0a6a30 0x7f2bfc149830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.810+0000 7f2c03e00640 1 -- 172.21.15.27:0/2665529842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bfc008960 msgr2=0x7f2bfc1445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.810+0000 7f2c03e00640 1 --2- 172.21.15.27:0/2665529842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bfc008960 0x7f2bfc1445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.810+0000 7f2c03e00640 1 -- 172.21.15.27:0/2665529842 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bfc149f30 con 0x7f2bfc0073a0 2024-09-21T00:35:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.811+0000 7f2c03400640 1 --2- 172.21.15.27:0/2665529842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bfc008960 0x7f2bfc1445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:35:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.811+0000 7f2c03e00640 1 --2- 172.21.15.27:0/2665529842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bfc0073a0 0x7f2bfc144000 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800c890 tx=0x7f2bf800cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.812+0000 7f2c00c00640 1 -- 172.21.15.27:0/2665529842 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8004220 con 0x7f2bfc0073a0 2024-09-21T00:35:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.813+0000 7f2c00c00640 1 -- 172.21.15.27:0/2665529842 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bf80043c0 con 0x7f2bfc0073a0 2024-09-21T00:35:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.813+0000 7f2c00c00640 1 -- 172.21.15.27:0/2665529842 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8004a00 con 0x7f2bfc0073a0 2024-09-21T00:35:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.813+0000 7f2c09400640 1 -- 172.21.15.27:0/2665529842 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bfc14a220 con 0x7f2bfc0073a0 2024-09-21T00:35:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.813+0000 7f2c09400640 1 -- 172.21.15.27:0/2665529842 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bfc1509b0 con 0x7f2bfc0073a0 2024-09-21T00:35:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.819+0000 7f2c11bbb740 1 Processor -- start 2024-09-21T00:35:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.820+0000 7f2c11bbb740 1 -- start start 2024-09-21T00:35:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.821+0000 7f2c11bbb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd82c1b0 0x55c8ddbd0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.821+0000 7f2c11bbb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd8032f0 0x55c8ddbd1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.821+0000 7f2c11bbb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbd1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.822+0000 7f2c11bbb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c8ddb466b0 con 0x55c8dd82c1b0 2024-09-21T00:35:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.822+0000 7f2c11bbb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c8ddb35ae0 con 0x55c8dd80b4d0 2024-09-21T00:35:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.822+0000 7f2c11bbb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c8ddb398f0 con 0x55c8dd8032f0 2024-09-21T00:35:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.823+0000 7f2c08a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd82c1b0 0x55c8ddbd0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.823+0000 7f2c08a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd82c1b0 0x55c8ddbd0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57566/0 (socket says 172.21.15.27:57566) 2024-09-21T00:35:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.823+0000 7f2c08a00640 1 -- 172.21.15.27:0/3103337428 learned_addr learned my addr 172.21.15.27:0/3103337428 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.823+0000 7f2c03400640 1 --2- 172.21.15.27:0/3103337428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbd1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.824+0000 7f2c00c00640 1 -- 172.21.15.27:0/2665529842 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2bf8004ba0 con 0x7f2bfc0073a0 2024-09-21T00:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.827+0000 7f2c08a00640 1 -- 172.21.15.27:0/3103337428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd8032f0 msgr2=0x55c8ddbd1350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T00:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.827+0000 7f2c08a00640 1 --2- 172.21.15.27:0/3103337428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd8032f0 0x55c8ddbd1350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.827+0000 7f2c08a00640 1 -- 172.21.15.27:0/3103337428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 msgr2=0x55c8ddbd1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.827+0000 7f2c08a00640 1 --2- 172.21.15.27:0/3103337428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbd1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.827+0000 7f2c08a00640 1 -- 172.21.15.27:0/3103337428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8ddb02780 con 0x55c8dd82c1b0 2024-09-21T00:35:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.829+0000 7f2c00c00640 1 --2- 172.21.15.27:0/2665529842 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2bec0b6780 0x7f2bec0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.830+0000 7f2c00c00640 1 -- 172.21.15.27:0/2665529842 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7f2bf80e78f0 con 0x7f2bfc0073a0 2024-09-21T00:35:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.831+0000 7f2c08a00640 1 --2- 172.21.15.27:0/3103337428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd82c1b0 0x55c8ddbd0e10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2c04051d10 tx=0x7f2c04098a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5923915177a61817 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.831+0000 7f2c03e00640 1 --2- 172.21.15.27:0/2665529842 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2bec0b6780 0x7f2bec0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.832+0000 7f2c03e00640 1 --2- 172.21.15.27:0/2665529842 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2bec0b6780 0x7f2bec0b8c90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55c8ddbcfa90 tx=0x7f2bf800a520 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.833+0000 7f2bf5200640 1 -- 172.21.15.27:0/3103337428 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04075380 con 0x55c8dd82c1b0 2024-09-21T00:35:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.834+0000 7f2bf5200640 1 -- 172.21.15.27:0/3103337428 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c0409c040 con 0x55c8dd82c1b0 2024-09-21T00:35:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.834+0000 7f2bf5200640 1 -- 172.21.15.27:0/3103337428 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04064500 con 0x55c8dd82c1b0 2024-09-21T00:35:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.835+0000 7f2c11bbb740 1 -- 172.21.15.27:0/3103337428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd82c1b0 msgr2=0x55c8ddbd0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.835+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/3103337428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd82c1b0 0x55c8ddbd0e10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2c04051d10 tx=0x7f2c04098a10 comp rx=0 tx=0).stop 2024-09-21T00:35:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.837+0000 7f2c11bbb740 1 -- 172.21.15.27:0/3103337428 shutdown_connections 2024-09-21T00:35:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.838+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/3103337428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbd1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.838+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/3103337428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd8032f0 0x55c8ddbd1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.838+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/3103337428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd82c1b0 0x55c8ddbd0e10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.838+0000 7f2c11bbb740 1 -- 172.21.15.27:0/3103337428 >> 172.21.15.27:0/3103337428 conn(0x55c8dd820300 msgr2=0x55c8ddbced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.839+0000 7f2c11bbb740 1 -- 172.21.15.27:0/3103337428 shutdown_connections 2024-09-21T00:35:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.839+0000 7f2c11bbb740 1 -- 172.21.15.27:0/3103337428 wait complete. 2024-09-21T00:35:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.842+0000 7f2c11bbb740 1 -- start start 2024-09-21T00:35:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.842+0000 7f2c11bbb740 1 Processor -- start 2024-09-21T00:35:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.844+0000 7f2c11bbb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd8032f0 0x55c8ddbdc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.844+0000 7f2c11bbb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbdc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.844+0000 7f2c11bbb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd82c1b0 0x55c8ddbdce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.844+0000 7f2c11bbb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c8dda6dab0 con 0x55c8dd8032f0 2024-09-21T00:35:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.844+0000 7f2c11bbb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c8ddadc570 con 0x55c8dd80b4d0 2024-09-21T00:35:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.844+0000 7f2c11bbb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c8ddbdf530 con 0x55c8dd82c1b0 2024-09-21T00:35:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.844+0000 7f2c11bbb740 10 client.0 _ll_register_callbacks cb 0x55c8ddaeb100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:35:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.845+0000 7f2c08a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd8032f0 0x55c8ddbdc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.845+0000 7f2c08a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd8032f0 0x55c8ddbdc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57576/0 (socket says 172.21.15.27:57576) 2024-09-21T00:35:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.845+0000 7f2c08a00640 1 -- 172.21.15.27:0/24070526 learned_addr learned my addr 172.21.15.27:0/24070526 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.845+0000 7f2c08a00640 1 --2- 172.21.15.27:0/24070526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd82c1b0 0x55c8ddbdce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.846+0000 7f2c08a00640 1 -- 172.21.15.27:0/24070526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 msgr2=0x55c8ddbdc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.846+0000 7f2c08a00640 1 --2- 172.21.15.27:0/24070526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbdc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.846+0000 7f2c08a00640 1 -- 172.21.15.27:0/24070526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd8032f0 msgr2=0x55c8ddbdc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.846+0000 7f2c08a00640 1 --2- 172.21.15.27:0/24070526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd8032f0 0x55c8ddbdc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.846+0000 7f2c08a00640 1 -- 172.21.15.27:0/24070526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8dd897790 con 0x55c8dd82c1b0 2024-09-21T00:35:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.849+0000 7f2c08a00640 1 --2- 172.21.15.27:0/24070526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd82c1b0 0x55c8ddbdce40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2c04073040 tx=0x7f2c04069790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6443791f82a7df1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.849+0000 7f2be9600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:35:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.851+0000 7f2be9600640 1 -- 172.21.15.27:0/24070526 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04063c10 con 0x55c8dd82c1b0 2024-09-21T00:35:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.851+0000 7f2c11bbb740 1 -- 172.21.15.27:0/24070526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c0409c040 con 0x55c8dd82c1b0 2024-09-21T00:35:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.851+0000 7f2be9600640 1 -- 172.21.15.27:0/24070526 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7f2c04063db0 con 0x55c8dd82c1b0 2024-09-21T00:35:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.853+0000 7f2be9600640 1 -- 172.21.15.27:0/24070526 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2c04063fb0 con 0x55c8dd82c1b0 2024-09-21T00:35:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.854+0000 7f2c11bbb740 10 client.26740 fetch_fsmap learned FSMap version 47 2024-09-21T00:35:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.855+0000 7f2c11bbb740 1 -- 172.21.15.27:0/24070526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c8dd910620 con 0x55c8dd82c1b0 2024-09-21T00:35:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.855+0000 7f2be9600640 1 -- 172.21.15.27:0/24070526 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c0409c040 con 0x55c8dd82c1b0 2024-09-21T00:35:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.856+0000 7f2be9600640 1 -- 172.21.15.27:0/24070526 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f2c040aa4d0 con 0x55c8dd82c1b0 2024-09-21T00:35:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.856+0000 7f2c11bbb740 10 client.26740 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:35:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.856+0000 7f2c11bbb740 10 client.26740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:35:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.856+0000 7f2c11bbb740 20 client.26740 populate_metadata read hostname 'smithi027' 2024-09-21T00:35:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.857+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/24070526 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c8dd7ead80 0x55c8ddbee330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.857+0000 7f2c11bbb740 4 client.26740 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:35:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.857+0000 7f2c11bbb740 1 -- 172.21.15.27:0/24070526 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8ddb48600 con 0x55c8dd7ead80 2024-09-21T00:35:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.857+0000 7f2c03400640 1 --2- 172.21.15.27:0/24070526 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c8dd7ead80 0x55c8ddbee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.858+0000 7f2c03400640 1 --2- 172.21.15.27:0/24070526 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c8dd7ead80 0x55c8ddbee330 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8b52ab7193dcf25 server_cookie=9ec15a57d52193d3 in_seq=0 out_seq=0 2024-09-21T00:35:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.858+0000 7f2be9600640 10 client.26740 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.859+0000 7f2be9600640 1 -- 172.21.15.27:0/24070526 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f2bf000b930 con 0x55c8dd7ead80 2024-09-21T00:35:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.859+0000 7f2be9600640 10 client.26740 handle_command_reply: tid=0 2024-09-21T00:35:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.859+0000 7f2c11bbb740 1 client.26740 shutdown 2024-09-21T00:35:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.864+0000 7f2c11bbb740 1 -- 172.21.15.27:0/24070526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd82c1b0 msgr2=0x55c8ddbdce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.864+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/24070526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd82c1b0 0x55c8ddbdce40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2c04073040 tx=0x7f2c04069790 comp rx=0 tx=0).stop 2024-09-21T00:35:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.864+0000 7f2c11bbb740 1 -- 172.21.15.27:0/24070526 shutdown_connections 2024-09-21T00:35:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.864+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/24070526 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c8dd7ead80 0x55c8ddbee330 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.864+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/24070526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd82c1b0 0x55c8ddbdce40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.864+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/24070526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbdc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.864+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/24070526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd8032f0 0x55c8ddbdc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.864+0000 7f2c11bbb740 1 -- 172.21.15.27:0/24070526 >> 172.21.15.27:0/24070526 conn(0x55c8dd820300 msgr2=0x55c8ddbd1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.865+0000 7f2be9600640 0 client.26740 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.865+0000 7f2c11bbb740 1 -- 172.21.15.27:0/24070526 shutdown_connections 2024-09-21T00:35:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.865+0000 7f2c11bbb740 1 -- 172.21.15.27:0/24070526 wait complete. 2024-09-21T00:35:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.865+0000 7f2c11bbb740 20 client.26740 trim_cache size 0 max 16384 2024-09-21T00:35:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.901+0000 7f2c11bbb740 1 Processor -- start 2024-09-21T00:35:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.901+0000 7f2c11bbb740 1 -- start start 2024-09-21T00:35:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.901+0000 7f2c11bbb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd8032f0 0x55c8ddbcdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.901+0000 7f2c11bbb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbddff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.901+0000 7f2c11bbb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd82c1b0 0x55c8ddbd9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.901+0000 7f2c11bbb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4012c00 con 0x55c8dd82c1b0 2024-09-21T00:35:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.901+0000 7f2c11bbb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4010410 con 0x55c8dd80b4d0 2024-09-21T00:35:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.901+0000 7f2c11bbb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2be400e200 con 0x55c8dd8032f0 2024-09-21T00:35:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.902+0000 7f2c03400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbddff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.902+0000 7f2c08a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd82c1b0 0x55c8ddbd9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.902+0000 7f2c03400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbddff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40252/0 (socket says 172.21.15.27:40252) 2024-09-21T00:35:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.902+0000 7f2c03400640 1 -- 172.21.15.27:0/2094138896 learned_addr learned my addr 172.21.15.27:0/2094138896 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.902+0000 7f2c08a00640 1 --2- 172.21.15.27:0/2094138896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd8032f0 0x55c8ddbcdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.903+0000 7f2c03400640 1 -- 172.21.15.27:0/2094138896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd8032f0 msgr2=0x55c8ddbcdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.903+0000 7f2c03400640 1 --2- 172.21.15.27:0/2094138896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd8032f0 0x55c8ddbcdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.903+0000 7f2c03400640 1 -- 172.21.15.27:0/2094138896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd82c1b0 msgr2=0x55c8ddbd9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.903+0000 7f2c03400640 1 --2- 172.21.15.27:0/2094138896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd82c1b0 0x55c8ddbd9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.903+0000 7f2c03400640 1 -- 172.21.15.27:0/2094138896 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8ddbcf080 con 0x55c8dd80b4d0 2024-09-21T00:35:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.903+0000 7f2c03400640 1 --2- 172.21.15.27:0/2094138896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbddff0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000bea0 tx=0x7f2bf000b3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68cddb31a2ad648e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.904+0000 7f2bebe00640 1 -- 172.21.15.27:0/2094138896 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0032480 con 0x55c8dd80b4d0 2024-09-21T00:35:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.904+0000 7f2bebe00640 1 -- 172.21.15.27:0/2094138896 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bf001c070 con 0x55c8dd80b4d0 2024-09-21T00:35:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.904+0000 7f2bebe00640 1 -- 172.21.15.27:0/2094138896 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00307a0 con 0x55c8dd80b4d0 2024-09-21T00:35:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.907+0000 7f2c11bbb740 1 -- 172.21.15.27:0/2094138896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 msgr2=0x55c8ddbddff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.907+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/2094138896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbddff0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000bea0 tx=0x7f2bf000b3f0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.908+0000 7f2c11bbb740 1 -- 172.21.15.27:0/2094138896 shutdown_connections 2024-09-21T00:35:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.908+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/2094138896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd82c1b0 0x55c8ddbd9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.908+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/2094138896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbddff0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.908+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/2094138896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd8032f0 0x55c8ddbcdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.908+0000 7f2c11bbb740 1 -- 172.21.15.27:0/2094138896 >> 172.21.15.27:0/2094138896 conn(0x55c8dd820300 msgr2=0x55c8ddbdd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.908+0000 7f2c11bbb740 1 -- 172.21.15.27:0/2094138896 shutdown_connections 2024-09-21T00:35:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.909+0000 7f2c11bbb740 1 -- 172.21.15.27:0/2094138896 wait complete. 2024-09-21T00:35:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.910+0000 7f2c11bbb740 1 -- start start 2024-09-21T00:35:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.910+0000 7f2c11bbb740 1 Processor -- start 2024-09-21T00:35:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.913+0000 7f2c11bbb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd8032f0 0x55c8ddbe2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.913+0000 7f2c11bbb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbe2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.913+0000 7f2c11bbb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd82c1b0 0x55c8ddc50cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.913+0000 7f2c11bbb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2be400e8d0 con 0x55c8dd80b4d0 2024-09-21T00:35:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.913+0000 7f2c11bbb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2be400fd40 con 0x55c8dd8032f0 2024-09-21T00:35:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.913+0000 7f2c11bbb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2be400efa0 con 0x55c8dd82c1b0 2024-09-21T00:35:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.914+0000 7f2c11bbb740 10 client.0 _ll_register_callbacks cb 0x7f2c04069400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:35:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.914+0000 7f2c08a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd8032f0 0x55c8ddbe2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.914+0000 7f2c03400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbe2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.914+0000 7f2c08a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd8032f0 0x55c8ddbe2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40262/0 (socket says 172.21.15.27:40262) 2024-09-21T00:35:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.914+0000 7f2c08a00640 1 -- 172.21.15.27:0/3698010953 learned_addr learned my addr 172.21.15.27:0/3698010953 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.915+0000 7f2c08a00640 1 --2- 172.21.15.27:0/3698010953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd82c1b0 0x55c8ddc50cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.915+0000 7f2c08a00640 1 -- 172.21.15.27:0/3698010953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd82c1b0 msgr2=0x55c8ddc50cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.915+0000 7f2c08a00640 1 --2- 172.21.15.27:0/3698010953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd82c1b0 0x55c8ddc50cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.915+0000 7f2c08a00640 1 -- 172.21.15.27:0/3698010953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd80b4d0 msgr2=0x55c8ddbe2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.915+0000 7f2c08a00640 1 --2- 172.21.15.27:0/3698010953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbe2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.915+0000 7f2c08a00640 1 -- 172.21.15.27:0/3698010953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8ddc51360 con 0x55c8dd8032f0 2024-09-21T00:35:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.918+0000 7f2be9600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T00:35:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.918+0000 7f2c08a00640 1 --2- 172.21.15.27:0/3698010953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd8032f0 0x55c8ddbe2380 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2c040524b0 tx=0x7f2c040ae040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74cded52a3f99167 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.918+0000 7f2be9600640 1 -- 172.21.15.27:0/3698010953 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04063fb0 con 0x55c8dd8032f0 2024-09-21T00:35:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.918+0000 7f2c11bbb740 1 -- 172.21.15.27:0/3698010953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bf001c070 con 0x55c8dd8032f0 2024-09-21T00:35:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.919+0000 7f2be9600640 1 -- 172.21.15.27:0/3698010953 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7f2c040bc070 con 0x55c8dd8032f0 2024-09-21T00:35:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.919+0000 7f2be9600640 1 -- 172.21.15.27:0/3698010953 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2c04063fb0 con 0x55c8dd8032f0 2024-09-21T00:35:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.919+0000 7f2c11bbb740 10 client.26717 fetch_fsmap learned FSMap version 47 2024-09-21T00:35:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.919+0000 7f2c11bbb740 1 -- 172.21.15.27:0/3698010953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c8ddc58750 con 0x55c8dd8032f0 2024-09-21T00:35:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.920+0000 7f2be9600640 1 -- 172.21.15.27:0/3698010953 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bf001c070 con 0x55c8dd8032f0 2024-09-21T00:35:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.920+0000 7f2be9600640 1 -- 172.21.15.27:0/3698010953 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f2c040a0410 con 0x55c8dd8032f0 2024-09-21T00:35:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.921+0000 7f2c11bbb740 10 client.26717 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:35:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.921+0000 7f2c11bbb740 10 client.26717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T00:35:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.921+0000 7f2c11bbb740 20 client.26717 populate_metadata read hostname 'smithi027' 2024-09-21T00:35:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.921+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/3698010953 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c8dd7c01c0 0x55c8ddc5adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.921+0000 7f2c11bbb740 4 client.26717 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T00:35:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.921+0000 7f2c11bbb740 1 -- 172.21.15.27:0/3698010953 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8ddb440b0 con 0x55c8dd7c01c0 2024-09-21T00:35:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.922+0000 7f2c03400640 1 --2- 172.21.15.27:0/3698010953 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c8dd7c01c0 0x55c8ddc5adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.924+0000 7f2c03400640 1 --2- 172.21.15.27:0/3698010953 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c8dd7c01c0 0x55c8ddc5adc0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a82d5765d348e6 server_cookie=59f864221d0bfc5c in_seq=0 out_seq=0 2024-09-21T00:35:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:34.924+0000 7f2be9600640 10 client.26717 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.091+0000 7f2be9600640 1 -- 172.21.15.27:0/3698010953 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8ddb440b0 con 0x55c8dd7c01c0 2024-09-21T00:35:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.091+0000 7f2be9600640 10 client.26717 handle_command_reply: tid=0 2024-09-21T00:35:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.091+0000 7f2c11bbb740 1 client.26717 shutdown 2024-09-21T00:35:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.091+0000 7f2c11bbb740 1 -- 172.21.15.27:0/3698010953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd8032f0 msgr2=0x55c8ddbe2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.091+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/3698010953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd8032f0 0x55c8ddbe2380 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2c040524b0 tx=0x7f2c040ae040 comp rx=0 tx=0).stop 2024-09-21T00:35:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.091+0000 7f2c11bbb740 1 -- 172.21.15.27:0/3698010953 shutdown_connections 2024-09-21T00:35:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.091+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/3698010953 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55c8dd7c01c0 0x55c8ddc5adc0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.091+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/3698010953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8dd82c1b0 0x55c8ddc50cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.091+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/3698010953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8dd80b4d0 0x55c8ddbe2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.091+0000 7f2c11bbb740 1 --2- 172.21.15.27:0/3698010953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8dd8032f0 0x55c8ddbe2380 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.091+0000 7f2c11bbb740 1 -- 172.21.15.27:0/3698010953 >> 172.21.15.27:0/3698010953 conn(0x55c8dd820300 msgr2=0x55c8ddbcdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.092+0000 7f2be9600640 0 client.26717 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.092+0000 7f2c11bbb740 1 -- 172.21.15.27:0/3698010953 shutdown_connections 2024-09-21T00:35:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.092+0000 7f2c11bbb740 1 -- 172.21.15.27:0/3698010953 wait complete. 2024-09-21T00:35:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.092+0000 7f2c11bbb740 20 client.26717 trim_cache size 0 max 16384 2024-09-21T00:35:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T00:35:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T00:35:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T00:35:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T00:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.093+0000 7f2c09400640 1 -- 172.21.15.27:0/2665529842 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2bec0b6780 msgr2=0x7f2bec0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.093+0000 7f2c09400640 1 --2- 172.21.15.27:0/2665529842 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2bec0b6780 0x7f2bec0b8c90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55c8ddbcfa90 tx=0x7f2bf800a520 comp rx=0 tx=0).stop 2024-09-21T00:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.093+0000 7f2c09400640 1 -- 172.21.15.27:0/2665529842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bfc0073a0 msgr2=0x7f2bfc144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.093+0000 7f2c09400640 1 --2- 172.21.15.27:0/2665529842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bfc0073a0 0x7f2bfc144000 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800c890 tx=0x7f2bf800cd50 comp rx=0 tx=0).stop 2024-09-21T00:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.094+0000 7f2c09400640 1 -- 172.21.15.27:0/2665529842 shutdown_connections 2024-09-21T00:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.094+0000 7f2c09400640 1 --2- 172.21.15.27:0/2665529842 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2bec0b6780 0x7f2bec0b8c90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.094+0000 7f2c09400640 1 --2- 172.21.15.27:0/2665529842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc0a6a30 0x7f2bfc149830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.094+0000 7f2c09400640 1 --2- 172.21.15.27:0/2665529842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bfc008960 0x7f2bfc1445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.094+0000 7f2c09400640 1 --2- 172.21.15.27:0/2665529842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bfc0073a0 0x7f2bfc144000 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.094+0000 7f2c09400640 1 -- 172.21.15.27:0/2665529842 >> 172.21.15.27:0/2665529842 conn(0x7f2bfc091b70 msgr2=0x7f2bfc0a2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.095+0000 7f2c09400640 1 -- 172.21.15.27:0/2665529842 shutdown_connections 2024-09-21T00:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T00:35:35.095+0000 7f2c09400640 1 -- 172.21.15.27:0/2665529842 wait complete. 2024-09-21T00:35:35.872 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:35:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 47, 2024-09-21T00:35:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24761, 2024-09-21T00:35:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 26, 2024-09-21T00:35:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": {} 2024-09-21T00:35:35.873 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:35:35.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.689+0000 7f349e000640 1 -- 172.21.15.27:0/4266499741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349810a650 msgr2=0x7f349810aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:35.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.689+0000 7f349e000640 1 --2- 172.21.15.27:0/4266499741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349810a650 0x7f349810aa60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f34880074e0 tx=0x7f348802b080 comp rx=0 tx=0).stop 2024-09-21T00:35:35.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.689+0000 7f349e000640 1 -- 172.21.15.27:0/4266499741 shutdown_connections 2024-09-21T00:35:35.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.689+0000 7f349e000640 1 --2- 172.21.15.27:0/4266499741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f349810d620 0x7f3498117090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.689+0000 7f349e000640 1 --2- 172.21.15.27:0/4266499741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f349810ccd0 0x7f349810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.689+0000 7f349e000640 1 --2- 172.21.15.27:0/4266499741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349810a650 0x7f349810aa60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.689+0000 7f349e000640 1 -- 172.21.15.27:0/4266499741 >> 172.21.15.27:0/4266499741 conn(0x7f3498070ec0 msgr2=0x7f34980712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:35.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.689+0000 7f349e000640 1 -- 172.21.15.27:0/4266499741 shutdown_connections 2024-09-21T00:35:35.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.689+0000 7f349e000640 1 -- 172.21.15.27:0/4266499741 wait complete. 2024-09-21T00:35:35.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.691+0000 7f349e000640 1 Processor -- start 2024-09-21T00:35:35.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.692+0000 7f349e000640 1 -- start start 2024-09-21T00:35:35.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.693+0000 7f349e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f349810a650 0x7f349811bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.693+0000 7f349e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349810ccd0 0x7f349811c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.693+0000 7f349e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f349810d620 0x7f34981268b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.693+0000 7f3497400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f349810a650 0x7f349811bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.693+0000 7f3496a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349810ccd0 0x7f349811c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.693+0000 7f3497400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f349810a650 0x7f349811bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40194/0 (socket says 172.21.15.27:40194) 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.693+0000 7f3496a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349810ccd0 0x7f349811c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57494/0 (socket says 172.21.15.27:57494) 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.693+0000 7f3497e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f349810d620 0x7f34981268b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.693+0000 7f3497e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f349810d620 0x7f34981268b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55792/0 (socket says 172.21.15.27:55792) 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.693+0000 7f3497400640 1 -- 172.21.15.27:0/2430247699 learned_addr learned my addr 172.21.15.27:0/2430247699 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.693+0000 7f349e000640 1 -- 172.21.15.27:0/2430247699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3498119e70 con 0x7f349810ccd0 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.693+0000 7f349e000640 1 -- 172.21.15.27:0/2430247699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3498119cf0 con 0x7f349810a650 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.694+0000 7f349e000640 1 -- 172.21.15.27:0/2430247699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3498119ff0 con 0x7f349810d620 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.698+0000 7f3497e00640 1 -- 172.21.15.27:0/2430247699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f349810a650 msgr2=0x7f349811bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.698+0000 7f3497e00640 1 --2- 172.21.15.27:0/2430247699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f349810a650 0x7f349811bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.698+0000 7f3497e00640 1 -- 172.21.15.27:0/2430247699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349810ccd0 msgr2=0x7f349811c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.698+0000 7f3497e00640 1 --2- 172.21.15.27:0/2430247699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349810ccd0 0x7f349811c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.698+0000 7f3497e00640 1 -- 172.21.15.27:0/2430247699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3498126fe0 con 0x7f349810d620 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.698+0000 7f3497400640 1 --2- 172.21.15.27:0/2430247699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f349810a650 0x7f349811bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:35:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.699+0000 7f3497e00640 1 --2- 172.21.15.27:0/2430247699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f349810d620 0x7f34981268b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f349000be20 tx=0x7f349000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.700+0000 7f348fe00640 1 -- 172.21.15.27:0/2430247699 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490019070 con 0x7f349810d620 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.700+0000 7f348fe00640 1 -- 172.21.15.27:0/2430247699 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34900148f0 con 0x7f349810d620 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.700+0000 7f348fe00640 1 -- 172.21.15.27:0/2430247699 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490012860 con 0x7f349810d620 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.700+0000 7f349e000640 1 -- 172.21.15.27:0/2430247699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3498121720 con 0x7f349810d620 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.700+0000 7f349e000640 1 -- 172.21.15.27:0/2430247699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3498121c10 con 0x7f349810d620 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.709+0000 7f34a4c29740 1 Processor -- start 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.713+0000 7f348fe00640 1 -- 172.21.15.27:0/2430247699 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f349001c070 con 0x7f349810d620 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.713+0000 7f34a4c29740 1 -- start start 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.715+0000 7f34a4c29740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ced32d0 0x55684d279290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.715+0000 7f34a4c29740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ceaa410 0x55684d2797d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.715+0000 7f34a4c29740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceb25f0 0x55684d279d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.716+0000 7f3497400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ceaa410 0x55684d2797d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.716+0000 7f3497400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ceaa410 0x55684d2797d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40196/0 (socket says 172.21.15.27:40196) 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.716+0000 7f34a4c29740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55684d1f0ff0 con 0x55684ced32d0 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.716+0000 7f34a4c29740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55684d1e0420 con 0x55684ceaa410 2024-09-21T00:35:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.716+0000 7f34a4c29740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55684d1e4230 con 0x55684ceb25f0 2024-09-21T00:35:35.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.716+0000 7f3496a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceb25f0 0x55684d279d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.716+0000 7f3496a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceb25f0 0x55684d279d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55804/0 (socket says 172.21.15.27:55804) 2024-09-21T00:35:35.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.717+0000 7f3496a00640 1 -- 172.21.15.27:0/3858012012 learned_addr learned my addr 172.21.15.27:0/3858012012 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:35.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.717+0000 7f348fe00640 1 --2- 172.21.15.27:0/2430247699 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f347c0b67d0 0x7f347c0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.717+0000 7f348fe00640 1 -- 172.21.15.27:0/2430247699 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7f34900e7320 con 0x7f349810d620 2024-09-21T00:35:35.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.718+0000 7f3497e00640 1 --2- 172.21.15.27:0/2430247699 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f347c0b67d0 0x7f347c0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.719+0000 7f3496a00640 1 --2- 172.21.15.27:0/3858012012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ced32d0 0x55684d279290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.720+0000 7f3496a00640 1 -- 172.21.15.27:0/3858012012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ceaa410 msgr2=0x55684d2797d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:35.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.720+0000 7f3496a00640 1 --2- 172.21.15.27:0/3858012012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ceaa410 0x55684d2797d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.720+0000 7f3496a00640 1 -- 172.21.15.27:0/3858012012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ced32d0 msgr2=0x55684d279290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:35.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.720+0000 7f3496a00640 1 --2- 172.21.15.27:0/3858012012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ced32d0 0x55684d279290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.720+0000 7f3496a00640 1 -- 172.21.15.27:0/3858012012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55684cf3e8b0 con 0x55684ceb25f0 2024-09-21T00:35:35.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.721+0000 7f3496a00640 1 --2- 172.21.15.27:0/3858012012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceb25f0 0x55684d279d10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f349806c970 tx=0x7f348400b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=150773a0e4567c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:35.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.721+0000 7f3483e00640 1 -- 172.21.15.27:0/3858012012 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484004220 con 0x55684ceb25f0 2024-09-21T00:35:35.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.721+0000 7f3497e00640 1 --2- 172.21.15.27:0/2430247699 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f347c0b67d0 0x7f347c0b8ce0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f34900aa010 tx=0x7f3490009320 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.722+0000 7f3483e00640 1 -- 172.21.15.27:0/3858012012 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34840043c0 con 0x55684ceb25f0 2024-09-21T00:35:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.724+0000 7f3483e00640 1 -- 172.21.15.27:0/3858012012 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34840035f0 con 0x55684ceb25f0 2024-09-21T00:35:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.725+0000 7f34a4c29740 1 -- 172.21.15.27:0/3858012012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceb25f0 msgr2=0x55684d279d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.725+0000 7f34a4c29740 1 --2- 172.21.15.27:0/3858012012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceb25f0 0x55684d279d10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f349806c970 tx=0x7f348400b650 comp rx=0 tx=0).stop 2024-09-21T00:35:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.725+0000 7f34a4c29740 1 -- 172.21.15.27:0/3858012012 shutdown_connections 2024-09-21T00:35:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.725+0000 7f34a4c29740 1 --2- 172.21.15.27:0/3858012012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceb25f0 0x55684d279d10 secure :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f349806c970 tx=0x7f348400b650 comp rx=0 tx=0).stop 2024-09-21T00:35:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.725+0000 7f34a4c29740 1 --2- 172.21.15.27:0/3858012012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ceaa410 0x55684d2797d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.725+0000 7f34a4c29740 1 --2- 172.21.15.27:0/3858012012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ced32d0 0x55684d279290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.725+0000 7f34a4c29740 1 -- 172.21.15.27:0/3858012012 >> 172.21.15.27:0/3858012012 conn(0x55684cec76c0 msgr2=0x55684d277180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.726+0000 7f34a4c29740 1 -- 172.21.15.27:0/3858012012 shutdown_connections 2024-09-21T00:35:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.726+0000 7f34a4c29740 1 -- 172.21.15.27:0/3858012012 wait complete. 2024-09-21T00:35:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.727+0000 7f34a4c29740 1 -- start start 2024-09-21T00:35:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.728+0000 7f34a4c29740 1 Processor -- start 2024-09-21T00:35:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.729+0000 7f34a4c29740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ceaa410 0x55684d284a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.729+0000 7f34a4c29740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ced32d0 0x55684d284f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.729+0000 7f34a4c29740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ce91c50 0x55684d290e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.729+0000 7f34a4c29740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55684d041810 con 0x55684ceaa410 2024-09-21T00:35:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.729+0000 7f34a4c29740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55684d17b550 con 0x55684ce91c50 2024-09-21T00:35:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.729+0000 7f34a4c29740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55684d287ac0 con 0x55684ced32d0 2024-09-21T00:35:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.729+0000 7f34a4c29740 10 client.0 _ll_register_callbacks cb 0x55684d18b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:35:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.730+0000 7f3497400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ced32d0 0x55684d284f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.730+0000 7f3497400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ced32d0 0x55684d284f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55808/0 (socket says 172.21.15.27:55808) 2024-09-21T00:35:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.731+0000 7f3497400640 1 -- 172.21.15.27:0/1169725949 learned_addr learned my addr 172.21.15.27:0/1169725949 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.731+0000 7f3496a00640 1 --2- 172.21.15.27:0/1169725949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ceaa410 0x55684d284a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.732+0000 7f3497400640 1 -- 172.21.15.27:0/1169725949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ce91c50 msgr2=0x55684d290e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T00:35:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.732+0000 7f3497400640 1 --2- 172.21.15.27:0/1169725949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ce91c50 0x55684d290e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.732+0000 7f3497400640 1 -- 172.21.15.27:0/1169725949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ceaa410 msgr2=0x55684d284a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.732+0000 7f3497400640 1 --2- 172.21.15.27:0/1169725949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ceaa410 0x55684d284a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.732+0000 7f3497400640 1 -- 172.21.15.27:0/1169725949 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55684d16d200 con 0x55684ced32d0 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.732+0000 7f3496a00640 1 --2- 172.21.15.27:0/1169725949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ceaa410 0x55684d284a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.733+0000 7f3497400640 1 --2- 172.21.15.27:0/1169725949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ced32d0 0x55684d284f50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f348802b670 tx=0x7f3488007e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6baed8279963e71 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.733+0000 7f3480c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.737+0000 7f3480c00640 1 -- 172.21.15.27:0/1169725949 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488043030 con 0x55684ced32d0 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.737+0000 7f3480c00640 1 -- 172.21.15.27:0/1169725949 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7f3488044070 con 0x55684ced32d0 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.737+0000 7f34a4c29740 1 -- 172.21.15.27:0/1169725949 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34840043c0 con 0x55684ced32d0 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.738+0000 7f3480c00640 1 -- 172.21.15.27:0/1169725949 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3488043030 con 0x55684ced32d0 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.738+0000 7f34a4c29740 10 client.26725 fetch_fsmap learned FSMap version 47 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.738+0000 7f34a4c29740 1 -- 172.21.15.27:0/1169725949 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55684d293920 con 0x55684ced32d0 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.738+0000 7f3480c00640 1 -- 172.21.15.27:0/1169725949 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f348800a810 con 0x55684ced32d0 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.740+0000 7f3480c00640 1 -- 172.21.15.27:0/1169725949 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f34880087a0 con 0x55684ced32d0 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.740+0000 7f34a4c29740 10 client.26725 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.740+0000 7f34a4c29740 10 client.26725 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.740+0000 7f34a4c29740 20 client.26725 populate_metadata read hostname 'smithi027' 2024-09-21T00:35:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.740+0000 7f34a4c29740 1 --2- 172.21.15.27:0/1169725949 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55684ce67000 0x55684d297380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.740+0000 7f34a4c29740 4 client.26725 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:35:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.740+0000 7f34a4c29740 1 -- 172.21.15.27:0/1169725949 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f34840043c0 con 0x55684ce67000 2024-09-21T00:35:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.741+0000 7f3496a00640 1 --2- 172.21.15.27:0/1169725949 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55684ce67000 0x55684d297380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.742+0000 7f3496a00640 1 --2- 172.21.15.27:0/1169725949 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55684ce67000 0x55684d297380 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19201ad23101d3c6 server_cookie=b25bac82f7c9ab02 in_seq=0 out_seq=0 2024-09-21T00:35:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.742+0000 7f3480c00640 10 client.26725 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.743+0000 7f3480c00640 1 -- 172.21.15.27:0/1169725949 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f34840043c0 con 0x55684ce67000 2024-09-21T00:35:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.743+0000 7f3480c00640 10 client.26725 handle_command_reply: tid=0 2024-09-21T00:35:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.743+0000 7f34a4c29740 1 client.26725 shutdown 2024-09-21T00:35:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.743+0000 7f34a4c29740 1 -- 172.21.15.27:0/1169725949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ced32d0 msgr2=0x55684d284f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.743+0000 7f34a4c29740 1 --2- 172.21.15.27:0/1169725949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ced32d0 0x55684d284f50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f348802b670 tx=0x7f3488007e90 comp rx=0 tx=0).stop 2024-09-21T00:35:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.743+0000 7f34a4c29740 1 -- 172.21.15.27:0/1169725949 shutdown_connections 2024-09-21T00:35:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.743+0000 7f34a4c29740 1 --2- 172.21.15.27:0/1169725949 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55684ce67000 0x55684d297380 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.743+0000 7f3480c00640 0 client.26725 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.743+0000 7f34a4c29740 1 --2- 172.21.15.27:0/1169725949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ce91c50 0x55684d290e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.743+0000 7f34a4c29740 1 --2- 172.21.15.27:0/1169725949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ced32d0 0x55684d284f50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.743+0000 7f34a4c29740 1 --2- 172.21.15.27:0/1169725949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ceaa410 0x55684d284a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.743+0000 7f34a4c29740 1 -- 172.21.15.27:0/1169725949 >> 172.21.15.27:0/1169725949 conn(0x55684cec76c0 msgr2=0x55684d27c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:35.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.744+0000 7f34a4c29740 1 -- 172.21.15.27:0/1169725949 shutdown_connections 2024-09-21T00:35:35.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.745+0000 7f34a4c29740 1 -- 172.21.15.27:0/1169725949 wait complete. 2024-09-21T00:35:35.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.745+0000 7f34a4c29740 20 client.26725 trim_cache size 0 max 16384 2024-09-21T00:35:35.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.779+0000 7f34a4c29740 1 Processor -- start 2024-09-21T00:35:35.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.782+0000 7f34a4c29740 1 -- start start 2024-09-21T00:35:35.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.782+0000 7f34a4c29740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ce91c50 0x55684d2868f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.782+0000 7f34a4c29740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceaa410 0x55684d27a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.782+0000 7f34a4c29740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ced32d0 0x55684d290d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.782+0000 7f34a4c29740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f34981cb360 con 0x55684ced32d0 2024-09-21T00:35:35.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.782+0000 7f34a4c29740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f34981c8b70 con 0x55684ce91c50 2024-09-21T00:35:35.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.782+0000 7f34a4c29740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f34981c69a0 con 0x55684ceaa410 2024-09-21T00:35:35.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.783+0000 7f3496a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ce91c50 0x55684d2868f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.783+0000 7f3496a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ce91c50 0x55684d2868f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40198/0 (socket says 172.21.15.27:40198) 2024-09-21T00:35:35.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.783+0000 7f3496a00640 1 -- 172.21.15.27:0/2255685853 learned_addr learned my addr 172.21.15.27:0/2255685853 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:35.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.783+0000 7f3497400640 1 --2- 172.21.15.27:0/2255685853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceaa410 0x55684d27a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.784+0000 7f3497400640 1 -- 172.21.15.27:0/2255685853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ce91c50 msgr2=0x55684d2868f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:35.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.784+0000 7f3497400640 1 --2- 172.21.15.27:0/2255685853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ce91c50 0x55684d2868f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.784+0000 7f3497400640 1 -- 172.21.15.27:0/2255685853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ced32d0 msgr2=0x55684d290d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:35.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.784+0000 7f3497400640 1 --2- 172.21.15.27:0/2255685853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ced32d0 0x55684d290d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.784+0000 7f3497400640 1 -- 172.21.15.27:0/2255685853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55684d271340 con 0x55684ceaa410 2024-09-21T00:35:35.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.784+0000 7f3497400640 1 --2- 172.21.15.27:0/2255685853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceaa410 0x55684d27a9d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f348803a660 tx=0x7f3488008570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17857cb09b64b64c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:35.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.785+0000 7f3483400640 1 -- 172.21.15.27:0/2255685853 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488043030 con 0x55684ceaa410 2024-09-21T00:35:35.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.785+0000 7f3483400640 1 -- 172.21.15.27:0/2255685853 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f348800a810 con 0x55684ceaa410 2024-09-21T00:35:35.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.785+0000 7f3483400640 1 -- 172.21.15.27:0/2255685853 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488046d90 con 0x55684ceaa410 2024-09-21T00:35:35.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.786+0000 7f34a4c29740 1 -- 172.21.15.27:0/2255685853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceaa410 msgr2=0x55684d27a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:35.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.786+0000 7f34a4c29740 1 --2- 172.21.15.27:0/2255685853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceaa410 0x55684d27a9d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f348803a660 tx=0x7f3488008570 comp rx=0 tx=0).stop 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.787+0000 7f34a4c29740 1 -- 172.21.15.27:0/2255685853 shutdown_connections 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.787+0000 7f34a4c29740 1 --2- 172.21.15.27:0/2255685853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ced32d0 0x55684d290d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.787+0000 7f34a4c29740 1 --2- 172.21.15.27:0/2255685853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceaa410 0x55684d27a9d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.787+0000 7f34a4c29740 1 --2- 172.21.15.27:0/2255685853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ce91c50 0x55684d2868f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.787+0000 7f34a4c29740 1 -- 172.21.15.27:0/2255685853 >> 172.21.15.27:0/2255685853 conn(0x55684cec76c0 msgr2=0x55684d27a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.787+0000 7f34a4c29740 1 -- 172.21.15.27:0/2255685853 shutdown_connections 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.788+0000 7f34a4c29740 1 -- 172.21.15.27:0/2255685853 wait complete. 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.789+0000 7f34a4c29740 1 -- start start 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.789+0000 7f34a4c29740 1 Processor -- start 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.790+0000 7f34a4c29740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ce91c50 0x55684d28cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.790+0000 7f34a4c29740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceaa410 0x55684d28d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.790+0000 7f34a4c29740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ced32d0 0x55684d287d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.790+0000 7f34a4c29740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f34981c7030 con 0x55684ce91c50 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.790+0000 7f34a4c29740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f34981c84a0 con 0x55684ced32d0 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.790+0000 7f34a4c29740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f34981c7700 con 0x55684ceaa410 2024-09-21T00:35:35.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.790+0000 7f34a4c29740 10 client.0 _ll_register_callbacks cb 0x7f348802bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:35:35.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.790+0000 7f3497400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceaa410 0x55684d28d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.790+0000 7f3497400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceaa410 0x55684d28d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55818/0 (socket says 172.21.15.27:55818) 2024-09-21T00:35:35.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.791+0000 7f3497400640 1 -- 172.21.15.27:0/818221090 learned_addr learned my addr 172.21.15.27:0/818221090 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:35.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.792+0000 7f3496a00640 1 --2- 172.21.15.27:0/818221090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ced32d0 0x55684d287d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.792+0000 7f3497400640 1 -- 172.21.15.27:0/818221090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ced32d0 msgr2=0x55684d287d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:35.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.792+0000 7f3496a00640 1 --2- 172.21.15.27:0/818221090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ce91c50 0x55684d28cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.792+0000 7f3497400640 1 --2- 172.21.15.27:0/818221090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ced32d0 0x55684d287d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.792+0000 7f3497400640 1 -- 172.21.15.27:0/818221090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ce91c50 msgr2=0x55684d28cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:35:35.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.792+0000 7f3497400640 1 --2- 172.21.15.27:0/818221090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ce91c50 0x55684d28cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.792+0000 7f3497400640 1 -- 172.21.15.27:0/818221090 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55684d288290 con 0x55684ceaa410 2024-09-21T00:35:35.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.792+0000 7f3497400640 1 --2- 172.21.15.27:0/818221090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceaa410 0x55684d28d2e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f348800ace0 tx=0x7f3488008ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19c49a040da4ed3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:35.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.792+0000 7f3480c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T00:35:35.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.793+0000 7f3480c00640 1 -- 172.21.15.27:0/818221090 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488056060 con 0x55684ceaa410 2024-09-21T00:35:35.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.793+0000 7f3480c00640 1 -- 172.21.15.27:0/818221090 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7f348803b330 con 0x55684ceaa410 2024-09-21T00:35:35.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.793+0000 7f34a4c29740 1 -- 172.21.15.27:0/818221090 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f348800a810 con 0x55684ceaa410 2024-09-21T00:35:35.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.793+0000 7f3480c00640 1 -- 172.21.15.27:0/818221090 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3488043070 con 0x55684ceaa410 2024-09-21T00:35:35.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.793+0000 7f34a4c29740 10 client.26731 fetch_fsmap learned FSMap version 47 2024-09-21T00:35:35.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.793+0000 7f34a4c29740 1 -- 172.21.15.27:0/818221090 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55684d302330 con 0x55684ceaa410 2024-09-21T00:35:35.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.794+0000 7f3480c00640 1 -- 172.21.15.27:0/818221090 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f348800a810 con 0x55684ceaa410 2024-09-21T00:35:35.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.795+0000 7f3480c00640 1 -- 172.21.15.27:0/818221090 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f34880087a0 con 0x55684ceaa410 2024-09-21T00:35:35.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.795+0000 7f34a4c29740 10 client.26731 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:35:35.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.795+0000 7f34a4c29740 10 client.26731 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T00:35:35.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.795+0000 7f34a4c29740 20 client.26731 populate_metadata read hostname 'smithi027' 2024-09-21T00:35:35.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.795+0000 7f34a4c29740 1 --2- 172.21.15.27:0/818221090 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55684d3029e0 0x55684d304e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:35.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.795+0000 7f34a4c29740 4 client.26731 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-21T00:35:35.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.795+0000 7f34a4c29740 1 -- 172.21.15.27:0/818221090 --> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55684d1f2f40 con 0x55684d3029e0 2024-09-21T00:35:35.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.800+0000 7f3496a00640 1 --2- 172.21.15.27:0/818221090 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55684d3029e0 0x55684d304e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:35.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.804+0000 7f3496a00640 1 --2- 172.21.15.27:0/818221090 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55684d3029e0 0x55684d304e00 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e0da2c9ea4035ef server_cookie=c2e5a3b9592c56e1 in_seq=0 out_seq=0 2024-09-21T00:35:35.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.804+0000 7f3480c00640 10 client.26731 ms_handle_connect on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:35.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.806+0000 7f3480c00640 1 -- 172.21.15.27:0/818221090 <== mds.0 v2:172.21.15.89:6838/2432693964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55684d1f2f40 con 0x55684d3029e0 2024-09-21T00:35:35.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.806+0000 7f3480c00640 10 client.26731 handle_command_reply: tid=0 2024-09-21T00:35:35.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.806+0000 7f34a4c29740 1 client.26731 shutdown 2024-09-21T00:35:35.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.807+0000 7f34a4c29740 1 -- 172.21.15.27:0/818221090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceaa410 msgr2=0x55684d28d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:35.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.807+0000 7f34a4c29740 1 --2- 172.21.15.27:0/818221090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceaa410 0x55684d28d2e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f348800ace0 tx=0x7f3488008ac0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.807+0000 7f34a4c29740 1 -- 172.21.15.27:0/818221090 shutdown_connections 2024-09-21T00:35:35.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.807+0000 7f34a4c29740 1 --2- 172.21.15.27:0/818221090 >> [v2:172.21.15.89:6838/2432693964,v1:172.21.15.89:6839/2432693964] conn(0x55684d3029e0 0x55684d304e00 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.808+0000 7f34a4c29740 1 --2- 172.21.15.27:0/818221090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55684ced32d0 0x55684d287d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.808+0000 7f34a4c29740 1 --2- 172.21.15.27:0/818221090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55684ceaa410 0x55684d28d2e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.808+0000 7f34a4c29740 1 --2- 172.21.15.27:0/818221090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55684ce91c50 0x55684d28cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.808+0000 7f34a4c29740 1 -- 172.21.15.27:0/818221090 >> 172.21.15.27:0/818221090 conn(0x55684cec76c0 msgr2=0x55684d27aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:35.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.810+0000 7f3480c00640 0 client.26731 ms_handle_reset on v2:172.21.15.89:6838/2432693964 2024-09-21T00:35:35.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.810+0000 7f34a4c29740 1 -- 172.21.15.27:0/818221090 shutdown_connections 2024-09-21T00:35:35.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.811+0000 7f34a4c29740 1 -- 172.21.15.27:0/818221090 wait complete. 2024-09-21T00:35:35.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.811+0000 7f34a4c29740 20 client.26731 trim_cache size 0 max 16384 2024-09-21T00:35:35.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.812+0000 7f349e000640 1 -- 172.21.15.27:0/2430247699 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f347c0b67d0 msgr2=0x7f347c0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:35.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.813+0000 7f349e000640 1 --2- 172.21.15.27:0/2430247699 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f347c0b67d0 0x7f347c0b8ce0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f34900aa010 tx=0x7f3490009320 comp rx=0 tx=0).stop 2024-09-21T00:35:35.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.813+0000 7f349e000640 1 -- 172.21.15.27:0/2430247699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f349810d620 msgr2=0x7f34981268b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:35.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.813+0000 7f349e000640 1 --2- 172.21.15.27:0/2430247699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f349810d620 0x7f34981268b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f349000be20 tx=0x7f349000bf10 comp rx=0 tx=0).stop 2024-09-21T00:35:35.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.817+0000 7f349e000640 1 -- 172.21.15.27:0/2430247699 shutdown_connections 2024-09-21T00:35:35.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.817+0000 7f349e000640 1 --2- 172.21.15.27:0/2430247699 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f347c0b67d0 0x7f347c0b8ce0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.817+0000 7f349e000640 1 --2- 172.21.15.27:0/2430247699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f349810d620 0x7f34981268b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.817+0000 7f349e000640 1 --2- 172.21.15.27:0/2430247699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349810ccd0 0x7f349811c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.817+0000 7f349e000640 1 --2- 172.21.15.27:0/2430247699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f349810a650 0x7f349811bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:35.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.817+0000 7f349e000640 1 -- 172.21.15.27:0/2430247699 >> 172.21.15.27:0/2430247699 conn(0x7f3498070ec0 msgr2=0x7f349810eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:35.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.817+0000 7f349e000640 1 -- 172.21.15.27:0/2430247699 shutdown_connections 2024-09-21T00:35:35.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:34.817+0000 7f349e000640 1 -- 172.21.15.27:0/2430247699 wait complete. 2024-09-21T00:35:36.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:35 smithi089 ceph-mon[34481]: pgmap v823: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 381 KiB/s wr, 238 op/s 2024-09-21T00:35:36.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:35 smithi098 ceph-mon[34725]: pgmap v823: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 381 KiB/s wr, 238 op/s 2024-09-21T00:35:36.302 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:35 smithi027 ceph-mon[30745]: pgmap v823: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 381 KiB/s wr, 238 op/s 2024-09-21T00:35:36.986 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/quiescer.py", line 422, in task stop_all_quiescers(ctx.ceph[cluster_name].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/quiescer.py", line 381, in stop_all_quiescers raise RuntimeError(f"error during quiesce thrashing: {thrasher.exception}") RuntimeError: error during quiesce thrashing: Error quiescing set '47099ce0': 1 (EPERM) 2024-09-21T00:35:36.987 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-21T00:35:36.996 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T00:35:37.020 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-21T00:35:37.055 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-21T00:35:37.055 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T00:35:37.055 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T00:35:37.055 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-21T00:35:37.187 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-21T00:35:37.324 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T00:35:37.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.723+0000 7f61a6a00640 1 -- 172.21.15.27:0/3346000788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61a0103050 msgr2=0x7f61a0107570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:37.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.723+0000 7f61a6a00640 1 --2- 172.21.15.27:0/3346000788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61a0103050 0x7f61a0107570 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6190009860 tx=0x7f619002a360 comp rx=0 tx=0).stop 2024-09-21T00:35:37.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.725+0000 7f61a6a00640 1 -- 172.21.15.27:0/3346000788 shutdown_connections 2024-09-21T00:35:37.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.725+0000 7f61a6a00640 1 --2- 172.21.15.27:0/3346000788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a0107bc0 0x7f61a0108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:37.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.725+0000 7f61a6a00640 1 --2- 172.21.15.27:0/3346000788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61a0103050 0x7f61a0107570 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:37.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.725+0000 7f61a6a00640 1 --2- 172.21.15.27:0/3346000788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61a0102700 0x7f61a0102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:37.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.725+0000 7f61a6a00640 1 -- 172.21.15.27:0/3346000788 >> 172.21.15.27:0/3346000788 conn(0x7f61a00fe3c0 msgr2=0x7f61a0100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:37.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.726+0000 7f61a6a00640 1 -- 172.21.15.27:0/3346000788 shutdown_connections 2024-09-21T00:35:37.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.726+0000 7f61a6a00640 1 -- 172.21.15.27:0/3346000788 wait complete. 2024-09-21T00:35:37.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.727+0000 7f61a6a00640 1 Processor -- start 2024-09-21T00:35:37.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.730+0000 7f61a6a00640 1 -- start start 2024-09-21T00:35:37.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.730+0000 7f61a6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61a0102700 0x7f61a01a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:37.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.730+0000 7f61a6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61a0103050 0x7f61a01a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:37.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.730+0000 7f61a6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a0107bc0 0x7f61a01a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:37.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.730+0000 7f61a6a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f61a0119fd0 con 0x7f61a0102700 2024-09-21T00:35:37.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.730+0000 7f61a6a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f61a0119e50 con 0x7f61a0107bc0 2024-09-21T00:35:37.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.730+0000 7f61a6a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f61a011a150 con 0x7f61a0103050 2024-09-21T00:35:37.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.731+0000 7f619f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61a0103050 0x7f61a01a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:37.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.731+0000 7f61a4a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a0107bc0 0x7f61a01a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:37.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.731+0000 7f61a4a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a0107bc0 0x7f61a01a6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40282/0 (socket says 172.21.15.27:40282) 2024-09-21T00:35:37.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.731+0000 7f61a4a00640 1 -- 172.21.15.27:0/1595494399 learned_addr learned my addr 172.21.15.27:0/1595494399 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:37.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.732+0000 7f619f400640 1 -- 172.21.15.27:0/1595494399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a0107bc0 msgr2=0x7f61a01a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:37.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.732+0000 7f619f400640 1 --2- 172.21.15.27:0/1595494399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a0107bc0 0x7f61a01a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:37.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.732+0000 7f619f400640 1 -- 172.21.15.27:0/1595494399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61a0102700 msgr2=0x7f61a01a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:37.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.732+0000 7f619f400640 1 --2- 172.21.15.27:0/1595494399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61a0102700 0x7f61a01a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:37.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.732+0000 7f619f400640 1 -- 172.21.15.27:0/1595494399 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a01a6960 con 0x7f61a0103050 2024-09-21T00:35:37.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.732+0000 7f619f400640 1 --2- 172.21.15.27:0/1595494399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61a0103050 0x7f61a01a0fd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f619002a880 tx=0x7f6190032dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:37.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.733+0000 7f619cc00640 1 -- 172.21.15.27:0/1595494399 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6190040020 con 0x7f61a0103050 2024-09-21T00:35:37.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.733+0000 7f61a6a00640 1 -- 172.21.15.27:0/1595494399 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a01a6bf0 con 0x7f61a0103050 2024-09-21T00:35:37.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.733+0000 7f619cc00640 1 -- 172.21.15.27:0/1595494399 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6190030450 con 0x7f61a0103050 2024-09-21T00:35:37.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.733+0000 7f619cc00640 1 -- 172.21.15.27:0/1595494399 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6190037040 con 0x7f61a0103050 2024-09-21T00:35:37.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.733+0000 7f61a6a00640 1 -- 172.21.15.27:0/1595494399 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a01ad3e0 con 0x7f61a0103050 2024-09-21T00:35:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.736+0000 7f61a6a00640 1 -- 172.21.15.27:0/1595494399 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a0107570 con 0x7f61a0103050 2024-09-21T00:35:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.742+0000 7f619cc00640 1 -- 172.21.15.27:0/1595494399 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6190043020 con 0x7f61a0103050 2024-09-21T00:35:37.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.745+0000 7f619cc00640 1 --2- 172.21.15.27:0/1595494399 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f61840b6700 0x7f61840b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.746+0000 7f619cc00640 1 -- 172.21.15.27:0/1595494399 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7f6190107b80 con 0x7f61a0103050 2024-09-21T00:35:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.747+0000 7f619fe00640 1 --2- 172.21.15.27:0/1595494399 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f61840b6700 0x7f61840b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.749+0000 7f619fe00640 1 --2- 172.21.15.27:0/1595494399 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f61840b6700 0x7f61840b8c10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6198005fd0 tx=0x7f6198009710 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:37.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:37.750+0000 7f619cc00640 1 -- 172.21.15.27:0/1595494399 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f619004a020 con 0x7f61a0103050 2024-09-21T00:35:38.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.197+0000 7f61a6a00640 1 -- 172.21.15.27:0/1595494399 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f61a0106c60 con 0x7f61840b6700 2024-09-21T00:35:38.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:37 smithi089 ceph-mon[34481]: pgmap v824: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 333 KiB/s wr, 234 op/s 2024-09-21T00:35:38.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:37 smithi098 ceph-mon[34725]: pgmap v824: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 333 KiB/s wr, 234 op/s 2024-09-21T00:35:38.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:37 smithi027 ceph-mon[30745]: pgmap v824: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 333 KiB/s wr, 234 op/s 2024-09-21T00:35:38.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.318+0000 7f619cc00640 1 -- 172.21.15.27:0/1595494399 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f61a0106c60 con 0x7f61840b6700 2024-09-21T00:35:38.319 INFO:teuthology.orchestra.run.smithi027.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T00:21:36", "first": "2024-09-21T00:22:00", "last": "2024-09-21T00:35:00", "last_pruned": "2024-09-21T00:35:02", "created_count": 14, "pruned_count": 8, "active": true} 2024-09-21T00:35:38.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.323+0000 7f61a6a00640 1 -- 172.21.15.27:0/1595494399 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f61840b6700 msgr2=0x7f61840b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:38.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.323+0000 7f61a6a00640 1 --2- 172.21.15.27:0/1595494399 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f61840b6700 0x7f61840b8c10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6198005fd0 tx=0x7f6198009710 comp rx=0 tx=0).stop 2024-09-21T00:35:38.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.323+0000 7f61a6a00640 1 -- 172.21.15.27:0/1595494399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61a0103050 msgr2=0x7f61a01a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:38.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.323+0000 7f61a6a00640 1 --2- 172.21.15.27:0/1595494399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61a0103050 0x7f61a01a0fd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f619002a880 tx=0x7f6190032dc0 comp rx=0 tx=0).stop 2024-09-21T00:35:38.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.323+0000 7f61a6a00640 1 -- 172.21.15.27:0/1595494399 shutdown_connections 2024-09-21T00:35:38.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.323+0000 7f61a6a00640 1 --2- 172.21.15.27:0/1595494399 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f61840b6700 0x7f61840b8c10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:38.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.323+0000 7f61a6a00640 1 --2- 172.21.15.27:0/1595494399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a0107bc0 0x7f61a01a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:38.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.323+0000 7f61a6a00640 1 --2- 172.21.15.27:0/1595494399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61a0103050 0x7f61a01a0fd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:38.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.323+0000 7f61a6a00640 1 --2- 172.21.15.27:0/1595494399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61a0102700 0x7f61a01a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:38.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.323+0000 7f61a6a00640 1 -- 172.21.15.27:0/1595494399 >> 172.21.15.27:0/1595494399 conn(0x7f61a00fe3c0 msgr2=0x7f61a0105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:38.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.323+0000 7f61a6a00640 1 -- 172.21.15.27:0/1595494399 shutdown_connections 2024-09-21T00:35:38.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.324+0000 7f61a6a00640 1 -- 172.21.15.27:0/1595494399 wait complete. 2024-09-21T00:35:38.365 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T00:35:38.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.767+0000 7f3a9ec00640 1 -- 172.21.15.27:0/947018307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a98104780 msgr2=0x7f3a98104b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:38.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.767+0000 7f3a9ec00640 1 --2- 172.21.15.27:0/947018307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a98104780 0x7f3a98104b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009900 tx=0x7f3a8402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:35:38.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.767+0000 7f3a9ec00640 1 -- 172.21.15.27:0/947018307 shutdown_connections 2024-09-21T00:35:38.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.767+0000 7f3a9ec00640 1 --2- 172.21.15.27:0/947018307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a9810dba0 0x7f3a981100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:38.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.767+0000 7f3a9ec00640 1 --2- 172.21.15.27:0/947018307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a981050d0 0x7f3a9810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:38.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.767+0000 7f3a9ec00640 1 --2- 172.21.15.27:0/947018307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a98104780 0x7f3a98104b90 secure :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009900 tx=0x7f3a8402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:35:38.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.767+0000 7f3a9ec00640 1 -- 172.21.15.27:0/947018307 >> 172.21.15.27:0/947018307 conn(0x7f3a980fe210 msgr2=0x7f3a98100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:38.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.768+0000 7f3a9ec00640 1 -- 172.21.15.27:0/947018307 shutdown_connections 2024-09-21T00:35:38.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.768+0000 7f3a9ec00640 1 -- 172.21.15.27:0/947018307 wait complete. 2024-09-21T00:35:38.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.769+0000 7f3a9ec00640 1 Processor -- start 2024-09-21T00:35:38.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.772+0000 7f3a9ec00640 1 -- start start 2024-09-21T00:35:38.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.773+0000 7f3a9ec00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a981050d0 0x7f3a981058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:38.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.773+0000 7f3a9ec00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9810dba0 0x7f3a98105e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.773+0000 7f3a9ec00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a9810b050 0x7f3a9810b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.773+0000 7f3a9ec00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9806d6e0 con 0x7f3a9810dba0 2024-09-21T00:35:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.773+0000 7f3a9ec00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9806d560 con 0x7f3a981050d0 2024-09-21T00:35:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.773+0000 7f3a9ec00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9806d860 con 0x7f3a9810b050 2024-09-21T00:35:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.773+0000 7f3a9d800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a981050d0 0x7f3a981058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.773+0000 7f3a9e200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a9810b050 0x7f3a9810b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.773+0000 7f3a9d800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a981050d0 0x7f3a981058e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40296/0 (socket says 172.21.15.27:40296) 2024-09-21T00:35:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.773+0000 7f3a9d800640 1 -- 172.21.15.27:0/2496365350 learned_addr learned my addr 172.21.15.27:0/2496365350 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.774+0000 7f3a9ce00640 1 --2- 172.21.15.27:0/2496365350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9810dba0 0x7f3a98105e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:38.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.774+0000 7f3a9e200640 1 -- 172.21.15.27:0/2496365350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a981050d0 msgr2=0x7f3a981058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:38.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.774+0000 7f3a9e200640 1 --2- 172.21.15.27:0/2496365350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a981050d0 0x7f3a981058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:38.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.774+0000 7f3a9e200640 1 -- 172.21.15.27:0/2496365350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9810dba0 msgr2=0x7f3a98105e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:38.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.774+0000 7f3a9e200640 1 --2- 172.21.15.27:0/2496365350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9810dba0 0x7f3a98105e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:38.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.774+0000 7f3a9e200640 1 -- 172.21.15.27:0/2496365350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9810bc00 con 0x7f3a9810b050 2024-09-21T00:35:38.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.775+0000 7f3a9e200640 1 --2- 172.21.15.27:0/2496365350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a9810b050 0x7f3a9810b460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000e940 tx=0x7f3a9000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:38.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.775+0000 7f3a96000640 1 -- 172.21.15.27:0/2496365350 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90004d20 con 0x7f3a9810b050 2024-09-21T00:35:38.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.775+0000 7f3a9ec00640 1 -- 172.21.15.27:0/2496365350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9810c1c0 con 0x7f3a9810b050 2024-09-21T00:35:38.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.775+0000 7f3a96000640 1 -- 172.21.15.27:0/2496365350 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a90005020 con 0x7f3a9810b050 2024-09-21T00:35:38.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.775+0000 7f3a96000640 1 -- 172.21.15.27:0/2496365350 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a900116d0 con 0x7f3a9810b050 2024-09-21T00:35:38.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.775+0000 7f3a9ec00640 1 -- 172.21.15.27:0/2496365350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a9810c760 con 0x7f3a9810b050 2024-09-21T00:35:38.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.777+0000 7f3a9ec00640 1 -- 172.21.15.27:0/2496365350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a9810be90 con 0x7f3a9810b050 2024-09-21T00:35:38.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.783+0000 7f3a96000640 1 -- 172.21.15.27:0/2496365350 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3a90005460 con 0x7f3a9810b050 2024-09-21T00:35:38.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.786+0000 7f3a96000640 1 --2- 172.21.15.27:0/2496365350 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3a800b66f0 0x7f3a800b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:38.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.787+0000 7f3a96000640 1 -- 172.21.15.27:0/2496365350 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7f3a900e7ef0 con 0x7f3a9810b050 2024-09-21T00:35:38.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.787+0000 7f3a9d800640 1 --2- 172.21.15.27:0/2496365350 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3a800b66f0 0x7f3a800b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:38.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.787+0000 7f3a96000640 1 -- 172.21.15.27:0/2496365350 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3a90016030 con 0x7f3a9810b050 2024-09-21T00:35:38.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:38.788+0000 7f3a9d800640 1 --2- 172.21.15.27:0/2496365350 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3a800b66f0 0x7f3a800b8c00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3a84031440 tx=0x7f3a840058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:39.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:39.235+0000 7f3a9ec00640 1 -- 172.21.15.27:0/2496365350 --> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f3a98101640 con 0x7f3a800b66f0 2024-09-21T00:35:39.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:38 smithi098 ceph-mon[34725]: from='client.26752 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:35:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:38 smithi027 ceph-mon[30745]: from='client.26752 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:35:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:39.341+0000 7f3a96000640 1 -- 172.21.15.27:0/2496365350 <== mgr.14226 v2:172.21.15.27:6800/2382590111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f3a98101640 con 0x7f3a800b66f0 2024-09-21T00:35:39.342 INFO:teuthology.orchestra.run.smithi027.stdout:/ 1m 6m3h 2024-09-21T00:35:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:39.345+0000 7f3a9ec00640 1 -- 172.21.15.27:0/2496365350 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3a800b66f0 msgr2=0x7f3a800b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:39.345+0000 7f3a9ec00640 1 --2- 172.21.15.27:0/2496365350 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3a800b66f0 0x7f3a800b8c00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3a84031440 tx=0x7f3a840058d0 comp rx=0 tx=0).stop 2024-09-21T00:35:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:39.345+0000 7f3a9ec00640 1 -- 172.21.15.27:0/2496365350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a9810b050 msgr2=0x7f3a9810b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:39.345+0000 7f3a9ec00640 1 --2- 172.21.15.27:0/2496365350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a9810b050 0x7f3a9810b460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000e940 tx=0x7f3a9000ee00 comp rx=0 tx=0).stop 2024-09-21T00:35:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:39.345+0000 7f3a9ec00640 1 -- 172.21.15.27:0/2496365350 shutdown_connections 2024-09-21T00:35:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:39.345+0000 7f3a9ec00640 1 --2- 172.21.15.27:0/2496365350 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f3a800b66f0 0x7f3a800b8c00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:39.345+0000 7f3a9ec00640 1 --2- 172.21.15.27:0/2496365350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a9810b050 0x7f3a9810b460 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:39.345+0000 7f3a9ec00640 1 --2- 172.21.15.27:0/2496365350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9810dba0 0x7f3a98105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:39.345+0000 7f3a9ec00640 1 --2- 172.21.15.27:0/2496365350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a981050d0 0x7f3a981058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:39.345+0000 7f3a9ec00640 1 -- 172.21.15.27:0/2496365350 >> 172.21.15.27:0/2496365350 conn(0x7f3a980fe210 msgr2=0x7f3a98103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:39.346+0000 7f3a9ec00640 1 -- 172.21.15.27:0/2496365350 shutdown_connections 2024-09-21T00:35:39.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:39.346+0000 7f3a9ec00640 1 -- 172.21.15.27:0/2496365350 wait complete. 2024-09-21T00:35:39.387 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T00:35:39.397 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:35:39.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:38 smithi089 ceph-mon[34481]: from='client.26752 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:35:40.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.114+0000 7fc469400640 1 -- 172.21.15.27:0/1398413000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc46410dd80 msgr2=0x7fc464110270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:40.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.114+0000 7fc469400640 1 --2- 172.21.15.27:0/1398413000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc46410dd80 0x7fc464110270 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009ca0 tx=0x7fc45c02a8c0 comp rx=0 tx=0).stop 2024-09-21T00:35:40.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.114+0000 7fc469400640 1 -- 172.21.15.27:0/1398413000 shutdown_connections 2024-09-21T00:35:40.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.114+0000 7fc469400640 1 --2- 172.21.15.27:0/1398413000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc46410dd80 0x7fc464110270 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:40.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.114+0000 7fc469400640 1 --2- 172.21.15.27:0/1398413000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc464105270 0x7fc46410d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.114+0000 7fc469400640 1 --2- 172.21.15.27:0/1398413000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc464104920 0x7fc464104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.114+0000 7fc469400640 1 -- 172.21.15.27:0/1398413000 >> 172.21.15.27:0/1398413000 conn(0x7fc46407a660 msgr2=0x7fc46407aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.114+0000 7fc469400640 1 -- 172.21.15.27:0/1398413000 shutdown_connections 2024-09-21T00:35:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.114+0000 7fc469400640 1 -- 172.21.15.27:0/1398413000 wait complete. 2024-09-21T00:35:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.116+0000 7fc469400640 1 Processor -- start 2024-09-21T00:35:40.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.116+0000 7fc469400640 1 -- start start 2024-09-21T00:35:40.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.116+0000 7fc469400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc464104920 0x7fc46419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:40.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.117+0000 7fc469400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc464105270 0x7fc46419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:40.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.117+0000 7fc469400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc46410dd80 0x7fc4641a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:40.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.117+0000 7fc469400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc464113a00 con 0x7fc46410dd80 2024-09-21T00:35:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.117+0000 7fc469400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc464113880 con 0x7fc464104920 2024-09-21T00:35:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.117+0000 7fc469400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc464113b80 con 0x7fc464105270 2024-09-21T00:35:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.117+0000 7fc461e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc464105270 0x7fc46419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.117+0000 7fc461e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc464105270 0x7fc46419cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55940/0 (socket says 172.21.15.27:55940) 2024-09-21T00:35:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.117+0000 7fc461e00640 1 -- 172.21.15.27:0/1821599767 learned_addr learned my addr 172.21.15.27:0/1821599767 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.117+0000 7fc463200640 1 --2- 172.21.15.27:0/1821599767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc46410dd80 0x7fc4641a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.117+0000 7fc462800640 1 --2- 172.21.15.27:0/1821599767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc464104920 0x7fc46419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:40.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.118+0000 7fc461e00640 1 -- 172.21.15.27:0/1821599767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc464104920 msgr2=0x7fc46419c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:40.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.118+0000 7fc461e00640 1 --2- 172.21.15.27:0/1821599767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc464104920 0x7fc46419c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:40.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.118+0000 7fc461e00640 1 -- 172.21.15.27:0/1821599767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc46410dd80 msgr2=0x7fc4641a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:40.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.118+0000 7fc461e00640 1 --2- 172.21.15.27:0/1821599767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc46410dd80 0x7fc4641a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:40.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.118+0000 7fc461e00640 1 -- 172.21.15.27:0/1821599767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4641a24b0 con 0x7fc464105270 2024-09-21T00:35:40.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.118+0000 7fc461e00640 1 --2- 172.21.15.27:0/1821599767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc464105270 0x7fc46419cb20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc45000d950 tx=0x7fc45000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:40.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.119+0000 7fc45b400640 1 -- 172.21.15.27:0/1821599767 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450017070 con 0x7fc464105270 2024-09-21T00:35:40.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.119+0000 7fc45b400640 1 -- 172.21.15.27:0/1821599767 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc450009c30 con 0x7fc464105270 2024-09-21T00:35:40.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.119+0000 7fc469400640 1 -- 172.21.15.27:0/1821599767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4641a27a0 con 0x7fc464105270 2024-09-21T00:35:40.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.120+0000 7fc45b400640 1 -- 172.21.15.27:0/1821599767 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4500047a0 con 0x7fc464105270 2024-09-21T00:35:40.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.120+0000 7fc469400640 1 -- 172.21.15.27:0/1821599767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46406cde0 con 0x7fc464105270 2024-09-21T00:35:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.126+0000 7fc458c00640 1 -- 172.21.15.27:0/1821599767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc46406af90 con 0x7fc464105270 2024-09-21T00:35:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.126+0000 7fc45b400640 1 -- 172.21.15.27:0/1821599767 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc450004020 con 0x7fc464105270 2024-09-21T00:35:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.128+0000 7fc45b400640 1 --2- 172.21.15.27:0/1821599767 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc4480b6700 0x7fc4480b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.129+0000 7fc45b400640 1 -- 172.21.15.27:0/1821599767 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7fc4500e78d0 con 0x7fc464105270 2024-09-21T00:35:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.129+0000 7fc462800640 1 --2- 172.21.15.27:0/1821599767 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc4480b6700 0x7fc4480b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:40.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.132+0000 7fc462800640 1 --2- 172.21.15.27:0/1821599767 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc4480b6700 0x7fc4480b8c10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc454009b60 tx=0x7fc454005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:40.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.135+0000 7fc45b400640 1 -- 172.21.15.27:0/1821599767 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc450020020 con 0x7fc464105270 2024-09-21T00:35:40.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:39 smithi098 ceph-mon[34725]: pgmap v825: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 320 KiB/s wr, 221 op/s 2024-09-21T00:35:40.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:39 smithi098 ceph-mon[34725]: from='client.26758 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:35:40.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:39 smithi027 ceph-mon[30745]: pgmap v825: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 320 KiB/s wr, 221 op/s 2024-09-21T00:35:40.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:39 smithi027 ceph-mon[30745]: from='client.26758 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:35:40.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:39 smithi089 ceph-mon[34481]: pgmap v825: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 320 KiB/s wr, 221 op/s 2024-09-21T00:35:40.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:39 smithi089 ceph-mon[34481]: from='client.26758 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T00:35:40.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.540+0000 7fc458c00640 1 -- 172.21.15.27:0/1821599767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc464104d30 con 0x7fc464105270 2024-09-21T00:35:40.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.543+0000 7fc45b400640 1 -- 172.21.15.27:0/1821599767 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+25579 (secure 0 0 0) 0x7fc4500ad9b0 con 0x7fc464105270 2024-09-21T00:35:40.545 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:35:40.545 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":110,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","created":"2024-09-21T00:09:20.149938+0000","modified":"2024-09-21T00:35:07.814442+0000","last_up_change":"2024-09-21T00:17:43.978785+0000","last_in_change":"2024-09-21T00:17:21.325177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:13:32.718543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:18:33.992414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:18:36.817691+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":110,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:18:46.707770+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":110,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63ccc094-62d5-446f-844d-1c60d41fda1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6803","nonce":354370090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6805","nonce":354370090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6809","nonce":354370090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6807","nonce":354370090}]},"public_addr":"172.21.15.27:6803/354370090","cluster_addr":"172.21.15.27:6805/354370090","heartbeat_back_addr":"172.21.15.27:6809/354370090","heartbeat_front_addr":"172.21.15.27:6807/354370090","state":["exists","up"]},{"osd":1,"uuid":"e59a8ed4-3a37-4477-a60a-4abc33d49b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6801","nonce":922528621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6803","nonce":922528621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6807","nonce":922528621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6805","nonce":922528621}]},"public_addr":"172.21.15.89:6801/922528621","cluster_addr":"172.21.15.89:6803/922528621","heartbeat_back_addr":"172.21.15.89:6807/922528621","heartbeat_front_addr":"172.21.15.89:6805/922528621","state":["exists","up"]},{"osd":2,"uuid":"6058a875-d5b2-4459-9407-2fd190cb34a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6801","nonce":257636259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6803","nonce":257636259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6807","nonce":257636259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6805","nonce":257636259}]},"public_addr":"172.21.15.98:6801/257636259","cluster_addr":"172.21.15.98:6803/257636259","heartbeat_back_addr":"172.21.15.98:6807/257636259","heartbeat_front_addr":"172.21.15.98:6805/257636259","state":["exists","up"]},{"osd":3,"uuid":"dfcdaede-e51c-4985-92af-3e296435a36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6811","nonce":821088824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6813","nonce":821088824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6817","nonce":821088824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6815","nonce":821088824}]},"public_addr":"172.21.15.27:6811/821088824","cluster_addr":"172.21.15.27:6813/821088824","heartbeat_back_addr":"172.21.15.27:6817/821088824","heartbeat_front_addr":"172.21.15.27:6815/821088824","state":["exists","up"]},{"osd":4,"uuid":"7c4af037-8ab6-47c0-b62b-76bd775b3491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6809","nonce":4001038690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6811","nonce":4001038690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6815","nonce":4001038690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6813","nonce":4001038690}]},"public_addr":"172.21.15.89:6809/4001038690","cluster_addr":"172.21.15.89:6811/4001038690","heartbeat_back_addr":"172.21.15.89:6815/4001038690","heartbeat_front_addr":"172.21.15.89:6813/4001038690","state":["exists","up"]},{"osd":5,"uuid":"e996ed0d-88ad-4aa7-891f-dfade4e09cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6809","nonce":3029903562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6811","nonce":3029903562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6815","nonce":3029903562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6813","nonce":3029903562}]},"public_addr":"172.21.15.98:6809/3029903562","cluster_addr":"172.21.15.98:6811/3029903562","heartbeat_back_addr":"172.21.15.98:6815/3029903562","heartbeat_front_addr":"172.21.15.98:6813/3029903562","state":["exists","up"]},{"osd":6,"uuid":"cf080497-57c2-42d0-8c8f-1eed0110702d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6819","nonce":1286808857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6821","nonce":1286808857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6825","nonce":1286808857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6823","nonce":1286808857}]},"public_addr":"172.21.15.27:6819/1286808857","cluster_addr":"172.21.15.27:6821/1286808857","heartbeat_back_addr":"172.21.15.27:6825/1286808857","heartbeat_front_addr":"172.21.15.27:6823/1286808857","state":["exists","up"]},{"osd":7,"uuid":"ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6817","nonce":2479349546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6819","nonce":2479349546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6823","nonce":2479349546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6821","nonce":2479349546}]},"public_addr":"172.21.15.89:6817/2479349546","cluster_addr":"172.21.15.89:6819/2479349546","heartbeat_back_addr":"172.21.15.89:6823/2479349546","heartbeat_front_addr":"172.21.15.89:6821/2479349546","state":["exists","up"]},{"osd":8,"uuid":"0b558458-fc2d-4ed1-b72a-e2ea8cf811b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6817","nonce":2830381485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6819","nonce":2830381485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6823","nonce":2830381485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6821","nonce":2830381485}]},"public_addr":"172.21.15.98:6817/2830381485","cluster_addr":"172.21.15.98:6819/2830381485","heartbeat_back_addr":"172.21.15.98:6823/2830381485","heartbeat_front_addr":"172.21.15.98:6821/2830381485","state":["exists","up"]},{"osd":9,"uuid":"1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6827","nonce":3033977113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6829","nonce":3033977113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6833","nonce":3033977113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6831","nonce":3033977113}]},"public_addr":"172.21.15.27:6827/3033977113","cluster_addr":"172.21.15.27:6829/3033977113","heartbeat_back_addr":"172.21.15.27:6833/3033977113","heartbeat_front_addr":"172.21.15.27:6831/3033977113","state":["exists","up"]},{"osd":10,"uuid":"324dd6ce-68fe-4c76-959b-1c9fb0ae92a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6825","nonce":2370332225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6827","nonce":2370332225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6831","nonce":2370332225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6829","nonce":2370332225}]},"public_addr":"172.21.15.89:6825/2370332225","cluster_addr":"172.21.15.89:6827/2370332225","heartbeat_back_addr":"172.21.15.89:6831/2370332225","heartbeat_front_addr":"172.21.15.89:6829/2370332225","state":["exists","up"]},{"osd":11,"uuid":"5e34686e-350a-491e-b9d7-59174ea90944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6825","nonce":2795999064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6827","nonce":2795999064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6831","nonce":2795999064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6829","nonce":2795999064}]},"public_addr":"172.21.15.98:6825/2795999064","cluster_addr":"172.21.15.98:6827/2795999064","heartbeat_back_addr":"172.21.15.98:6831/2795999064","heartbeat_front_addr":"172.21.15.98:6829/2795999064","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:12:39.841958+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:06.684417+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:31.041378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:01.612108+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:28.592326+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:54.303714+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:25.523475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:52.523483+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:19.430503+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:50.599120+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:17.520452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:42.725550+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/502883064":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/831048090":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/3880378905":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/793991452":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/2077956479":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6800/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6801/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/1098432608":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6800/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/528019907":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/2173601508":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:6800/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/764776233":"2024-09-22T00:11:24.231101+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":10,"length":1},{"begin":21,"length":1},{"begin":23,"length":1}]},{"pool":4,"snaps":[{"begin":8,"length":1},{"begin":10,"length":1},{"begin":21,"length":1},{"begin":23,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]},{"pool":4,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:35:40.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.547+0000 7fc458c00640 1 -- 172.21.15.27:0/1821599767 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc4480b6700 msgr2=0x7fc4480b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:40.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.547+0000 7fc458c00640 1 --2- 172.21.15.27:0/1821599767 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc4480b6700 0x7fc4480b8c10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc454009b60 tx=0x7fc454005f90 comp rx=0 tx=0).stop 2024-09-21T00:35:40.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.547+0000 7fc458c00640 1 -- 172.21.15.27:0/1821599767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc464105270 msgr2=0x7fc46419cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:40.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.547+0000 7fc458c00640 1 --2- 172.21.15.27:0/1821599767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc464105270 0x7fc46419cb20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc45000d950 tx=0x7fc45000de10 comp rx=0 tx=0).stop 2024-09-21T00:35:40.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.548+0000 7fc458c00640 1 -- 172.21.15.27:0/1821599767 shutdown_connections 2024-09-21T00:35:40.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.548+0000 7fc458c00640 1 --2- 172.21.15.27:0/1821599767 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fc4480b6700 0x7fc4480b8c10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:40.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.548+0000 7fc458c00640 1 --2- 172.21.15.27:0/1821599767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc46410dd80 0x7fc4641a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:40.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.548+0000 7fc458c00640 1 --2- 172.21.15.27:0/1821599767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc464105270 0x7fc46419cb20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:40.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.548+0000 7fc458c00640 1 --2- 172.21.15.27:0/1821599767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc464104920 0x7fc46419c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:40.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.548+0000 7fc458c00640 1 -- 172.21.15.27:0/1821599767 >> 172.21.15.27:0/1821599767 conn(0x7fc46407a660 msgr2=0x7fc464103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:40.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.548+0000 7fc458c00640 1 -- 172.21.15.27:0/1821599767 shutdown_connections 2024-09-21T00:35:40.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:40.551+0000 7fc458c00640 1 -- 172.21.15.27:0/1821599767 wait complete. 2024-09-21T00:35:40.598 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T00:13:32.718543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T00:18:33.992414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T00:18:36.817691+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 110, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-21T00:18:46.707770+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 110, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-21T00:35:40.599 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T00:35:41.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.001+0000 7f5d1ac00640 1 -- 172.21.15.27:0/1293465466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d1410db40 msgr2=0x7f5d14110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:41.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.001+0000 7f5d1ac00640 1 --2- 172.21.15.27:0/1293465466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d1410db40 0x7f5d14110030 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009ca0 tx=0x7f5d0c02a8e0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.001+0000 7f5d1ac00640 1 -- 172.21.15.27:0/1293465466 shutdown_connections 2024-09-21T00:35:41.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.001+0000 7f5d1ac00640 1 --2- 172.21.15.27:0/1293465466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d1410db40 0x7f5d14110030 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.001+0000 7f5d1ac00640 1 --2- 172.21.15.27:0/1293465466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d14105030 0x7f5d1410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.001+0000 7f5d1ac00640 1 --2- 172.21.15.27:0/1293465466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d141046e0 0x7f5d14104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.001+0000 7f5d1ac00640 1 -- 172.21.15.27:0/1293465466 >> 172.21.15.27:0/1293465466 conn(0x7f5d140fe190 msgr2=0x7f5d141005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:41.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.002+0000 7f5d1ac00640 1 -- 172.21.15.27:0/1293465466 shutdown_connections 2024-09-21T00:35:41.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.002+0000 7f5d1ac00640 1 -- 172.21.15.27:0/1293465466 wait complete. 2024-09-21T00:35:41.006 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:41 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1821599767' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:35:41.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.006+0000 7f5d1ac00640 1 Processor -- start 2024-09-21T00:35:41.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.006+0000 7f5d1ac00640 1 -- start start 2024-09-21T00:35:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.008+0000 7f5d1ac00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d141046e0 0x7f5d1410d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.008+0000 7f5d1ac00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d14105030 0x7f5d1410b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.008+0000 7f5d1ac00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d1410db40 0x7f5d1410bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.008+0000 7f5d1ac00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14111650 con 0x7f5d141046e0 2024-09-21T00:35:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.008+0000 7f5d1ac00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141114d0 con 0x7f5d1410db40 2024-09-21T00:35:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.008+0000 7f5d1ac00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141117d0 con 0x7f5d14105030 2024-09-21T00:35:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.009+0000 7f5d18e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d14105030 0x7f5d1410b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.009+0000 7f5d1a200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d1410db40 0x7f5d1410bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.009+0000 7f5d18e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d14105030 0x7f5d1410b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55960/0 (socket says 172.21.15.27:55960) 2024-09-21T00:35:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.009+0000 7f5d18e00640 1 -- 172.21.15.27:0/1827506364 learned_addr learned my addr 172.21.15.27:0/1827506364 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.010+0000 7f5d18e00640 1 -- 172.21.15.27:0/1827506364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d1410db40 msgr2=0x7f5d1410bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.010+0000 7f5d18e00640 1 --2- 172.21.15.27:0/1827506364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d1410db40 0x7f5d1410bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.010+0000 7f5d18e00640 1 -- 172.21.15.27:0/1827506364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d141046e0 msgr2=0x7f5d1410d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.010+0000 7f5d18e00640 1 --2- 172.21.15.27:0/1827506364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d141046e0 0x7f5d1410d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.010+0000 7f5d18e00640 1 -- 172.21.15.27:0/1827506364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d141ac900 con 0x7f5d14105030 2024-09-21T00:35:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.011+0000 7f5d18e00640 1 --2- 172.21.15.27:0/1827506364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d14105030 0x7f5d1410b800 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000c8c0 tx=0x7f5d0000cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.012+0000 7f5d12000640 1 -- 172.21.15.27:0/1827506364 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00007bb0 con 0x7f5d14105030 2024-09-21T00:35:41.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.012+0000 7f5d12000640 1 -- 172.21.15.27:0/1827506364 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d00007d50 con 0x7f5d14105030 2024-09-21T00:35:41.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.012+0000 7f5d12000640 1 -- 172.21.15.27:0/1827506364 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d000187c0 con 0x7f5d14105030 2024-09-21T00:35:41.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.012+0000 7f5d1ac00640 1 -- 172.21.15.27:0/1827506364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d141acad0 con 0x7f5d14105030 2024-09-21T00:35:41.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.012+0000 7f5d1ac00640 1 -- 172.21.15.27:0/1827506364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d141ad050 con 0x7f5d14105030 2024-09-21T00:35:41.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.016+0000 7f5d1ac00640 1 -- 172.21.15.27:0/1827506364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d14104af0 con 0x7f5d14105030 2024-09-21T00:35:41.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.021+0000 7f5d12000640 1 -- 172.21.15.27:0/1827506364 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5d0000fa20 con 0x7f5d14105030 2024-09-21T00:35:41.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.027+0000 7f5d12000640 1 --2- 172.21.15.27:0/1827506364 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5cfc0b6850 0x7f5cfc0b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:41.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.027+0000 7f5d12000640 1 -- 172.21.15.27:0/1827506364 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7f5d000e8070 con 0x7f5d14105030 2024-09-21T00:35:41.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.028+0000 7f5d12000640 1 -- 172.21.15.27:0/1827506364 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5d00014030 con 0x7f5d14105030 2024-09-21T00:35:41.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.029+0000 7f5d19800640 1 --2- 172.21.15.27:0/1827506364 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5cfc0b6850 0x7f5cfc0b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:41.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.030+0000 7f5d19800640 1 --2- 172.21.15.27:0/1827506364 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5cfc0b6850 0x7f5cfc0b8d60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5d1410d250 tx=0x7f5d08006c90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:41.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.428+0000 7f5d1ac00640 1 -- 172.21.15.27:0/1827506364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5d141ad4f0 con 0x7f5d14105030 2024-09-21T00:35:41.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.429+0000 7f5d12000640 1 -- 172.21.15.27:0/1827506364 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7f5d000ae1e0 con 0x7f5d14105030 2024-09-21T00:35:41.434 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-21T00:35:41.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.433+0000 7f5d1ac00640 1 -- 172.21.15.27:0/1827506364 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5cfc0b6850 msgr2=0x7f5cfc0b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:41.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.433+0000 7f5d1ac00640 1 --2- 172.21.15.27:0/1827506364 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5cfc0b6850 0x7f5cfc0b8d60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5d1410d250 tx=0x7f5d08006c90 comp rx=0 tx=0).stop 2024-09-21T00:35:41.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.433+0000 7f5d1ac00640 1 -- 172.21.15.27:0/1827506364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d14105030 msgr2=0x7f5d1410b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:41.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.433+0000 7f5d1ac00640 1 --2- 172.21.15.27:0/1827506364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d14105030 0x7f5d1410b800 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000c8c0 tx=0x7f5d0000cd80 comp rx=0 tx=0).stop 2024-09-21T00:35:41.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.437+0000 7f5d1ac00640 1 -- 172.21.15.27:0/1827506364 shutdown_connections 2024-09-21T00:35:41.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.437+0000 7f5d1ac00640 1 --2- 172.21.15.27:0/1827506364 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5cfc0b6850 0x7f5cfc0b8d60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.437+0000 7f5d1ac00640 1 --2- 172.21.15.27:0/1827506364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d1410db40 0x7f5d1410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.437+0000 7f5d1ac00640 1 --2- 172.21.15.27:0/1827506364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d14105030 0x7f5d1410b800 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.437+0000 7f5d1ac00640 1 --2- 172.21.15.27:0/1827506364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d141046e0 0x7f5d1410d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.437+0000 7f5d1ac00640 1 -- 172.21.15.27:0/1827506364 >> 172.21.15.27:0/1827506364 conn(0x7f5d140fe190 msgr2=0x7f5d14111dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:41.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.437+0000 7f5d1ac00640 1 -- 172.21.15.27:0/1827506364 shutdown_connections 2024-09-21T00:35:41.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.437+0000 7f5d1ac00640 1 -- 172.21.15.27:0/1827506364 wait complete. 2024-09-21T00:35:41.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:41 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1821599767' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:35:41.485 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T00:35:41.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:41 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1821599767' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:35:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.919+0000 7f1283200640 1 -- 172.21.15.27:0/2660579806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f127c104730 msgr2=0x7f127c104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.919+0000 7f1283200640 1 --2- 172.21.15.27:0/2660579806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f127c104730 0x7f127c104b40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1278009ca0 tx=0x7f127802a8c0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.919+0000 7f1283200640 1 -- 172.21.15.27:0/2660579806 shutdown_connections 2024-09-21T00:35:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.919+0000 7f1283200640 1 --2- 172.21.15.27:0/2660579806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f127c10db90 0x7f127c110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.919+0000 7f1283200640 1 --2- 172.21.15.27:0/2660579806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f127c105080 0x7f127c10d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.919+0000 7f1283200640 1 --2- 172.21.15.27:0/2660579806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f127c104730 0x7f127c104b40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.919+0000 7f1283200640 1 -- 172.21.15.27:0/2660579806 >> 172.21.15.27:0/2660579806 conn(0x7f127c0fe240 msgr2=0x7f127c100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.919+0000 7f1283200640 1 -- 172.21.15.27:0/2660579806 shutdown_connections 2024-09-21T00:35:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.919+0000 7f1283200640 1 -- 172.21.15.27:0/2660579806 wait complete. 2024-09-21T00:35:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.921+0000 7f1283200640 1 Processor -- start 2024-09-21T00:35:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.922+0000 7f1283200640 1 -- start start 2024-09-21T00:35:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.922+0000 7f1283200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f127c104730 0x7f127c198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.922+0000 7f1283200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f127c105080 0x7f127c199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.922+0000 7f1283200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f127c10db90 0x7f127c19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.922+0000 7f1283200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f127c113840 con 0x7f127c105080 2024-09-21T00:35:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.922+0000 7f1283200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1136c0 con 0x7f127c104730 2024-09-21T00:35:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.922+0000 7f1283200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1139c0 con 0x7f127c10db90 2024-09-21T00:35:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.923+0000 7f1282800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f127c10db90 0x7f127c19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.923+0000 7f1281e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f127c104730 0x7f127c198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.923+0000 7f1282800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f127c10db90 0x7f127c19fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55972/0 (socket says 172.21.15.27:55972) 2024-09-21T00:35:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.923+0000 7f1282800640 1 -- 172.21.15.27:0/58021474 learned_addr learned my addr 172.21.15.27:0/58021474 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.923+0000 7f1281e00640 1 -- 172.21.15.27:0/58021474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f127c10db90 msgr2=0x7f127c19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.923+0000 7f1281e00640 1 --2- 172.21.15.27:0/58021474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f127c10db90 0x7f127c19fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.923+0000 7f1281e00640 1 -- 172.21.15.27:0/58021474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f127c105080 msgr2=0x7f127c199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.923+0000 7f1281e00640 1 --2- 172.21.15.27:0/58021474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f127c105080 0x7f127c199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.923+0000 7f1281e00640 1 -- 172.21.15.27:0/58021474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127c199b20 con 0x7f127c104730 2024-09-21T00:35:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.924+0000 7f1282800640 1 --2- 172.21.15.27:0/58021474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f127c10db90 0x7f127c19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:35:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.924+0000 7f1281e00640 1 --2- 172.21.15.27:0/58021474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f127c104730 0x7f127c198d10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1278000f80 tx=0x7f12780319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.926+0000 7f1276a00640 1 -- 172.21.15.27:0/58021474 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278003b70 con 0x7f127c104730 2024-09-21T00:35:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.926+0000 7f1276a00640 1 -- 172.21.15.27:0/58021474 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1278003d10 con 0x7f127c104730 2024-09-21T00:35:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.926+0000 7f1276a00640 1 -- 172.21.15.27:0/58021474 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127802fb70 con 0x7f127c104730 2024-09-21T00:35:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.926+0000 7f1283200640 1 -- 172.21.15.27:0/58021474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f127c1a0370 con 0x7f127c104730 2024-09-21T00:35:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.926+0000 7f1283200640 1 -- 172.21.15.27:0/58021474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f127c1a0840 con 0x7f127c104730 2024-09-21T00:35:41.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.929+0000 7f126be00640 1 -- 172.21.15.27:0/58021474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f127c104b40 con 0x7f127c104730 2024-09-21T00:35:41.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.932+0000 7f1276a00640 1 -- 172.21.15.27:0/58021474 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f127802fd10 con 0x7f127c104730 2024-09-21T00:35:41.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.938+0000 7f1276a00640 1 --2- 172.21.15.27:0/58021474 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f12640b6700 0x7f12640b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:41.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.938+0000 7f1276a00640 1 -- 172.21.15.27:0/58021474 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7f1278108d30 con 0x7f127c104730 2024-09-21T00:35:41.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.939+0000 7f1281400640 1 --2- 172.21.15.27:0/58021474 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f12640b6700 0x7f12640b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:41.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.940+0000 7f1281400640 1 --2- 172.21.15.27:0/58021474 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f12640b6700 0x7f12640b8c10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f126c005fd0 tx=0x7f126c005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:41.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:41.942+0000 7f1276a00640 1 -- 172.21.15.27:0/58021474 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12780cee00 con 0x7f127c104730 2024-09-21T00:35:42.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:42 smithi027 ceph-mon[30745]: pgmap v826: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 180 KiB/s wr, 199 op/s 2024-09-21T00:35:42.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:42 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1827506364' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T00:35:42.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.337+0000 7f126be00640 1 -- 172.21.15.27:0/58021474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f127c1a1260 con 0x7f127c104730 2024-09-21T00:35:42.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.338+0000 7f1276a00640 1 -- 172.21.15.27:0/58021474 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) ==== 104+0+11 (secure 0 0 0) 0x7f12780d31e0 con 0x7f127c104730 2024-09-21T00:35:42.343 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-09-21T00:35:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.343+0000 7f126be00640 1 -- 172.21.15.27:0/58021474 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f12640b6700 msgr2=0x7f12640b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.343+0000 7f126be00640 1 --2- 172.21.15.27:0/58021474 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f12640b6700 0x7f12640b8c10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f126c005fd0 tx=0x7f126c005850 comp rx=0 tx=0).stop 2024-09-21T00:35:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.343+0000 7f126be00640 1 -- 172.21.15.27:0/58021474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f127c104730 msgr2=0x7f127c198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.343+0000 7f126be00640 1 --2- 172.21.15.27:0/58021474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f127c104730 0x7f127c198d10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1278000f80 tx=0x7f12780319f0 comp rx=0 tx=0).stop 2024-09-21T00:35:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.343+0000 7f126be00640 1 -- 172.21.15.27:0/58021474 shutdown_connections 2024-09-21T00:35:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.343+0000 7f126be00640 1 --2- 172.21.15.27:0/58021474 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f12640b6700 0x7f12640b8c10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.343+0000 7f126be00640 1 --2- 172.21.15.27:0/58021474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f127c10db90 0x7f127c19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.343+0000 7f126be00640 1 --2- 172.21.15.27:0/58021474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f127c105080 0x7f127c199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:42.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.343+0000 7f126be00640 1 --2- 172.21.15.27:0/58021474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f127c104730 0x7f127c198d10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:42.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.343+0000 7f126be00640 1 -- 172.21.15.27:0/58021474 >> 172.21.15.27:0/58021474 conn(0x7f127c0fe240 msgr2=0x7f127c102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:42.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.344+0000 7f126be00640 1 -- 172.21.15.27:0/58021474 shutdown_connections 2024-09-21T00:35:42.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.344+0000 7f126be00640 1 -- 172.21.15.27:0/58021474 wait complete. 2024-09-21T00:35:42.391 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T00:35:42.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:42 smithi089 ceph-mon[34481]: pgmap v826: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 180 KiB/s wr, 199 op/s 2024-09-21T00:35:42.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:42 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1827506364' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T00:35:42.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:42 smithi098 ceph-mon[34725]: pgmap v826: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 180 KiB/s wr, 199 op/s 2024-09-21T00:35:42.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:42 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1827506364' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T00:35:42.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.804+0000 7fe2dd400640 1 -- 172.21.15.27:0/4167176501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2d8103af0 msgr2=0x7fe2d8103f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.804+0000 7fe2dd400640 1 --2- 172.21.15.27:0/4167176501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2d8103af0 0x7fe2d8103f00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009900 tx=0x7fe2c402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:35:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.804+0000 7fe2dd400640 1 -- 172.21.15.27:0/4167176501 shutdown_connections 2024-09-21T00:35:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.804+0000 7fe2dd400640 1 --2- 172.21.15.27:0/4167176501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d8109d70 0x7fe2d81105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.804+0000 7fe2dd400640 1 --2- 172.21.15.27:0/4167176501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2d8109420 0x7fe2d8109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.804+0000 7fe2dd400640 1 --2- 172.21.15.27:0/4167176501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2d8103af0 0x7fe2d8103f00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.804+0000 7fe2dd400640 1 -- 172.21.15.27:0/4167176501 >> 172.21.15.27:0/4167176501 conn(0x7fe2d80ff570 msgr2=0x7fe2d81019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.804+0000 7fe2dd400640 1 -- 172.21.15.27:0/4167176501 shutdown_connections 2024-09-21T00:35:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.804+0000 7fe2dd400640 1 -- 172.21.15.27:0/4167176501 wait complete. 2024-09-21T00:35:42.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.806+0000 7fe2dd400640 1 Processor -- start 2024-09-21T00:35:42.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.806+0000 7fe2dd400640 1 -- start start 2024-09-21T00:35:42.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.807+0000 7fe2dd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d8103af0 0x7fe2d819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:42.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.807+0000 7fe2dd400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2d8109420 0x7fe2d819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:42.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.807+0000 7fe2dd400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2d8109d70 0x7fe2d81a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:42.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.808+0000 7fe2d5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2d8109420 0x7fe2d819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:42.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.808+0000 7fe2d5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2d8109420 0x7fe2d819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40374/0 (socket says 172.21.15.27:40374) 2024-09-21T00:35:42.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.808+0000 7fe2d6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d8103af0 0x7fe2d819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:42.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.808+0000 7fe2d7200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2d8109d70 0x7fe2d81a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:42.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.808+0000 7fe2d7200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2d8109d70 0x7fe2d81a1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56000/0 (socket says 172.21.15.27:56000) 2024-09-21T00:35:42.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.808+0000 7fe2d6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d8103af0 0x7fe2d819c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57674/0 (socket says 172.21.15.27:57674) 2024-09-21T00:35:42.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.808+0000 7fe2d5e00640 1 -- 172.21.15.27:0/746457524 learned_addr learned my addr 172.21.15.27:0/746457524 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:42.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.808+0000 7fe2dd400640 1 -- 172.21.15.27:0/746457524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d8076c80 con 0x7fe2d8103af0 2024-09-21T00:35:42.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.808+0000 7fe2dd400640 1 -- 172.21.15.27:0/746457524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d8076b00 con 0x7fe2d8109420 2024-09-21T00:35:42.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.808+0000 7fe2dd400640 1 -- 172.21.15.27:0/746457524 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d8076e00 con 0x7fe2d8109d70 2024-09-21T00:35:42.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.809+0000 7fe2d5e00640 1 -- 172.21.15.27:0/746457524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2d8109d70 msgr2=0x7fe2d81a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:42.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.809+0000 7fe2d5e00640 1 --2- 172.21.15.27:0/746457524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2d8109d70 0x7fe2d81a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:42.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.809+0000 7fe2d5e00640 1 -- 172.21.15.27:0/746457524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d8103af0 msgr2=0x7fe2d819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:42.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.809+0000 7fe2d5e00640 1 --2- 172.21.15.27:0/746457524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d8103af0 0x7fe2d819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:42.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.809+0000 7fe2d5e00640 1 -- 172.21.15.27:0/746457524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d81a25d0 con 0x7fe2d8109420 2024-09-21T00:35:42.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.809+0000 7fe2d6800640 1 --2- 172.21.15.27:0/746457524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d8103af0 0x7fe2d819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:35:42.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.809+0000 7fe2d5e00640 1 --2- 172.21.15.27:0/746457524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2d8109420 0x7fe2d819cc40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800d740 tx=0x7fe2c800dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.809+0000 7fe2cf400640 1 -- 172.21.15.27:0/746457524 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8017070 con 0x7fe2d8109420 2024-09-21T00:35:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.809+0000 7fe2dd400640 1 -- 172.21.15.27:0/746457524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d81a28c0 con 0x7fe2d8109420 2024-09-21T00:35:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.810+0000 7fe2dd400640 1 -- 172.21.15.27:0/746457524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d81a9050 con 0x7fe2d8109420 2024-09-21T00:35:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.810+0000 7fe2cf400640 1 -- 172.21.15.27:0/746457524 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c800de50 con 0x7fe2d8109420 2024-09-21T00:35:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.810+0000 7fe2cf400640 1 -- 172.21.15.27:0/746457524 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c800f620 con 0x7fe2d8109420 2024-09-21T00:35:42.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.813+0000 7fe2dd400640 1 -- 172.21.15.27:0/746457524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b80052c0 con 0x7fe2d8109420 2024-09-21T00:35:42.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.816+0000 7fe2cf400640 1 -- 172.21.15.27:0/746457524 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe2c801a070 con 0x7fe2d8109420 2024-09-21T00:35:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.818+0000 7fe2cf400640 1 --2- 172.21.15.27:0/746457524 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe2bc0b6780 0x7fe2bc0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.819+0000 7fe2cf400640 1 -- 172.21.15.27:0/746457524 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7fe2c80e73b0 con 0x7fe2d8109420 2024-09-21T00:35:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.819+0000 7fe2d6800640 1 --2- 172.21.15.27:0/746457524 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe2bc0b6780 0x7fe2bc0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.821+0000 7fe2d6800640 1 --2- 172.21.15.27:0/746457524 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe2bc0b6780 0x7fe2bc0b8c90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4005ab0 tx=0x7fe2c4005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:42.822+0000 7fe2cf400640 1 -- 172.21.15.27:0/746457524 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe2c80b1800 con 0x7fe2d8109420 2024-09-21T00:35:43.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.280+0000 7fe2dd400640 1 -- 172.21.15.27:0/746457524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe2b8005880 con 0x7fe2d8109420 2024-09-21T00:35:43.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.281+0000 7fe2cf400640 1 -- 172.21.15.27:0/746457524 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) ==== 100+0+11 (secure 0 0 0) 0x7fe2c80b6740 con 0x7fe2d8109420 2024-09-21T00:35:43.284 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-09-21T00:35:43.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.288+0000 7fe2dd400640 1 -- 172.21.15.27:0/746457524 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe2bc0b6780 msgr2=0x7fe2bc0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:43.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.288+0000 7fe2dd400640 1 --2- 172.21.15.27:0/746457524 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe2bc0b6780 0x7fe2bc0b8c90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4005ab0 tx=0x7fe2c4005a20 comp rx=0 tx=0).stop 2024-09-21T00:35:43.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.288+0000 7fe2dd400640 1 -- 172.21.15.27:0/746457524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2d8109420 msgr2=0x7fe2d819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:43.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.288+0000 7fe2dd400640 1 --2- 172.21.15.27:0/746457524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2d8109420 0x7fe2d819cc40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800d740 tx=0x7fe2c800dc00 comp rx=0 tx=0).stop 2024-09-21T00:35:43.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.288+0000 7fe2dd400640 1 -- 172.21.15.27:0/746457524 shutdown_connections 2024-09-21T00:35:43.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.288+0000 7fe2dd400640 1 --2- 172.21.15.27:0/746457524 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe2bc0b6780 0x7fe2bc0b8c90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:43.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.288+0000 7fe2dd400640 1 --2- 172.21.15.27:0/746457524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2d8109d70 0x7fe2d81a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:43.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.288+0000 7fe2dd400640 1 --2- 172.21.15.27:0/746457524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2d8109420 0x7fe2d819cc40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:43.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.288+0000 7fe2dd400640 1 --2- 172.21.15.27:0/746457524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d8103af0 0x7fe2d819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:43.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.288+0000 7fe2dd400640 1 -- 172.21.15.27:0/746457524 >> 172.21.15.27:0/746457524 conn(0x7fe2d80ff570 msgr2=0x7fe2d8107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:43.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.289+0000 7fe2dd400640 1 -- 172.21.15.27:0/746457524 shutdown_connections 2024-09-21T00:35:43.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.289+0000 7fe2dd400640 1 -- 172.21.15.27:0/746457524 wait complete. 2024-09-21T00:35:43.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:43 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/58021474' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T00:35:43.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:43 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:35:43.332 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-21T00:35:43.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:43 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/58021474' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T00:35:43.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:43 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:35:43.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:43 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/58021474' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T00:35:43.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:43 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:35:43.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.756+0000 7f2a44c00640 1 -- 172.21.15.27:0/640120558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a401023a0 msgr2=0x7f2a401027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:43.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.756+0000 7f2a44c00640 1 --2- 172.21.15.27:0/640120558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a401023a0 0x7f2a401027b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009900 tx=0x7f2a3002a480 comp rx=0 tx=0).stop 2024-09-21T00:35:43.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.756+0000 7f2a44c00640 1 -- 172.21.15.27:0/640120558 shutdown_connections 2024-09-21T00:35:43.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.756+0000 7f2a44c00640 1 --2- 172.21.15.27:0/640120558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a4010ff50 0x7f2a40112370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:43.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.756+0000 7f2a44c00640 1 --2- 172.21.15.27:0/640120558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a40102cf0 0x7f2a4010fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:43.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.756+0000 7f2a44c00640 1 --2- 172.21.15.27:0/640120558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a401023a0 0x7f2a401027b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:43.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.756+0000 7f2a44c00640 1 -- 172.21.15.27:0/640120558 >> 172.21.15.27:0/640120558 conn(0x7f2a400fe240 msgr2=0x7f2a40100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:43.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.756+0000 7f2a44c00640 1 -- 172.21.15.27:0/640120558 shutdown_connections 2024-09-21T00:35:43.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.756+0000 7f2a44c00640 1 -- 172.21.15.27:0/640120558 wait complete. 2024-09-21T00:35:43.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.758+0000 7f2a44c00640 1 Processor -- start 2024-09-21T00:35:43.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.758+0000 7f2a44c00640 1 -- start start 2024-09-21T00:35:43.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.759+0000 7f2a44c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a401023a0 0x7f2a401a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:43.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.759+0000 7f2a44c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a40102cf0 0x7f2a401a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:43.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.759+0000 7f2a44c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a4010ff50 0x7f2a401a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:43.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.759+0000 7f2a44c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a40115a40 con 0x7f2a401023a0 2024-09-21T00:35:43.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.759+0000 7f2a44c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401158c0 con 0x7f2a40102cf0 2024-09-21T00:35:43.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.759+0000 7f2a44c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a40115bc0 con 0x7f2a4010ff50 2024-09-21T00:35:43.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.759+0000 7f2a3ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a40102cf0 0x7f2a401a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:43.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.759+0000 7f2a3fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a4010ff50 0x7f2a401a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:43.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.759+0000 7f2a3ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a40102cf0 0x7f2a401a1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:51408/0 (socket says 172.21.15.27:51408) 2024-09-21T00:35:43.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.759+0000 7f2a3ea00640 1 -- 172.21.15.27:0/4209162044 learned_addr learned my addr 172.21.15.27:0/4209162044 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:43.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.759+0000 7f2a3f400640 1 --2- 172.21.15.27:0/4209162044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a401023a0 0x7f2a401a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:43.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.760+0000 7f2a3fe00640 1 -- 172.21.15.27:0/4209162044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a40102cf0 msgr2=0x7f2a401a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:43.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.760+0000 7f2a3fe00640 1 --2- 172.21.15.27:0/4209162044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a40102cf0 0x7f2a401a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:43.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.760+0000 7f2a3fe00640 1 -- 172.21.15.27:0/4209162044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a401023a0 msgr2=0x7f2a401a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:43.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.760+0000 7f2a3fe00640 1 --2- 172.21.15.27:0/4209162044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a401023a0 0x7f2a401a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:43.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.760+0000 7f2a3fe00640 1 -- 172.21.15.27:0/4209162044 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4010e870 con 0x7f2a4010ff50 2024-09-21T00:35:43.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.761+0000 7f2a3fe00640 1 --2- 172.21.15.27:0/4209162044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a4010ff50 0x7f2a401a2000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800cab0 tx=0x7f2a3800cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:43.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.762+0000 7f2a37e00640 1 -- 172.21.15.27:0/4209162044 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3800ae20 con 0x7f2a4010ff50 2024-09-21T00:35:43.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.762+0000 7f2a37e00640 1 -- 172.21.15.27:0/4209162044 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a3800f900 con 0x7f2a4010ff50 2024-09-21T00:35:43.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.762+0000 7f2a37e00640 1 -- 172.21.15.27:0/4209162044 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380189f0 con 0x7f2a4010ff50 2024-09-21T00:35:43.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.762+0000 7f2a44c00640 1 -- 172.21.15.27:0/4209162044 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4010eb60 con 0x7f2a4010ff50 2024-09-21T00:35:43.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.762+0000 7f2a44c00640 1 -- 172.21.15.27:0/4209162044 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a4010f060 con 0x7f2a4010ff50 2024-09-21T00:35:43.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.764+0000 7f2a44c00640 1 -- 172.21.15.27:0/4209162044 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a401027b0 con 0x7f2a4010ff50 2024-09-21T00:35:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.768+0000 7f2a37e00640 1 -- 172.21.15.27:0/4209162044 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2a38018bd0 con 0x7f2a4010ff50 2024-09-21T00:35:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.771+0000 7f2a37e00640 1 --2- 172.21.15.27:0/4209162044 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2a240b6800 0x7f2a240b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.772+0000 7f2a37e00640 1 -- 172.21.15.27:0/4209162044 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7f2a380e7e90 con 0x7f2a4010ff50 2024-09-21T00:35:43.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.773+0000 7f2a3f400640 1 --2- 172.21.15.27:0/4209162044 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2a240b6800 0x7f2a240b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.774+0000 7f2a3f400640 1 --2- 172.21.15.27:0/4209162044 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2a240b6800 0x7f2a240b8d10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009850 tx=0x7f2a30005fd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:43.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:43.779+0000 7f2a37e00640 1 -- 172.21.15.27:0/4209162044 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2a38014030 con 0x7f2a4010ff50 2024-09-21T00:35:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:44 smithi027 ceph-mon[30745]: pgmap v827: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 173 op/s 2024-09-21T00:35:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:44 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/746457524' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T00:35:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:44 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:35:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:44 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:35:44.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:44 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:35:44.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.178+0000 7f2a44c00640 1 -- 172.21.15.27:0/4209162044 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2a4010f500 con 0x7f2a4010ff50 2024-09-21T00:35:44.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.178+0000 7f2a37e00640 1 -- 172.21.15.27:0/4209162044 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v110) ==== 103+0+11 (secure 0 0 0) 0x7f2a380adf60 con 0x7f2a4010ff50 2024-09-21T00:35:44.179 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-09-21T00:35:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.183+0000 7f2a44c00640 1 -- 172.21.15.27:0/4209162044 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2a240b6800 msgr2=0x7f2a240b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.183+0000 7f2a44c00640 1 --2- 172.21.15.27:0/4209162044 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2a240b6800 0x7f2a240b8d10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009850 tx=0x7f2a30005fd0 comp rx=0 tx=0).stop 2024-09-21T00:35:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.183+0000 7f2a44c00640 1 -- 172.21.15.27:0/4209162044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a4010ff50 msgr2=0x7f2a401a2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.183+0000 7f2a44c00640 1 --2- 172.21.15.27:0/4209162044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a4010ff50 0x7f2a401a2000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800cab0 tx=0x7f2a3800cf70 comp rx=0 tx=0).stop 2024-09-21T00:35:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.183+0000 7f2a44c00640 1 -- 172.21.15.27:0/4209162044 shutdown_connections 2024-09-21T00:35:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.183+0000 7f2a44c00640 1 --2- 172.21.15.27:0/4209162044 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f2a240b6800 0x7f2a240b8d10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.183+0000 7f2a44c00640 1 --2- 172.21.15.27:0/4209162044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a4010ff50 0x7f2a401a2000 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.183+0000 7f2a44c00640 1 --2- 172.21.15.27:0/4209162044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a40102cf0 0x7f2a401a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.183+0000 7f2a44c00640 1 --2- 172.21.15.27:0/4209162044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a401023a0 0x7f2a401a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:44.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.183+0000 7f2a44c00640 1 -- 172.21.15.27:0/4209162044 >> 172.21.15.27:0/4209162044 conn(0x7f2a400fe240 msgr2=0x7f2a401056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:44.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.183+0000 7f2a44c00640 1 -- 172.21.15.27:0/4209162044 shutdown_connections 2024-09-21T00:35:44.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.183+0000 7f2a44c00640 1 -- 172.21.15.27:0/4209162044 wait complete. 2024-09-21T00:35:44.234 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T00:35:44.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:44 smithi089 ceph-mon[34481]: pgmap v827: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 173 op/s 2024-09-21T00:35:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:44 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/746457524' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T00:35:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:44 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:35:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:44 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:35:44.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:44 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:35:44.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:44 smithi098 ceph-mon[34725]: pgmap v827: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 173 op/s 2024-09-21T00:35:44.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:44 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/746457524' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T00:35:44.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:44 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:35:44.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:44 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:35:44.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:44 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:35:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.639+0000 7fd025c00640 1 -- 172.21.15.27:0/1922395683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0201080f0 msgr2=0x7fd020108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.639+0000 7fd025c00640 1 --2- 172.21.15.27:0/1922395683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0201080f0 0x7fd020108500 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009900 tx=0x7fd00c02a3f0 comp rx=0 tx=0).stop 2024-09-21T00:35:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.639+0000 7fd025c00640 1 -- 172.21.15.27:0/1922395683 shutdown_connections 2024-09-21T00:35:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.639+0000 7fd025c00640 1 --2- 172.21.15.27:0/1922395683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd020108a40 0x7fd02010f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.639+0000 7fd025c00640 1 --2- 172.21.15.27:0/1922395683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0201080f0 0x7fd020108500 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.639+0000 7fd025c00640 1 --2- 172.21.15.27:0/1922395683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201027c0 0x7fd020102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.639+0000 7fd025c00640 1 -- 172.21.15.27:0/1922395683 >> 172.21.15.27:0/1922395683 conn(0x7fd0200fe1e0 msgr2=0x7fd020100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.640+0000 7fd025c00640 1 -- 172.21.15.27:0/1922395683 shutdown_connections 2024-09-21T00:35:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.640+0000 7fd025c00640 1 -- 172.21.15.27:0/1922395683 wait complete. 2024-09-21T00:35:44.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.642+0000 7fd025c00640 1 Processor -- start 2024-09-21T00:35:44.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.642+0000 7fd025c00640 1 -- start start 2024-09-21T00:35:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.643+0000 7fd025c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0201027c0 0x7fd02019c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.643+0000 7fd025c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201080f0 0x7fd02019ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.643+0000 7fd025c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020108a40 0x7fd0201a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.643+0000 7fd025c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd020111600 con 0x7fd0201080f0 2024-09-21T00:35:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.643+0000 7fd025c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd020111480 con 0x7fd020108a40 2024-09-21T00:35:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.643+0000 7fd025c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd020111780 con 0x7fd0201027c0 2024-09-21T00:35:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.643+0000 7fd025200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020108a40 0x7fd0201a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.643+0000 7fd025200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020108a40 0x7fd0201a1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:51434/0 (socket says 172.21.15.27:51434) 2024-09-21T00:35:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.643+0000 7fd025200640 1 -- 172.21.15.27:0/1107284538 learned_addr learned my addr 172.21.15.27:0/1107284538 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:35:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.643+0000 7fd01f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201080f0 0x7fd02019ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.643+0000 7fd01fe00640 1 --2- 172.21.15.27:0/1107284538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0201027c0 0x7fd02019c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:44.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.645+0000 7fd025200640 1 -- 172.21.15.27:0/1107284538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0201027c0 msgr2=0x7fd02019c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:44.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.645+0000 7fd025200640 1 --2- 172.21.15.27:0/1107284538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0201027c0 0x7fd02019c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:44.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.645+0000 7fd025200640 1 -- 172.21.15.27:0/1107284538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201080f0 msgr2=0x7fd02019ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:44.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.645+0000 7fd025200640 1 --2- 172.21.15.27:0/1107284538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201080f0 0x7fd02019ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:44.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.645+0000 7fd025200640 1 -- 172.21.15.27:0/1107284538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0201a23a0 con 0x7fd020108a40 2024-09-21T00:35:44.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.646+0000 7fd025200640 1 --2- 172.21.15.27:0/1107284538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020108a40 0x7fd0201a1ca0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd018009db0 tx=0x7fd01800e490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:44.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.646+0000 7fd01cc00640 1 -- 172.21.15.27:0/1107284538 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01800b2d0 con 0x7fd020108a40 2024-09-21T00:35:44.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.646+0000 7fd01cc00640 1 -- 172.21.15.27:0/1107284538 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd018003e20 con 0x7fd020108a40 2024-09-21T00:35:44.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.647+0000 7fd01cc00640 1 -- 172.21.15.27:0/1107284538 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0180047d0 con 0x7fd020108a40 2024-09-21T00:35:44.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.647+0000 7fd025c00640 1 -- 172.21.15.27:0/1107284538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0201a2690 con 0x7fd020108a40 2024-09-21T00:35:44.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.647+0000 7fd025c00640 1 -- 172.21.15.27:0/1107284538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0201a8e20 con 0x7fd020108a40 2024-09-21T00:35:44.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.650+0000 7fd025c00640 1 -- 172.21.15.27:0/1107284538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0000052c0 con 0x7fd020108a40 2024-09-21T00:35:44.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.655+0000 7fd01cc00640 1 -- 172.21.15.27:0/1107284538 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fd01801b020 con 0x7fd020108a40 2024-09-21T00:35:44.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.658+0000 7fd01cc00640 1 --2- 172.21.15.27:0/1107284538 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd0040b6700 0x7fd0040b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:35:44.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.658+0000 7fd01cc00640 1 -- 172.21.15.27:0/1107284538 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9671+0+0 (secure 0 0 0) 0x7fd0180e8090 con 0x7fd020108a40 2024-09-21T00:35:44.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.658+0000 7fd01fe00640 1 --2- 172.21.15.27:0/1107284538 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd0040b6700 0x7fd0040b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:35:44.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.661+0000 7fd01fe00640 1 --2- 172.21.15.27:0/1107284538 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd0040b6700 0x7fd0040b8c10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd0100058c0 tx=0x7fd01000a4c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:35:44.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:44.663+0000 7fd01cc00640 1 -- 172.21.15.27:0/1107284538 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd0180ae100 con 0x7fd020108a40 2024-09-21T00:35:45.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:45.172+0000 7fd025c00640 1 -- 172.21.15.27:0/1107284538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fd000005880 con 0x7fd020108a40 2024-09-21T00:35:45.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:45.184+0000 7fd01cc00640 1 -- 172.21.15.27:0/1107284538 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) ==== 81+0+834053 (secure 0 0 0) 0x7fd0180b24e0 con 0x7fd020108a40 2024-09-21T00:35:45.189 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:35:45.189 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:35:45.189 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 24, 2024-09-21T00:35:45.189 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.189 INFO:teuthology.orchestra.run.smithi027.stdout: "active_gid": 14226, 2024-09-21T00:35:45.189 INFO:teuthology.orchestra.run.smithi027.stdout: "active_name": "x", 2024-09-21T00:35:45.189 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addrs": { 2024-09-21T00:35:45.189 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T00:35:45.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.189 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T00:35:45.189 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-09-21T00:35:45.190 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2382590111 2024-09-21T00:35:45.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.190 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-21T00:35:45.190 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-09-21T00:35:45.190 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2382590111 2024-09-21T00:35:45.190 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.190 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T00:35:45.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.190 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addr": "172.21.15.27:6801/2382590111", 2024-09-21T00:35:45.190 INFO:teuthology.orchestra.run.smithi027.stdout: "active_change": "2024-09-21T00:11:24.231237+0000", 2024-09-21T00:35:45.190 INFO:teuthology.orchestra.run.smithi027.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T00:35:45.190 INFO:teuthology.orchestra.run.smithi027.stdout: "available": true, 2024-09-21T00:35:45.191 INFO:teuthology.orchestra.run.smithi027.stdout: "standbys": [ 2024-09-21T00:35:45.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.191 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 24166, 2024-09-21T00:35:45.191 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "z", 2024-09-21T00:35:45.191 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_features": 4540719139924082687, 2024-09-21T00:35:45.191 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-09-21T00:35:45.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.191 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-09-21T00:35:45.191 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.191 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.191 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.191 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T00:35:45.191 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T00:35:45.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.192 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.192 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.192 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T00:35:45.192 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.192 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.192 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.192 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T00:35:45.192 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.192 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.192 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.192 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.192 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.193 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.193 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.193 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.193 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.193 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.193 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.193 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.193 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.193 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.193 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.193 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.193 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.193 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.193 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.194 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.194 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.194 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.194 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.194 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.194 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.194 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.194 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.194 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.195 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.195 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.195 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.195 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.195 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.195 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.195 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.195 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.195 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.195 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.195 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.195 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.196 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.196 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.196 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.196 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.196 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.196 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.196 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.196 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.196 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.196 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.196 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.196 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.196 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.196 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.198 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.198 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-09-21T00:35:45.198 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-09-21T00:35:45.198 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.198 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.198 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.198 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.198 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.198 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.198 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.198 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T00:35:45.198 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.198 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-09-21T00:35:45.200 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-09-21T00:35:45.201 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.201 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.201 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.201 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.201 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.201 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.201 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.201 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-09-21T00:35:45.201 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.201 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.201 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.201 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-09-21T00:35:45.201 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-09-21T00:35:45.201 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.202 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.203 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.203 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.203 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.203 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.203 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-09-21T00:35:45.203 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.203 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.203 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.203 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-09-21T00:35:45.203 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-09-21T00:35:45.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.203 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.204 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.205 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.206 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.206 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.206 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.206 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.206 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.206 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.206 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.206 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-09-21T00:35:45.206 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.206 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.206 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.206 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-09-21T00:35:45.207 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-09-21T00:35:45.207 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.207 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.207 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.207 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.207 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.207 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.207 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.207 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T00:35:45.207 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.207 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.207 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.207 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-09-21T00:35:45.207 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T00:35:45.208 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-09-21T00:35:45.209 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T00:35:45.210 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-09-21T00:35:45.210 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.210 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T00:35:45.210 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.210 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.210 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.210 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-09-21T00:35:45.210 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-09-21T00:35:45.210 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.210 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.210 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.210 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T00:35:45.211 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.211 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.211 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.211 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T00:35:45.211 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T00:35:45.211 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.211 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.211 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-09-21T00:35:45.211 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-09-21T00:35:45.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:45.211 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.211 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.211 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-09-21T00:35:45.212 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-09-21T00:35:45.212 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-09-21T00:35:45.212 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.212 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-09-21T00:35:45.212 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T00:35:45.212 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.212 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.212 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-09-21T00:35:45.212 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-09-21T00:35:45.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.212 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.212 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.212 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-09-21T00:35:45.213 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.213 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.213 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.213 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T00:35:45.213 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T00:35:45.213 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.213 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.213 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-09-21T00:35:45.213 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-09-21T00:35:45.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T00:35:45.213 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.213 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.213 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.214 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T00:35:45.214 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T00:35:45.214 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.214 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T00:35:45.214 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T00:35:45.214 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.214 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.214 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.214 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.214 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.214 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.214 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.214 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.215 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.215 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.215 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.215 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.215 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.215 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.215 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.215 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.215 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.215 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.215 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.215 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.215 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.215 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.216 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.216 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.216 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.216 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.216 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.216 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.216 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.216 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.216 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.216 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.216 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.216 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.216 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.217 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.217 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.217 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.217 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.217 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.217 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.217 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.217 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.217 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.217 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.217 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.217 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.218 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.218 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.218 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.218 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.218 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.218 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.218 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.218 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.218 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.218 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.219 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.219 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.219 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.219 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.219 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.219 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.219 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.219 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.219 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.220 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-09-21T00:35:45.220 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-09-21T00:35:45.220 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:45.220 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.220 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.220 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.220 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.220 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.220 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.220 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T00:35:45.220 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.220 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.220 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.221 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T00:35:45.221 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T00:35:45.221 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.221 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.221 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.221 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-09-21T00:35:45.221 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.221 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.221 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.221 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-09-21T00:35:45.221 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-09-21T00:35:45.221 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-09-21T00:35:45.221 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-09-21T00:35:45.221 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-09-21T00:35:45.222 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.222 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-09-21T00:35:45.222 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.222 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.222 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.222 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-09-21T00:35:45.222 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-09-21T00:35:45.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.222 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.222 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.222 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.222 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.223 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.223 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.223 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T00:35:45.223 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.223 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.223 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.223 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T00:35:45.223 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T00:35:45.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.223 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.223 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.224 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T00:35:45.224 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.224 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.224 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.224 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T00:35:45.224 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.224 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.224 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.224 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.224 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.224 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.225 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.225 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.225 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.225 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.225 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.225 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.225 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.225 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.225 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.225 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-09-21T00:35:45.225 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-09-21T00:35:45.225 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.226 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.226 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.226 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T00:35:45.226 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T00:35:45.226 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.226 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.226 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T00:35:45.226 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T00:35:45.226 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.226 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.226 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-09-21T00:35:45.226 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-09-21T00:35:45.226 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T00:35:45.227 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.227 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.227 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:45.227 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.227 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.227 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.227 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T00:35:45.227 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.227 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.227 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.227 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.227 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.227 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-09-21T00:35:45.228 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.228 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.228 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.228 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-09-21T00:35:45.228 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-09-21T00:35:45.228 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:45.228 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.228 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.228 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T00:35:45.228 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.228 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.228 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.228 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T00:35:45.228 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.229 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.229 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.229 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-09-21T00:35:45.229 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-09-21T00:35:45.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.229 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.229 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-09-21T00:35:45.229 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.229 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.229 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.230 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T00:35:45.230 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.230 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.230 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.230 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-09-21T00:35:45.230 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-09-21T00:35:45.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.230 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.230 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.230 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-09-21T00:35:45.230 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.230 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.230 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.231 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T00:35:45.231 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.231 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.231 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.231 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-09-21T00:35:45.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-09-21T00:35:45.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.231 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.231 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.231 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.231 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.232 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.232 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.232 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T00:35:45.232 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T00:35:45.232 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.232 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.232 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-09-21T00:35:45.232 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-09-21T00:35:45.232 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.232 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.232 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.232 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T00:35:45.232 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.233 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.233 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.233 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T00:35:45.233 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.233 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.233 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.233 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-09-21T00:35:45.233 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T00:35:45.233 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:45.233 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.233 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.234 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-09-21T00:35:45.234 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.234 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.234 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.234 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T00:35:45.234 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.234 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.234 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.234 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-09-21T00:35:45.234 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-09-21T00:35:45.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.234 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.235 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.235 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.235 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.235 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.235 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.235 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-09-21T00:35:45.235 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-09-21T00:35:45.235 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-09-21T00:35:45.235 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.235 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T00:35:45.235 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.235 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.235 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.236 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-09-21T00:35:45.236 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-09-21T00:35:45.236 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.236 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.236 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.236 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.236 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.236 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.236 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.237 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T00:35:45.237 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.237 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.237 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.237 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-09-21T00:35:45.237 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-09-21T00:35:45.237 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.237 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.237 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.237 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.237 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.237 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.237 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.237 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T00:35:45.238 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.238 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.238 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.238 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-09-21T00:35:45.238 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-09-21T00:35:45.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.238 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.238 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.238 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.238 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.238 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.238 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.238 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T00:35:45.239 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.239 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.239 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.239 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-09-21T00:35:45.239 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-09-21T00:35:45.239 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.239 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.239 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.239 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T00:35:45.239 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.239 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.239 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.239 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T00:35:45.239 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.240 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.240 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.240 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-09-21T00:35:45.240 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-09-21T00:35:45.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.240 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.240 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.240 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T00:35:45.240 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.240 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.240 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.240 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-09-21T00:35:45.241 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.241 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.241 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.241 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-09-21T00:35:45.241 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-09-21T00:35:45.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.241 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.241 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.241 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T00:35:45.241 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.241 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.241 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.241 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-09-21T00:35:45.241 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.242 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.242 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.242 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-09-21T00:35:45.242 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-09-21T00:35:45.242 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.242 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.242 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.242 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T00:35:45.242 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.242 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.242 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.242 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T00:35:45.243 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.243 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.243 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.243 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-09-21T00:35:45.243 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-09-21T00:35:45.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.243 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.243 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T00:35:45.243 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.243 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.243 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.243 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-09-21T00:35:45.244 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.244 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.244 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.244 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-09-21T00:35:45.244 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-09-21T00:35:45.244 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.244 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.244 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.244 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T00:35:45.244 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.244 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.244 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.244 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-09-21T00:35:45.245 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.245 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.245 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.245 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-09-21T00:35:45.245 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-09-21T00:35:45.245 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.245 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.245 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.245 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T00:35:45.245 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.245 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.245 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.245 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-09-21T00:35:45.246 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.246 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.246 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.246 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-09-21T00:35:45.246 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-09-21T00:35:45.246 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.246 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.246 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T00:35:45.246 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.246 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.246 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.247 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-09-21T00:35:45.247 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.247 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.247 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.247 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-09-21T00:35:45.247 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-09-21T00:35:45.247 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.247 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.247 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.247 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T00:35:45.247 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.247 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.247 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.248 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-09-21T00:35:45.248 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.248 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.248 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.248 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-09-21T00:35:45.248 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-09-21T00:35:45.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.248 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.248 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.248 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T00:35:45.248 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.248 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.248 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.249 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-09-21T00:35:45.249 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.249 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.249 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.249 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nginx": { 2024-09-21T00:35:45.249 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nginx", 2024-09-21T00:35:45.249 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.249 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.249 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.249 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T00:35:45.249 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.249 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.249 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.250 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nginx container image", 2024-09-21T00:35:45.250 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.250 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.250 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.250 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-09-21T00:35:45.250 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-09-21T00:35:45.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.250 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.250 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.250 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T00:35:45.250 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.250 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.250 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.250 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T00:35:45.251 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.251 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.251 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.251 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-09-21T00:35:45.251 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-09-21T00:35:45.251 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.251 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.251 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T00:35:45.251 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.251 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.251 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.251 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-09-21T00:35:45.251 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.252 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.252 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.252 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_oauth2_proxy": { 2024-09-21T00:35:45.252 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T00:35:45.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.252 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.252 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.252 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T00:35:45.252 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.252 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.252 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.252 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "oauth2-proxy container image", 2024-09-21T00:35:45.252 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.252 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.254 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-09-21T00:35:45.254 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-09-21T00:35:45.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.254 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.254 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.254 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T00:35:45.254 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.254 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.254 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.254 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-09-21T00:35:45.254 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.254 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.254 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-09-21T00:35:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-09-21T00:35:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T00:35:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-09-21T00:35:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba_metrics": { 2024-09-21T00:35:45.256 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba_metrics", 2024-09-21T00:35:45.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.256 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.256 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T00:35:45.256 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.256 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.256 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.256 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T00:35:45.256 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.256 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.256 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.256 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-09-21T00:35:45.256 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-09-21T00:35:45.257 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.257 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.257 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.257 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T00:35:45.257 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.257 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.257 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.257 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-09-21T00:35:45.257 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.257 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.257 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.257 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-09-21T00:35:45.257 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-09-21T00:35:45.258 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.258 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.258 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.258 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.258 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.258 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.258 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.258 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T00:35:45.258 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.258 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.258 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.258 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-09-21T00:35:45.258 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-09-21T00:35:45.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.259 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.259 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T00:35:45.259 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.259 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.259 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.259 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T00:35:45.259 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.259 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.259 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.259 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-09-21T00:35:45.259 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T00:35:45.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.260 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.260 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-09-21T00:35:45.260 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.260 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.260 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.260 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T00:35:45.260 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.260 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.260 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.260 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-09-21T00:35:45.260 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-09-21T00:35:45.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.260 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-09-21T00:35:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T00:35:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-09-21T00:35:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-09-21T00:35:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.262 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-09-21T00:35:45.262 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.262 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.262 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.262 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-09-21T00:35:45.262 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.262 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.262 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.262 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-09-21T00:35:45.262 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-09-21T00:35:45.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.262 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.262 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.263 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.263 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.263 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T00:35:45.263 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.263 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.263 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.263 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-09-21T00:35:45.263 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-09-21T00:35:45.263 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.263 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.263 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.263 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T00:35:45.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.264 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.264 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.264 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-09-21T00:35:45.264 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.264 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.264 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.264 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-09-21T00:35:45.264 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-09-21T00:35:45.264 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.264 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.264 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.264 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T00:35:45.265 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.265 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.265 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.265 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T00:35:45.265 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.265 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.265 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.265 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-09-21T00:35:45.265 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-09-21T00:35:45.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.265 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.266 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T00:35:45.266 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.266 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.266 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.266 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-09-21T00:35:45.266 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.266 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.266 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.266 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-09-21T00:35:45.266 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-09-21T00:35:45.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.266 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.266 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T00:35:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-09-21T00:35:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-09-21T00:35:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.268 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-09-21T00:35:45.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.270 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.270 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.270 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.270 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.270 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.270 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.270 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T00:35:45.270 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.270 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.270 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.270 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.270 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.270 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.271 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.271 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.271 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.271 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.271 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.271 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.271 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T00:35:45.271 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.271 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.271 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.271 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.271 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.272 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T00:35:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T00:35:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T00:35:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T00:35:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T00:35:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-09-21T00:35:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T00:35:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-09-21T00:35:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-09-21T00:35:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-09-21T00:35:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-09-21T00:35:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-09-21T00:35:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-09-21T00:35:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T00:35:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T00:35:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-09-21T00:35:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-09-21T00:35:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-09-21T00:35:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.280 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T00:35:45.280 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.280 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.280 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.280 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-09-21T00:35:45.280 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-09-21T00:35:45.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.280 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.280 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.280 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-09-21T00:35:45.280 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.280 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.280 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T00:35:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-09-21T00:35:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-09-21T00:35:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T00:35:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.282 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T00:35:45.282 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.282 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.282 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.282 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-09-21T00:35:45.282 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-09-21T00:35:45.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.282 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.282 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.282 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.282 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.282 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.282 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.283 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T00:35:45.283 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.283 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.283 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-09-21T00:35:45.283 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-09-21T00:35:45.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.283 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.283 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.283 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.283 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.283 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.284 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T00:35:45.284 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.284 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.284 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.284 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-09-21T00:35:45.284 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-09-21T00:35:45.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.284 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.284 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.284 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.284 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.284 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.284 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.284 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T00:35:45.285 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.285 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.285 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.285 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-09-21T00:35:45.285 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-09-21T00:35:45.285 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.285 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.285 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.285 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.285 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.285 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.285 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T00:35:45.285 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.286 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.286 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.286 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-09-21T00:35:45.286 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-09-21T00:35:45.286 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.286 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.286 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.286 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.286 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.286 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.286 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.286 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T00:35:45.286 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.286 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.287 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.287 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-09-21T00:35:45.287 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-09-21T00:35:45.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.287 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.287 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-09-21T00:35:45.287 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.287 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.287 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.287 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-09-21T00:35:45.287 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.288 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.288 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.288 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.288 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.288 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.288 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.288 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.288 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.288 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.288 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.288 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.288 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.288 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.288 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.289 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-09-21T00:35:45.289 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-09-21T00:35:45.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.289 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.289 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.289 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.289 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.289 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.289 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T00:35:45.289 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.289 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.290 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.290 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_count_max": { 2024-09-21T00:35:45.290 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T00:35:45.290 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.290 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.290 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.290 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T00:35:45.290 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.290 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.290 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.290 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T00:35:45.290 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.290 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_interval": { 2024-09-21T00:35:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_interval", 2024-09-21T00:35:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T00:35:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T00:35:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.292 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-09-21T00:35:45.292 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-09-21T00:35:45.292 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.292 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.292 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.292 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.292 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.292 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.292 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.292 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T00:35:45.292 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.292 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-09-21T00:35:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-09-21T00:35:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T00:35:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.294 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-09-21T00:35:45.294 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-09-21T00:35:45.294 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.294 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.294 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.294 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.294 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.294 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.294 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T00:35:45.294 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.294 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.294 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.295 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-09-21T00:35:45.295 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-09-21T00:35:45.295 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.295 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.295 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.295 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.295 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.295 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.295 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.295 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T00:35:45.295 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.295 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.296 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.296 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-09-21T00:35:45.296 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-09-21T00:35:45.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.296 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.296 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.296 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.296 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.296 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.296 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T00:35:45.296 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.296 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.297 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.297 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.297 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.297 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-09-21T00:35:45.297 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.297 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.297 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.297 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.297 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.297 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.297 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.297 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.298 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.298 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.298 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.298 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.298 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.298 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.298 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.298 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.298 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.298 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.298 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.298 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.298 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.299 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.299 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.299 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.299 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.299 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.299 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.299 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.299 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.299 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.299 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.299 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.300 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.300 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.300 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.300 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.300 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.300 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.300 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.300 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.300 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.300 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.300 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.300 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.301 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.301 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.301 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.301 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.301 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.301 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.301 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.301 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.301 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.301 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-09-21T00:35:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-09-21T00:35:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.303 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.303 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-09-21T00:35:45.303 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.303 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.303 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.303 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T00:35:45.303 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.303 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.303 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.303 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.303 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.304 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.304 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.304 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.304 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.304 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.304 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.304 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.304 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.304 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.304 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.304 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-09-21T00:35:45.304 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-09-21T00:35:45.304 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.304 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.305 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.305 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-09-21T00:35:45.305 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.305 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.305 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.305 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T00:35:45.305 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.305 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.305 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.305 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.305 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.306 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-09-21T00:35:45.306 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.306 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.306 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.306 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T00:35:45.306 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T00:35:45.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.306 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.306 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.306 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:45.306 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.306 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.306 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.306 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.307 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.307 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.307 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.307 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T00:35:45.307 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T00:35:45.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.307 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.307 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.307 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.307 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.307 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.307 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.307 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.308 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.308 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T00:35:45.308 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T00:35:45.308 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.308 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.308 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.308 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.308 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.308 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.308 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.308 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.309 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.309 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.309 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-09-21T00:35:45.309 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T00:35:45.309 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.309 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.309 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.309 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.309 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.309 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.309 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.309 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.310 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.310 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.310 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.310 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T00:35:45.310 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T00:35:45.310 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.310 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.310 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.310 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.310 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.310 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.310 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.310 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.310 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.311 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.312 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T00:35:45.312 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T00:35:45.312 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.312 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.312 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.312 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.312 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.312 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.312 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.312 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.312 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.313 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T00:35:45.320 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T00:35:45.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.320 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.321 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.322 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.322 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.322 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.322 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.322 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.322 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.322 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.322 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T00:35:45.322 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T00:35:45.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.322 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.323 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.323 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.323 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.323 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.323 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.323 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.323 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.323 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T00:35:45.323 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T00:35:45.323 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.323 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T00:35:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T00:35:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.324 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.325 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.326 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.326 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.326 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.326 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.326 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.326 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T00:35:45.326 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T00:35:45.326 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.326 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.326 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.326 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.326 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.327 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.327 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.327 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.327 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.327 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T00:35:45.327 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T00:35:45.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.327 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.327 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.327 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T00:35:45.327 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.327 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.327 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T00:35:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T00:35:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.329 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.330 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.330 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T00:35:45.330 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T00:35:45.330 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.330 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.330 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.330 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T00:35:45.330 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.330 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.330 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.330 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.330 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.330 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T00:35:45.332 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T00:35:45.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.333 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.333 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.333 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.333 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.333 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.333 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.333 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.333 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.333 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.333 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.333 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T00:35:45.333 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T00:35:45.334 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.334 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.334 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.334 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.334 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.334 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.334 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.334 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.334 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.334 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.334 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.334 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T00:35:45.334 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T00:35:45.334 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.335 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.335 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.335 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-09-21T00:35:45.335 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.335 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.335 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.335 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.335 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.335 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.335 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.335 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T00:35:45.335 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T00:35:45.335 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.336 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.336 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.336 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-09-21T00:35:45.336 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.336 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.336 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.336 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.336 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.336 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.336 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.336 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T00:35:45.336 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T00:35:45.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.336 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.337 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.337 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.337 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.337 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.337 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.337 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.337 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.337 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.337 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.337 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T00:35:45.337 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T00:35:45.337 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.337 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.337 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.338 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.338 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.338 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.338 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.338 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.338 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.338 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.338 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.338 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T00:35:45.338 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T00:35:45.338 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.338 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.338 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.338 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.339 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.339 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.339 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.339 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.339 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.339 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.339 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.339 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T00:35:45.339 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T00:35:45.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.339 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.340 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.340 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.340 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.340 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.340 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.340 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.340 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.340 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.340 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T00:35:45.340 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T00:35:45.340 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.340 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T00:35:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T00:35:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.342 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.342 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.342 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.342 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.342 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.342 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.342 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.342 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.342 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T00:35:45.342 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T00:35:45.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.342 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.343 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.343 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.343 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.343 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.343 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.343 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.343 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.343 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.343 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T00:35:45.343 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T00:35:45.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.343 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.344 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.344 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.344 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.344 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.344 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.344 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.344 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.344 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.344 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.344 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T00:35:45.344 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T00:35:45.344 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.344 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.345 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.345 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.345 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.345 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.345 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.345 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.345 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.345 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.345 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.345 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T00:35:45.345 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T00:35:45.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.346 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.346 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.346 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.346 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.346 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.346 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.346 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.346 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.346 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.346 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.346 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T00:35:45.346 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T00:35:45.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.347 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.347 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.347 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T00:35:45.347 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.347 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.347 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.347 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.347 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.347 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.347 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.347 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T00:35:45.347 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T00:35:45.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.348 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.348 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.348 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:45.348 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.348 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.348 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.348 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.348 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.348 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.348 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.348 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T00:35:45.348 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T00:35:45.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.349 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.349 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.349 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-09-21T00:35:45.349 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.349 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.349 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.349 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.349 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.349 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.349 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.349 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T00:35:45.350 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T00:35:45.350 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.350 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.350 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.350 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-09-21T00:35:45.350 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.350 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.350 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.350 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.350 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.350 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.350 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.350 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T00:35:45.351 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T00:35:45.351 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.351 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.351 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.351 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.351 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.351 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.351 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.351 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.351 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.351 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.351 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.351 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T00:35:45.352 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T00:35:45.352 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.352 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.352 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.352 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T00:35:45.352 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.352 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.352 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.352 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.352 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.352 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.352 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.353 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T00:35:45.353 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T00:35:45.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.353 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.353 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.353 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.353 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.353 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.353 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.353 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.353 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.353 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.353 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T00:35:45.354 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T00:35:45.354 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.354 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.354 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.354 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.354 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.354 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.354 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.354 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.354 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.354 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.354 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.354 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T00:35:45.354 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T00:35:45.355 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.355 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.355 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.355 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.355 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.355 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.355 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.355 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.355 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.355 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.355 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.355 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T00:35:45.356 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T00:35:45.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.356 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.356 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.356 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.356 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.356 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.356 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.356 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.356 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.356 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.356 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T00:35:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T00:35:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.358 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T00:35:45.358 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T00:35:45.358 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.358 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.358 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T00:35:45.358 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.358 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.358 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.358 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.358 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.358 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.358 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.359 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-09-21T00:35:45.359 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-09-21T00:35:45.359 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.359 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.359 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.359 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.359 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.359 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.359 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.359 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.359 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.359 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.360 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-09-21T00:35:45.360 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-09-21T00:35:45.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.360 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.360 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.360 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.360 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.360 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.360 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.360 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.360 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.360 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.361 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-09-21T00:35:45.361 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-09-21T00:35:45.361 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.361 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.361 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.361 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.361 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.361 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.361 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.361 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-09-21T00:35:45.361 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.361 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.361 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-09-21T00:35:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-09-21T00:35:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-09-21T00:35:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.363 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T00:35:45.363 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T00:35:45.363 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.363 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.363 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.363 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.363 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.363 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.363 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.363 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.363 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.363 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.363 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.364 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.365 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.365 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.365 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.365 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.365 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.365 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.365 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.365 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.365 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.365 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.365 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.365 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.365 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.366 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.366 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.366 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.366 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.366 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.366 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.366 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.366 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.366 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.366 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.366 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.366 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.366 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.367 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.367 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.367 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.367 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.367 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.367 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.367 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.367 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.367 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.367 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.367 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.367 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.368 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.369 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.369 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.369 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-09-21T00:35:45.369 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.369 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.369 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.369 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-09-21T00:35:45.369 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T00:35:45.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.369 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.369 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.369 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.370 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.370 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.370 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.370 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.370 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.370 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.370 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T00:35:45.370 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T00:35:45.370 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.370 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.370 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.370 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T00:35:45.371 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.371 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.371 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.371 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.371 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.371 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.371 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.371 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T00:35:45.371 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T00:35:45.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.371 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.371 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.371 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-09-21T00:35:45.372 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.372 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.372 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.372 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.372 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.372 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.372 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.372 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.372 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.373 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.373 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.373 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.373 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.373 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.373 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.373 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.373 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.373 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.373 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T00:35:45.373 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T00:35:45.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.373 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.374 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.374 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.374 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.374 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.374 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.374 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.374 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.374 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.374 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-09-21T00:35:45.374 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-09-21T00:35:45.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.374 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.375 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.375 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-09-21T00:35:45.375 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.375 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.375 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.375 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.375 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.375 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.375 INFO:teuthology.orchestra.run.smithi027.stdout: "sso_oauth2": { 2024-09-21T00:35:45.375 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sso_oauth2", 2024-09-21T00:35:45.375 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.376 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.376 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.376 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.376 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.376 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.376 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.376 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.376 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.376 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T00:35:45.376 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T00:35:45.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.377 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.377 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.377 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-09-21T00:35:45.377 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.377 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.377 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.377 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.377 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-09-21T00:35:45.377 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.377 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.377 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.378 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.378 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.378 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T00:35:45.378 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T00:35:45.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.378 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.378 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T00:35:45.378 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T00:35:45.378 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T00:35:45.378 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.379 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.379 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.379 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.379 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-09-21T00:35:45.379 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-09-21T00:35:45.379 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.379 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.379 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.379 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.379 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.379 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.380 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.380 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.380 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.380 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.380 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.380 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.380 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-09-21T00:35:45.380 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.380 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.380 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.381 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-09-21T00:35:45.381 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-09-21T00:35:45.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.381 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.381 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.381 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.381 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.381 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.381 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.381 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-09-21T00:35:45.381 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.381 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.381 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.382 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.382 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.382 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.382 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.382 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.382 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.382 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.382 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.382 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.382 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.383 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.383 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.383 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.383 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.383 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.383 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.383 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.383 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.383 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.383 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.383 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.384 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.384 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.384 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.384 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.384 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.384 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.384 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.384 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.384 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.385 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.385 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.385 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.385 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.385 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.385 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.385 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.385 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.385 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.385 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.385 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.386 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.386 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.386 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.386 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.386 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.386 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.386 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.386 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.386 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.386 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-09-21T00:35:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-09-21T00:35:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-09-21T00:35:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T00:35:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-09-21T00:35:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-09-21T00:35:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-09-21T00:35:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T00:35:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-09-21T00:35:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-09-21T00:35:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-09-21T00:35:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-09-21T00:35:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-09-21T00:35:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-09-21T00:35:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T00:35:45.391 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.391 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.391 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.391 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T00:35:45.391 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.391 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.391 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.391 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-09-21T00:35:45.391 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-09-21T00:35:45.391 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.391 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.391 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.391 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.391 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.392 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.392 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.392 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T00:35:45.392 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.392 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.392 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.392 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T00:35:45.392 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T00:35:45.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.392 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.392 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T00:35:45.392 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.392 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.393 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.393 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T00:35:45.393 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.393 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.393 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.393 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.393 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.393 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.393 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.393 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.393 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.394 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.394 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.394 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.394 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-09-21T00:35:45.394 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-09-21T00:35:45.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.394 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.394 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.394 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-09-21T00:35:45.394 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.394 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.394 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.395 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T00:35:45.395 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.395 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.395 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.395 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.395 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.395 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-09-21T00:35:45.395 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.395 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.395 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.395 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.395 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.397 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.397 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.397 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.397 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.397 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.397 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.397 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.397 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.397 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.397 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.397 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.397 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-09-21T00:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-09-21T00:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T00:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-09-21T00:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-09-21T00:35:45.402 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.402 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.402 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.402 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-09-21T00:35:45.402 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.402 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.402 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.402 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.402 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.402 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.402 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.403 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T00:35:45.403 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T00:35:45.403 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.403 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.403 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.403 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T00:35:45.403 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.403 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.403 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.403 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.403 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.403 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.403 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.404 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.404 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.404 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.404 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.404 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.404 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.404 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.404 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.404 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.404 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.404 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.404 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.405 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.405 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.405 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-09-21T00:35:45.405 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-09-21T00:35:45.405 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-09-21T00:35:45.405 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.405 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-09-21T00:35:45.405 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-09-21T00:35:45.405 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.405 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.405 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.405 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-09-21T00:35:45.406 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.406 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.406 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.406 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T00:35:45.406 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.406 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.406 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.406 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-09-21T00:35:45.406 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-09-21T00:35:45.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.406 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.406 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.407 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-09-21T00:35:45.407 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.407 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.407 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.407 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T00:35:45.407 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.407 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.407 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.407 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-09-21T00:35:45.407 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-09-21T00:35:45.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.407 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.407 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.408 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.408 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.408 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.408 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.408 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-09-21T00:35:45.408 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.408 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.408 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.408 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T00:35:45.408 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T00:35:45.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.408 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.408 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.409 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-09-21T00:35:45.409 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-09-21T00:35:45.409 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.409 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.409 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T00:35:45.409 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.409 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.409 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.409 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.409 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.409 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.410 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.410 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.410 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.410 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.410 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.410 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.410 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.410 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.410 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.410 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.410 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.411 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.411 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.411 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.411 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.411 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.411 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.411 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.411 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.411 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.411 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.411 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.411 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.411 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.412 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.412 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.412 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.412 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.412 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.412 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.412 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.412 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.412 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.412 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.412 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.412 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.413 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.413 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.413 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.413 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.413 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.413 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.413 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.413 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.413 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.413 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.413 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.413 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.413 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-09-21T00:35:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-09-21T00:35:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-09-21T00:35:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T00:35:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T00:35:45.418 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.418 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.418 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.418 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-09-21T00:35:45.418 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.418 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.418 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.418 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T00:35:45.418 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.418 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.418 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.418 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-09-21T00:35:45.418 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.419 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.420 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.420 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.420 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.420 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.420 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.420 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-09-21T00:35:45.420 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.420 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.420 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.420 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-09-21T00:35:45.420 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-09-21T00:35:45.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.420 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.420 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.421 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.422 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.422 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.422 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.422 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.422 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.422 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.422 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.422 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.422 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.422 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.422 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.422 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.423 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.423 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.423 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.423 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.423 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.423 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.423 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.423 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.423 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.423 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.423 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.423 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.424 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.424 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.424 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.424 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.424 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.424 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.424 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.424 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.424 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.424 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.424 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.424 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.424 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.425 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.425 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.425 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.425 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.425 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.425 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.425 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.425 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.425 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.425 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.425 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.425 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.426 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.426 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.426 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.426 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.426 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.426 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.426 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.426 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.426 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.426 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.426 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.427 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.427 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.427 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.427 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.427 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.427 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.427 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.427 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.427 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.427 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.427 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.427 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.428 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.428 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.428 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.428 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.428 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.428 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.428 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-09-21T00:35:45.428 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.428 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.428 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.428 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.428 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.429 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.429 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.429 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.429 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.429 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.429 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.429 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.429 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.429 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.429 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.429 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.429 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.429 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.429 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.430 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.430 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.430 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.430 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.430 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.430 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.430 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.430 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.430 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.430 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.430 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.430 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.430 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.431 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.431 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.431 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.431 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.431 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.431 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.431 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.431 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.431 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.431 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.431 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.431 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.431 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.431 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.432 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.432 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.432 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.432 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.432 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.432 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.432 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.432 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.432 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.432 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.432 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.432 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.432 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.433 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.433 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.433 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.433 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.433 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.433 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.433 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.433 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.433 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.433 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.433 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.433 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.433 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.434 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.434 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.434 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.434 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.434 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.434 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.434 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.434 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.434 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.434 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.434 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.434 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.435 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-09-21T00:35:45.435 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.435 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.435 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.435 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-09-21T00:35:45.435 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-09-21T00:35:45.435 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.435 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.435 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.435 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T00:35:45.435 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.435 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.435 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.435 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T00:35:45.435 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.436 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.437 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.437 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.437 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.437 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.437 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.437 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.437 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.437 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.437 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.437 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.437 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.437 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.438 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.438 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.438 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.438 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.438 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.438 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.438 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.438 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.438 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.438 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.438 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.438 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.439 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.439 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.439 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.439 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.439 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.439 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.439 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.439 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.439 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.439 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.439 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.439 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.442 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.442 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.442 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.442 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.442 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.442 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.442 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.442 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.442 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.442 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.442 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.442 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.443 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.443 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.443 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.443 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-09-21T00:35:45.443 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.443 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.443 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.443 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-09-21T00:35:45.443 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-09-21T00:35:45.443 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.443 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.443 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.444 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-09-21T00:35:45.444 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.444 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.444 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.444 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-09-21T00:35:45.444 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T00:35:45.444 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.444 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.444 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.444 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.444 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.444 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.444 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.446 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.447 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-09-21T00:35:45.449 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-09-21T00:35:45.450 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.450 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.450 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.450 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T00:35:45.450 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.450 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.450 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.450 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-09-21T00:35:45.450 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.450 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.450 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.450 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-09-21T00:35:45.450 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-09-21T00:35:45.451 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.451 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.451 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.451 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-09-21T00:35:45.451 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.451 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.451 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.451 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T00:35:45.451 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.451 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.451 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.451 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-09-21T00:35:45.451 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-09-21T00:35:45.452 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.452 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.452 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.452 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.452 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.452 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.452 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.452 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-09-21T00:35:45.452 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.452 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.452 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.452 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.452 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.453 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.453 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.453 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.453 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.453 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.453 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.453 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.453 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.453 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.453 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.453 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.453 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-09-21T00:35:45.453 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-09-21T00:35:45.453 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.454 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.454 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.454 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-09-21T00:35:45.454 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.454 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.454 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.454 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T00:35:45.454 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T00:35:45.454 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.454 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.454 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.454 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.454 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-09-21T00:35:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.456 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.456 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.456 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.456 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.456 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.456 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.456 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.456 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.456 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.456 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.456 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.456 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.457 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.457 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.457 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.457 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.457 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.457 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.457 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.457 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.457 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.457 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.457 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.457 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.457 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.457 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.458 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.458 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.458 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.458 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.458 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.458 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.458 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.458 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.458 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.458 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.458 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.458 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.458 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.458 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.461 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.462 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.462 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.462 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.462 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.462 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.462 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.462 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.462 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.462 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.462 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.462 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.462 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.462 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.463 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.463 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.463 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.463 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.463 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.463 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.463 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.463 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.463 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.463 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.463 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.463 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.463 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.464 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.464 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.464 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.464 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.464 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.464 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.464 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.464 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.464 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.464 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.464 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.464 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.464 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.464 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.464 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.465 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.466 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.466 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.466 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.466 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.466 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.466 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.466 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.466 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.466 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.466 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.466 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.467 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-09-21T00:35:45.467 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.467 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.467 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.467 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.467 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.467 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.467 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.467 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.467 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.467 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.468 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.468 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.468 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.468 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.468 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.468 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.468 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.468 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.468 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.468 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.468 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.468 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.468 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.468 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.469 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.469 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.469 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.469 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.469 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.469 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.469 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.469 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.469 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.469 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.469 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.469 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.469 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.469 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.470 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.470 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.470 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.470 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.470 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.470 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.470 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.470 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.470 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.470 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.470 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.470 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.470 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.470 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.471 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.471 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.471 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.471 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.471 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.471 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.471 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.471 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.471 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.471 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.471 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.471 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.471 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.471 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.472 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.472 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.472 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.472 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.472 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.472 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.472 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.472 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.472 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.472 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.473 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.473 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.473 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.473 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.473 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.473 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.473 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T00:35:45.473 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.473 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.473 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.473 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-09-21T00:35:45.473 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-09-21T00:35:45.474 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.474 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.474 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.474 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.474 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.474 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.474 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.474 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T00:35:45.474 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.474 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.474 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.474 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.474 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.475 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.475 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.475 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.475 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.475 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.475 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.475 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.475 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.475 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.475 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.475 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.475 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.475 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.475 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.476 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.476 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.476 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.476 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.476 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.476 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.476 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.476 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.476 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.476 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.476 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.476 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.476 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.477 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.477 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.477 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.477 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.477 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.477 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.477 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.477 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.477 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.477 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.477 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.477 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.477 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.477 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.478 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.478 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.478 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.478 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.478 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.478 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.478 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.478 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.478 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.478 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.478 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.478 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.478 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.479 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.479 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.479 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.479 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.479 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.479 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.479 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.479 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.479 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-09-21T00:35:45.479 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T00:35:45.479 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.479 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.479 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.480 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.480 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.480 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.480 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.480 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-09-21T00:35:45.480 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-09-21T00:35:45.480 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-09-21T00:35:45.480 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.480 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-09-21T00:35:45.480 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.480 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.480 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.481 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.481 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.481 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.481 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.481 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.481 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.481 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.481 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.481 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.481 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.481 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.481 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.481 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.482 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-09-21T00:35:45.482 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.482 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.482 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.482 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.482 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.482 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.482 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.483 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.483 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.483 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.483 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.483 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.483 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.483 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.483 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.483 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.483 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.483 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.483 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.483 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.484 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.484 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.484 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.484 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.484 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.484 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.484 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.484 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.484 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.484 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.484 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.485 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.485 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.485 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.485 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.485 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.485 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.485 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.485 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.485 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.485 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.485 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.485 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.485 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.486 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.486 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.486 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.486 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.486 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.486 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.486 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.486 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.486 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.486 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.486 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.486 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.487 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.487 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.487 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.487 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.487 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.487 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.487 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.487 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.487 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.487 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.487 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.487 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.487 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.488 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.488 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.488 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.488 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.488 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.488 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.488 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.488 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.488 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.488 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.488 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.489 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-09-21T00:35:45.489 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.489 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.489 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.489 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.489 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.489 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.489 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.489 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.489 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.489 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.489 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.490 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.490 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.490 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.490 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.490 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.490 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.490 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.490 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.490 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.490 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.490 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.490 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.491 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.491 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.491 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.491 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.491 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.491 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.491 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.491 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.491 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.491 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.491 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.491 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.491 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.491 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.492 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.492 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.492 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.492 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.492 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.492 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.492 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.492 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.492 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.492 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.492 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.492 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.492 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.492 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.493 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.493 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.493 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.493 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.493 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.493 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.493 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.493 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.493 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.493 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.493 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.493 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.494 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.494 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.494 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.494 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.494 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.494 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.494 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.494 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.494 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.494 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.494 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.494 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.494 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.495 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.495 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.495 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.495 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.495 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.495 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.495 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-09-21T00:35:45.495 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.495 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.495 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.495 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.496 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.496 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.496 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.496 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.496 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.496 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.496 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.496 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.496 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.496 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.496 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.496 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.496 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.496 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.497 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.497 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.497 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.497 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.497 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.497 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.497 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.497 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.497 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.497 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.497 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.498 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.498 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.498 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.498 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.498 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.498 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.498 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.498 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.498 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.498 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.498 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.498 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.498 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.499 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.499 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.499 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.499 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.499 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.499 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.499 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.499 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.499 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.499 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.499 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.499 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.499 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.499 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.500 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.500 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.500 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.500 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.500 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.500 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.500 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.500 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.500 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.500 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.500 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.500 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.500 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T00:35:45.501 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T00:35:45.501 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.501 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.501 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.501 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T00:35:45.501 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.501 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.501 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.501 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.501 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.501 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.501 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.501 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.502 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.502 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.502 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.502 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.502 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.502 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.502 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.502 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.502 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.502 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.502 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.502 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.502 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-09-21T00:35:45.503 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-09-21T00:35:45.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:45.503 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.503 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.503 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T00:35:45.503 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-09-21T00:35:45.503 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.503 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.503 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-09-21T00:35:45.503 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T00:35:45.503 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.503 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.503 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.504 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.504 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-09-21T00:35:45.504 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.504 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.504 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.504 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-09-21T00:35:45.504 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-09-21T00:35:45.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.504 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.504 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.504 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.504 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.505 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.505 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.505 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T00:35:45.505 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.505 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.505 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.505 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T00:35:45.505 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T00:35:45.505 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.505 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.505 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.505 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.505 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.505 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.506 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.506 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.506 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.506 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.506 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.506 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.506 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.506 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.506 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.506 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.506 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.506 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.507 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.507 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.507 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.507 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.507 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.507 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.507 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.507 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.507 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.507 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.507 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.507 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.507 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.508 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.508 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.508 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.508 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.508 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.508 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.508 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.508 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.508 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.508 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.508 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.508 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.508 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.509 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.509 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.509 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.509 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.509 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.509 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.509 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.509 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.509 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.509 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.511 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.511 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.511 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.511 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.511 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.511 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.511 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.511 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.511 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-09-21T00:35:45.511 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-09-21T00:35:45.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.511 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.511 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.512 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-09-21T00:35:45.512 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.512 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.512 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.512 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-09-21T00:35:45.512 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.512 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.512 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.512 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T00:35:45.512 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T00:35:45.512 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.512 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.512 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.512 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T00:35:45.513 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.513 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.513 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.513 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-09-21T00:35:45.513 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.513 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.513 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.513 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.513 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.513 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.513 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.513 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.513 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.514 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.514 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.514 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.514 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.514 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.514 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.514 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.514 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.514 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-09-21T00:35:45.514 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.514 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.514 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.515 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-09-21T00:35:45.515 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-09-21T00:35:45.515 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.515 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.515 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.515 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.515 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.515 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.515 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.515 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.515 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.515 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.515 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.516 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-09-21T00:35:45.516 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-09-21T00:35:45.516 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.516 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.516 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.516 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.516 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.516 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.516 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.516 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T00:35:45.516 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T00:35:45.516 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.517 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.517 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.517 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.517 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.517 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.517 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.517 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.517 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.517 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.517 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.517 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.517 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.517 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.518 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.518 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.518 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.518 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.518 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.518 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.518 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.518 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.518 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.518 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.518 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.518 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.518 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.519 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.519 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.519 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.519 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.519 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.519 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.519 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.519 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.519 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.519 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.519 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.519 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.519 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.521 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.521 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.521 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.521 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.521 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.521 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.521 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.521 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.521 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.521 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.521 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.521 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.521 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-09-21T00:35:45.522 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-09-21T00:35:45.522 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.522 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.522 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.522 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.522 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.522 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.522 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.522 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.522 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.522 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.522 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T00:35:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T00:35:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-09-21T00:35:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-09-21T00:35:45.524 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-09-21T00:35:45.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:45.524 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.524 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.524 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-09-21T00:35:45.524 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.524 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.524 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.524 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.524 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.524 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.524 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.524 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T00:35:45.524 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T00:35:45.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.525 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.525 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.525 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T00:35:45.525 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.525 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.525 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.525 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T00:35:45.525 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.525 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.525 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.525 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T00:35:45.525 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T00:35:45.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.526 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.526 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.526 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-09-21T00:35:45.526 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.526 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.526 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.526 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T00:35:45.526 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.526 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.526 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.526 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.526 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.526 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.527 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.527 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.527 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.527 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.527 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.527 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.527 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.527 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.527 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.527 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.527 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-09-21T00:35:45.528 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-09-21T00:35:45.528 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.528 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.528 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.528 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-09-21T00:35:45.528 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.528 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.528 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.528 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.528 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.528 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.528 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.528 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T00:35:45.529 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T00:35:45.529 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.529 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.529 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.529 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-09-21T00:35:45.529 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.529 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.529 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.529 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-09-21T00:35:45.529 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-09-21T00:35:45.529 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.529 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.529 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.529 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.530 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.530 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T00:35:45.530 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T00:35:45.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.530 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.530 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.530 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T00:35:45.530 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T00:35:45.530 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T00:35:45.530 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.530 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.530 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.530 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.531 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.531 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.531 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.531 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-09-21T00:35:45.531 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.531 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.531 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.531 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.531 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.531 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.531 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.532 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.532 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.532 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.532 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.532 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.532 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.532 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.532 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.532 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.532 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.532 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.532 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.532 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.532 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.532 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.533 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.533 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.533 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.533 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.533 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.533 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.533 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.533 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.533 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.533 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.533 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.533 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.534 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.534 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.534 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.534 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.534 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.534 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.534 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.534 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.534 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.534 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.534 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.535 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.535 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.535 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.535 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.535 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.535 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.535 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.535 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.535 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.535 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.535 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.535 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.536 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.536 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.536 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.536 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.536 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.536 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.536 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.536 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.536 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-09-21T00:35:45.536 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-09-21T00:35:45.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.536 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.536 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.538 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.538 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.538 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.538 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.538 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.538 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.538 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.538 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.538 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.538 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.539 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.539 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.539 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.539 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.539 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.539 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.539 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.539 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-09-21T00:35:45.539 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-09-21T00:35:45.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.539 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.540 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.540 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.540 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.540 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.540 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.540 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.540 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.540 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.540 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.540 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.540 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.540 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-09-21T00:35:45.541 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.541 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.541 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-09-21T00:35:45.541 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-09-21T00:35:45.541 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.541 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.541 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.541 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.541 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.541 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.541 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.541 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.542 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.542 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.542 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T00:35:45.542 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T00:35:45.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.542 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.542 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.542 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.542 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.542 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.542 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.542 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.543 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.544 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.544 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.544 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.544 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.544 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.544 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.544 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.544 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.544 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.544 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.544 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.545 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.545 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.545 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.545 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.545 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.545 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.545 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.545 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.545 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.545 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.545 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.546 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.546 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.546 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.546 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.546 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.546 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.546 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.546 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.546 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.546 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.546 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.546 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.547 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.547 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.547 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.547 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.547 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.547 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.547 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.547 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.547 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.547 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.547 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.548 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.548 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.548 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.548 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.548 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.548 INFO:teuthology.orchestra.run.smithi027.stdout: "max_requests": { 2024-09-21T00:35:45.548 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_requests", 2024-09-21T00:35:45.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.548 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.548 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.548 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T00:35:45.548 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.548 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.548 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.549 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T00:35:45.549 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.549 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.549 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.549 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T00:35:45.549 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T00:35:45.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.549 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.549 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.549 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.549 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.549 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.549 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.550 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.550 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.550 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.550 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T00:35:45.550 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T00:35:45.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.550 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.550 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.550 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.550 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.550 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-09-21T00:35:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T00:35:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T00:35:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T00:35:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T00:35:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-09-21T00:35:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-09-21T00:35:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-09-21T00:35:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-09-21T00:35:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T00:35:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-09-21T00:35:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.568 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.568 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.568 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.568 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.568 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.568 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.568 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.568 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.568 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.568 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.568 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.568 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.568 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.569 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.569 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.569 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.569 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.569 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.569 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.569 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.569 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.569 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.569 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.569 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.569 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.570 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.570 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.570 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.570 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.570 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.570 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.570 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.570 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.570 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.570 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.570 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.570 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.570 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.571 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.571 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.571 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.571 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.571 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.571 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.571 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.571 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.571 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.571 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.571 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.572 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.572 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.572 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.572 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.572 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.572 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.572 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.572 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.572 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-09-21T00:35:45.572 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-09-21T00:35:45.572 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.572 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.573 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.573 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.573 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.573 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.573 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.573 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.573 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.573 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.573 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.573 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-09-21T00:35:45.574 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-09-21T00:35:45.574 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.574 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.574 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.574 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-09-21T00:35:45.574 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.574 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.574 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.574 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.574 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.574 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.574 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.575 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-09-21T00:35:45.575 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-09-21T00:35:45.575 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.575 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.575 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.575 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.575 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.575 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.575 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.575 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.575 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.576 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.576 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.576 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-09-21T00:35:45.576 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-09-21T00:35:45.576 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.576 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.576 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.576 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.576 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.576 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.576 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.576 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.576 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.577 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.577 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.577 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-09-21T00:35:45.577 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-09-21T00:35:45.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:45.577 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.577 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.577 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.577 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.577 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.577 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.577 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.578 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.578 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.578 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.578 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-09-21T00:35:45.578 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-09-21T00:35:45.578 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.578 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.578 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.578 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.578 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.578 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.579 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.579 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.579 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.579 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.579 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.579 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-09-21T00:35:45.579 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-09-21T00:35:45.579 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.579 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.579 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.579 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.579 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.580 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.580 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.580 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.580 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.580 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.580 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.580 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-09-21T00:35:45.580 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-09-21T00:35:45.580 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.580 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.580 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.580 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.581 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.581 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.581 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.581 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.581 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.581 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.581 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.581 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-09-21T00:35:45.581 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-09-21T00:35:45.581 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.581 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.582 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.582 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.582 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T00:35:45.582 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-09-21T00:35:45.582 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.582 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.582 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.582 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.582 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.582 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.582 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.582 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.583 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.583 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.583 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.583 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.583 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.583 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.583 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.583 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.583 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.583 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.583 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-09-21T00:35:45.583 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-09-21T00:35:45.583 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.584 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.584 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.584 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.584 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.584 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.584 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.584 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.584 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.584 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.584 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.584 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-09-21T00:35:45.584 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-09-21T00:35:45.584 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.584 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.585 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.585 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.585 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.585 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.585 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.585 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.585 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.585 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.585 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.585 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-09-21T00:35:45.585 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-09-21T00:35:45.585 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.585 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.586 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.586 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.586 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.586 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.586 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.586 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.586 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.586 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.586 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.586 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.586 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.587 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smb", 2024-09-21T00:35:45.587 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.587 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.587 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.587 INFO:teuthology.orchestra.run.smithi027.stdout: "internal_store_backend": { 2024-09-21T00:35:45.587 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "internal_store_backend", 2024-09-21T00:35:45.587 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.587 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.587 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.587 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.587 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.587 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.587 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.587 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T00:35:45.587 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.588 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.588 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.588 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.588 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.588 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.588 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.588 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.588 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.588 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.588 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.588 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.588 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.588 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.589 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.589 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.589 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.589 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.589 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.589 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.589 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.589 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.589 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.589 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.589 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.589 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.589 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.590 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.590 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.590 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.590 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.590 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.590 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.590 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.590 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.590 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.590 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.590 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.590 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.590 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.591 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.591 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.591 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.591 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.591 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.591 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.591 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.591 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.591 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.591 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.591 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.591 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.591 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.591 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.592 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.592 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.592 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.592 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.592 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.592 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.592 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.592 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.592 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.592 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.592 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.592 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.592 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.593 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.593 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.593 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.593 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.593 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.593 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.593 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.593 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.593 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.593 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.593 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.593 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.593 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "update_orchestration": { 2024-09-21T00:35:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "update_orchestration", 2024-09-21T00:35:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T00:35:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-21T00:35:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-09-21T00:35:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-09-21T00:35:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T00:35:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-09-21T00:35:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-09-21T00:35:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-09-21T00:35:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-09-21T00:35:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-09-21T00:35:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-09-21T00:35:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-09-21T00:35:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T00:35:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T00:35:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T00:35:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-09-21T00:35:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-09-21T00:35:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-09-21T00:35:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-09-21T00:35:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T00:35:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-09-21T00:35:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-09-21T00:35:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T00:35:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-09-21T00:35:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-09-21T00:35:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T00:35:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-09-21T00:35:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-09-21T00:35:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T00:35:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-09-21T00:35:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-09-21T00:35:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T00:35:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-09-21T00:35:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-09-21T00:35:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-09-21T00:35:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-09-21T00:35:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-09-21T00:35:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-09-21T00:35:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T00:35:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T00:35:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T00:35:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T00:35:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T00:35:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-09-21T00:35:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-09-21T00:35:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-09-21T00:35:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-09-21T00:35:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-09-21T00:35:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-09-21T00:35:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-09-21T00:35:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-09-21T00:35:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-09-21T00:35:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-09-21T00:35:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-09-21T00:35:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-09-21T00:35:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T00:35:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-09-21T00:35:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-09-21T00:35:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-09-21T00:35:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-09-21T00:35:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T00:35:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-09-21T00:35:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-09-21T00:35:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-09-21T00:35:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-09-21T00:35:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-09-21T00:35:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T00:35:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-09-21T00:35:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T00:35:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T00:35:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-09-21T00:35:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-09-21T00:35:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-09-21T00:35:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T00:35:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-09-21T00:35:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-09-21T00:35:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-09-21T00:35:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T00:35:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T00:35:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T00:35:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.669 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.669 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.669 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.669 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.669 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.669 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.669 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.669 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.669 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.669 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.669 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.669 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.669 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.669 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.670 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.670 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.670 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.670 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.670 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.670 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.670 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.670 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.670 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.670 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.670 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.670 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.671 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.671 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.671 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.671 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.671 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.671 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.671 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.671 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.671 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.671 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.671 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.671 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.671 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.671 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.672 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.672 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.672 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.672 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.672 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.672 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.672 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-09-21T00:35:45.672 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-09-21T00:35:45.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.672 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.672 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.672 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.672 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.673 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.673 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.673 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.673 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.673 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.673 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.673 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-09-21T00:35:45.673 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-09-21T00:35:45.673 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.673 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.673 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.673 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-09-21T00:35:45.673 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.674 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.674 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.674 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.674 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.674 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.674 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.674 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-09-21T00:35:45.674 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-09-21T00:35:45.674 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.674 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.674 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.674 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T00:35:45.674 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.675 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.675 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.675 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.675 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.675 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.675 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.675 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.675 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.675 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.675 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T00:35:45.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.675 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 24182, 2024-09-21T00:35:45.675 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "y", 2024-09-21T00:35:45.676 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_features": 4540719139924082687, 2024-09-21T00:35:45.676 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-09-21T00:35:45.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.676 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-09-21T00:35:45.676 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.676 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.676 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.676 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T00:35:45.676 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T00:35:45.676 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.676 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.676 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.676 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T00:35:45.676 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.676 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.677 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.678 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.678 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.678 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.678 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.678 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.678 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.678 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.678 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.678 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.678 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.678 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.678 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.679 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.679 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.679 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.679 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.679 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.679 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.679 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.679 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.679 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.679 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.679 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.679 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.679 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.679 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.680 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.680 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.680 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.680 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.680 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.680 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.680 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.680 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.680 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.680 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.680 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.680 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.680 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.680 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.681 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.681 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.681 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.681 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.681 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.681 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.681 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.681 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.681 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.681 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.681 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.681 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.681 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.681 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.682 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.682 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-09-21T00:35:45.682 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-09-21T00:35:45.682 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.682 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.682 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.682 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.682 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.682 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.682 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.682 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T00:35:45.682 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.683 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.683 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.683 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-09-21T00:35:45.683 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-09-21T00:35:45.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.683 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.683 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.683 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-09-21T00:35:45.683 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.683 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.683 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.683 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-09-21T00:35:45.683 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.684 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.685 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-09-21T00:35:45.685 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-09-21T00:35:45.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.685 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.685 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.685 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.685 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.685 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.685 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.685 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-09-21T00:35:45.685 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.685 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.685 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.686 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-09-21T00:35:45.686 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-09-21T00:35:45.686 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.686 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.686 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.686 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-09-21T00:35:45.686 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.686 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.686 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.686 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-09-21T00:35:45.686 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.686 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.686 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.687 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-09-21T00:35:45.687 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-09-21T00:35:45.687 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.687 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.687 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.687 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.687 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.687 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.687 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.687 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-09-21T00:35:45.687 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.687 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.687 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.687 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-09-21T00:35:45.688 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-09-21T00:35:45.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.688 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.688 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.688 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.688 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.688 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.688 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.688 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T00:35:45.688 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.688 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.688 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.688 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.689 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.690 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.690 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.690 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.690 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.690 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.690 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.690 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.690 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.690 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.690 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.690 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.690 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.691 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.692 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.692 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-09-21T00:35:45.692 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-09-21T00:35:45.692 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.692 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.692 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.692 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-09-21T00:35:45.692 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.692 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.692 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.692 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T00:35:45.692 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T00:35:45.692 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.693 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.693 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-09-21T00:35:45.693 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-09-21T00:35:45.693 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T00:35:45.693 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.693 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.693 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.693 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T00:35:45.693 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T00:35:45.693 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.693 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T00:35:45.693 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T00:35:45.693 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.693 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.694 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-09-21T00:35:45.694 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-09-21T00:35:45.694 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T00:35:45.694 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.694 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.694 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-09-21T00:35:45.694 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T00:35:45.694 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-09-21T00:35:45.694 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.694 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T00:35:45.694 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.694 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.694 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-09-21T00:35:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-09-21T00:35:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T00:35:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T00:35:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T00:35:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-09-21T00:35:45.696 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-09-21T00:35:45.697 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.697 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.697 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.697 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-09-21T00:35:45.697 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.697 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.697 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.697 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T00:35:45.697 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T00:35:45.697 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.697 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.697 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-09-21T00:35:45.697 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-09-21T00:35:45.698 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T00:35:45.698 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.698 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.698 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.698 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T00:35:45.698 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T00:35:45.698 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.698 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T00:35:45.698 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T00:35:45.698 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.698 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.698 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.698 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.699 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.699 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.699 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.699 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.699 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.699 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.699 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.699 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.699 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.699 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.699 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.699 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.699 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.699 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.699 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.700 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.700 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.700 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.700 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.700 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.700 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.700 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.700 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.700 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.700 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.700 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.700 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.700 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.702 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.702 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.702 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.702 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.702 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.702 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.702 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.702 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.702 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.702 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.702 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.702 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.702 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.703 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.703 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.703 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.703 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.703 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.703 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.703 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.703 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.703 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-09-21T00:35:45.703 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-09-21T00:35:45.703 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:45.703 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.703 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.703 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.704 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.704 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.704 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.704 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T00:35:45.704 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.704 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.704 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.704 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T00:35:45.704 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T00:35:45.704 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.704 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.704 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.704 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-09-21T00:35:45.704 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.705 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.705 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.705 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-09-21T00:35:45.705 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-09-21T00:35:45.705 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-09-21T00:35:45.705 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-09-21T00:35:45.705 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-09-21T00:35:45.705 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.705 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-09-21T00:35:45.705 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.705 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.705 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.705 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-09-21T00:35:45.705 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-09-21T00:35:45.706 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.706 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.706 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.706 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.706 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.706 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.706 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.706 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T00:35:45.706 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.706 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.706 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.706 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T00:35:45.707 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T00:35:45.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.707 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.707 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.707 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T00:35:45.707 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.707 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.707 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.707 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T00:35:45.707 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.707 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.707 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.707 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.707 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.708 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.708 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.708 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.708 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.708 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.708 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.708 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.708 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.708 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.708 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.708 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.708 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-09-21T00:35:45.708 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-09-21T00:35:45.708 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.709 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.709 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.709 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T00:35:45.709 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T00:35:45.709 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.709 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.709 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T00:35:45.709 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T00:35:45.709 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.709 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.709 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-09-21T00:35:45.709 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-09-21T00:35:45.709 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T00:35:45.709 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T00:35:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.711 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-09-21T00:35:45.711 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.711 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.711 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.711 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-09-21T00:35:45.711 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-09-21T00:35:45.711 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:45.711 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.711 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.711 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T00:35:45.711 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.711 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.711 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.711 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T00:35:45.711 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.712 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.712 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.712 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-09-21T00:35:45.712 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-09-21T00:35:45.712 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.712 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.712 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.712 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-09-21T00:35:45.712 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.712 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.712 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.712 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T00:35:45.712 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.713 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.713 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.713 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-09-21T00:35:45.713 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-09-21T00:35:45.713 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.713 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.713 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.713 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-09-21T00:35:45.713 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.713 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.713 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.713 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T00:35:45.714 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.714 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.714 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.714 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-09-21T00:35:45.714 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-09-21T00:35:45.714 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.714 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.714 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.714 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.714 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.714 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.714 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.714 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T00:35:45.714 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T00:35:45.715 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.715 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.715 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-09-21T00:35:45.715 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-09-21T00:35:45.715 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.715 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.715 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.715 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T00:35:45.715 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.715 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.715 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.715 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T00:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-09-21T00:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T00:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-09-21T00:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-09-21T00:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-09-21T00:35:45.718 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.718 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T00:35:45.718 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.718 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.718 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.718 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-09-21T00:35:45.718 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-09-21T00:35:45.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.718 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.718 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T00:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-09-21T00:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-09-21T00:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T00:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-09-21T00:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-09-21T00:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T00:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-09-21T00:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-09-21T00:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T00:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T00:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-09-21T00:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-09-21T00:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T00:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-09-21T00:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-09-21T00:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-09-21T00:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T00:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-09-21T00:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-09-21T00:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-09-21T00:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T00:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T00:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-09-21T00:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-09-21T00:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T00:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-09-21T00:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-09-21T00:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-09-21T00:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T00:35:45.727 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.727 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.727 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.727 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-09-21T00:35:45.727 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.727 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.727 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.727 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-09-21T00:35:45.727 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-09-21T00:35:45.727 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.727 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.727 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.727 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T00:35:45.727 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.728 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.728 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.728 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-09-21T00:35:45.728 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.728 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.728 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.728 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-09-21T00:35:45.728 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-09-21T00:35:45.728 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.728 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.728 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T00:35:45.728 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.728 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.729 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-09-21T00:35:45.730 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.730 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.730 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.730 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-09-21T00:35:45.730 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-09-21T00:35:45.730 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.730 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.730 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.730 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T00:35:45.730 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.730 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.730 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.730 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-09-21T00:35:45.731 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.731 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.731 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.731 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nginx": { 2024-09-21T00:35:45.731 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nginx", 2024-09-21T00:35:45.731 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.731 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.731 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.731 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T00:35:45.731 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.731 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.731 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.731 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nginx container image", 2024-09-21T00:35:45.731 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.732 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.733 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-09-21T00:35:45.733 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-09-21T00:35:45.733 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.733 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.733 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.733 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T00:35:45.733 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.733 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.733 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.733 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-09-21T00:35:45.733 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.733 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.734 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.734 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_oauth2_proxy": { 2024-09-21T00:35:45.734 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T00:35:45.734 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.734 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.734 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.734 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T00:35:45.734 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.734 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.734 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.734 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "oauth2-proxy container image", 2024-09-21T00:35:45.734 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.735 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.735 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.735 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-09-21T00:35:45.735 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-09-21T00:35:45.735 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.735 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.735 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.735 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T00:35:45.735 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.735 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.735 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.735 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T00:35:45.735 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.736 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.736 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.736 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-09-21T00:35:45.736 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-09-21T00:35:45.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.736 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.736 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.736 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T00:35:45.736 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.736 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.736 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.736 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-09-21T00:35:45.736 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.736 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.737 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.737 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-09-21T00:35:45.737 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-09-21T00:35:45.737 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.737 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.737 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.737 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T00:35:45.737 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.737 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.737 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.737 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-09-21T00:35:45.737 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.737 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.737 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.738 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba_metrics": { 2024-09-21T00:35:45.738 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba_metrics", 2024-09-21T00:35:45.738 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.738 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.738 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.738 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T00:35:45.738 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.738 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.738 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.738 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T00:35:45.738 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.738 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.739 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.739 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-09-21T00:35:45.739 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-09-21T00:35:45.739 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.739 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.739 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.739 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T00:35:45.739 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.739 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.739 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.739 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-09-21T00:35:45.739 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.739 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.740 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.740 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-09-21T00:35:45.740 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-09-21T00:35:45.740 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.740 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.740 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.740 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.740 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.740 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.740 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.740 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T00:35:45.740 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.740 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.740 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.741 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-09-21T00:35:45.741 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-09-21T00:35:45.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.741 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.741 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.741 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T00:35:45.741 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.741 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.741 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.741 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T00:35:45.741 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.741 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.741 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.742 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-09-21T00:35:45.742 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T00:35:45.742 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.742 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.742 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.742 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-09-21T00:35:45.742 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.742 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.742 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.742 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T00:35:45.742 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.742 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.742 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.743 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-09-21T00:35:45.744 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-09-21T00:35:45.744 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.744 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.744 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.744 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-09-21T00:35:45.744 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.744 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.744 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.744 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-09-21T00:35:45.744 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.744 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.744 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.744 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-09-21T00:35:45.745 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-09-21T00:35:45.745 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.745 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.745 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.745 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.745 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.745 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.745 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.745 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T00:35:45.745 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.745 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.745 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.745 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-09-21T00:35:45.745 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-09-21T00:35:45.746 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.746 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.746 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.746 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T00:35:45.746 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.746 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.746 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.746 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-09-21T00:35:45.746 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.746 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.746 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.746 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-09-21T00:35:45.746 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-09-21T00:35:45.746 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.747 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.747 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.747 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T00:35:45.747 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.747 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.747 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.747 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T00:35:45.747 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.747 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.747 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.747 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-09-21T00:35:45.748 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-09-21T00:35:45.748 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.748 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.748 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.748 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T00:35:45.748 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.748 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.748 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.748 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-09-21T00:35:45.748 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.748 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.748 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.748 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-09-21T00:35:45.749 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-09-21T00:35:45.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.749 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.749 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.749 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.749 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.749 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.749 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.749 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T00:35:45.749 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.749 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.749 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.749 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-09-21T00:35:45.749 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-09-21T00:35:45.750 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.750 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.750 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.750 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.750 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.750 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.750 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.750 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T00:35:45.750 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.750 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.750 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.750 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.751 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.751 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.751 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.751 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.751 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.751 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.751 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.751 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.751 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.751 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.751 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.751 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.751 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.751 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.751 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.752 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.752 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.752 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.752 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.752 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-09-21T00:35:45.752 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-09-21T00:35:45.752 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.752 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.752 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.752 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.752 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.752 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.752 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.752 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T00:35:45.753 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.753 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.753 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.753 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.753 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.753 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.753 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.753 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.753 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.753 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.753 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.753 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.753 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T00:35:45.753 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.754 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.754 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.754 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.754 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.754 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.754 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.754 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.754 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.754 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.754 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.754 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.754 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.754 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.754 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.755 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.755 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.755 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.755 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.755 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.755 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.755 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.755 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.755 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.755 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.755 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.755 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.755 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.756 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.756 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.756 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.756 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.756 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.756 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.756 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.756 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.756 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T00:35:45.756 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T00:35:45.756 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.756 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.756 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.756 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.757 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.757 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.757 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.757 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T00:35:45.757 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.757 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.757 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.757 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T00:35:45.757 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T00:35:45.757 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.757 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.757 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.757 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-09-21T00:35:45.758 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.758 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.758 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.758 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T00:35:45.758 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.758 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.758 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.758 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-09-21T00:35:45.758 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-09-21T00:35:45.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.758 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.758 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.758 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:45.758 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.759 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.759 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.759 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-09-21T00:35:45.759 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.759 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.759 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.759 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-09-21T00:35:45.759 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-09-21T00:35:45.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.759 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.759 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.759 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:45.759 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.760 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.760 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.760 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T00:35:45.760 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.760 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.760 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.760 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-09-21T00:35:45.760 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-09-21T00:35:45.760 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.760 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.760 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.760 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.760 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.761 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.761 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.761 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-09-21T00:35:45.761 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.761 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.761 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.761 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T00:35:45.761 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T00:35:45.761 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.761 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.761 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.762 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-09-21T00:35:45.762 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.762 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.762 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.762 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-09-21T00:35:45.762 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-09-21T00:35:45.762 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.762 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T00:35:45.762 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.762 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.762 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.763 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-09-21T00:35:45.763 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-09-21T00:35:45.763 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.763 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.763 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.763 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-09-21T00:35:45.763 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.763 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.763 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.763 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T00:35:45.763 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.764 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.764 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.764 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-09-21T00:35:45.764 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-09-21T00:35:45.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.764 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.764 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.764 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T00:35:45.764 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.764 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.764 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.764 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T00:35:45.765 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.765 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.765 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.765 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-09-21T00:35:45.765 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-09-21T00:35:45.765 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.765 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.765 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.765 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.765 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.765 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.765 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.765 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T00:35:45.766 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.766 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.766 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.766 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-09-21T00:35:45.766 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-09-21T00:35:45.766 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.766 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.766 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.766 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.766 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.766 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.766 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.766 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T00:35:45.767 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.767 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.767 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.767 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-09-21T00:35:45.767 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-09-21T00:35:45.767 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.767 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.767 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.767 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.767 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.767 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.767 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.767 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T00:35:45.768 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.768 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.768 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.768 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-09-21T00:35:45.768 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-09-21T00:35:45.768 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.768 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.768 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.768 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.768 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.768 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.768 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.768 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T00:35:45.768 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.769 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.769 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.769 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-09-21T00:35:45.769 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-09-21T00:35:45.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.769 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.769 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.769 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.769 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.769 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.769 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.769 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T00:35:45.770 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.770 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.770 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.770 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-09-21T00:35:45.770 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-09-21T00:35:45.770 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.770 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.770 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.770 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-09-21T00:35:45.770 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.770 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.770 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.771 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-09-21T00:35:45.771 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.771 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.771 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.771 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.771 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.771 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.771 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.771 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.771 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.771 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.771 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.772 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.772 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.772 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.772 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.772 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.772 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-09-21T00:35:45.772 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-09-21T00:35:45.772 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.772 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.772 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.772 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.773 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.773 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.773 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.773 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T00:35:45.773 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.773 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.773 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.773 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_count_max": { 2024-09-21T00:35:45.773 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T00:35:45.773 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.773 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.773 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.774 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T00:35:45.774 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.774 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.774 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.774 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T00:35:45.774 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.774 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.774 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.774 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_interval": { 2024-09-21T00:35:45.774 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_interval", 2024-09-21T00:35:45.774 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.774 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.775 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.775 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T00:35:45.775 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.775 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.775 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.775 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T00:35:45.775 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.775 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.775 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.775 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-09-21T00:35:45.775 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-09-21T00:35:45.775 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.775 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.776 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.776 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.776 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.776 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.776 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.776 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T00:35:45.776 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.776 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.776 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.776 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-09-21T00:35:45.776 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-09-21T00:35:45.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.777 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.778 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.779 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.779 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.779 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T00:35:45.779 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.779 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.779 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.779 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-09-21T00:35:45.779 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-09-21T00:35:45.779 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.779 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.779 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.779 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.779 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.779 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.780 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.780 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T00:35:45.780 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.780 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.780 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.780 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.780 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.780 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-09-21T00:35:45.780 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.780 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.780 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.780 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.780 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.781 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.781 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.781 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.781 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.781 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.781 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.781 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.781 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.781 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.781 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.781 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.781 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.781 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.781 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.781 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.782 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.782 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.782 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.782 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.782 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.782 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.782 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.782 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.782 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.782 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.782 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.782 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.782 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.782 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.783 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.783 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.783 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.783 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.783 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.783 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.783 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.783 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.783 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.783 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.783 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.783 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.784 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.785 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.785 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.785 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.785 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.785 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.785 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.785 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.785 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.785 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-09-21T00:35:45.785 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-09-21T00:35:45.785 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.785 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.785 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.785 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-09-21T00:35:45.786 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.786 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.786 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.786 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T00:35:45.786 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.786 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.786 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.786 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.786 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.786 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.786 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.786 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.787 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.787 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.787 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.787 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.787 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.787 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.787 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.787 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.787 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-09-21T00:35:45.787 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-09-21T00:35:45.787 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.787 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.788 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.788 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-09-21T00:35:45.788 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.788 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.788 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.788 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T00:35:45.788 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.788 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.788 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.788 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.788 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.788 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-09-21T00:35:45.788 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.789 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.789 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.789 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T00:35:45.789 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T00:35:45.789 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.789 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.789 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.789 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:45.789 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.789 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.789 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.789 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.789 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.789 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.789 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.790 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T00:35:45.790 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T00:35:45.790 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.790 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.790 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.790 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.790 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.790 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.790 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.790 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.790 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.790 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.790 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.791 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T00:35:45.791 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T00:35:45.791 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.791 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.791 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.791 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.791 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.791 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.791 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.791 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.791 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.791 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.792 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.792 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-09-21T00:35:45.792 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T00:35:45.792 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.792 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.792 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.792 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.792 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.792 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.792 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.792 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.792 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.792 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.793 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.793 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T00:35:45.793 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T00:35:45.793 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.793 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.793 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.793 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.793 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.793 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.793 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.793 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.793 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.793 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.794 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.794 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T00:35:45.794 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T00:35:45.794 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.794 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.794 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.794 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.794 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.794 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.794 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.794 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.795 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.796 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T00:35:45.796 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T00:35:45.796 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.796 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.796 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.796 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.796 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.796 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.796 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.796 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.796 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.796 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.796 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T00:35:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T00:35:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T00:35:45.798 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T00:35:45.798 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.798 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.798 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.798 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.798 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.798 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.798 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.798 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.798 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.798 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.798 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.798 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T00:35:45.799 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T00:35:45.799 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.799 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.799 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.799 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.799 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.799 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.799 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.799 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.799 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.799 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.799 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.799 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T00:35:45.799 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T00:35:45.800 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.800 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.800 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.800 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.800 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.800 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.800 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.800 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.800 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.800 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.801 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.801 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T00:35:45.801 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T00:35:45.801 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.801 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.801 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.801 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.801 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.801 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.801 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.801 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.801 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.801 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.801 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T00:35:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T00:35:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.803 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T00:35:45.803 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T00:35:45.803 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.803 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.803 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.803 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T00:35:45.803 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.803 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.803 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.803 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.803 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.803 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.803 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.804 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T00:35:45.804 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T00:35:45.804 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.804 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.804 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.804 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.804 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.804 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.804 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.804 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.804 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.804 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.804 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.805 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-09-21T00:35:45.805 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-09-21T00:35:45.805 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.805 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.805 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.805 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.805 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.805 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.805 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.805 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.805 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.805 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.805 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.806 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T00:35:45.806 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T00:35:45.806 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.806 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.806 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.806 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T00:35:45.806 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.806 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.806 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.806 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.806 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.806 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.806 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.807 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T00:35:45.807 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T00:35:45.807 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.807 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.807 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.807 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.807 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.807 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.807 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.808 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.808 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.808 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.808 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.808 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T00:35:45.808 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T00:35:45.808 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.808 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.808 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.808 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.808 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.808 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.809 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.809 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.809 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.809 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.809 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.809 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T00:35:45.809 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T00:35:45.809 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.809 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.809 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.809 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.809 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.809 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.810 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.810 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.810 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.810 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.810 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.810 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T00:35:45.810 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T00:35:45.810 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.810 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.810 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.810 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.810 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.810 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.811 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.811 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.811 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.811 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.811 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.811 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T00:35:45.811 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T00:35:45.811 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.811 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.811 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.811 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-09-21T00:35:45.812 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.812 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.812 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.812 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.812 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.812 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.812 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.812 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T00:35:45.812 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T00:35:45.812 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.812 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.813 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.813 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-09-21T00:35:45.813 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.813 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.813 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.813 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.813 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.813 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.813 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.813 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T00:35:45.813 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T00:35:45.813 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.813 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.813 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.814 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.814 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.814 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.814 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.814 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.814 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.814 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.814 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.814 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T00:35:45.814 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T00:35:45.814 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.814 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.814 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.814 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.815 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.816 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.816 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.816 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.816 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.816 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.816 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T00:35:45.816 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T00:35:45.816 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.816 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.816 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.816 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.816 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.816 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.817 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.817 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.817 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.817 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.817 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.817 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T00:35:45.817 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T00:35:45.817 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.817 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.817 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.817 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.817 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T00:35:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T00:35:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.818 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.819 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.819 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.819 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.819 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.819 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.819 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T00:35:45.819 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T00:35:45.819 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.819 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.819 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.819 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.820 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.820 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.820 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.820 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.820 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.820 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.820 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.820 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T00:35:45.820 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T00:35:45.820 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.820 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.820 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.820 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.821 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.821 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.821 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.821 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.821 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.821 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.821 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.821 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T00:35:45.821 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T00:35:45.821 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.821 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.821 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.821 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.821 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.822 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.822 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.822 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.822 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.822 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.822 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.822 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T00:35:45.822 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T00:35:45.822 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.822 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.822 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.822 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.822 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.822 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.823 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.823 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.823 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.823 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.823 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.823 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T00:35:45.823 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T00:35:45.823 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.823 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.823 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.823 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T00:35:45.823 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T00:35:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T00:35:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.825 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.825 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.825 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.825 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.825 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.825 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T00:35:45.825 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T00:35:45.825 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.825 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.825 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.825 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-09-21T00:35:45.825 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.825 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T00:35:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T00:35:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-09-21T00:35:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.827 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.827 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.827 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.827 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.827 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T00:35:45.827 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T00:35:45.827 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.827 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.827 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.827 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.827 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.827 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.828 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.828 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.828 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.828 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.828 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.828 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T00:35:45.828 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T00:35:45.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.828 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.828 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T00:35:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T00:35:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T00:35:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.830 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.830 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.830 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.830 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.830 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.830 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.830 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.830 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T00:35:45.830 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T00:35:45.830 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.830 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.830 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.830 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.830 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.831 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.831 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.831 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.831 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.831 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.831 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.831 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T00:35:45.831 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T00:35:45.831 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.831 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.831 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.831 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.831 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.831 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.832 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.832 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.832 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.832 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.832 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.832 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T00:35:45.832 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T00:35:45.832 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.832 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.832 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.832 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.832 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.832 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.832 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.833 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.833 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.833 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.833 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.833 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T00:35:45.833 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T00:35:45.833 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.833 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.833 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.833 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:45.833 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.833 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.834 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.834 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.834 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.834 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.834 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.834 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T00:35:45.834 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T00:35:45.834 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.834 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.834 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.834 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T00:35:45.834 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.834 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.835 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.835 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.835 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.835 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.835 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.835 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-09-21T00:35:45.835 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-09-21T00:35:45.835 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.835 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.835 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.835 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.835 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.835 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.835 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.836 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.836 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.836 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.836 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.836 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-09-21T00:35:45.836 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-09-21T00:35:45.836 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.836 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.836 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.836 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.836 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-09-21T00:35:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-09-21T00:35:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.841 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.841 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.841 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.841 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.841 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.841 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.841 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.841 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.841 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.841 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.841 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.841 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.841 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.842 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.842 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.842 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.842 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.842 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.842 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.842 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.842 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.842 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.842 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.842 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.842 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-09-21T00:35:45.844 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.845 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.846 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.846 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.846 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.846 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.846 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.846 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.846 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.846 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.846 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.846 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T00:35:45.846 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T00:35:45.846 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.847 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.847 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.847 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T00:35:45.847 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.847 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.847 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.847 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.847 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.847 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.847 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.847 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T00:35:45.847 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T00:35:45.847 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.847 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.848 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.849 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.849 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.849 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.849 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.849 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.849 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.849 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.849 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T00:35:45.849 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T00:35:45.849 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.850 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.850 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.850 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.850 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.850 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.850 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.850 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.850 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.850 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.850 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.850 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-09-21T00:35:45.850 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-09-21T00:35:45.850 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.851 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.851 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.851 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-09-21T00:35:45.851 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.851 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.851 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.851 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.851 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.851 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.851 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.851 INFO:teuthology.orchestra.run.smithi027.stdout: "sso_oauth2": { 2024-09-21T00:35:45.851 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sso_oauth2", 2024-09-21T00:35:45.851 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.852 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.852 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.852 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.852 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.852 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.852 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.852 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.852 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.852 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.852 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.852 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T00:35:45.852 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T00:35:45.853 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.853 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.853 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.853 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-09-21T00:35:45.853 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.853 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.853 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.853 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.853 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-09-21T00:35:45.853 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.853 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.853 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.853 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.853 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.854 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T00:35:45.854 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T00:35:45.854 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.854 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.854 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.854 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T00:35:45.854 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T00:35:45.854 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T00:35:45.854 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.854 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.854 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.854 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.855 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.855 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-09-21T00:35:45.855 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-09-21T00:35:45.855 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.855 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.855 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.855 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.855 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.855 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.855 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.855 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.855 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.855 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.855 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.856 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-09-21T00:35:45.856 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.856 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.856 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.856 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-09-21T00:35:45.856 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-09-21T00:35:45.856 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.856 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.856 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.856 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.856 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.857 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.857 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.857 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-09-21T00:35:45.857 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.857 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.857 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.857 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.857 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.857 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.857 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.857 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.857 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.857 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.858 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.858 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.858 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.858 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.858 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.858 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.858 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.858 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.858 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.858 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.858 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.858 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.858 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.858 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.859 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.860 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.860 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.860 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.860 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.860 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.860 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.860 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.860 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.860 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.860 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.860 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.860 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.860 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.860 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.861 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.861 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.861 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.861 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.861 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.861 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.861 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.861 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.861 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.861 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.861 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.862 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.862 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.862 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.862 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-09-21T00:35:45.862 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-09-21T00:35:45.862 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.862 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.862 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.862 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-09-21T00:35:45.862 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.862 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.863 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.863 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T00:35:45.863 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.863 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.863 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.863 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-09-21T00:35:45.863 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-09-21T00:35:45.863 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.863 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.863 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.863 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-09-21T00:35:45.863 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.864 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.864 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.864 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T00:35:45.864 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.864 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.864 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.864 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-09-21T00:35:45.864 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-09-21T00:35:45.864 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.864 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.864 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.864 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-09-21T00:35:45.864 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.864 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.865 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T00:35:45.866 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.866 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.866 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.866 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-09-21T00:35:45.866 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-09-21T00:35:45.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.866 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.866 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.866 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.866 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.866 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.866 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T00:35:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T00:35:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T00:35:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T00:35:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T00:35:45.868 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.868 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.868 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.868 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.868 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.868 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.868 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.868 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.868 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.868 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.868 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.868 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.868 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.868 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.869 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.870 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.870 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.870 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-09-21T00:35:45.870 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.870 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.870 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.870 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.870 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.870 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.870 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.870 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.870 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.870 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.871 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.872 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.872 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.872 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.872 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.872 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.872 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.872 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.872 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.872 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.872 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.872 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.872 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.874 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.874 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.874 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.874 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.874 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.874 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.874 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.874 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.874 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.874 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.874 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.874 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.875 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.875 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.875 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.875 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.875 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.875 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.875 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-09-21T00:35:45.875 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-09-21T00:35:45.875 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.875 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.875 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.875 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T00:35:45.875 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.875 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.876 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.876 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.876 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.876 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.876 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.876 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-09-21T00:35:45.876 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-09-21T00:35:45.876 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.876 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.876 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.876 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-09-21T00:35:45.876 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.876 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.876 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.877 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.877 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.877 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.877 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.877 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T00:35:45.877 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T00:35:45.877 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.877 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.877 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.877 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T00:35:45.877 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.877 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.877 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.877 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.878 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.878 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.878 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.878 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.878 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.878 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.878 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.878 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.878 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.878 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.878 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.878 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.878 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.878 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.879 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.879 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.879 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.879 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.879 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-09-21T00:35:45.879 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-09-21T00:35:45.879 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-09-21T00:35:45.879 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.879 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-09-21T00:35:45.879 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-09-21T00:35:45.879 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.879 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.879 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.880 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-09-21T00:35:45.880 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.880 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.880 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.880 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T00:35:45.880 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.880 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.880 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.880 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-09-21T00:35:45.880 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-09-21T00:35:45.880 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.880 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.880 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.880 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-09-21T00:35:45.881 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.881 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.881 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.881 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T00:35:45.881 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.881 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.881 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.881 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-09-21T00:35:45.881 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-09-21T00:35:45.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.881 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.881 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.882 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.882 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.882 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.882 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.882 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-09-21T00:35:45.882 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.882 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.882 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.882 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T00:35:45.882 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T00:35:45.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.882 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.882 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.883 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-09-21T00:35:45.883 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-09-21T00:35:45.883 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.883 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.883 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T00:35:45.883 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.883 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.883 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.883 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.883 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.883 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.883 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.883 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.883 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.884 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.884 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.884 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.884 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.884 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.884 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.884 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.884 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.884 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.884 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.884 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.884 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.884 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.884 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.885 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.886 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.886 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.886 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.886 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.886 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.886 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.886 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.886 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.886 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.886 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.886 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.886 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.886 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.886 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.887 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.888 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.889 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T00:35:45.890 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.891 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.891 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.891 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-09-21T00:35:45.891 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.891 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.891 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.891 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T00:35:45.891 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.891 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.891 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.891 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-09-21T00:35:45.891 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-09-21T00:35:45.891 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.891 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.892 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.892 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T00:35:45.892 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T00:35:45.892 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-09-21T00:35:45.892 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.892 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T00:35:45.892 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.892 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.892 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.892 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-09-21T00:35:45.892 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-09-21T00:35:45.892 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.893 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.893 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.893 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.893 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.893 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.893 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.893 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-09-21T00:35:45.893 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.893 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.893 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.893 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-09-21T00:35:45.893 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-09-21T00:35:45.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.893 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.894 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.894 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-09-21T00:35:45.894 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.894 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.894 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.894 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T00:35:45.894 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.894 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.894 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.894 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.894 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.894 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-09-21T00:35:45.894 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.895 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.896 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.897 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.898 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.898 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.898 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.898 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.898 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.898 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.898 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.898 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.898 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.898 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.898 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.898 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.898 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.899 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.900 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.900 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.900 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.900 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.900 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.900 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-09-21T00:35:45.900 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.900 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.900 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.900 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.900 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.900 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.900 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.901 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.901 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.901 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.901 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.901 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.901 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.901 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.901 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.901 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.901 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.901 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.901 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.901 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.901 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.901 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.902 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.902 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.902 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.902 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.902 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.902 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.902 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.902 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.902 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.902 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.902 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.902 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.902 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.903 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.903 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.903 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.903 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.903 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.903 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.903 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.903 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.903 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.903 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.903 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.903 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.903 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.904 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.904 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.904 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.904 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.904 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.904 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.904 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.904 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.904 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.904 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.904 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.904 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.904 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.905 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.906 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.906 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.906 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.906 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.906 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.906 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.906 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.906 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.906 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-09-21T00:35:45.906 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.906 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.906 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.906 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-09-21T00:35:45.907 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-09-21T00:35:45.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.907 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.907 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.907 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T00:35:45.907 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.907 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.907 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.907 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T00:35:45.907 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.907 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.907 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.907 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-09-21T00:35:45.907 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-09-21T00:35:45.908 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.908 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.908 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.908 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:45.908 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-09-21T00:35:45.908 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.908 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.908 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T00:35:45.908 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.908 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.908 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.908 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.908 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.908 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.909 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.909 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.909 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.909 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.909 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.909 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.909 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.909 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.909 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.909 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.909 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.909 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.909 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.909 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.910 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.910 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.910 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.910 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.910 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.910 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.910 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.910 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.910 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.910 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.910 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.910 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.913 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.913 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.913 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.913 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.913 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.913 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.913 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.913 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.913 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.913 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.913 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.913 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.913 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.914 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.914 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.914 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.914 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.914 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.914 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.914 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.914 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.914 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.914 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.914 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.914 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.915 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-09-21T00:35:45.915 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.915 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.915 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.915 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-09-21T00:35:45.915 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-09-21T00:35:45.915 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.915 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.915 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.915 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-09-21T00:35:45.915 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.915 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.916 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.916 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-09-21T00:35:45.916 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T00:35:45.916 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.916 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.916 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.916 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.916 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.916 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.916 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.916 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.916 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.916 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.917 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.917 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.917 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.917 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.917 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.917 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.917 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.917 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.917 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.917 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.917 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.917 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.917 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.917 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.918 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.918 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.918 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.918 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.918 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.918 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.918 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.918 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.918 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.918 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.918 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.918 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.918 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.919 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.919 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.919 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.919 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.919 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.919 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.919 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.919 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.919 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.919 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.919 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.919 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.920 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.921 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.921 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.921 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.921 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.921 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-09-21T00:35:45.921 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-09-21T00:35:45.921 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.921 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.921 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.921 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.921 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.921 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.921 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.922 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-09-21T00:35:45.922 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T00:35:45.922 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.922 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.922 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-09-21T00:35:45.922 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-09-21T00:35:45.922 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.922 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.922 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.922 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T00:35:45.922 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.923 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.923 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.923 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-09-21T00:35:45.923 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.923 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.923 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.923 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-09-21T00:35:45.923 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-09-21T00:35:45.923 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.923 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.923 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.923 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-09-21T00:35:45.923 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.924 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.924 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.924 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T00:35:45.924 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.924 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.924 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.924 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-09-21T00:35:45.924 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-09-21T00:35:45.924 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.924 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.924 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.924 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.924 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.924 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.925 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.925 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-09-21T00:35:45.925 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.925 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.925 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.925 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.925 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.925 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.925 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.925 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.925 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.925 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.925 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.926 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.926 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.926 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.926 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.926 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.926 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-09-21T00:35:45.926 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-09-21T00:35:45.926 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.926 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.926 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.926 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-09-21T00:35:45.926 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.926 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T00:35:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T00:35:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-09-21T00:35:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.928 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.928 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.928 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.928 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.928 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.928 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.928 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.928 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.928 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.928 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.928 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.929 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.930 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.930 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.930 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.930 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.930 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.930 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.930 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.930 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.930 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.930 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.930 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.930 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.932 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.932 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.932 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.932 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.932 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.932 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.932 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.932 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.932 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.932 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.932 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.932 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.932 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.932 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.933 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.933 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.933 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.933 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.933 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.933 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.933 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.933 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.933 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.933 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.933 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.933 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.934 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.934 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-09-21T00:35:45.934 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.934 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.934 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.934 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.934 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.934 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.934 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.934 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.934 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.935 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.935 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.935 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.935 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.935 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.935 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.935 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.935 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.935 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.935 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.935 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.935 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.935 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.936 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.936 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.936 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.936 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.936 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.936 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.936 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.936 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.936 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.936 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.936 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.936 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.936 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.937 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.937 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.937 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.937 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.937 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.937 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.937 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.938 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.940 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.940 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.940 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.940 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.940 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.940 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.940 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.940 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.940 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.940 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.940 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-09-21T00:35:45.940 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.940 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.941 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.941 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.941 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.941 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.941 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.941 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.941 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.941 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.941 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.941 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.941 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.941 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.941 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.941 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.941 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.942 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.942 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.942 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.942 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.942 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.942 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.942 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.942 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.942 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.942 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.942 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.942 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.942 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.942 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.943 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.943 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.943 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.943 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.943 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.943 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.943 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.943 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.944 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.944 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.944 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.944 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.944 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.944 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.944 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.944 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.944 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.944 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.944 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.944 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.944 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.944 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.944 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.945 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.945 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.945 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.945 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.945 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.945 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.945 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.945 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.945 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.945 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.945 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.945 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.945 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.946 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.946 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.946 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.946 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.946 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.946 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.946 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.946 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.946 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.946 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.946 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.946 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.946 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.947 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.948 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T00:35:45.948 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.948 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.948 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.948 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.948 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.948 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.948 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.948 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.948 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.948 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.948 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.948 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.949 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.949 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.949 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.949 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.949 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.949 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.949 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.949 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.949 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.949 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.949 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.949 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.949 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.950 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.950 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.950 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.950 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.950 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.950 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.950 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.950 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.950 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.950 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.950 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.950 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.950 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.951 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.951 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.951 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.951 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.951 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.951 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.951 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.951 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.951 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.951 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.951 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.951 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.951 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.951 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.952 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.952 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.952 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.952 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.952 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.952 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.952 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.952 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.952 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.952 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.952 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.952 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.952 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-09-21T00:35:45.953 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-09-21T00:35:45.954 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.954 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-09-21T00:35:45.954 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.954 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.954 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.954 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.954 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.954 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.954 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.954 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.954 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.954 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.954 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.955 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.955 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.955 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.955 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.955 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.955 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.955 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.955 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-09-21T00:35:45.955 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.955 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.955 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.955 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.956 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.956 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.956 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.956 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.956 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.956 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.956 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.956 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.956 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.956 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.956 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.956 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.956 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.957 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.957 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.957 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.957 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.957 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.957 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.957 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.957 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.957 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.957 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.957 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.958 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.958 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.958 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.958 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.958 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.958 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.958 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.958 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.958 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.958 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.958 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.958 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.959 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.959 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.959 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.959 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.959 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.959 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.959 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.959 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.959 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.959 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.959 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.959 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.959 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.960 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.960 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.960 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.960 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.960 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.960 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.960 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.960 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.960 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.960 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.960 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.960 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.961 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.961 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.961 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.961 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.961 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.961 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.961 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.961 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.961 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.961 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.962 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.962 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.962 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.962 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.962 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.962 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.962 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-09-21T00:35:45.962 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.962 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.962 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.962 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.963 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.963 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.963 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.963 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.963 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.963 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.963 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.963 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.963 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.963 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.963 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.963 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.964 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.964 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.964 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.964 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.964 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.964 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.964 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.964 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.964 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.964 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.964 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.964 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.964 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.965 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.965 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.965 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.965 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.965 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.965 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.965 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.965 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.965 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.965 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.965 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.965 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.966 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.966 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.966 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.966 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.966 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.966 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.966 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.966 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.966 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.966 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.966 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.966 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.966 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.966 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.967 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.967 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.967 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.967 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.967 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.967 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.967 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.967 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.967 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.967 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.967 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.967 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.969 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-09-21T00:35:45.969 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.969 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.969 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.969 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.969 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.969 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.969 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.969 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.969 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.969 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.969 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.970 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.970 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.970 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.970 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.970 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.970 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.970 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.970 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.970 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.970 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.970 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.970 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.970 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.971 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.971 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.971 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.971 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.971 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.971 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.971 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.971 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.971 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.971 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.971 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.971 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.972 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.972 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.972 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.972 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.972 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.972 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.972 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.972 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.972 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.972 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.972 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.972 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.972 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.973 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.973 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.973 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.973 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.973 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.973 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.973 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.973 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.973 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.973 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.973 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.973 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.973 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.974 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.974 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.974 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.974 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.974 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T00:35:45.974 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T00:35:45.974 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.974 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.974 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.974 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T00:35:45.974 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.974 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.974 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.974 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.975 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.975 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.975 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.975 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.975 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.975 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.975 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.975 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.975 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.975 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.975 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.975 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.976 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.976 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.976 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.976 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-09-21T00:35:45.976 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-09-21T00:35:45.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:45.976 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.976 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.976 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T00:35:45.976 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-09-21T00:35:45.976 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.976 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.976 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-09-21T00:35:45.977 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T00:35:45.977 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.977 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.977 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.977 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.977 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-09-21T00:35:45.977 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.977 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.977 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.977 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-09-21T00:35:45.977 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-09-21T00:35:45.977 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.978 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.978 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.978 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.978 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.978 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.978 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.978 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T00:35:45.978 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.978 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.978 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.978 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T00:35:45.978 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T00:35:45.978 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.979 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.979 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.979 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.979 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.979 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.979 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.979 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.979 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.979 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.979 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.979 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.979 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.979 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.980 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.980 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.980 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.980 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.980 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.980 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.980 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.980 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.980 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.980 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.980 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.980 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.980 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.980 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.980 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.981 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.981 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.981 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.981 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.981 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.981 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.981 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.981 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.981 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.981 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.981 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.981 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.981 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.982 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.982 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.982 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.982 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.982 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.982 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.982 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.982 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.982 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.982 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.982 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.983 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.983 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.983 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.983 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.983 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.983 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.983 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.983 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.983 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.983 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.983 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.984 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.984 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.984 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.984 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.984 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.984 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.984 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.984 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.984 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.984 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.984 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.984 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.984 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-09-21T00:35:45.984 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-09-21T00:35:45.985 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.985 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.985 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.985 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-09-21T00:35:45.985 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.985 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.985 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.985 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-09-21T00:35:45.985 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.985 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.985 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.985 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T00:35:45.985 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T00:35:45.985 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:45.986 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.986 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.986 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T00:35:45.986 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.986 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.986 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.986 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-09-21T00:35:45.986 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.986 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.986 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.986 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.986 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.986 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.987 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:45.987 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.987 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:45.987 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.987 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.987 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.987 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.987 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.987 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.987 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.987 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.987 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:45.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:45.987 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-09-21T00:35:45.988 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:45.988 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:45.988 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:45.988 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-09-21T00:35:45.988 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-09-21T00:35:45.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.988 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.988 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.988 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.988 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.988 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.988 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.988 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.988 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.989 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.990 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.991 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.991 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.991 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.991 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.991 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.991 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:45.991 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:45.991 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.991 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.991 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.991 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.991 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.991 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.992 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.992 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.992 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.992 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.992 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.992 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:45.992 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:45.992 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.992 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.992 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.992 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:45.992 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.992 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.992 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:45.993 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.994 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.994 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:45.994 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.994 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.994 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.994 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.994 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.994 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.994 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.994 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-09-21T00:35:45.994 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-09-21T00:35:45.994 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.995 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.995 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.995 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:45.995 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.995 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.995 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.995 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.995 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.995 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.995 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.995 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T00:35:45.995 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T00:35:45.995 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.995 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.996 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.996 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-09-21T00:35:45.996 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.996 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.996 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.996 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.996 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.996 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.996 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.996 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-09-21T00:35:45.996 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-09-21T00:35:45.996 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:45.997 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.997 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-09-21T00:35:45.997 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.997 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.997 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.997 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:45.997 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.997 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.997 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.997 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T00:35:45.997 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T00:35:45.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:45.997 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.998 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:45.998 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T00:35:45.998 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.998 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.998 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.998 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T00:35:45.998 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.998 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.998 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.998 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T00:35:45.998 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T00:35:45.998 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.998 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:45.999 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:45.999 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-09-21T00:35:45.999 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:45.999 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:45.999 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:45.999 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T00:35:45.999 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:45.999 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:45.999 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:45.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:45.999 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:45.999 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:45.999 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:45.999 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.000 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.000 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.000 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.000 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.000 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.000 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.000 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.000 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.000 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.000 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-09-21T00:35:46.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-09-21T00:35:46.000 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.000 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.000 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.001 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-09-21T00:35:46.001 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.001 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.001 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.001 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.001 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.001 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.001 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.001 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T00:35:46.001 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T00:35:46.001 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.001 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.001 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.002 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-09-21T00:35:46.002 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.002 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.002 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.002 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-09-21T00:35:46.002 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-09-21T00:35:46.002 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.002 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.002 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.002 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.002 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.002 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T00:35:46.003 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T00:35:46.003 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.003 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.003 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.003 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T00:35:46.003 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T00:35:46.003 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T00:35:46.003 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.003 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.003 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.003 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.003 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.003 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.003 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.004 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-09-21T00:35:46.004 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.004 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.004 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.004 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.004 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.004 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.004 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.004 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.004 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.004 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.004 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.004 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.005 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.006 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.006 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.006 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.006 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.006 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.006 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.006 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.006 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.006 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.006 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.006 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.006 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.006 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.006 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.008 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.008 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.008 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.008 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.008 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.008 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.008 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.008 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.008 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.008 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.008 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.008 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.008 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.009 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-09-21T00:35:46.009 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-09-21T00:35:46.009 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.009 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.009 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.009 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:46.009 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.009 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.009 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.009 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.009 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.009 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.009 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.009 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-09-21T00:35:46.010 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T00:35:46.010 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.010 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.010 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.010 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.010 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.010 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.010 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.010 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.010 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.010 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.010 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.010 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.011 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.011 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.011 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.011 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.011 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.011 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.011 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.011 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.011 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.011 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.011 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.011 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.012 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-09-21T00:35:46.012 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-09-21T00:35:46.012 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.012 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.012 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.012 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.012 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.012 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.012 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.012 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.012 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.012 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.012 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.012 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-09-21T00:35:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-09-21T00:35:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-09-21T00:35:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.014 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.014 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.014 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.014 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.014 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.014 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T00:35:46.014 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T00:35:46.014 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.014 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.014 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.014 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.015 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.015 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.015 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.015 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.015 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.015 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.015 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.015 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.015 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.015 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.015 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.015 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.015 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.016 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.016 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.016 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.016 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.016 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.016 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.016 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.016 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.016 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.016 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.016 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.016 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.016 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.016 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.017 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.017 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.017 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.017 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.017 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.017 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.017 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.017 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.017 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.017 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.017 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.017 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.017 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.017 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.018 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.018 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.018 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.018 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.018 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.018 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.018 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.018 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.018 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.018 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.018 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.018 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.018 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.019 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.019 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.019 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.019 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.019 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.019 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.019 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.019 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.019 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.019 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.019 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.019 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.019 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.020 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.020 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.020 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.020 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.020 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.020 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.020 INFO:teuthology.orchestra.run.smithi027.stdout: "max_requests": { 2024-09-21T00:35:46.020 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_requests", 2024-09-21T00:35:46.020 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.020 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.020 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.020 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T00:35:46.020 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.021 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.021 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.021 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T00:35:46.021 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.021 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.021 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.021 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T00:35:46.021 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T00:35:46.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.021 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.022 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.022 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.022 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.022 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.022 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.022 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.022 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.022 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.022 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T00:35:46.022 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T00:35:46.022 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.022 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.023 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.023 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.023 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.023 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.023 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.023 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.023 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.023 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.023 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.023 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.023 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.023 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.024 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.024 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.024 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.024 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.024 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.024 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.024 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.024 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.024 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.024 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.024 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.024 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.025 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-09-21T00:35:46.025 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.025 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.025 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.025 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.025 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.025 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.025 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.025 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.025 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.025 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.025 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.025 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.026 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.026 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.026 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.026 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.026 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.026 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.026 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.026 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.026 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.026 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.026 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.026 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T00:35:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T00:35:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T00:35:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T00:35:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.033 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.033 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.033 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.033 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.033 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.033 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.033 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.033 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.033 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.033 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.033 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.033 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.033 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.034 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.034 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.034 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.034 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.034 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.034 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.034 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.034 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.034 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.034 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.034 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.034 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.034 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.035 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.037 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.037 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.037 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.037 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.037 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.037 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.037 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.037 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.113 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.113 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-09-21T00:35:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-09-21T00:35:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-09-21T00:35:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T00:35:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-09-21T00:35:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-09-21T00:35:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-09-21T00:35:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-09-21T00:35:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-09-21T00:35:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-09-21T00:35:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-09-21T00:35:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-09-21T00:35:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-09-21T00:35:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-09-21T00:35:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-09-21T00:35:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-09-21T00:35:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.126 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-09-21T00:35:46.126 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-09-21T00:35:46.126 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.126 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.126 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.126 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.126 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.126 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.126 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.126 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.126 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.126 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.126 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.127 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-09-21T00:35:46.127 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-09-21T00:35:46.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.127 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.127 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.127 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.127 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.127 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.127 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.127 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.127 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.127 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.128 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.128 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-09-21T00:35:46.128 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-09-21T00:35:46.128 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.128 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.128 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.128 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.128 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.128 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.128 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.128 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.128 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.129 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.129 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.129 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-09-21T00:35:46.129 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-09-21T00:35:46.129 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.129 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.129 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.129 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.129 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T00:35:46.129 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-09-21T00:35:46.129 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.129 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.129 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.130 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.130 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.130 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.130 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.130 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.130 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.130 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.130 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.130 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.130 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.130 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.130 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.130 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.131 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.131 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.131 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-09-21T00:35:46.131 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-09-21T00:35:46.131 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.131 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.131 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.131 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.131 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.131 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.131 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.131 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.131 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.132 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.132 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.132 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-09-21T00:35:46.132 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-09-21T00:35:46.132 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.132 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.132 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.132 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.132 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.132 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.132 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.132 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.132 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.133 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.133 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.133 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-09-21T00:35:46.133 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-09-21T00:35:46.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.133 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.133 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.133 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.133 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.133 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.133 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.133 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.133 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.133 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.134 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.134 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smb", 2024-09-21T00:35:46.134 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.134 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.134 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.134 INFO:teuthology.orchestra.run.smithi027.stdout: "internal_store_backend": { 2024-09-21T00:35:46.134 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "internal_store_backend", 2024-09-21T00:35:46.134 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.134 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.134 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.135 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.135 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.135 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.135 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.135 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T00:35:46.135 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.135 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.135 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.135 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.135 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.135 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.135 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.135 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.136 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.136 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.136 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.136 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.136 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.136 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.136 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.136 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.136 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.136 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.136 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.136 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.136 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.136 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.136 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.137 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.137 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.137 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.137 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.137 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.137 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.137 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.137 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.137 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.137 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.137 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.137 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.138 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.138 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.138 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.138 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.138 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.138 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.138 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.138 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.138 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.138 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.138 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.138 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.139 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.139 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.139 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.139 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.139 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.139 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.139 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.139 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.139 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.139 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.139 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.139 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.139 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.140 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.140 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.140 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.140 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.140 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.140 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.140 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.140 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.140 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.140 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.140 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.140 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.140 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.140 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.141 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.141 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.141 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.141 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.141 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.141 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.141 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.141 INFO:teuthology.orchestra.run.smithi027.stdout: "update_orchestration": { 2024-09-21T00:35:46.141 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "update_orchestration", 2024-09-21T00:35:46.141 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.141 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.141 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.142 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.142 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.142 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.142 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.142 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T00:35:46.142 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.142 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.142 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.142 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.142 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.142 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-21T00:35:46.142 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.143 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.143 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.143 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-09-21T00:35:46.143 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-09-21T00:35:46.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.143 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.143 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.143 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.143 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.143 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.143 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.143 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T00:35:46.143 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.146 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.146 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.146 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.146 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.146 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.146 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.146 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.146 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.146 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.146 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.146 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.146 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.147 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.147 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.147 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.147 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.147 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.147 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.147 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.147 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.147 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.147 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.147 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.147 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.147 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-09-21T00:35:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-09-21T00:35:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.160 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.160 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.160 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.160 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.160 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.160 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.160 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.160 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.160 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.160 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.160 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.160 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.160 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.160 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.161 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.161 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.161 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.161 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.161 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.161 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.161 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.161 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.161 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.161 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.161 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.161 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.161 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.161 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.162 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.162 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.162 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.162 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.162 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.162 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.162 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.162 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.162 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.162 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.162 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.162 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.162 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.163 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.163 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.163 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.163 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.163 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.163 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.163 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.163 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.163 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.163 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.163 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.163 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.163 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-09-21T00:35:46.164 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.164 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.164 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.164 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-09-21T00:35:46.164 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-09-21T00:35:46.164 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.164 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.164 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.164 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T00:35:46.164 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.164 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.164 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.164 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.164 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.164 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.165 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.165 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T00:35:46.165 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T00:35:46.165 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.165 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.165 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.165 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-09-21T00:35:46.165 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.165 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.165 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.165 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.165 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.165 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.166 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.166 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.166 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.166 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.166 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.166 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.166 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.166 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.166 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.166 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.166 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.166 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.167 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.167 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.167 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.167 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.167 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.167 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.167 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.167 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.167 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.167 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.167 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.167 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.167 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.168 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.168 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.168 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.168 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.168 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.168 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.168 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.168 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.168 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.168 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.168 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.168 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.169 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.169 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.169 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.169 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.169 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.169 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.169 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.169 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.169 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.169 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.169 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.169 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.169 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.170 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.170 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.170 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.170 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.170 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.170 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.170 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.170 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.170 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.170 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.170 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.171 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.171 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.171 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.171 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.171 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.171 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.171 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.171 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.171 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.171 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.171 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.171 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.172 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.172 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.172 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.172 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.172 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.172 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.172 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.172 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-09-21T00:35:46.172 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.172 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.172 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.172 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-09-21T00:35:46.173 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-09-21T00:35:46.173 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.173 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.173 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.173 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.173 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.173 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.173 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.173 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T00:35:46.173 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.173 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.173 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.173 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-09-21T00:35:46.174 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-09-21T00:35:46.174 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.174 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.174 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.174 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.174 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.174 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.174 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T00:35:46.174 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.174 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.174 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.175 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-09-21T00:35:46.175 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-09-21T00:35:46.175 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.175 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.175 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.175 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.175 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.175 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.175 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.175 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T00:35:46.176 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.176 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.176 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.176 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-09-21T00:35:46.176 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-09-21T00:35:46.176 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.176 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.176 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.176 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.176 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.176 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.176 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.176 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T00:35:46.177 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.177 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.177 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.177 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-09-21T00:35:46.177 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-09-21T00:35:46.177 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.177 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.177 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.177 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.177 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.177 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.177 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.177 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T00:35:46.177 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.178 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.178 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.178 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-09-21T00:35:46.178 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-09-21T00:35:46.178 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.178 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.178 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.178 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.178 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.178 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.178 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.178 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.178 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.179 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.179 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.179 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-09-21T00:35:46.179 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-09-21T00:35:46.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.179 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.179 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.179 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.179 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.179 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.179 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.179 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.179 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.180 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.180 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.180 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-09-21T00:35:46.180 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-09-21T00:35:46.180 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.180 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.180 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.180 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T00:35:46.180 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.180 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.180 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.180 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.181 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.181 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.181 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.181 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T00:35:46.181 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T00:35:46.181 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.181 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.181 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.181 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.181 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.181 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.181 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.182 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.182 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.182 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.182 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.182 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T00:35:46.182 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T00:35:46.183 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.183 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.183 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.183 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-09-21T00:35:46.183 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-09-21T00:35:46.183 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.183 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.183 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.183 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.183 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.183 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.184 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-09-21T00:35:46.184 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-09-21T00:35:46.184 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.184 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.184 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.184 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-09-21T00:35:46.184 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.184 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.184 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.184 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.184 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.185 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.185 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.185 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-09-21T00:35:46.185 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-09-21T00:35:46.185 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.186 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.186 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.186 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.186 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.186 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.186 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.187 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.187 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.187 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.187 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.187 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-09-21T00:35:46.188 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-09-21T00:35:46.188 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.188 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.188 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.188 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.188 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.188 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.188 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.188 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.188 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.188 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.188 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.188 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.189 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.189 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.189 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.189 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.189 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.189 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.189 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.189 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.189 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.189 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.189 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.189 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.189 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.189 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.190 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.190 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.190 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.190 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.190 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.190 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.190 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.190 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.190 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.190 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.190 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.190 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.190 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.191 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.191 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.191 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.191 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.191 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.191 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.191 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.191 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.191 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.191 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.191 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.191 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.191 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.192 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.192 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.192 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.192 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.192 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.192 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.192 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.192 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.192 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.192 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.192 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.192 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.192 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.192 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.193 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.193 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.193 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.193 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.193 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.193 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.193 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.193 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.193 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.193 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.193 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.193 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-09-21T00:35:46.193 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-09-21T00:35:46.193 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.194 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.194 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.194 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.194 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.194 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.194 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.194 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.194 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.194 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.194 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.194 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-09-21T00:35:46.194 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-09-21T00:35:46.194 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.195 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.195 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.195 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.195 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.195 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.195 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.195 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.195 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.195 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.195 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.195 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.195 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.196 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.196 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.196 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.196 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.196 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.196 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.196 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.196 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.196 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.196 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.196 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-09-21T00:35:46.196 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-09-21T00:35:46.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.196 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.197 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.197 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T00:35:46.197 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.197 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.197 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.197 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.197 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.197 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.197 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.197 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.197 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.197 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-09-21T00:35:46.198 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.198 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.198 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.198 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.198 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.198 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.199 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.199 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.199 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.199 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.199 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.199 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.200 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.200 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.200 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.200 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.200 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.200 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.200 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.200 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.200 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.201 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.201 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.201 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.201 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.201 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.201 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.201 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.202 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.202 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.202 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.202 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.202 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.202 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.202 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.202 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.203 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.203 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.203 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.203 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.203 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.203 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.203 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.203 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.203 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.203 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.203 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.203 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.203 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.204 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.204 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.204 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.204 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.204 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.204 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.204 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.204 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.204 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.204 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.204 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.204 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.204 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.204 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.205 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.205 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.205 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.205 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.205 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.205 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.205 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.205 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.205 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.205 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.205 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.205 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.205 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.206 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.206 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.206 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.206 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.206 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.206 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.206 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-21T00:35:46.206 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.206 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.206 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.206 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.207 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.207 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.207 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.207 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.207 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.207 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.207 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.207 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.207 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.207 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.207 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.207 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.207 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.207 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.208 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.208 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.208 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.208 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.208 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.208 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.208 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.208 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.208 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.208 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.208 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.208 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.208 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.209 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.210 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.210 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.210 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.210 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.210 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.210 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.210 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.210 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.210 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.211 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.211 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.211 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.211 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.211 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.211 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.211 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.211 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.211 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.211 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.211 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.211 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-09-21T00:35:46.211 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-09-21T00:35:46.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.212 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.212 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.212 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-09-21T00:35:46.212 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.212 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.212 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.212 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T00:35:46.212 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.212 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.212 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.212 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-09-21T00:35:46.212 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-09-21T00:35:46.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-09-21T00:35:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-09-21T00:35:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-09-21T00:35:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.213 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.214 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.214 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.214 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.214 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.214 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.214 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T00:35:46.214 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.214 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.214 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.214 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-09-21T00:35:46.214 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T00:35:46.214 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.215 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.215 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.215 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.215 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.215 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.215 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.215 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T00:35:46.215 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.215 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.215 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.215 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.215 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.215 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.215 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.216 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.216 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.216 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.216 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.216 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.216 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.216 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.216 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.216 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.216 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.216 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.216 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-09-21T00:35:46.217 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.217 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.217 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.217 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-09-21T00:35:46.217 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-09-21T00:35:46.217 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T00:35:46.217 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.217 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.217 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-09-21T00:35:46.217 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.217 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.217 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.217 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.217 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.218 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.218 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.218 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-09-21T00:35:46.218 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-09-21T00:35:46.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.218 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.218 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.218 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.218 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.218 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.218 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.218 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.218 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.219 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.219 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T00:35:46.219 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T00:35:46.219 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.219 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.219 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T00:35:46.219 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.219 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.219 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.220 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.220 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.220 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.220 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.220 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.220 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.220 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.220 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.220 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.221 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.221 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.221 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.221 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.221 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.221 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.221 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.222 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.222 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.222 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.222 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.222 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.222 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.223 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.223 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.223 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.223 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.223 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.224 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.224 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.224 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.224 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.224 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.224 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.224 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.225 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.225 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.225 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.225 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.225 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.225 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.226 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.226 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.226 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.226 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.226 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.226 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.227 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.227 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.227 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.227 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.227 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.227 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.227 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.228 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.228 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.228 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.228 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.228 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.229 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.229 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.229 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.229 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.229 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.230 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.230 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.230 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.230 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.230 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.231 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.231 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.231 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.231 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.231 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.232 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.232 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.232 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.232 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.232 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.233 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-09-21T00:35:46.233 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-09-21T00:35:46.233 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.233 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.233 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.233 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.234 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.234 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.234 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.234 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.234 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.234 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.235 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.235 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-09-21T00:35:46.235 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-09-21T00:35:46.235 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.235 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.235 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.236 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-09-21T00:35:46.236 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.236 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.236 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.236 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.236 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.237 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.237 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.237 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-09-21T00:35:46.237 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-09-21T00:35:46.237 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.238 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.238 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.238 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T00:35:46.238 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.238 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.238 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.239 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.239 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.239 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.239 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.240 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T00:35:46.240 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.240 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.240 INFO:teuthology.orchestra.run.smithi027.stdout: "modules": [ 2024-09-21T00:35:46.240 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-09-21T00:35:46.240 INFO:teuthology.orchestra.run.smithi027.stdout: "dashboard", 2024-09-21T00:35:46.241 INFO:teuthology.orchestra.run.smithi027.stdout: "iostat", 2024-09-21T00:35:46.241 INFO:teuthology.orchestra.run.smithi027.stdout: "nfs", 2024-09-21T00:35:46.241 INFO:teuthology.orchestra.run.smithi027.stdout: "restful", 2024-09-21T00:35:46.241 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_schedule" 2024-09-21T00:35:46.241 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.241 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-09-21T00:35:46.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.242 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-09-21T00:35:46.242 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.242 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.242 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.242 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T00:35:46.243 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T00:35:46.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.243 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.243 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T00:35:46.243 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.244 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.244 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.244 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T00:35:46.244 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.244 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.244 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.245 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.245 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.245 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.245 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.245 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.245 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.246 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.246 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.246 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.246 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.246 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.246 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.247 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.247 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.247 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.247 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.247 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.247 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.248 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.248 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.248 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.248 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.249 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.249 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.249 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.249 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.249 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.249 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.250 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.250 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.250 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.250 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.250 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.251 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.251 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.251 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.251 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.251 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.252 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.252 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.252 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.252 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.252 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.252 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.253 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.253 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.253 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.253 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.253 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.253 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.253 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.254 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.254 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.254 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.254 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.254 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.254 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.254 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.255 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.255 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.255 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.255 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.255 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-09-21T00:35:46.255 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-09-21T00:35:46.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.256 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.256 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.256 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.256 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.256 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.256 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.257 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.258 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.258 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-09-21T00:35:46.258 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-09-21T00:35:46.258 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.258 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.258 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.258 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.258 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.258 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.258 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.258 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-09-21T00:35:46.258 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.259 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.259 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.259 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-09-21T00:35:46.259 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-09-21T00:35:46.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.259 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.259 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.259 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.259 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.259 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.259 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-09-21T00:35:46.259 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.259 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.260 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.260 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-09-21T00:35:46.260 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-09-21T00:35:46.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.260 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.260 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-09-21T00:35:46.260 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.260 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.260 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.260 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-09-21T00:35:46.260 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.260 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.260 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.261 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-09-21T00:35:46.261 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-09-21T00:35:46.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.261 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.261 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.261 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.261 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.261 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.261 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-09-21T00:35:46.261 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.261 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.262 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-09-21T00:35:46.262 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-09-21T00:35:46.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.262 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.262 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.262 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.262 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.262 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.262 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T00:35:46.262 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.262 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.262 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.263 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-09-21T00:35:46.263 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-09-21T00:35:46.263 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.263 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.263 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.263 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.263 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.263 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.263 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-09-21T00:35:46.263 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.263 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.263 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.264 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.264 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.264 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.264 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.264 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.264 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.264 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.264 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.264 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.264 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.264 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.264 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.264 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-09-21T00:35:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-09-21T00:35:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-09-21T00:35:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.266 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.266 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.266 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T00:35:46.266 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.266 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.266 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.266 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-09-21T00:35:46.266 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-09-21T00:35:46.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.266 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.266 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.266 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-09-21T00:35:46.267 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.267 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.267 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.267 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T00:35:46.267 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T00:35:46.267 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.267 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.267 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-09-21T00:35:46.267 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-09-21T00:35:46.267 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T00:35:46.267 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.267 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.267 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.268 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T00:35:46.268 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T00:35:46.268 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.268 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T00:35:46.268 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T00:35:46.268 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.268 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.268 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-09-21T00:35:46.268 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-09-21T00:35:46.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T00:35:46.268 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.268 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-09-21T00:35:46.269 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T00:35:46.269 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-09-21T00:35:46.269 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.269 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T00:35:46.269 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.269 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.269 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.269 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-09-21T00:35:46.269 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-09-21T00:35:46.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.269 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.269 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.269 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T00:35:46.269 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.270 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.270 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.270 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T00:35:46.270 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T00:35:46.270 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.270 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.270 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-09-21T00:35:46.270 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-09-21T00:35:46.270 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:46.270 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.270 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.270 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.271 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.272 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.272 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T00:35:46.272 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T00:35:46.272 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.272 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.272 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-09-21T00:35:46.272 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-09-21T00:35:46.272 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T00:35:46.272 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T00:35:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T00:35:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T00:35:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T00:35:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.274 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.274 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.274 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.274 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.274 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.274 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.274 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.274 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.274 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.274 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.274 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.274 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.274 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.274 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.274 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.275 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.275 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.275 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.275 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.275 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.275 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.275 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.275 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.275 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.275 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.275 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.275 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.275 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.275 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.276 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.276 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.276 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.276 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.276 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.276 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.276 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.276 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.276 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.276 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.276 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.276 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.276 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.276 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.277 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.277 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.277 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.277 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.277 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.277 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.277 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.277 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.277 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.277 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.277 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.278 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.278 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.278 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.278 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.278 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.278 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-09-21T00:35:46.278 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-09-21T00:35:46.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:46.278 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.279 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.279 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.279 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.279 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.279 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T00:35:46.279 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.279 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.279 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.279 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T00:35:46.279 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T00:35:46.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.279 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.280 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-09-21T00:35:46.280 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.280 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.280 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.280 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-09-21T00:35:46.280 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-09-21T00:35:46.280 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-09-21T00:35:46.280 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-09-21T00:35:46.280 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-09-21T00:35:46.280 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.280 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-09-21T00:35:46.280 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.281 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.281 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.281 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-09-21T00:35:46.281 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-09-21T00:35:46.281 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.281 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.281 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.281 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.281 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.281 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.281 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.281 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T00:35:46.281 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.282 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.282 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.282 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T00:35:46.282 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T00:35:46.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.282 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.282 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.282 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T00:35:46.282 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.282 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.282 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.282 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T00:35:46.282 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.283 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.283 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.283 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.283 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.283 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.283 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.283 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.283 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.283 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.283 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.284 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.284 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.284 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-09-21T00:35:46.284 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-09-21T00:35:46.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.284 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.284 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.284 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T00:35:46.284 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T00:35:46.284 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.284 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.284 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T00:35:46.284 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T00:35:46.284 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.285 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.285 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-09-21T00:35:46.285 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-09-21T00:35:46.285 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T00:35:46.285 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.285 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:46.285 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.285 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.285 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.285 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T00:35:46.285 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.285 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.285 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.286 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.286 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.286 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-09-21T00:35:46.286 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.286 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.286 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.286 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-09-21T00:35:46.286 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-09-21T00:35:46.286 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:46.286 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.286 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.286 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T00:35:46.286 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.287 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.287 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.287 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T00:35:46.287 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.287 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.287 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-09-21T00:35:46.287 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-09-21T00:35:46.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.287 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.287 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-09-21T00:35:46.287 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.288 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T00:35:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T00:35:46.290 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.290 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.290 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-09-21T00:35:46.290 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-09-21T00:35:46.290 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.290 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.290 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.291 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T00:35:46.291 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.291 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.291 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.291 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T00:35:46.291 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.292 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.292 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-09-21T00:35:46.292 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T00:35:46.292 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:46.292 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.293 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.293 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-09-21T00:35:46.293 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.293 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.293 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.293 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T00:35:46.293 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.294 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.294 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.294 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-09-21T00:35:46.294 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-09-21T00:35:46.294 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.294 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.295 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.295 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.295 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.295 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.295 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.295 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-09-21T00:35:46.296 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-09-21T00:35:46.296 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-09-21T00:35:46.296 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.296 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T00:35:46.296 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.296 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.297 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.297 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-09-21T00:35:46.297 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-09-21T00:35:46.297 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.297 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.298 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.298 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.298 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.298 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.298 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.298 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T00:35:46.299 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.299 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.299 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-09-21T00:35:46.299 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-09-21T00:35:46.300 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.300 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.300 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.300 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.300 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.300 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.300 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.301 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T00:35:46.301 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.301 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.301 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-09-21T00:35:46.302 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-09-21T00:35:46.302 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.302 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.302 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.302 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.302 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.303 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.303 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.303 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T00:35:46.303 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.303 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.303 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.304 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-09-21T00:35:46.304 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-09-21T00:35:46.304 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.304 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.304 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.304 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T00:35:46.305 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.305 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.305 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.305 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T00:35:46.305 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.305 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.306 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.306 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-09-21T00:35:46.306 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-09-21T00:35:46.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.306 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.307 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T00:35:46.307 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.307 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.307 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.307 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-09-21T00:35:46.308 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.308 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.308 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-09-21T00:35:46.308 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-09-21T00:35:46.309 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.309 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.309 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.309 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T00:35:46.309 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.309 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.309 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.310 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-09-21T00:35:46.310 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.310 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.310 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.310 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-09-21T00:35:46.311 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-09-21T00:35:46.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.311 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.311 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.311 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T00:35:46.311 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.312 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.312 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.312 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T00:35:46.312 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.312 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.312 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-09-21T00:35:46.312 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-09-21T00:35:46.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.313 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.313 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.313 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T00:35:46.313 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.313 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.313 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.313 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-09-21T00:35:46.314 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.314 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.314 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-09-21T00:35:46.314 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-09-21T00:35:46.314 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.314 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.314 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.315 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T00:35:46.315 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.315 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.315 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.315 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-09-21T00:35:46.315 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.315 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.315 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.316 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-09-21T00:35:46.316 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-09-21T00:35:46.316 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.316 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.316 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.316 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T00:35:46.316 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.316 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.316 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.317 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-09-21T00:35:46.317 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.317 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.317 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.317 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-09-21T00:35:46.317 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-09-21T00:35:46.317 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.317 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.318 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.318 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T00:35:46.318 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.318 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.318 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.318 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-09-21T00:35:46.318 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.319 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.319 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.319 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-09-21T00:35:46.319 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-09-21T00:35:46.319 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.319 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.320 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.320 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T00:35:46.320 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.320 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.320 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.320 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-09-21T00:35:46.321 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.321 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.321 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.321 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-09-21T00:35:46.321 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-09-21T00:35:46.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.322 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.322 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T00:35:46.322 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.322 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.322 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.323 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-09-21T00:35:46.323 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.323 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.323 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nginx": { 2024-09-21T00:35:46.323 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nginx", 2024-09-21T00:35:46.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.324 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.324 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.324 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T00:35:46.324 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.325 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.325 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.325 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nginx container image", 2024-09-21T00:35:46.325 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.325 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.325 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.326 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-09-21T00:35:46.326 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-09-21T00:35:46.326 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.326 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.327 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T00:35:46.327 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.327 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.327 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.327 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T00:35:46.327 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.328 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.328 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-09-21T00:35:46.328 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-09-21T00:35:46.328 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.328 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.329 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.329 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T00:35:46.329 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.329 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.329 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.329 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-09-21T00:35:46.330 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.330 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.330 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.330 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_oauth2_proxy": { 2024-09-21T00:35:46.330 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T00:35:46.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.331 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.331 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.331 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T00:35:46.331 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.331 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.332 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.332 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "oauth2-proxy container image", 2024-09-21T00:35:46.332 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.332 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.333 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-09-21T00:35:46.333 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-09-21T00:35:46.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.333 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.333 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.333 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T00:35:46.333 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.333 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.334 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.334 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T00:35:46.334 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.334 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.334 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.334 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-09-21T00:35:46.335 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-09-21T00:35:46.335 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.335 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.335 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.335 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T00:35:46.335 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.335 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.336 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.336 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-09-21T00:35:46.336 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.336 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.336 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.336 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-09-21T00:35:46.336 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-09-21T00:35:46.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.336 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.336 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.336 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T00:35:46.336 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.336 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.337 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.337 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-09-21T00:35:46.337 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.337 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.337 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.337 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba_metrics": { 2024-09-21T00:35:46.337 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba_metrics", 2024-09-21T00:35:46.337 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.337 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.337 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.337 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T00:35:46.337 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.337 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.338 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.338 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T00:35:46.338 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.338 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.338 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.338 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-09-21T00:35:46.338 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-09-21T00:35:46.338 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.338 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.338 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.338 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T00:35:46.338 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.338 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.339 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.339 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-09-21T00:35:46.339 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.339 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.339 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.339 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-09-21T00:35:46.339 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-09-21T00:35:46.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.339 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.339 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.339 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.339 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.340 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.340 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T00:35:46.340 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.340 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.340 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.340 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-09-21T00:35:46.340 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-09-21T00:35:46.340 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.340 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.340 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.340 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T00:35:46.340 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.340 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.341 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.341 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T00:35:46.341 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.341 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.341 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.341 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-09-21T00:35:46.341 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T00:35:46.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.341 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.341 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-09-21T00:35:46.341 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.341 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.341 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.342 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T00:35:46.342 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.342 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.342 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.342 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-09-21T00:35:46.342 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-09-21T00:35:46.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.342 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.342 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-09-21T00:35:46.342 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.342 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.342 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.343 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T00:35:46.343 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.343 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.343 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.343 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-09-21T00:35:46.343 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-09-21T00:35:46.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.343 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.343 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.343 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-09-21T00:35:46.343 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.343 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.343 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.343 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-09-21T00:35:46.344 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.344 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.344 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.344 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-09-21T00:35:46.344 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-09-21T00:35:46.344 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.344 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.344 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.344 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.345 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.345 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.345 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.345 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T00:35:46.345 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.345 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.346 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.346 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-09-21T00:35:46.346 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-09-21T00:35:46.346 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.346 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.347 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.347 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T00:35:46.347 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.347 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.347 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.347 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-09-21T00:35:46.347 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.348 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.348 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.348 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-09-21T00:35:46.348 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-09-21T00:35:46.348 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.348 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.349 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.349 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T00:35:46.349 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.349 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.349 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.349 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T00:35:46.349 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.350 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.350 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.350 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-09-21T00:35:46.350 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-09-21T00:35:46.350 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.351 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.351 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.351 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T00:35:46.351 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.351 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.351 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.352 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-09-21T00:35:46.352 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.352 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.352 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.352 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-09-21T00:35:46.352 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-09-21T00:35:46.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.353 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.353 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.353 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.353 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.353 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.354 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T00:35:46.354 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.354 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.354 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.354 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-09-21T00:35:46.354 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-09-21T00:35:46.355 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.355 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.355 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.355 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.355 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.355 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.355 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.356 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T00:35:46.356 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.356 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.356 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.356 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.358 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.358 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.358 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.358 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.358 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.358 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.358 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.359 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.359 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.359 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.359 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.359 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-09-21T00:35:46.360 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-09-21T00:35:46.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.360 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.360 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.360 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.360 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.361 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.361 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T00:35:46.361 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.361 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.361 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.362 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.362 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.362 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.362 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.363 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.363 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.363 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.363 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T00:35:46.363 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.363 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.363 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.364 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.364 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.364 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.364 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.364 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.365 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.365 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.365 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.365 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.365 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.365 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.366 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.366 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.366 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.366 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.366 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.366 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.367 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.367 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.367 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.367 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.368 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.368 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.368 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.368 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.368 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.369 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.369 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.369 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.369 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T00:35:46.369 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T00:35:46.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T00:35:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T00:35:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T00:35:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-09-21T00:35:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T00:35:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-09-21T00:35:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-09-21T00:35:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-09-21T00:35:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-09-21T00:35:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-09-21T00:35:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T00:35:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-09-21T00:35:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-09-21T00:35:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-09-21T00:35:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T00:35:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T00:35:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-09-21T00:35:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-09-21T00:35:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-09-21T00:35:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T00:35:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-09-21T00:35:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-09-21T00:35:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.376 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.376 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-09-21T00:35:46.376 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.376 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.376 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T00:35:46.376 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.376 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.376 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.376 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-09-21T00:35:46.376 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-09-21T00:35:46.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.376 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.377 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.377 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T00:35:46.377 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.377 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.377 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.377 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T00:35:46.377 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.377 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.377 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.377 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-09-21T00:35:46.377 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-09-21T00:35:46.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.377 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.377 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.378 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.378 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.378 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.378 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.378 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T00:35:46.378 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.378 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.378 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.378 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-09-21T00:35:46.378 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-09-21T00:35:46.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.378 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.379 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.379 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.379 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.379 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.379 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T00:35:46.379 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.379 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.379 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-09-21T00:35:46.379 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-09-21T00:35:46.379 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.379 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.379 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.379 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.380 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.380 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.380 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T00:35:46.380 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.380 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.380 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.380 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-09-21T00:35:46.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-09-21T00:35:46.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.380 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.380 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.380 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.381 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.381 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.381 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T00:35:46.381 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.381 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.381 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.381 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-09-21T00:35:46.381 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-09-21T00:35:46.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.381 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.381 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.381 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.381 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.381 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.382 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.382 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T00:35:46.382 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.382 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.382 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.382 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-09-21T00:35:46.382 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-09-21T00:35:46.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.382 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.382 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.382 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-09-21T00:35:46.382 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.382 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.382 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.383 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-09-21T00:35:46.383 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.383 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.383 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.383 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.383 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.383 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.383 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.383 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.383 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.383 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.383 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.383 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.384 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.384 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.384 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-09-21T00:35:46.384 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-09-21T00:35:46.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.384 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.384 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.384 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.384 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.384 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.384 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T00:35:46.385 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.385 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.385 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.385 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_count_max": { 2024-09-21T00:35:46.385 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T00:35:46.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.385 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.385 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T00:35:46.385 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.385 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.385 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T00:35:46.386 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.386 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.386 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.386 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_interval": { 2024-09-21T00:35:46.386 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_interval", 2024-09-21T00:35:46.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.386 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.386 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T00:35:46.386 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.386 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.386 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.386 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T00:35:46.386 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.387 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.387 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.387 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-09-21T00:35:46.387 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-09-21T00:35:46.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.387 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.387 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.387 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.387 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.387 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.387 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.387 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T00:35:46.387 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.387 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.388 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.388 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-09-21T00:35:46.388 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-09-21T00:35:46.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.388 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.388 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.388 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.388 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.388 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.388 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.388 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T00:35:46.388 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.389 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.389 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-09-21T00:35:46.389 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-09-21T00:35:46.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.389 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.389 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.389 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.389 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.389 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T00:35:46.389 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.389 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.389 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.390 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-09-21T00:35:46.390 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-09-21T00:35:46.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.390 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.390 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.390 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.390 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.390 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.390 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.390 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T00:35:46.390 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.390 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.390 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.391 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-09-21T00:35:46.391 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-09-21T00:35:46.391 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.391 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.391 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.391 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.391 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.391 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.391 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.391 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T00:35:46.391 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.391 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.391 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.391 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.392 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.392 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-09-21T00:35:46.392 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.392 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.392 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.392 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.392 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.392 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.392 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.392 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.393 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.393 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.393 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.393 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.393 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.393 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.393 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.393 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.393 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.393 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.393 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.393 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.393 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.394 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.394 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.394 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.394 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.394 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.394 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.394 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.394 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.394 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.394 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.394 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.395 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.395 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.395 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.395 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.395 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.395 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.395 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.395 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.395 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.395 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.395 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.395 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.395 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.395 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.396 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.397 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.397 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.397 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-09-21T00:35:46.397 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-09-21T00:35:46.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.397 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.397 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.397 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-09-21T00:35:46.397 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.397 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.397 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.397 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T00:35:46.397 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.397 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.398 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.398 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.398 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.398 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.398 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.398 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.398 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.398 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.398 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.398 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.398 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.398 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.398 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.399 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.399 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-09-21T00:35:46.399 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-09-21T00:35:46.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.399 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.399 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.399 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-09-21T00:35:46.399 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.399 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.399 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.399 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T00:35:46.399 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.399 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:46.400 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.401 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.401 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.401 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.401 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.401 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.401 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.401 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T00:35:46.401 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T00:35:46.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.401 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.401 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.401 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.401 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.401 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.402 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.402 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.402 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.402 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.402 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.402 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T00:35:46.402 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T00:35:46.402 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.402 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.402 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.402 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.402 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.402 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.402 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:45.202+0000 7fd025c00640 1 -- 172.21.15.27:0/1107284538 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd0040b6700 msgr2=0x7fd0040b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:46.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:45.202+0000 7fd025c00640 1 --2- 172.21.15.27:0/1107284538 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd0040b6700 0x7fd0040b8c10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd0100058c0 tx=0x7fd01000a4c0 comp rx=0 tx=0).stop 2024-09-21T00:35:46.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:45.202+0000 7fd025c00640 1 -- 172.21.15.27:0/1107284538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020108a40 msgr2=0x7fd0201a1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:35:46.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:45.202+0000 7fd025c00640 1 --2- 172.21.15.27:0/1107284538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020108a40 0x7fd0201a1ca0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd018009db0 tx=0x7fd01800e490 comp rx=0 tx=0).stop 2024-09-21T00:35:46.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:45.202+0000 7fd025c00640 1 -- 172.21.15.27:0/1107284538 shutdown_connections 2024-09-21T00:35:46.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:45.202+0000 7fd025c00640 1 --2- 172.21.15.27:0/1107284538 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd0040b6700 0x7fd0040b8c10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:46.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:45.202+0000 7fd025c00640 1 --2- 172.21.15.27:0/1107284538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020108a40 0x7fd0201a1ca0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:46.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:45.202+0000 7fd025c00640 1 --2- 172.21.15.27:0/1107284538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201080f0 0x7fd02019ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:46.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:45.202+0000 7fd025c00640 1 --2- 172.21.15.27:0/1107284538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0201027c0 0x7fd02019c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:35:46.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:45.202+0000 7fd025c00640 1 -- 172.21.15.27:0/1107284538 >> 172.21.15.27:0/1107284538 conn(0x7fd0200fe1e0 msgr2=0x7fd0200fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:35:46.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:45.202+0000 7fd025c00640 1 -- 172.21.15.27:0/1107284538 shutdown_connections 2024-09-21T00:35:46.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:35:45.202+0000 7fd025c00640 1 -- 172.21.15.27:0/1107284538 wait complete. 2024-09-21T00:35:46.406 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.406 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.406 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.406 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.406 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-09-21T00:35:46.406 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T00:35:46.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.406 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.406 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.406 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.407 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.407 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.407 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.407 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.407 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.407 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.407 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.407 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T00:35:46.407 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T00:35:46.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.407 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.407 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.407 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.408 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.408 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.408 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.408 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.408 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.408 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.408 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.408 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T00:35:46.408 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T00:35:46.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.408 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.408 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.408 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.408 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.409 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.409 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.409 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.409 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.409 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.409 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.409 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T00:35:46.409 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T00:35:46.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.409 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.409 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.409 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.409 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.409 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.410 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.410 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.410 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.410 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.410 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.410 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T00:35:46.410 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T00:35:46.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.410 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.410 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.410 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.410 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.411 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.412 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.412 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.412 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.412 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.412 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T00:35:46.412 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T00:35:46.412 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.412 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.412 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.412 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.412 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.412 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.412 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.412 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.413 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.414 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.414 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T00:35:46.414 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T00:35:46.414 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.414 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.414 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.414 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.414 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.414 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.414 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.414 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.414 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T00:35:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T00:35:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T00:35:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T00:35:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-09-21T00:35:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.420 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.421 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.421 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.421 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.421 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.421 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.421 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.421 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.421 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T00:35:46.421 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T00:35:46.421 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.421 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.421 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.421 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.422 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.422 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.422 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.422 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.422 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.422 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.422 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.422 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T00:35:46.422 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T00:35:46.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.422 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.422 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.422 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.423 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.423 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.423 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.423 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.423 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.423 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.423 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T00:35:46.423 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T00:35:46.423 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.423 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.423 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.423 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.423 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.424 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.424 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.424 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.424 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.424 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.424 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.424 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T00:35:46.424 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T00:35:46.424 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.424 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.424 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.424 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-09-21T00:35:46.424 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.425 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.425 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.425 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.425 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.425 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.425 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.425 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T00:35:46.425 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T00:35:46.425 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.425 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.425 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.425 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-09-21T00:35:46.425 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.425 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.426 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.426 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.426 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.426 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.426 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.426 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T00:35:46.426 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T00:35:46.426 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.426 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.426 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.426 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.426 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.427 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.428 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.429 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.429 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T00:35:46.429 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T00:35:46.429 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.429 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.429 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.429 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.429 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.429 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.429 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.429 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.429 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.429 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.429 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.430 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T00:35:46.430 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T00:35:46.430 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.430 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.430 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.430 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.430 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.430 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.430 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.430 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.430 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.430 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.430 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.431 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T00:35:46.431 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T00:35:46.431 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.431 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.431 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.431 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.431 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.431 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.431 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.431 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.431 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.431 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.431 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.431 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T00:35:46.432 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T00:35:46.432 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.432 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.432 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.432 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.432 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.432 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.432 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.432 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.432 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.432 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.432 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.432 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T00:35:46.432 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T00:35:46.433 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.433 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.433 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.433 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.433 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.433 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.433 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.433 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.433 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.433 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.433 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.433 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T00:35:46.433 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T00:35:46.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.434 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.434 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.434 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.434 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.434 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.434 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.434 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.434 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.434 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.434 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.434 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T00:35:46.434 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T00:35:46.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.435 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.436 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T00:35:46.436 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.436 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.436 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.436 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.436 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.436 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.436 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.436 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T00:35:46.436 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T00:35:46.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.436 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.436 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.436 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.437 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.438 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.438 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.438 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.438 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.438 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.438 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T00:35:46.438 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T00:35:46.438 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.438 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.438 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.438 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-09-21T00:35:46.438 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.438 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.438 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T00:35:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T00:35:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T00:35:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T00:35:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T00:35:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T00:35:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T00:35:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T00:35:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T00:35:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T00:35:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T00:35:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T00:35:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T00:35:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.445 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T00:35:46.445 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T00:35:46.445 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.445 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.445 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.445 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:46.445 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.445 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.445 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.445 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.445 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.445 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.445 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.446 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T00:35:46.446 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T00:35:46.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.446 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.446 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.446 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T00:35:46.446 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.446 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.446 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.446 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.446 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.446 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.446 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.447 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-09-21T00:35:46.447 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-09-21T00:35:46.447 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.447 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.447 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.447 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.447 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.447 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.447 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.447 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.447 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.447 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.447 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.447 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-09-21T00:35:46.448 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-09-21T00:35:46.448 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.448 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.448 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.448 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.448 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.448 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.448 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.448 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.448 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.448 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.448 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.448 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-09-21T00:35:46.448 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-09-21T00:35:46.449 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.449 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.449 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.449 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.449 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.449 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.449 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.449 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-09-21T00:35:46.449 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.449 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.449 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-09-21T00:35:46.449 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-09-21T00:35:46.449 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.450 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.450 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.450 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-09-21T00:35:46.450 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.450 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.450 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.450 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.450 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.450 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.450 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.450 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T00:35:46.450 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T00:35:46.450 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.451 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.453 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-09-21T00:35:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-09-21T00:35:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-09-21T00:35:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-09-21T00:35:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T00:35:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T00:35:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T00:35:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T00:35:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T00:35:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T00:35:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-09-21T00:35:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "sso_oauth2": { 2024-09-21T00:35:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sso_oauth2", 2024-09-21T00:35:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-09-21T00:35:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-09-21T00:35:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-09-21T00:35:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-09-21T00:35:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-09-21T00:35:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-09-21T00:35:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-09-21T00:35:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T00:35:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-09-21T00:35:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-09-21T00:35:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-09-21T00:35:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-09-21T00:35:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-09-21T00:35:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-09-21T00:35:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T00:35:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T00:35:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-09-21T00:35:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-09-21T00:35:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T00:35:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T00:35:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T00:35:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T00:35:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T00:35:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-09-21T00:35:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-09-21T00:35:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-09-21T00:35:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T00:35:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-09-21T00:35:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-09-21T00:35:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-09-21T00:35:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T00:35:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-09-21T00:35:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-09-21T00:35:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-09-21T00:35:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T00:35:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T00:35:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T00:35:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-09-21T00:35:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-09-21T00:35:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-09-21T00:35:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-09-21T00:35:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-09-21T00:35:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-09-21T00:35:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T00:35:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-09-21T00:35:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-09-21T00:35:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-09-21T00:35:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T00:35:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-09-21T00:35:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-09-21T00:35:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-09-21T00:35:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T00:35:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T00:35:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-09-21T00:35:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-09-21T00:35:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T00:35:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-09-21T00:35:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-09-21T00:35:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-09-21T00:35:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-09-21T00:35:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-09-21T00:35:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-09-21T00:35:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-09-21T00:35:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T00:35:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T00:35:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-09-21T00:35:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T00:35:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-09-21T00:35:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-09-21T00:35:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T00:35:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T00:35:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-09-21T00:35:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T00:35:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-09-21T00:35:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-09-21T00:35:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-09-21T00:35:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-09-21T00:35:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-09-21T00:35:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-09-21T00:35:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T00:35:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-09-21T00:35:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-09-21T00:35:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-09-21T00:35:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-09-21T00:35:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T00:35:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-09-21T00:35:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-09-21T00:35:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-09-21T00:35:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-09-21T00:35:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-09-21T00:35:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T00:35:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-09-21T00:35:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-09-21T00:35:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-09-21T00:35:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T00:35:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-09-21T00:35:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-09-21T00:35:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T00:35:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-09-21T00:35:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-09-21T00:35:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-09-21T00:35:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-09-21T00:35:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T00:35:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-09-21T00:35:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-09-21T00:35:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-09-21T00:35:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-09-21T00:35:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-09-21T00:35:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-09-21T00:35:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T00:35:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T00:35:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-09-21T00:35:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-09-21T00:35:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T00:35:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-09-21T00:35:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-09-21T00:35:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T00:35:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-09-21T00:35:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T00:35:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-09-21T00:35:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-09-21T00:35:46.564 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-09-21T00:35:46.564 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.564 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-09-21T00:35:46.564 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.564 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.564 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.564 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.564 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.564 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.564 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.564 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.564 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.565 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.565 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.565 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.565 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.565 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.565 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.565 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.565 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.565 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.565 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-09-21T00:35:46.565 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.565 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.565 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.566 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.566 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.566 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.566 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.566 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.566 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.566 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.566 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.566 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.566 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.566 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.566 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.566 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.566 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.567 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.567 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.567 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.567 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.567 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.567 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.567 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.567 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.567 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.567 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.567 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.567 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.568 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.568 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.568 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.568 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.568 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.568 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.568 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.568 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.568 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.568 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.568 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.568 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.568 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.568 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.569 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.569 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.569 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.569 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.569 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.569 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.569 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.569 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.569 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.569 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.569 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.569 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.569 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.570 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.570 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.570 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.570 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.570 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.570 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.570 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.570 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.570 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.570 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.570 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.570 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.570 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.571 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.571 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.571 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.571 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.571 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.571 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.571 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.571 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.571 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.571 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.571 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.572 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.573 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.573 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.573 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.573 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.573 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.573 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.573 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.573 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.573 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.573 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.573 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.573 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.573 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.574 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.574 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.574 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.574 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.574 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.574 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.574 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.574 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.574 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.574 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.574 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.574 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.574 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-09-21T00:35:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T00:35:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T00:35:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T00:35:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-09-21T00:35:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-09-21T00:35:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T00:35:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-09-21T00:35:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-09-21T00:35:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T00:35:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-09-21T00:35:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-09-21T00:35:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-09-21T00:35:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T00:35:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T00:35:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T00:35:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-09-21T00:35:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-09-21T00:35:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-09-21T00:35:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-09-21T00:35:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T00:35:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T00:35:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T00:35:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-09-21T00:35:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-09-21T00:35:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-09-21T00:35:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-09-21T00:35:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-09-21T00:35:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-09-21T00:35:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T00:35:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T00:35:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.604 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-09-21T00:35:46.604 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-09-21T00:35:46.604 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.604 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.604 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.604 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.604 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.604 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.604 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.604 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.604 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.604 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.604 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T00:35:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T00:35:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-09-21T00:35:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-09-21T00:35:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-09-21T00:35:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-09-21T00:35:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T00:35:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T00:35:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T00:35:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T00:35:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T00:35:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T00:35:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-09-21T00:35:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T00:35:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-09-21T00:35:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-09-21T00:35:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-09-21T00:35:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T00:35:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T00:35:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-09-21T00:35:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-09-21T00:35:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-09-21T00:35:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T00:35:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T00:35:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T00:35:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T00:35:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T00:35:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-09-21T00:35:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-09-21T00:35:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-09-21T00:35:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T00:35:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-09-21T00:35:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T00:35:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-09-21T00:35:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-09-21T00:35:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-09-21T00:35:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-09-21T00:35:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-09-21T00:35:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T00:35:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T00:35:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "max_requests": { 2024-09-21T00:35:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_requests", 2024-09-21T00:35:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T00:35:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T00:35:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T00:35:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T00:35:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-09-21T00:35:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T00:35:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T00:35:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T00:35:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T00:35:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-09-21T00:35:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-09-21T00:35:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-09-21T00:35:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-09-21T00:35:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T00:35:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-09-21T00:35:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-09-21T00:35:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-09-21T00:35:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-09-21T00:35:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-09-21T00:35:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-09-21T00:35:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-09-21T00:35:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-09-21T00:35:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-09-21T00:35:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-09-21T00:35:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-09-21T00:35:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-09-21T00:35:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T00:35:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-09-21T00:35:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-09-21T00:35:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-09-21T00:35:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-09-21T00:35:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-09-21T00:35:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-09-21T00:35:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T00:35:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-09-21T00:35:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-09-21T00:35:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-09-21T00:35:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-09-21T00:35:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-09-21T00:35:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-09-21T00:35:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-09-21T00:35:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smb", 2024-09-21T00:35:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "internal_store_backend": { 2024-09-21T00:35:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "internal_store_backend", 2024-09-21T00:35:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T00:35:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "update_orchestration": { 2024-09-21T00:35:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "update_orchestration", 2024-09-21T00:35:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T00:35:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T00:35:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-09-21T00:35:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-09-21T00:35:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-09-21T00:35:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-09-21T00:35:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-09-21T00:35:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-09-21T00:35:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-09-21T00:35:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T00:35:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T00:35:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T00:35:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-09-21T00:35:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T00:35:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-09-21T00:35:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-09-21T00:35:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T00:35:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-09-21T00:35:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-09-21T00:35:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T00:35:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-09-21T00:35:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-09-21T00:35:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T00:35:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-09-21T00:35:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-09-21T00:35:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T00:35:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-09-21T00:35:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-09-21T00:35:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-09-21T00:35:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-09-21T00:35:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T00:35:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T00:35:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T00:35:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T00:35:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-09-21T00:35:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-09-21T00:35:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-09-21T00:35:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-09-21T00:35:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-09-21T00:35:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-09-21T00:35:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-09-21T00:35:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-09-21T00:35:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-09-21T00:35:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-09-21T00:35:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-09-21T00:35:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-09-21T00:35:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-09-21T00:35:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-09-21T00:35:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-09-21T00:35:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T00:35:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-09-21T00:35:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-21T00:35:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-09-21T00:35:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-09-21T00:35:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-09-21T00:35:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T00:35:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-09-21T00:35:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-09-21T00:35:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-09-21T00:35:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-09-21T00:35:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-09-21T00:35:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T00:35:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-09-21T00:35:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-09-21T00:35:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T00:35:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T00:35:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T00:35:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T00:35:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T00:35:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T00:35:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T00:35:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T00:35:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T00:35:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T00:35:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T00:35:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T00:35:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T00:35:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T00:35:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T00:35:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T00:35:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T00:35:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T00:35:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T00:35:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T00:35:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T00:35:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T00:35:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T00:35:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T00:35:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T00:35:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T00:35:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T00:35:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-09-21T00:35:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-09-21T00:35:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T00:35:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-09-21T00:35:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-09-21T00:35:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T00:35:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-09-21T00:35:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-09-21T00:35:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-09-21T00:35:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T00:35:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T00:35:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T00:35:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T00:35:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T00:35:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T00:35:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T00:35:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T00:35:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T00:35:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T00:35:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T00:35:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "services": { 2024-09-21T00:35:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "dashboard": "https://172.21.15.27:8443/" 2024-09-21T00:35:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "always_on_modules": { 2024-09-21T00:35:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "octopus": [ 2024-09-21T00:35:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-21T00:35:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-21T00:35:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-21T00:35:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-21T00:35:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-21T00:35:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-21T00:35:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-21T00:35:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-21T00:35:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-21T00:35:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "pacific": [ 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "quincy": [ 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-21T00:35:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-21T00:35:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-21T00:35:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-21T00:35:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-21T00:35:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-21T00:35:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-21T00:35:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-21T00:35:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-21T00:35:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-21T00:35:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "reef": [ 2024-09-21T00:35:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-21T00:35:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-21T00:35:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-21T00:35:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-21T00:35:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-21T00:35:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-21T00:35:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-21T00:35:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-21T00:35:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-21T00:35:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-21T00:35:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T00:35:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "squid": [ 2024-09-21T00:35:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-21T00:35:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-21T00:35:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-21T00:35:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-21T00:35:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-21T00:35:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-21T00:35:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-21T00:35:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-21T00:35:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-21T00:35:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-21T00:35:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T00:35:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "last_failure_osd_epoch": 5, 2024-09-21T00:35:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "active_clients": [ 2024-09-21T00:35:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-09-21T00:35:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T00:35:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T00:35:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T00:35:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2210567748 2024-09-21T00:35:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "libcephsqlite", 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 4278830807 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T00:35:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T00:35:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 663091323 2024-09-21T00:35:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T00:35:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-21T00:35:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T00:35:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T00:35:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T00:35:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1037566755 2024-09-21T00:35:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T00:35:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-21T00:35:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T00:35:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T00:35:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T00:35:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2932334074 2024-09-21T00:35:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T00:35:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-21T00:35:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T00:35:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T00:35:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T00:35:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T00:35:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2644715933 2024-09-21T00:35:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T00:35:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T00:35:46.759 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:35:46.766 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T00:35:46.766 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T00:35:46.767 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:45 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/4209162044' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T00:35:46.767 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:46 smithi027 ceph-mon[30745]: pgmap v828: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 49 KiB/s wr, 124 op/s 2024-09-21T00:35:46.767 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:46 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/1107284538' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T00:35:46.767 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:45 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/4209162044' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T00:35:46.767 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:46 smithi098 ceph-mon[34725]: pgmap v828: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 49 KiB/s wr, 124 op/s 2024-09-21T00:35:46.767 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:46 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/1107284538' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T00:35:46.768 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:45 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/4209162044' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T00:35:46.768 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:46 smithi089 ceph-mon[34481]: pgmap v828: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 49 KiB/s wr, 124 op/s 2024-09-21T00:35:46.768 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:46 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/1107284538' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T00:35:47.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:47 smithi089 ceph-mon[34481]: osdmap e111: 12 total, 12 up, 12 in 2024-09-21T00:35:47.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:47 smithi098 ceph-mon[34725]: osdmap e111: 12 total, 12 up, 12 in 2024-09-21T00:35:47.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:47 smithi027 ceph-mon[30745]: osdmap e111: 12 total, 12 up, 12 in 2024-09-21T00:35:48.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:48 smithi089 ceph-mon[34481]: pgmap v830: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 32 KiB/s wr, 75 op/s 2024-09-21T00:35:48.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:48 smithi098 ceph-mon[34725]: pgmap v830: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 32 KiB/s wr, 75 op/s 2024-09-21T00:35:48.802 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:35:48.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:48 smithi027 ceph-mon[30745]: pgmap v830: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 32 KiB/s wr, 75 op/s 2024-09-21T00:35:49.975 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:35:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T00:35:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 70744, 2024-09-21T00:35:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 70670, 2024-09-21T00:35:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 84399837, 2024-09-21T00:35:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 99261043, 2024-09-21T00:35:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 10, 2024-09-21T00:35:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 4, 2024-09-21T00:35:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 13.065002054, 2024-09-21T00:35:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 5.970329913, 2024-09-21T00:35:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 5.695216703, 2024-09-21T00:35:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 2.584848378, 2024-09-21T00:35:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:35:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 70671, 2024-09-21T00:35:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 23.759296927, 2024-09-21T00:35:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000336195 2024-09-21T00:35:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:35:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 109737, 2024-09-21T00:35:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.051728962, 2024-09-21T00:35:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000471 2024-09-21T00:35:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T00:35:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T00:35:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T00:35:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 9573, 2024-09-21T00:35:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 9827, 2024-09-21T00:35:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 3519964, 2024-09-21T00:35:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 67453758, 2024-09-21T00:35:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 11, 2024-09-21T00:35:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 8, 2024-09-21T00:35:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 3.522388202, 2024-09-21T00:35:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.926911313, 2024-09-21T00:35:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.393700719, 2024-09-21T00:35:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 1.973767890, 2024-09-21T00:35:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:35:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 9825, 2024-09-21T00:35:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 3.784947326, 2024-09-21T00:35:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000385236 2024-09-21T00:35:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:35:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 72, 2024-09-21T00:35:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000021684, 2024-09-21T00:35:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000301 2024-09-21T00:35:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 4176, 2024-09-21T00:35:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 464832 2024-09-21T00:35:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T00:35:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 246747, 2024-09-21T00:35:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 316092, 2024-09-21T00:35:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 81546168, 2024-09-21T00:35:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 279533320, 2024-09-21T00:35:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 15, 2024-09-21T00:35:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 8, 2024-09-21T00:35:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 75.750525226, 2024-09-21T00:35:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 61.706804674, 2024-09-21T00:35:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 80.825138643, 2024-09-21T00:35:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 2.446229917, 2024-09-21T00:35:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:35:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 316089, 2024-09-21T00:35:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 237.761282635, 2024-09-21T00:35:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000752197 2024-09-21T00:35:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:35:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 223396, 2024-09-21T00:35:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.198352214, 2024-09-21T00:35:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000887 2024-09-21T00:35:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1003664, 2024-09-21T00:35:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 183184 2024-09-21T00:35:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-09-21T00:35:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 1, 2024-09-21T00:35:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-09-21T00:35:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-MDSRank": { 2024-09-21T00:35:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-09-21T00:35:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-09-21T00:35:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 5103, 2024-09-21T00:35:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 80.798742897, 2024-09-21T00:35:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.015833576 2024-09-21T00:35:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-PurgeQueue": { 2024-09-21T00:35:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-09-21T00:35:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-09-21T00:35:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 26311, 2024-09-21T00:35:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 9.450511163, 2024-09-21T00:35:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000359184 2024-09-21T00:35:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2024-09-21T00:35:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "request": 147728, 2024-09-21T00:35:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "reply": 137759, 2024-09-21T00:35:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "reply_latency": { 2024-09-21T00:35:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 137759, 2024-09-21T00:35:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 7125.636344891, 2024-09-21T00:35:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.051725377 2024-09-21T00:35:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_reply": 0, 2024-09-21T00:35:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "forward": 9911, 2024-09-21T00:35:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_complete": 189, 2024-09-21T00:35:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_keys": 0, 2024-09-21T00:35:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_commit": 399, 2024-09-21T00:35:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_split": 44, 2024-09-21T00:35:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_merge": 0, 2024-09-21T00:35:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes": 31366, 2024-09-21T00:35:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_top": 1243, 2024-09-21T00:35:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_bottom": 8879, 2024-09-21T00:35:49.984 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pin_tail": 21244, 2024-09-21T00:35:49.984 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pinned": 29587, 2024-09-21T00:35:49.984 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_expired": 82124, 2024-09-21T00:35:49.984 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_with_caps": 1, 2024-09-21T00:35:49.984 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": 2, 2024-09-21T00:35:49.984 INFO:teuthology.orchestra.run.smithi027.stdout: "subtrees": 3, 2024-09-21T00:35:49.984 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse": 270040, 2024-09-21T00:35:49.984 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_hit": 197065, 2024-09-21T00:35:49.984 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_forward": 9678, 2024-09-21T00:35:49.984 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_discover": 667, 2024-09-21T00:35:49.984 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_dir_fetch": 32, 2024-09-21T00:35:49.984 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_remote_ino": 0, 2024-09-21T00:35:49.984 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_lock": 488, 2024-09-21T00:35:49.985 INFO:teuthology.orchestra.run.smithi027.stdout: "load_cent": 0, 2024-09-21T00:35:49.985 INFO:teuthology.orchestra.run.smithi027.stdout: "q": 0, 2024-09-21T00:35:49.985 INFO:teuthology.orchestra.run.smithi027.stdout: "exported": 233, 2024-09-21T00:35:49.985 INFO:teuthology.orchestra.run.smithi027.stdout: "exported_inodes": 69087, 2024-09-21T00:35:49.985 INFO:teuthology.orchestra.run.smithi027.stdout: "imported": 154, 2024-09-21T00:35:49.985 INFO:teuthology.orchestra.run.smithi027.stdout: "imported_inodes": 86832, 2024-09-21T00:35:49.985 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_dir_fetch": 0, 2024-09-21T00:35:49.985 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_backtrace_fetch": 0, 2024-09-21T00:35:49.985 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_peer_discover": 0, 2024-09-21T00:35:49.985 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rfiles": 1050, 2024-09-21T00:35:49.985 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rbytes": 23776636, 2024-09-21T00:35:49.985 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rsnaps": 6, 2024-09-21T00:35:49.985 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T00:35:49.985 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_set_tag": 0, 2024-09-21T00:35:49.985 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T00:35:49.986 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_inotable_repaired": 0, 2024-09-21T00:35:49.986 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_inodes": 0, 2024-09-21T00:35:49.986 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T00:35:49.986 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T00:35:49.986 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_file_inodes": 0, 2024-09-21T00:35:49.986 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_inode_file_caps": 346, 2024-09-21T00:35:49.986 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_revoke": 41222, 2024-09-21T00:35:49.986 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_grant": 36384, 2024-09-21T00:35:49.986 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T00:35:49.986 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T00:35:49.986 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T00:35:49.986 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps": 48320, 2024-09-21T00:35:49.986 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps_dirty": 8724, 2024-09-21T00:35:49.986 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_cap_release": 2120, 2024-09-21T00:35:49.987 INFO:teuthology.orchestra.run.smithi027.stdout: "process_request_cap_release": 24355 2024-09-21T00:35:49.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.987 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache": { 2024-09-21T00:35:49.987 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update": 2129, 2024-09-21T00:35:49.987 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update_receipt": 5863, 2024-09-21T00:35:49.987 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_try_discover": 3532, 2024-09-21T00:35:49.987 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_send_discover": 4132, 2024-09-21T00:35:49.987 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_handle_discover": 2498, 2024-09-21T00:35:49.987 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays": 93, 2024-09-21T00:35:49.987 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_delayed": 0, 2024-09-21T00:35:49.987 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_enqueuing": 0, 2024-09-21T00:35:49.987 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_created": 14358, 2024-09-21T00:35:49.987 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_enqueued": 14265, 2024-09-21T00:35:49.987 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_reintegrated": 0, 2024-09-21T00:35:49.987 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_migrated": 0, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_processing": 0, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_enqueued": 0, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_prioritized": 0, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_started": 0, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_completed": 0, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_path": 7, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_inode": 30335, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_exportdir": 234, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_flush": 0, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragmentdir": 44, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragstats": 0, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_inodestats": 0, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_succeeded": 0, 2024-09-21T00:35:49.988 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_write_failed": 0 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log": { 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: "evlrg": 9, 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: "evadd": 64442, 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: "evex": 0, 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: "evtrm": 0, 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: "ev": 64442, 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: "evexg": 0, 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: "evexd": 0, 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: "segadd": 92, 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: "segex": 0, 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: "segtrm": 0, 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: "seg": 92, 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: "segmjr": 7, 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: "segexg": 0, 2024-09-21T00:35:49.989 INFO:teuthology.orchestra.run.smithi027.stdout: "segexd": 0, 2024-09-21T00:35:49.990 INFO:teuthology.orchestra.run.smithi027.stdout: "expos": 4194304, 2024-09-21T00:35:49.990 INFO:teuthology.orchestra.run.smithi027.stdout: "wrpos": 255007474, 2024-09-21T00:35:49.990 INFO:teuthology.orchestra.run.smithi027.stdout: "rdpos": 0, 2024-09-21T00:35:49.990 INFO:teuthology.orchestra.run.smithi027.stdout: "jlat": { 2024-09-21T00:35:49.990 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3213, 2024-09-21T00:35:49.990 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 253.095304918, 2024-09-21T00:35:49.990 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.078772270 2024-09-21T00:35:49.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.990 INFO:teuthology.orchestra.run.smithi027.stdout: "replayed": 0 2024-09-21T00:35:49.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.990 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mem": { 2024-09-21T00:35:49.990 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 17866, 2024-09-21T00:35:49.990 INFO:teuthology.orchestra.run.smithi027.stdout: "ino+": 87959, 2024-09-21T00:35:49.991 INFO:teuthology.orchestra.run.smithi027.stdout: "ino-": 70093, 2024-09-21T00:35:49.991 INFO:teuthology.orchestra.run.smithi027.stdout: "dir": 673, 2024-09-21T00:35:49.991 INFO:teuthology.orchestra.run.smithi027.stdout: "dir+": 4160, 2024-09-21T00:35:49.991 INFO:teuthology.orchestra.run.smithi027.stdout: "dir-": 3487, 2024-09-21T00:35:49.991 INFO:teuthology.orchestra.run.smithi027.stdout: "dn": 31492, 2024-09-21T00:35:49.991 INFO:teuthology.orchestra.run.smithi027.stdout: "dn+": 134601, 2024-09-21T00:35:49.991 INFO:teuthology.orchestra.run.smithi027.stdout: "dn-": 103109, 2024-09-21T00:35:49.991 INFO:teuthology.orchestra.run.smithi027.stdout: "cap": 2, 2024-09-21T00:35:49.991 INFO:teuthology.orchestra.run.smithi027.stdout: "cap+": 27526, 2024-09-21T00:35:49.991 INFO:teuthology.orchestra.run.smithi027.stdout: "cap-": 27524, 2024-09-21T00:35:49.991 INFO:teuthology.orchestra.run.smithi027.stdout: "rss": 278508, 2024-09-21T00:35:49.991 INFO:teuthology.orchestra.run.smithi027.stdout: "heap": 545206272 2024-09-21T00:35:49.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.991 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server": { 2024-09-21T00:35:49.991 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_client_request": 153510, 2024-09-21T00:35:49.992 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_server_request": 1745, 2024-09-21T00:35:49.992 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_request": 147728, 2024-09-21T00:35:49.992 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_session": 293, 2024-09-21T00:35:49.992 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_peer_request": 3355, 2024-09-21T00:35:49.992 INFO:teuthology.orchestra.run.smithi027.stdout: "req_create_latency": { 2024-09-21T00:35:49.992 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 4643, 2024-09-21T00:35:49.992 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 385.959731532, 2024-09-21T00:35:49.992 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.083127230 2024-09-21T00:35:49.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.992 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getattr_latency": { 2024-09-21T00:35:49.992 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 124, 2024-09-21T00:35:49.992 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 32.091928219, 2024-09-21T00:35:49.992 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.258805872 2024-09-21T00:35:49.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.992 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getfilelock_latency": { 2024-09-21T00:35:49.993 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:49.993 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:49.993 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:49.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.993 INFO:teuthology.orchestra.run.smithi027.stdout: "req_link_latency": { 2024-09-21T00:35:49.993 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:49.993 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:49.993 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:49.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.993 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookup_latency": { 2024-09-21T00:35:49.993 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 110079, 2024-09-21T00:35:49.993 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 6002.976387426, 2024-09-21T00:35:49.993 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.054533347 2024-09-21T00:35:49.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.993 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookuphash_latency": { 2024-09-21T00:35:49.994 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:49.994 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:49.994 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:49.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.994 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupino_latency": { 2024-09-21T00:35:49.994 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1, 2024-09-21T00:35:49.994 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000139648, 2024-09-21T00:35:49.994 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000139648 2024-09-21T00:35:49.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.994 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupname_latency": { 2024-09-21T00:35:49.994 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:49.994 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:49.994 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:49.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.994 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupparent_latency": { 2024-09-21T00:35:49.995 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:49.995 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:49.995 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:49.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.995 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupsnap_latency": { 2024-09-21T00:35:49.995 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:49.995 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:49.995 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:49.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.996 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lssnap_latency": { 2024-09-21T00:35:49.996 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:49.996 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:49.996 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:49.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.996 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mkdir_latency": { 2024-09-21T00:35:49.996 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 18, 2024-09-21T00:35:49.996 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 6.431168546, 2024-09-21T00:35:49.996 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.357287141 2024-09-21T00:35:49.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.996 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mknod_latency": { 2024-09-21T00:35:49.996 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:49.996 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:49.996 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:49.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.997 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mksnap_latency": { 2024-09-21T00:35:49.997 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:49.997 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:49.997 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:49.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.997 INFO:teuthology.orchestra.run.smithi027.stdout: "req_open_latency": { 2024-09-21T00:35:49.997 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 4504, 2024-09-21T00:35:49.997 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 157.550068840, 2024-09-21T00:35:49.997 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.034980033 2024-09-21T00:35:49.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.998 INFO:teuthology.orchestra.run.smithi027.stdout: "req_readdir_latency": { 2024-09-21T00:35:49.998 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 648, 2024-09-21T00:35:49.998 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 4.402375578, 2024-09-21T00:35:49.998 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.006793789 2024-09-21T00:35:49.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.998 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rename_latency": { 2024-09-21T00:35:49.998 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 4662, 2024-09-21T00:35:49.998 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 465.156971751, 2024-09-21T00:35:49.998 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.099776270 2024-09-21T00:35:49.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.998 INFO:teuthology.orchestra.run.smithi027.stdout: "req_renamesnap_latency": { 2024-09-21T00:35:49.998 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:49.998 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:49.999 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:49.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.999 INFO:teuthology.orchestra.run.smithi027.stdout: "req_snapdiff_latency": { 2024-09-21T00:35:49.999 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:49.999 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:49.999 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:49.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.999 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmdir_latency": { 2024-09-21T00:35:49.999 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 93, 2024-09-21T00:35:49.999 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 31.708653968, 2024-09-21T00:35:49.999 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.340953268 2024-09-21T00:35:49.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:49.999 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmsnap_latency": { 2024-09-21T00:35:49.999 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:50.000 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:50.000 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:50.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.000 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmxattr_latency": { 2024-09-21T00:35:50.000 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:50.000 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:50.000 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:50.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.000 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setattr_latency": { 2024-09-21T00:35:50.000 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:50.000 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:50.000 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:50.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.000 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setdirlayout_latency": { 2024-09-21T00:35:50.000 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:50.001 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:50.001 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:50.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.001 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setfilelock_latency": { 2024-09-21T00:35:50.001 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:50.001 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:50.001 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:50.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.001 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setlayout_latency": { 2024-09-21T00:35:50.001 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:50.001 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:50.001 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:50.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.001 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setxattr_latency": { 2024-09-21T00:35:50.001 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:50.002 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:50.002 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:50.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.002 INFO:teuthology.orchestra.run.smithi027.stdout: "req_symlink_latency": { 2024-09-21T00:35:50.002 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:50.002 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:50.002 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:50.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.002 INFO:teuthology.orchestra.run.smithi027.stdout: "req_unlink_latency": { 2024-09-21T00:35:50.002 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 12987, 2024-09-21T00:35:50.002 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 39.358919383, 2024-09-21T00:35:50.002 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.003030639 2024-09-21T00:35:50.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.002 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_revoke_eviction": 0, 2024-09-21T00:35:50.002 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_acquisition_throttle": 0, 2024-09-21T00:35:50.003 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getvxattr_latency": { 2024-09-21T00:35:50.003 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:50.003 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:50.003 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:50.003 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:50.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.003 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessions": { 2024-09-21T00:35:50.003 INFO:teuthology.orchestra.run.smithi027.stdout: "session_count": 2, 2024-09-21T00:35:50.003 INFO:teuthology.orchestra.run.smithi027.stdout: "session_add": 3, 2024-09-21T00:35:50.003 INFO:teuthology.orchestra.run.smithi027.stdout: "session_remove": 1, 2024-09-21T00:35:50.003 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_open": 2, 2024-09-21T00:35:50.003 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_stale": 0, 2024-09-21T00:35:50.003 INFO:teuthology.orchestra.run.smithi027.stdout: "total_load": 10323, 2024-09-21T00:35:50.003 INFO:teuthology.orchestra.run.smithi027.stdout: "average_load": 5161, 2024-09-21T00:35:50.003 INFO:teuthology.orchestra.run.smithi027.stdout: "avg_session_uptime": 827, 2024-09-21T00:35:50.004 INFO:teuthology.orchestra.run.smithi027.stdout: "mdthresh_evicted": 0 2024-09-21T00:35:50.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.004 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-09-21T00:35:50.004 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-09-21T00:35:50.004 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-09-21T00:35:50.004 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T00:35:50.004 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-09-21T00:35:50.004 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T00:35:50.004 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-09-21T00:35:50.004 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T00:35:50.004 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T00:35:50.005 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T00:35:50.005 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T00:35:50.005 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T00:35:50.005 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-09-21T00:35:50.005 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T00:35:50.005 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T00:35:50.005 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-09-21T00:35:50.005 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-09-21T00:35:50.005 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-09-21T00:35:50.005 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-09-21T00:35:50.005 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T00:35:50.005 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T00:35:50.005 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T00:35:50.005 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T00:35:50.005 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T00:35:50.006 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-09-21T00:35:50.006 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-09-21T00:35:50.006 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-09-21T00:35:50.006 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T00:35:50.006 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T00:35:50.006 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-09-21T00:35:50.006 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-09-21T00:35:50.006 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-09-21T00:35:50.006 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-09-21T00:35:50.006 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T00:35:50.006 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-09-21T00:35:50.006 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T00:35:50.006 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-09-21T00:35:50.006 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 1608797, 2024-09-21T00:35:50.006 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 15200, 2024-09-21T00:35:50.007 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-09-21T00:35:50.007 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-09-21T00:35:50.007 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-09-21T00:35:50.007 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-09-21T00:35:50.007 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-09-21T00:35:50.007 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-09-21T00:35:50.007 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-09-21T00:35:50.007 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-09-21T00:35:50.007 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 9912, 2024-09-21T00:35:50.007 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 204, 2024-09-21T00:35:50.007 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T00:35:50.007 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-09-21T00:35:50.007 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-09-21T00:35:50.007 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-09-21T00:35:50.007 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 65199427, 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 769077, 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter": { 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: "op_active": 0, 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: "op_laggy": 0, 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send": 33814, 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send_bytes": 261703724, 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: "op_resend": 27, 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: "op_reply": 33814, 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: "op_latency": { 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 33814, 2024-09-21T00:35:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 1463.331338437, 2024-09-21T00:35:50.009 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.043275901 2024-09-21T00:35:50.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.009 INFO:teuthology.orchestra.run.smithi027.stdout: "op_inflight": 0, 2024-09-21T00:35:50.009 INFO:teuthology.orchestra.run.smithi027.stdout: "oplen_avg": { 2024-09-21T00:35:50.009 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 33814, 2024-09-21T00:35:50.009 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 35064 2024-09-21T00:35:50.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.009 INFO:teuthology.orchestra.run.smithi027.stdout: "op": 33814, 2024-09-21T00:35:50.009 INFO:teuthology.orchestra.run.smithi027.stdout: "op_r": 578, 2024-09-21T00:35:50.009 INFO:teuthology.orchestra.run.smithi027.stdout: "op_w": 33236, 2024-09-21T00:35:50.009 INFO:teuthology.orchestra.run.smithi027.stdout: "op_rmw": 0, 2024-09-21T00:35:50.009 INFO:teuthology.orchestra.run.smithi027.stdout: "op_pg": 0, 2024-09-21T00:35:50.009 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_stat": 0, 2024-09-21T00:35:50.009 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_create": 10, 2024-09-21T00:35:50.009 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_read": 389, 2024-09-21T00:35:50.010 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_write": 3605, 2024-09-21T00:35:50.010 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writefull": 204, 2024-09-21T00:35:50.010 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writesame": 0, 2024-09-21T00:35:50.010 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_append": 0, 2024-09-21T00:35:50.010 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_zero": 0, 2024-09-21T00:35:50.010 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_truncate": 44, 2024-09-21T00:35:50.010 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_delete": 28601, 2024-09-21T00:35:50.010 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_mapext": 0, 2024-09-21T00:35:50.010 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_sparse_read": 0, 2024-09-21T00:35:50.010 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_clonerange": 0, 2024-09-21T00:35:50.010 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_getxattr": 5, 2024-09-21T00:35:50.010 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_setxattr": 20, 2024-09-21T00:35:50.010 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_cmpxattr": 0, 2024-09-21T00:35:50.010 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_rmxattr": 0, 2024-09-21T00:35:50.011 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_resetxattrs": 0, 2024-09-21T00:35:50.011 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_call": 0, 2024-09-21T00:35:50.011 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_watch": 0, 2024-09-21T00:35:50.011 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_notify": 0, 2024-09-21T00:35:50.011 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T00:35:50.011 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls": 0, 2024-09-21T00:35:50.011 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls_filter": 0, 2024-09-21T00:35:50.011 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_other": 0, 2024-09-21T00:35:50.011 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_active": 0, 2024-09-21T00:35:50.011 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_send": 0, 2024-09-21T00:35:50.011 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_resend": 0, 2024-09-21T00:35:50.011 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_ping": 0, 2024-09-21T00:35:50.011 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_active": 0, 2024-09-21T00:35:50.011 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_send": 0, 2024-09-21T00:35:50.011 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_resend": 0, 2024-09-21T00:35:50.012 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_active": 0, 2024-09-21T00:35:50.012 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_send": 0, 2024-09-21T00:35:50.012 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_resend": 0, 2024-09-21T00:35:50.012 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_active": 0, 2024-09-21T00:35:50.012 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_send": 0, 2024-09-21T00:35:50.012 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_resend": 0, 2024-09-21T00:35:50.012 INFO:teuthology.orchestra.run.smithi027.stdout: "command_active": 0, 2024-09-21T00:35:50.012 INFO:teuthology.orchestra.run.smithi027.stdout: "command_send": 0, 2024-09-21T00:35:50.012 INFO:teuthology.orchestra.run.smithi027.stdout: "command_resend": 0, 2024-09-21T00:35:50.012 INFO:teuthology.orchestra.run.smithi027.stdout: "map_epoch": 111, 2024-09-21T00:35:50.012 INFO:teuthology.orchestra.run.smithi027.stdout: "map_full": 0, 2024-09-21T00:35:50.012 INFO:teuthology.orchestra.run.smithi027.stdout: "map_inc": 23, 2024-09-21T00:35:50.012 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_sessions": 12, 2024-09-21T00:35:50.012 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_open": 12, 2024-09-21T00:35:50.012 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_close": 0, 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_laggy": 0, 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_wr": 1502, 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_rd": 378, 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_del": 306 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: "oft": { 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_objs": 1, 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_kv_pairs": 0, 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_updates": 0, 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_removes": 0 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: "purge_queue": { 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops": 0, 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops_high_water": 16, 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing": 0, 2024-09-21T00:35:50.013 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_high_water": 8, 2024-09-21T00:35:50.014 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed_ops": 28530, 2024-09-21T00:35:50.014 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed": 14265, 2024-09-21T00:35:50.014 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_item_in_journal": 0 2024-09-21T00:35:50.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.014 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T00:35:50.014 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T00:35:50.014 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T00:35:50.014 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T00:35:50.014 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 327060, 2024-09-21T00:35:50.014 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 143737313, 2024-09-21T00:35:50.014 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T00:35:50.014 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 327060, 2024-09-21T00:35:50.014 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T00:35:50.014 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T00:35:50.014 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 327060, 2024-09-21T00:35:50.015 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 143737313, 2024-09-21T00:35:50.015 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T00:35:50.015 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:50.015 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:50.015 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:50.015 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:50.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.015 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_bytes": { 2024-09-21T00:35:50.015 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T00:35:50.015 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T00:35:50.015 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T00:35:50.015 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T00:35:50.015 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T00:35:50.015 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T00:35:50.015 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 33814, 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 263879065, 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 33814, 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 263879065, 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_ops": { 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 1024, 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T00:35:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T00:35:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T00:35:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T00:35:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 33814, 2024-09-21T00:35:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 33814, 2024-09-21T00:35:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 33814, 2024-09-21T00:35:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 33814, 2024-09-21T00:35:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T00:35:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle": { 2024-09-21T00:35:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T00:35:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-09-21T00:35:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T00:35:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 14265, 2024-09-21T00:35:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 2178365, 2024-09-21T00:35:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T00:35:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 14265, 2024-09-21T00:35:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T00:35:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T00:35:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 390, 2024-09-21T00:35:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 2178365, 2024-09-21T00:35:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T00:35:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle-0x5574a051b450": { 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 64442, 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 250813170, 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 64442, 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 3213, 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 250813170, 2024-09-21T00:35:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T00:35:50.020 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:50.020 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:50.020 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:50.020 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:50.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:50.020 INFO:teuthology.orchestra.run.smithi027.stdout: "trackedop": { 2024-09-21T00:35:50.020 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_ops_count": 30268 2024-09-21T00:35:50.020 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:50.020 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:35:50.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:50 smithi089 ceph-mon[34481]: pgmap v831: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.7 KiB/s wr, 30 op/s 2024-09-21T00:35:50.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:50 smithi098 ceph-mon[34725]: pgmap v831: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.7 KiB/s wr, 30 op/s 2024-09-21T00:35:50.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:50 smithi027 ceph-mon[30745]: pgmap v831: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.7 KiB/s wr, 30 op/s 2024-09-21T00:35:51.608 INFO:tasks.check_counter:Daemon mds.a mds.exported=233 2024-09-21T00:35:51.608 INFO:tasks.check_counter:Daemon mds.a mds.imported=154 2024-09-21T00:35:51.608 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=2129 2024-09-21T00:35:51.608 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=5863 2024-09-21T00:35:51.608 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-09-21T00:35:51.608 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-21T00:35:51.608 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T00:35:51.608 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=44 2024-09-21T00:35:51.608 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T00:35:51.609 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T00:35:52.331 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:35:52.719 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:52 smithi027 ceph-mon[30745]: pgmap v832: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-21T00:35:52.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:52 smithi089 ceph-mon[34481]: pgmap v832: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-21T00:35:52.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:52 smithi098 ceph-mon[34725]: pgmap v832: 129 pgs: 19 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 50 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-21T00:35:54.429 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:35:54.429 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T00:35:54.429 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 4, 2024-09-21T00:35:54.430 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 4, 2024-09-21T00:35:54.430 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 2992, 2024-09-21T00:35:54.430 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 448, 2024-09-21T00:35:54.430 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-09-21T00:35:54.430 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-09-21T00:35:54.430 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.008450440, 2024-09-21T00:35:54.430 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000089031, 2024-09-21T00:35:54.430 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.004963785, 2024-09-21T00:35:54.430 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:35:54.430 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:35:54.430 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3, 2024-09-21T00:35:54.430 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.008788741, 2024-09-21T00:35:54.430 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.002929580 2024-09-21T00:35:54.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:54.431 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:35:54.431 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:54.431 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:54.431 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:54.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:54.431 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T00:35:54.431 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T00:35:54.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:54.431 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T00:35:54.431 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 3, 2024-09-21T00:35:54.432 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 220, 2024-09-21T00:35:54.432 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 480, 2024-09-21T00:35:54.432 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 279968, 2024-09-21T00:35:54.432 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 5, 2024-09-21T00:35:54.432 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 2, 2024-09-21T00:35:54.432 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.034288645, 2024-09-21T00:35:54.432 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.028172570, 2024-09-21T00:35:54.432 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.003209834, 2024-09-21T00:35:54.432 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:35:54.432 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:35:54.432 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 217, 2024-09-21T00:35:54.432 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.035537544, 2024-09-21T00:35:54.432 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000163767 2024-09-21T00:35:54.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:54.433 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:35:54.433 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:54.433 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:54.433 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:54.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:54.433 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-21T00:35:54.433 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 279968 2024-09-21T00:35:54.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:54.433 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T00:35:54.433 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 290, 2024-09-21T00:35:54.433 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 285, 2024-09-21T00:35:54.433 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 857344, 2024-09-21T00:35:54.433 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 176064, 2024-09-21T00:35:54.433 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-09-21T00:35:54.434 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-09-21T00:35:54.434 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.082657051, 2024-09-21T00:35:54.434 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.045204202, 2024-09-21T00:35:54.434 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.047868283, 2024-09-21T00:35:54.434 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:35:54.434 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:35:54.434 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 284, 2024-09-21T00:35:54.434 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.034498263, 2024-09-21T00:35:54.434 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000121472 2024-09-21T00:35:54.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:54.434 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:35:54.435 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:54.435 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:54.435 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:54.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:54.435 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 857344, 2024-09-21T00:35:54.435 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 176064 2024-09-21T00:35:54.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:54.435 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-09-21T00:35:54.435 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-09-21T00:35:54.435 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-09-21T00:35:54.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:54.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-09-21T00:35:54.436 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-09-21T00:35:54.436 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-09-21T00:35:54.436 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T00:35:54.436 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-09-21T00:35:54.436 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T00:35:54.436 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-09-21T00:35:54.436 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T00:35:54.436 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T00:35:54.436 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T00:35:54.436 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T00:35:54.436 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T00:35:54.437 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-09-21T00:35:54.437 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T00:35:54.437 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T00:35:54.437 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-09-21T00:35:54.437 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-09-21T00:35:54.437 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-09-21T00:35:54.437 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-09-21T00:35:54.437 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T00:35:54.437 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T00:35:54.437 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T00:35:54.437 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T00:35:54.437 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T00:35:54.437 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-09-21T00:35:54.437 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-09-21T00:35:54.438 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-09-21T00:35:54.438 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T00:35:54.438 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T00:35:54.438 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-09-21T00:35:54.438 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-09-21T00:35:54.438 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-09-21T00:35:54.438 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-09-21T00:35:54.438 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T00:35:54.438 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-09-21T00:35:54.438 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T00:35:54.438 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-09-21T00:35:54.438 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 147920, 2024-09-21T00:35:54.438 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 27, 2024-09-21T00:35:54.439 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-09-21T00:35:54.439 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-09-21T00:35:54.439 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-09-21T00:35:54.439 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-09-21T00:35:54.439 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-09-21T00:35:54.439 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-09-21T00:35:54.439 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-09-21T00:35:54.439 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-09-21T00:35:54.439 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-09-21T00:35:54.439 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-09-21T00:35:54.439 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T00:35:54.440 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-09-21T00:35:54.440 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-09-21T00:35:54.440 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-09-21T00:35:54.440 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 968, 2024-09-21T00:35:54.440 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-09-21T00:35:54.440 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-09-21T00:35:54.440 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-09-21T00:35:54.440 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-09-21T00:35:54.440 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-09-21T00:35:54.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:54.440 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T00:35:54.440 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T00:35:54.440 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T00:35:54.440 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T00:35:54.441 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 293, 2024-09-21T00:35:54.441 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 816373, 2024-09-21T00:35:54.441 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T00:35:54.441 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 293, 2024-09-21T00:35:54.441 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T00:35:54.441 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T00:35:54.441 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 293, 2024-09-21T00:35:54.441 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 816373, 2024-09-21T00:35:54.441 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T00:35:54.441 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:54.441 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:54.442 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:54.442 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:54.442 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:54.442 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:35:54.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:54 smithi089 ceph-mon[34481]: pgmap v833: 129 pgs: 18 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 51 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-21T00:35:54.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:54 smithi098 ceph-mon[34725]: pgmap v833: 129 pgs: 18 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 51 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-21T00:35:54.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:54 smithi027 ceph-mon[30745]: pgmap v833: 129 pgs: 18 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 51 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-21T00:35:55.267 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-21T00:35:55.268 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-21T00:35:55.268 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-21T00:35:55.268 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-21T00:35:55.268 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-21T00:35:55.268 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-21T00:35:55.268 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-21T00:35:55.268 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-21T00:35:55.268 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T00:35:55.269 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T00:35:55.951 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:35:56.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:56 smithi089 ceph-mon[34481]: pgmap v834: 129 pgs: 18 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 51 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-21T00:35:56.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:56 smithi098 ceph-mon[34725]: pgmap v834: 129 pgs: 18 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 51 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-21T00:35:56.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:56 smithi027 ceph-mon[30745]: pgmap v834: 129 pgs: 18 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 51 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-21T00:35:57.006 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:35:57.006 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T00:35:57.006 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 287, 2024-09-21T00:35:57.006 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 282, 2024-09-21T00:35:57.006 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 856528, 2024-09-21T00:35:57.006 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 174144, 2024-09-21T00:35:57.006 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-09-21T00:35:57.006 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-09-21T00:35:57.006 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.082826781, 2024-09-21T00:35:57.007 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.044020237, 2024-09-21T00:35:57.007 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.044055294, 2024-09-21T00:35:57.007 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:35:57.007 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:35:57.007 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 281, 2024-09-21T00:35:57.007 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.042873188, 2024-09-21T00:35:57.007 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000152573 2024-09-21T00:35:57.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:57.007 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:35:57.007 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:57.007 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:57.007 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:57.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:57.007 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 856528, 2024-09-21T00:35:57.008 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 174144 2024-09-21T00:35:57.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:57.008 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T00:35:57.008 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2, 2024-09-21T00:35:57.008 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 216, 2024-09-21T00:35:57.008 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 320, 2024-09-21T00:35:57.008 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 198848, 2024-09-21T00:35:57.008 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-09-21T00:35:57.008 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 2, 2024-09-21T00:35:57.008 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.033364393, 2024-09-21T00:35:57.008 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.026631644, 2024-09-21T00:35:57.008 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.003834690, 2024-09-21T00:35:57.008 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:35:57.008 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:35:57.008 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 214, 2024-09-21T00:35:57.009 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.036211515, 2024-09-21T00:35:57.009 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000169212 2024-09-21T00:35:57.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:57.009 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:35:57.009 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:57.009 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:57.009 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:57.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:57.009 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T00:35:57.009 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 198848 2024-09-21T00:35:57.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:57.009 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T00:35:57.009 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 4, 2024-09-21T00:35:57.009 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 4, 2024-09-21T00:35:57.009 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 2992, 2024-09-21T00:35:57.010 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 448, 2024-09-21T00:35:57.010 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-09-21T00:35:57.010 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-09-21T00:35:57.010 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.004208899, 2024-09-21T00:35:57.010 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000088540, 2024-09-21T00:35:57.010 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.004055539, 2024-09-21T00:35:57.010 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:35:57.010 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:35:57.010 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3, 2024-09-21T00:35:57.010 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.006848578, 2024-09-21T00:35:57.010 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.002282859 2024-09-21T00:35:57.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:57.010 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:35:57.010 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:57.010 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:57.011 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:57.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:57.011 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T00:35:57.011 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T00:35:57.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:57.011 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-09-21T00:35:57.011 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-09-21T00:35:57.011 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-09-21T00:35:57.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:57.011 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-09-21T00:35:57.011 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-09-21T00:35:57.011 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-09-21T00:35:57.011 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T00:35:57.011 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-09-21T00:35:57.011 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T00:35:57.012 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-09-21T00:35:57.012 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T00:35:57.012 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T00:35:57.012 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T00:35:57.012 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T00:35:57.012 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T00:35:57.012 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-09-21T00:35:57.012 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T00:35:57.012 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T00:35:57.012 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-09-21T00:35:57.012 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-09-21T00:35:57.012 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-09-21T00:35:57.012 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-09-21T00:35:57.013 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T00:35:57.013 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T00:35:57.013 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T00:35:57.013 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T00:35:57.013 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T00:35:57.013 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-09-21T00:35:57.013 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-09-21T00:35:57.013 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-09-21T00:35:57.013 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T00:35:57.013 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T00:35:57.013 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-09-21T00:35:57.013 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-09-21T00:35:57.013 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-09-21T00:35:57.013 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-09-21T00:35:57.013 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T00:35:57.014 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-09-21T00:35:57.014 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T00:35:57.014 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-09-21T00:35:57.014 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 148951, 2024-09-21T00:35:57.014 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 28, 2024-09-21T00:35:57.014 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-09-21T00:35:57.014 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-09-21T00:35:57.014 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-09-21T00:35:57.014 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-09-21T00:35:57.014 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-09-21T00:35:57.014 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-09-21T00:35:57.014 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-09-21T00:35:57.014 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-09-21T00:35:57.014 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-09-21T00:35:57.014 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-09-21T00:35:57.015 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T00:35:57.015 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-09-21T00:35:57.015 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-09-21T00:35:57.015 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-09-21T00:35:57.015 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 968, 2024-09-21T00:35:57.015 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-09-21T00:35:57.015 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-09-21T00:35:57.015 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-09-21T00:35:57.015 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-09-21T00:35:57.015 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-09-21T00:35:57.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:35:57.015 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T00:35:57.015 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T00:35:57.015 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T00:35:57.015 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T00:35:57.016 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 289, 2024-09-21T00:35:57.016 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 815956, 2024-09-21T00:35:57.016 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T00:35:57.016 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 289, 2024-09-21T00:35:57.016 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T00:35:57.016 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T00:35:57.016 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 289, 2024-09-21T00:35:57.016 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 815956, 2024-09-21T00:35:57.016 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T00:35:57.016 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:35:57.016 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:35:57.016 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:35:57.016 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:57.016 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:35:57.017 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:35:58.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:35:58 smithi089 ceph-mon[34481]: pgmap v835: 129 pgs: 18 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 51 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:35:58.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:35:58 smithi098 ceph-mon[34725]: pgmap v835: 129 pgs: 18 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 51 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:35:58.747 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:35:58 smithi027 ceph-mon[30745]: pgmap v835: 129 pgs: 18 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 51 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T00:35:58.819 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-21T00:35:58.819 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-21T00:35:58.819 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-21T00:35:58.819 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-21T00:35:58.819 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-21T00:35:58.819 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-21T00:35:58.819 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-21T00:35:58.819 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-09-21T00:35:58.819 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T00:35:58.820 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T00:35:59.533 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:36:00.566 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T00:36:00.567 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T00:36:00.567 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 4, 2024-09-21T00:36:00.567 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 4, 2024-09-21T00:36:00.567 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 2992, 2024-09-21T00:36:00.567 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 448, 2024-09-21T00:36:00.567 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-09-21T00:36:00.567 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-09-21T00:36:00.567 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.007095857, 2024-09-21T00:36:00.567 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000062876, 2024-09-21T00:36:00.567 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.004242341, 2024-09-21T00:36:00.567 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:36:00.567 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:00.567 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3, 2024-09-21T00:36:00.567 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.007451317, 2024-09-21T00:36:00.568 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.002483772 2024-09-21T00:36:00.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:36:00.568 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:00.568 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:36:00.568 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:36:00.568 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:36:00.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:36:00.568 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T00:36:00.568 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T00:36:00.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:36:00.568 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T00:36:00.568 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2, 2024-09-21T00:36:00.568 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 221, 2024-09-21T00:36:00.568 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 320, 2024-09-21T00:36:00.569 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 199728, 2024-09-21T00:36:00.569 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-09-21T00:36:00.569 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 2, 2024-09-21T00:36:00.569 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.035176162, 2024-09-21T00:36:00.569 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.029894393, 2024-09-21T00:36:00.569 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.002418383, 2024-09-21T00:36:00.569 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:36:00.569 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:00.569 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 219, 2024-09-21T00:36:00.569 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.030459048, 2024-09-21T00:36:00.569 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000139082 2024-09-21T00:36:00.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:36:00.569 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:00.569 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:36:00.569 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:36:00.570 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:36:00.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:36:00.570 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T00:36:00.570 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 199728 2024-09-21T00:36:00.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:36:00.570 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T00:36:00.570 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 294, 2024-09-21T00:36:00.570 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 289, 2024-09-21T00:36:00.570 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 859056, 2024-09-21T00:36:00.570 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 178320, 2024-09-21T00:36:00.570 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-09-21T00:36:00.570 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-09-21T00:36:00.570 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.066911916, 2024-09-21T00:36:00.570 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.025870177, 2024-09-21T00:36:00.571 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.046714553, 2024-09-21T00:36:00.571 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:36:00.571 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:00.571 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 288, 2024-09-21T00:36:00.571 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.053500458, 2024-09-21T00:36:00.571 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000185765 2024-09-21T00:36:00.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:36:00.571 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:00.571 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:36:00.571 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:36:00.571 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:36:00.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:36:00.571 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 859056, 2024-09-21T00:36:00.571 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 178320 2024-09-21T00:36:00.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T00:36:00.572 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T00:36:00.573 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-09-21T00:36:00.573 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T00:36:00.573 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T00:36:00.573 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-09-21T00:36:00.573 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-09-21T00:36:00.573 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-09-21T00:36:00.573 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-09-21T00:36:00.573 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T00:36:00.573 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T00:36:00.573 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T00:36:00.573 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T00:36:00.573 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T00:36:00.573 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-09-21T00:36:00.573 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-09-21T00:36:00.574 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-09-21T00:36:00.574 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T00:36:00.574 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T00:36:00.574 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-09-21T00:36:00.574 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-09-21T00:36:00.574 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-09-21T00:36:00.574 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-09-21T00:36:00.574 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T00:36:00.574 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-09-21T00:36:00.574 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T00:36:00.574 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-09-21T00:36:00.574 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 149982, 2024-09-21T00:36:00.574 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 29, 2024-09-21T00:36:00.574 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-09-21T00:36:00.574 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-09-21T00:36:00.575 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-09-21T00:36:00.575 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-09-21T00:36:00.575 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-09-21T00:36:00.575 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-09-21T00:36:00.575 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-09-21T00:36:00.575 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-09-21T00:36:00.575 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-09-21T00:36:00.575 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-09-21T00:36:00.575 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T00:36:00.575 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-09-21T00:36:00.575 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-09-21T00:36:00.575 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-09-21T00:36:00.575 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 968, 2024-09-21T00:36:00.575 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-09-21T00:36:00.575 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 296, 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 817503, 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 296, 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 296, 2024-09-21T00:36:00.576 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 817503, 2024-09-21T00:36:00.577 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T00:36:00.577 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T00:36:00.577 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T00:36:00.577 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T00:36:00.577 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:36:00.577 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T00:36:00.577 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T00:36:00.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:00 smithi089 ceph-mon[34481]: pgmap v836: 129 pgs: 18 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 51 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:36:00.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:00 smithi098 ceph-mon[34725]: pgmap v836: 129 pgs: 18 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 51 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:36:00.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:00 smithi027 ceph-mon[30745]: pgmap v836: 129 pgs: 18 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 51 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T00:36:02.008 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-21T00:36:02.008 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-21T00:36:02.008 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-21T00:36:02.008 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-21T00:36:02.008 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-21T00:36:02.008 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-21T00:36:02.008 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-21T00:36:02.008 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-21T00:36:02.009 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T00:36:02.009 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T00:36:02.346 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T00:36:02.419 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:02 smithi089 ceph-mon[34481]: pgmap v837: 129 pgs: 16 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 57 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:36:02.419 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:02 smithi089 ceph-mon[34481]: osdmap e112: 12 total, 12 up, 12 in 2024-09-21T00:36:02.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:02 smithi098 ceph-mon[34725]: pgmap v837: 129 pgs: 16 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 57 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:36:02.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:02 smithi098 ceph-mon[34725]: osdmap e112: 12 total, 12 up, 12 in 2024-09-21T00:36:02.810 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:02 smithi027 ceph-mon[30745]: pgmap v837: 129 pgs: 16 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 57 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T00:36:02.811 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:02 smithi027 ceph-mon[30745]: osdmap e112: 12 total, 12 up, 12 in 2024-09-21T00:36:03.005 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T00:36:03.005 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T00:36:03.005 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 78048, 2024-09-21T00:36:03.005 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 81060, 2024-09-21T00:36:03.005 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 73527966, 2024-09-21T00:36:03.006 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 283161808, 2024-09-21T00:36:03.006 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 12, 2024-09-21T00:36:03.006 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 4, 2024-09-21T00:36:03.006 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 7.702091972, 2024-09-21T00:36:03.006 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 3.574729145, 2024-09-21T00:36:03.006 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 2.316283444, 2024-09-21T00:36:03.006 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 1.986141868, 2024-09-21T00:36:03.006 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:03.006 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 81061, 2024-09-21T00:36:03.006 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 51.649880063, 2024-09-21T00:36:03.006 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000637172 2024-09-21T00:36:03.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.006 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:03.006 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 63822, 2024-09-21T00:36:03.007 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.011969339, 2024-09-21T00:36:03.007 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000187 2024-09-21T00:36:03.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 451152, 2024-09-21T00:36:03.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 145760 2024-09-21T00:36:03.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.007 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T00:36:03.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 318461, 2024-09-21T00:36:03.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 463659, 2024-09-21T00:36:03.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 106783284, 2024-09-21T00:36:03.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 409878527, 2024-09-21T00:36:03.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 145, 2024-09-21T00:36:03.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 10, 2024-09-21T00:36:03.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 21.463704445, 2024-09-21T00:36:03.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 12.427028854, 2024-09-21T00:36:03.008 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 10.049671382, 2024-09-21T00:36:03.008 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 1.654722441, 2024-09-21T00:36:03.008 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:03.008 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 463654, 2024-09-21T00:36:03.008 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 58372.377939156, 2024-09-21T00:36:03.008 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.125896418 2024-09-21T00:36:03.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.008 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:03.008 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 309907, 2024-09-21T00:36:03.008 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.082829395, 2024-09-21T00:36:03.008 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000267 2024-09-21T00:36:03.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.008 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 52720, 2024-09-21T00:36:03.008 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 587232 2024-09-21T00:36:03.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.009 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T00:36:03.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 78315, 2024-09-21T00:36:03.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 78865, 2024-09-21T00:36:03.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 37940450, 2024-09-21T00:36:03.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 151751680, 2024-09-21T00:36:03.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 25, 2024-09-21T00:36:03.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 6, 2024-09-21T00:36:03.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 6.854013914, 2024-09-21T00:36:03.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 3.085660767, 2024-09-21T00:36:03.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 2.084813621, 2024-09-21T00:36:03.010 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 1.752316344, 2024-09-21T00:36:03.010 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:03.010 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 78865, 2024-09-21T00:36:03.010 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 48.103321797, 2024-09-21T00:36:03.010 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000609945 2024-09-21T00:36:03.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.010 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:03.010 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 70902, 2024-09-21T00:36:03.010 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.012419080, 2024-09-21T00:36:03.010 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000175 2024-09-21T00:36:03.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.010 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 1025648, 2024-09-21T00:36:03.010 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 194160 2024-09-21T00:36:03.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.011 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-21T00:36:03.011 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 1, 2024-09-21T00:36:03.011 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-21T00:36:03.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.011 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-MDSRank": { 2024-09-21T00:36:03.011 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2024-09-21T00:36:03.011 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2024-09-21T00:36:03.011 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 15767, 2024-09-21T00:36:03.011 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 126.184515239, 2024-09-21T00:36:03.011 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.008003077 2024-09-21T00:36:03.011 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:03.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.011 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-PurgeQueue": { 2024-09-21T00:36:03.012 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2024-09-21T00:36:03.012 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2024-09-21T00:36:03.012 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 34986, 2024-09-21T00:36:03.012 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 7.990706464, 2024-09-21T00:36:03.012 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000228397 2024-09-21T00:36:03.012 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:03.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.012 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": { 2024-09-21T00:36:03.012 INFO:teuthology.orchestra.run.smithi089.stdout: "request": 175511, 2024-09-21T00:36:03.012 INFO:teuthology.orchestra.run.smithi089.stdout: "reply": 155915, 2024-09-21T00:36:03.012 INFO:teuthology.orchestra.run.smithi089.stdout: "reply_latency": { 2024-09-21T00:36:03.012 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 155915, 2024-09-21T00:36:03.012 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 3237.562738429, 2024-09-21T00:36:03.012 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.020764921 2024-09-21T00:36:03.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.013 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_reply": 0, 2024-09-21T00:36:03.013 INFO:teuthology.orchestra.run.smithi089.stdout: "forward": 19532, 2024-09-21T00:36:03.013 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_complete": 454, 2024-09-21T00:36:03.013 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_keys": 0, 2024-09-21T00:36:03.013 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_commit": 6225, 2024-09-21T00:36:03.013 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_split": 109, 2024-09-21T00:36:03.013 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_merge": 1, 2024-09-21T00:36:03.013 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes": 32779, 2024-09-21T00:36:03.013 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_top": 22338, 2024-09-21T00:36:03.013 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_bottom": 10318, 2024-09-21T00:36:03.013 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pin_tail": 123, 2024-09-21T00:36:03.013 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pinned": 140, 2024-09-21T00:36:03.013 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_expired": 116316, 2024-09-21T00:36:03.013 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_with_caps": 2, 2024-09-21T00:36:03.014 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": 2, 2024-09-21T00:36:03.014 INFO:teuthology.orchestra.run.smithi089.stdout: "subtrees": 4, 2024-09-21T00:36:03.014 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse": 281300, 2024-09-21T00:36:03.014 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_hit": 229052, 2024-09-21T00:36:03.014 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_forward": 19337, 2024-09-21T00:36:03.014 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_discover": 622, 2024-09-21T00:36:03.014 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_dir_fetch": 130, 2024-09-21T00:36:03.014 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_remote_ino": 0, 2024-09-21T00:36:03.014 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_lock": 335, 2024-09-21T00:36:03.014 INFO:teuthology.orchestra.run.smithi089.stdout: "load_cent": 110, 2024-09-21T00:36:03.014 INFO:teuthology.orchestra.run.smithi089.stdout: "q": 0, 2024-09-21T00:36:03.014 INFO:teuthology.orchestra.run.smithi089.stdout: "exported": 212, 2024-09-21T00:36:03.014 INFO:teuthology.orchestra.run.smithi089.stdout: "exported_inodes": 86057, 2024-09-21T00:36:03.014 INFO:teuthology.orchestra.run.smithi089.stdout: "imported": 223, 2024-09-21T00:36:03.015 INFO:teuthology.orchestra.run.smithi089.stdout: "imported_inodes": 77938, 2024-09-21T00:36:03.015 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_dir_fetch": 0, 2024-09-21T00:36:03.015 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_backtrace_fetch": 0, 2024-09-21T00:36:03.015 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_peer_discover": 0, 2024-09-21T00:36:03.015 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rfiles": 4, 2024-09-21T00:36:03.015 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rbytes": 234, 2024-09-21T00:36:03.015 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rsnaps": 6, 2024-09-21T00:36:03.015 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T00:36:03.015 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_set_tag": 0, 2024-09-21T00:36:03.015 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T00:36:03.015 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_inotable_repaired": 0, 2024-09-21T00:36:03.015 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_inodes": 0, 2024-09-21T00:36:03.015 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T00:36:03.015 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T00:36:03.015 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_file_inodes": 0, 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_inode_file_caps": 163, 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_revoke": 52948, 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_grant": 62112, 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_trunc": 6, 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps": 75910, 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps_dirty": 26843, 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_cap_release": 5400, 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: "process_request_cap_release": 47214 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_cache": { 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update": 2630, 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update_receipt": 2047, 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_try_discover": 1107, 2024-09-21T00:36:03.016 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_send_discover": 1806, 2024-09-21T00:36:03.017 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_handle_discover": 7039, 2024-09-21T00:36:03.017 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays": 127, 2024-09-21T00:36:03.017 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_delayed": 0, 2024-09-21T00:36:03.017 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_enqueuing": 0, 2024-09-21T00:36:03.017 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_created": 18984, 2024-09-21T00:36:03.017 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_enqueued": 18858, 2024-09-21T00:36:03.017 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_reintegrated": 0, 2024-09-21T00:36:03.017 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_migrated": 0, 2024-09-21T00:36:03.017 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_processing": 0, 2024-09-21T00:36:03.017 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_enqueued": 0, 2024-09-21T00:36:03.017 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_prioritized": 0, 2024-09-21T00:36:03.017 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_started": 0, 2024-09-21T00:36:03.017 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_completed": 0, 2024-09-21T00:36:03.017 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_quiesce_path": 7, 2024-09-21T00:36:03.018 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_quiesce_inode": 42389, 2024-09-21T00:36:03.018 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T00:36:03.018 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_exportdir": 220, 2024-09-21T00:36:03.018 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_flush": 0, 2024-09-21T00:36:03.018 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragmentdir": 116, 2024-09-21T00:36:03.018 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragstats": 0, 2024-09-21T00:36:03.018 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_inodestats": 0, 2024-09-21T00:36:03.018 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_started": 0, 2024-09-21T00:36:03.018 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_succeeded": 0, 2024-09-21T00:36:03.018 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_write_failed": 0 2024-09-21T00:36:03.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.018 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_log": { 2024-09-21T00:36:03.018 INFO:teuthology.orchestra.run.smithi089.stdout: "evlrg": 7, 2024-09-21T00:36:03.019 INFO:teuthology.orchestra.run.smithi089.stdout: "evadd": 141352, 2024-09-21T00:36:03.019 INFO:teuthology.orchestra.run.smithi089.stdout: "evex": 141339, 2024-09-21T00:36:03.019 INFO:teuthology.orchestra.run.smithi089.stdout: "evtrm": 141339, 2024-09-21T00:36:03.019 INFO:teuthology.orchestra.run.smithi089.stdout: "ev": 13, 2024-09-21T00:36:03.019 INFO:teuthology.orchestra.run.smithi089.stdout: "evexg": 0, 2024-09-21T00:36:03.019 INFO:teuthology.orchestra.run.smithi089.stdout: "evexd": 1541, 2024-09-21T00:36:03.019 INFO:teuthology.orchestra.run.smithi089.stdout: "segadd": 242, 2024-09-21T00:36:03.019 INFO:teuthology.orchestra.run.smithi089.stdout: "segex": 241, 2024-09-21T00:36:03.019 INFO:teuthology.orchestra.run.smithi089.stdout: "segtrm": 241, 2024-09-21T00:36:03.019 INFO:teuthology.orchestra.run.smithi089.stdout: "seg": 1, 2024-09-21T00:36:03.019 INFO:teuthology.orchestra.run.smithi089.stdout: "segmjr": 1, 2024-09-21T00:36:03.019 INFO:teuthology.orchestra.run.smithi089.stdout: "segexg": 0, 2024-09-21T00:36:03.019 INFO:teuthology.orchestra.run.smithi089.stdout: "segexd": 2, 2024-09-21T00:36:03.019 INFO:teuthology.orchestra.run.smithi089.stdout: "expos": 446864325, 2024-09-21T00:36:03.019 INFO:teuthology.orchestra.run.smithi089.stdout: "wrpos": 446899741, 2024-09-21T00:36:03.020 INFO:teuthology.orchestra.run.smithi089.stdout: "rdpos": 0, 2024-09-21T00:36:03.020 INFO:teuthology.orchestra.run.smithi089.stdout: "jlat": { 2024-09-21T00:36:03.020 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 14931, 2024-09-21T00:36:03.020 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 1740.648419517, 2024-09-21T00:36:03.020 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.116579493 2024-09-21T00:36:03.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.020 INFO:teuthology.orchestra.run.smithi089.stdout: "replayed": 0 2024-09-21T00:36:03.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.020 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_mem": { 2024-09-21T00:36:03.020 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 15946, 2024-09-21T00:36:03.020 INFO:teuthology.orchestra.run.smithi089.stdout: "ino+": 108090, 2024-09-21T00:36:03.020 INFO:teuthology.orchestra.run.smithi089.stdout: "ino-": 92144, 2024-09-21T00:36:03.020 INFO:teuthology.orchestra.run.smithi089.stdout: "dir": 879, 2024-09-21T00:36:03.020 INFO:teuthology.orchestra.run.smithi089.stdout: "dir+": 3455, 2024-09-21T00:36:03.020 INFO:teuthology.orchestra.run.smithi089.stdout: "dir-": 2576, 2024-09-21T00:36:03.021 INFO:teuthology.orchestra.run.smithi089.stdout: "dn": 32779, 2024-09-21T00:36:03.021 INFO:teuthology.orchestra.run.smithi089.stdout: "dn+": 176482, 2024-09-21T00:36:03.021 INFO:teuthology.orchestra.run.smithi089.stdout: "dn-": 143703, 2024-09-21T00:36:03.021 INFO:teuthology.orchestra.run.smithi089.stdout: "cap": 2, 2024-09-21T00:36:03.021 INFO:teuthology.orchestra.run.smithi089.stdout: "cap+": 49099, 2024-09-21T00:36:03.021 INFO:teuthology.orchestra.run.smithi089.stdout: "cap-": 49097, 2024-09-21T00:36:03.021 INFO:teuthology.orchestra.run.smithi089.stdout: "rss": 292396, 2024-09-21T00:36:03.021 INFO:teuthology.orchestra.run.smithi089.stdout: "heap": 733413376 2024-09-21T00:36:03.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.021 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_server": { 2024-09-21T00:36:03.021 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_client_request": 185497, 2024-09-21T00:36:03.021 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_server_request": 3249, 2024-09-21T00:36:03.021 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_request": 175511, 2024-09-21T00:36:03.021 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_session": 384, 2024-09-21T00:36:03.022 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_peer_request": 5929, 2024-09-21T00:36:03.022 INFO:teuthology.orchestra.run.smithi089.stdout: "req_create_latency": { 2024-09-21T00:36:03.022 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 26320, 2024-09-21T00:36:03.022 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 218.243610317, 2024-09-21T00:36:03.022 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.008291930 2024-09-21T00:36:03.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.022 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getattr_latency": { 2024-09-21T00:36:03.022 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2716, 2024-09-21T00:36:03.022 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 149.138084602, 2024-09-21T00:36:03.022 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.054910929 2024-09-21T00:36:03.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.022 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getfilelock_latency": { 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: "req_link_latency": { 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookup_latency": { 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 72025, 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 2146.527323861, 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.029802531 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookuphash_latency": { 2024-09-21T00:36:03.023 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.024 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.024 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.024 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupino_latency": { 2024-09-21T00:36:03.024 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.024 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.024 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.024 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupname_latency": { 2024-09-21T00:36:03.024 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.024 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.024 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.024 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupparent_latency": { 2024-09-21T00:36:03.025 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.025 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.025 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.025 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupsnap_latency": { 2024-09-21T00:36:03.026 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.026 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.026 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.026 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lssnap_latency": { 2024-09-21T00:36:03.026 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 15, 2024-09-21T00:36:03.026 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.006768846, 2024-09-21T00:36:03.026 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000451256 2024-09-21T00:36:03.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.026 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mkdir_latency": { 2024-09-21T00:36:03.026 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 207, 2024-09-21T00:36:03.026 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 39.475414391, 2024-09-21T00:36:03.026 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.190702484 2024-09-21T00:36:03.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.027 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mknod_latency": { 2024-09-21T00:36:03.027 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.027 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.027 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.027 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mksnap_latency": { 2024-09-21T00:36:03.027 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 15, 2024-09-21T00:36:03.027 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 47.777460499, 2024-09-21T00:36:03.027 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 3.185164033 2024-09-21T00:36:03.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.027 INFO:teuthology.orchestra.run.smithi089.stdout: "req_open_latency": { 2024-09-21T00:36:03.027 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 11061, 2024-09-21T00:36:03.027 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 69.827360774, 2024-09-21T00:36:03.027 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.006312933 2024-09-21T00:36:03.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.028 INFO:teuthology.orchestra.run.smithi089.stdout: "req_readdir_latency": { 2024-09-21T00:36:03.028 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 908, 2024-09-21T00:36:03.028 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 4.172730969, 2024-09-21T00:36:03.028 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.004595518 2024-09-21T00:36:03.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.028 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rename_latency": { 2024-09-21T00:36:03.028 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 25922, 2024-09-21T00:36:03.028 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 500.206971796, 2024-09-21T00:36:03.028 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.019296619 2024-09-21T00:36:03.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.028 INFO:teuthology.orchestra.run.smithi089.stdout: "req_renamesnap_latency": { 2024-09-21T00:36:03.028 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.028 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.028 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.029 INFO:teuthology.orchestra.run.smithi089.stdout: "req_snapdiff_latency": { 2024-09-21T00:36:03.029 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.029 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.029 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.029 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmdir_latency": { 2024-09-21T00:36:03.029 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 130, 2024-09-21T00:36:03.029 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 26.580020378, 2024-09-21T00:36:03.029 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.204461695 2024-09-21T00:36:03.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.029 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmsnap_latency": { 2024-09-21T00:36:03.029 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 9, 2024-09-21T00:36:03.029 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.690555070, 2024-09-21T00:36:03.030 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.076728341 2024-09-21T00:36:03.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.030 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmxattr_latency": { 2024-09-21T00:36:03.030 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.030 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.030 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.030 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setattr_latency": { 2024-09-21T00:36:03.030 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 8, 2024-09-21T00:36:03.030 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.816892633, 2024-09-21T00:36:03.030 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.102111579 2024-09-21T00:36:03.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.030 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setdirlayout_latency": { 2024-09-21T00:36:03.030 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.031 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.031 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.031 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setfilelock_latency": { 2024-09-21T00:36:03.031 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.031 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.031 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.031 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setlayout_latency": { 2024-09-21T00:36:03.031 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.031 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.031 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.031 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setxattr_latency": { 2024-09-21T00:36:03.031 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 10, 2024-09-21T00:36:03.032 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.026030882, 2024-09-21T00:36:03.032 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.002603088 2024-09-21T00:36:03.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.032 INFO:teuthology.orchestra.run.smithi089.stdout: "req_symlink_latency": { 2024-09-21T00:36:03.032 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 3, 2024-09-21T00:36:03.032 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.001890963, 2024-09-21T00:36:03.032 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000630321 2024-09-21T00:36:03.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.032 INFO:teuthology.orchestra.run.smithi089.stdout: "req_unlink_latency": { 2024-09-21T00:36:03.032 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 16566, 2024-09-21T00:36:03.032 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 34.071622448, 2024-09-21T00:36:03.032 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.002056719 2024-09-21T00:36:03.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.033 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_revoke_eviction": 0, 2024-09-21T00:36:03.033 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_acquisition_throttle": 0, 2024-09-21T00:36:03.033 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getvxattr_latency": { 2024-09-21T00:36:03.033 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.033 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.033 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.033 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:03.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.033 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_sessions": { 2024-09-21T00:36:03.033 INFO:teuthology.orchestra.run.smithi089.stdout: "session_count": 2, 2024-09-21T00:36:03.033 INFO:teuthology.orchestra.run.smithi089.stdout: "session_add": 4, 2024-09-21T00:36:03.033 INFO:teuthology.orchestra.run.smithi089.stdout: "session_remove": 2, 2024-09-21T00:36:03.033 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_open": 2, 2024-09-21T00:36:03.033 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_stale": 0, 2024-09-21T00:36:03.033 INFO:teuthology.orchestra.run.smithi089.stdout: "total_load": 8045, 2024-09-21T00:36:03.034 INFO:teuthology.orchestra.run.smithi089.stdout: "average_load": 4022, 2024-09-21T00:36:03.034 INFO:teuthology.orchestra.run.smithi089.stdout: "avg_session_uptime": 855, 2024-09-21T00:36:03.034 INFO:teuthology.orchestra.run.smithi089.stdout: "mdthresh_evicted": 0 2024-09-21T00:36:03.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.034 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-21T00:36:03.034 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 480, 2024-09-21T00:36:03.034 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 480, 2024-09-21T00:36:03.034 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T00:36:03.034 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-21T00:36:03.034 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T00:36:03.034 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-21T00:36:03.034 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T00:36:03.034 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T00:36:03.034 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T00:36:03.035 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T00:36:03.035 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T00:36:03.035 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-21T00:36:03.035 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T00:36:03.035 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T00:36:03.035 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-21T00:36:03.035 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-21T00:36:03.035 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-21T00:36:03.035 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-21T00:36:03.035 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T00:36:03.035 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T00:36:03.035 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T00:36:03.035 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T00:36:03.035 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T00:36:03.036 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-21T00:36:03.036 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-21T00:36:03.036 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-21T00:36:03.036 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T00:36:03.036 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T00:36:03.036 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-21T00:36:03.036 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-21T00:36:03.036 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-21T00:36:03.036 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-21T00:36:03.036 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T00:36:03.036 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-21T00:36:03.036 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T00:36:03.036 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-21T00:36:03.036 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 1941214, 2024-09-21T00:36:03.036 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 16365, 2024-09-21T00:36:03.037 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 88, 2024-09-21T00:36:03.037 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 1, 2024-09-21T00:36:03.037 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-21T00:36:03.037 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-21T00:36:03.037 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-21T00:36:03.037 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-21T00:36:03.037 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-21T00:36:03.037 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-21T00:36:03.037 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 9912, 2024-09-21T00:36:03.037 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 204, 2024-09-21T00:36:03.037 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T00:36:03.037 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-21T00:36:03.037 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-21T00:36:03.037 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-21T00:36:03.037 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 62476283, 2024-09-21T00:36:03.038 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 746513, 2024-09-21T00:36:03.038 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-21T00:36:03.038 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-21T00:36:03.038 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-21T00:36:03.038 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-21T00:36:03.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.038 INFO:teuthology.orchestra.run.smithi089.stdout: "objecter": { 2024-09-21T00:36:03.038 INFO:teuthology.orchestra.run.smithi089.stdout: "op_active": 0, 2024-09-21T00:36:03.038 INFO:teuthology.orchestra.run.smithi089.stdout: "op_laggy": 0, 2024-09-21T00:36:03.038 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send": 126428, 2024-09-21T00:36:03.038 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send_bytes": 550273638, 2024-09-21T00:36:03.038 INFO:teuthology.orchestra.run.smithi089.stdout: "op_resend": 37, 2024-09-21T00:36:03.038 INFO:teuthology.orchestra.run.smithi089.stdout: "op_reply": 126428, 2024-09-21T00:36:03.038 INFO:teuthology.orchestra.run.smithi089.stdout: "op_latency": { 2024-09-21T00:36:03.039 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 126428, 2024-09-21T00:36:03.039 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 68214.933030520, 2024-09-21T00:36:03.039 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.539555581 2024-09-21T00:36:03.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.039 INFO:teuthology.orchestra.run.smithi089.stdout: "op_inflight": 0, 2024-09-21T00:36:03.039 INFO:teuthology.orchestra.run.smithi089.stdout: "oplen_avg": { 2024-09-21T00:36:03.039 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 126428, 2024-09-21T00:36:03.039 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 238629 2024-09-21T00:36:03.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.039 INFO:teuthology.orchestra.run.smithi089.stdout: "op": 126428, 2024-09-21T00:36:03.039 INFO:teuthology.orchestra.run.smithi089.stdout: "op_r": 885, 2024-09-21T00:36:03.039 INFO:teuthology.orchestra.run.smithi089.stdout: "op_w": 125543, 2024-09-21T00:36:03.039 INFO:teuthology.orchestra.run.smithi089.stdout: "op_rmw": 0, 2024-09-21T00:36:03.039 INFO:teuthology.orchestra.run.smithi089.stdout: "op_pg": 0, 2024-09-21T00:36:03.039 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_stat": 5169, 2024-09-21T00:36:03.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_create": 64520, 2024-09-21T00:36:03.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_read": 431, 2024-09-21T00:36:03.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_write": 15364, 2024-09-21T00:36:03.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writefull": 384, 2024-09-21T00:36:03.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writesame": 0, 2024-09-21T00:36:03.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_append": 0, 2024-09-21T00:36:03.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_zero": 1, 2024-09-21T00:36:03.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_truncate": 109, 2024-09-21T00:36:03.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_delete": 38437, 2024-09-21T00:36:03.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_mapext": 0, 2024-09-21T00:36:03.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_sparse_read": 0, 2024-09-21T00:36:03.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_clonerange": 0, 2024-09-21T00:36:03.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_getxattr": 13, 2024-09-21T00:36:03.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_setxattr": 97884, 2024-09-21T00:36:03.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_cmpxattr": 0, 2024-09-21T00:36:03.041 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_rmxattr": 0, 2024-09-21T00:36:03.041 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_resetxattrs": 0, 2024-09-21T00:36:03.041 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_call": 0, 2024-09-21T00:36:03.041 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_watch": 0, 2024-09-21T00:36:03.041 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_notify": 0, 2024-09-21T00:36:03.041 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T00:36:03.041 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls": 0, 2024-09-21T00:36:03.041 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls_filter": 0, 2024-09-21T00:36:03.041 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_other": 6, 2024-09-21T00:36:03.041 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_active": 0, 2024-09-21T00:36:03.041 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_send": 0, 2024-09-21T00:36:03.042 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_resend": 0, 2024-09-21T00:36:03.042 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_ping": 0, 2024-09-21T00:36:03.042 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_active": 0, 2024-09-21T00:36:03.042 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_send": 0, 2024-09-21T00:36:03.042 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_resend": 0, 2024-09-21T00:36:03.042 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_active": 0, 2024-09-21T00:36:03.042 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_send": 0, 2024-09-21T00:36:03.042 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_resend": 0, 2024-09-21T00:36:03.042 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_active": 0, 2024-09-21T00:36:03.042 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_send": 0, 2024-09-21T00:36:03.042 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_resend": 0, 2024-09-21T00:36:03.043 INFO:teuthology.orchestra.run.smithi089.stdout: "command_active": 0, 2024-09-21T00:36:03.043 INFO:teuthology.orchestra.run.smithi089.stdout: "command_send": 0, 2024-09-21T00:36:03.043 INFO:teuthology.orchestra.run.smithi089.stdout: "command_resend": 0, 2024-09-21T00:36:03.043 INFO:teuthology.orchestra.run.smithi089.stdout: "map_epoch": 112, 2024-09-21T00:36:03.043 INFO:teuthology.orchestra.run.smithi089.stdout: "map_full": 0, 2024-09-21T00:36:03.043 INFO:teuthology.orchestra.run.smithi089.stdout: "map_inc": 33, 2024-09-21T00:36:03.043 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_sessions": 12, 2024-09-21T00:36:03.043 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_open": 12, 2024-09-21T00:36:03.043 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_close": 0, 2024-09-21T00:36:03.043 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_laggy": 0, 2024-09-21T00:36:03.043 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_wr": 12540, 2024-09-21T00:36:03.044 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_rd": 908, 2024-09-21T00:36:03.044 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_del": 2863 2024-09-21T00:36:03.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.044 INFO:teuthology.orchestra.run.smithi089.stdout: "oft": { 2024-09-21T00:36:03.044 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_objs": 1, 2024-09-21T00:36:03.044 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_kv_pairs": 0, 2024-09-21T00:36:03.044 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_updates": 0, 2024-09-21T00:36:03.044 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_removes": 0 2024-09-21T00:36:03.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.044 INFO:teuthology.orchestra.run.smithi089.stdout: "purge_queue": { 2024-09-21T00:36:03.044 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops": 0, 2024-09-21T00:36:03.044 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops_high_water": 17, 2024-09-21T00:36:03.044 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing": 0, 2024-09-21T00:36:03.045 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_high_water": 9, 2024-09-21T00:36:03.045 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed_ops": 37708, 2024-09-21T00:36:03.045 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed": 18858, 2024-09-21T00:36:03.045 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_item_in_journal": 0 2024-09-21T00:36:03.045 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.045 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T00:36:03.045 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T00:36:03.045 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T00:36:03.045 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T00:36:03.045 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 474820, 2024-09-21T00:36:03.045 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 179881578, 2024-09-21T00:36:03.045 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:03.046 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 474820, 2024-09-21T00:36:03.046 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T00:36:03.046 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T00:36:03.046 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 474820, 2024-09-21T00:36:03.046 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 179881578, 2024-09-21T00:36:03.046 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T00:36:03.046 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.046 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.046 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.046 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:03.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.046 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_bytes": { 2024-09-21T00:36:03.047 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T00:36:03.047 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T00:36:03.047 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T00:36:03.047 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T00:36:03.047 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T00:36:03.047 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:03.047 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:03.047 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 126428, 2024-09-21T00:36:03.047 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 553105976, 2024-09-21T00:36:03.047 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 126428, 2024-09-21T00:36:03.048 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 553105976, 2024-09-21T00:36:03.048 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T00:36:03.048 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.048 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.048 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.048 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:03.048 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.048 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_ops": { 2024-09-21T00:36:03.048 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T00:36:03.048 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 1024, 2024-09-21T00:36:03.048 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T00:36:03.048 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T00:36:03.049 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T00:36:03.049 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:03.049 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:03.049 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 126428, 2024-09-21T00:36:03.049 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 126428, 2024-09-21T00:36:03.049 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 126428, 2024-09-21T00:36:03.049 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 126428, 2024-09-21T00:36:03.049 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T00:36:03.049 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.049 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.049 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.049 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:03.049 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.049 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle": { 2024-09-21T00:36:03.050 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T00:36:03.050 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2024-09-21T00:36:03.050 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T00:36:03.050 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 18858, 2024-09-21T00:36:03.050 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 2839602, 2024-09-21T00:36:03.050 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:03.050 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 18858, 2024-09-21T00:36:03.050 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T00:36:03.050 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T00:36:03.050 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 432, 2024-09-21T00:36:03.050 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 2839602, 2024-09-21T00:36:03.050 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T00:36:03.050 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.050 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.051 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.051 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:03.051 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.051 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle-0x55e0cf495520": { 2024-09-21T00:36:03.051 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 60, 2024-09-21T00:36:03.051 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2024-09-21T00:36:03.051 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T00:36:03.051 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 141352, 2024-09-21T00:36:03.051 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 442705437, 2024-09-21T00:36:03.051 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:03.051 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 141352, 2024-09-21T00:36:03.052 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T00:36:03.052 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T00:36:03.052 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 14931, 2024-09-21T00:36:03.052 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 442705377, 2024-09-21T00:36:03.052 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T00:36:03.052 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:03.052 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:03.052 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:03.052 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:03.052 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:03.052 INFO:teuthology.orchestra.run.smithi089.stdout: "trackedop": { 2024-09-21T00:36:03.052 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_ops_count": 42249 2024-09-21T00:36:03.053 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:03.053 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T00:36:04.108 INFO:tasks.check_counter:Daemon mds.b mds.exported=212 2024-09-21T00:36:04.109 INFO:tasks.check_counter:Daemon mds.b mds.imported=223 2024-09-21T00:36:04.109 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=2630 2024-09-21T00:36:04.109 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=2047 2024-09-21T00:36:04.109 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-21T00:36:04.109 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=15 2024-09-21T00:36:04.109 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=9 2024-09-21T00:36:04.109 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=109 2024-09-21T00:36:04.109 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T00:36:04.109 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T00:36:04.418 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T00:36:04.699 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:04 smithi089 ceph-mon[34481]: pgmap v839: 129 pgs: 16 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 57 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 52 KiB/s wr, 25 op/s 2024-09-21T00:36:04.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:04 smithi098 ceph-mon[34725]: pgmap v839: 129 pgs: 16 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 57 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 52 KiB/s wr, 25 op/s 2024-09-21T00:36:04.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:04 smithi027 ceph-mon[30745]: pgmap v839: 129 pgs: 16 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 57 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 52 KiB/s wr, 25 op/s 2024-09-21T00:36:05.007 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T00:36:05.007 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T00:36:05.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 292, 2024-09-21T00:36:05.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 287, 2024-09-21T00:36:05.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 857888, 2024-09-21T00:36:05.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 177344, 2024-09-21T00:36:05.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-09-21T00:36:05.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2024-09-21T00:36:05.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.047390853, 2024-09-21T00:36:05.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.022240098, 2024-09-21T00:36:05.007 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.032927897, 2024-09-21T00:36:05.008 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:36:05.008 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:05.008 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 286, 2024-09-21T00:36:05.008 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.017501546, 2024-09-21T00:36:05.008 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000061194 2024-09-21T00:36:05.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:05.008 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:05.008 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:05.008 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:05.009 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:05.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:05.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 857888, 2024-09-21T00:36:05.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 177344 2024-09-21T00:36:05.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:05.009 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T00:36:05.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 2, 2024-09-21T00:36:05.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 220, 2024-09-21T00:36:05.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 320, 2024-09-21T00:36:05.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 199552, 2024-09-21T00:36:05.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-09-21T00:36:05.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2024-09-21T00:36:05.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.017522397, 2024-09-21T00:36:05.009 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.013690128, 2024-09-21T00:36:05.010 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.001460831, 2024-09-21T00:36:05.010 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:36:05.010 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:05.010 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 218, 2024-09-21T00:36:05.010 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.016086733, 2024-09-21T00:36:05.010 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000073792 2024-09-21T00:36:05.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:05.010 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:05.010 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:05.010 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:05.010 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:05.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:05.010 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T00:36:05.011 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 199552 2024-09-21T00:36:05.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:05.011 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T00:36:05.011 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 4, 2024-09-21T00:36:05.011 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 4, 2024-09-21T00:36:05.011 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 2992, 2024-09-21T00:36:05.011 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 448, 2024-09-21T00:36:05.011 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2024-09-21T00:36:05.011 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2024-09-21T00:36:05.011 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.002210670, 2024-09-21T00:36:05.011 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000037047, 2024-09-21T00:36:05.011 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.002161286, 2024-09-21T00:36:05.011 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:36:05.011 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:05.011 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 3, 2024-09-21T00:36:05.012 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.003764558, 2024-09-21T00:36:05.012 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.001254852 2024-09-21T00:36:05.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:05.012 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:05.012 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:05.012 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:05.012 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:05.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:05.012 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T00:36:05.012 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T00:36:05.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:05.012 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-21T00:36:05.012 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2024-09-21T00:36:05.012 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-21T00:36:05.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:05.013 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-21T00:36:05.013 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-09-21T00:36:05.013 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-09-21T00:36:05.013 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T00:36:05.013 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-21T00:36:05.013 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T00:36:05.013 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-21T00:36:05.013 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T00:36:05.013 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T00:36:05.013 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T00:36:05.013 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T00:36:05.013 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T00:36:05.013 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-21T00:36:05.013 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T00:36:05.013 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T00:36:05.014 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-21T00:36:05.014 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-21T00:36:05.014 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-21T00:36:05.014 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-21T00:36:05.014 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T00:36:05.014 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T00:36:05.014 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T00:36:05.014 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T00:36:05.014 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T00:36:05.014 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-21T00:36:05.014 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-21T00:36:05.014 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-21T00:36:05.014 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T00:36:05.014 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T00:36:05.015 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-21T00:36:05.015 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-21T00:36:05.015 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-21T00:36:05.015 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-21T00:36:05.015 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T00:36:05.015 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-21T00:36:05.015 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T00:36:05.015 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-21T00:36:05.015 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 147954, 2024-09-21T00:36:05.015 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 29, 2024-09-21T00:36:05.015 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-09-21T00:36:05.015 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-09-21T00:36:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-21T00:36:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-21T00:36:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-21T00:36:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-21T00:36:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-21T00:36:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-21T00:36:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2024-09-21T00:36:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2024-09-21T00:36:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T00:36:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-21T00:36:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-21T00:36:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-21T00:36:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 968, 2024-09-21T00:36:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2024-09-21T00:36:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-21T00:36:05.017 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-21T00:36:05.017 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-21T00:36:05.017 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-21T00:36:05.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:05.017 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T00:36:05.017 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T00:36:05.017 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T00:36:05.017 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T00:36:05.017 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 294, 2024-09-21T00:36:05.017 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 816606, 2024-09-21T00:36:05.017 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:05.017 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 294, 2024-09-21T00:36:05.017 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T00:36:05.017 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T00:36:05.017 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 294, 2024-09-21T00:36:05.018 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 816606, 2024-09-21T00:36:05.018 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T00:36:05.018 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:05.018 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:05.018 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:05.018 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:05.018 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:05.018 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T00:36:06.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:06 smithi089 ceph-mon[34481]: pgmap v840: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 52 KiB/s wr, 25 op/s 2024-09-21T00:36:06.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:06 smithi098 ceph-mon[34725]: pgmap v840: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 52 KiB/s wr, 25 op/s 2024-09-21T00:36:06.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:06 smithi027 ceph-mon[30745]: pgmap v840: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 52 KiB/s wr, 25 op/s 2024-09-21T00:36:06.825 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-09-21T00:36:06.825 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-09-21T00:36:06.825 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-09-21T00:36:06.825 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-09-21T00:36:06.825 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-09-21T00:36:06.825 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-09-21T00:36:06.825 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-09-21T00:36:06.825 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-09-21T00:36:06.825 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T00:36:06.826 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T00:36:07.152 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T00:36:07.870 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T00:36:07.870 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T00:36:07.870 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 220, 2024-09-21T00:36:07.870 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 221, 2024-09-21T00:36:07.870 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 31728777, 2024-09-21T00:36:07.871 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 69470, 2024-09-21T00:36:07.871 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 8, 2024-09-21T00:36:07.871 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 4, 2024-09-21T00:36:07.871 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.048053719, 2024-09-21T00:36:07.871 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.008016849, 2024-09-21T00:36:07.871 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.029233636, 2024-09-21T00:36:07.871 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.006698474, 2024-09-21T00:36:07.871 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:07.871 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 220, 2024-09-21T00:36:07.871 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.032419768, 2024-09-21T00:36:07.871 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000147362 2024-09-21T00:36:07.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.871 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:07.871 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.871 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.872 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.872 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 2656, 2024-09-21T00:36:07.872 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T00:36:07.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.872 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T00:36:07.872 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 369, 2024-09-21T00:36:07.872 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 410, 2024-09-21T00:36:07.872 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 33672045, 2024-09-21T00:36:07.872 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 278637, 2024-09-21T00:36:07.872 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 6, 2024-09-21T00:36:07.872 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 5, 2024-09-21T00:36:07.872 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.063937448, 2024-09-21T00:36:07.872 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.017953812, 2024-09-21T00:36:07.873 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.030808729, 2024-09-21T00:36:07.873 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.011886809, 2024-09-21T00:36:07.873 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:07.873 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 408, 2024-09-21T00:36:07.873 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.025147922, 2024-09-21T00:36:07.873 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000061637 2024-09-21T00:36:07.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.873 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:07.873 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.873 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.873 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.873 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T00:36:07.873 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 161552 2024-09-21T00:36:07.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.874 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T00:36:07.874 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 232, 2024-09-21T00:36:07.874 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 229, 2024-09-21T00:36:07.874 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 37551602, 2024-09-21T00:36:07.874 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 86400, 2024-09-21T00:36:07.874 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 5, 2024-09-21T00:36:07.874 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 4, 2024-09-21T00:36:07.874 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.047218527, 2024-09-21T00:36:07.874 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.010016105, 2024-09-21T00:36:07.874 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.034614237, 2024-09-21T00:36:07.874 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.005305414, 2024-09-21T00:36:07.874 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:07.874 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 228, 2024-09-21T00:36:07.874 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.018389779, 2024-09-21T00:36:07.875 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000080656 2024-09-21T00:36:07.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.875 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:07.875 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.875 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.875 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.875 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 202336, 2024-09-21T00:36:07.875 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 35520 2024-09-21T00:36:07.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.875 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-21T00:36:07.875 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 1, 2024-09-21T00:36:07.875 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-21T00:36:07.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.876 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-MDSRank": { 2024-09-21T00:36:07.876 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2024-09-21T00:36:07.876 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2024-09-21T00:36:07.876 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1203, 2024-09-21T00:36:07.876 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.226409226, 2024-09-21T00:36:07.876 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000188203 2024-09-21T00:36:07.876 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:07.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.876 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-PurgeQueue": { 2024-09-21T00:36:07.876 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2024-09-21T00:36:07.876 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2024-09-21T00:36:07.876 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.876 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.876 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.877 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:07.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.877 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": { 2024-09-21T00:36:07.877 INFO:teuthology.orchestra.run.smithi089.stdout: "request": 0, 2024-09-21T00:36:07.877 INFO:teuthology.orchestra.run.smithi089.stdout: "reply": 0, 2024-09-21T00:36:07.877 INFO:teuthology.orchestra.run.smithi089.stdout: "reply_latency": { 2024-09-21T00:36:07.877 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.877 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.877 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.877 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_reply": 0, 2024-09-21T00:36:07.877 INFO:teuthology.orchestra.run.smithi089.stdout: "forward": 0, 2024-09-21T00:36:07.877 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_complete": 0, 2024-09-21T00:36:07.877 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_keys": 0, 2024-09-21T00:36:07.877 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_commit": 0, 2024-09-21T00:36:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_split": 0, 2024-09-21T00:36:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_merge": 0, 2024-09-21T00:36:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes": 44762, 2024-09-21T00:36:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_top": 29716, 2024-09-21T00:36:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_bottom": 15046, 2024-09-21T00:36:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pin_tail": 0, 2024-09-21T00:36:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pinned": 104, 2024-09-21T00:36:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_expired": 0, 2024-09-21T00:36:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_with_caps": 0, 2024-09-21T00:36:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": 0, 2024-09-21T00:36:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: "subtrees": 2, 2024-09-21T00:36:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse": 0, 2024-09-21T00:36:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_hit": 0, 2024-09-21T00:36:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_forward": 0, 2024-09-21T00:36:07.878 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_discover": 0, 2024-09-21T00:36:07.879 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_dir_fetch": 0, 2024-09-21T00:36:07.879 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_remote_ino": 0, 2024-09-21T00:36:07.879 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_lock": 0, 2024-09-21T00:36:07.879 INFO:teuthology.orchestra.run.smithi089.stdout: "load_cent": 0, 2024-09-21T00:36:07.879 INFO:teuthology.orchestra.run.smithi089.stdout: "q": 0, 2024-09-21T00:36:07.879 INFO:teuthology.orchestra.run.smithi089.stdout: "exported": 0, 2024-09-21T00:36:07.879 INFO:teuthology.orchestra.run.smithi089.stdout: "exported_inodes": 0, 2024-09-21T00:36:07.879 INFO:teuthology.orchestra.run.smithi089.stdout: "imported": 0, 2024-09-21T00:36:07.879 INFO:teuthology.orchestra.run.smithi089.stdout: "imported_inodes": 0, 2024-09-21T00:36:07.879 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_dir_fetch": 0, 2024-09-21T00:36:07.879 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_backtrace_fetch": 0, 2024-09-21T00:36:07.879 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_peer_discover": 0, 2024-09-21T00:36:07.879 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rfiles": 4, 2024-09-21T00:36:07.879 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rbytes": 234, 2024-09-21T00:36:07.879 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rsnaps": 6, 2024-09-21T00:36:07.880 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T00:36:07.880 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_set_tag": 0, 2024-09-21T00:36:07.880 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T00:36:07.880 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_inotable_repaired": 0, 2024-09-21T00:36:07.880 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_inodes": 0, 2024-09-21T00:36:07.880 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T00:36:07.880 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T00:36:07.880 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_file_inodes": 0, 2024-09-21T00:36:07.880 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_inode_file_caps": 0, 2024-09-21T00:36:07.880 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T00:36:07.880 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_grant": 0, 2024-09-21T00:36:07.880 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T00:36:07.880 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T00:36:07.880 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps": 0, 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps_dirty": 0, 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_cap_release": 0, 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: "process_request_cap_release": 0 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_cache": { 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update": 0, 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update_receipt": 0, 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_try_discover": 0, 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_send_discover": 0, 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_handle_discover": 0, 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays": 0, 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_delayed": 0, 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_enqueuing": 0, 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_created": 0, 2024-09-21T00:36:07.881 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_enqueued": 0, 2024-09-21T00:36:07.882 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_reintegrated": 0, 2024-09-21T00:36:07.882 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_migrated": 0, 2024-09-21T00:36:07.882 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_processing": 0, 2024-09-21T00:36:07.882 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_enqueued": 0, 2024-09-21T00:36:07.882 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_prioritized": 0, 2024-09-21T00:36:07.882 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_started": 0, 2024-09-21T00:36:07.882 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_completed": 0, 2024-09-21T00:36:07.882 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_quiesce_path": 0, 2024-09-21T00:36:07.882 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_quiesce_inode": 0, 2024-09-21T00:36:07.882 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T00:36:07.882 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_exportdir": 0, 2024-09-21T00:36:07.882 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_flush": 0, 2024-09-21T00:36:07.882 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragmentdir": 0, 2024-09-21T00:36:07.882 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragstats": 0, 2024-09-21T00:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_inodestats": 0, 2024-09-21T00:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_started": 0, 2024-09-21T00:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_succeeded": 0, 2024-09-21T00:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_write_failed": 0 2024-09-21T00:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_log": { 2024-09-21T00:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout: "evlrg": 0, 2024-09-21T00:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout: "evadd": 0, 2024-09-21T00:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout: "evex": 0, 2024-09-21T00:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout: "evtrm": 0, 2024-09-21T00:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout: "ev": 23969, 2024-09-21T00:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout: "evexg": 0, 2024-09-21T00:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout: "evexd": 0, 2024-09-21T00:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout: "segadd": 0, 2024-09-21T00:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout: "segex": 0, 2024-09-21T00:36:07.884 INFO:teuthology.orchestra.run.smithi089.stdout: "segtrm": 0, 2024-09-21T00:36:07.884 INFO:teuthology.orchestra.run.smithi089.stdout: "seg": 2, 2024-09-21T00:36:07.884 INFO:teuthology.orchestra.run.smithi089.stdout: "segmjr": 12, 2024-09-21T00:36:07.884 INFO:teuthology.orchestra.run.smithi089.stdout: "segexg": 0, 2024-09-21T00:36:07.884 INFO:teuthology.orchestra.run.smithi089.stdout: "segexd": 0, 2024-09-21T00:36:07.884 INFO:teuthology.orchestra.run.smithi089.stdout: "expos": 446864325, 2024-09-21T00:36:07.884 INFO:teuthology.orchestra.run.smithi089.stdout: "wrpos": 446899741, 2024-09-21T00:36:07.884 INFO:teuthology.orchestra.run.smithi089.stdout: "rdpos": 446899681, 2024-09-21T00:36:07.884 INFO:teuthology.orchestra.run.smithi089.stdout: "jlat": { 2024-09-21T00:36:07.884 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.884 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.884 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.884 INFO:teuthology.orchestra.run.smithi089.stdout: "replayed": 23969 2024-09-21T00:36:07.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.885 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_mem": { 2024-09-21T00:36:07.885 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 15661, 2024-09-21T00:36:07.885 INFO:teuthology.orchestra.run.smithi089.stdout: "ino+": 40654, 2024-09-21T00:36:07.885 INFO:teuthology.orchestra.run.smithi089.stdout: "ino-": 24993, 2024-09-21T00:36:07.885 INFO:teuthology.orchestra.run.smithi089.stdout: "dir": 603, 2024-09-21T00:36:07.885 INFO:teuthology.orchestra.run.smithi089.stdout: "dir+": 982, 2024-09-21T00:36:07.885 INFO:teuthology.orchestra.run.smithi089.stdout: "dir-": 379, 2024-09-21T00:36:07.885 INFO:teuthology.orchestra.run.smithi089.stdout: "dn": 44762, 2024-09-21T00:36:07.885 INFO:teuthology.orchestra.run.smithi089.stdout: "dn+": 64816, 2024-09-21T00:36:07.885 INFO:teuthology.orchestra.run.smithi089.stdout: "dn-": 20054, 2024-09-21T00:36:07.885 INFO:teuthology.orchestra.run.smithi089.stdout: "cap": 0, 2024-09-21T00:36:07.885 INFO:teuthology.orchestra.run.smithi089.stdout: "cap+": 0, 2024-09-21T00:36:07.885 INFO:teuthology.orchestra.run.smithi089.stdout: "cap-": 0, 2024-09-21T00:36:07.885 INFO:teuthology.orchestra.run.smithi089.stdout: "rss": 200716, 2024-09-21T00:36:07.885 INFO:teuthology.orchestra.run.smithi089.stdout: "heap": 434663424 2024-09-21T00:36:07.886 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.886 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_server": { 2024-09-21T00:36:07.886 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_client_request": 0, 2024-09-21T00:36:07.886 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_server_request": 0, 2024-09-21T00:36:07.886 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_request": 0, 2024-09-21T00:36:07.886 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_session": 0, 2024-09-21T00:36:07.886 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_peer_request": 0, 2024-09-21T00:36:07.886 INFO:teuthology.orchestra.run.smithi089.stdout: "req_create_latency": { 2024-09-21T00:36:07.886 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.886 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.886 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.886 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.886 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getattr_latency": { 2024-09-21T00:36:07.886 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.886 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getfilelock_latency": { 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: "req_link_latency": { 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookup_latency": { 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.888 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookuphash_latency": { 2024-09-21T00:36:07.888 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.888 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.888 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.888 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupino_latency": { 2024-09-21T00:36:07.888 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.888 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.888 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.888 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupname_latency": { 2024-09-21T00:36:07.888 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.888 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.889 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.889 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupparent_latency": { 2024-09-21T00:36:07.889 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.889 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.889 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.889 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupsnap_latency": { 2024-09-21T00:36:07.889 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.889 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.889 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.889 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lssnap_latency": { 2024-09-21T00:36:07.889 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.889 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.890 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.890 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mkdir_latency": { 2024-09-21T00:36:07.890 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.890 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.890 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.890 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mknod_latency": { 2024-09-21T00:36:07.890 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.890 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.890 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.890 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mksnap_latency": { 2024-09-21T00:36:07.890 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.891 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.891 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.891 INFO:teuthology.orchestra.run.smithi089.stdout: "req_open_latency": { 2024-09-21T00:36:07.891 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.891 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.891 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.891 INFO:teuthology.orchestra.run.smithi089.stdout: "req_readdir_latency": { 2024-09-21T00:36:07.891 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.891 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.891 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.891 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rename_latency": { 2024-09-21T00:36:07.892 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.892 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.892 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.892 INFO:teuthology.orchestra.run.smithi089.stdout: "req_renamesnap_latency": { 2024-09-21T00:36:07.892 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.892 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.892 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.892 INFO:teuthology.orchestra.run.smithi089.stdout: "req_snapdiff_latency": { 2024-09-21T00:36:07.892 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.892 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.892 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.892 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmdir_latency": { 2024-09-21T00:36:07.893 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.893 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.893 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.893 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmsnap_latency": { 2024-09-21T00:36:07.893 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.893 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.893 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.893 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmxattr_latency": { 2024-09-21T00:36:07.893 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.893 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.893 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.893 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setattr_latency": { 2024-09-21T00:36:07.894 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.894 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.894 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.894 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setdirlayout_latency": { 2024-09-21T00:36:07.894 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.894 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.894 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.894 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setfilelock_latency": { 2024-09-21T00:36:07.894 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.894 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.894 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.895 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setlayout_latency": { 2024-09-21T00:36:07.895 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.895 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.895 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.895 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setxattr_latency": { 2024-09-21T00:36:07.895 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.895 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.895 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.895 INFO:teuthology.orchestra.run.smithi089.stdout: "req_symlink_latency": { 2024-09-21T00:36:07.895 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.895 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.896 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.896 INFO:teuthology.orchestra.run.smithi089.stdout: "req_unlink_latency": { 2024-09-21T00:36:07.896 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.896 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.896 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.896 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_revoke_eviction": 0, 2024-09-21T00:36:07.896 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_acquisition_throttle": 0, 2024-09-21T00:36:07.896 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getvxattr_latency": { 2024-09-21T00:36:07.896 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.896 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.896 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.896 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:07.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.897 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_sessions": { 2024-09-21T00:36:07.897 INFO:teuthology.orchestra.run.smithi089.stdout: "session_count": 2, 2024-09-21T00:36:07.897 INFO:teuthology.orchestra.run.smithi089.stdout: "session_add": 2, 2024-09-21T00:36:07.897 INFO:teuthology.orchestra.run.smithi089.stdout: "session_remove": 0, 2024-09-21T00:36:07.897 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_open": 0, 2024-09-21T00:36:07.897 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_stale": 0, 2024-09-21T00:36:07.897 INFO:teuthology.orchestra.run.smithi089.stdout: "total_load": 0, 2024-09-21T00:36:07.897 INFO:teuthology.orchestra.run.smithi089.stdout: "average_load": 0, 2024-09-21T00:36:07.897 INFO:teuthology.orchestra.run.smithi089.stdout: "avg_session_uptime": 2477, 2024-09-21T00:36:07.897 INFO:teuthology.orchestra.run.smithi089.stdout: "mdthresh_evicted": 0 2024-09-21T00:36:07.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.897 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-21T00:36:07.897 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-09-21T00:36:07.898 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-09-21T00:36:07.898 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T00:36:07.898 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-21T00:36:07.898 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T00:36:07.898 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-21T00:36:07.898 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T00:36:07.898 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T00:36:07.898 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T00:36:07.898 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T00:36:07.898 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T00:36:07.898 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-21T00:36:07.898 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T00:36:07.898 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T00:36:07.898 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-21T00:36:07.899 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-21T00:36:07.899 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-21T00:36:07.899 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-21T00:36:07.899 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T00:36:07.899 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T00:36:07.899 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T00:36:07.899 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T00:36:07.899 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T00:36:07.899 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-21T00:36:07.899 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-21T00:36:07.899 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-21T00:36:07.899 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T00:36:07.899 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T00:36:07.900 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-21T00:36:07.900 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-21T00:36:07.900 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-21T00:36:07.900 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-21T00:36:07.900 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T00:36:07.900 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-21T00:36:07.900 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T00:36:07.900 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-21T00:36:07.900 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 819160, 2024-09-21T00:36:07.900 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 16001, 2024-09-21T00:36:07.900 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-09-21T00:36:07.900 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-09-21T00:36:07.900 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-21T00:36:07.900 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-21T00:36:07.901 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-21T00:36:07.901 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-21T00:36:07.901 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-21T00:36:07.901 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-21T00:36:07.901 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 9912, 2024-09-21T00:36:07.901 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 204, 2024-09-21T00:36:07.901 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T00:36:07.901 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-21T00:36:07.901 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-21T00:36:07.901 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-21T00:36:07.901 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 68474111, 2024-09-21T00:36:07.901 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 724642, 2024-09-21T00:36:07.901 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-21T00:36:07.901 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-21T00:36:07.902 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-21T00:36:07.902 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-21T00:36:07.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.902 INFO:teuthology.orchestra.run.smithi089.stdout: "objecter": { 2024-09-21T00:36:07.902 INFO:teuthology.orchestra.run.smithi089.stdout: "op_active": 0, 2024-09-21T00:36:07.902 INFO:teuthology.orchestra.run.smithi089.stdout: "op_laggy": 0, 2024-09-21T00:36:07.902 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send": 745, 2024-09-21T00:36:07.902 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send_bytes": 26, 2024-09-21T00:36:07.902 INFO:teuthology.orchestra.run.smithi089.stdout: "op_resend": 4, 2024-09-21T00:36:07.902 INFO:teuthology.orchestra.run.smithi089.stdout: "op_reply": 745, 2024-09-21T00:36:07.902 INFO:teuthology.orchestra.run.smithi089.stdout: "op_latency": { 2024-09-21T00:36:07.902 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 745, 2024-09-21T00:36:07.902 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 5.444879152, 2024-09-21T00:36:07.902 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.007308562 2024-09-21T00:36:07.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.903 INFO:teuthology.orchestra.run.smithi089.stdout: "op_inflight": 0, 2024-09-21T00:36:07.903 INFO:teuthology.orchestra.run.smithi089.stdout: "oplen_avg": { 2024-09-21T00:36:07.903 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 745, 2024-09-21T00:36:07.903 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 746 2024-09-21T00:36:07.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.903 INFO:teuthology.orchestra.run.smithi089.stdout: "op": 745, 2024-09-21T00:36:07.903 INFO:teuthology.orchestra.run.smithi089.stdout: "op_r": 745, 2024-09-21T00:36:07.903 INFO:teuthology.orchestra.run.smithi089.stdout: "op_w": 0, 2024-09-21T00:36:07.903 INFO:teuthology.orchestra.run.smithi089.stdout: "op_rmw": 0, 2024-09-21T00:36:07.903 INFO:teuthology.orchestra.run.smithi089.stdout: "op_pg": 0, 2024-09-21T00:36:07.903 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_stat": 429, 2024-09-21T00:36:07.903 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_create": 0, 2024-09-21T00:36:07.903 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_read": 313, 2024-09-21T00:36:07.903 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_write": 0, 2024-09-21T00:36:07.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writefull": 0, 2024-09-21T00:36:07.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writesame": 0, 2024-09-21T00:36:07.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_append": 0, 2024-09-21T00:36:07.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_zero": 0, 2024-09-21T00:36:07.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_truncate": 0, 2024-09-21T00:36:07.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_delete": 0, 2024-09-21T00:36:07.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_mapext": 0, 2024-09-21T00:36:07.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_sparse_read": 0, 2024-09-21T00:36:07.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_clonerange": 0, 2024-09-21T00:36:07.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_getxattr": 2, 2024-09-21T00:36:07.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_setxattr": 0, 2024-09-21T00:36:07.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_cmpxattr": 0, 2024-09-21T00:36:07.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_rmxattr": 0, 2024-09-21T00:36:07.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_resetxattrs": 0, 2024-09-21T00:36:07.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_call": 0, 2024-09-21T00:36:07.905 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_watch": 0, 2024-09-21T00:36:07.905 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_notify": 0, 2024-09-21T00:36:07.905 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T00:36:07.905 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls": 0, 2024-09-21T00:36:07.905 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls_filter": 0, 2024-09-21T00:36:07.905 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_other": 0, 2024-09-21T00:36:07.905 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_active": 0, 2024-09-21T00:36:07.905 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_send": 0, 2024-09-21T00:36:07.905 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_resend": 0, 2024-09-21T00:36:07.905 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_ping": 0, 2024-09-21T00:36:07.905 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_active": 0, 2024-09-21T00:36:07.905 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_send": 0, 2024-09-21T00:36:07.905 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_resend": 0, 2024-09-21T00:36:07.906 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_active": 0, 2024-09-21T00:36:07.906 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_send": 0, 2024-09-21T00:36:07.906 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_resend": 0, 2024-09-21T00:36:07.906 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_active": 0, 2024-09-21T00:36:07.906 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_send": 0, 2024-09-21T00:36:07.906 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_resend": 0, 2024-09-21T00:36:07.906 INFO:teuthology.orchestra.run.smithi089.stdout: "command_active": 0, 2024-09-21T00:36:07.906 INFO:teuthology.orchestra.run.smithi089.stdout: "command_send": 0, 2024-09-21T00:36:07.906 INFO:teuthology.orchestra.run.smithi089.stdout: "command_resend": 0, 2024-09-21T00:36:07.906 INFO:teuthology.orchestra.run.smithi089.stdout: "map_epoch": 112, 2024-09-21T00:36:07.906 INFO:teuthology.orchestra.run.smithi089.stdout: "map_full": 0, 2024-09-21T00:36:07.906 INFO:teuthology.orchestra.run.smithi089.stdout: "map_inc": 6, 2024-09-21T00:36:07.906 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_sessions": 10, 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_open": 10, 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_close": 0, 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_laggy": 0, 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_wr": 0, 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_rd": 2, 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_del": 0 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: "oft": { 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_objs": 0, 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_kv_pairs": 0, 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_updates": 0, 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_removes": 0 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: "purge_queue": { 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops": 0, 2024-09-21T00:36:07.907 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing": 0, 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_high_water": 0, 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed_ops": 0, 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed": 0, 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_item_in_journal": 0 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 818, 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 102872493, 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 818, 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T00:36:07.908 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T00:36:07.909 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 818, 2024-09-21T00:36:07.909 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 102872493, 2024-09-21T00:36:07.909 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T00:36:07.909 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.909 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.909 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.909 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:07.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.909 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_bytes": { 2024-09-21T00:36:07.909 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T00:36:07.909 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T00:36:07.909 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T00:36:07.909 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T00:36:07.909 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T00:36:07.910 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:07.910 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:07.910 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 745, 2024-09-21T00:36:07.910 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 102536008, 2024-09-21T00:36:07.910 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 745, 2024-09-21T00:36:07.910 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 102536008, 2024-09-21T00:36:07.910 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T00:36:07.910 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.910 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.910 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.910 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:07.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.910 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_ops": { 2024-09-21T00:36:07.911 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T00:36:07.911 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 1024, 2024-09-21T00:36:07.911 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T00:36:07.911 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T00:36:07.911 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T00:36:07.911 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:07.911 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:07.911 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 745, 2024-09-21T00:36:07.911 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 745, 2024-09-21T00:36:07.911 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 745, 2024-09-21T00:36:07.911 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 745, 2024-09-21T00:36:07.911 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T00:36:07.911 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.911 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.911 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.912 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:07.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.912 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle": { 2024-09-21T00:36:07.912 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T00:36:07.912 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2024-09-21T00:36:07.912 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T00:36:07.912 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T00:36:07.912 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T00:36:07.912 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:07.912 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:07.912 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T00:36:07.912 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T00:36:07.912 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 0, 2024-09-21T00:36:07.912 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 0, 2024-09-21T00:36:07.913 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T00:36:07.913 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.913 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.913 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.913 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:07.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.913 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle-0x560e7ac9da00": { 2024-09-21T00:36:07.913 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T00:36:07.913 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2024-09-21T00:36:07.913 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T00:36:07.913 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T00:36:07.913 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T00:36:07.913 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:07.913 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:07.913 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T00:36:07.914 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T00:36:07.914 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 0, 2024-09-21T00:36:07.914 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 0, 2024-09-21T00:36:07.914 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T00:36:07.914 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:07.914 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:07.914 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:07.914 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:07.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:07.914 INFO:teuthology.orchestra.run.smithi089.stdout: "trackedop": { 2024-09-21T00:36:07.914 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_ops_count": 0 2024-09-21T00:36:07.914 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:07.914 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T00:36:08.702 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:08 smithi089 ceph-mon[34481]: pgmap v841: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-21T00:36:08.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:08 smithi098 ceph-mon[34725]: pgmap v841: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-21T00:36:08.750 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-21T00:36:08.750 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-21T00:36:08.750 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-21T00:36:08.751 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-09-21T00:36:08.751 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-21T00:36:08.751 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-21T00:36:08.751 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T00:36:08.751 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-09-21T00:36:08.751 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T00:36:08.751 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T00:36:08.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:08 smithi027 ceph-mon[30745]: pgmap v841: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-21T00:36:09.157 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T00:36:09.615 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:09 smithi089 ceph-mon[34481]: osdmap e113: 12 total, 12 up, 12 in 2024-09-21T00:36:09.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:09 smithi098 ceph-mon[34725]: osdmap e113: 12 total, 12 up, 12 in 2024-09-21T00:36:09.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:09 smithi027 ceph-mon[30745]: osdmap e113: 12 total, 12 up, 12 in 2024-09-21T00:36:10.265 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T00:36:10.265 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T00:36:10.266 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 301, 2024-09-21T00:36:10.266 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 294, 2024-09-21T00:36:10.266 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 862864, 2024-09-21T00:36:10.266 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 180688, 2024-09-21T00:36:10.266 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-09-21T00:36:10.266 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2024-09-21T00:36:10.266 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.051443275, 2024-09-21T00:36:10.266 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.023027273, 2024-09-21T00:36:10.266 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.035800479, 2024-09-21T00:36:10.266 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:36:10.266 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:10.266 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 294, 2024-09-21T00:36:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.018498542, 2024-09-21T00:36:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000062920 2024-09-21T00:36:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 862864, 2024-09-21T00:36:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 180688 2024-09-21T00:36:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T00:36:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 2, 2024-09-21T00:36:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 223, 2024-09-21T00:36:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 320, 2024-09-21T00:36:10.268 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 200080, 2024-09-21T00:36:10.268 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-09-21T00:36:10.268 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2024-09-21T00:36:10.268 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.016344751, 2024-09-21T00:36:10.268 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.012899720, 2024-09-21T00:36:10.268 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.001245490, 2024-09-21T00:36:10.268 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:36:10.268 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:10.268 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 221, 2024-09-21T00:36:10.268 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.012916022, 2024-09-21T00:36:10.268 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000058443 2024-09-21T00:36:10.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:10.268 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:10.269 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:10.269 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:10.269 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:10.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:10.269 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T00:36:10.269 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 200080 2024-09-21T00:36:10.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:10.269 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T00:36:10.269 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 0, 2024-09-21T00:36:10.269 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2, 2024-09-21T00:36:10.269 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 0, 2024-09-21T00:36:10.269 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 0, 2024-09-21T00:36:10.269 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2024-09-21T00:36:10.269 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2024-09-21T00:36:10.270 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.000397433, 2024-09-21T00:36:10.270 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T00:36:10.270 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.000316045, 2024-09-21T00:36:10.270 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:36:10.270 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:10.270 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:10.270 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:10.270 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:10.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:10.270 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:10.270 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:10.270 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:10.270 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:10.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:10.271 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T00:36:10.271 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T00:36:10.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:10.271 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-21T00:36:10.271 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2024-09-21T00:36:10.271 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-21T00:36:10.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:10.271 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-21T00:36:10.271 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-09-21T00:36:10.271 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-09-21T00:36:10.271 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T00:36:10.271 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-21T00:36:10.271 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T00:36:10.271 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-21T00:36:10.272 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T00:36:10.272 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T00:36:10.272 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T00:36:10.272 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T00:36:10.272 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T00:36:10.272 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-21T00:36:10.272 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T00:36:10.272 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T00:36:10.272 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-21T00:36:10.272 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-21T00:36:10.272 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-21T00:36:10.272 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-21T00:36:10.272 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T00:36:10.272 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T00:36:10.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T00:36:10.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T00:36:10.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T00:36:10.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-21T00:36:10.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-21T00:36:10.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-21T00:36:10.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T00:36:10.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T00:36:10.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-21T00:36:10.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-21T00:36:10.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-21T00:36:10.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-21T00:36:10.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T00:36:10.274 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-21T00:36:10.274 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T00:36:10.274 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-21T00:36:10.274 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 148784, 2024-09-21T00:36:10.274 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 31, 2024-09-21T00:36:10.274 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-09-21T00:36:10.274 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-09-21T00:36:10.274 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-21T00:36:10.274 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-21T00:36:10.274 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-21T00:36:10.274 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-21T00:36:10.274 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-21T00:36:10.274 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-21T00:36:10.274 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2024-09-21T00:36:10.275 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2024-09-21T00:36:10.275 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T00:36:10.275 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-21T00:36:10.275 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-21T00:36:10.275 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-21T00:36:10.275 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 968, 2024-09-21T00:36:10.275 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2024-09-21T00:36:10.275 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-21T00:36:10.275 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-21T00:36:10.275 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-21T00:36:10.275 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-21T00:36:10.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T00:36:10.275 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T00:36:10.275 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T00:36:10.276 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T00:36:10.276 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T00:36:10.276 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 299, 2024-09-21T00:36:10.276 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 817893, 2024-09-21T00:36:10.276 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:10.276 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 299, 2024-09-21T00:36:10.276 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T00:36:10.276 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T00:36:10.276 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 299, 2024-09-21T00:36:10.276 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 817893, 2024-09-21T00:36:10.276 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T00:36:10.276 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T00:36:10.276 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T00:36:10.276 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T00:36:10.277 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:10.277 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T00:36:10.277 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T00:36:10.717 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:10 smithi027 ceph-mon[30745]: pgmap v843: 129 pgs: 14 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 75 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 KiB/s wr, 31 op/s 2024-09-21T00:36:10.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:10 smithi089 ceph-mon[34481]: pgmap v843: 129 pgs: 14 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 75 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 KiB/s wr, 31 op/s 2024-09-21T00:36:10.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:10 smithi098 ceph-mon[34725]: pgmap v843: 129 pgs: 14 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 75 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 KiB/s wr, 31 op/s 2024-09-21T00:36:12.254 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-21T00:36:12.255 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-21T00:36:12.255 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-21T00:36:12.255 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-21T00:36:12.255 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-21T00:36:12.255 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-21T00:36:12.255 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-21T00:36:12.255 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-09-21T00:36:12.255 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T00:36:12.256 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T00:36:12.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:12 smithi098 ceph-mon[34725]: pgmap v844: 129 pgs: 21 active+clean+snaptrim, 61 active+clean+snaptrim_wait, 47 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 58 KiB/s wr, 28 op/s 2024-09-21T00:36:12.778 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:36:12.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:12 smithi027 ceph-mon[30745]: pgmap v844: 129 pgs: 21 active+clean+snaptrim, 61 active+clean+snaptrim_wait, 47 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 58 KiB/s wr, 28 op/s 2024-09-21T00:36:12.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:12 smithi089 ceph-mon[34481]: pgmap v844: 129 pgs: 21 active+clean+snaptrim, 61 active+clean+snaptrim_wait, 47 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 58 KiB/s wr, 28 op/s 2024-09-21T00:36:13.447 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-21T00:36:13.447 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T00:36:13.448 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 2107, 2024-09-21T00:36:13.448 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 2109, 2024-09-21T00:36:13.448 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 84520294, 2024-09-21T00:36:13.448 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 670033, 2024-09-21T00:36:13.448 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 8, 2024-09-21T00:36:13.448 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-09-21T00:36:13.448 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.271336143, 2024-09-21T00:36:13.448 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.097286913, 2024-09-21T00:36:13.448 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.093095734, 2024-09-21T00:36:13.448 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.064811659, 2024-09-21T00:36:13.448 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:13.448 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2107, 2024-09-21T00:36:13.448 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.103570628, 2024-09-21T00:36:13.449 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000049155 2024-09-21T00:36:13.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.449 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:13.449 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.449 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.449 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.449 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T00:36:13.449 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T00:36:13.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.449 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T00:36:13.449 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 520, 2024-09-21T00:36:13.449 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 740, 2024-09-21T00:36:13.449 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 78985757, 2024-09-21T00:36:13.450 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 704080, 2024-09-21T00:36:13.450 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 8, 2024-09-21T00:36:13.450 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 6, 2024-09-21T00:36:13.450 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.105512535, 2024-09-21T00:36:13.450 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.030195365, 2024-09-21T00:36:13.450 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.055163320, 2024-09-21T00:36:13.450 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.014391752, 2024-09-21T00:36:13.450 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:13.450 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 738, 2024-09-21T00:36:13.450 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.032514486, 2024-09-21T00:36:13.450 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000044057 2024-09-21T00:36:13.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.450 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:13.451 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.451 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.451 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.451 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T00:36:13.451 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 538976 2024-09-21T00:36:13.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.451 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T00:36:13.451 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 1170, 2024-09-21T00:36:13.451 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 1140, 2024-09-21T00:36:13.451 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 89307472, 2024-09-21T00:36:13.451 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 444442, 2024-09-21T00:36:13.451 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 5, 2024-09-21T00:36:13.452 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-09-21T00:36:13.452 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.182386300, 2024-09-21T00:36:13.452 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.055623413, 2024-09-21T00:36:13.452 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.108527110, 2024-09-21T00:36:13.452 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.022725389, 2024-09-21T00:36:13.452 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:13.452 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1140, 2024-09-21T00:36:13.452 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.111096380, 2024-09-21T00:36:13.452 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000097452 2024-09-21T00:36:13.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.452 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:13.452 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.452 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.452 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.453 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 994912, 2024-09-21T00:36:13.453 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 184000 2024-09-21T00:36:13.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.453 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-09-21T00:36:13.453 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-09-21T00:36:13.453 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-09-21T00:36:13.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.453 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-09-21T00:36:13.453 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-09-21T00:36:13.453 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-09-21T00:36:13.453 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 5685, 2024-09-21T00:36:13.453 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.151932618, 2024-09-21T00:36:13.454 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000026725 2024-09-21T00:36:13.454 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:13.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.454 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-09-21T00:36:13.454 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-09-21T00:36:13.454 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-09-21T00:36:13.454 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.454 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.454 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.454 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:13.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.454 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-09-21T00:36:13.454 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 0, 2024-09-21T00:36:13.455 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 0, 2024-09-21T00:36:13.455 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-09-21T00:36:13.455 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.455 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.455 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.455 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-09-21T00:36:13.455 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 0, 2024-09-21T00:36:13.455 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-09-21T00:36:13.455 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-09-21T00:36:13.455 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 0, 2024-09-21T00:36:13.455 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-09-21T00:36:13.456 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-09-21T00:36:13.456 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 58839, 2024-09-21T00:36:13.456 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 9202, 2024-09-21T00:36:13.456 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 49637, 2024-09-21T00:36:13.456 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-09-21T00:36:13.456 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 45692, 2024-09-21T00:36:13.456 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 1, 2024-09-21T00:36:13.456 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 0, 2024-09-21T00:36:13.456 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 0, 2024-09-21T00:36:13.456 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 1, 2024-09-21T00:36:13.456 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 0, 2024-09-21T00:36:13.456 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 0, 2024-09-21T00:36:13.458 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 0, 2024-09-21T00:36:13.458 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 0, 2024-09-21T00:36:13.458 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-09-21T00:36:13.458 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-09-21T00:36:13.458 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-09-21T00:36:13.458 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-09-21T00:36:13.458 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-09-21T00:36:13.458 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-09-21T00:36:13.458 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-09-21T00:36:13.459 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-09-21T00:36:13.459 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-09-21T00:36:13.459 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-09-21T00:36:13.459 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-09-21T00:36:13.459 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-09-21T00:36:13.459 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 0, 2024-09-21T00:36:13.459 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 0, 2024-09-21T00:36:13.459 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2024-09-21T00:36:13.459 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T00:36:13.459 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-09-21T00:36:13.459 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T00:36:13.459 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-09-21T00:36:13.459 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-09-21T00:36:13.460 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T00:36:13.460 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T00:36:13.460 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-09-21T00:36:13.460 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-09-21T00:36:13.460 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T00:36:13.460 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2024-09-21T00:36:13.460 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T00:36:13.460 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T00:36:13.460 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T00:36:13.460 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 0, 2024-09-21T00:36:13.460 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2024-09-21T00:36:13.460 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2024-09-21T00:36:13.460 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 0 2024-09-21T00:36:13.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.461 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-09-21T00:36:13.461 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-09-21T00:36:13.461 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2024-09-21T00:36:13.461 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2024-09-21T00:36:13.461 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 0, 2024-09-21T00:36:13.461 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2024-09-21T00:36:13.461 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-09-21T00:36:13.461 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-09-21T00:36:13.461 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-09-21T00:36:13.461 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2024-09-21T00:36:13.461 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-09-21T00:36:13.461 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-09-21T00:36:13.461 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-09-21T00:36:13.462 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-09-21T00:36:13.462 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-09-21T00:36:13.462 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-09-21T00:36:13.462 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-09-21T00:36:13.462 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-09-21T00:36:13.462 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 0, 2024-09-21T00:36:13.462 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 0, 2024-09-21T00:36:13.462 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T00:36:13.462 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-09-21T00:36:13.462 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-09-21T00:36:13.462 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-09-21T00:36:13.462 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-09-21T00:36:13.462 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0, 2024-09-21T00:36:13.462 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_started": 0, 2024-09-21T00:36:13.463 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_succeeded": 0, 2024-09-21T00:36:13.463 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_write_failed": 0 2024-09-21T00:36:13.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.463 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-09-21T00:36:13.463 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-09-21T00:36:13.463 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 0, 2024-09-21T00:36:13.463 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-09-21T00:36:13.463 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-09-21T00:36:13.463 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 64442, 2024-09-21T00:36:13.463 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-09-21T00:36:13.463 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-09-21T00:36:13.463 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 0, 2024-09-21T00:36:13.463 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-09-21T00:36:13.464 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-09-21T00:36:13.464 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 92, 2024-09-21T00:36:13.464 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 7, 2024-09-21T00:36:13.464 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-09-21T00:36:13.464 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-09-21T00:36:13.464 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2024-09-21T00:36:13.464 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 255007474, 2024-09-21T00:36:13.464 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 255000656, 2024-09-21T00:36:13.464 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-09-21T00:36:13.464 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.464 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.464 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.464 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 64442 2024-09-21T00:36:13.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.465 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-09-21T00:36:13.465 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 22214, 2024-09-21T00:36:13.465 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 85619, 2024-09-21T00:36:13.465 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 63405, 2024-09-21T00:36:13.465 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 650, 2024-09-21T00:36:13.465 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 2190, 2024-09-21T00:36:13.465 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 1540, 2024-09-21T00:36:13.465 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 58839, 2024-09-21T00:36:13.465 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 130446, 2024-09-21T00:36:13.465 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 71607, 2024-09-21T00:36:13.465 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 0, 2024-09-21T00:36:13.465 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 0, 2024-09-21T00:36:13.465 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 0, 2024-09-21T00:36:13.466 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 244712, 2024-09-21T00:36:13.466 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 482598912 2024-09-21T00:36:13.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.466 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-09-21T00:36:13.466 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 0, 2024-09-21T00:36:13.466 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2024-09-21T00:36:13.466 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 0, 2024-09-21T00:36:13.466 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 0, 2024-09-21T00:36:13.466 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 0, 2024-09-21T00:36:13.466 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-09-21T00:36:13.466 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.466 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.466 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.466 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-09-21T00:36:13.467 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.467 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.467 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.467 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-09-21T00:36:13.467 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.467 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.467 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.467 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-09-21T00:36:13.467 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.467 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.467 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.468 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-09-21T00:36:13.468 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.468 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.468 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.468 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-09-21T00:36:13.468 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.468 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.468 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.468 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-09-21T00:36:13.468 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.469 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.469 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.469 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-09-21T00:36:13.469 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.469 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.469 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.469 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-09-21T00:36:13.469 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.469 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.469 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.469 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-09-21T00:36:13.470 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.470 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.470 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.470 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-09-21T00:36:13.470 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.471 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.471 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.471 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-09-21T00:36:13.471 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.471 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.471 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.471 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-09-21T00:36:13.471 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.471 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.471 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.471 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-09-21T00:36:13.472 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.472 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.472 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.472 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-09-21T00:36:13.472 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.472 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.472 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.472 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-09-21T00:36:13.472 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.472 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.472 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.473 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-09-21T00:36:13.473 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.473 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.473 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.473 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-09-21T00:36:13.473 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.473 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.473 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.473 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-09-21T00:36:13.473 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.473 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.474 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.474 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-09-21T00:36:13.474 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.474 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.474 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.474 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-09-21T00:36:13.474 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.474 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.474 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.474 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-09-21T00:36:13.474 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.474 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.475 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.475 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-09-21T00:36:13.475 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.475 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.475 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.475 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-09-21T00:36:13.475 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.475 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.475 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.475 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-09-21T00:36:13.475 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.476 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.476 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.476 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-09-21T00:36:13.476 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.476 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.476 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.476 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-09-21T00:36:13.476 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.476 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.476 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.476 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-09-21T00:36:13.476 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.477 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.477 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.477 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-09-21T00:36:13.477 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.477 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.477 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.477 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-09-21T00:36:13.477 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-09-21T00:36:13.477 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-09-21T00:36:13.477 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.477 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.478 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.478 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:13.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.478 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-09-21T00:36:13.478 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 2, 2024-09-21T00:36:13.478 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 3, 2024-09-21T00:36:13.478 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 1, 2024-09-21T00:36:13.478 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 2, 2024-09-21T00:36:13.478 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-09-21T00:36:13.478 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 0, 2024-09-21T00:36:13.478 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 0, 2024-09-21T00:36:13.478 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 2490, 2024-09-21T00:36:13.478 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-09-21T00:36:13.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.479 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-09-21T00:36:13.479 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-09-21T00:36:13.479 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-09-21T00:36:13.479 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T00:36:13.479 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-09-21T00:36:13.479 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T00:36:13.479 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-09-21T00:36:13.479 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T00:36:13.479 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T00:36:13.479 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T00:36:13.479 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T00:36:13.479 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T00:36:13.479 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-09-21T00:36:13.479 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T00:36:13.480 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T00:36:13.480 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-09-21T00:36:13.480 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-09-21T00:36:13.480 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-09-21T00:36:13.480 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-09-21T00:36:13.480 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T00:36:13.480 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T00:36:13.480 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T00:36:13.480 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T00:36:13.480 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T00:36:13.480 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-09-21T00:36:13.480 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-09-21T00:36:13.480 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-09-21T00:36:13.480 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T00:36:13.481 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T00:36:13.481 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-09-21T00:36:13.481 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-09-21T00:36:13.481 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-09-21T00:36:13.481 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-09-21T00:36:13.481 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T00:36:13.481 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-09-21T00:36:13.481 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T00:36:13.481 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-09-21T00:36:13.481 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 1072382, 2024-09-21T00:36:13.481 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 22571, 2024-09-21T00:36:13.481 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-09-21T00:36:13.481 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-09-21T00:36:13.481 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-09-21T00:36:13.481 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-09-21T00:36:13.482 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-09-21T00:36:13.482 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-09-21T00:36:13.482 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-09-21T00:36:13.482 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-09-21T00:36:13.482 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 10112, 2024-09-21T00:36:13.482 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 212, 2024-09-21T00:36:13.482 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T00:36:13.482 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-09-21T00:36:13.482 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-09-21T00:36:13.482 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-09-21T00:36:13.482 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 93946982, 2024-09-21T00:36:13.482 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 1064733, 2024-09-21T00:36:13.482 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-09-21T00:36:13.482 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-09-21T00:36:13.483 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-09-21T00:36:13.483 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-09-21T00:36:13.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.483 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-09-21T00:36:13.483 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-09-21T00:36:13.483 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-09-21T00:36:13.483 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 3445, 2024-09-21T00:36:13.483 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 21, 2024-09-21T00:36:13.483 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 3, 2024-09-21T00:36:13.483 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 3445, 2024-09-21T00:36:13.483 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-09-21T00:36:13.483 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3445, 2024-09-21T00:36:13.483 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 22.962709482, 2024-09-21T00:36:13.483 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.006665517 2024-09-21T00:36:13.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.484 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-09-21T00:36:13.484 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-09-21T00:36:13.484 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3445, 2024-09-21T00:36:13.484 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 3446 2024-09-21T00:36:13.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.484 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 3445, 2024-09-21T00:36:13.484 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 3445, 2024-09-21T00:36:13.484 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 0, 2024-09-21T00:36:13.484 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-09-21T00:36:13.484 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-09-21T00:36:13.484 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 2080, 2024-09-21T00:36:13.484 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 0, 2024-09-21T00:36:13.484 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 1363, 2024-09-21T00:36:13.484 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 0, 2024-09-21T00:36:13.485 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2024-09-21T00:36:13.485 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-09-21T00:36:13.485 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-09-21T00:36:13.485 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-09-21T00:36:13.485 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-09-21T00:36:13.485 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2024-09-21T00:36:13.485 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-09-21T00:36:13.485 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-09-21T00:36:13.485 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-09-21T00:36:13.485 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 1, 2024-09-21T00:36:13.485 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 0, 2024-09-21T00:36:13.485 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-09-21T00:36:13.485 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-09-21T00:36:13.485 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-09-21T00:36:13.485 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-09-21T00:36:13.486 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-09-21T00:36:13.486 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-09-21T00:36:13.486 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T00:36:13.486 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-09-21T00:36:13.486 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-09-21T00:36:13.486 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-09-21T00:36:13.486 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-09-21T00:36:13.486 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-09-21T00:36:13.486 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-09-21T00:36:13.486 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-09-21T00:36:13.486 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-09-21T00:36:13.486 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-09-21T00:36:13.486 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-09-21T00:36:13.486 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-09-21T00:36:13.486 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-09-21T00:36:13.487 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-09-21T00:36:13.487 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-09-21T00:36:13.487 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-09-21T00:36:13.487 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-09-21T00:36:13.487 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-09-21T00:36:13.487 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-09-21T00:36:13.487 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-09-21T00:36:13.487 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 113, 2024-09-21T00:36:13.487 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-09-21T00:36:13.487 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 25, 2024-09-21T00:36:13.487 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 11, 2024-09-21T00:36:13.487 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 11, 2024-09-21T00:36:13.487 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-09-21T00:36:13.487 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-09-21T00:36:13.487 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2024-09-21T00:36:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 2, 2024-09-21T00:36:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2024-09-21T00:36:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-09-21T00:36:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 0, 2024-09-21T00:36:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-09-21T00:36:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-09-21T00:36:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-09-21T00:36:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-09-21T00:36:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-09-21T00:36:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T00:36:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-09-21T00:36:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-09-21T00:36:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-09-21T00:36:13.489 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-09-21T00:36:13.489 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-09-21T00:36:13.489 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.489 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T00:36:13.489 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:13.489 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T00:36:13.489 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:13.489 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 3793, 2024-09-21T00:36:13.489 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 252451534, 2024-09-21T00:36:13.489 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:13.489 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 3793, 2024-09-21T00:36:13.489 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T00:36:13.489 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T00:36:13.489 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3793, 2024-09-21T00:36:13.489 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 252451534, 2024-09-21T00:36:13.490 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:13.490 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.490 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.490 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.490 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:13.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.490 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-09-21T00:36:13.490 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:13.490 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T00:36:13.490 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:13.490 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T00:36:13.490 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T00:36:13.490 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:13.490 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:13.490 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 3445, 2024-09-21T00:36:13.491 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 250860323, 2024-09-21T00:36:13.491 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3445, 2024-09-21T00:36:13.491 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 250860323, 2024-09-21T00:36:13.491 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:13.491 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.491 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.491 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.491 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:13.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.491 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-09-21T00:36:13.491 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:13.491 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-09-21T00:36:13.491 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:13.491 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T00:36:13.491 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T00:36:13.492 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:13.492 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:13.492 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 3445, 2024-09-21T00:36:13.492 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 3445, 2024-09-21T00:36:13.492 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3445, 2024-09-21T00:36:13.492 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 3445, 2024-09-21T00:36:13.492 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:13.492 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.492 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.492 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.492 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:13.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.492 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-09-21T00:36:13.492 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:13.493 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-09-21T00:36:13.493 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:13.493 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T00:36:13.493 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T00:36:13.493 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:13.493 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:13.493 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T00:36:13.493 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T00:36:13.493 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-09-21T00:36:13.493 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-09-21T00:36:13.493 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:13.493 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.493 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.493 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.493 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:13.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.494 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x560aa23391e0": { 2024-09-21T00:36:13.494 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:13.494 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-09-21T00:36:13.494 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:13.494 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T00:36:13.494 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T00:36:13.494 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:13.494 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:13.494 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T00:36:13.494 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T00:36:13.494 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-09-21T00:36:13.494 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-09-21T00:36:13.494 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:13.495 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:13.495 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:13.495 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:13.495 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:13.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:13.495 INFO:teuthology.orchestra.run.smithi098.stdout: "trackedop": { 2024-09-21T00:36:13.495 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2024-09-21T00:36:13.495 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:13.495 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-21T00:36:14.475 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-21T00:36:14.476 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-21T00:36:14.476 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-21T00:36:14.476 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-21T00:36:14.476 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-09-21T00:36:14.476 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-21T00:36:14.476 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T00:36:14.476 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-09-21T00:36:14.476 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T00:36:14.476 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T00:36:14.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:14 smithi098 ceph-mon[34725]: pgmap v845: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-21T00:36:14.814 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:36:14.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:14 smithi027 ceph-mon[30745]: pgmap v845: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-21T00:36:14.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:14 smithi089 ceph-mon[34481]: pgmap v845: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-21T00:36:15.437 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-21T00:36:15.438 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T00:36:15.438 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 1436, 2024-09-21T00:36:15.438 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 1438, 2024-09-21T00:36:15.438 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 83622791, 2024-09-21T00:36:15.438 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 456712, 2024-09-21T00:36:15.438 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 7, 2024-09-21T00:36:15.438 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 3, 2024-09-21T00:36:15.438 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.174512814, 2024-09-21T00:36:15.438 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.053516872, 2024-09-21T00:36:15.438 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.072032024, 2024-09-21T00:36:15.439 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.038965126, 2024-09-21T00:36:15.439 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:15.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1436, 2024-09-21T00:36:15.439 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.063288063, 2024-09-21T00:36:15.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000044072 2024-09-21T00:36:15.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.439 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:15.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.439 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.439 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T00:36:15.440 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T00:36:15.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.440 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T00:36:15.440 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 377, 2024-09-21T00:36:15.440 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 597, 2024-09-21T00:36:15.440 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 54629946, 2024-09-21T00:36:15.440 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 668055, 2024-09-21T00:36:15.440 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 7, 2024-09-21T00:36:15.440 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 5, 2024-09-21T00:36:15.440 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.069281008, 2024-09-21T00:36:15.440 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.026611519, 2024-09-21T00:36:15.440 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.030015268, 2024-09-21T00:36:15.441 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.008556762, 2024-09-21T00:36:15.441 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:15.441 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 595, 2024-09-21T00:36:15.441 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.024892746, 2024-09-21T00:36:15.441 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000041836 2024-09-21T00:36:15.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.441 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:15.441 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.441 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.441 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.441 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T00:36:15.441 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 548800 2024-09-21T00:36:15.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.442 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T00:36:15.442 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 1734, 2024-09-21T00:36:15.442 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 1704, 2024-09-21T00:36:15.442 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 60148227, 2024-09-21T00:36:15.442 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 624116, 2024-09-21T00:36:15.442 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 5, 2024-09-21T00:36:15.442 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-09-21T00:36:15.442 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.258030451, 2024-09-21T00:36:15.442 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.093205343, 2024-09-21T00:36:15.442 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.115188626, 2024-09-21T00:36:15.442 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.048491294, 2024-09-21T00:36:15.442 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:15.443 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1704, 2024-09-21T00:36:15.443 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.080580809, 2024-09-21T00:36:15.443 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000047289 2024-09-21T00:36:15.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.443 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:15.443 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.443 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.443 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.443 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 995184, 2024-09-21T00:36:15.443 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 184640 2024-09-21T00:36:15.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.443 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-09-21T00:36:15.444 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-09-21T00:36:15.444 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-09-21T00:36:15.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.444 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-09-21T00:36:15.444 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-09-21T00:36:15.444 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-09-21T00:36:15.444 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 5288, 2024-09-21T00:36:15.444 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.150102826, 2024-09-21T00:36:15.444 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000028385 2024-09-21T00:36:15.444 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:15.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.444 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-09-21T00:36:15.444 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-09-21T00:36:15.444 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-09-21T00:36:15.444 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.445 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.445 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.445 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:15.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.445 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-09-21T00:36:15.445 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 0, 2024-09-21T00:36:15.445 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 0, 2024-09-21T00:36:15.445 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-09-21T00:36:15.445 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.445 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.445 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.445 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-09-21T00:36:15.445 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 0, 2024-09-21T00:36:15.445 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-09-21T00:36:15.446 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-09-21T00:36:15.446 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 0, 2024-09-21T00:36:15.446 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-09-21T00:36:15.446 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-09-21T00:36:15.446 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 23363, 2024-09-21T00:36:15.446 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 1358, 2024-09-21T00:36:15.446 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 22005, 2024-09-21T00:36:15.446 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-09-21T00:36:15.446 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 21422, 2024-09-21T00:36:15.446 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 0, 2024-09-21T00:36:15.446 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 0, 2024-09-21T00:36:15.446 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 0, 2024-09-21T00:36:15.446 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 1, 2024-09-21T00:36:15.446 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 0, 2024-09-21T00:36:15.446 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 0, 2024-09-21T00:36:15.447 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 0, 2024-09-21T00:36:15.447 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 0, 2024-09-21T00:36:15.447 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-09-21T00:36:15.447 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-09-21T00:36:15.447 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-09-21T00:36:15.447 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-09-21T00:36:15.447 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-09-21T00:36:15.447 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-09-21T00:36:15.447 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-09-21T00:36:15.447 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-09-21T00:36:15.447 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-09-21T00:36:15.447 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-09-21T00:36:15.447 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-09-21T00:36:15.447 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-09-21T00:36:15.447 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 0, 2024-09-21T00:36:15.448 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 0, 2024-09-21T00:36:15.448 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2024-09-21T00:36:15.448 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T00:36:15.448 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-09-21T00:36:15.448 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T00:36:15.448 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-09-21T00:36:15.448 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-09-21T00:36:15.448 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T00:36:15.448 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T00:36:15.448 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-09-21T00:36:15.448 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-09-21T00:36:15.448 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T00:36:15.448 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2024-09-21T00:36:15.448 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T00:36:15.448 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T00:36:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T00:36:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 0, 2024-09-21T00:36:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2024-09-21T00:36:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2024-09-21T00:36:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 0 2024-09-21T00:36:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-09-21T00:36:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-09-21T00:36:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2024-09-21T00:36:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2024-09-21T00:36:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 0, 2024-09-21T00:36:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2024-09-21T00:36:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-09-21T00:36:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-09-21T00:36:15.450 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-09-21T00:36:15.450 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2024-09-21T00:36:15.450 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-09-21T00:36:15.450 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-09-21T00:36:15.450 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-09-21T00:36:15.450 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-09-21T00:36:15.450 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-09-21T00:36:15.450 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-09-21T00:36:15.450 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-09-21T00:36:15.450 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-09-21T00:36:15.450 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 0, 2024-09-21T00:36:15.450 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 0, 2024-09-21T00:36:15.450 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T00:36:15.450 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0, 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_started": 0, 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_succeeded": 0, 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_write_failed": 0 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 0, 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 67379, 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-09-21T00:36:15.451 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 0, 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 93, 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 7, 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 200676597, 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 200670252, 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.452 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 67379 2024-09-21T00:36:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-09-21T00:36:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 9138, 2024-09-21T00:36:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 37504, 2024-09-21T00:36:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 28366, 2024-09-21T00:36:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 255, 2024-09-21T00:36:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 943, 2024-09-21T00:36:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 688, 2024-09-21T00:36:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 23363, 2024-09-21T00:36:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 55611, 2024-09-21T00:36:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 32248, 2024-09-21T00:36:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 0, 2024-09-21T00:36:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 0, 2024-09-21T00:36:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 0, 2024-09-21T00:36:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 198748, 2024-09-21T00:36:15.454 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 440651776 2024-09-21T00:36:15.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.454 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-09-21T00:36:15.454 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 0, 2024-09-21T00:36:15.454 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2024-09-21T00:36:15.454 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 0, 2024-09-21T00:36:15.454 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 0, 2024-09-21T00:36:15.454 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 0, 2024-09-21T00:36:15.454 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-09-21T00:36:15.454 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.454 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.454 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.454 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-09-21T00:36:15.455 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.456 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.457 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.457 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-09-21T00:36:15.457 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.457 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.457 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.457 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-09-21T00:36:15.457 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.457 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.457 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.457 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-09-21T00:36:15.458 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.458 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.458 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.458 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-09-21T00:36:15.458 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.458 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.458 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.458 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-09-21T00:36:15.458 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.459 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.459 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.459 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-09-21T00:36:15.459 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.459 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.459 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.459 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-09-21T00:36:15.459 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.459 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.459 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.460 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-09-21T00:36:15.460 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.460 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.460 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.460 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-09-21T00:36:15.460 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.460 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.460 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.460 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-09-21T00:36:15.460 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.460 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.460 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.461 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-09-21T00:36:15.461 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.461 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.461 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.461 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-09-21T00:36:15.461 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.461 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.461 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.461 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-09-21T00:36:15.461 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.461 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.461 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.462 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-09-21T00:36:15.462 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.462 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.462 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.462 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-09-21T00:36:15.462 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.462 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.462 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.462 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-09-21T00:36:15.462 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.463 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.463 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.463 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-09-21T00:36:15.463 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.463 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.463 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.463 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-09-21T00:36:15.463 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.463 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.463 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.464 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-09-21T00:36:15.464 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.464 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.464 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.464 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-09-21T00:36:15.464 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.464 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.464 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.464 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-09-21T00:36:15.464 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.464 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.464 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.465 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-09-21T00:36:15.465 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-09-21T00:36:15.465 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-09-21T00:36:15.465 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.465 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.465 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.465 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:15.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.465 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-09-21T00:36:15.465 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 2, 2024-09-21T00:36:15.465 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 2, 2024-09-21T00:36:15.465 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 0, 2024-09-21T00:36:15.465 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 2, 2024-09-21T00:36:15.465 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-09-21T00:36:15.466 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 0, 2024-09-21T00:36:15.466 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 0, 2024-09-21T00:36:15.466 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 2492, 2024-09-21T00:36:15.466 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-09-21T00:36:15.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.466 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-09-21T00:36:15.466 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-09-21T00:36:15.466 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-09-21T00:36:15.466 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T00:36:15.466 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-09-21T00:36:15.466 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T00:36:15.466 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-09-21T00:36:15.466 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T00:36:15.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T00:36:15.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T00:36:15.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T00:36:15.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T00:36:15.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-09-21T00:36:15.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T00:36:15.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T00:36:15.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-09-21T00:36:15.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-09-21T00:36:15.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-09-21T00:36:15.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-09-21T00:36:15.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T00:36:15.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T00:36:15.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T00:36:15.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T00:36:15.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T00:36:15.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-09-21T00:36:15.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-09-21T00:36:15.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-09-21T00:36:15.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T00:36:15.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T00:36:15.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-09-21T00:36:15.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-09-21T00:36:15.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-09-21T00:36:15.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-09-21T00:36:15.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T00:36:15.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-09-21T00:36:15.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T00:36:15.469 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-09-21T00:36:15.469 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 566609, 2024-09-21T00:36:15.469 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 9311, 2024-09-21T00:36:15.469 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-09-21T00:36:15.469 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-09-21T00:36:15.469 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-09-21T00:36:15.469 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-09-21T00:36:15.469 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-09-21T00:36:15.469 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-09-21T00:36:15.469 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-09-21T00:36:15.469 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-09-21T00:36:15.469 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 10112, 2024-09-21T00:36:15.469 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 212, 2024-09-21T00:36:15.469 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T00:36:15.470 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-09-21T00:36:15.470 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-09-21T00:36:15.470 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-09-21T00:36:15.470 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 38203796, 2024-09-21T00:36:15.470 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 440465, 2024-09-21T00:36:15.470 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-09-21T00:36:15.470 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-09-21T00:36:15.470 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-09-21T00:36:15.470 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-09-21T00:36:15.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.470 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-09-21T00:36:15.470 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-09-21T00:36:15.470 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-09-21T00:36:15.471 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 3193, 2024-09-21T00:36:15.471 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 21, 2024-09-21T00:36:15.471 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 3, 2024-09-21T00:36:15.471 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 3193, 2024-09-21T00:36:15.471 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-09-21T00:36:15.471 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3193, 2024-09-21T00:36:15.471 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 41.568467296, 2024-09-21T00:36:15.471 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.013018624 2024-09-21T00:36:15.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.471 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-09-21T00:36:15.471 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-09-21T00:36:15.471 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3193, 2024-09-21T00:36:15.473 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 3194 2024-09-21T00:36:15.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.474 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 3193, 2024-09-21T00:36:15.474 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 3193, 2024-09-21T00:36:15.474 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 0, 2024-09-21T00:36:15.474 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-09-21T00:36:15.474 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-09-21T00:36:15.474 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 1951, 2024-09-21T00:36:15.474 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 0, 2024-09-21T00:36:15.474 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 1240, 2024-09-21T00:36:15.474 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 0, 2024-09-21T00:36:15.475 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2024-09-21T00:36:15.475 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-09-21T00:36:15.475 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-09-21T00:36:15.475 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-09-21T00:36:15.475 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-09-21T00:36:15.475 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2024-09-21T00:36:15.475 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-09-21T00:36:15.475 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-09-21T00:36:15.475 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-09-21T00:36:15.475 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 1, 2024-09-21T00:36:15.475 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 0, 2024-09-21T00:36:15.475 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-09-21T00:36:15.475 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-09-21T00:36:15.475 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-09-21T00:36:15.476 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-09-21T00:36:15.476 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-09-21T00:36:15.476 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-09-21T00:36:15.476 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T00:36:15.476 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-09-21T00:36:15.476 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-09-21T00:36:15.476 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-09-21T00:36:15.476 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-09-21T00:36:15.476 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-09-21T00:36:15.476 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-09-21T00:36:15.476 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-09-21T00:36:15.476 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-09-21T00:36:15.476 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-09-21T00:36:15.476 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-09-21T00:36:15.476 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-09-21T00:36:15.477 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-09-21T00:36:15.477 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-09-21T00:36:15.477 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-09-21T00:36:15.477 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-09-21T00:36:15.477 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-09-21T00:36:15.477 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-09-21T00:36:15.477 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-09-21T00:36:15.477 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-09-21T00:36:15.477 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 113, 2024-09-21T00:36:15.477 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-09-21T00:36:15.477 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 25, 2024-09-21T00:36:15.477 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 9, 2024-09-21T00:36:15.477 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 9, 2024-09-21T00:36:15.478 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-09-21T00:36:15.478 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-09-21T00:36:15.478 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2024-09-21T00:36:15.478 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 2, 2024-09-21T00:36:15.478 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2024-09-21T00:36:15.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.478 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-09-21T00:36:15.478 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 0, 2024-09-21T00:36:15.478 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-09-21T00:36:15.478 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-09-21T00:36:15.478 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-09-21T00:36:15.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.478 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-09-21T00:36:15.478 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-09-21T00:36:15.479 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T00:36:15.479 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-09-21T00:36:15.479 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-09-21T00:36:15.479 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-09-21T00:36:15.479 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-09-21T00:36:15.479 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-09-21T00:36:15.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.479 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T00:36:15.479 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:15.479 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T00:36:15.479 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:15.480 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 3543, 2024-09-21T00:36:15.480 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 198061380, 2024-09-21T00:36:15.480 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:15.480 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 3543, 2024-09-21T00:36:15.480 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T00:36:15.480 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T00:36:15.480 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3543, 2024-09-21T00:36:15.480 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 198061380, 2024-09-21T00:36:15.480 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:15.480 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.480 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.480 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.480 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:15.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.481 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-09-21T00:36:15.481 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:15.481 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T00:36:15.481 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:15.481 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T00:36:15.481 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T00:36:15.481 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:15.481 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:15.481 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 3193, 2024-09-21T00:36:15.481 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 196512798, 2024-09-21T00:36:15.481 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3193, 2024-09-21T00:36:15.481 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 196512798, 2024-09-21T00:36:15.481 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:15.482 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.482 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.482 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.482 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:15.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.482 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-09-21T00:36:15.482 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:15.482 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-09-21T00:36:15.482 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:15.482 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T00:36:15.482 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T00:36:15.482 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:15.483 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:15.483 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 3193, 2024-09-21T00:36:15.483 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 3193, 2024-09-21T00:36:15.483 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3193, 2024-09-21T00:36:15.483 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 3193, 2024-09-21T00:36:15.483 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:15.483 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.483 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.483 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.483 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:15.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.483 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-09-21T00:36:15.483 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:15.483 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.484 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x555ffc761380": { 2024-09-21T00:36:15.485 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:15.485 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-09-21T00:36:15.485 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:15.485 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T00:36:15.485 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T00:36:15.485 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:15.485 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:15.485 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T00:36:15.485 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T00:36:15.485 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-09-21T00:36:15.485 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-09-21T00:36:15.485 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:15.485 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:15.485 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:15.485 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:15.486 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:15.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:15.486 INFO:teuthology.orchestra.run.smithi098.stdout: "trackedop": { 2024-09-21T00:36:15.486 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2024-09-21T00:36:15.486 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:15.486 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-21T00:36:16.316 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-21T00:36:16.316 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-21T00:36:16.316 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-21T00:36:16.316 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-21T00:36:16.316 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-21T00:36:16.316 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-21T00:36:16.316 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T00:36:16.317 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-21T00:36:16.317 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T00:36:16.317 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T00:36:16.675 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:36:16.745 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:16 smithi098 ceph-mon[34725]: pgmap v846: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-21T00:36:16.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:16 smithi027 ceph-mon[30745]: pgmap v846: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-21T00:36:16.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:16 smithi089 ceph-mon[34481]: pgmap v846: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-21T00:36:17.285 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-21T00:36:17.285 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T00:36:17.285 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 8023, 2024-09-21T00:36:17.285 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 8044, 2024-09-21T00:36:17.285 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 3680924, 2024-09-21T00:36:17.285 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 75105109, 2024-09-21T00:36:17.285 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 13, 2024-09-21T00:36:17.285 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 5, 2024-09-21T00:36:17.286 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 1.108502158, 2024-09-21T00:36:17.286 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.471935674, 2024-09-21T00:36:17.286 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.163152892, 2024-09-21T00:36:17.286 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.426315184, 2024-09-21T00:36:17.286 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:17.286 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 8043, 2024-09-21T00:36:17.286 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.828875204, 2024-09-21T00:36:17.286 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000103055 2024-09-21T00:36:17.286 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.286 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:17.286 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 48, 2024-09-21T00:36:17.286 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000009607, 2024-09-21T00:36:17.286 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000200 2024-09-21T00:36:17.287 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.287 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 5040, 2024-09-21T00:36:17.287 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 8880 2024-09-21T00:36:17.287 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.287 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T00:36:17.287 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 216535, 2024-09-21T00:36:17.287 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 281699, 2024-09-21T00:36:17.287 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 70122881, 2024-09-21T00:36:17.287 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 232149988, 2024-09-21T00:36:17.287 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 16, 2024-09-21T00:36:17.287 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 8, 2024-09-21T00:36:17.287 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 11.801806043, 2024-09-21T00:36:17.287 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 7.179477546, 2024-09-21T00:36:17.287 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 8.268932795, 2024-09-21T00:36:17.287 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.385275506, 2024-09-21T00:36:17.288 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:17.288 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 281697, 2024-09-21T00:36:17.288 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 2253.938872595, 2024-09-21T00:36:17.288 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.008001288 2024-09-21T00:36:17.288 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.288 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:17.288 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 174721, 2024-09-21T00:36:17.288 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.050781539, 2024-09-21T00:36:17.288 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000290 2024-09-21T00:36:17.288 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.288 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T00:36:17.288 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 548448 2024-09-21T00:36:17.288 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.288 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 68731, 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 64384, 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 37961164, 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 59539614, 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 7, 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 6, 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 4.298099175, 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 1.984322296, 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 2.707932056, 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.296038632, 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 64383, 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 3.149403612, 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000048916 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.289 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:17.290 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 117212, 2024-09-21T00:36:17.290 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.018752591, 2024-09-21T00:36:17.290 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000159 2024-09-21T00:36:17.290 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.290 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 1003648, 2024-09-21T00:36:17.290 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 189056 2024-09-21T00:36:17.290 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.290 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-09-21T00:36:17.290 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-09-21T00:36:17.290 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-09-21T00:36:17.290 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.290 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-09-21T00:36:17.290 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-09-21T00:36:17.291 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-09-21T00:36:17.291 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 5606, 2024-09-21T00:36:17.291 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 38.494766806, 2024-09-21T00:36:17.291 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.006866708 2024-09-21T00:36:17.291 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:17.291 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.291 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-09-21T00:36:17.291 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-09-21T00:36:17.291 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-09-21T00:36:17.291 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 12757, 2024-09-21T00:36:17.291 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 2.103200194, 2024-09-21T00:36:17.291 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000164866 2024-09-21T00:36:17.291 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:17.291 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.291 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-09-21T00:36:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 102719, 2024-09-21T00:36:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 89763, 2024-09-21T00:36:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-09-21T00:36:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 89763, 2024-09-21T00:36:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 2380.409497351, 2024-09-21T00:36:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.026518827 2024-09-21T00:36:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-09-21T00:36:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 12956, 2024-09-21T00:36:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 87, 2024-09-21T00:36:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-09-21T00:36:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 531, 2024-09-21T00:36:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 59, 2024-09-21T00:36:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-09-21T00:36:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 13226, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 2, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 5, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 13219, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 13222, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 37774, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 1, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 1, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 3, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 183567, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 142642, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 12506, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 958, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 66, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 419, 2024-09-21T00:36:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-09-21T00:36:17.294 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-09-21T00:36:17.294 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 134, 2024-09-21T00:36:17.294 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 29360, 2024-09-21T00:36:17.294 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 202, 2024-09-21T00:36:17.294 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 19734, 2024-09-21T00:36:17.294 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-09-21T00:36:17.294 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-09-21T00:36:17.294 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-09-21T00:36:17.294 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 1050, 2024-09-21T00:36:17.294 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 23776636, 2024-09-21T00:36:17.294 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 6, 2024-09-21T00:36:17.294 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T00:36:17.294 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-09-21T00:36:17.294 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T00:36:17.294 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 428, 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 55049, 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 34550, 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 66353, 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 14792, 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 5271, 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 20909 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.295 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-09-21T00:36:17.296 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 1202, 2024-09-21T00:36:17.296 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 7031, 2024-09-21T00:36:17.296 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 4341, 2024-09-21T00:36:17.296 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 5294, 2024-09-21T00:36:17.296 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 2048, 2024-09-21T00:36:17.296 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 34, 2024-09-21T00:36:17.296 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-09-21T00:36:17.296 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-09-21T00:36:17.296 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 6552, 2024-09-21T00:36:17.296 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 6518, 2024-09-21T00:36:17.296 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-09-21T00:36:17.296 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-09-21T00:36:17.296 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-09-21T00:36:17.296 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-09-21T00:36:17.296 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 7, 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 44382, 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 134, 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 67, 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0, 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_started": 0, 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_succeeded": 0, 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_write_failed": 0 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-09-21T00:36:17.297 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 2, 2024-09-21T00:36:17.298 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 67379, 2024-09-21T00:36:17.298 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-09-21T00:36:17.298 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-09-21T00:36:17.298 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 67379, 2024-09-21T00:36:17.298 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-09-21T00:36:17.298 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-09-21T00:36:17.298 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 93, 2024-09-21T00:36:17.298 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-09-21T00:36:17.298 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-09-21T00:36:17.298 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 93, 2024-09-21T00:36:17.298 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 7, 2024-09-21T00:36:17.298 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-09-21T00:36:17.298 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-09-21T00:36:17.298 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2024-09-21T00:36:17.298 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 200676597, 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3697, 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 232.510535043, 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.062891678 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 8280, 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 39428, 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 31148, 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 264, 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 3449, 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 3185, 2024-09-21T00:36:17.299 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 13226, 2024-09-21T00:36:17.300 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 59302, 2024-09-21T00:36:17.300 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 46076, 2024-09-21T00:36:17.300 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 1, 2024-09-21T00:36:17.300 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 24254, 2024-09-21T00:36:17.300 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 24253, 2024-09-21T00:36:17.300 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 262672, 2024-09-21T00:36:17.300 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 567988224 2024-09-21T00:36:17.300 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.300 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-09-21T00:36:17.300 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 108249, 2024-09-21T00:36:17.300 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 4056, 2024-09-21T00:36:17.300 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 102719, 2024-09-21T00:36:17.300 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 267, 2024-09-21T00:36:17.300 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 6568, 2024-09-21T00:36:17.300 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-09-21T00:36:17.301 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 8675, 2024-09-21T00:36:17.301 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 106.197913525, 2024-09-21T00:36:17.301 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.012241834 2024-09-21T00:36:17.301 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.301 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-09-21T00:36:17.301 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 14, 2024-09-21T00:36:17.301 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1.599917265, 2024-09-21T00:36:17.301 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.114279804 2024-09-21T00:36:17.301 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.301 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-09-21T00:36:17.301 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.301 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.301 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.301 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.301 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-09-21T00:36:17.302 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.302 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.302 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.302 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.302 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-09-21T00:36:17.302 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 62272, 2024-09-21T00:36:17.302 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1766.678239477, 2024-09-21T00:36:17.302 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.028370346 2024-09-21T00:36:17.302 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.302 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-09-21T00:36:17.302 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.302 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.302 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.303 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.303 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-09-21T00:36:17.303 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2024-09-21T00:36:17.303 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000136874, 2024-09-21T00:36:17.303 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000136874 2024-09-21T00:36:17.303 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.303 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-09-21T00:36:17.303 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.303 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.303 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.303 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.303 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-09-21T00:36:17.303 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.303 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.304 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.304 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.304 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-09-21T00:36:17.304 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.304 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.304 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.304 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.304 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-09-21T00:36:17.305 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.305 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.305 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.305 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.305 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-09-21T00:36:17.305 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 38, 2024-09-21T00:36:17.305 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 17.545126225, 2024-09-21T00:36:17.305 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.461713848 2024-09-21T00:36:17.305 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.305 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-09-21T00:36:17.305 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.305 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.305 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.305 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.305 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-09-21T00:36:17.306 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.306 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.306 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.306 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.306 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-09-21T00:36:17.306 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 5498, 2024-09-21T00:36:17.306 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 70.166300149, 2024-09-21T00:36:17.306 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.012762149 2024-09-21T00:36:17.306 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.306 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-09-21T00:36:17.306 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 234, 2024-09-21T00:36:17.306 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1.187164166, 2024-09-21T00:36:17.306 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.005073351 2024-09-21T00:36:17.306 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.307 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-09-21T00:36:17.307 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 8676, 2024-09-21T00:36:17.307 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 226.466842269, 2024-09-21T00:36:17.307 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.026102678 2024-09-21T00:36:17.307 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.307 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-09-21T00:36:17.307 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.307 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.307 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.307 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.307 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-09-21T00:36:17.307 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.307 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.307 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.308 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.308 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-09-21T00:36:17.308 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 34, 2024-09-21T00:36:17.308 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 180.073257330, 2024-09-21T00:36:17.308 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 5.296272274 2024-09-21T00:36:17.308 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.308 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-09-21T00:36:17.308 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.308 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.308 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.308 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.308 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-09-21T00:36:17.308 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.308 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.309 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.310 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.310 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-09-21T00:36:17.310 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.310 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.310 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.310 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.310 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-09-21T00:36:17.310 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.310 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.310 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.310 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.310 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-09-21T00:36:17.310 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.310 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.310 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.311 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.311 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-09-21T00:36:17.311 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4321, 2024-09-21T00:36:17.311 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 10.494600071, 2024-09-21T00:36:17.311 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002428743 2024-09-21T00:36:17.311 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.311 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-09-21T00:36:17.311 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-09-21T00:36:17.311 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-09-21T00:36:17.311 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.311 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.311 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.311 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:17.311 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.312 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-09-21T00:36:17.312 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 2, 2024-09-21T00:36:17.312 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 2, 2024-09-21T00:36:17.312 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 0, 2024-09-21T00:36:17.312 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 2, 2024-09-21T00:36:17.312 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-09-21T00:36:17.312 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 1084, 2024-09-21T00:36:17.312 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 542, 2024-09-21T00:36:17.312 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 854, 2024-09-21T00:36:17.312 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-09-21T00:36:17.312 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.312 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-09-21T00:36:17.312 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-09-21T00:36:17.312 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-09-21T00:36:17.313 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T00:36:17.313 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-09-21T00:36:17.313 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T00:36:17.313 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-09-21T00:36:17.313 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T00:36:17.313 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T00:36:17.313 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T00:36:17.313 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T00:36:17.313 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T00:36:17.313 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-09-21T00:36:17.313 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T00:36:17.313 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T00:36:17.313 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-09-21T00:36:17.313 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-09-21T00:36:17.313 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-09-21T00:36:17.314 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-09-21T00:36:17.314 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T00:36:17.314 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T00:36:17.314 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T00:36:17.314 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T00:36:17.314 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T00:36:17.314 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-09-21T00:36:17.314 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-09-21T00:36:17.314 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-09-21T00:36:17.314 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T00:36:17.314 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T00:36:17.314 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-09-21T00:36:17.314 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-09-21T00:36:17.314 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-09-21T00:36:17.315 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-09-21T00:36:17.315 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T00:36:17.315 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-09-21T00:36:17.315 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T00:36:17.315 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-09-21T00:36:17.315 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 836385, 2024-09-21T00:36:17.315 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 6106, 2024-09-21T00:36:17.315 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-09-21T00:36:17.315 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-09-21T00:36:17.315 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-09-21T00:36:17.315 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-09-21T00:36:17.315 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-09-21T00:36:17.315 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-09-21T00:36:17.315 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-09-21T00:36:17.315 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-09-21T00:36:17.316 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 10112, 2024-09-21T00:36:17.316 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 212, 2024-09-21T00:36:17.316 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T00:36:17.316 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-09-21T00:36:17.316 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-09-21T00:36:17.316 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-09-21T00:36:17.316 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 29121928, 2024-09-21T00:36:17.316 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 329552, 2024-09-21T00:36:17.316 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-09-21T00:36:17.316 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-09-21T00:36:17.316 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-09-21T00:36:17.316 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-09-21T00:36:17.316 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.316 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-09-21T00:36:17.316 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-09-21T00:36:17.317 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-09-21T00:36:17.317 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 18663, 2024-09-21T00:36:17.317 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 209291724, 2024-09-21T00:36:17.317 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 33, 2024-09-21T00:36:17.317 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 18663, 2024-09-21T00:36:17.317 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-09-21T00:36:17.317 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 18663, 2024-09-21T00:36:17.317 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 639.377827719, 2024-09-21T00:36:17.317 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.034259113 2024-09-21T00:36:17.317 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.317 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-09-21T00:36:17.317 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-09-21T00:36:17.317 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 18663, 2024-09-21T00:36:17.317 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 19855 2024-09-21T00:36:17.318 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.318 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 18663, 2024-09-21T00:36:17.318 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 419, 2024-09-21T00:36:17.318 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 18244, 2024-09-21T00:36:17.318 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-09-21T00:36:17.318 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-09-21T00:36:17.318 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-09-21T00:36:17.318 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 10, 2024-09-21T00:36:17.318 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 332, 2024-09-21T00:36:17.318 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 4032, 2024-09-21T00:36:17.318 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 197, 2024-09-21T00:36:17.318 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-09-21T00:36:17.318 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-09-21T00:36:17.318 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-09-21T00:36:17.319 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 59, 2024-09-21T00:36:17.319 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 13094, 2024-09-21T00:36:17.319 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-09-21T00:36:17.319 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-09-21T00:36:17.319 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-09-21T00:36:17.319 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 1, 2024-09-21T00:36:17.319 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 20, 2024-09-21T00:36:17.319 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-09-21T00:36:17.319 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-09-21T00:36:17.319 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-09-21T00:36:17.319 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-09-21T00:36:17.319 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-09-21T00:36:17.319 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-09-21T00:36:17.319 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T00:36:17.319 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-09-21T00:36:17.320 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-09-21T00:36:17.320 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-09-21T00:36:17.320 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-09-21T00:36:17.320 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-09-21T00:36:17.320 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-09-21T00:36:17.320 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-09-21T00:36:17.320 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-09-21T00:36:17.320 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-09-21T00:36:17.320 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-09-21T00:36:17.320 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-09-21T00:36:17.320 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-09-21T00:36:17.320 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-09-21T00:36:17.320 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-09-21T00:36:17.320 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-09-21T00:36:17.320 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-09-21T00:36:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-09-21T00:36:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-09-21T00:36:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-09-21T00:36:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 113, 2024-09-21T00:36:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-09-21T00:36:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 25, 2024-09-21T00:36:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 12, 2024-09-21T00:36:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 12, 2024-09-21T00:36:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-09-21T00:36:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-09-21T00:36:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 1651, 2024-09-21T00:36:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 174, 2024-09-21T00:36:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 285 2024-09-21T00:36:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 16, 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 8, 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 13036, 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 6518, 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T00:36:17.322 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 293285, 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 88708390, 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 293285, 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 293285, 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 88708390, 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.323 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-09-21T00:36:17.324 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:17.324 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T00:36:17.324 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:17.324 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T00:36:17.324 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T00:36:17.324 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:17.324 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:17.324 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 18663, 2024-09-21T00:36:17.324 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 210188154, 2024-09-21T00:36:17.324 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 18663, 2024-09-21T00:36:17.324 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 210188154, 2024-09-21T00:36:17.324 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:17.324 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.324 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.325 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.325 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:17.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.325 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-09-21T00:36:17.325 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:17.325 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-09-21T00:36:17.325 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:17.325 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T00:36:17.325 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T00:36:17.325 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:17.325 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T00:36:17.325 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 18663, 2024-09-21T00:36:17.325 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 18663, 2024-09-21T00:36:17.325 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 18663, 2024-09-21T00:36:17.325 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 18663, 2024-09-21T00:36:17.326 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:17.326 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.326 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.326 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.326 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:17.326 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.326 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-09-21T00:36:17.326 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:17.326 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-09-21T00:36:17.326 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:17.326 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 6518, 2024-09-21T00:36:17.326 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 897822, 2024-09-21T00:36:17.326 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:17.326 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 6518, 2024-09-21T00:36:17.326 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T00:36:17.327 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T00:36:17.327 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 335, 2024-09-21T00:36:17.327 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 897822, 2024-09-21T00:36:17.327 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:17.327 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.327 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.327 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.327 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:17.327 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.327 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x55c30a2112b0": { 2024-09-21T00:36:17.327 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:17.327 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-09-21T00:36:17.327 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:17.327 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 67379, 2024-09-21T00:36:17.327 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 196482293, 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 67379, 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3697, 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 196482293, 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout: "trackedop": { 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 44306 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:17.328 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-21T00:36:17.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:17 smithi027 ceph-mon[30745]: pgmap v847: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T00:36:17.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:17 smithi089 ceph-mon[34481]: pgmap v847: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T00:36:17.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:17 smithi098 ceph-mon[34725]: pgmap v847: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T00:36:18.268 INFO:tasks.check_counter:Daemon mds.i mds.exported=134 2024-09-21T00:36:18.268 INFO:tasks.check_counter:Daemon mds.i mds.imported=202 2024-09-21T00:36:18.268 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=1202 2024-09-21T00:36:18.268 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=7031 2024-09-21T00:36:18.268 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-21T00:36:18.269 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-21T00:36:18.269 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T00:36:18.269 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=59 2024-09-21T00:36:18.269 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T00:36:18.269 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T00:36:18.628 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T00:36:19.267 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-21T00:36:19.267 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T00:36:19.267 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 0, 2024-09-21T00:36:19.267 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 2, 2024-09-21T00:36:19.267 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 0, 2024-09-21T00:36:19.267 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 0, 2024-09-21T00:36:19.267 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2024-09-21T00:36:19.267 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 0, 2024-09-21T00:36:19.268 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.001108682, 2024-09-21T00:36:19.268 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T00:36:19.268 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.000225469, 2024-09-21T00:36:19.268 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:36:19.268 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:19.268 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:19.268 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:19.268 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:19.268 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:19.268 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:19.268 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:19.268 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:19.268 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:19.269 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:19.269 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T00:36:19.269 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T00:36:19.269 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:19.269 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T00:36:19.269 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 6, 2024-09-21T00:36:19.269 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 228, 2024-09-21T00:36:19.269 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 3312, 2024-09-21T00:36:19.269 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 201056, 2024-09-21T00:36:19.269 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2024-09-21T00:36:19.269 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2024-09-21T00:36:19.269 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.020040923, 2024-09-21T00:36:19.269 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.014558755, 2024-09-21T00:36:19.269 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.003109862, 2024-09-21T00:36:19.270 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:36:19.270 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:19.270 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 227, 2024-09-21T00:36:19.270 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.017700909, 2024-09-21T00:36:19.270 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000077977 2024-09-21T00:36:19.270 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:19.270 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:19.270 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:19.270 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:19.270 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:19.270 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:19.270 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-21T00:36:19.270 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 201056 2024-09-21T00:36:19.270 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:19.271 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T00:36:19.271 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 300, 2024-09-21T00:36:19.271 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 295, 2024-09-21T00:36:19.271 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 860688, 2024-09-21T00:36:19.271 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 182160, 2024-09-21T00:36:19.271 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 2, 2024-09-21T00:36:19.271 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 1, 2024-09-21T00:36:19.271 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.050527269, 2024-09-21T00:36:19.271 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.023968530, 2024-09-21T00:36:19.271 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.035689433, 2024-09-21T00:36:19.271 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T00:36:19.271 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T00:36:19.271 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 294, 2024-09-21T00:36:19.271 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.017262629, 2024-09-21T00:36:19.272 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000058716 2024-09-21T00:36:19.272 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:19.272 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T00:36:19.272 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:19.272 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:19.272 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:19.272 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:19.272 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 860688, 2024-09-21T00:36:19.272 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 182160 2024-09-21T00:36:19.272 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:19.272 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-09-21T00:36:19.272 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 0, 2024-09-21T00:36:19.272 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-09-21T00:36:19.272 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:19.272 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-09-21T00:36:19.273 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-09-21T00:36:19.273 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-09-21T00:36:19.273 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T00:36:19.273 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-09-21T00:36:19.273 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T00:36:19.273 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-09-21T00:36:19.273 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T00:36:19.273 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T00:36:19.273 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T00:36:19.273 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T00:36:19.273 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T00:36:19.273 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-09-21T00:36:19.273 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T00:36:19.273 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T00:36:19.274 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-09-21T00:36:19.274 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-09-21T00:36:19.274 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-09-21T00:36:19.274 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-09-21T00:36:19.274 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T00:36:19.274 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T00:36:19.274 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T00:36:19.274 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T00:36:19.274 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T00:36:19.274 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-09-21T00:36:19.274 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-09-21T00:36:19.274 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-09-21T00:36:19.274 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T00:36:19.274 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T00:36:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-09-21T00:36:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-09-21T00:36:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-09-21T00:36:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-09-21T00:36:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T00:36:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-09-21T00:36:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T00:36:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-09-21T00:36:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 148704, 2024-09-21T00:36:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 29, 2024-09-21T00:36:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-09-21T00:36:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-09-21T00:36:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-09-21T00:36:19.276 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-09-21T00:36:19.276 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-09-21T00:36:19.276 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-09-21T00:36:19.276 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-09-21T00:36:19.276 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-09-21T00:36:19.276 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 0, 2024-09-21T00:36:19.276 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 0, 2024-09-21T00:36:19.276 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T00:36:19.276 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-09-21T00:36:19.276 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-09-21T00:36:19.276 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-09-21T00:36:19.276 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 968, 2024-09-21T00:36:19.276 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 2, 2024-09-21T00:36:19.276 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-09-21T00:36:19.276 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-09-21T00:36:19.277 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-09-21T00:36:19.277 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-09-21T00:36:19.277 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T00:36:19.277 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T00:36:19.277 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T00:36:19.277 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T00:36:19.277 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T00:36:19.277 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 302, 2024-09-21T00:36:19.277 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 818283, 2024-09-21T00:36:19.277 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T00:36:19.277 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 302, 2024-09-21T00:36:19.277 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T00:36:19.277 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T00:36:19.277 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 302, 2024-09-21T00:36:19.278 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 818283, 2024-09-21T00:36:19.278 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T00:36:19.278 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T00:36:19.278 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T00:36:19.278 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T00:36:19.278 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:19.278 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T00:36:19.278 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-21T00:36:19.957 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-21T00:36:19.957 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-21T00:36:19.957 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-21T00:36:19.958 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-21T00:36:19.958 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-21T00:36:19.958 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-21T00:36:19.958 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-21T00:36:19.958 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-09-21T00:36:19.958 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T00:36:19.967 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T00:36:19.968 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:36:19.968 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:36:20.049 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:36:20.049 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:36:20.119 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T00:36:20.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:19 smithi089 ceph-mon[34481]: pgmap v848: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 198 B/s wr, 6 op/s 2024-09-21T00:36:20.231 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:19 smithi027 ceph-mon[30745]: pgmap v848: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 198 B/s wr, 6 op/s 2024-09-21T00:36:20.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:19 smithi098 ceph-mon[34725]: pgmap v848: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 198 B/s wr, 6 op/s 2024-09-21T00:36:21.369 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:36:22.124 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:21 smithi027 ceph-mon[30745]: pgmap v849: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:36:22.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:21 smithi089 ceph-mon[34481]: pgmap v849: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:36:22.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:21 smithi098 ceph-mon[34725]: pgmap v849: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T00:36:22.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.365+0000 7fe970e00640 1 -- 172.21.15.27:0/3786766848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe96c1025b0 msgr2=0x7fe96c1029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:22.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.365+0000 7fe970e00640 1 --2- 172.21.15.27:0/3786766848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe96c1025b0 0x7fe96c1029c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009930 tx=0x7fe95c02a460 comp rx=0 tx=0).stop 2024-09-21T00:36:22.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.365+0000 7fe970e00640 1 -- 172.21.15.27:0/3786766848 shutdown_connections 2024-09-21T00:36:22.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.365+0000 7fe970e00640 1 --2- 172.21.15.27:0/3786766848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c1079f0 0x7fe96c107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:22.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.365+0000 7fe970e00640 1 --2- 172.21.15.27:0/3786766848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe96c102f00 0x7fe96c107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:22.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.365+0000 7fe970e00640 1 --2- 172.21.15.27:0/3786766848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe96c1025b0 0x7fe96c1029c0 secure :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009930 tx=0x7fe95c02a460 comp rx=0 tx=0).stop 2024-09-21T00:36:22.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.365+0000 7fe970e00640 1 -- 172.21.15.27:0/3786766848 >> 172.21.15.27:0/3786766848 conn(0x7fe96c0fe2c0 msgr2=0x7fe96c100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:22.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.368+0000 7fe970e00640 1 -- 172.21.15.27:0/3786766848 shutdown_connections 2024-09-21T00:36:22.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.368+0000 7fe970e00640 1 -- 172.21.15.27:0/3786766848 wait complete. 2024-09-21T00:36:22.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.370+0000 7fe970e00640 1 Processor -- start 2024-09-21T00:36:22.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.370+0000 7fe970e00640 1 -- start start 2024-09-21T00:36:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.371+0000 7fe970e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe96c102f00 0x7fe96c1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.371+0000 7fe970e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe96c1079f0 0x7fe96c1a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.371+0000 7fe970e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c1a6210 0x7fe96c1a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.371+0000 7fe970e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe96c119f80 con 0x7fe96c1079f0 2024-09-21T00:36:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.371+0000 7fe970e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe96c119e00 con 0x7fe96c102f00 2024-09-21T00:36:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.371+0000 7fe970e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe96c11a100 con 0x7fe96c1a6210 2024-09-21T00:36:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.371+0000 7fe96aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe96c1079f0 0x7fe96c1a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.371+0000 7fe96aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe96c1079f0 0x7fe96c1a1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38838/0 (socket says 172.21.15.27:38838) 2024-09-21T00:36:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.371+0000 7fe96aa00640 1 -- 172.21.15.27:0/414993816 learned_addr learned my addr 172.21.15.27:0/414993816 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:36:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.372+0000 7fe96be00640 1 --2- 172.21.15.27:0/414993816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c1a6210 0x7fe96c1a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.372+0000 7fe96aa00640 1 -- 172.21.15.27:0/414993816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c1a6210 msgr2=0x7fe96c1a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.372+0000 7fe96aa00640 1 --2- 172.21.15.27:0/414993816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c1a6210 0x7fe96c1a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:22.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.372+0000 7fe96aa00640 1 -- 172.21.15.27:0/414993816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe96c102f00 msgr2=0x7fe96c1a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:22.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.372+0000 7fe96aa00640 1 --2- 172.21.15.27:0/414993816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe96c102f00 0x7fe96c1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:22.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.372+0000 7fe96aa00640 1 -- 172.21.15.27:0/414993816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96c1a6d60 con 0x7fe96c1079f0 2024-09-21T00:36:22.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.373+0000 7fe96aa00640 1 --2- 172.21.15.27:0/414993816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe96c1079f0 0x7fe96c1a1030 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe95800c8c0 tx=0x7fe95800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.376+0000 7fe963e00640 1 -- 172.21.15.27:0/414993816 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958007bb0 con 0x7fe96c1079f0 2024-09-21T00:36:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.376+0000 7fe970e00640 1 -- 172.21.15.27:0/414993816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96c1a7050 con 0x7fe96c1079f0 2024-09-21T00:36:22.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.376+0000 7fe970e00640 1 -- 172.21.15.27:0/414993816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe96c10add0 con 0x7fe96c1079f0 2024-09-21T00:36:22.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.377+0000 7fe963e00640 1 -- 172.21.15.27:0/414993816 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe958007d50 con 0x7fe96c1079f0 2024-09-21T00:36:22.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.377+0000 7fe963e00640 1 -- 172.21.15.27:0/414993816 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95803a7c0 con 0x7fe96c1079f0 2024-09-21T00:36:22.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.379+0000 7fe970e00640 1 -- 172.21.15.27:0/414993816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe94c0052c0 con 0x7fe96c1079f0 2024-09-21T00:36:22.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.390+0000 7fe963e00640 1 -- 172.21.15.27:0/414993816 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe95803a960 con 0x7fe96c1079f0 2024-09-21T00:36:22.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.392+0000 7fe963e00640 1 --2- 172.21.15.27:0/414993816 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9500b67b0 0x7fe9500b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:22.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.393+0000 7fe963e00640 1 -- 172.21.15.27:0/414993816 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9703+0+0 (secure 0 0 0) 0x7fe958109c80 con 0x7fe96c1079f0 2024-09-21T00:36:22.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.397+0000 7fe96b400640 1 --2- 172.21.15.27:0/414993816 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9500b67b0 0x7fe9500b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:22.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.399+0000 7fe963e00640 1 -- 172.21.15.27:0/414993816 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe958036030 con 0x7fe96c1079f0 2024-09-21T00:36:22.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.400+0000 7fe96b400640 1 --2- 172.21.15.27:0/414993816 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9500b67b0 0x7fe9500b8cc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009900 tx=0x7fe95c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.806+0000 7fe970e00640 1 -- 172.21.15.27:0/414993816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe94c0055b0 con 0x7fe96c1079f0 2024-09-21T00:36:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.807+0000 7fe963e00640 1 -- 172.21.15.27:0/414993816 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v113) ==== 81+0+881 (secure 0 0 0) 0x7fe9580cfde0 con 0x7fe96c1079f0 2024-09-21T00:36:22.815 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/502883064 2024-09-22T00:11:24.231101+0000 2024-09-21T00:36:22.815 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/831048090 2024-09-22T00:11:24.231101+0000 2024-09-21T00:36:22.815 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3880378905 2024-09-22T00:09:46.276169+0000 2024-09-21T00:36:22.815 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/793991452 2024-09-22T00:09:46.276169+0000 2024-09-21T00:36:22.815 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2077956479 2024-09-22T00:09:46.276169+0000 2024-09-21T00:36:22.815 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/3752372220 2024-09-22T00:09:46.276169+0000 2024-09-21T00:36:22.815 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/3752372220 2024-09-22T00:09:46.276169+0000 2024-09-21T00:36:22.815 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1098432608 2024-09-22T00:10:16.413167+0000 2024-09-21T00:36:22.815 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/2344239267 2024-09-22T00:10:16.413167+0000 2024-09-21T00:36:22.815 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/2344239267 2024-09-22T00:10:16.413167+0000 2024-09-21T00:36:22.816 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/528019907 2024-09-22T00:10:16.413167+0000 2024-09-21T00:36:22.816 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2173601508 2024-09-22T00:10:16.413167+0000 2024-09-21T00:36:22.816 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/827586431 2024-09-22T00:11:24.231101+0000 2024-09-21T00:36:22.816 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/827586431 2024-09-22T00:11:24.231101+0000 2024-09-21T00:36:22.816 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/764776233 2024-09-22T00:11:24.231101+0000 2024-09-21T00:36:22.816 INFO:teuthology.orchestra.run.smithi027.stderr:listed 15 entries 2024-09-21T00:36:22.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.814+0000 7fe970e00640 1 -- 172.21.15.27:0/414993816 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9500b67b0 msgr2=0x7fe9500b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:22.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.814+0000 7fe970e00640 1 --2- 172.21.15.27:0/414993816 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9500b67b0 0x7fe9500b8cc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009900 tx=0x7fe95c031400 comp rx=0 tx=0).stop 2024-09-21T00:36:22.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.814+0000 7fe970e00640 1 -- 172.21.15.27:0/414993816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe96c1079f0 msgr2=0x7fe96c1a1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:22.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.814+0000 7fe970e00640 1 --2- 172.21.15.27:0/414993816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe96c1079f0 0x7fe96c1a1030 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe95800c8c0 tx=0x7fe95800cd80 comp rx=0 tx=0).stop 2024-09-21T00:36:22.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.815+0000 7fe970e00640 1 -- 172.21.15.27:0/414993816 shutdown_connections 2024-09-21T00:36:22.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.815+0000 7fe970e00640 1 --2- 172.21.15.27:0/414993816 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fe9500b67b0 0x7fe9500b8cc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:22.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.815+0000 7fe970e00640 1 --2- 172.21.15.27:0/414993816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c1a6210 0x7fe96c1a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:22.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.815+0000 7fe970e00640 1 --2- 172.21.15.27:0/414993816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe96c1079f0 0x7fe96c1a1030 secure :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe95800c8c0 tx=0x7fe95800cd80 comp rx=0 tx=0).stop 2024-09-21T00:36:22.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.815+0000 7fe970e00640 1 --2- 172.21.15.27:0/414993816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe96c102f00 0x7fe96c1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:22.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.815+0000 7fe970e00640 1 -- 172.21.15.27:0/414993816 >> 172.21.15.27:0/414993816 conn(0x7fe96c0fe2c0 msgr2=0x7fe96c105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:22.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.816+0000 7fe970e00640 1 -- 172.21.15.27:0/414993816 shutdown_connections 2024-09-21T00:36:22.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:22.816+0000 7fe970e00640 1 -- 172.21.15.27:0/414993816 wait complete. 2024-09-21T00:36:23.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:22 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/414993816' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T00:36:23.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:22 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/414993816' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T00:36:23.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:22 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/414993816' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T00:36:23.852 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T00:36:23.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:36:23.852 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T00:36:24.074 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T00:36:24.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:36:24.075 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T00:36:24.118 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:23 smithi027 ceph-mon[30745]: pgmap v850: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T00:36:24.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:36:24.146 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-21T00:36:24.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:23 smithi089 ceph-mon[34481]: pgmap v850: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T00:36:24.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:23 smithi098 ceph-mon[34725]: pgmap v850: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T00:36:24.296 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T00:36:24.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:36:24.297 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:36:24.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:36:24.446 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-09-21T00:36:24.774 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T00:36:24.774 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T00:36:24.774 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:36:24.850 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052612,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497948k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248972k,nr_inodes=812243,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/08e374f91e3f80cd3ea8f340f5c395fc83d29ceb6575cdaefd2bd7164e506973/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/S35MXYA7FVRVYAXHLINUXNIFXE:/var/lib/containers/storage/overlay/l/DDSMN6NCDNIHBEYLIFJQAKJVLP,upperdir=/var/lib/containers/storage/overlay/08e374f91e3f80cd3ea8f340f5c395fc83d29ceb6575cdaefd2bd7164e506973/diff,workdir=/var/lib/containers/storage/overlay/08e374f91e3f80cd3ea8f340f5c395fc83d29ceb6575cdaefd2bd7164e506973/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0135fc2a8b9604a3a6fc96c258e086ef85db81c8efb5732cb389891fc55a73ca/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/S35MXYA7FVRVYAXHLINUXNIFXE:/var/lib/containers/storage/overlay/l/DDSMN6NCDNIHBEYLIFJQAKJVLP,upperdir=/var/lib/containers/storage/overlay/0135fc2a8b9604a3a6fc96c258e086ef85db81c8efb5732cb389891fc55a73ca/diff,workdir=/var/lib/containers/storage/overlay/0135fc2a8b9604a3a6fc96c258e086ef85db81c8efb5732cb389891fc55a73ca/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248972k,nr_inodes=812243,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/327bb41e128901be0655a8ce441b4e7f8360cec8b75cf9e890c7c1126d7c2cbf/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/S35MXYA7FVRVYAXHLINUXNIFXE:/var/lib/containers/storage/overlay/l/DDSMN6NCDNIHBEYLIFJQAKJVLP,upperdir=/var/lib/containers/storage/overlay/327bb41e128901be0655a8ce441b4e7f8360cec8b75cf9e890c7c1126d7c2cbf/diff,workdir=/var/lib/containers/storage/overlay/327bb41e128901be0655a8ce441b4e7f8360cec8b75cf9e890c7c1126d7c2cbf/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1c95e4c60ca6987d8aad22098d0facb8e8838870f491f4d733050b748fe2e622/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/S35MXYA7FVRVYAXHLINUXNIFXE:/var/lib/containers/storage/overlay/l/DDSMN6NCDNIHBEYLIFJQAKJVLP,upperdir=/var/lib/containers/storage/overlay/1c95e4c60ca6987d8aad22098d0facb8e8838870f491f4d733050b748fe2e622/diff,workdir=/var/lib/containers/storage/overlay/1c95e4c60ca6987d8aad22098d0facb8e8838870f491f4d733050b748fe2e622/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/77ce322d2ad31f9eb611d80507ea7f1c267ace32f9178d902bb93e63af7261ed/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/S35MXYA7FVRVYAXHLINUXNIFXE:/var/lib/containers/storage/overlay/l/DDSMN6NCDNIHBEYLIFJQAKJVLP,upperdir=/var/lib/containers/storage/overlay/77ce322d2ad31f9eb611d80507ea7f1c267ace32f9178d902bb93e63af7261ed/diff,workdir=/var/lib/containers/storage/overlay/77ce322d2ad31f9eb611d80507ea7f1c267ace32f9178d902bb93e63af7261ed/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f9d6a1d830c0955bb6d884ad76b133e2510f4c8e616143327b04c4311fba0e20/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/S35MXYA7FVRVYAXHLINUXNIFXE:/var/lib/containers/storage/overlay/l/DDSMN6NCDNIHBEYLIFJQAKJVLP,upperdir=/var/lib/containers/storage/overlay/f9d6a1d830c0955bb6d884ad76b133e2510f4c8e616143327b04c4311fba0e20/diff,workdir=/var/lib/containers/storage/overlay/f9d6a1d830c0955bb6d884ad76b133e2510f4c8e616143327b04c4311fba0e20/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9f75a3971628c70b5b7e428d0b8bb0064c0b2ecc0c3fa9d480e38c37bea26e9f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/S35MXYA7FVRVYAXHLINUXNIFXE:/var/lib/containers/storage/overlay/l/DDSMN6NCDNIHBEYLIFJQAKJVLP,upperdir=/var/lib/containers/storage/overlay/9f75a3971628c70b5b7e428d0b8bb0064c0b2ecc0c3fa9d480e38c37bea26e9f/diff,workdir=/var/lib/containers/storage/overlay/9f75a3971628c70b5b7e428d0b8bb0064c0b2ecc0c3fa9d480e38c37bea26e9f/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b1f640be5d83b9acd502e859a77673f3709b56228a30b5dc4119376f7eb719ae/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/S35MXYA7FVRVYAXHLINUXNIFXE:/var/lib/containers/storage/overlay/l/DDSMN6NCDNIHBEYLIFJQAKJVLP,upperdir=/var/lib/containers/storage/overlay/b1f640be5d83b9acd502e859a77673f3709b56228a30b5dc4119376f7eb719ae/diff,workdir=/var/lib/containers/storage/overlay/b1f640be5d83b9acd502e859a77673f3709b56228a30b5dc4119376f7eb719ae/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4773807c6467d64ec9277792e5e9184ebd979b705d11a9ae8bd8c66db0299733/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/S35MXYA7FVRVYAXHLINUXNIFXE:/var/lib/containers/storage/overlay/l/DDSMN6NCDNIHBEYLIFJQAKJVLP,upperdir=/var/lib/containers/storage/overlay/4773807c6467d64ec9277792e5e9184ebd979b705d11a9ae8bd8c66db0299733/diff,workdir=/var/lib/containers/storage/overlay/4773807c6467d64ec9277792e5e9184ebd979b705d11a9ae8bd8c66db0299733/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/65ca6b378fc3e89ea660b3fbab808e96b574eeb9b8394121d68789e879b536ca/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/S35MXYA7FVRVYAXHLINUXNIFXE:/var/lib/containers/storage/overlay/l/DDSMN6NCDNIHBEYLIFJQAKJVLP,upperdir=/var/lib/containers/storage/overlay/65ca6b378fc3e89ea660b3fbab808e96b574eeb9b8394121d68789e879b536ca/diff,workdir=/var/lib/containers/storage/overlay/65ca6b378fc3e89ea660b3fbab808e96b574eeb9b8394121d68789e879b536ca/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497948k,nr_inodes=819200,mode=755 0 0 2024-09-21T00:36:24.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:36:24.851 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-21T00:36:24.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:36:24.926 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-09-21T00:36:25.150 INFO:teuthology.orchestra.run.smithi027.stderr:Cannot find device "ceph-brx" 2024-09-21T00:36:25.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:36:25.155 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T00:36:25.166 INFO:tasks.cephadm:Teardown begin 2024-09-21T00:36:25.166 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T00:36:25.166 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:36:26.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:25 smithi089 ceph-mon[34481]: pgmap v851: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T00:36:26.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:25 smithi098 ceph-mon[34725]: pgmap v851: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T00:36:26.273 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:36:26.320 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:25 smithi027 ceph-mon[30745]: pgmap v851: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T00:36:26.320 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:36:25 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:36:25.971+0000 7fbf41e00640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-21T00:36:26.321 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:36:25 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:36:25.972+0000 7fbf41e00640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-21T00:36:28.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:27 smithi027 ceph-mon[30745]: pgmap v852: 129 pgs: 22 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 40 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-21T00:36:28.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:27 smithi089 ceph-mon[34481]: pgmap v852: 129 pgs: 22 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 40 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-21T00:36:28.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:27 smithi098 ceph-mon[34725]: pgmap v852: 129 pgs: 22 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 40 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-21T00:36:28.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.471+0000 7f0a27200640 1 -- 172.21.15.27:0/602483633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a28105bb0 msgr2=0x7f0a28105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:28.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.471+0000 7f0a27200640 1 --2- 172.21.15.27:0/602483633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a28105bb0 0x7f0a28105fc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009900 tx=0x7f0a1802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:36:28.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.471+0000 7f0a27200640 1 -- 172.21.15.27:0/602483633 shutdown_connections 2024-09-21T00:36:28.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.471+0000 7f0a27200640 1 --2- 172.21.15.27:0/602483633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a2810be40 0x7f0a28112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:28.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.471+0000 7f0a27200640 1 --2- 172.21.15.27:0/602483633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2810b4f0 0x7f0a2810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:28.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.471+0000 7f0a27200640 1 --2- 172.21.15.27:0/602483633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a28105bb0 0x7f0a28105fc0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:28.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.471+0000 7f0a27200640 1 -- 172.21.15.27:0/602483633 >> 172.21.15.27:0/602483633 conn(0x7f0a28101680 msgr2=0x7f0a28103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:28.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.471+0000 7f0a27200640 1 -- 172.21.15.27:0/602483633 shutdown_connections 2024-09-21T00:36:28.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.471+0000 7f0a27200640 1 -- 172.21.15.27:0/602483633 wait complete. 2024-09-21T00:36:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.472+0000 7f0a27200640 1 Processor -- start 2024-09-21T00:36:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.473+0000 7f0a27200640 1 -- start start 2024-09-21T00:36:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.473+0000 7f0a27200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a28105bb0 0x7f0a2819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.473+0000 7f0a27200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2810b4f0 0x7f0a2819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.473+0000 7f0a27200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a2810be40 0x7f0a281a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.473+0000 7f0a27200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0a28115b00 con 0x7f0a2810b4f0 2024-09-21T00:36:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.473+0000 7f0a27200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0a28115980 con 0x7f0a28105bb0 2024-09-21T00:36:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.473+0000 7f0a27200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0a28115c80 con 0x7f0a2810be40 2024-09-21T00:36:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.473+0000 7f0a25e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a28105bb0 0x7f0a2819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.473+0000 7f0a25e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a28105bb0 0x7f0a2819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:35686/0 (socket says 172.21.15.27:35686) 2024-09-21T00:36:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.473+0000 7f0a25e00640 1 -- 172.21.15.27:0/3977510258 learned_addr learned my addr 172.21.15.27:0/3977510258 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:36:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.473+0000 7f0a25400640 1 --2- 172.21.15.27:0/3977510258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2810b4f0 0x7f0a2819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:28.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.473+0000 7f0a26800640 1 --2- 172.21.15.27:0/3977510258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a2810be40 0x7f0a281a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:28.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.474+0000 7f0a25e00640 1 -- 172.21.15.27:0/3977510258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a2810be40 msgr2=0x7f0a281a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:28.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.474+0000 7f0a25e00640 1 --2- 172.21.15.27:0/3977510258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a2810be40 0x7f0a281a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.474+0000 7f0a25e00640 1 -- 172.21.15.27:0/3977510258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2810b4f0 msgr2=0x7f0a2819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.474+0000 7f0a25e00640 1 --2- 172.21.15.27:0/3977510258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2810b4f0 0x7f0a2819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.474+0000 7f0a25e00640 1 -- 172.21.15.27:0/3977510258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a281a2480 con 0x7f0a28105bb0 2024-09-21T00:36:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.474+0000 7f0a25e00640 1 --2- 172.21.15.27:0/3977510258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a28105bb0 0x7f0a2819c510 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0a18005940 tx=0x7f0a180040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.475+0000 7f0a1ea00640 1 -- 172.21.15.27:0/3977510258 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1803f030 con 0x7f0a28105bb0 2024-09-21T00:36:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.475+0000 7f0a1ea00640 1 -- 172.21.15.27:0/3977510258 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a180044c0 con 0x7f0a28105bb0 2024-09-21T00:36:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.475+0000 7f0a1ea00640 1 -- 172.21.15.27:0/3977510258 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18035070 con 0x7f0a28105bb0 2024-09-21T00:36:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.475+0000 7f0a27200640 1 -- 172.21.15.27:0/3977510258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a281a2710 con 0x7f0a28105bb0 2024-09-21T00:36:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.475+0000 7f0a27200640 1 -- 172.21.15.27:0/3977510258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a281a8ed0 con 0x7f0a28105bb0 2024-09-21T00:36:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.479+0000 7f0a27200640 1 -- 172.21.15.27:0/3977510258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a28105fc0 con 0x7f0a28105bb0 2024-09-21T00:36:28.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.487+0000 7f0a1ea00640 1 -- 172.21.15.27:0/3977510258 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0a18042030 con 0x7f0a28105bb0 2024-09-21T00:36:28.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.491+0000 7f0a1ea00640 1 --2- 172.21.15.27:0/3977510258 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0a0c0b6700 0x7f0a0c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:28.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.492+0000 7f0a1ea00640 1 -- 172.21.15.27:0/3977510258 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9703+0+0 (secure 0 0 0) 0x7f0a181080f0 con 0x7f0a28105bb0 2024-09-21T00:36:28.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.492+0000 7f0a1ea00640 1 -- 172.21.15.27:0/3977510258 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0a18108550 con 0x7f0a28105bb0 2024-09-21T00:36:28.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.493+0000 7f0a25400640 1 --2- 172.21.15.27:0/3977510258 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0a0c0b6700 0x7f0a0c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:28.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.495+0000 7f0a25400640 1 --2- 172.21.15.27:0/3977510258 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0a0c0b6700 0x7f0a0c0b8c10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0a140095f0 tx=0x7f0a14005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:28.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.955+0000 7f0a27200640 1 -- 172.21.15.27:0/3977510258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a2810dc10 con 0x7f0a28105bb0 2024-09-21T00:36:28.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.957+0000 7f0a1ea00640 1 -- 172.21.15.27:0/3977510258 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11902 (secure 0 0 0) 0x7f0a180ce140 con 0x7f0a28105bb0 2024-09-21T00:36:28.962 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:36:28.962 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 47 2024-09-21T00:36:28.963 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":47,"btime":"2024-09-21T00:33:03:420304+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:33:02.469717+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26548":{"gid":26548,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2873147127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2873147127},{"type":"v1","addr":"172.21.15.89:6837","nonce":2873147127}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24751,24727]},"id":1}]} 2024-09-21T00:36:28.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.961+0000 7f0a27200640 1 -- 172.21.15.27:0/3977510258 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0a0c0b6700 msgr2=0x7f0a0c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:28.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.961+0000 7f0a27200640 1 --2- 172.21.15.27:0/3977510258 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0a0c0b6700 0x7f0a0c0b8c10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0a140095f0 tx=0x7f0a14005f90 comp rx=0 tx=0).stop 2024-09-21T00:36:28.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.961+0000 7f0a27200640 1 -- 172.21.15.27:0/3977510258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a28105bb0 msgr2=0x7f0a2819c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:28.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.961+0000 7f0a27200640 1 --2- 172.21.15.27:0/3977510258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a28105bb0 0x7f0a2819c510 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0a18005940 tx=0x7f0a180040c0 comp rx=0 tx=0).stop 2024-09-21T00:36:28.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.962+0000 7f0a27200640 1 -- 172.21.15.27:0/3977510258 shutdown_connections 2024-09-21T00:36:28.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.962+0000 7f0a27200640 1 --2- 172.21.15.27:0/3977510258 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f0a0c0b6700 0x7f0a0c0b8c10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:28.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.962+0000 7f0a27200640 1 --2- 172.21.15.27:0/3977510258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a2810be40 0x7f0a281a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:28.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.962+0000 7f0a27200640 1 --2- 172.21.15.27:0/3977510258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2810b4f0 0x7f0a2819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:28.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.962+0000 7f0a27200640 1 --2- 172.21.15.27:0/3977510258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a28105bb0 0x7f0a2819c510 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:28.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.962+0000 7f0a27200640 1 -- 172.21.15.27:0/3977510258 >> 172.21.15.27:0/3977510258 conn(0x7f0a28101680 msgr2=0x7f0a28103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:28.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.962+0000 7f0a27200640 1 -- 172.21.15.27:0/3977510258 shutdown_connections 2024-09-21T00:36:28.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:28.963+0000 7f0a27200640 1 -- 172.21.15.27:0/3977510258 wait complete. 2024-09-21T00:36:29.820 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T00:36:30.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:29 smithi027 ceph-mon[30745]: pgmap v853: 129 pgs: 22 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 40 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T00:36:30.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:29 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3977510258' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:36:30.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:29 smithi089 ceph-mon[34481]: pgmap v853: 129 pgs: 22 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 40 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T00:36:30.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:29 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3977510258' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:36:30.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:29 smithi098 ceph-mon[34725]: pgmap v853: 129 pgs: 22 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 40 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T00:36:30.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:29 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3977510258' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:36:30.537 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:36:31.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.404+0000 7fd605e00640 1 -- 172.21.15.27:0/1651330901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd60010a650 msgr2=0x7fd60010aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:31.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.404+0000 7fd605e00640 1 --2- 172.21.15.27:0/1651330901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd60010a650 0x7fd60010aa60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009900 tx=0x7fd5f002a3f0 comp rx=0 tx=0).stop 2024-09-21T00:36:31.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.405+0000 7fd605e00640 1 -- 172.21.15.27:0/1651330901 shutdown_connections 2024-09-21T00:36:31.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.405+0000 7fd605e00640 1 --2- 172.21.15.27:0/1651330901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd60010d620 0x7fd600117090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:31.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.405+0000 7fd605e00640 1 --2- 172.21.15.27:0/1651330901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd60010ccd0 0x7fd60010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:31.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.405+0000 7fd605e00640 1 --2- 172.21.15.27:0/1651330901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd60010a650 0x7fd60010aa60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:31.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.405+0000 7fd605e00640 1 -- 172.21.15.27:0/1651330901 >> 172.21.15.27:0/1651330901 conn(0x7fd600070ec0 msgr2=0x7fd6000712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:31.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.405+0000 7fd605e00640 1 -- 172.21.15.27:0/1651330901 shutdown_connections 2024-09-21T00:36:31.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.405+0000 7fd605e00640 1 -- 172.21.15.27:0/1651330901 wait complete. 2024-09-21T00:36:31.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.408+0000 7fd605e00640 1 Processor -- start 2024-09-21T00:36:31.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.408+0000 7fd605e00640 1 -- start start 2024-09-21T00:36:31.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.409+0000 7fd605e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd60010a650 0x7fd6001aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:31.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.410+0000 7fd605e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd60010ccd0 0x7fd6001aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:31.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.410+0000 7fd605e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd60010d620 0x7fd6001a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.411+0000 7fd5fe000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd60010ccd0 0x7fd6001aaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.413+0000 7fd605e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd600118680 con 0x7fd60010a650 2024-09-21T00:36:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.413+0000 7fd605e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd600118500 con 0x7fd60010ccd0 2024-09-21T00:36:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.413+0000 7fd605e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd600118800 con 0x7fd60010d620 2024-09-21T00:36:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.413+0000 7fd5fe000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd60010ccd0 0x7fd6001aaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:35706/0 (socket says 172.21.15.27:35706) 2024-09-21T00:36:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.413+0000 7fd5fe000640 1 -- 172.21.15.27:0/3747604206 learned_addr learned my addr 172.21.15.27:0/3747604206 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:36:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.411+0000 7fd5ff400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd60010d620 0x7fd6001a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.413+0000 7fd5fea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd60010a650 0x7fd6001aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:31.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.415+0000 7fd5fe000640 1 -- 172.21.15.27:0/3747604206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd60010d620 msgr2=0x7fd6001a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:31.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.415+0000 7fd5fe000640 1 --2- 172.21.15.27:0/3747604206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd60010d620 0x7fd6001a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:31.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.415+0000 7fd5fe000640 1 -- 172.21.15.27:0/3747604206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd60010a650 msgr2=0x7fd6001aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:31.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.415+0000 7fd5fe000640 1 --2- 172.21.15.27:0/3747604206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd60010a650 0x7fd6001aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:31.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.415+0000 7fd5fe000640 1 -- 172.21.15.27:0/3747604206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6001a5770 con 0x7fd60010ccd0 2024-09-21T00:36:31.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.415+0000 7fd5ff400640 1 --2- 172.21.15.27:0/3747604206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd60010d620 0x7fd6001a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:36:31.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.415+0000 7fd5fe000640 1 --2- 172.21.15.27:0/3747604206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd60010ccd0 0x7fd6001aaf70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00ea90 tx=0x7fd5ec00ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:31.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.416+0000 7fd604a00640 1 -- 172.21.15.27:0/3747604206 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec01b070 con 0x7fd60010ccd0 2024-09-21T00:36:31.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.416+0000 7fd604a00640 1 -- 172.21.15.27:0/3747604206 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ec00cce0 con 0x7fd60010ccd0 2024-09-21T00:36:31.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.416+0000 7fd604a00640 1 -- 172.21.15.27:0/3747604206 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec00b650 con 0x7fd60010ccd0 2024-09-21T00:36:31.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.416+0000 7fd605e00640 1 -- 172.21.15.27:0/3747604206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6001a5a60 con 0x7fd60010ccd0 2024-09-21T00:36:31.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.416+0000 7fd605e00640 1 -- 172.21.15.27:0/3747604206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6001b1830 con 0x7fd60010ccd0 2024-09-21T00:36:31.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.418+0000 7fd605e00640 1 -- 172.21.15.27:0/3747604206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60010b870 con 0x7fd60010ccd0 2024-09-21T00:36:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.425+0000 7fd604a00640 1 -- 172.21.15.27:0/3747604206 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fd5ec01e070 con 0x7fd60010ccd0 2024-09-21T00:36:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.428+0000 7fd604a00640 1 --2- 172.21.15.27:0/3747604206 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd5e40b6700 0x7fd5e40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:31.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.428+0000 7fd604a00640 1 -- 172.21.15.27:0/3747604206 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9703+0+0 (secure 0 0 0) 0x7fd5ec0e7bd0 con 0x7fd60010ccd0 2024-09-21T00:36:31.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.428+0000 7fd5fea00640 1 --2- 172.21.15.27:0/3747604206 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd5e40b6700 0x7fd5e40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:31.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.429+0000 7fd604a00640 1 -- 172.21.15.27:0/3747604206 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd5ec0b2000 con 0x7fd60010ccd0 2024-09-21T00:36:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.430+0000 7fd5fea00640 1 --2- 172.21.15.27:0/3747604206 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd5e40b6700 0x7fd5e40b8c10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0005ab0 tx=0x7fd5f0005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:31.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.868+0000 7fd605e00640 1 -- 172.21.15.27:0/3747604206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd60010aa60 con 0x7fd60010ccd0 2024-09-21T00:36:31.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.869+0000 7fd604a00640 1 -- 172.21.15.27:0/3747604206 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11902 (secure 0 0 0) 0x7fd5ec0b6f40 con 0x7fd60010ccd0 2024-09-21T00:36:31.872 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:36:31.872 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":47,"btime":"2024-09-21T00:33:03:420304+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1265394249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1265394249},{"type":"v1","addr":"172.21.15.27:6837","nonce":1265394249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3523607333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3523607333},{"type":"v1","addr":"172.21.15.27:6839","nonce":3523607333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24658,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3448953882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3448953882},{"type":"v1","addr":"172.21.15.98:6833","nonce":3448953882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24662,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/191964694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":191964694},{"type":"v1","addr":"172.21.15.89:6833","nonce":191964694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4068274538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":4068274538},{"type":"v1","addr":"172.21.15.27:6835","nonce":4068274538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24692,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/811483028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":811483028},{"type":"v1","addr":"172.21.15.89:6835","nonce":811483028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T00:18:39.809850+0000","modified":"2024-09-21T00:33:02.469717+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24761,"mds_1":24751,"mds_2":24727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/388589971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388589971},{"type":"v1","addr":"172.21.15.98:6835","nonce":388589971}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6837/3858073278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3858073278},{"type":"v1","addr":"172.21.15.98:6837","nonce":3858073278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.98:6839/3074699335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3074699335},{"type":"v1","addr":"172.21.15.98:6839","nonce":3074699335}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24751":{"gid":24751,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6841/378707351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":378707351},{"type":"v1","addr":"172.21.15.27:6841","nonce":378707351}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/2432693964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2432693964},{"type":"v1","addr":"172.21.15.89:6839","nonce":2432693964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26548":{"gid":26548,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2873147127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2873147127},{"type":"v1","addr":"172.21.15.89:6837","nonce":2873147127}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24751,24727]},"id":1}]} 2024-09-21T00:36:31.872 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 47 2024-09-21T00:36:31.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.873+0000 7fd605e00640 1 -- 172.21.15.27:0/3747604206 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd5e40b6700 msgr2=0x7fd5e40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:31.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.873+0000 7fd605e00640 1 --2- 172.21.15.27:0/3747604206 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd5e40b6700 0x7fd5e40b8c10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0005ab0 tx=0x7fd5f0005a20 comp rx=0 tx=0).stop 2024-09-21T00:36:31.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.873+0000 7fd605e00640 1 -- 172.21.15.27:0/3747604206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd60010ccd0 msgr2=0x7fd6001aaf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:31.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.873+0000 7fd605e00640 1 --2- 172.21.15.27:0/3747604206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd60010ccd0 0x7fd6001aaf70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00ea90 tx=0x7fd5ec00ef50 comp rx=0 tx=0).stop 2024-09-21T00:36:31.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.873+0000 7fd605e00640 1 -- 172.21.15.27:0/3747604206 shutdown_connections 2024-09-21T00:36:31.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.873+0000 7fd605e00640 1 --2- 172.21.15.27:0/3747604206 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fd5e40b6700 0x7fd5e40b8c10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:31.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.873+0000 7fd605e00640 1 --2- 172.21.15.27:0/3747604206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd60010d620 0x7fd6001a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:31.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.873+0000 7fd605e00640 1 --2- 172.21.15.27:0/3747604206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd60010ccd0 0x7fd6001aaf70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:31.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.873+0000 7fd605e00640 1 --2- 172.21.15.27:0/3747604206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd60010a650 0x7fd6001aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:31.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.873+0000 7fd605e00640 1 -- 172.21.15.27:0/3747604206 >> 172.21.15.27:0/3747604206 conn(0x7fd600070ec0 msgr2=0x7fd600073e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:31.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.874+0000 7fd605e00640 1 -- 172.21.15.27:0/3747604206 shutdown_connections 2024-09-21T00:36:31.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:31.874+0000 7fd605e00640 1 -- 172.21.15.27:0/3747604206 wait complete. 2024-09-21T00:36:32.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:31 smithi089 ceph-mon[34481]: pgmap v854: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T00:36:32.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:31 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/3747604206' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:36:32.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:31 smithi098 ceph-mon[34725]: pgmap v854: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T00:36:32.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:31 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/3747604206' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:36:32.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:31 smithi027 ceph-mon[30745]: pgmap v854: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T00:36:32.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:31 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/3747604206' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T00:36:33.059 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T00:36:33.745 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:36:34.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:33 smithi027 ceph-mon[30745]: pgmap v855: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T00:36:34.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:33 smithi089 ceph-mon[34481]: pgmap v855: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T00:36:34.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:33 smithi098 ceph-mon[34725]: pgmap v855: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T00:36:34.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.628+0000 7f5bb2400640 1 -- 172.21.15.27:0/429487705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac10b500 msgr2=0x7f5bac10b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:34.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.629+0000 7f5bb2400640 1 --2- 172.21.15.27:0/429487705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac10b500 0x7f5bac10b910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009900 tx=0x7f5b9c02a400 comp rx=0 tx=0).stop 2024-09-21T00:36:34.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.630+0000 7f5bb2400640 1 -- 172.21.15.27:0/429487705 shutdown_connections 2024-09-21T00:36:34.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.630+0000 7f5bb2400640 1 --2- 172.21.15.27:0/429487705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bac10be50 0x7f5bac112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:34.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.631+0000 7f5bb2400640 1 --2- 172.21.15.27:0/429487705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac10b500 0x7f5bac10b910 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:34.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.631+0000 7f5bb2400640 1 --2- 172.21.15.27:0/429487705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bac105bc0 0x7f5bac105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:34.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.631+0000 7f5bb2400640 1 -- 172.21.15.27:0/429487705 >> 172.21.15.27:0/429487705 conn(0x7f5bac101630 msgr2=0x7f5bac103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:34.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.632+0000 7f5bb2400640 1 -- 172.21.15.27:0/429487705 shutdown_connections 2024-09-21T00:36:34.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.632+0000 7f5bb2400640 1 -- 172.21.15.27:0/429487705 wait complete. 2024-09-21T00:36:34.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.637+0000 7f5bb2400640 1 Processor -- start 2024-09-21T00:36:34.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.637+0000 7f5bb2400640 1 -- start start 2024-09-21T00:36:34.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.637+0000 7f5bb2400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bac105bc0 0x7f5bac07b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:34.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.637+0000 7f5bb2400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac10b500 0x7f5bac077920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:34.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.637+0000 7f5bb2400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bac10be50 0x7f5bac078040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:34.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.637+0000 7f5bb2400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac115b60 con 0x7f5bac10be50 2024-09-21T00:36:34.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.638+0000 7f5bb2400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac1159e0 con 0x7f5bac10b500 2024-09-21T00:36:34.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.638+0000 7f5bb2400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac115ce0 con 0x7f5bac105bc0 2024-09-21T00:36:34.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.638+0000 7f5babe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac10b500 0x7f5bac077920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:34.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.638+0000 7f5babe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac10b500 0x7f5bac077920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53876/0 (socket says 172.21.15.27:53876) 2024-09-21T00:36:34.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.638+0000 7f5babe00640 1 -- 172.21.15.27:0/2441379397 learned_addr learned my addr 172.21.15.27:0/2441379397 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:36:34.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.639+0000 7f5babe00640 1 -- 172.21.15.27:0/2441379397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bac105bc0 msgr2=0x7f5bac07b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:34.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.639+0000 7f5babe00640 1 --2- 172.21.15.27:0/2441379397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bac105bc0 0x7f5bac07b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:34.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.639+0000 7f5babe00640 1 -- 172.21.15.27:0/2441379397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bac10be50 msgr2=0x7f5bac078040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:34.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.639+0000 7f5babe00640 1 --2- 172.21.15.27:0/2441379397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bac10be50 0x7f5bac078040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:34.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.639+0000 7f5babe00640 1 -- 172.21.15.27:0/2441379397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bac19c200 con 0x7f5bac10b500 2024-09-21T00:36:34.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.640+0000 7f5babe00640 1 --2- 172.21.15.27:0/2441379397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac10b500 0x7f5bac077920 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c005ab0 tx=0x7f5b9c003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:34.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.640+0000 7f5ba9600640 1 -- 172.21.15.27:0/2441379397 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c03f070 con 0x7f5bac10b500 2024-09-21T00:36:34.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.641+0000 7f5ba9600640 1 -- 172.21.15.27:0/2441379397 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b9c031d60 con 0x7f5bac10b500 2024-09-21T00:36:34.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.641+0000 7f5ba9600640 1 -- 172.21.15.27:0/2441379397 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c035030 con 0x7f5bac10b500 2024-09-21T00:36:34.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.641+0000 7f5bb2400640 1 -- 172.21.15.27:0/2441379397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bac19c490 con 0x7f5bac10b500 2024-09-21T00:36:34.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.642+0000 7f5bb2400640 1 -- 172.21.15.27:0/2441379397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bac19c920 con 0x7f5bac10b500 2024-09-21T00:36:34.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.644+0000 7f5bb2400640 1 -- 172.21.15.27:0/2441379397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bac106e40 con 0x7f5bac10b500 2024-09-21T00:36:34.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.652+0000 7f5ba9600640 1 -- 172.21.15.27:0/2441379397 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5b9c042070 con 0x7f5bac10b500 2024-09-21T00:36:34.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.656+0000 7f5ba9600640 1 --2- 172.21.15.27:0/2441379397 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5b940b6700 0x7f5b940b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:34.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.656+0000 7f5ba9600640 1 -- 172.21.15.27:0/2441379397 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9703+0+0 (secure 0 0 0) 0x7f5b9c1086e0 con 0x7f5bac10b500 2024-09-21T00:36:34.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.656+0000 7f5bb1000640 1 --2- 172.21.15.27:0/2441379397 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5b940b6700 0x7f5b940b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:34.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.657+0000 7f5ba9600640 1 -- 172.21.15.27:0/2441379397 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5b9c108be0 con 0x7f5bac10b500 2024-09-21T00:36:34.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:34.658+0000 7f5bb1000640 1 --2- 172.21.15.27:0/2441379397 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5b940b6700 0x7f5b940b8c10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5bac07b390 tx=0x7f5ba0005ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:35.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:35.039+0000 7f5bb2400640 1 -- 172.21.15.27:0/2441379397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5bac078c20 con 0x7f5bac10b500 2024-09-21T00:36:35.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:35.042+0000 7f5ba9600640 1 -- 172.21.15.27:0/2441379397 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+25629 (secure 0 0 0) 0x7f5b9c0ce790 con 0x7f5bac10b500 2024-09-21T00:36:35.045 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:36:35.045 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":113,"fsid":"9c21de84-77ad-11ef-baf5-efdc52797490","created":"2024-09-21T00:09:20.149938+0000","modified":"2024-09-21T00:36:08.437825+0000","last_up_change":"2024-09-21T00:17:43.978785+0000","last_in_change":"2024-09-21T00:17:21.325177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:13:32.718543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:18:33.992414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:18:36.817691+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:18:46.707770+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63ccc094-62d5-446f-844d-1c60d41fda1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6803","nonce":354370090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6805","nonce":354370090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6809","nonce":354370090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":354370090},{"type":"v1","addr":"172.21.15.27:6807","nonce":354370090}]},"public_addr":"172.21.15.27:6803/354370090","cluster_addr":"172.21.15.27:6805/354370090","heartbeat_back_addr":"172.21.15.27:6809/354370090","heartbeat_front_addr":"172.21.15.27:6807/354370090","state":["exists","up"]},{"osd":1,"uuid":"e59a8ed4-3a37-4477-a60a-4abc33d49b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6801","nonce":922528621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6803","nonce":922528621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6807","nonce":922528621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":922528621},{"type":"v1","addr":"172.21.15.89:6805","nonce":922528621}]},"public_addr":"172.21.15.89:6801/922528621","cluster_addr":"172.21.15.89:6803/922528621","heartbeat_back_addr":"172.21.15.89:6807/922528621","heartbeat_front_addr":"172.21.15.89:6805/922528621","state":["exists","up"]},{"osd":2,"uuid":"6058a875-d5b2-4459-9407-2fd190cb34a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6801","nonce":257636259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6803","nonce":257636259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6807","nonce":257636259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":257636259},{"type":"v1","addr":"172.21.15.98:6805","nonce":257636259}]},"public_addr":"172.21.15.98:6801/257636259","cluster_addr":"172.21.15.98:6803/257636259","heartbeat_back_addr":"172.21.15.98:6807/257636259","heartbeat_front_addr":"172.21.15.98:6805/257636259","state":["exists","up"]},{"osd":3,"uuid":"dfcdaede-e51c-4985-92af-3e296435a36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6811","nonce":821088824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6813","nonce":821088824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6817","nonce":821088824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":821088824},{"type":"v1","addr":"172.21.15.27:6815","nonce":821088824}]},"public_addr":"172.21.15.27:6811/821088824","cluster_addr":"172.21.15.27:6813/821088824","heartbeat_back_addr":"172.21.15.27:6817/821088824","heartbeat_front_addr":"172.21.15.27:6815/821088824","state":["exists","up"]},{"osd":4,"uuid":"7c4af037-8ab6-47c0-b62b-76bd775b3491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6809","nonce":4001038690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6811","nonce":4001038690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6815","nonce":4001038690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4001038690},{"type":"v1","addr":"172.21.15.89:6813","nonce":4001038690}]},"public_addr":"172.21.15.89:6809/4001038690","cluster_addr":"172.21.15.89:6811/4001038690","heartbeat_back_addr":"172.21.15.89:6815/4001038690","heartbeat_front_addr":"172.21.15.89:6813/4001038690","state":["exists","up"]},{"osd":5,"uuid":"e996ed0d-88ad-4aa7-891f-dfade4e09cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6809","nonce":3029903562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6811","nonce":3029903562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6815","nonce":3029903562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3029903562},{"type":"v1","addr":"172.21.15.98:6813","nonce":3029903562}]},"public_addr":"172.21.15.98:6809/3029903562","cluster_addr":"172.21.15.98:6811/3029903562","heartbeat_back_addr":"172.21.15.98:6815/3029903562","heartbeat_front_addr":"172.21.15.98:6813/3029903562","state":["exists","up"]},{"osd":6,"uuid":"cf080497-57c2-42d0-8c8f-1eed0110702d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6819","nonce":1286808857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6821","nonce":1286808857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6825","nonce":1286808857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1286808857},{"type":"v1","addr":"172.21.15.27:6823","nonce":1286808857}]},"public_addr":"172.21.15.27:6819/1286808857","cluster_addr":"172.21.15.27:6821/1286808857","heartbeat_back_addr":"172.21.15.27:6825/1286808857","heartbeat_front_addr":"172.21.15.27:6823/1286808857","state":["exists","up"]},{"osd":7,"uuid":"ac2e6fec-00fa-4f27-bb5c-dfed5a6471f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6817","nonce":2479349546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6819","nonce":2479349546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6823","nonce":2479349546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2479349546},{"type":"v1","addr":"172.21.15.89:6821","nonce":2479349546}]},"public_addr":"172.21.15.89:6817/2479349546","cluster_addr":"172.21.15.89:6819/2479349546","heartbeat_back_addr":"172.21.15.89:6823/2479349546","heartbeat_front_addr":"172.21.15.89:6821/2479349546","state":["exists","up"]},{"osd":8,"uuid":"0b558458-fc2d-4ed1-b72a-e2ea8cf811b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6817","nonce":2830381485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6819","nonce":2830381485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6823","nonce":2830381485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2830381485},{"type":"v1","addr":"172.21.15.98:6821","nonce":2830381485}]},"public_addr":"172.21.15.98:6817/2830381485","cluster_addr":"172.21.15.98:6819/2830381485","heartbeat_back_addr":"172.21.15.98:6823/2830381485","heartbeat_front_addr":"172.21.15.98:6821/2830381485","state":["exists","up"]},{"osd":9,"uuid":"1ad29ad6-f660-4a37-96e7-5e2b7e8abf9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6827","nonce":3033977113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6829","nonce":3033977113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6833","nonce":3033977113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3033977113},{"type":"v1","addr":"172.21.15.27:6831","nonce":3033977113}]},"public_addr":"172.21.15.27:6827/3033977113","cluster_addr":"172.21.15.27:6829/3033977113","heartbeat_back_addr":"172.21.15.27:6833/3033977113","heartbeat_front_addr":"172.21.15.27:6831/3033977113","state":["exists","up"]},{"osd":10,"uuid":"324dd6ce-68fe-4c76-959b-1c9fb0ae92a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6825","nonce":2370332225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6827","nonce":2370332225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6831","nonce":2370332225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2370332225},{"type":"v1","addr":"172.21.15.89:6829","nonce":2370332225}]},"public_addr":"172.21.15.89:6825/2370332225","cluster_addr":"172.21.15.89:6827/2370332225","heartbeat_back_addr":"172.21.15.89:6831/2370332225","heartbeat_front_addr":"172.21.15.89:6829/2370332225","state":["exists","up"]},{"osd":11,"uuid":"5e34686e-350a-491e-b9d7-59174ea90944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6825","nonce":2795999064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6827","nonce":2795999064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6831","nonce":2795999064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2795999064},{"type":"v1","addr":"172.21.15.98:6829","nonce":2795999064}]},"public_addr":"172.21.15.98:6825/2795999064","cluster_addr":"172.21.15.98:6827/2795999064","heartbeat_back_addr":"172.21.15.98:6831/2795999064","heartbeat_front_addr":"172.21.15.98:6829/2795999064","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:12:39.841958+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:06.684417+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:13:31.041378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:01.612108+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:28.592326+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:14:54.303714+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:25.523475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:15:52.523483+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:19.430503+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:16:50.599120+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:17.520452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:17:42.725550+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/502883064":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/831048090":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/3880378905":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/793991452":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/2077956479":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6800/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:6801/3752372220":"2024-09-22T00:09:46.276169+0000","172.21.15.27:0/1098432608":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6800/2344239267":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/528019907":"2024-09-22T00:10:16.413167+0000","172.21.15.27:0/2173601508":"2024-09-22T00:10:16.413167+0000","172.21.15.27:6801/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:6800/827586431":"2024-09-22T00:11:24.231101+0000","172.21.15.27:0/764776233":"2024-09-22T00:11:24.231101+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":12,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":25,"length":1}]},{"pool":4,"snaps":[{"begin":10,"length":1},{"begin":12,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":25,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]},{"pool":4,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:36:35.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:35.045+0000 7f5bb2400640 1 -- 172.21.15.27:0/2441379397 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5b940b6700 msgr2=0x7f5b940b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:35.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:35.045+0000 7f5bb2400640 1 --2- 172.21.15.27:0/2441379397 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5b940b6700 0x7f5b940b8c10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5bac07b390 tx=0x7f5ba0005ed0 comp rx=0 tx=0).stop 2024-09-21T00:36:35.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:35.046+0000 7f5bb2400640 1 -- 172.21.15.27:0/2441379397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac10b500 msgr2=0x7f5bac077920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:35.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:35.046+0000 7f5bb2400640 1 --2- 172.21.15.27:0/2441379397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac10b500 0x7f5bac077920 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c005ab0 tx=0x7f5b9c003cd0 comp rx=0 tx=0).stop 2024-09-21T00:36:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:35.050+0000 7f5bb2400640 1 -- 172.21.15.27:0/2441379397 shutdown_connections 2024-09-21T00:36:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:35.050+0000 7f5bb2400640 1 --2- 172.21.15.27:0/2441379397 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f5b940b6700 0x7f5b940b8c10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:35.050+0000 7f5bb2400640 1 --2- 172.21.15.27:0/2441379397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bac10be50 0x7f5bac078040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:35.050+0000 7f5bb2400640 1 --2- 172.21.15.27:0/2441379397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac10b500 0x7f5bac077920 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:35.050+0000 7f5bb2400640 1 --2- 172.21.15.27:0/2441379397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bac105bc0 0x7f5bac07b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:35.050+0000 7f5bb2400640 1 -- 172.21.15.27:0/2441379397 >> 172.21.15.27:0/2441379397 conn(0x7f5bac101630 msgr2=0x7f5bac103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:35.053+0000 7f5bb2400640 1 -- 172.21.15.27:0/2441379397 shutdown_connections 2024-09-21T00:36:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:35.053+0000 7f5bb2400640 1 -- 172.21.15.27:0/2441379397 wait complete. 2024-09-21T00:36:36.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:35 smithi089 ceph-mon[34481]: pgmap v856: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-21T00:36:36.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:35 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2441379397' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:36:36.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:35 smithi098 ceph-mon[34725]: pgmap v856: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-21T00:36:36.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:35 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2441379397' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:36:36.290 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:35 smithi027 ceph-mon[30745]: pgmap v856: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-21T00:36:36.290 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:35 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2441379397' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T00:36:36.368 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T00:36:37.100 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:36:38.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:37 smithi089 ceph-mon[34481]: pgmap v857: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T00:36:38.245 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:37 smithi098 ceph-mon[34725]: pgmap v857: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T00:36:38.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.986+0000 7f1ff8c00640 1 -- 172.21.15.27:0/2076838607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ff4115360 msgr2=0x7f1ff4117850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.986+0000 7f1ff8c00640 1 --2- 172.21.15.27:0/2076838607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ff4115360 0x7f1ff4117850 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009900 tx=0x7f1fe002a3f0 comp rx=0 tx=0).stop 2024-09-21T00:36:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.986+0000 7f1ff8c00640 1 -- 172.21.15.27:0/2076838607 shutdown_connections 2024-09-21T00:36:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.986+0000 7f1ff8c00640 1 --2- 172.21.15.27:0/2076838607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ff4115360 0x7f1ff4117850 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.986+0000 7f1ff8c00640 1 --2- 172.21.15.27:0/2076838607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ff410c880 0x7f1ff4114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.986+0000 7f1ff8c00640 1 --2- 172.21.15.27:0/2076838607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ff410bf30 0x7f1ff410c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.986+0000 7f1ff8c00640 1 -- 172.21.15.27:0/2076838607 >> 172.21.15.27:0/2076838607 conn(0x7f1ff406ee50 msgr2=0x7f1ff406f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.987+0000 7f1ff8c00640 1 -- 172.21.15.27:0/2076838607 shutdown_connections 2024-09-21T00:36:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.990+0000 7f1ff8c00640 1 -- 172.21.15.27:0/2076838607 wait complete. 2024-09-21T00:36:38.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:37 smithi027 ceph-mon[30745]: pgmap v857: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T00:36:38.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.991+0000 7f1ff8c00640 1 Processor -- start 2024-09-21T00:36:38.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.991+0000 7f1ff8c00640 1 -- start start 2024-09-21T00:36:38.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.992+0000 7f1ff8c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ff410bf30 0x7f1ff41a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:38.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.992+0000 7f1ff8c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ff410c880 0x7f1ff41aa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:38.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.992+0000 7f1ff8c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ff4115360 0x7f1ff41b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:38.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.992+0000 7f1ff8c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff410b6a0 con 0x7f1ff4115360 2024-09-21T00:36:38.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.992+0000 7f1ff8c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff410b520 con 0x7f1ff410bf30 2024-09-21T00:36:38.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.992+0000 7f1ff8c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff410b820 con 0x7f1ff410c880 2024-09-21T00:36:38.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.992+0000 7f1ff3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ff410bf30 0x7f1ff41a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:38.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.992+0000 7f1ff2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ff410c880 0x7f1ff41aa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:38.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.992+0000 7f1ff2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ff410c880 0x7f1ff41aa3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:46014/0 (socket says 172.21.15.27:46014) 2024-09-21T00:36:38.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.992+0000 7f1ff2a00640 1 -- 172.21.15.27:0/2696567424 learned_addr learned my addr 172.21.15.27:0/2696567424 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:36:38.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.993+0000 7f1ff3e00640 1 --2- 172.21.15.27:0/2696567424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ff4115360 0x7f1ff41b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:38.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.994+0000 7f1ff3400640 1 -- 172.21.15.27:0/2696567424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ff410c880 msgr2=0x7f1ff41aa3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:38.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.994+0000 7f1ff3400640 1 --2- 172.21.15.27:0/2696567424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ff410c880 0x7f1ff41aa3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:38.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.994+0000 7f1ff3400640 1 -- 172.21.15.27:0/2696567424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ff4115360 msgr2=0x7f1ff41b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:38.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.994+0000 7f1ff3400640 1 --2- 172.21.15.27:0/2696567424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ff4115360 0x7f1ff41b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:38.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.994+0000 7f1ff3400640 1 -- 172.21.15.27:0/2696567424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff41aabc0 con 0x7f1ff410bf30 2024-09-21T00:36:38.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.995+0000 7f1ff3400640 1 --2- 172.21.15.27:0/2696567424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ff410bf30 0x7f1ff41a9e60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009fd0 tx=0x7f1fec00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:38.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.995+0000 7f1febe00640 1 -- 172.21.15.27:0/2696567424 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec004b20 con 0x7f1ff410bf30 2024-09-21T00:36:38.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.995+0000 7f1febe00640 1 -- 172.21.15.27:0/2696567424 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fec004cc0 con 0x7f1ff410bf30 2024-09-21T00:36:38.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.995+0000 7f1febe00640 1 -- 172.21.15.27:0/2696567424 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec010720 con 0x7f1ff410bf30 2024-09-21T00:36:38.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.996+0000 7f1ff8c00640 1 -- 172.21.15.27:0/2696567424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff41b5480 con 0x7f1ff410bf30 2024-09-21T00:36:38.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.996+0000 7f1ff8c00640 1 -- 172.21.15.27:0/2696567424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff41b5910 con 0x7f1ff410bf30 2024-09-21T00:36:38.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:37.998+0000 7f1ff8c00640 1 -- 172.21.15.27:0/2696567424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff410c340 con 0x7f1ff410bf30 2024-09-21T00:36:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.003+0000 7f1febe00640 1 -- 172.21.15.27:0/2696567424 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1fec0108c0 con 0x7f1ff410bf30 2024-09-21T00:36:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.006+0000 7f1febe00640 1 --2- 172.21.15.27:0/2696567424 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1fd80b6730 0x7f1fd80b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.008+0000 7f1febe00640 1 -- 172.21.15.27:0/2696567424 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9703+0+0 (secure 0 0 0) 0x7f1fec0e8020 con 0x7f1ff410bf30 2024-09-21T00:36:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.008+0000 7f1ff2a00640 1 --2- 172.21.15.27:0/2696567424 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1fd80b6730 0x7f1fd80b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.008+0000 7f1febe00640 1 -- 172.21.15.27:0/2696567424 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1fec0ae0d0 con 0x7f1ff410bf30 2024-09-21T00:36:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.009+0000 7f1ff2a00640 1 --2- 172.21.15.27:0/2696567424 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1fd80b6730 0x7f1fd80b8c40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4005e70 tx=0x7f1fe4005e00 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:38.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.443+0000 7f1ff8c00640 1 -- 172.21.15.27:0/2696567424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1ff41b5ec0 con 0x7f1ff410bf30 2024-09-21T00:36:38.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.458+0000 7f1febe00640 1 -- 172.21.15.27:0/2696567424 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7f1fec0b24b0 con 0x7f1ff410bf30 2024-09-21T00:36:38.461 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T00:36:38.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.465+0000 7f1ff8c00640 1 -- 172.21.15.27:0/2696567424 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1fd80b6730 msgr2=0x7f1fd80b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:38.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.465+0000 7f1ff8c00640 1 --2- 172.21.15.27:0/2696567424 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1fd80b6730 0x7f1fd80b8c40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4005e70 tx=0x7f1fe4005e00 comp rx=0 tx=0).stop 2024-09-21T00:36:38.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.465+0000 7f1ff8c00640 1 -- 172.21.15.27:0/2696567424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ff410bf30 msgr2=0x7f1ff41a9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:38.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.465+0000 7f1ff8c00640 1 --2- 172.21.15.27:0/2696567424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ff410bf30 0x7f1ff41a9e60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009fd0 tx=0x7f1fec00ef90 comp rx=0 tx=0).stop 2024-09-21T00:36:38.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.465+0000 7f1ff8c00640 1 -- 172.21.15.27:0/2696567424 shutdown_connections 2024-09-21T00:36:38.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.465+0000 7f1ff8c00640 1 --2- 172.21.15.27:0/2696567424 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f1fd80b6730 0x7f1fd80b8c40 secure :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4005e70 tx=0x7f1fe4005e00 comp rx=0 tx=0).stop 2024-09-21T00:36:38.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.465+0000 7f1ff8c00640 1 --2- 172.21.15.27:0/2696567424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ff4115360 0x7f1ff41b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:38.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.465+0000 7f1ff8c00640 1 --2- 172.21.15.27:0/2696567424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ff410c880 0x7f1ff41aa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:38.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.465+0000 7f1ff8c00640 1 --2- 172.21.15.27:0/2696567424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ff410bf30 0x7f1ff41a9e60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:38.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.465+0000 7f1ff8c00640 1 -- 172.21.15.27:0/2696567424 >> 172.21.15.27:0/2696567424 conn(0x7f1ff406ee50 msgr2=0x7f1ff4071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:38.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.467+0000 7f1ff8c00640 1 -- 172.21.15.27:0/2696567424 shutdown_connections 2024-09-21T00:36:38.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:38.467+0000 7f1ff8c00640 1 -- 172.21.15.27:0/2696567424 wait complete. 2024-09-21T00:36:38.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:38 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a[30741]: 2024-09-21T00:36:38.445+0000 7f2145200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T00:36:38.944 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:36:38 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-a[70832]: ignoring --setuser ceph since I am not root 2024-09-21T00:36:38.944 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:36:38 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-a[70832]: ignoring --setgroup ceph since I am not root 2024-09-21T00:36:38.944 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:36:38 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-a[70832]: starting mds.a at 2024-09-21T00:36:38.945 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-f[63853]: ignoring --setuser ceph since I am not root 2024-09-21T00:36:38.945 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-f[63853]: ignoring --setgroup ceph since I am not root 2024-09-21T00:36:38.945 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-mds[63857]: --2- 172.21.15.98:0/1641511559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633ed863600 0x5633ee46c580 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x5633ed874ba0 tx=0x5633ed79a480 comp rx=0 tx=0).write_message sending message m=0x5633ed7288c0 seq=3 auth(proto 2 2 bytes epoch 0) 2024-09-21T00:36:38.945 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-mds[63857]: -- 172.21.15.98:0/1641511559 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5633e96c4ea0 con 0x5633ed863600 2024-09-21T00:36:38.945 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-mds[63857]: -- 172.21.15.98:0/1641511559 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5633e96bbc20 con 0x5633ed863600 2024-09-21T00:36:38.945 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-f[63853]: starting mds.f at 2024-09-21T00:36:38.945 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-b[63110]: ignoring --setuser ceph since I am not root 2024-09-21T00:36:38.945 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-b[63110]: ignoring --setgroup ceph since I am not root 2024-09-21T00:36:38.946 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-mds[63114]: -- 172.21.15.89:0/506606275 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564701576ea0 con 0x564705715600 2024-09-21T00:36:38.946 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-mds[63114]: -- 172.21.15.89:0/506606275 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56470156d860 con 0x564705715600 2024-09-21T00:36:38.946 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-b[63110]: starting mds.b at 2024-09-21T00:36:38.946 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-c[64121]: ignoring --setuser ceph since I am not root 2024-09-21T00:36:38.946 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-c[64121]: ignoring --setgroup ceph since I am not root 2024-09-21T00:36:38.946 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-c[64121]: starting mds.c at 2024-09-21T00:36:38.946 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h[62840]: ignoring --setuser ceph since I am not root 2024-09-21T00:36:38.946 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h[62840]: ignoring --setgroup ceph since I am not root 2024-09-21T00:36:38.947 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-mds[62844]: -- 172.21.15.89:0/1114670180 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5591de272ea0 con 0x5591e2411a80 2024-09-21T00:36:38.947 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-mds[62844]: -- 172.21.15.89:0/1114670180 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5591de269860 con 0x5591e2411a80 2024-09-21T00:36:38.947 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h[62840]: starting mds.h at 2024-09-21T00:36:38.947 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-i[64389]: ignoring --setuser ceph since I am not root 2024-09-21T00:36:38.947 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-i[64389]: ignoring --setgroup ceph since I am not root 2024-09-21T00:36:38.947 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-i[64389]: starting mds.i at 2024-09-21T00:36:38.947 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/2696567424' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T00:36:39.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/2696567424' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T00:36:39.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T00:36:39.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-mon[34481]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T00:36:39.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-mon[34481]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T00:36:39.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T00:36:39.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-mon[34481]: osdmap e114: 12 total, 12 up, 12 in 2024-09-21T00:36:39.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:38 smithi089 ceph-mon[34481]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-21T00:36:39.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T00:36:39.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-mon[34725]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T00:36:39.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-mon[34725]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T00:36:39.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T00:36:39.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-mon[34725]: osdmap e114: 12 total, 12 up, 12 in 2024-09-21T00:36:39.247 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:38 smithi098 ceph-mon[34725]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-21T00:36:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:38 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/2696567424' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T00:36:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:38 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T00:36:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:38 smithi027 ceph-mon[30745]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T00:36:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:38 smithi027 ceph-mon[30745]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T00:36:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:38 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T00:36:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:38 smithi027 ceph-mon[30745]: osdmap e114: 12 total, 12 up, 12 in 2024-09-21T00:36:39.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:38 smithi027 ceph-mon[30745]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-21T00:36:39.878 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T00:36:40.589 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:36:40.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:40 smithi089 ceph-mon[34481]: pgmap v859: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T00:36:40.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:40 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.89:6836/500964380,v1:172.21.15.89:6837/500964380] up:boot 2024-09-21T00:36:40.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:40 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.89:6838/1915115801,v1:172.21.15.89:6839/1915115801] up:boot 2024-09-21T00:36:40.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:40 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.98:6836/3434102438,v1:172.21.15.98:6837/3434102438] up:boot 2024-09-21T00:36:40.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:40 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.98:6834/3160469976,v1:172.21.15.98:6835/3160469976] up:boot 2024-09-21T00:36:40.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:40 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.98:6838/2137071245,v1:172.21.15.98:6839/2137071245] up:boot 2024-09-21T00:36:40.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:40 smithi089 ceph-mon[34481]: mds.? [v2:172.21.15.27:6840/1247455159,v1:172.21.15.27:6841/1247455159] up:boot 2024-09-21T00:36:40.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:40 smithi089 ceph-mon[34481]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-21T00:36:40.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:40 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T00:36:40.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:40 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T00:36:40.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:40 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T00:36:40.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:40 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T00:36:40.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:40 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T00:36:40.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:40 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T00:36:40.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:40 smithi098 ceph-mon[34725]: pgmap v859: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T00:36:40.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:40 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.89:6836/500964380,v1:172.21.15.89:6837/500964380] up:boot 2024-09-21T00:36:40.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:40 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.89:6838/1915115801,v1:172.21.15.89:6839/1915115801] up:boot 2024-09-21T00:36:40.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:40 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.98:6836/3434102438,v1:172.21.15.98:6837/3434102438] up:boot 2024-09-21T00:36:40.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:40 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.98:6834/3160469976,v1:172.21.15.98:6835/3160469976] up:boot 2024-09-21T00:36:40.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:40 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.98:6838/2137071245,v1:172.21.15.98:6839/2137071245] up:boot 2024-09-21T00:36:40.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:40 smithi098 ceph-mon[34725]: mds.? [v2:172.21.15.27:6840/1247455159,v1:172.21.15.27:6841/1247455159] up:boot 2024-09-21T00:36:40.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:40 smithi098 ceph-mon[34725]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-21T00:36:40.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:40 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T00:36:40.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:40 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T00:36:40.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:40 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T00:36:40.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:40 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T00:36:40.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:40 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T00:36:40.747 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:40 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T00:36:40.767 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:40 smithi027 ceph-mon[30745]: pgmap v859: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T00:36:40.768 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:40 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.89:6836/500964380,v1:172.21.15.89:6837/500964380] up:boot 2024-09-21T00:36:40.768 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:40 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.89:6838/1915115801,v1:172.21.15.89:6839/1915115801] up:boot 2024-09-21T00:36:40.768 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:40 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.98:6836/3434102438,v1:172.21.15.98:6837/3434102438] up:boot 2024-09-21T00:36:40.768 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:40 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.98:6834/3160469976,v1:172.21.15.98:6835/3160469976] up:boot 2024-09-21T00:36:40.768 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:40 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.98:6838/2137071245,v1:172.21.15.98:6839/2137071245] up:boot 2024-09-21T00:36:40.768 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:40 smithi027 ceph-mon[30745]: mds.? [v2:172.21.15.27:6840/1247455159,v1:172.21.15.27:6841/1247455159] up:boot 2024-09-21T00:36:40.768 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:40 smithi027 ceph-mon[30745]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-21T00:36:40.768 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:40 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T00:36:40.768 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:40 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T00:36:40.768 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:40 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T00:36:40.768 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:40 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T00:36:40.768 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:40 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T00:36:40.768 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:40 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T00:36:41.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.453+0000 7f81aa200640 1 -- 172.21.15.27:0/1674812430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81a4110210 msgr2=0x7f81a407b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:41.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.453+0000 7f81aa200640 1 --2- 172.21.15.27:0/1674812430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81a4110210 0x7f81a407b690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8194009860 tx=0x7f819402a360 comp rx=0 tx=0).stop 2024-09-21T00:36:41.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.454+0000 7f81aa200640 1 -- 172.21.15.27:0/1674812430 shutdown_connections 2024-09-21T00:36:41.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.454+0000 7f81aa200640 1 --2- 172.21.15.27:0/1674812430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81a4110210 0x7f81a407b690 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:41.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.454+0000 7f81aa200640 1 --2- 172.21.15.27:0/1674812430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81a410f8c0 0x7f81a410fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:41.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.454+0000 7f81aa200640 1 --2- 172.21.15.27:0/1674812430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81a410b8e0 0x7f81a410bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:41.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.454+0000 7f81aa200640 1 -- 172.21.15.27:0/1674812430 >> 172.21.15.27:0/1674812430 conn(0x7f81a406e9f0 msgr2=0x7f81a406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:41.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.454+0000 7f81aa200640 1 -- 172.21.15.27:0/1674812430 shutdown_connections 2024-09-21T00:36:41.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.454+0000 7f81aa200640 1 -- 172.21.15.27:0/1674812430 wait complete. 2024-09-21T00:36:41.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.455+0000 7f81aa200640 1 Processor -- start 2024-09-21T00:36:41.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.456+0000 7f81aa200640 1 -- start start 2024-09-21T00:36:41.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.457+0000 7f81aa200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81a410b8e0 0x7f81a40806b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:41.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.457+0000 7f81aa200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81a410f8c0 0x7f81a4080bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:41.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.458+0000 7f81aa200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81a4110210 0x7f81a4138a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:41.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.458+0000 7f81a3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81a410f8c0 0x7f81a4080bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:41.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.458+0000 7f81aa200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f81a407e9c0 con 0x7f81a410b8e0 2024-09-21T00:36:41.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.458+0000 7f81aa200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f81a407e840 con 0x7f81a410f8c0 2024-09-21T00:36:41.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.458+0000 7f81aa200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f81a407eb40 con 0x7f81a4110210 2024-09-21T00:36:41.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.458+0000 7f81a8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81a410b8e0 0x7f81a40806b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:41.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.458+0000 7f81a8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81a410b8e0 0x7f81a40806b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57414/0 (socket says 172.21.15.27:57414) 2024-09-21T00:36:41.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.458+0000 7f81a8e00640 1 -- 172.21.15.27:0/494658508 learned_addr learned my addr 172.21.15.27:0/494658508 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:36:41.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.458+0000 7f81a3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81a410f8c0 0x7f81a4080bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53924/0 (socket says 172.21.15.27:53924) 2024-09-21T00:36:41.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.458+0000 7f81a9800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81a4110210 0x7f81a4138a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:41.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.459+0000 7f81a8e00640 1 -- 172.21.15.27:0/494658508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81a4110210 msgr2=0x7f81a4138a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:41.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.459+0000 7f81a8e00640 1 --2- 172.21.15.27:0/494658508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81a4110210 0x7f81a4138a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:41.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.459+0000 7f81a8e00640 1 -- 172.21.15.27:0/494658508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81a410f8c0 msgr2=0x7f81a4080bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:41.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.459+0000 7f81a8e00640 1 --2- 172.21.15.27:0/494658508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81a410f8c0 0x7f81a4080bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:41.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.459+0000 7f81a8e00640 1 -- 172.21.15.27:0/494658508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a4081470 con 0x7f81a410b8e0 2024-09-21T00:36:41.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.460+0000 7f81a8e00640 1 --2- 172.21.15.27:0/494658508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81a410b8e0 0x7f81a40806b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f819c00a9d0 tx=0x7f819c00e4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:41.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.461+0000 7f81a1600640 1 -- 172.21.15.27:0/494658508 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c00ebb0 con 0x7f81a410b8e0 2024-09-21T00:36:41.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.461+0000 7f81aa200640 1 -- 172.21.15.27:0/494658508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a4138fa0 con 0x7f81a410b8e0 2024-09-21T00:36:41.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.461+0000 7f81aa200640 1 -- 172.21.15.27:0/494658508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a4139530 con 0x7f81a410b8e0 2024-09-21T00:36:41.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.464+0000 7f81aa200640 1 -- 172.21.15.27:0/494658508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a410bcf0 con 0x7f81a410b8e0 2024-09-21T00:36:41.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.466+0000 7f81a1600640 1 -- 172.21.15.27:0/494658508 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f819c007410 con 0x7f81a410b8e0 2024-09-21T00:36:41.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.466+0000 7f81a1600640 1 -- 172.21.15.27:0/494658508 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c0076c0 con 0x7f81a410b8e0 2024-09-21T00:36:41.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.471+0000 7f81a1600640 1 -- 172.21.15.27:0/494658508 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f819c01b030 con 0x7f81a410b8e0 2024-09-21T00:36:41.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.473+0000 7f81a1600640 1 --2- 172.21.15.27:0/494658508 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f81880b6680 0x7f81880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:41.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.474+0000 7f81a1600640 1 -- 172.21.15.27:0/494658508 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9873+0+0 (secure 0 0 0) 0x7f819c0e7380 con 0x7f81a410b8e0 2024-09-21T00:36:41.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.475+0000 7f81a3e00640 1 --2- 172.21.15.27:0/494658508 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f81880b6680 0x7f81880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:41.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.478+0000 7f81a3e00640 1 --2- 172.21.15.27:0/494658508 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f81880b6680 0x7f81880b8b90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f81a410a620 tx=0x7f8190009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:41.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.483+0000 7f81a1600640 1 -- 172.21.15.27:0/494658508 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f819c011070 con 0x7f81a410b8e0 2024-09-21T00:36:41.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:41.944+0000 7f81aa200640 1 -- 172.21.15.27:0/494658508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f81a402e640 con 0x7f81a410b8e0 2024-09-21T00:36:42.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:42.492+0000 7f81a1600640 1 -- 172.21.15.27:0/494658508 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) ==== 104+0+0 (secure 0 0 0) 0x7f819c0ad410 con 0x7f81a410b8e0 2024-09-21T00:36:42.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:42.499+0000 7f81aa200640 1 -- 172.21.15.27:0/494658508 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f81880b6680 msgr2=0x7f81880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:42.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:42.499+0000 7f81aa200640 1 --2- 172.21.15.27:0/494658508 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f81880b6680 0x7f81880b8b90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f81a410a620 tx=0x7f8190009110 comp rx=0 tx=0).stop 2024-09-21T00:36:42.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:42.499+0000 7f81aa200640 1 -- 172.21.15.27:0/494658508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81a410b8e0 msgr2=0x7f81a40806b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:42.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:42.499+0000 7f81aa200640 1 --2- 172.21.15.27:0/494658508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81a410b8e0 0x7f81a40806b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f819c00a9d0 tx=0x7f819c00e4a0 comp rx=0 tx=0).stop 2024-09-21T00:36:42.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:42.499+0000 7f81aa200640 1 -- 172.21.15.27:0/494658508 shutdown_connections 2024-09-21T00:36:42.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:42.499+0000 7f81aa200640 1 --2- 172.21.15.27:0/494658508 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f81880b6680 0x7f81880b8b90 secure :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f81a410a620 tx=0x7f8190009110 comp rx=0 tx=0).stop 2024-09-21T00:36:42.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:42.499+0000 7f81aa200640 1 --2- 172.21.15.27:0/494658508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81a4110210 0x7f81a4138a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:42.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:42.499+0000 7f81aa200640 1 --2- 172.21.15.27:0/494658508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81a410f8c0 0x7f81a4080bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:42.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:42.499+0000 7f81aa200640 1 --2- 172.21.15.27:0/494658508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81a410b8e0 0x7f81a40806b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:42.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:42.500+0000 7f81aa200640 1 -- 172.21.15.27:0/494658508 >> 172.21.15.27:0/494658508 conn(0x7f81a406e9f0 msgr2=0x7f81a410da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:42.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:42.500+0000 7f81aa200640 1 -- 172.21.15.27:0/494658508 shutdown_connections 2024-09-21T00:36:42.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:42.500+0000 7f81aa200640 1 -- 172.21.15.27:0/494658508 wait complete. 2024-09-21T00:36:42.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:42 smithi027 ceph-mon[30745]: pgmap v860: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-21T00:36:42.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:42 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/494658508' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T00:36:42.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:42 smithi089 ceph-mon[34481]: pgmap v860: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-21T00:36:42.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:42 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/494658508' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T00:36:42.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:42 smithi098 ceph-mon[34725]: pgmap v860: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-21T00:36:42.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:42 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/494658508' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T00:36:43.659 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T00:36:43.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:43 smithi027 ceph-mon[30745]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T00:36:43.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:43 smithi027 ceph-mon[30745]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T00:36:43.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:43 smithi027 ceph-mon[30745]: Cluster is now healthy 2024-09-21T00:36:43.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:43 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/494658508' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T00:36:43.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:43 smithi027 ceph-mon[30745]: fsmap 12 up:standby 2024-09-21T00:36:43.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:43 smithi089 ceph-mon[34481]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T00:36:43.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:43 smithi089 ceph-mon[34481]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T00:36:43.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:43 smithi089 ceph-mon[34481]: Cluster is now healthy 2024-09-21T00:36:43.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:43 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/494658508' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T00:36:43.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:43 smithi089 ceph-mon[34481]: fsmap 12 up:standby 2024-09-21T00:36:43.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:43 smithi098 ceph-mon[34725]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T00:36:43.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:43 smithi098 ceph-mon[34725]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T00:36:43.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:43 smithi098 ceph-mon[34725]: Cluster is now healthy 2024-09-21T00:36:43.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:43 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/494658508' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T00:36:43.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:43 smithi098 ceph-mon[34725]: fsmap 12 up:standby 2024-09-21T00:36:44.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:44 smithi027 ceph-mon[30745]: pgmap v861: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T00:36:44.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:44 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:36:44.914 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:36:44.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:44 smithi089 ceph-mon[34481]: pgmap v861: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T00:36:44.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:44 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:36:44.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:44 smithi098 ceph-mon[34725]: pgmap v861: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T00:36:44.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:44 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T00:36:45.779 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:45 smithi027 ceph-mon[30745]: pgmap v862: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T00:36:45.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:45 smithi089 ceph-mon[34481]: pgmap v862: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T00:36:45.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:45 smithi098 ceph-mon[34725]: pgmap v862: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T00:36:46.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.043+0000 7f5285800640 1 -- 172.21.15.27:0/3167511140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280109c60 msgr2=0x7f5280110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:46.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.043+0000 7f5285800640 1 --2- 172.21.15.27:0/3167511140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280109c60 0x7f5280110490 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5278009be0 tx=0x7f527802a710 comp rx=0 tx=0).stop 2024-09-21T00:36:46.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.046+0000 7f5285800640 1 -- 172.21.15.27:0/3167511140 shutdown_connections 2024-09-21T00:36:46.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.046+0000 7f5285800640 1 --2- 172.21.15.27:0/3167511140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280109c60 0x7f5280110490 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:46.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.046+0000 7f5285800640 1 --2- 172.21.15.27:0/3167511140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5280109310 0x7f5280109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:46.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.046+0000 7f5285800640 1 --2- 172.21.15.27:0/3167511140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52801039d0 0x7f5280103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:46.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.046+0000 7f5285800640 1 -- 172.21.15.27:0/3167511140 >> 172.21.15.27:0/3167511140 conn(0x7f52800ff460 msgr2=0x7f52801018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:46.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.047+0000 7f5285800640 1 -- 172.21.15.27:0/3167511140 shutdown_connections 2024-09-21T00:36:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.047+0000 7f5285800640 1 -- 172.21.15.27:0/3167511140 wait complete. 2024-09-21T00:36:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.048+0000 7f5285800640 1 Processor -- start 2024-09-21T00:36:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.049+0000 7f5285800640 1 -- start start 2024-09-21T00:36:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.050+0000 7f5285800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52801039d0 0x7f528019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.050+0000 7f5285800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280109310 0x7f528019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.050+0000 7f5285800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280109c60 0x7f528019faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.050+0000 7f527e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280109310 0x7f528019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.050+0000 7f527e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280109310 0x7f528019a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41410/0 (socket says 172.21.15.27:41410) 2024-09-21T00:36:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.051+0000 7f5285800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5280113920 con 0x7f5280109c60 2024-09-21T00:36:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.051+0000 7f5285800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f52801137a0 con 0x7f52801039d0 2024-09-21T00:36:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.051+0000 7f527f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280109c60 0x7f528019faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.051+0000 7f5285800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5280113aa0 con 0x7f5280109310 2024-09-21T00:36:46.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.051+0000 7f527e000640 1 -- 172.21.15.27:0/4148442485 learned_addr learned my addr 172.21.15.27:0/4148442485 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:36:46.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.052+0000 7f527ea00640 1 --2- 172.21.15.27:0/4148442485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52801039d0 0x7f528019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:46.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.052+0000 7f527f400640 1 -- 172.21.15.27:0/4148442485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280109310 msgr2=0x7f528019a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:46.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.052+0000 7f527f400640 1 --2- 172.21.15.27:0/4148442485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280109310 0x7f528019a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:46.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.052+0000 7f527f400640 1 -- 172.21.15.27:0/4148442485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52801039d0 msgr2=0x7f528019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:46.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.052+0000 7f527f400640 1 --2- 172.21.15.27:0/4148442485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52801039d0 0x7f528019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:46.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.052+0000 7f527f400640 1 -- 172.21.15.27:0/4148442485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52801a0220 con 0x7f5280109c60 2024-09-21T00:36:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.052+0000 7f527e000640 1 --2- 172.21.15.27:0/4148442485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280109310 0x7f528019a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:36:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.058+0000 7f527f400640 1 --2- 172.21.15.27:0/4148442485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280109c60 0x7f528019faf0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5278034040 tx=0x7f5278030e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:46.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.061+0000 7f5277400640 1 -- 172.21.15.27:0/4148442485 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278036040 con 0x7f5280109c60 2024-09-21T00:36:46.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.061+0000 7f5277400640 1 -- 172.21.15.27:0/4148442485 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527803cb20 con 0x7f5280109c60 2024-09-21T00:36:46.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.061+0000 7f5277400640 1 -- 172.21.15.27:0/4148442485 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278044a80 con 0x7f5280109c60 2024-09-21T00:36:46.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.062+0000 7f5285800640 1 -- 172.21.15.27:0/4148442485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52801a04b0 con 0x7f5280109c60 2024-09-21T00:36:46.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.062+0000 7f5285800640 1 -- 172.21.15.27:0/4148442485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52801a0910 con 0x7f5280109c60 2024-09-21T00:36:46.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.063+0000 7f5285800640 1 -- 172.21.15.27:0/4148442485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5280103de0 con 0x7f5280109c60 2024-09-21T00:36:46.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.069+0000 7f5277400640 1 -- 172.21.15.27:0/4148442485 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5278044c20 con 0x7f5280109c60 2024-09-21T00:36:46.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.072+0000 7f5277400640 1 --2- 172.21.15.27:0/4148442485 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f52640b6780 0x7f52640b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:46.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.073+0000 7f5277400640 1 -- 172.21.15.27:0/4148442485 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9873+0+0 (secure 0 0 0) 0x7f527810cc10 con 0x7f5280109c60 2024-09-21T00:36:46.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.073+0000 7f527ea00640 1 --2- 172.21.15.27:0/4148442485 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f52640b6780 0x7f52640b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.075+0000 7f527ea00640 1 --2- 172.21.15.27:0/4148442485 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f52640b6780 0x7f52640b8c90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5270005fd0 tx=0x7f527000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:46.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.078+0000 7f5277400640 1 -- 172.21.15.27:0/4148442485 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f52780d2c30 con 0x7f5280109c60 2024-09-21T00:36:46.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:46.471+0000 7f5285800640 1 -- 172.21.15.27:0/4148442485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f528002e640 con 0x7f5280109c60 2024-09-21T00:36:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:46 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:36:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:46 smithi027 ceph-mon[30745]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:36:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:46 smithi027 ceph-mon[30745]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:36:46.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:46 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/4148442485' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T00:36:46.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:46 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:36:46.978 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:46 smithi089 ceph-mon[34481]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:36:46.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:46 smithi089 ceph-mon[34481]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:36:46.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:46 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/4148442485' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T00:36:46.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:46 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T00:36:46.995 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:46 smithi098 ceph-mon[34725]: from='mgr.14226 172.21.15.27:0/2893263106' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T00:36:46.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:46 smithi098 ceph-mon[34725]: from='mgr.14226 ' entity='mgr.x' 2024-09-21T00:36:46.996 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:46 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/4148442485' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T00:36:47.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:47.002+0000 7f5277400640 1 -- 172.21.15.27:0/4148442485 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v115) ==== 181+0+0 (secure 0 0 0) 0x7f52780d7010 con 0x7f5280109c60 2024-09-21T00:36:47.003 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-09-21T00:36:47.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:47.005+0000 7f5285800640 1 -- 172.21.15.27:0/4148442485 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f52640b6780 msgr2=0x7f52640b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:47.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:47.005+0000 7f5285800640 1 --2- 172.21.15.27:0/4148442485 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f52640b6780 0x7f52640b8c90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5270005fd0 tx=0x7f527000b410 comp rx=0 tx=0).stop 2024-09-21T00:36:47.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:47.005+0000 7f5285800640 1 -- 172.21.15.27:0/4148442485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280109c60 msgr2=0x7f528019faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:47.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:47.005+0000 7f5285800640 1 --2- 172.21.15.27:0/4148442485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280109c60 0x7f528019faf0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5278034040 tx=0x7f5278030e30 comp rx=0 tx=0).stop 2024-09-21T00:36:47.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:47.006+0000 7f5285800640 1 -- 172.21.15.27:0/4148442485 shutdown_connections 2024-09-21T00:36:47.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:47.006+0000 7f5285800640 1 --2- 172.21.15.27:0/4148442485 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f52640b6780 0x7f52640b8c90 secure :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5270005fd0 tx=0x7f527000b410 comp rx=0 tx=0).stop 2024-09-21T00:36:47.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:47.006+0000 7f5285800640 1 --2- 172.21.15.27:0/4148442485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280109c60 0x7f528019faf0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:47.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:47.006+0000 7f5285800640 1 --2- 172.21.15.27:0/4148442485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280109310 0x7f528019a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:47.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:47.006+0000 7f5285800640 1 --2- 172.21.15.27:0/4148442485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52801039d0 0x7f528019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:47.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:47.006+0000 7f5285800640 1 -- 172.21.15.27:0/4148442485 >> 172.21.15.27:0/4148442485 conn(0x7f52800ff460 msgr2=0x7f5280101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:47.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:47.006+0000 7f5285800640 1 -- 172.21.15.27:0/4148442485 shutdown_connections 2024-09-21T00:36:47.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:47.008+0000 7f5285800640 1 -- 172.21.15.27:0/4148442485 wait complete. 2024-09-21T00:36:47.849 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T00:36:48.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:48 smithi027 ceph-mon[30745]: pgmap v863: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T00:36:48.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:48 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/4148442485' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T00:36:48.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:48 smithi027 ceph-mon[30745]: osdmap e115: 12 total, 12 up, 12 in 2024-09-21T00:36:48.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:48 smithi089 ceph-mon[34481]: pgmap v863: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T00:36:48.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:48 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/4148442485' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T00:36:48.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:48 smithi089 ceph-mon[34481]: osdmap e115: 12 total, 12 up, 12 in 2024-09-21T00:36:48.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:48 smithi098 ceph-mon[34725]: pgmap v863: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T00:36:48.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:48 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/4148442485' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T00:36:48.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:48 smithi098 ceph-mon[34725]: osdmap e115: 12 total, 12 up, 12 in 2024-09-21T00:36:48.624 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:36:50.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:50 smithi027 ceph-mon[30745]: pgmap v865: 97 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 9 active+clean; 683 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:36:50.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.467+0000 7fbcdb600640 1 -- 172.21.15.27:0/2273580327 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcd410a590 msgr2=0x7fbcd410a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:50.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.467+0000 7fbcdb600640 1 --2- 172.21.15.27:0/2273580327 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcd410a590 0x7fbcd410a9a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009900 tx=0x7fbcc802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:36:50.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.469+0000 7fbcdb600640 1 -- 172.21.15.27:0/2273580327 shutdown_connections 2024-09-21T00:36:50.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.469+0000 7fbcdb600640 1 --2- 172.21.15.27:0/2273580327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcd4116640 0x7fbcd4118b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:50.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.469+0000 7fbcdb600640 1 --2- 172.21.15.27:0/2273580327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd410aee0 0x7fbcd4115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:50.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.469+0000 7fbcdb600640 1 --2- 172.21.15.27:0/2273580327 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcd410a590 0x7fbcd410a9a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:50.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.469+0000 7fbcdb600640 1 -- 172.21.15.27:0/2273580327 >> 172.21.15.27:0/2273580327 conn(0x7fbcd4070ec0 msgr2=0x7fbcd40712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:50.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.469+0000 7fbcdb600640 1 -- 172.21.15.27:0/2273580327 shutdown_connections 2024-09-21T00:36:50.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.469+0000 7fbcdb600640 1 -- 172.21.15.27:0/2273580327 wait complete. 2024-09-21T00:36:50.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.471+0000 7fbcdb600640 1 Processor -- start 2024-09-21T00:36:50.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.471+0000 7fbcdb600640 1 -- start start 2024-09-21T00:36:50.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.471+0000 7fbcdb600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd410a590 0x7fbcd41a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:50.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.472+0000 7fbcdb600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcd410aee0 0x7fbcd41a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:50.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.472+0000 7fbcdb600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcd4116640 0x7fbcd41a8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:50.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.472+0000 7fbcdb600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd411c340 con 0x7fbcd410a590 2024-09-21T00:36:50.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.473+0000 7fbcd3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcd410aee0 0x7fbcd41a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:50.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.473+0000 7fbcdb600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd411c1c0 con 0x7fbcd4116640 2024-09-21T00:36:50.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.473+0000 7fbcdb600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd411c4c0 con 0x7fbcd410aee0 2024-09-21T00:36:50.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.473+0000 7fbcd9600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcd4116640 0x7fbcd41a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:50.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.473+0000 7fbcd9600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcd4116640 0x7fbcd41a8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:42080/0 (socket says 172.21.15.27:42080) 2024-09-21T00:36:50.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.473+0000 7fbcd9600640 1 -- 172.21.15.27:0/445528692 learned_addr learned my addr 172.21.15.27:0/445528692 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:36:50.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.473+0000 7fbcd9600640 1 -- 172.21.15.27:0/445528692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcd410aee0 msgr2=0x7fbcd41a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:50.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.473+0000 7fbcd9600640 1 --2- 172.21.15.27:0/445528692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcd410aee0 0x7fbcd41a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:50.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.473+0000 7fbcd9600640 1 -- 172.21.15.27:0/445528692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd410a590 msgr2=0x7fbcd41a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:50.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.474+0000 7fbcd9600640 1 --2- 172.21.15.27:0/445528692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd410a590 0x7fbcd41a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:50.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.474+0000 7fbcd9600640 1 -- 172.21.15.27:0/445528692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd41a25b0 con 0x7fbcd4116640 2024-09-21T00:36:50.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.474+0000 7fbcd3e00640 1 --2- 172.21.15.27:0/445528692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcd410aee0 0x7fbcd41a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:36:50.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.474+0000 7fbcd9600640 1 --2- 172.21.15.27:0/445528692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcd4116640 0x7fbcd41a8870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00cad0 tx=0x7fbccc00ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:50.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.475+0000 7fbcd1600640 1 -- 172.21.15.27:0/445528692 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc00ad60 con 0x7fbcd4116640 2024-09-21T00:36:50.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.475+0000 7fbcdb600640 1 -- 172.21.15.27:0/445528692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd41a8e70 con 0x7fbcd4116640 2024-09-21T00:36:50.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.475+0000 7fbcdb600640 1 -- 172.21.15.27:0/445528692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd41a9400 con 0x7fbcd4116640 2024-09-21T00:36:50.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:50 smithi089 ceph-mon[34481]: pgmap v865: 97 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 9 active+clean; 683 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:36:50.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.477+0000 7fbcd1600640 1 -- 172.21.15.27:0/445528692 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbccc005020 con 0x7fbcd4116640 2024-09-21T00:36:50.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.477+0000 7fbcd1600640 1 -- 172.21.15.27:0/445528692 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc00f6b0 con 0x7fbcd4116640 2024-09-21T00:36:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.483+0000 7fbcd1600640 1 -- 172.21.15.27:0/445528692 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fbccc004a40 con 0x7fbcd4116640 2024-09-21T00:36:50.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.486+0000 7fbcd1600640 1 --2- 172.21.15.27:0/445528692 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbcbc0b67b0 0x7fbcbc0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.486+0000 7fbcd1600640 1 -- 172.21.15.27:0/445528692 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9290+0+0 (secure 0 0 0) 0x7fbccc0e7750 con 0x7fbcd4116640 2024-09-21T00:36:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.487+0000 7fbcdb600640 1 -- 172.21.15.27:0/445528692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb40052c0 con 0x7fbcd4116640 2024-09-21T00:36:50.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.488+0000 7fbcd8c00640 1 --2- 172.21.15.27:0/445528692 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbcbc0b67b0 0x7fbcbc0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:50.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.491+0000 7fbcd8c00640 1 --2- 172.21.15.27:0/445528692 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbcbc0b67b0 0x7fbcbc0b8cc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80096c0 tx=0x7fbcc8031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:50.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:50 smithi098 ceph-mon[34725]: pgmap v865: 97 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 9 active+clean; 683 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:36:50.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.497+0000 7fbcd1600640 1 -- 172.21.15.27:0/445528692 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbccc014030 con 0x7fbcd4116640 2024-09-21T00:36:50.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:50.903+0000 7fbcdb600640 1 -- 172.21.15.27:0/445528692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbcb40055b0 con 0x7fbcd4116640 2024-09-21T00:36:51.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:51.028+0000 7fbcd1600640 1 -- 172.21.15.27:0/445528692 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v116) ==== 169+0+0 (secure 0 0 0) 0x7fbccc0ad940 con 0x7fbcd4116640 2024-09-21T00:36:51.031 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-09-21T00:36:51.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:51.036+0000 7fbcdb600640 1 -- 172.21.15.27:0/445528692 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbcbc0b67b0 msgr2=0x7fbcbc0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:51.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:51.036+0000 7fbcdb600640 1 --2- 172.21.15.27:0/445528692 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbcbc0b67b0 0x7fbcbc0b8cc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80096c0 tx=0x7fbcc8031400 comp rx=0 tx=0).stop 2024-09-21T00:36:51.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:51.036+0000 7fbcdb600640 1 -- 172.21.15.27:0/445528692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcd4116640 msgr2=0x7fbcd41a8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:51.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:51.036+0000 7fbcdb600640 1 --2- 172.21.15.27:0/445528692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcd4116640 0x7fbcd41a8870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00cad0 tx=0x7fbccc00ced0 comp rx=0 tx=0).stop 2024-09-21T00:36:51.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:51.036+0000 7fbcdb600640 1 -- 172.21.15.27:0/445528692 shutdown_connections 2024-09-21T00:36:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:51.036+0000 7fbcdb600640 1 --2- 172.21.15.27:0/445528692 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7fbcbc0b67b0 0x7fbcbc0b8cc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:51.036+0000 7fbcdb600640 1 --2- 172.21.15.27:0/445528692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcd4116640 0x7fbcd41a8870 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:51.036+0000 7fbcdb600640 1 --2- 172.21.15.27:0/445528692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcd410aee0 0x7fbcd41a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:51.036+0000 7fbcdb600640 1 --2- 172.21.15.27:0/445528692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd410a590 0x7fbcd41a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:51.036+0000 7fbcdb600640 1 -- 172.21.15.27:0/445528692 >> 172.21.15.27:0/445528692 conn(0x7fbcd4070ec0 msgr2=0x7fbcd410c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:51.037+0000 7fbcdb600640 1 -- 172.21.15.27:0/445528692 shutdown_connections 2024-09-21T00:36:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:51.037+0000 7fbcdb600640 1 -- 172.21.15.27:0/445528692 wait complete. 2024-09-21T00:36:51.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:51 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/445528692' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T00:36:51.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:51 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T00:36:51.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:51 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/445528692' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T00:36:51.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:51 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T00:36:51.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:51 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/445528692' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T00:36:51.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:51 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T00:36:52.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:52 smithi027 ceph-mon[30745]: pgmap v866: 97 pgs: 20 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 17 active+clean; 683 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:36:52.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:52 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T00:36:52.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:52 smithi027 ceph-mon[30745]: osdmap e116: 12 total, 12 up, 12 in 2024-09-21T00:36:52.379 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:52 smithi089 ceph-mon[34481]: pgmap v866: 97 pgs: 20 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 17 active+clean; 683 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:36:52.379 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:52 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T00:36:52.379 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:52 smithi089 ceph-mon[34481]: osdmap e116: 12 total, 12 up, 12 in 2024-09-21T00:36:52.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:52 smithi098 ceph-mon[34725]: pgmap v866: 97 pgs: 20 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 17 active+clean; 683 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:36:52.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:52 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T00:36:52.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:52 smithi098 ceph-mon[34725]: osdmap e116: 12 total, 12 up, 12 in 2024-09-21T00:36:52.613 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-21T00:36:53.527 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T00:36:54.228 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:54 smithi027 ceph-mon[30745]: pgmap v868: 33 pgs: 14 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 2 active+clean; 683 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:36:54.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:54 smithi089 ceph-mon[34481]: pgmap v868: 33 pgs: 14 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 2 active+clean; 683 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:36:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.479+0000 7f71be600640 1 -- 172.21.15.27:0/3467091282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b810b500 msgr2=0x7f71b810b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.479+0000 7f71be600640 1 --2- 172.21.15.27:0/3467091282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b810b500 0x7f71b810b910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009900 tx=0x7f71a402a410 comp rx=0 tx=0).stop 2024-09-21T00:36:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.479+0000 7f71be600640 1 -- 172.21.15.27:0/3467091282 shutdown_connections 2024-09-21T00:36:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.479+0000 7f71be600640 1 --2- 172.21.15.27:0/3467091282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b810be50 0x7f71b8112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.479+0000 7f71be600640 1 --2- 172.21.15.27:0/3467091282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b810b500 0x7f71b810b910 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.479+0000 7f71be600640 1 --2- 172.21.15.27:0/3467091282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b8105bc0 0x7f71b8105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:54.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.479+0000 7f71be600640 1 -- 172.21.15.27:0/3467091282 >> 172.21.15.27:0/3467091282 conn(0x7f71b8101630 msgr2=0x7f71b8103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:54.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.481+0000 7f71be600640 1 -- 172.21.15.27:0/3467091282 shutdown_connections 2024-09-21T00:36:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.481+0000 7f71be600640 1 -- 172.21.15.27:0/3467091282 wait complete. 2024-09-21T00:36:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.483+0000 7f71be600640 1 Processor -- start 2024-09-21T00:36:54.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.484+0000 7f71be600640 1 -- start start 2024-09-21T00:36:54.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.484+0000 7f71be600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b8105bc0 0x7f71b8112620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:54.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.484+0000 7f71be600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b810b500 0x7f71b810f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:54.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.484+0000 7f71be600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b810be50 0x7f71b81100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.485+0000 7f71b7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b810b500 0x7f71b810f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.485+0000 7f71bd200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b8105bc0 0x7f71b8112620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.485+0000 7f71bd200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b8105bc0 0x7f71b8112620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38098/0 (socket says 172.21.15.27:38098) 2024-09-21T00:36:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.485+0000 7f71bdc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b810be50 0x7f71b81100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:54.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.485+0000 7f71bdc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b810be50 0x7f71b81100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:40344/0 (socket says 172.21.15.27:40344) 2024-09-21T00:36:54.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.486+0000 7f71bd200640 1 -- 172.21.15.27:0/4087825890 learned_addr learned my addr 172.21.15.27:0/4087825890 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T00:36:54.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.486+0000 7f71be600640 1 -- 172.21.15.27:0/4087825890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f71b8115b60 con 0x7f71b8105bc0 2024-09-21T00:36:54.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.486+0000 7f71be600640 1 -- 172.21.15.27:0/4087825890 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81159e0 con 0x7f71b810b500 2024-09-21T00:36:54.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.486+0000 7f71be600640 1 -- 172.21.15.27:0/4087825890 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f71b8115ce0 con 0x7f71b810be50 2024-09-21T00:36:54.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.487+0000 7f71bdc00640 1 -- 172.21.15.27:0/4087825890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b810b500 msgr2=0x7f71b810f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:54.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.487+0000 7f71bdc00640 1 --2- 172.21.15.27:0/4087825890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b810b500 0x7f71b810f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:54.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.487+0000 7f71bdc00640 1 -- 172.21.15.27:0/4087825890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b8105bc0 msgr2=0x7f71b8112620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:54.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.487+0000 7f71bdc00640 1 --2- 172.21.15.27:0/4087825890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b8105bc0 0x7f71b8112620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:54.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.488+0000 7f71bdc00640 1 -- 172.21.15.27:0/4087825890 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b81a2020 con 0x7f71b810be50 2024-09-21T00:36:54.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.488+0000 7f71bd200640 1 --2- 172.21.15.27:0/4087825890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b8105bc0 0x7f71b8112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:36:54.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.488+0000 7f71b7e00640 1 --2- 172.21.15.27:0/4087825890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b810b500 0x7f71b810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:36:54.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.489+0000 7f71bdc00640 1 --2- 172.21.15.27:0/4087825890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b810be50 0x7f71b81100d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009fd0 tx=0x7f71b000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:54.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.490+0000 7f71b5600640 1 -- 172.21.15.27:0/4087825890 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b000ce20 con 0x7f71b810be50 2024-09-21T00:36:54.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.490+0000 7f71b5600640 1 -- 172.21.15.27:0/4087825890 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b0005020 con 0x7f71b810be50 2024-09-21T00:36:54.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.490+0000 7f71b5600640 1 -- 172.21.15.27:0/4087825890 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0010690 con 0x7f71b810be50 2024-09-21T00:36:54.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.490+0000 7f71be600640 1 -- 172.21.15.27:0/4087825890 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b81a22b0 con 0x7f71b810be50 2024-09-21T00:36:54.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.490+0000 7f71be600640 1 -- 172.21.15.27:0/4087825890 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b81a2770 con 0x7f71b810be50 2024-09-21T00:36:54.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:54 smithi098 ceph-mon[34725]: pgmap v868: 33 pgs: 14 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 2 active+clean; 683 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:36:54.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.497+0000 7f71be600640 1 -- 172.21.15.27:0/4087825890 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b8106e40 con 0x7f71b810be50 2024-09-21T00:36:54.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.503+0000 7f71b5600640 1 -- 172.21.15.27:0/4087825890 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f71b00054d0 con 0x7f71b810be50 2024-09-21T00:36:54.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.508+0000 7f71b5600640 1 --2- 172.21.15.27:0/4087825890 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f71a00b6850 0x7f71a00b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:36:54.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.509+0000 7f71b5600640 1 -- 172.21.15.27:0/4087825890 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8702+0+0 (secure 0 0 0) 0x7f71b00e8910 con 0x7f71b810be50 2024-09-21T00:36:54.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.509+0000 7f71b5600640 1 -- 172.21.15.27:0/4087825890 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f71b001d060 con 0x7f71b810be50 2024-09-21T00:36:54.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.510+0000 7f71bd200640 1 --2- 172.21.15.27:0/4087825890 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f71a00b6850 0x7f71a00b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:36:54.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.511+0000 7f71bd200640 1 --2- 172.21.15.27:0/4087825890 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f71a00b6850 0x7f71a00b8d60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f71b8112370 tx=0x7f71ac005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:36:54.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:54.911+0000 7f71be600640 1 -- 172.21.15.27:0/4087825890 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f71b8110c10 con 0x7f71b810be50 2024-09-21T00:36:55.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:55.055+0000 7f71b5600640 1 -- 172.21.15.27:0/4087825890 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v117) ==== 178+0+0 (secure 0 0 0) 0x7f71b8110c10 con 0x7f71b810be50 2024-09-21T00:36:55.056 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data_ec' removed 2024-09-21T00:36:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:55.059+0000 7f71be600640 1 -- 172.21.15.27:0/4087825890 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f71a00b6850 msgr2=0x7f71a00b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:55.059+0000 7f71be600640 1 --2- 172.21.15.27:0/4087825890 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f71a00b6850 0x7f71a00b8d60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f71b8112370 tx=0x7f71ac005f90 comp rx=0 tx=0).stop 2024-09-21T00:36:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:55.059+0000 7f71be600640 1 -- 172.21.15.27:0/4087825890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b810be50 msgr2=0x7f71b81100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:36:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:55.059+0000 7f71be600640 1 --2- 172.21.15.27:0/4087825890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b810be50 0x7f71b81100d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009fd0 tx=0x7f71b000ef10 comp rx=0 tx=0).stop 2024-09-21T00:36:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:55.059+0000 7f71be600640 1 -- 172.21.15.27:0/4087825890 shutdown_connections 2024-09-21T00:36:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:55.059+0000 7f71be600640 1 --2- 172.21.15.27:0/4087825890 >> [v2:172.21.15.27:6800/2382590111,v1:172.21.15.27:6801/2382590111] conn(0x7f71a00b6850 0x7f71a00b8d60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:55.059+0000 7f71be600640 1 --2- 172.21.15.27:0/4087825890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b810be50 0x7f71b81100d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:55.059+0000 7f71be600640 1 --2- 172.21.15.27:0/4087825890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b810b500 0x7f71b810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:55.059+0000 7f71be600640 1 --2- 172.21.15.27:0/4087825890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b8105bc0 0x7f71b8112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:36:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:55.059+0000 7f71be600640 1 -- 172.21.15.27:0/4087825890 >> 172.21.15.27:0/4087825890 conn(0x7f71b8101630 msgr2=0x7f71b8108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:36:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:55.060+0000 7f71be600640 1 -- 172.21.15.27:0/4087825890 shutdown_connections 2024-09-21T00:36:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T00:36:55.060+0000 7f71be600640 1 -- 172.21.15.27:0/4087825890 wait complete. 2024-09-21T00:36:55.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:55 smithi027 ceph-mon[30745]: from='client.? 172.21.15.27:0/4087825890' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T00:36:55.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:55 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T00:36:55.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:55 smithi089 ceph-mon[34481]: from='client.? 172.21.15.27:0/4087825890' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T00:36:55.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:55 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T00:36:55.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:55 smithi098 ceph-mon[34725]: from='client.? 172.21.15.27:0/4087825890' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T00:36:55.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:55 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T00:36:56.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:56 smithi027 ceph-mon[30745]: pgmap v869: 33 pgs: 16 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 2 active+clean; 683 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:36:56.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:56 smithi027 ceph-mon[30745]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T00:36:56.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:56 smithi027 ceph-mon[30745]: osdmap e117: 12 total, 12 up, 12 in 2024-09-21T00:36:56.074 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T00:36:56.171 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T00:36:56.236 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T00:36:56.310 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T00:36:56.310 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T00:36:56.366 DEBUG:teuthology.orchestra.run.smithi089:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T00:36:56.391 DEBUG:teuthology.orchestra.run.smithi098:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T00:36:56.413 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T00:36:56.414 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T00:36:56.414 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mon.a 2024-09-21T00:36:56.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:56 smithi089 ceph-mon[34481]: pgmap v869: 33 pgs: 16 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 2 active+clean; 683 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:36:56.478 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:56 smithi089 ceph-mon[34481]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T00:36:56.479 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:56 smithi089 ceph-mon[34481]: osdmap e117: 12 total, 12 up, 12 in 2024-09-21T00:36:56.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:56 smithi098 ceph-mon[34725]: pgmap v869: 33 pgs: 16 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 2 active+clean; 683 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T00:36:56.495 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:56 smithi098 ceph-mon[34725]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T00:36:56.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:36:56 smithi098 ceph-mon[34725]: osdmap e117: 12 total, 12 up, 12 in 2024-09-21T00:36:56.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:56 smithi027 systemd[1]: Stopping Ceph mon.a for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:36:57.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:57 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a[30741]: 2024-09-21T00:36:57.111+0000 7f214c000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:36:57.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:57 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a[30741]: 2024-09-21T00:36:57.111+0000 7f214c000640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T00:36:57.422 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:57 smithi027 podman[98617]: 2024-09-21 00:36:57.171844387 +0000 UTC m=+0.230149713 container died 3adf3fc84023f274831c402345f8068f8ecf738e6dc5c5cac51bf36d19bd0a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T00:36:58.040 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:57 smithi027 podman[98617]: 2024-09-21 00:36:57.72968857 +0000 UTC m=+0.787993903 container cleanup 3adf3fc84023f274831c402345f8068f8ecf738e6dc5c5cac51bf36d19bd0a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T00:36:58.041 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:57 smithi027 bash[98617]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a 2024-09-21T00:36:58.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 00:36:58 smithi027 podman[98628]: 2024-09-21 00:36:58.039910082 +0000 UTC m=+0.853511253 container remove 3adf3fc84023f274831c402345f8068f8ecf738e6dc5c5cac51bf36d19bd0a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-a, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T00:36:58.339 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T00:36:58.439 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:36:58.439 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T00:36:58.439 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T00:36:58.439 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mon.b 2024-09-21T00:36:58.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:58 smithi089 systemd[1]: Stopping Ceph mon.b for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:36:59.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:58 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-b[34477]: 2024-09-21T00:36:58.784+0000 7f7b115fc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:36:59.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:58 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-b[34477]: 2024-09-21T00:36:58.784+0000 7f7b115fc640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T00:36:59.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:58 smithi089 podman[67411]: 2024-09-21 00:36:58.957873425 +0000 UTC m=+0.315313108 container died 834528238ca832d9ddbc62e56dff915993f98dba5b00903cfebb1154cf3f74f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T00:37:00.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:59 smithi089 podman[67411]: 2024-09-21 00:36:59.776071392 +0000 UTC m=+1.133511074 container cleanup 834528238ca832d9ddbc62e56dff915993f98dba5b00903cfebb1154cf3f74f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-b, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T00:37:00.228 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:36:59 smithi089 bash[67411]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-b 2024-09-21T00:37:00.567 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:37:00.067+0000 7fbede200640 -1 client.14238: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-21T00:37:00.567 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:37:00.069+0000 7fbede200640 -1 client.14238: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-21T00:37:00.567 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: Exception in thread Thread-20: 2024-09-21T00:37:00.567 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: Traceback (most recent call last): 2024-09-21T00:37:00.567 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib64/python3.9/threading.py", line 980, in _bootstrap_inner 2024-09-21T00:37:00.567 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: self.run() 2024-09-21T00:37:00.568 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/lib64/python3.9/threading.py", line 1306, in run 2024-09-21T00:37:00.568 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: self.function(*self.args, **self.kwargs) 2024-09-21T00:37:00.568 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line 355, in create_scheduled_snapshot 2024-09-21T00:37:00.568 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: self.prune_snapshots(sched) 2024-09-21T00:37:00.568 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: UnboundLocalError: local variable 'sched' referenced before assignment 2024-09-21T00:37:02.803 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 00:37:02 smithi089 podman[67422]: 2024-09-21 00:37:02.546530036 +0000 UTC m=+3.591036063 container remove 834528238ca832d9ddbc62e56dff915993f98dba5b00903cfebb1154cf3f74f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T00:37:02.807 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T00:37:03.097 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:37:03.098 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T00:37:03.098 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T00:37:03.098 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mon.c 2024-09-21T00:37:03.465 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:37:03 smithi098 systemd[1]: Stopping Ceph mon.c for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:37:03.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:37:03 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-c[34707]: 2024-09-21T00:37:03.463+0000 7f804a681640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:37:03.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:37:03 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-c[34707]: 2024-09-21T00:37:03.463+0000 7f804a681640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T00:37:03.746 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:37:03 smithi098 podman[68794]: 2024-09-21 00:37:03.69887118 +0000 UTC m=+0.362637217 container died e211261a2fc96ffd88b03b5787efdcfe94e444afd0bc9ef3e437a71e54343166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T00:37:04.409 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:37:04 smithi098 podman[68794]: 2024-09-21 00:37:04.081529733 +0000 UTC m=+0.745295768 container cleanup e211261a2fc96ffd88b03b5787efdcfe94e444afd0bc9ef3e437a71e54343166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-c, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T00:37:04.409 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 00:37:04 smithi098 bash[68794]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mon-c 2024-09-21T00:37:04.534 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T00:37:04.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:37:04.584 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T00:37:04.585 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T00:37:04.585 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.x 2024-09-21T00:37:05.033 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:04 smithi027 systemd[1]: Stopping Ceph mgr.x for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:37:05.033 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:05 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:37:05.030+0000 7fbfa6400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:37:05.033 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:05 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x[30970]: 2024-09-21T00:37:05.030+0000 7fbfa6400640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-21T00:37:05.317 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:05 smithi027 podman[98747]: 2024-09-21 00:37:05.133136006 +0000 UTC m=+0.250340888 container died c336f5fa731be341ff4792f6fae2a737fac1aeb18978046e6a322ab0786e9ab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T00:37:05.817 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:05 smithi027 podman[98747]: 2024-09-21 00:37:05.603118371 +0000 UTC m=+0.720323252 container cleanup c336f5fa731be341ff4792f6fae2a737fac1aeb18978046e6a322ab0786e9ab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:37:05.817 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:05 smithi027 bash[98747]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x 2024-09-21T00:37:06.299 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T00:37:06.350 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:06 smithi027 podman[98761]: 2024-09-21 00:37:06.071693762 +0000 UTC m=+0.924364723 container remove c336f5fa731be341ff4792f6fae2a737fac1aeb18978046e6a322ab0786e9ab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-x, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T00:37:06.350 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:06 smithi027 systemd[1]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.x.service: Deactivated successfully. 2024-09-21T00:37:06.350 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:06 smithi027 systemd[1]: Stopped Ceph mgr.x for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:37:06.350 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 00:37:06 smithi027 systemd[1]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.x.service: Consumed 1min 7.878s CPU time. 2024-09-21T00:37:06.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:37:06.379 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T00:37:06.379 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T00:37:06.379 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.y 2024-09-21T00:37:06.711 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:37:06 smithi089 systemd[1]: Stopping Ceph mgr.y for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:37:06.979 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:37:06 smithi089 podman[67531]: 2024-09-21 00:37:06.710955568 +0000 UTC m=+0.144935761 container stop 69bc3dd205031e490a9797533171f17f72dbb360b46781187bd583e1cb67e5cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-y, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T00:37:06.979 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:37:06 smithi089 podman[67531]: 2024-09-21 00:37:06.736975959 +0000 UTC m=+0.170956140 container died 69bc3dd205031e490a9797533171f17f72dbb360b46781187bd583e1cb67e5cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-y, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T00:37:07.326 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:37:07 smithi089 podman[67531]: 2024-09-21 00:37:07.099014882 +0000 UTC m=+0.532995065 container cleanup 69bc3dd205031e490a9797533171f17f72dbb360b46781187bd583e1cb67e5cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-y, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-21T00:37:07.326 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:37:07 smithi089 bash[67531]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-y 2024-09-21T00:37:07.578 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T00:37:07.599 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:37:07 smithi089 podman[67543]: 2024-09-21 00:37:07.325966062 +0000 UTC m=+0.609392948 container remove 69bc3dd205031e490a9797533171f17f72dbb360b46781187bd583e1cb67e5cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-y, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T00:37:07.599 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:37:07 smithi089 systemd[1]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-21T00:37:07.599 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:37:07 smithi089 systemd[1]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.y.service: Failed with result 'exit-code'. 2024-09-21T00:37:07.599 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:37:07 smithi089 systemd[1]: Stopped Ceph mgr.y for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:37:07.599 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 00:37:07 smithi089 systemd[1]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.y.service: Consumed 7.388s CPU time. 2024-09-21T00:37:07.612 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:37:07.612 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T00:37:07.612 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T00:37:07.613 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.z 2024-09-21T00:37:07.941 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 00:37:07 smithi098 systemd[1]: Stopping Ceph mgr.z for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:37:08.239 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 00:37:07 smithi098 podman[68917]: 2024-09-21 00:37:07.941096974 +0000 UTC m=+0.132476271 container died 422b28bd1a063f7f7f5664d4d47f3ce2e8d2b33b8d3152a6ae3020e3e837b2e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-z, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T00:37:08.495 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 00:37:08 smithi098 podman[68917]: 2024-09-21 00:37:08.329472558 +0000 UTC m=+0.520851858 container cleanup 422b28bd1a063f7f7f5664d4d47f3ce2e8d2b33b8d3152a6ae3020e3e837b2e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T00:37:08.515 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 00:37:08 smithi098 bash[68917]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mgr-z 2024-09-21T00:37:08.785 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T00:37:08.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:37:08.820 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T00:37:08.820 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T00:37:08.820 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.0 2024-09-21T00:37:09.235 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:37:08 smithi027 systemd[1]: Stopping Ceph osd.0 for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:37:09.567 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:37:09 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-0[45001]: 2024-09-21T00:37:09.303+0000 7f3890e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:37:09.567 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:37:09 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-0[45001]: 2024-09-21T00:37:09.303+0000 7f3890e00640 -1 osd.0 117 *** Got signal Terminated *** 2024-09-21T00:37:09.567 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:37:09 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-0[45001]: 2024-09-21T00:37:09.303+0000 7f3890e00640 -1 osd.0 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T00:37:15.237 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:37:14 smithi027 podman[98867]: 2024-09-21 00:37:14.941166481 +0000 UTC m=+5.821130706 container died c9317691a4a7091e372826ff0637a740c8a3a3620d8d35c74f6b3aca25715f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:37:15.567 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:37:15 smithi027 podman[98867]: 2024-09-21 00:37:15.358674179 +0000 UTC m=+6.238638400 container cleanup c9317691a4a7091e372826ff0637a740c8a3a3620d8d35c74f6b3aca25715f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T00:37:15.567 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:37:15 smithi027 bash[98867]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-0 2024-09-21T00:37:16.066 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:37:15 smithi027 podman[98896]: 2024-09-21 00:37:15.635403144 +0000 UTC m=+0.682074838 container remove c9317691a4a7091e372826ff0637a740c8a3a3620d8d35c74f6b3aca25715f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T00:37:16.540 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:37:16 smithi027 podman[98987]: 2024-09-21 00:37:16.127407607 +0000 UTC m=+0.027815987 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:37:16.540 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:37:16 smithi027 podman[98987]: 2024-09-21 00:37:16.261590728 +0000 UTC m=+0.161999093 container create 212ea60c123ba64d4cfa1fe9833d5058e4a39cd2e7a8b0f0dfe95963a9c69348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T00:37:16.541 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:37:16 smithi027 podman[98987]: 2024-09-21 00:37:16.539709967 +0000 UTC m=+0.440118349 container init 212ea60c123ba64d4cfa1fe9833d5058e4a39cd2e7a8b0f0dfe95963a9c69348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:37:16.817 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:37:16 smithi027 podman[98987]: 2024-09-21 00:37:16.546843737 +0000 UTC m=+0.447252095 container start 212ea60c123ba64d4cfa1fe9833d5058e4a39cd2e7a8b0f0dfe95963a9c69348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T00:37:16.817 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:37:16 smithi027 podman[98987]: 2024-09-21 00:37:16.639548769 +0000 UTC m=+0.539957128 container attach 212ea60c123ba64d4cfa1fe9833d5058e4a39cd2e7a8b0f0dfe95963a9c69348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T00:37:17.316 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 00:37:16 smithi027 podman[99010]: 2024-09-21 00:37:16.984996051 +0000 UTC m=+0.028005555 container died 212ea60c123ba64d4cfa1fe9833d5058e4a39cd2e7a8b0f0dfe95963a9c69348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:37:17.727 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T00:37:17.806 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:37:17.806 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T00:37:17.806 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T00:37:17.806 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.1 2024-09-21T00:37:18.122 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:37:17 smithi089 systemd[1]: Stopping Ceph osd.1 for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:37:18.479 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:37:18 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-1[39091]: 2024-09-21T00:37:18.120+0000 7fac3e616640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:37:18.479 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:37:18 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-1[39091]: 2024-09-21T00:37:18.120+0000 7fac3e616640 -1 osd.1 117 *** Got signal Terminated *** 2024-09-21T00:37:18.479 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:37:18 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-1[39091]: 2024-09-21T00:37:18.120+0000 7fac3e616640 -1 osd.1 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T00:37:23.973 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:37:23 smithi089 podman[67658]: 2024-09-21 00:37:23.653992183 +0000 UTC m=+5.655272014 container died 60346f87f015f7e23b086f2b9a6975397bec29580ee76a62d2c353de7ea2cc8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-1, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T00:37:24.228 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:37:24 smithi089 podman[67658]: 2024-09-21 00:37:24.079648938 +0000 UTC m=+6.080928758 container cleanup 60346f87f015f7e23b086f2b9a6975397bec29580ee76a62d2c353de7ea2cc8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T00:37:24.228 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:37:24 smithi089 bash[67658]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-1 2024-09-21T00:37:24.715 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:37:24 smithi089 podman[67674]: 2024-09-21 00:37:24.322955979 +0000 UTC m=+0.659939711 container remove 60346f87f015f7e23b086f2b9a6975397bec29580ee76a62d2c353de7ea2cc8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-1, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:37:24.978 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:37:24 smithi089 podman[67763]: 2024-09-21 00:37:24.616917261 +0000 UTC m=+0.015684762 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:37:24.979 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:37:24 smithi089 podman[67763]: 2024-09-21 00:37:24.74905386 +0000 UTC m=+0.147821401 container create 5457efa2543f9e327d0d7ac58e970b0029a65e3c19ccf5dc503af2dd52ae25a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T00:37:25.233 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:37:25 smithi089 podman[67763]: 2024-09-21 00:37:25.01831588 +0000 UTC m=+0.417083378 container init 5457efa2543f9e327d0d7ac58e970b0029a65e3c19ccf5dc503af2dd52ae25a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T00:37:25.233 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:37:25 smithi089 podman[67763]: 2024-09-21 00:37:25.021631448 +0000 UTC m=+0.420398947 container start 5457efa2543f9e327d0d7ac58e970b0029a65e3c19ccf5dc503af2dd52ae25a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T00:37:25.233 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:37:25 smithi089 podman[67763]: 2024-09-21 00:37:25.101687751 +0000 UTC m=+0.500455257 container attach 5457efa2543f9e327d0d7ac58e970b0029a65e3c19ccf5dc503af2dd52ae25a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T00:37:25.525 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 00:37:25 smithi089 podman[67763]: 2024-09-21 00:37:25.233020378 +0000 UTC m=+0.631787885 container died 5457efa2543f9e327d0d7ac58e970b0029a65e3c19ccf5dc503af2dd52ae25a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:37:25.876 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T00:37:25.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:37:25.915 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T00:37:25.916 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T00:37:25.916 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.2 2024-09-21T00:37:26.211 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:37:25 smithi098 systemd[1]: Stopping Ceph osd.2 for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:37:26.496 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:37:26 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-2[39902]: 2024-09-21T00:37:26.209+0000 7fa4dc8f9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:37:26.496 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:37:26 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-2[39902]: 2024-09-21T00:37:26.209+0000 7fa4dc8f9640 -1 osd.2 117 *** Got signal Terminated *** 2024-09-21T00:37:26.496 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:37:26 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-2[39902]: 2024-09-21T00:37:26.209+0000 7fa4dc8f9640 -1 osd.2 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T00:37:31.996 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:37:31 smithi098 podman[69053]: 2024-09-21 00:37:31.714122534 +0000 UTC m=+5.603044345 container died 996f59ee0e0af8e6c232cbde8bf2b59bee18477b0211743c8016cb153263f01c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:37:32.396 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:37:32 smithi098 podman[69053]: 2024-09-21 00:37:32.110303066 +0000 UTC m=+5.999224876 container cleanup 996f59ee0e0af8e6c232cbde8bf2b59bee18477b0211743c8016cb153263f01c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-21T00:37:32.397 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:37:32 smithi098 bash[69053]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-2 2024-09-21T00:37:32.702 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:37:32 smithi098 podman[69068]: 2024-09-21 00:37:32.395887998 +0000 UTC m=+0.682812380 container remove 996f59ee0e0af8e6c232cbde8bf2b59bee18477b0211743c8016cb153263f01c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T00:37:32.996 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:37:32 smithi098 podman[69157]: 2024-09-21 00:37:32.604590152 +0000 UTC m=+0.015971422 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:37:32.996 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:37:32 smithi098 podman[69157]: 2024-09-21 00:37:32.738413979 +0000 UTC m=+0.149795250 container create f6ad39d37cc7709bf274626821a27071a2e16490b176c598905ef672968d2d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T00:37:33.247 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:37:32 smithi098 podman[69157]: 2024-09-21 00:37:32.999744291 +0000 UTC m=+0.411125560 container init f6ad39d37cc7709bf274626821a27071a2e16490b176c598905ef672968d2d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T00:37:33.247 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:37:33 smithi098 podman[69157]: 2024-09-21 00:37:33.003368184 +0000 UTC m=+0.414749455 container start f6ad39d37cc7709bf274626821a27071a2e16490b176c598905ef672968d2d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T00:37:33.247 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:37:33 smithi098 podman[69157]: 2024-09-21 00:37:33.099798565 +0000 UTC m=+0.511179848 container attach f6ad39d37cc7709bf274626821a27071a2e16490b176c598905ef672968d2d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-21T00:37:33.525 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 00:37:33 smithi098 podman[69157]: 2024-09-21 00:37:33.247378358 +0000 UTC m=+0.658759637 container died f6ad39d37cc7709bf274626821a27071a2e16490b176c598905ef672968d2d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T00:37:33.924 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T00:37:33.964 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:37:33.964 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T00:37:33.964 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T00:37:33.964 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.3 2024-09-21T00:37:34.317 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:37:34 smithi027 systemd[1]: Stopping Ceph osd.3 for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:37:34.817 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:37:34 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-3[51258]: 2024-09-21T00:37:34.417+0000 7f3675600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:37:34.817 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:37:34 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-3[51258]: 2024-09-21T00:37:34.417+0000 7f3675600640 -1 osd.3 117 *** Got signal Terminated *** 2024-09-21T00:37:34.817 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:37:34 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-3[51258]: 2024-09-21T00:37:34.417+0000 7f3675600640 -1 osd.3 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T00:37:36.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:36 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:36.194+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:36.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:36 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:36.258+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:37.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:36 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:36.688+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:37.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:37 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:37.209+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:37.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:37 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:37.262+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:38.066 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:37 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:37.736+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:38.495 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:38.230+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:38.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:38.304+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:38.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:38 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:38.422+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:38.817 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:37:38 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-3[51258]: 2024-09-21T00:37:38.508+0000 7f3671000640 -1 osd.3 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.637286+0000 front 2024-09-21T00:37:13.637215+0000 (oldest deadline 2024-09-21T00:37:37.737202+0000) 2024-09-21T00:37:38.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:38 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:38.703+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:39.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:39 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:39.194+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:39.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:39 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:39.262+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:39.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:39 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:39.434+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:39.995 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:39 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:39.605+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:40.066 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:39 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:39.685+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:40.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:40 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:40.231+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:40.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:40 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:40.264+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:40.566 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:37:40 smithi027 podman[99091]: 2024-09-21 00:37:40.29380854 +0000 UTC m=+6.027137731 container died a2a08cdf0d8991e280e24b818d7d2ad5e170a1de787ca813cd13f632125fcb8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:37:40.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:40 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:40.433+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:40.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:40 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:40.624+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:41.067 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:37:40 smithi027 podman[99091]: 2024-09-21 00:37:40.772376664 +0000 UTC m=+6.505705861 container cleanup a2a08cdf0d8991e280e24b818d7d2ad5e170a1de787ca813cd13f632125fcb8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-3, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T00:37:41.067 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:37:40 smithi027 bash[99091]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-3 2024-09-21T00:37:41.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:40 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:40.643+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:41.566 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:37:41 smithi027 podman[99109]: 2024-09-21 00:37:41.107016427 +0000 UTC m=+0.781101124 container remove a2a08cdf0d8991e280e24b818d7d2ad5e170a1de787ca813cd13f632125fcb8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-3, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:37:41.665 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:41 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:41.270+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:41.665 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:41 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:41.253+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:41.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:41 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:41.411+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:41.995 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:41 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:41.664+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:42.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:41 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:41.638+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:42.067 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:37:41 smithi027 podman[99194]: 2024-09-21 00:37:41.606908925 +0000 UTC m=+0.028778011 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:37:42.067 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:37:41 smithi027 podman[99194]: 2024-09-21 00:37:41.724938606 +0000 UTC m=+0.146807675 container create 563c892c06fe45783100050fa9dbcb121109ba85bedea606e32ad28eab001def (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T00:37:42.387 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:37:42 smithi027 podman[99194]: 2024-09-21 00:37:42.111059124 +0000 UTC m=+0.532928220 container init 563c892c06fe45783100050fa9dbcb121109ba85bedea606e32ad28eab001def (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T00:37:42.388 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:37:42 smithi027 podman[99194]: 2024-09-21 00:37:42.118421823 +0000 UTC m=+0.540290894 container start 563c892c06fe45783100050fa9dbcb121109ba85bedea606e32ad28eab001def (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T00:37:42.388 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:37:42 smithi027 podman[99194]: 2024-09-21 00:37:42.219494924 +0000 UTC m=+0.641364024 container attach 563c892c06fe45783100050fa9dbcb121109ba85bedea606e32ad28eab001def (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T00:37:42.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:42 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:42.242+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:42.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:42 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:42.207+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:42.655 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 00:37:42 smithi027 podman[99216]: 2024-09-21 00:37:42.430082884 +0000 UTC m=+0.029564248 container died 563c892c06fe45783100050fa9dbcb121109ba85bedea606e32ad28eab001def (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T00:37:42.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:42 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:42.454+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:42.995 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:42 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:42.666+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:43.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:42 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:42.653+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:43.171 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T00:37:43.249 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:37:43.250 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T00:37:43.250 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T00:37:43.250 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.4 2024-09-21T00:37:43.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:43 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:43.276+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:43.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:43 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:43.208+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:43.568 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:43 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:43.497+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:43.568 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:43 smithi089 systemd[1]: Stopping Ceph osd.4 for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:37:43.945 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:43 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:43.652+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:43.978 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:43 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4[45233]: 2024-09-21T00:37:43.567+0000 7f5b64685640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:37:43.978 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:43 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4[45233]: 2024-09-21T00:37:43.567+0000 7f5b64685640 -1 osd.4 117 *** Got signal Terminated *** 2024-09-21T00:37:43.978 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:43 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4[45233]: 2024-09-21T00:37:43.567+0000 7f5b64685640 -1 osd.4 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T00:37:43.995 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:43 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:43.681+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:44.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:44 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:44.267+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:44.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:44 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:44.222+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:44.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:44 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:44.459+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:44.995 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:44 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:44.657+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:44.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:44 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:44.657+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.235461+0000 front 2024-09-21T00:37:21.235500+0000 (oldest deadline 2024-09-21T00:37:44.135354+0000) 2024-09-21T00:37:45.066 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:44 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:44.611+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:45.457 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:45 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4[45233]: 2024-09-21T00:37:45.132+0000 7f5b5cc68640 -1 osd.4 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:20.423174+0000 front 2024-09-21T00:37:20.423207+0000 (oldest deadline 2024-09-21T00:37:44.523319+0000) 2024-09-21T00:37:45.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:45 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:45.289+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:45.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:45 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:45.204+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:45.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:45 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:45.456+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:45.995 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:45 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:45.669+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:45.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:45 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:45.669+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.235461+0000 front 2024-09-21T00:37:21.235500+0000 (oldest deadline 2024-09-21T00:37:44.135354+0000) 2024-09-21T00:37:46.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:45 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:45.653+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:46.478 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:46 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4[45233]: 2024-09-21T00:37:46.086+0000 7f5b5cc68640 -1 osd.4 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:20.423174+0000 front 2024-09-21T00:37:20.423207+0000 (oldest deadline 2024-09-21T00:37:44.523319+0000) 2024-09-21T00:37:46.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:46 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:46.219+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:46.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:46 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:46.291+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:46.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:46 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:46.291+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:37:46.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:46 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:46.484+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:46.995 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:46 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:46.690+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:46.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:46 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:46.690+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.235461+0000 front 2024-09-21T00:37:21.235500+0000 (oldest deadline 2024-09-21T00:37:44.135354+0000) 2024-09-21T00:37:47.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:46 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:46.637+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:47.478 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:47 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4[45233]: 2024-09-21T00:37:47.132+0000 7f5b5cc68640 -1 osd.4 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:20.423174+0000 front 2024-09-21T00:37:20.423207+0000 (oldest deadline 2024-09-21T00:37:44.523319+0000) 2024-09-21T00:37:47.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:47 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:47.261+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:47.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:47 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:47.261+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:37:47.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:47 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:47.221+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:47.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:47 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:47.221+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:37:47.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:47 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:47.493+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:47.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:47 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:47.493+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:37:47.995 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:47 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:47.685+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:47.995 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:47 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:47.685+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.235461+0000 front 2024-09-21T00:37:21.235500+0000 (oldest deadline 2024-09-21T00:37:44.135354+0000) 2024-09-21T00:37:48.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:47 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:47.657+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:48.478 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:48 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4[45233]: 2024-09-21T00:37:48.141+0000 7f5b5cc68640 -1 osd.4 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:20.423174+0000 front 2024-09-21T00:37:20.423207+0000 (oldest deadline 2024-09-21T00:37:44.523319+0000) 2024-09-21T00:37:48.479 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:48 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:48.076+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:37:48.479 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:48 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:48.219+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:48.479 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:48 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:48.219+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:37:48.480 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:48 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:48.263+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:48.480 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:48 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:48.263+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:37:48.567 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:48 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:48.304+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:37:48.746 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:48 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:48.692+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:48.746 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:48 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:48.692+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.235461+0000 front 2024-09-21T00:37:21.235500+0000 (oldest deadline 2024-09-21T00:37:44.135354+0000) 2024-09-21T00:37:48.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:48 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:48.500+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:48.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:48 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:48.500+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:37:49.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:48 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:48.682+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:49.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:48 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:48.682+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:37:49.452 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:49 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:49.039+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:37:49.453 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:49 smithi089 podman[67877]: 2024-09-21 00:37:49.134484757 +0000 UTC m=+5.685683308 container died f274b0470b3b60f5a891d9773c0b9080089dde80914ae40d1783325aa145a681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T00:37:49.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:49 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:49.225+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:49.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:49 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:49.225+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:37:49.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:49 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:49.206+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:49.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:49 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:49.206+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:37:49.654 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:49.330+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:37:49.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:49 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:49.517+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:49.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:49 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:49.517+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:37:49.729 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:49 smithi089 podman[67877]: 2024-09-21 00:37:49.57579251 +0000 UTC m=+6.126991057 container cleanup f274b0470b3b60f5a891d9773c0b9080089dde80914ae40d1783325aa145a681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T00:37:49.729 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:49 smithi089 bash[67877]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4 2024-09-21T00:37:49.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:49 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:49.680+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:49.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:49 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:49.680+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.235461+0000 front 2024-09-21T00:37:21.235500+0000 (oldest deadline 2024-09-21T00:37:44.135354+0000) 2024-09-21T00:37:50.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:49.654+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:50.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:49.654+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:37:50.077 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:49 smithi089 podman[67895]: 2024-09-21 00:37:49.818976525 +0000 UTC m=+0.678902399 container remove f274b0470b3b60f5a891d9773c0b9080089dde80914ae40d1783325aa145a681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T00:37:50.330 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:50 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:50.075+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:37:50.330 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:50 smithi089 podman[67982]: 2024-09-21 00:37:50.11076158 +0000 UTC m=+0.015752126 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:37:50.330 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:50 smithi089 podman[67982]: 2024-09-21 00:37:50.211540351 +0000 UTC m=+0.116530937 container create 31d14ea421881c5f21b1090f380c1ef2ebff60180f2afa356c156ebcbdf5961c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T00:37:50.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:50 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:50.196+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:50.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:50 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:50.196+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:37:50.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:50 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:50.210+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:50.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:50 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:50.210+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:37:50.567 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:50 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:50.302+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:37:50.729 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:50 smithi089 podman[67982]: 2024-09-21 00:37:50.455937321 +0000 UTC m=+0.360927873 container init 31d14ea421881c5f21b1090f380c1ef2ebff60180f2afa356c156ebcbdf5961c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T00:37:50.729 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:50 smithi089 podman[67982]: 2024-09-21 00:37:50.459616871 +0000 UTC m=+0.364607412 container start 31d14ea421881c5f21b1090f380c1ef2ebff60180f2afa356c156ebcbdf5961c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T00:37:50.729 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:50 smithi089 podman[67982]: 2024-09-21 00:37:50.539034441 +0000 UTC m=+0.444024993 container attach 31d14ea421881c5f21b1090f380c1ef2ebff60180f2afa356c156ebcbdf5961c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T00:37:50.729 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 00:37:50 smithi089 podman[67982]: 2024-09-21 00:37:50.653841428 +0000 UTC m=+0.558831983 container died 31d14ea421881c5f21b1090f380c1ef2ebff60180f2afa356c156ebcbdf5961c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T00:37:50.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:50 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:50.561+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:50.730 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:50 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:50.561+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:37:50.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:50 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:50.662+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:50.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:50 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:50.662+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.235461+0000 front 2024-09-21T00:37:21.235500+0000 (oldest deadline 2024-09-21T00:37:44.135354+0000) 2024-09-21T00:37:51.066 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:50 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:50.652+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:51.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:50 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:50.652+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:37:51.310 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:51 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:51.077+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:37:51.337 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T00:37:51.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:37:51.377 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T00:37:51.377 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T00:37:51.377 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.5 2024-09-21T00:37:51.405 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:51 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:51.153+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:51.406 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:51 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:51.153+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:37:51.406 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:51 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:51.241+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:51.406 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:51 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:51.241+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:37:51.656 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:51 smithi098 systemd[1]: Stopping Ceph osd.5 for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:37:51.656 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:51 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:51.654+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:51.656 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:51 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:51.654+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.235461+0000 front 2024-09-21T00:37:21.235500+0000 (oldest deadline 2024-09-21T00:37:44.135354+0000) 2024-09-21T00:37:51.685 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:51 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:51.325+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:37:51.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:51 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:51.553+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:51.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:51 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:51.553+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:37:51.995 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:51 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:51.682+0000 7f937aad3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:37:51.995 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:51 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:51.682+0000 7f937aad3640 -1 osd.5 117 *** Got signal Terminated *** 2024-09-21T00:37:51.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:51 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:51.682+0000 7f937aad3640 -1 osd.5 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T00:37:52.066 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:51 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:51.684+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:52.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:51 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:51.684+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:37:52.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:52 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:52.110+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:37:52.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:52 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:52.166+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:52.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:52 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:52.166+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:37:52.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:52 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:52.265+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:52.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:52 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:52.265+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:37:52.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:52 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:52.265+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:37:52.683 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:52 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:52.338+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:37:52.982 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:52 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:52.539+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:52.982 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:52 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:52.539+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:37:52.982 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:52 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:52.539+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:37:52.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:52 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:52.675+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:52.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:52 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:52.675+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.235461+0000 front 2024-09-21T00:37:21.235500+0000 (oldest deadline 2024-09-21T00:37:44.135354+0000) 2024-09-21T00:37:53.066 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:52 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:52.681+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:53.066 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:52 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:52.681+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:37:53.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:53 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:53.113+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:37:53.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:53 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:53.113+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:37:53.495 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:53 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:53.231+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:53.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:53 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:53.231+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:37:53.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:53 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:53.231+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:37:53.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:53 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:53.210+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:53.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:53 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:53.210+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:37:53.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:53 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:53.210+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:37:53.668 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:53.359+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:37:53.742 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:53 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:53.532+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:53.742 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:53 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:53.532+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:37:53.743 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:53 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:53.532+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:37:53.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:53 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:53.636+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:53.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:53 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:53.636+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.235461+0000 front 2024-09-21T00:37:21.235500+0000 (oldest deadline 2024-09-21T00:37:44.135354+0000) 2024-09-21T00:37:53.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:53 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:53.636+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.836129+0000 front 2024-09-21T00:37:28.836176+0000 (oldest deadline 2024-09-21T00:37:52.936144+0000) 2024-09-21T00:37:54.039 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:53.666+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:54.039 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:53 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:53.666+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:37:54.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:54 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:54.154+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:37:54.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:54 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:54.154+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:37:54.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:54 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:54.206+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:54.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:54 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:54.206+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:37:54.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:54 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:54.206+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:37:54.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:54 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:54.248+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:54.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:54 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:54.248+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:37:54.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:54 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:54.248+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:37:54.627 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:54 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:54.346+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:37:54.627 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:54 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:54.346+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.852840+0000 front 2024-09-21T00:37:28.852819+0000 (oldest deadline 2024-09-21T00:37:54.152865+0000) 2024-09-21T00:37:54.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:54 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:54.503+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:54.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:54 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:54.503+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:37:54.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:54 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:54.503+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:37:54.995 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:54 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:54.672+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:54.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:54 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:54.672+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.235461+0000 front 2024-09-21T00:37:21.235500+0000 (oldest deadline 2024-09-21T00:37:44.135354+0000) 2024-09-21T00:37:54.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:54 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:54.672+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.836129+0000 front 2024-09-21T00:37:28.836176+0000 (oldest deadline 2024-09-21T00:37:52.936144+0000) 2024-09-21T00:37:55.066 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:54 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:54.625+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:55.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:54 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:54.625+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:37:55.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:54 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:54.625+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:37:55.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:55 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:55.135+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:37:55.479 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:55 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:55.135+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:37:55.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:55 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:55.243+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:55.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:55 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:55.243+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:37:55.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:55 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:55.243+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:37:55.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:55 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:55.239+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:55.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:55 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:55.239+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:37:55.497 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:55 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:55.239+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:37:55.658 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:55 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:55.395+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:37:55.658 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:55 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:55.395+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.852840+0000 front 2024-09-21T00:37:28.852819+0000 (oldest deadline 2024-09-21T00:37:54.152865+0000) 2024-09-21T00:37:55.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:55 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:55.548+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:55.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:55 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:55.548+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:37:55.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:55 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:55.548+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:37:55.995 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:55 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:55.669+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:55.995 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:55 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:55.669+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.235461+0000 front 2024-09-21T00:37:21.235500+0000 (oldest deadline 2024-09-21T00:37:44.135354+0000) 2024-09-21T00:37:55.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:55 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:55.669+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.836129+0000 front 2024-09-21T00:37:28.836176+0000 (oldest deadline 2024-09-21T00:37:52.936144+0000) 2024-09-21T00:37:56.066 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:55 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:55.656+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:56.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:55 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:55.656+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:37:56.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:55 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:55.656+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:37:56.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:56 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:56.096+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:37:56.479 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:56 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:56.096+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:37:56.656 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:56 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:56.274+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:56.656 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:56 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:56.274+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:37:56.656 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:56 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:56.274+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:37:56.656 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:56 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:56.245+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:56.656 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:56 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:56.245+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:37:56.656 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:56 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:56.245+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:37:56.817 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:56 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:56.422+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:37:56.817 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:56 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:56.422+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.852840+0000 front 2024-09-21T00:37:28.852819+0000 (oldest deadline 2024-09-21T00:37:54.152865+0000) 2024-09-21T00:37:56.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:56 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:56.647+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:56.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:56 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:56.647+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:37:56.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:56 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:56.647+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:37:56.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:56 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:56.553+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:56.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:56 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:56.553+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:37:56.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:56 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:56.553+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:37:56.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:56 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:56.654+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:13.733958+0000 front 2024-09-21T00:37:13.734023+0000 (oldest deadline 2024-09-21T00:37:39.034072+0000) 2024-09-21T00:37:56.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:56 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:56.654+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.235461+0000 front 2024-09-21T00:37:21.235500+0000 (oldest deadline 2024-09-21T00:37:44.135354+0000) 2024-09-21T00:37:56.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:56 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5[46466]: 2024-09-21T00:37:56.654+0000 7f93770be640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.836129+0000 front 2024-09-21T00:37:28.836176+0000 (oldest deadline 2024-09-21T00:37:52.936144+0000) 2024-09-21T00:37:57.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:57 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:57.093+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:37:57.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:57 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:57.093+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:37:57.495 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:57 smithi098 podman[69271]: 2024-09-21 00:37:57.380966219 +0000 UTC m=+5.816770027 container died b2e4837abc90a222f72b3a932ce0a90c2fb79103455c86353903b24fb24b02f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T00:37:57.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:57 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:57.303+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:57.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:57 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:57.303+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:37:57.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:57 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:57.303+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:37:57.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:57 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:57.199+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:57.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:57 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:57.199+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:37:57.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:57 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:57.199+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:37:57.511 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:57 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:57.509+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:57.511 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:57 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:57.509+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:37:57.817 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:57 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:57.430+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:37:57.817 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:57 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:57.430+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.852840+0000 front 2024-09-21T00:37:28.852819+0000 (oldest deadline 2024-09-21T00:37:54.152865+0000) 2024-09-21T00:37:57.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:57 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:57.635+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:57.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:57 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:57.635+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:37:57.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:57 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:57.635+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:37:57.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:57 smithi098 podman[69271]: 2024-09-21 00:37:57.783808949 +0000 UTC m=+6.219612756 container cleanup b2e4837abc90a222f72b3a932ce0a90c2fb79103455c86353903b24fb24b02f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T00:37:57.996 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:57 smithi098 bash[69271]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5 2024-09-21T00:37:58.478 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:57 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:57.509+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:37:58.479 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:58 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:58.121+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:37:58.479 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:58 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:58.121+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:37:58.495 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:58 smithi098 podman[69286]: 2024-09-21 00:37:58.144159845 +0000 UTC m=+0.765419507 container remove b2e4837abc90a222f72b3a932ce0a90c2fb79103455c86353903b24fb24b02f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:37:58.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:58 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:58.320+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:58.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:58 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:58.320+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:37:58.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:58 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:58.320+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:37:58.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:58 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:58.171+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:58.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:58 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:58.171+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:37:58.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:58 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:58.171+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:37:58.817 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:58 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:58.385+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:37:58.817 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:58 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:58.385+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.852840+0000 front 2024-09-21T00:37:28.852819+0000 (oldest deadline 2024-09-21T00:37:54.152865+0000) 2024-09-21T00:37:58.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:58 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:58.606+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:58.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:58 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:58.606+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:37:58.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:58 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:58.606+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:37:58.857 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:58 smithi098 podman[69375]: 2024-09-21 00:37:58.458353688 +0000 UTC m=+0.015498773 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:37:58.857 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:58 smithi098 podman[69375]: 2024-09-21 00:37:58.595317527 +0000 UTC m=+0.152462593 container create 8d3c4ffe084bf26522f40fc4db5db97e952909533915eaad259a5498223f14ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T00:37:58.857 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:58 smithi098 podman[69375]: 2024-09-21 00:37:58.856617176 +0000 UTC m=+0.413762251 container init 8d3c4ffe084bf26522f40fc4db5db97e952909533915eaad259a5498223f14ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T00:37:58.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:58 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:58.481+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:58.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:58 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:58.481+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:37:58.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:58 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:58.481+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:37:59.171 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:58 smithi098 podman[69375]: 2024-09-21 00:37:58.859833964 +0000 UTC m=+0.416979030 container start 8d3c4ffe084bf26522f40fc4db5db97e952909533915eaad259a5498223f14ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T00:37:59.171 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:58 smithi098 podman[69375]: 2024-09-21 00:37:58.948196249 +0000 UTC m=+0.505341336 container attach 8d3c4ffe084bf26522f40fc4db5db97e952909533915eaad259a5498223f14ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T00:37:59.171 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 00:37:59 smithi098 podman[69375]: 2024-09-21 00:37:59.050470332 +0000 UTC m=+0.607615399 container died 8d3c4ffe084bf26522f40fc4db5db97e952909533915eaad259a5498223f14ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T00:37:59.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:59 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:59.147+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:37:59.479 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:37:59 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:37:59.147+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:37:59.495 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:59 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:59.312+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:37:59.495 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:59 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:59.312+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:37:59.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:37:59 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:37:59.312+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:37:59.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:59 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:59.170+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:37:59.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:59 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:59.170+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:37:59.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:37:59 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:37:59.170+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:37:59.752 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T00:37:59.816 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:59 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:59.401+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:37:59.817 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:59 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:37:59.401+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.852840+0000 front 2024-09-21T00:37:28.852819+0000 (oldest deadline 2024-09-21T00:37:54.152865+0000) 2024-09-21T00:37:59.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:59 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:59.560+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:37:59.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:59 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:59.560+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:37:59.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:37:59 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:37:59.560+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:37:59.831 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:37:59.831 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T00:37:59.831 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T00:37:59.831 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.6 2024-09-21T00:37:59.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:59 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:59.504+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:37:59.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:59 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:59.504+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:37:59.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:37:59 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:37:59.504+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:00.264 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:37:59 smithi027 systemd[1]: Stopping Ceph osd.6 for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:38:00.265 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:38:00.262+0000 7fb7cec00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:38:00.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:00 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:00.186+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:38:00.479 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:00 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:00.186+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:38:00.495 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:00 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:00.321+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:00.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:00 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:00.321+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:00.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:00 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:00.321+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:00.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:00 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:00.171+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:00.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:00 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:00.171+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:00.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:00 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:00.171+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:00.521 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:38:00.262+0000 7fb7cec00640 -1 osd.6 117 *** Got signal Terminated *** 2024-09-21T00:38:00.521 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:38:00.262+0000 7fb7cec00640 -1 osd.6 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T00:38:00.521 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:38:00.412+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:38:00.521 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:38:00.412+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.852840+0000 front 2024-09-21T00:37:28.852819+0000 (oldest deadline 2024-09-21T00:37:54.152865+0000) 2024-09-21T00:38:00.816 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:00.518+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:00.816 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:00.518+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:00.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:00 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:00.518+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:00.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:00 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:00.530+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:00.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:00 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:00.530+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:00.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:00 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:00.530+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:01.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:01 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:01.222+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:38:01.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:01 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:01.222+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:38:01.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:01 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:01.312+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:01.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:01 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:01.312+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:01.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:01 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:01.312+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:01.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:01 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:01.160+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:01.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:01 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:01.160+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:01.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:01 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:01.160+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:01.816 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:01 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:38:01.379+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:38:01.817 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:01 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:38:01.379+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.852840+0000 front 2024-09-21T00:37:28.852819+0000 (oldest deadline 2024-09-21T00:37:54.152865+0000) 2024-09-21T00:38:01.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:01 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:01.482+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:01.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:01 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:01.482+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:01.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:01 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:01.482+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:01.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:01 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:01.492+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:01.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:01 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:01.492+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:01.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:01 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:01.492+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:02.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:02 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:02.172+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:02.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:02 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:02.172+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:02.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:02 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:02.172+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:02.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:02 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:02.317+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:02.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:02 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:02.317+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:02.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:02 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:02.317+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:02.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:02 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:02.317+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:02.728 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:02 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:02.257+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:38:02.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:02 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:02.257+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:38:02.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:02 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:02.497+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:02.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:02 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:02.497+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:02.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:02 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:02.497+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:02.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:02 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:02.497+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:02.817 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:02 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:38:02.341+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:38:02.817 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:02 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:38:02.341+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.852840+0000 front 2024-09-21T00:37:28.852819+0000 (oldest deadline 2024-09-21T00:37:54.152865+0000) 2024-09-21T00:38:02.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:02 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:02.463+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:02.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:02 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:02.463+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:02.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:02 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:02.463+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:03.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:03 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:03.189+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:03.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:03 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:03.189+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:03.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:03 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:03.189+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:03.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:03 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:03.313+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:03.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:03 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:03.313+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:03.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:03 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:03.313+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:03.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:03 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:03.313+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:03.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:03 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:03.288+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:38:03.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:03 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:03.288+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:38:03.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:03 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:03.288+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:39.155970+0000 front 2024-09-21T00:37:39.155917+0000 (oldest deadline 2024-09-21T00:38:03.156181+0000) 2024-09-21T00:38:03.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:03 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:03.497+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:03.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:03 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:03.497+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:03.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:03 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:03.497+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:03.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:03 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:03.497+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:03.816 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:03 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:38:03.351+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:38:03.817 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:03 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:38:03.351+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.852840+0000 front 2024-09-21T00:37:28.852819+0000 (oldest deadline 2024-09-21T00:37:54.152865+0000) 2024-09-21T00:38:03.817 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:03 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:38:03.351+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.653826+0000 front 2024-09-21T00:37:37.653845+0000 (oldest deadline 2024-09-21T00:38:02.953809+0000) 2024-09-21T00:38:03.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:03 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:03.486+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:03.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:03 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:03.486+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:03.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:03 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:03.486+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:04.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:04 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:04.330+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:38:04.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:04 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:04.330+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:38:04.479 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:04 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:04.330+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:39.155970+0000 front 2024-09-21T00:37:39.155917+0000 (oldest deadline 2024-09-21T00:38:03.156181+0000) 2024-09-21T00:38:04.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:04 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:04.155+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:04.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:04 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:04.155+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:04.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:04 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:04.155+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:04.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:04 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:04.266+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:04.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:04 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:04.266+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:04.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:04 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:04.266+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:04.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:04 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:04.266+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:04.817 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:04 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:38:04.400+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.451012+0000 front 2024-09-21T00:37:22.451050+0000 (oldest deadline 2024-09-21T00:37:47.751307+0000) 2024-09-21T00:38:04.817 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:04 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:38:04.400+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.852840+0000 front 2024-09-21T00:37:28.852819+0000 (oldest deadline 2024-09-21T00:37:54.152865+0000) 2024-09-21T00:38:04.817 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:04 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6[57861]: 2024-09-21T00:38:04.400+0000 7fb7c5600640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.653826+0000 front 2024-09-21T00:37:37.653845+0000 (oldest deadline 2024-09-21T00:38:02.953809+0000) 2024-09-21T00:38:04.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:04 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:04.515+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:04.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:04 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:04.515+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:04.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:04 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:04.515+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:04.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:04 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:04.548+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:04.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:04 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:04.548+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:04.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:04 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:04.548+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:04.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:04 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:04.548+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:05.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:05 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:05.118+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:05.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:05 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:05.118+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:05.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:05 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:05.118+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:05.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:05 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:05.250+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:05.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:05 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:05.250+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:05.497 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:05 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:05.250+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:05.497 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:05 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:05.250+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:05.732 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:05 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:05.372+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:38:05.732 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:05 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:05.372+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:38:05.732 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:05 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:05.372+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:39.155970+0000 front 2024-09-21T00:37:39.155917+0000 (oldest deadline 2024-09-21T00:38:03.156181+0000) 2024-09-21T00:38:05.733 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:05 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:05.591+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:05.733 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:05 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:05.591+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:05.733 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:05 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:05.591+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:05.733 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:05 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:05.591+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:05.818 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:05 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:05.545+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:05.818 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:05 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:05.545+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:05.818 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:05 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:05.545+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:05.818 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:05 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:05.545+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:06.317 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:06 smithi027 podman[99307]: 2024-09-21 00:38:06.03236211 +0000 UTC m=+5.908124613 container died 41b19b872c1f9b7c114964aca94c17a6cba9e0bb493096b929903666d805ee9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T00:38:06.495 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:06 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:06.087+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:06.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:06 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:06.087+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:06.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:06 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:06.087+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:06.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:06 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:06.218+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:06.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:06 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:06.218+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:06.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:06 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:06.218+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:06.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:06 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:06.218+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:06.720 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:06 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:06.519+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:06.720 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:06 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:06.519+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:06.720 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:06 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:06.519+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:06.721 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:06 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:06.519+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:06.721 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:06 smithi027 podman[99307]: 2024-09-21 00:38:06.460236208 +0000 UTC m=+6.335998709 container cleanup 41b19b872c1f9b7c114964aca94c17a6cba9e0bb493096b929903666d805ee9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T00:38:06.721 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:06 smithi027 bash[99307]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6 2024-09-21T00:38:06.728 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:06 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:06.336+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:38:06.728 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:06 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:06.336+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:38:06.728 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:06 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:06.336+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:39.155970+0000 front 2024-09-21T00:37:39.155917+0000 (oldest deadline 2024-09-21T00:38:03.156181+0000) 2024-09-21T00:38:06.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:06 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:06.585+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:06.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:06 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:06.585+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:06.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:06 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:06.585+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:06.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:06 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:06.585+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:07.067 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:06 smithi027 podman[99325]: 2024-09-21 00:38:06.720006064 +0000 UTC m=+0.675535982 container remove 41b19b872c1f9b7c114964aca94c17a6cba9e0bb493096b929903666d805ee9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:38:07.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:07 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:07.053+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:07.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:07 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:07.053+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:07.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:07 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:07.053+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:07.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:07 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:07.212+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:07.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:07 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:07.212+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:07.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:07 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:07.212+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:07.497 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:07 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:07.212+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:07.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:07 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:07.500+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:07.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:07 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:07.500+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:07.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:07 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:07.500+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:07.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:07 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:07.500+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:07.567 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:07 smithi027 podman[99413]: 2024-09-21 00:38:07.205455285 +0000 UTC m=+0.028171751 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:38:07.567 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:07 smithi027 podman[99413]: 2024-09-21 00:38:07.321281556 +0000 UTC m=+0.143998039 container create da05f6e0d3c6f4967aae9350d061cabba33ad4d27ea8e21bf81f38ed45aee910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T00:38:07.567 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:07 smithi027 podman[99413]: 2024-09-21 00:38:07.532269637 +0000 UTC m=+0.354986102 container init da05f6e0d3c6f4967aae9350d061cabba33ad4d27ea8e21bf81f38ed45aee910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T00:38:07.567 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:07 smithi027 podman[99413]: 2024-09-21 00:38:07.539217326 +0000 UTC m=+0.361933770 container start da05f6e0d3c6f4967aae9350d061cabba33ad4d27ea8e21bf81f38ed45aee910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T00:38:07.728 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:07 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:07.362+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:38:07.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:07 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:07.362+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:38:07.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:07 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:07.362+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:39.155970+0000 front 2024-09-21T00:37:39.155917+0000 (oldest deadline 2024-09-21T00:38:03.156181+0000) 2024-09-21T00:38:07.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:07 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:07.588+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:07.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:07 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:07.588+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:07.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:07 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:07.588+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:07.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:07 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:07.588+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:08.067 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:07 smithi027 podman[99413]: 2024-09-21 00:38:07.631973668 +0000 UTC m=+0.454690138 container attach da05f6e0d3c6f4967aae9350d061cabba33ad4d27ea8e21bf81f38ed45aee910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T00:38:08.067 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 00:38:07 smithi027 podman[99413]: 2024-09-21 00:38:07.81086729 +0000 UTC m=+0.633583733 container died da05f6e0d3c6f4967aae9350d061cabba33ad4d27ea8e21bf81f38ed45aee910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T00:38:08.397 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:08 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:08.176+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:08.397 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:08 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:08.176+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:08.397 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:08 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:08.176+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:08.397 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:08 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:08.176+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:08.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:08 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:08.029+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:08.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:08 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:08.029+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:08.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:08 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:08.029+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:08.507 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T00:38:08.585 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:38:08.585 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T00:38:08.585 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T00:38:08.585 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.7 2024-09-21T00:38:08.612 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:08 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:08.356+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:38:08.612 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:08 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:08.356+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:38:08.612 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:08 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:08.356+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:39.155970+0000 front 2024-09-21T00:37:39.155917+0000 (oldest deadline 2024-09-21T00:38:03.156181+0000) 2024-09-21T00:38:08.612 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:08 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:08.554+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:08.613 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:08 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:08.554+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:08.613 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:08 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:08.554+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:08.613 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:08 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:08.554+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:08.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:08 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:08.457+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:08.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:08 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:08.457+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:08.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:08 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:08.457+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:08.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:08 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:08.457+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:08.905 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:08 smithi089 systemd[1]: Stopping Ceph osd.7 for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:38:09.228 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:08 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:08.903+0000 7ff9fc142640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:38:09.228 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:08 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:08.903+0000 7ff9fc142640 -1 osd.7 117 *** Got signal Terminated *** 2024-09-21T00:38:09.228 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:08 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:08.903+0000 7ff9fc142640 -1 osd.7 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T00:38:09.413 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:09 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:09.006+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:09.414 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:09 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:09.006+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:09.414 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:09 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:09.006+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:09.414 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:09 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:09.138+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:09.414 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:09 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:09.138+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:09.414 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:09 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:09.138+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:09.414 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:09 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:09.138+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:09.728 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:09 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:09.330+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:38:09.728 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:09 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:09.330+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:38:09.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:09 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:09.330+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:39.155970+0000 front 2024-09-21T00:37:39.155917+0000 (oldest deadline 2024-09-21T00:38:03.156181+0000) 2024-09-21T00:38:09.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:09 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:09.526+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:09.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:09 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:09.526+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:09.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:09 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:09.526+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:09.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:09 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:09.526+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:09.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:09 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:09.456+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:09.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:09 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:09.456+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:09.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:09 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:09.456+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:09.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:09 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:09.456+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:10.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:10 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:10.028+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:10.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:10 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:10.028+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:10.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:10 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:10.028+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:10.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:10 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:10.160+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:10.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:10 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:10.160+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:10.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:10 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:10.160+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:10.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:10 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:10.160+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:10.718 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:10 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:10.412+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:10.718 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:10 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:10.412+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:10.719 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:10 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:10.412+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:10.719 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:10 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:10.412+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:10.719 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:10 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:10.412+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:10.728 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:10 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:10.302+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:38:10.728 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:10 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:10.302+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:38:10.728 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:10 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:10.302+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:39.155970+0000 front 2024-09-21T00:37:39.155917+0000 (oldest deadline 2024-09-21T00:38:03.156181+0000) 2024-09-21T00:38:10.728 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:10 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:10.302+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:44.857240+0000 front 2024-09-21T00:37:44.857186+0000 (oldest deadline 2024-09-21T00:38:09.557328+0000) 2024-09-21T00:38:10.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:10 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:10.513+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:10.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:10 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:10.513+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:10.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:10 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:10.513+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:10.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:10 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:10.513+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:11.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:11 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:11.030+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:11.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:11 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:11.030+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:11.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:11 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:11.030+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:11.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:11 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:11.200+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:11.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:11 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:11.200+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:11.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:11 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:11.200+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:11.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:11 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:11.200+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:11.728 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:11 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:11.346+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:38:11.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:11 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:11.346+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:38:11.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:11 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:11.346+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:39.155970+0000 front 2024-09-21T00:37:39.155917+0000 (oldest deadline 2024-09-21T00:38:03.156181+0000) 2024-09-21T00:38:11.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:11 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:11.346+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:44.857240+0000 front 2024-09-21T00:37:44.857186+0000 (oldest deadline 2024-09-21T00:38:09.557328+0000) 2024-09-21T00:38:11.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:11 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:11.549+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:11.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:11 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:11.549+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:11.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:11 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:11.549+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:11.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:11 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:11.549+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:11.816 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:11 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:11.380+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:11.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:11 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:11.380+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:11.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:11 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:11.380+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:11.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:11 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:11.380+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:11.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:11 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:11.380+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:12.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:12 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:12.010+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:12.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:12 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:12.010+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:12.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:12 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:12.010+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:12.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:12 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:12.221+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:12.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:12 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:12.221+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:12.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:12 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:12.221+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:12.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:12 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:12.221+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:12.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:12 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:12.371+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:12.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:12 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:12.371+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:12.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:12 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:12.371+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:12.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:12 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:12.371+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:12.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:12 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:12.371+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:12.728 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:12 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:12.340+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:38:12.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:12 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:12.340+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:38:12.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:12 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:12.340+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:39.155970+0000 front 2024-09-21T00:37:39.155917+0000 (oldest deadline 2024-09-21T00:38:03.156181+0000) 2024-09-21T00:38:12.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:12 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:12.340+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:44.857240+0000 front 2024-09-21T00:37:44.857186+0000 (oldest deadline 2024-09-21T00:38:09.557328+0000) 2024-09-21T00:38:12.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:12 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:12.504+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:12.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:12 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:12.504+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:12.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:12 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:12.504+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:12.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:12 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:12.504+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:12.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:12 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:12.504+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:13.246 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:13 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:13.235+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:13.246 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:13 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:13.235+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:13.246 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:13 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:13.235+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:13.246 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:13 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:13.235+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:13.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:12 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:12.988+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:13.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:12 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:12.988+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:13.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:12 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:12.988+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:13.728 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:13 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:13.328+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.752048+0000 front 2024-09-21T00:37:21.752082+0000 (oldest deadline 2024-09-21T00:37:47.652298+0000) 2024-09-21T00:38:13.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:13 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:13.328+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.953587+0000 front 2024-09-21T00:37:29.953753+0000 (oldest deadline 2024-09-21T00:37:52.754076+0000) 2024-09-21T00:38:13.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:13 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:13.328+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:39.155970+0000 front 2024-09-21T00:37:39.155917+0000 (oldest deadline 2024-09-21T00:38:03.156181+0000) 2024-09-21T00:38:13.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:13 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7[51554]: 2024-09-21T00:38:13.328+0000 7ff9f872d640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:44.857240+0000 front 2024-09-21T00:37:44.857186+0000 (oldest deadline 2024-09-21T00:38:09.557328+0000) 2024-09-21T00:38:13.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:13 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:13.456+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:13.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:13 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:13.456+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:13.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:13 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:13.456+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:13.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:13 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:13.456+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:13.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:13 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:13.456+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:13.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:13 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:13.351+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:13.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:13 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:13.351+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:13.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:13 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:13.351+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:13.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:13 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:13.351+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:13.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:13 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:13.351+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:14.254 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:14 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:14.003+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:14.254 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:14 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:14.003+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:14.254 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:14 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:14.003+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:14.254 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:14 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:14.003+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:14.728 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:14 smithi089 podman[68097]: 2024-09-21 00:38:14.531708418 +0000 UTC m=+5.756054483 container died fc0e65b168e006c656f865b7aec87f0d795efc51b987819a1a1af2ae0a370ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T00:38:14.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:14 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:14.457+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:14.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:14 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:14.457+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:14.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:14 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:14.457+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:14.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:14 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:14.457+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:14.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:14 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:14.457+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:14.745 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:14 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:14.253+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:14.745 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:14 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:14.253+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:14.745 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:14 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:14.253+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:14.746 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:14 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:14.253+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:14.816 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:14 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:14.337+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:14.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:14 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:14.337+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:14.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:14 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:14.337+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:14.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:14 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:14.337+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:14.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:14 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:14.337+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:15.205 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:14 smithi089 podman[68097]: 2024-09-21 00:38:14.94516961 +0000 UTC m=+6.169515631 container cleanup fc0e65b168e006c656f865b7aec87f0d795efc51b987819a1a1af2ae0a370ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T00:38:15.205 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:14 smithi089 bash[68097]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7 2024-09-21T00:38:15.245 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:14 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:14.957+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:15.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:14 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:14.957+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:15.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:14 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:14.957+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:15.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:14 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:14.957+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:15.478 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:15 smithi089 podman[68112]: 2024-09-21 00:38:15.20521211 +0000 UTC m=+0.665024732 container remove fc0e65b168e006c656f865b7aec87f0d795efc51b987819a1a1af2ae0a370ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T00:38:15.746 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:15 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:15.256+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:15.746 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:15 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:15.256+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:15.746 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:15 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:15.256+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:15.746 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:15 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:15.256+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:15.751 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:15 smithi089 podman[68199]: 2024-09-21 00:38:15.412835377 +0000 UTC m=+0.015724031 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:38:15.751 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:15 smithi089 podman[68199]: 2024-09-21 00:38:15.531084466 +0000 UTC m=+0.133973119 container create 384b39f6d12b2b1d1924899aaf889decd9cd93b8ae5e3fd39b31e65302884a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-09-21T00:38:15.751 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:15 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:15.484+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:15.751 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:15 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:15.484+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:15.751 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:15 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:15.484+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:15.751 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:15 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:15.484+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:15.751 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:15 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:15.484+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:15.816 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:15 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:15.372+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:15.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:15 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:15.372+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:15.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:15 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:15.372+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:15.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:15 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:15.372+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:15.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:15 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:15.372+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:16.228 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:15 smithi089 podman[68199]: 2024-09-21 00:38:15.750296878 +0000 UTC m=+0.353185532 container init 384b39f6d12b2b1d1924899aaf889decd9cd93b8ae5e3fd39b31e65302884a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:38:16.229 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:15 smithi089 podman[68199]: 2024-09-21 00:38:15.753925393 +0000 UTC m=+0.356814047 container start 384b39f6d12b2b1d1924899aaf889decd9cd93b8ae5e3fd39b31e65302884a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T00:38:16.229 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:15 smithi089 podman[68199]: 2024-09-21 00:38:15.866961943 +0000 UTC m=+0.469850599 container attach 384b39f6d12b2b1d1924899aaf889decd9cd93b8ae5e3fd39b31e65302884a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T00:38:16.229 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 00:38:15 smithi089 podman[68199]: 2024-09-21 00:38:15.940949339 +0000 UTC m=+0.543837994 container died 384b39f6d12b2b1d1924899aaf889decd9cd93b8ae5e3fd39b31e65302884a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T00:38:16.246 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:16 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:16.222+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:16.246 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:16 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:16.222+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:16.246 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:16 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:16.222+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:16.246 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:16 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:16.222+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:16.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:15 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:15.976+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:16.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:15 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:15.976+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:16.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:15 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:15.976+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:16.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:15 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:15.976+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:16.621 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:16 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:16.528+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:16.621 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:16 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:16.528+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:16.622 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:16 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:16.528+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:16.622 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:16 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:16.528+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:16.622 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:16 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:16.528+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:16.643 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T00:38:16.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:38:16.723 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T00:38:16.723 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T00:38:16.723 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.8 2024-09-21T00:38:16.816 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:16 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:16.403+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:16.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:16 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:16.403+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:16.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:16 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:16.403+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:16.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:16 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:16.403+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:16.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:16 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:16.403+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:17.002 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:16 smithi098 systemd[1]: Stopping Ceph osd.8 for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:38:17.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:17 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:17.001+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:17.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:17 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:17.001+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:17.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:17 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:17.001+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:17.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:17 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:17.001+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:17.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:17 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:17.030+0000 7ff39bd05640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:38:17.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:17 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:17.030+0000 7ff39bd05640 -1 osd.8 117 *** Got signal Terminated *** 2024-09-21T00:38:17.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:17 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:17.030+0000 7ff39bd05640 -1 osd.8 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T00:38:17.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:17 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:17.202+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:17.497 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:17 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:17.202+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:17.497 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:17 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:17.202+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:17.497 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:17 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:17.202+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:17.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:17 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:17.378+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:17.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:17 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:17.378+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:17.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:17 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:17.378+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:17.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:17 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:17.378+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:17.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:17 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:17.378+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:17.838 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:17 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:17.548+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:17.838 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:17 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:17.548+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:17.838 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:17 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:17.548+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:17.838 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:17 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:17.548+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:17.838 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:17 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:17.548+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:18.495 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:18 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:18.032+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:18.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:18 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:18.032+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:18.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:18 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:18.032+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:18.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:18 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:18.032+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:18.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:18 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:18.165+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:18.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:18 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:18.165+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:18.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:18 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:18.165+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:18.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:18 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:18.165+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:18.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:18.334+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:18.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:18.334+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:18.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:18.334+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:18.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:18.334+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:18.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:18 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:18.334+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:18.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:18 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:18.569+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:18.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:18 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:18.569+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:18.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:18 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:18.569+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:18.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:18 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:18.569+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:18.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:18 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:18.569+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:18.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:18 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:18.569+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:19.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:19 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:19.079+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:19.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:19 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:19.079+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:19.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:19 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:19.079+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:19.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:19 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:19.079+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:19.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:19 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:19.208+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:19.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:19 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:19.208+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:19.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:19 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:19.208+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:19.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:19 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:19.208+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:19.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:19 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:19.208+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:53.020628+0000 front 2024-09-21T00:37:53.020560+0000 (oldest deadline 2024-09-21T00:38:18.320644+0000) 2024-09-21T00:38:19.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:19 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:19.342+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:19.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:19 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:19.342+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:19.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:19 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:19.342+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:19.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:19 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:19.342+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:19.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:19 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:19.342+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:19.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:19 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:19.529+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:19.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:19 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:19.529+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:19.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:19 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:19.529+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:19.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:19 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:19.529+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:19.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:19 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:19.529+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:19.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:19 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:19.529+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:20.495 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:20 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:20.099+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:20.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:20 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:20.099+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:20.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:20 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:20.099+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:20.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:20 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:20.099+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:20.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:20 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:20.217+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:20.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:20 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:20.217+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:20.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:20 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:20.217+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:20.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:20 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:20.217+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:20.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:20 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:20.217+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:53.020628+0000 front 2024-09-21T00:37:53.020560+0000 (oldest deadline 2024-09-21T00:38:18.320644+0000) 2024-09-21T00:38:20.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:20 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:20.340+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:20.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:20 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:20.340+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:20.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:20 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:20.340+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:20.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:20 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:20.340+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:20.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:20 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:20.340+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:20.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:20 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:20.487+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:20.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:20 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:20.487+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:20.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:20 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:20.487+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:20.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:20 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:20.487+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:20.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:20 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:20.487+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:20.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:20 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:20.487+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:21.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:21 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:21.093+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:21.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:21 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:21.093+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:21.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:21 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:21.093+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:21.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:21 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:21.093+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:21.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:21 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:21.216+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.512660+0000 front 2024-09-21T00:37:09.512603+0000 (oldest deadline 2024-09-21T00:37:35.412724+0000) 2024-09-21T00:38:21.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:21 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:21.216+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.413994+0000 front 2024-09-21T00:37:22.414109+0000 (oldest deadline 2024-09-21T00:37:45.913947+0000) 2024-09-21T00:38:21.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:21 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:21.216+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:31.115603+0000 front 2024-09-21T00:37:31.115555+0000 (oldest deadline 2024-09-21T00:37:52.815719+0000) 2024-09-21T00:38:21.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:21 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:21.216+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.417670+0000 front 2024-09-21T00:37:37.417633+0000 (oldest deadline 2024-09-21T00:38:02.117530+0000) 2024-09-21T00:38:21.496 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:21 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8[52771]: 2024-09-21T00:38:21.216+0000 7ff3942e8640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:53.020628+0000 front 2024-09-21T00:37:53.020560+0000 (oldest deadline 2024-09-21T00:38:18.320644+0000) 2024-09-21T00:38:21.750 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:21 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:21.316+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:21.750 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:21 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:21.316+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:21.750 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:21 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:21.316+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:21.750 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:21 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:21.316+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:21.750 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:21 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:21.316+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:21.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:21 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:21.491+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:21.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:21 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:21.491+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:21.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:21 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:21.491+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:21.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:21 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:21.491+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:21.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:21 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:21.491+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:21.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:21 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:21.491+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:22.495 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:22 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:22.101+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:22.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:22 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:22.101+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:22.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:22 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:22.101+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:22.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:22 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:22.101+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:22.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:22 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:22.281+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:22.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:22 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:22.281+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:22.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:22 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:22.281+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:22.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:22 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:22.281+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:22.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:22 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:22.281+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:22.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:22 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:22.281+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:54.432006+0000 front 2024-09-21T00:37:54.431896+0000 (oldest deadline 2024-09-21T00:38:22.032364+0000) 2024-09-21T00:38:22.849 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:22 smithi098 podman[69488]: 2024-09-21 00:38:22.529219188 +0000 UTC m=+5.608514375 container died 5acc01f958b567f8abaee790fccf70aa38520d0b198e0e726f08382169f32786 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T00:38:22.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:22 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:22.531+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:22.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:22 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:22.531+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:22.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:22 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:22.531+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:22.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:22 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:22.531+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:22.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:22 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:22.531+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:22.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:22 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:22.531+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:23.208 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:22 smithi098 podman[69488]: 2024-09-21 00:38:22.939302127 +0000 UTC m=+6.018597304 container cleanup 5acc01f958b567f8abaee790fccf70aa38520d0b198e0e726f08382169f32786 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T00:38:23.208 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:22 smithi098 bash[69488]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8 2024-09-21T00:38:23.208 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:23 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:23.052+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:23.209 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:23 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:23.052+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:23.209 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:23 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:23.052+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:23.209 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:23 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:23.052+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:23.209 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:23 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:23.052+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:23.495 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:23 smithi098 podman[69507]: 2024-09-21 00:38:23.207870184 +0000 UTC m=+0.677475749 container remove 5acc01f958b567f8abaee790fccf70aa38520d0b198e0e726f08382169f32786 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T00:38:23.787 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:23 smithi098 podman[69600]: 2024-09-21 00:38:23.416200703 +0000 UTC m=+0.015752515 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:38:23.787 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:23 smithi098 podman[69600]: 2024-09-21 00:38:23.542039644 +0000 UTC m=+0.141591446 container create 6eb1d60251ca0bbaea3a98b231386411fbc4829786993456454ab104d6f51353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T00:38:23.787 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:23 smithi098 podman[69600]: 2024-09-21 00:38:23.78643123 +0000 UTC m=+0.385983038 container init 6eb1d60251ca0bbaea3a98b231386411fbc4829786993456454ab104d6f51353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True) 2024-09-21T00:38:23.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:23 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:23.321+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:23.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:23 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:23.321+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:23.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:23 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:23.321+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:23.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:23 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:23.321+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:23.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:23 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:23.321+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:23.817 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:23 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:23.321+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:54.432006+0000 front 2024-09-21T00:37:54.431896+0000 (oldest deadline 2024-09-21T00:38:22.032364+0000) 2024-09-21T00:38:23.880 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:23 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:23.523+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:23.880 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:23 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:23.523+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:23.880 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:23 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:23.523+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:23.880 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:23 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:23.523+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:23.880 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:23 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:23.523+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:23.880 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:23 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:23.523+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:24.075 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:23 smithi098 podman[69600]: 2024-09-21 00:38:23.78975877 +0000 UTC m=+0.389310573 container start 6eb1d60251ca0bbaea3a98b231386411fbc4829786993456454ab104d6f51353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T00:38:24.075 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:23 smithi098 podman[69600]: 2024-09-21 00:38:23.869561695 +0000 UTC m=+0.469113514 container attach 6eb1d60251ca0bbaea3a98b231386411fbc4829786993456454ab104d6f51353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:38:24.075 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 00:38:23 smithi098 podman[69600]: 2024-09-21 00:38:23.982591989 +0000 UTC m=+0.582143793 container died 6eb1d60251ca0bbaea3a98b231386411fbc4829786993456454ab104d6f51353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-8-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:38:24.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:24 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:24.073+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:24.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:24 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:24.073+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:24.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:24 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:24.073+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:24.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:24 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:24.073+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:24.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:24 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:24.073+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:24.556 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T00:38:24.595 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:38:24.595 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T00:38:24.596 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T00:38:24.596 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.9 2024-09-21T00:38:24.652 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:24 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:24.323+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:24.652 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:24 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:24.323+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:24.653 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:24 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:24.323+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:24.653 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:24 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:24.323+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:24.653 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:24 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:24.323+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:24.653 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:24 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:24.323+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:54.432006+0000 front 2024-09-21T00:37:54.431896+0000 (oldest deadline 2024-09-21T00:38:22.032364+0000) 2024-09-21T00:38:24.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:24 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:24.525+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:24.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:24 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:24.525+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:24.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:24 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:24.525+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:24.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:24 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:24.525+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:24.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:24 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:24.525+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:24.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:24 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:24.525+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:25.034 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:24 smithi027 systemd[1]: Stopping Ceph osd.9 for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:38:25.034 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:25 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:25.033+0000 7f0dac800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:38:25.293 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:25 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:25.033+0000 7f0dac800640 -1 osd.9 117 *** Got signal Terminated *** 2024-09-21T00:38:25.293 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:25 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:25.033+0000 7f0dac800640 -1 osd.9 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T00:38:25.293 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:25 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:25.292+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:25.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:25 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:25.120+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:25.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:25 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:25.120+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:25.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:25 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:25.120+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:25.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:25 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:25.120+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:25.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:25 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:25.120+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:25.566 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:25 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:25.292+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:25.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:25 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:25.292+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:25.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:25 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:25.292+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:25.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:25 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:25.292+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:25.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:25 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:25.292+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:54.432006+0000 front 2024-09-21T00:37:54.431896+0000 (oldest deadline 2024-09-21T00:38:22.032364+0000) 2024-09-21T00:38:25.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:25 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:25.514+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:25.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:25 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:25.514+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:25.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:25 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:25.514+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:25.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:25 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:25.514+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:25.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:25 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:25.514+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:25.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:25 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:25.514+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:26.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:26 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:26.083+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:26.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:26 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:26.083+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:26.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:26 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:26.083+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:26.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:26 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:26.083+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:26.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:26 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:26.083+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:26.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:26 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:26.272+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:26.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:26 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:26.272+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:26.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:26 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:26.272+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:26.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:26 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:26.272+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:26.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:26 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:26.272+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:26.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:26 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:26.272+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:54.432006+0000 front 2024-09-21T00:37:54.431896+0000 (oldest deadline 2024-09-21T00:38:22.032364+0000) 2024-09-21T00:38:26.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:26 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:26.554+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:26.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:26 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:26.554+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:26.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:26 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:26.554+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:26.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:26 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:26.554+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:26.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:26 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:26.554+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:26.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:26 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:26.554+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:27.495 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:27 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:27.045+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:27.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:27 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:27.045+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:27.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:27 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:27.045+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:27.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:27 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:27.045+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:27.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:27 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:27.045+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:27.566 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:27.252+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:27.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:27.252+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:27.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:27.252+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:27.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:27.252+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:27.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:27.252+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:27.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:27 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:27.252+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:54.432006+0000 front 2024-09-21T00:37:54.431896+0000 (oldest deadline 2024-09-21T00:38:22.032364+0000) 2024-09-21T00:38:27.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:27 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:27.542+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:27.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:27 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:27.542+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:27.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:27 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:27.542+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:27.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:27 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:27.542+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:27.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:27 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:27.542+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:27.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:27 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:27.542+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:28.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:28 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:28.012+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:28.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:28 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:28.012+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:28.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:28 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:28.012+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:28.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:28 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:28.012+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:28.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:28 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:28.012+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:28.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:28 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:28.012+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:28.566 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:28 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:28.284+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:28.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:28 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:28.284+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:28.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:28 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:28.284+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:28.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:28 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:28.284+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:28.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:28 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:28.284+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:28.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:28 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:28.284+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:54.432006+0000 front 2024-09-21T00:37:54.431896+0000 (oldest deadline 2024-09-21T00:38:22.032364+0000) 2024-09-21T00:38:28.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:28 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:28.534+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:28.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:28 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:28.534+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:28.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:28 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:28.534+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:28.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:28 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:28.534+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:28.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:28 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:28.534+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:28.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:28 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:28.534+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:29.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:29 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:29.002+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:29.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:29 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:29.002+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:29.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:29 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:29.002+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:29.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:29 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:29.002+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:29.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:29 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:29.002+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:29.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:29 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:29.002+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:29.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:29 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:29.309+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:12.224840+0000 front 2024-09-21T00:37:12.225044+0000 (oldest deadline 2024-09-21T00:37:36.324867+0000) 2024-09-21T00:38:29.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:29 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:29.309+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.727014+0000 front 2024-09-21T00:37:22.727189+0000 (oldest deadline 2024-09-21T00:37:48.027149+0000) 2024-09-21T00:38:29.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:29 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:29.309+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:28.028115+0000 front 2024-09-21T00:37:28.028081+0000 (oldest deadline 2024-09-21T00:37:53.928006+0000) 2024-09-21T00:38:29.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:29 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:29.309+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:36.229269+0000 front 2024-09-21T00:37:36.229240+0000 (oldest deadline 2024-09-21T00:38:05.029710+0000) 2024-09-21T00:38:29.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:29 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:29.309+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:45.030395+0000 front 2024-09-21T00:37:45.030566+0000 (oldest deadline 2024-09-21T00:38:10.330443+0000) 2024-09-21T00:38:29.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:29 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9[64244]: 2024-09-21T00:38:29.309+0000 7f0da3200640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:54.432006+0000 front 2024-09-21T00:37:54.431896+0000 (oldest deadline 2024-09-21T00:38:22.032364+0000) 2024-09-21T00:38:29.978 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:29 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:29.514+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:29.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:29 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:29.514+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:29.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:29 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:29.514+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:29.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:29 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:29.514+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:29.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:29 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:29.514+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:29.979 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:29 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:29.514+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:30.245 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:29 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:29.992+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:30.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:29 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:29.992+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:30.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:29 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:29.992+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:30.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:29 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:29.992+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:30.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:29 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:29.992+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:30.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:29 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:29.992+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:30.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:30 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:30.474+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:30.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:30 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:30.474+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:30.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:30 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:30.474+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:30.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:30 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:30.474+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:30.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:30 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:30.474+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:30.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:30 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:30.474+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:31.040 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:30 smithi027 podman[99523]: 2024-09-21 00:38:30.715415079 +0000 UTC m=+5.816708730 container died 1865bd23c2c0ebfd8af95f63fd148ecfe0f94e30edf56c80dfeb9dc406152bf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T00:38:31.317 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:31 smithi027 podman[99523]: 2024-09-21 00:38:31.171161973 +0000 UTC m=+6.272455612 container cleanup 1865bd23c2c0ebfd8af95f63fd148ecfe0f94e30edf56c80dfeb9dc406152bf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T00:38:31.317 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:31 smithi027 bash[99523]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9 2024-09-21T00:38:31.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:30 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:30.997+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:31.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:30 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:30.997+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:31.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:30 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:30.997+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:31.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:30 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:30.997+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:31.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:30 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:30.997+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:31.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:30 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:30.997+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:31.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:31 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:31.425+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:31.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:31 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:31.425+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:31.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:31 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:31.425+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:31.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:31 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:31.425+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:31.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:31 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:31.425+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:31.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:31 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:31.425+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:31.730 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:31 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:31.425+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:01.776323+0000 front 2024-09-21T00:38:01.776266+0000 (oldest deadline 2024-09-21T00:38:30.576607+0000) 2024-09-21T00:38:31.816 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:31 smithi027 podman[99542]: 2024-09-21 00:38:31.381333581 +0000 UTC m=+0.653161788 container remove 1865bd23c2c0ebfd8af95f63fd148ecfe0f94e30edf56c80dfeb9dc406152bf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T00:38:32.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:31 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:31.966+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:32.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:31 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:31.966+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:32.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:31 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:31.966+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:32.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:31 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:31.966+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:32.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:31 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:31.966+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:32.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:31 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:31.966+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:32.276 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:31 smithi027 podman[99628]: 2024-09-21 00:38:31.830348365 +0000 UTC m=+0.028563587 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:38:32.277 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:31 smithi027 podman[99628]: 2024-09-21 00:38:31.93221629 +0000 UTC m=+0.130431507 container create b0f93a818d828ebb9328f499c7a2a20addc6461b376a852c96471a1ab72cf8c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:38:32.277 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:32 smithi027 podman[99628]: 2024-09-21 00:38:32.151485453 +0000 UTC m=+0.349700678 container init b0f93a818d828ebb9328f499c7a2a20addc6461b376a852c96471a1ab72cf8c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T00:38:32.277 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:32 smithi027 podman[99628]: 2024-09-21 00:38:32.158435948 +0000 UTC m=+0.356651163 container start b0f93a818d828ebb9328f499c7a2a20addc6461b376a852c96471a1ab72cf8c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T00:38:32.277 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:32 smithi027 podman[99628]: 2024-09-21 00:38:32.2761437 +0000 UTC m=+0.474358934 container attach b0f93a818d828ebb9328f499c7a2a20addc6461b376a852c96471a1ab72cf8c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-09-21T00:38:32.567 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 00:38:32 smithi027 podman[99628]: 2024-09-21 00:38:32.428229565 +0000 UTC m=+0.626444777 container died b0f93a818d828ebb9328f499c7a2a20addc6461b376a852c96471a1ab72cf8c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:38:32.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:32 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:32.425+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:32.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:32 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:32.425+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:32.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:32 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:32.425+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:32.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:32 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:32.425+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:32.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:32 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:32.425+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:32.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:32 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:32.425+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:32.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:32 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:32.425+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:01.776323+0000 front 2024-09-21T00:38:01.776266+0000 (oldest deadline 2024-09-21T00:38:30.576607+0000) 2024-09-21T00:38:33.187 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T00:38:33.264 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:38:33.265 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T00:38:33.265 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T00:38:33.265 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.10 2024-09-21T00:38:33.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:33 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:33.013+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:33.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:33 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:33.013+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:33.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:33 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:33.013+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:33.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:33 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:33.013+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:33.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:33 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:33.013+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:33.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:33 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:33.013+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:33.560 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:33 smithi089 systemd[1]: Stopping Ceph osd.10 for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:38:33.560 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:33 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:33.427+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:33.560 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:33 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:33.427+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:33.561 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:33 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:33.427+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:33.561 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:33 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:33.427+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:33.561 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:33 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:33.427+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:33.561 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:33 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:33.427+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:33.561 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:33 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:33.427+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:01.776323+0000 front 2024-09-21T00:38:01.776266+0000 (oldest deadline 2024-09-21T00:38:30.576607+0000) 2024-09-21T00:38:33.561 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:33 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:33.559+0000 7f97d7854640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:38:33.561 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:33 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:33.559+0000 7f97d7854640 -1 osd.10 117 *** Got signal Terminated *** 2024-09-21T00:38:33.561 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:33 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:33.559+0000 7f97d7854640 -1 osd.10 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T00:38:34.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:33 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:33.996+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:34.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:33 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:33.996+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:34.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:33 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:33.996+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:34.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:33 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:33.996+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:34.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:33 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:33.996+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:34.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:33 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:33.996+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:34.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:34 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:34.394+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:34.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:34 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:34.394+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:34.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:34 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:34.394+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:34.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:34 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:34.394+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:34.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:34 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:34.394+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:34.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:34 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:34.394+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:34.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:34 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:34.394+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:01.776323+0000 front 2024-09-21T00:38:01.776266+0000 (oldest deadline 2024-09-21T00:38:30.576607+0000) 2024-09-21T00:38:35.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:35 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:35.045+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:35.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:35 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:35.045+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:35.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:35 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:35.045+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:35.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:35 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:35.045+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:35.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:35 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:35.045+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:35.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:35 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:35.045+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:35.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:35 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:35.442+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:35.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:35 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:35.442+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:35.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:35 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:35.442+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:35.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:35 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:35.442+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:35.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:35 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:35.442+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:35.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:35 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:35.442+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:35.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:35 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:35.442+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:01.776323+0000 front 2024-09-21T00:38:01.776266+0000 (oldest deadline 2024-09-21T00:38:30.576607+0000) 2024-09-21T00:38:36.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:36 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:36.043+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:36.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:36 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:36.043+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:36.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:36 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:36.043+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:36.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:36 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:36.043+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:36.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:36 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:36.043+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:36.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:36 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:36.043+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:36.246 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:36 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:36.043+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T00:38:11.674750+0000 front 2024-09-21T00:38:11.674912+0000 (oldest deadline 2024-09-21T00:38:35.075083+0000) 2024-09-21T00:38:36.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:36 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:36.404+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:36.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:36 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:36.404+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:36.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:36 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:36.404+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:36.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:36 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:36.404+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:36.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:36 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:36.404+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:36.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:36 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:36.404+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:36.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:36 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:36.404+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:01.776323+0000 front 2024-09-21T00:38:01.776266+0000 (oldest deadline 2024-09-21T00:38:30.576607+0000) 2024-09-21T00:38:37.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:37 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:37.072+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:37.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:37 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:37.072+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:37.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:37 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:37.072+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:37.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:37 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:37.072+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:37.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:37 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:37.072+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:37.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:37 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:37.072+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:37.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:37 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:37.072+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T00:38:11.674750+0000 front 2024-09-21T00:38:11.674912+0000 (oldest deadline 2024-09-21T00:38:35.075083+0000) 2024-09-21T00:38:37.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:37 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:37.440+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:37.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:37 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:37.440+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:37.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:37 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:37.440+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:37.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:37 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:37.440+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:37.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:37 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:37.440+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:37.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:37 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:37.440+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:37.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:37 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:37.440+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:01.776323+0000 front 2024-09-21T00:38:01.776266+0000 (oldest deadline 2024-09-21T00:38:30.576607+0000) 2024-09-21T00:38:38.397 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:38.082+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:38.397 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:38.082+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:38.397 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:38.082+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:38.397 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:38.082+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:38.397 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:38.082+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:38.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:38.082+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:38.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:38 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:38.082+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T00:38:11.674750+0000 front 2024-09-21T00:38:11.674912+0000 (oldest deadline 2024-09-21T00:38:35.075083+0000) 2024-09-21T00:38:38.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:38 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:38.458+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:08.867511+0000 front 2024-09-21T00:37:08.867741+0000 (oldest deadline 2024-09-21T00:37:37.668229+0000) 2024-09-21T00:38:38.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:38 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:38.458+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:22.368983+0000 front 2024-09-21T00:37:22.369114+0000 (oldest deadline 2024-09-21T00:37:47.069062+0000) 2024-09-21T00:38:38.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:38 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:38.458+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:29.369936+0000 front 2024-09-21T00:37:29.370038+0000 (oldest deadline 2024-09-21T00:37:52.269929+0000) 2024-09-21T00:38:38.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:38 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:38.458+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T00:37:37.972234+0000 front 2024-09-21T00:37:37.972292+0000 (oldest deadline 2024-09-21T00:38:02.072318+0000) 2024-09-21T00:38:38.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:38 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:38.458+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:46.574074+0000 front 2024-09-21T00:37:46.574127+0000 (oldest deadline 2024-09-21T00:38:12.474175+0000) 2024-09-21T00:38:38.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:38 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:38.458+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:55.375189+0000 front 2024-09-21T00:37:55.375080+0000 (oldest deadline 2024-09-21T00:38:18.275116+0000) 2024-09-21T00:38:38.729 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:38 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10[58230]: 2024-09-21T00:38:38.458+0000 7f97cfe37640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:01.776323+0000 front 2024-09-21T00:38:01.776266+0000 (oldest deadline 2024-09-21T00:38:30.576607+0000) 2024-09-21T00:38:39.390 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:39 smithi089 podman[68316]: 2024-09-21 00:38:39.135239637 +0000 UTC m=+5.675529045 container died 456854bb1edd4b8a800715a4da53ad715cc343e7b186f5c341a30935c3487241 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T00:38:39.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:39 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:39.082+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:39.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:39 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:39.082+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:39.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:39 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:39.082+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:39.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:39 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:39.082+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:39.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:39 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:39.082+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:39.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:39 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:39.082+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:39.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:39 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:39.082+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T00:38:11.674750+0000 front 2024-09-21T00:38:11.674912+0000 (oldest deadline 2024-09-21T00:38:35.075083+0000) 2024-09-21T00:38:39.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:39 smithi089 podman[68316]: 2024-09-21 00:38:39.48957287 +0000 UTC m=+6.029862278 container cleanup 456854bb1edd4b8a800715a4da53ad715cc343e7b186f5c341a30935c3487241 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:38:39.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:39 smithi089 bash[68316]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10 2024-09-21T00:38:40.100 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:39 smithi089 podman[68331]: 2024-09-21 00:38:39.774270586 +0000 UTC m=+0.640723794 container remove 456854bb1edd4b8a800715a4da53ad715cc343e7b186f5c341a30935c3487241 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T00:38:40.403 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:40 smithi089 podman[68422]: 2024-09-21 00:38:40.100003882 +0000 UTC m=+0.108133912 container create 967f32eecd6c505356aa574e8058e16ee20b42c0bf7e5b500899ff86efd8ea11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T00:38:40.403 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:40 smithi089 podman[68422]: 2024-09-21 00:38:40.007565791 +0000 UTC m=+0.015695789 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:38:40.403 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:40 smithi089 podman[68422]: 2024-09-21 00:38:40.318951962 +0000 UTC m=+0.327081950 container init 967f32eecd6c505356aa574e8058e16ee20b42c0bf7e5b500899ff86efd8ea11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T00:38:40.403 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:40 smithi089 podman[68422]: 2024-09-21 00:38:40.322228965 +0000 UTC m=+0.330358956 container start 967f32eecd6c505356aa574e8058e16ee20b42c0bf7e5b500899ff86efd8ea11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T00:38:40.403 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:40 smithi089 podman[68422]: 2024-09-21 00:38:40.402247799 +0000 UTC m=+0.410377789 container attach 967f32eecd6c505356aa574e8058e16ee20b42c0bf7e5b500899ff86efd8ea11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:38:40.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:40 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:40.104+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:40.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:40 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:40.104+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:40.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:40 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:40.104+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:40.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:40 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:40.104+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:40.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:40 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:40.104+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:40.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:40 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:40.104+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:40.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:40 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:40.104+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T00:38:11.674750+0000 front 2024-09-21T00:38:11.674912+0000 (oldest deadline 2024-09-21T00:38:35.075083+0000) 2024-09-21T00:38:40.728 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 00:38:40 smithi089 podman[68422]: 2024-09-21 00:38:40.509365069 +0000 UTC m=+0.517495063 container died 967f32eecd6c505356aa574e8058e16ee20b42c0bf7e5b500899ff86efd8ea11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T00:38:41.069 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T00:38:41.107 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:38:41.108 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T00:38:41.108 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T00:38:41.108 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.11 2024-09-21T00:38:41.445 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:41 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:41.091+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:41.445 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:41 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:41.091+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:41.445 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:41 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:41.091+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:41.445 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:41 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:41.091+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:41.445 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:41 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:41.091+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:41.445 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:41 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:41.091+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:41.445 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:41 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:41.091+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T00:38:11.674750+0000 front 2024-09-21T00:38:11.674912+0000 (oldest deadline 2024-09-21T00:38:35.075083+0000) 2024-09-21T00:38:41.445 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:41 smithi098 systemd[1]: Stopping Ceph osd.11 for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:38:41.745 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:41 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:41.443+0000 7fcbb56a9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:38:41.745 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:41 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:41.443+0000 7fcbb56a9640 -1 osd.11 117 *** Got signal Terminated *** 2024-09-21T00:38:41.746 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:41 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:41.443+0000 7fcbb56a9640 -1 osd.11 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T00:38:42.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:42 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:42.098+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:42.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:42 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:42.098+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:42.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:42 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:42.098+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:42.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:42 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:42.098+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:42.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:42 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:42.098+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:42.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:42 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:42.098+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:42.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:42 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:42.098+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T00:38:11.674750+0000 front 2024-09-21T00:38:11.674912+0000 (oldest deadline 2024-09-21T00:38:35.075083+0000) 2024-09-21T00:38:43.495 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:43 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:43.050+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:43.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:43 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:43.050+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:43.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:43 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:43.050+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:43.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:43 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:43.050+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:43.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:43 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:43.050+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:43.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:43 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:43.050+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:43.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:43 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:43.050+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T00:38:11.674750+0000 front 2024-09-21T00:38:11.674912+0000 (oldest deadline 2024-09-21T00:38:35.075083+0000) 2024-09-21T00:38:44.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:44 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:44.063+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:44.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:44 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:44.063+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:44.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:44 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:44.063+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:44.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:44 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:44.063+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:44.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:44 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:44.063+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:44.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:44 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:44.063+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:44.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:44 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:44.063+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T00:38:11.674750+0000 front 2024-09-21T00:38:11.674912+0000 (oldest deadline 2024-09-21T00:38:35.075083+0000) 2024-09-21T00:38:44.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:44 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:44.063+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6820 osd.8 since back 2024-09-21T00:38:19.175839+0000 front 2024-09-21T00:38:19.175879+0000 (oldest deadline 2024-09-21T00:38:43.875860+0000) 2024-09-21T00:38:45.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:45 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:45.048+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:45.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:45 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:45.048+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:45.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:45 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:45.048+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:45.497 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:45 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:45.048+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:45.497 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:45 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:45.048+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:45.497 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:45 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:45.048+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:45.498 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:45 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:45.048+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T00:38:11.674750+0000 front 2024-09-21T00:38:11.674912+0000 (oldest deadline 2024-09-21T00:38:35.075083+0000) 2024-09-21T00:38:45.498 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:45 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:45.048+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6820 osd.8 since back 2024-09-21T00:38:19.175839+0000 front 2024-09-21T00:38:19.175879+0000 (oldest deadline 2024-09-21T00:38:43.875860+0000) 2024-09-21T00:38:46.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:46 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:46.044+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T00:37:09.964832+0000 front 2024-09-21T00:37:09.964697+0000 (oldest deadline 2024-09-21T00:37:35.864607+0000) 2024-09-21T00:38:46.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:46 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:46.044+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T00:37:21.066205+0000 front 2024-09-21T00:37:21.066182+0000 (oldest deadline 2024-09-21T00:37:46.866832+0000) 2024-09-21T00:38:46.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:46 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:46.044+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T00:37:27.367989+0000 front 2024-09-21T00:37:27.368081+0000 (oldest deadline 2024-09-21T00:37:51.968738+0000) 2024-09-21T00:38:46.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:46 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:46.044+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T00:37:43.170309+0000 front 2024-09-21T00:37:43.170288+0000 (oldest deadline 2024-09-21T00:38:13.170771+0000) 2024-09-21T00:38:46.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:46 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:46.044+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T00:37:56.671701+0000 front 2024-09-21T00:37:56.671736+0000 (oldest deadline 2024-09-21T00:38:22.571804+0000) 2024-09-21T00:38:46.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:46 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:46.044+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T00:38:03.573262+0000 front 2024-09-21T00:38:03.573331+0000 (oldest deadline 2024-09-21T00:38:27.573564+0000) 2024-09-21T00:38:46.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:46 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:46.044+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T00:38:11.674750+0000 front 2024-09-21T00:38:11.674912+0000 (oldest deadline 2024-09-21T00:38:35.075083+0000) 2024-09-21T00:38:46.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:46 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11[59002]: 2024-09-21T00:38:46.044+0000 7fcbb1c94640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.98:6820 osd.8 since back 2024-09-21T00:38:19.175839+0000 front 2024-09-21T00:38:19.175879+0000 (oldest deadline 2024-09-21T00:38:43.875860+0000) 2024-09-21T00:38:47.235 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:46 smithi098 podman[69714]: 2024-09-21 00:38:46.949677056 +0000 UTC m=+5.643041296 container died 7ce233965c13102a425bb2fe04d8832c37f316908354e0659c678068cc7fab18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T00:38:47.235 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:47 smithi098 podman[69714]: 2024-09-21 00:38:47.234497855 +0000 UTC m=+5.927862097 container cleanup 7ce233965c13102a425bb2fe04d8832c37f316908354e0659c678068cc7fab18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:38:47.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:47 smithi098 bash[69714]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11 2024-09-21T00:38:47.496 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:47 smithi098 podman[69729]: 2024-09-21 00:38:47.46078301 +0000 UTC m=+0.505333411 container remove 7ce233965c13102a425bb2fe04d8832c37f316908354e0659c678068cc7fab18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T00:38:48.081 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:47 smithi098 podman[69814]: 2024-09-21 00:38:47.668107928 +0000 UTC m=+0.015530683 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T00:38:48.081 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:47 smithi098 podman[69814]: 2024-09-21 00:38:47.786903212 +0000 UTC m=+0.134325956 container create 5da9623b3a71be4ed3951c8bb005670d05443b00cdcd17b2c467de42cf76c6e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T00:38:48.082 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:48 smithi098 podman[69814]: 2024-09-21 00:38:48.005728708 +0000 UTC m=+0.353151457 container init 5da9623b3a71be4ed3951c8bb005670d05443b00cdcd17b2c467de42cf76c6e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T00:38:48.082 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:48 smithi098 podman[69814]: 2024-09-21 00:38:48.009041452 +0000 UTC m=+0.356464204 container start 5da9623b3a71be4ed3951c8bb005670d05443b00cdcd17b2c467de42cf76c6e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T00:38:48.082 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:48 smithi098 podman[69814]: 2024-09-21 00:38:48.080798958 +0000 UTC m=+0.428221730 container attach 5da9623b3a71be4ed3951c8bb005670d05443b00cdcd17b2c467de42cf76c6e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T00:38:48.475 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 00:38:48 smithi098 podman[69814]: 2024-09-21 00:38:48.195218585 +0000 UTC m=+0.542641331 container died 5da9623b3a71be4ed3951c8bb005670d05443b00cdcd17b2c467de42cf76c6e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T00:38:48.769 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T00:38:48.809 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:38:48.809 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T00:38:48.809 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T00:38:48.809 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.a 2024-09-21T00:38:49.219 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:38:48 smithi027 systemd[1]: Stopping Ceph mds.a for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:38:49.567 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:38:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-a[70832]: 2024-09-21T00:38:49.218+0000 7f287de00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:38:49.567 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:38:49 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-a[70832]: 2024-09-21T00:38:49.218+0000 7f287de00640 -1 mds.a *** got signal Terminated *** 2024-09-21T00:38:51.093 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:38:50 smithi027 podman[99729]: 2024-09-21 00:38:50.818462446 +0000 UTC m=+1.721673205 container died c5af4bbf1b58f760a8d77456a01ed693dbfdacbd1ef89e082dd0e4b82a16f810 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:38:51.417 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:38:51 smithi027 podman[99729]: 2024-09-21 00:38:51.215117492 +0000 UTC m=+2.118328240 container cleanup c5af4bbf1b58f760a8d77456a01ed693dbfdacbd1ef89e082dd0e4b82a16f810 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-a, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T00:38:51.417 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:38:51 smithi027 bash[99729]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-a 2024-09-21T00:38:51.690 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 00:38:51 smithi027 podman[99742]: 2024-09-21 00:38:51.416323248 +0000 UTC m=+0.584893687 container remove c5af4bbf1b58f760a8d77456a01ed693dbfdacbd1ef89e082dd0e4b82a16f810 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T00:38:51.711 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T00:38:51.784 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:38:51.785 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T00:38:51.785 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T00:38:51.785 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.d 2024-09-21T00:38:52.066 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:38:51 smithi027 systemd[1]: Stopping Ceph mds.d for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:38:52.567 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:38:52 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-d[69737]: 2024-09-21T00:38:52.196+0000 7fd280000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:38:52.567 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:38:52 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-d[69737]: 2024-09-21T00:38:52.196+0000 7fd280000640 -1 mds.d *** got signal Terminated *** 2024-09-21T00:38:55.930 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:38:55 smithi027 podman[99851]: 2024-09-21 00:38:55.648296379 +0000 UTC m=+3.570992279 container died e2ac6703cc5d1e07568936bdbc61ca156ad7c043a381d4d05e5596d390abfa9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-d, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T00:38:56.180 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:38:56 smithi027 podman[99851]: 2024-09-21 00:38:56.174228436 +0000 UTC m=+4.096924336 container cleanup e2ac6703cc5d1e07568936bdbc61ca156ad7c043a381d4d05e5596d390abfa9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:38:56.181 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:38:56 smithi027 bash[99851]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-d 2024-09-21T00:38:56.809 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 00:38:56 smithi027 podman[99865]: 2024-09-21 00:38:56.528641364 +0000 UTC m=+0.861937027 container remove e2ac6703cc5d1e07568936bdbc61ca156ad7c043a381d4d05e5596d390abfa9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-d, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T00:38:56.828 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T00:38:56.899 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:38:56.899 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T00:38:56.900 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T00:38:56.900 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.g 2024-09-21T00:38:57.302 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 00:38:56 smithi027 systemd[1]: Stopping Ceph mds.g for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:38:57.566 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 00:38:57 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-g[70459]: 2024-09-21T00:38:57.301+0000 7fb385600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:38:57.567 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 00:38:57 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-g[70459]: 2024-09-21T00:38:57.301+0000 7fb385600640 -1 mds.g *** got signal Terminated *** 2024-09-21T00:39:01.206 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 00:39:00 smithi027 podman[99970]: 2024-09-21 00:39:00.906866642 +0000 UTC m=+3.725595939 container died a57428cef61dca59ef0b5f18c2971d9fef7b75bf1dff85554be4618522328b16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-g, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T00:39:01.550 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 00:39:01 smithi027 podman[99970]: 2024-09-21 00:39:01.355953172 +0000 UTC m=+4.174682476 container cleanup a57428cef61dca59ef0b5f18c2971d9fef7b75bf1dff85554be4618522328b16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-g, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T00:39:01.550 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 00:39:01 smithi027 bash[99970]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-g 2024-09-21T00:39:01.818 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 00:39:01 smithi027 podman[99982]: 2024-09-21 00:39:01.548980262 +0000 UTC m=+0.635080019 container remove a57428cef61dca59ef0b5f18c2971d9fef7b75bf1dff85554be4618522328b16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-g, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:39:01.850 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T00:39:01.920 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:39:01.921 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T00:39:01.921 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T00:39:01.921 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.j 2024-09-21T00:39:02.317 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:39:01 smithi027 systemd[1]: Stopping Ceph mds.j for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:39:02.816 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:39:02 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-j[69350]: 2024-09-21T00:39:02.346+0000 7fa6a3400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:39:02.817 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:39:02 smithi027 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-j[69350]: 2024-09-21T00:39:02.346+0000 7fa6a3400640 -1 mds.j *** got signal Terminated *** 2024-09-21T00:39:04.168 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:39:03 smithi027 podman[100089]: 2024-09-21 00:39:03.906113707 +0000 UTC m=+1.708901859 container died 8f3c7975b433443c4d7aefdf58d73401de893890d55154726883bdcabad44fd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T00:39:04.478 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:39:04 smithi027 podman[100089]: 2024-09-21 00:39:04.284682585 +0000 UTC m=+2.087470736 container cleanup 8f3c7975b433443c4d7aefdf58d73401de893890d55154726883bdcabad44fd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-j, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T00:39:04.478 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:39:04 smithi027 bash[100089]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-j 2024-09-21T00:39:04.754 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 00:39:04 smithi027 podman[100100]: 2024-09-21 00:39:04.478080631 +0000 UTC m=+0.556192961 container remove 8f3c7975b433443c4d7aefdf58d73401de893890d55154726883bdcabad44fd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-j, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T00:39:04.773 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T00:39:04.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:39:04.844 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T00:39:04.845 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T00:39:04.845 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.b 2024-09-21T00:39:05.149 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:39:04 smithi089 systemd[1]: Stopping Ceph mds.b for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:39:05.478 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:39:05 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-b[63110]: 2024-09-21T00:39:05.146+0000 7fafcc16e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:39:05.478 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:39:05 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-b[63110]: 2024-09-21T00:39:05.146+0000 7fafcc16e640 -1 mds.b *** got signal Terminated *** 2024-09-21T00:39:06.882 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:39:06 smithi089 podman[68525]: 2024-09-21 00:39:06.606390511 +0000 UTC m=+1.564476818 container died a4b42d6d83682708340db1d7de0b3bd8b309a3f9567daf82bf738f89490983aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T00:39:07.166 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:39:06 smithi089 podman[68525]: 2024-09-21 00:39:06.98137823 +0000 UTC m=+1.939464523 container cleanup a4b42d6d83682708340db1d7de0b3bd8b309a3f9567daf82bf738f89490983aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-b, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T00:39:07.166 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:39:06 smithi089 bash[68525]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-b 2024-09-21T00:39:07.406 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T00:39:07.429 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:39:07 smithi089 podman[68550]: 2024-09-21 00:39:07.25797954 +0000 UTC m=+0.642270135 container remove a4b42d6d83682708340db1d7de0b3bd8b309a3f9567daf82bf738f89490983aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-b, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T00:39:07.429 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:39:07 smithi089 systemd[1]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.b.service: Deactivated successfully. 2024-09-21T00:39:07.429 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:39:07 smithi089 systemd[1]: Stopped Ceph mds.b for 9c21de84-77ad-11ef-baf5-efdc52797490. 2024-09-21T00:39:07.429 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 00:39:07 smithi089 systemd[1]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.b.service: Consumed 7min 51.200s CPU time. 2024-09-21T00:39:07.443 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:39:07.443 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T00:39:07.443 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T00:39:07.444 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.e 2024-09-21T00:39:07.728 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 00:39:07 smithi089 systemd[1]: Stopping Ceph mds.e for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:39:08.229 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 00:39:07 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-e[62569]: 2024-09-21T00:39:07.768+0000 7f405e956640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:39:08.229 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 00:39:07 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-e[62569]: 2024-09-21T00:39:07.768+0000 7f405e956640 -1 mds.e *** got signal Terminated *** 2024-09-21T00:39:10.478 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 00:39:10 smithi089 podman[68657]: 2024-09-21 00:39:10.228000588 +0000 UTC m=+2.555305371 container died 976b1886d114aa685091894c12951f091f01de9572ea6acabeb07252d094852e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T00:39:10.861 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 00:39:10 smithi089 podman[68657]: 2024-09-21 00:39:10.601764969 +0000 UTC m=+2.929069749 container cleanup 976b1886d114aa685091894c12951f091f01de9572ea6acabeb07252d094852e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T00:39:10.861 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 00:39:10 smithi089 bash[68657]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-e 2024-09-21T00:39:11.040 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T00:39:11.074 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:39:11.074 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T00:39:11.074 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T00:39:11.075 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.h 2024-09-21T00:39:11.398 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:39:11 smithi089 systemd[1]: Stopping Ceph mds.h for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:39:11.728 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:39:11 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h[62840]: 2024-09-21T00:39:11.396+0000 7f7e150d5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:39:11.728 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:39:11 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h[62840]: 2024-09-21T00:39:11.397+0000 7f7e150d5640 -1 mds.h *** got signal Terminated *** 2024-09-21T00:39:14.882 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:39:14 smithi089 podman[68776]: 2024-09-21 00:39:14.603937837 +0000 UTC m=+3.317127165 container died 3483ea512a72f742480e7b141d9e39827b3ce214a614ed3154e2a8176ec9e254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T00:39:15.207 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:39:14 smithi089 podman[68776]: 2024-09-21 00:39:14.972068391 +0000 UTC m=+3.685257720 container cleanup 3483ea512a72f742480e7b141d9e39827b3ce214a614ed3154e2a8176ec9e254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:39:15.207 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 00:39:14 smithi089 bash[68776]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-h 2024-09-21T00:39:15.345 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T00:39:15.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:39:15.377 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T00:39:15.378 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T00:39:15.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.k 2024-09-21T00:39:15.728 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 00:39:15 smithi089 systemd[1]: Stopping Ceph mds.k for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:39:16.228 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 00:39:15 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-k[62288]: 2024-09-21T00:39:15.800+0000 7fb3ffa20640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:39:16.228 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 00:39:15 smithi089 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-k[62288]: 2024-09-21T00:39:15.800+0000 7fb3ffa20640 -1 mds.k *** got signal Terminated *** 2024-09-21T00:39:20.126 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 00:39:19 smithi089 podman[68897]: 2024-09-21 00:39:19.814939439 +0000 UTC m=+4.222290880 container died 8038e914072b155c3b42b767b9011f805b2640dc4e028ee03d2ff7961327d9ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-k, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T00:39:20.444 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 00:39:20 smithi089 podman[68897]: 2024-09-21 00:39:20.225465313 +0000 UTC m=+4.632816746 container cleanup 8038e914072b155c3b42b767b9011f805b2640dc4e028ee03d2ff7961327d9ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T00:39:20.444 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 00:39:20 smithi089 bash[68897]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-k 2024-09-21T00:39:20.665 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T00:39:20.697 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:39:20.698 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T00:39:20.698 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T00:39:20.698 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.c 2024-09-21T00:39:20.995 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:39:20 smithi098 systemd[1]: Stopping Ceph mds.c for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:39:21.250 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:39:21 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-c[64121]: 2024-09-21T00:39:21.000+0000 7fd35a435640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:39:21.250 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:39:21 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-c[64121]: 2024-09-21T00:39:21.000+0000 7fd35a435640 -1 mds.c *** got signal Terminated *** 2024-09-21T00:39:22.919 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:39:22 smithi098 podman[69917]: 2024-09-21 00:39:22.627690307 +0000 UTC m=+1.737889010 container died 9bacc637e50c55d3b16f4a1738e4decb617ef0748b64d8f5ebb12d0565c4423f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T00:39:23.246 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:39:23 smithi098 podman[69917]: 2024-09-21 00:39:23.018030671 +0000 UTC m=+2.128229380 container cleanup 9bacc637e50c55d3b16f4a1738e4decb617ef0748b64d8f5ebb12d0565c4423f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-c, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T00:39:23.246 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 00:39:23 smithi098 bash[69917]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-c 2024-09-21T00:39:23.400 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T00:39:23.436 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:39:23.436 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T00:39:23.436 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T00:39:23.436 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.f 2024-09-21T00:39:23.995 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:39:23 smithi098 systemd[1]: Stopping Ceph mds.f for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:39:23.996 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:39:23 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-f[63853]: 2024-09-21T00:39:23.746+0000 7fc94a122640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:39:23.996 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:39:23 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-f[63853]: 2024-09-21T00:39:23.746+0000 7fc94a122640 -1 mds.f *** got signal Terminated *** 2024-09-21T00:39:26.923 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:39:26 smithi098 podman[70053]: 2024-09-21 00:39:26.624325331 +0000 UTC m=+2.967529144 container died fac7a7a53ad7db4321c4b5e6ca451c68d53836b71b3ad4c9e1413fafb322e9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T00:39:27.246 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:39:27 smithi098 podman[70053]: 2024-09-21 00:39:27.005090303 +0000 UTC m=+3.348294127 container cleanup fac7a7a53ad7db4321c4b5e6ca451c68d53836b71b3ad4c9e1413fafb322e9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T00:39:27.246 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 00:39:27 smithi098 bash[70053]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-f 2024-09-21T00:39:27.486 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T00:39:27.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:39:27.520 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T00:39:27.521 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T00:39:27.521 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.i 2024-09-21T00:39:27.844 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:39:27 smithi098 systemd[1]: Stopping Ceph mds.i for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:39:28.245 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:39:27 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-i[64389]: 2024-09-21T00:39:27.843+0000 7f9b18d5f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:39:28.246 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:39:27 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-i[64389]: 2024-09-21T00:39:27.843+0000 7f9b18d5f640 -1 mds.i *** got signal Terminated *** 2024-09-21T00:39:30.927 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:39:30 smithi098 podman[70174]: 2024-09-21 00:39:30.649377685 +0000 UTC m=+2.910877056 container died 6d1c23916cc1da5cfe00dec7332208f9bb400324cd546e3b7da6068111e05e8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-i, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T00:39:31.245 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:39:31 smithi098 podman[70174]: 2024-09-21 00:39:31.026243921 +0000 UTC m=+3.287743294 container cleanup 6d1c23916cc1da5cfe00dec7332208f9bb400324cd546e3b7da6068111e05e8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-i, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T00:39:31.246 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 00:39:31 smithi098 bash[70174]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-i 2024-09-21T00:39:31.444 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T00:39:31.476 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:39:31.477 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T00:39:31.477 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T00:39:31.477 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.l 2024-09-21T00:39:31.806 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 00:39:31 smithi098 systemd[1]: Stopping Ceph mds.l for 9c21de84-77ad-11ef-baf5-efdc52797490... 2024-09-21T00:39:32.245 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 00:39:31 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-l[63577]: 2024-09-21T00:39:31.804+0000 7fd7ae0b7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T00:39:32.246 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 00:39:31 smithi098 ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-l[63577]: 2024-09-21T00:39:31.804+0000 7fd7ae0b7640 -1 mds.l *** got signal Terminated *** 2024-09-21T00:39:33.746 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 00:39:33 smithi098 podman[70298]: 2024-09-21 00:39:33.484612896 +0000 UTC m=+1.792805035 container died e5614accb06847eb34f6a1d31d6b96fc7e13e44a893c0de2d4dec197dc584074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-l, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T00:39:34.007 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 00:39:33 smithi098 podman[70298]: 2024-09-21 00:39:33.839037371 +0000 UTC m=+2.147229510 container cleanup e5614accb06847eb34f6a1d31d6b96fc7e13e44a893c0de2d4dec197dc584074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-l, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T00:39:34.007 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 00:39:33 smithi098 bash[70298]: ceph-9c21de84-77ad-11ef-baf5-efdc52797490-mds-l 2024-09-21T00:39:34.221 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c21de84-77ad-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T00:39:34.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T00:39:34.253 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T00:39:34.253 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T00:39:34.541 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:39:38.569 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T00:39:38.770 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:39:41.302 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T00:39:41.505 INFO:teuthology.orchestra.run.smithi098.stdout:Deleting cluster with fsid: 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:39:43.934 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T00:39:44.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T00:39:44.041 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T00:39:44.069 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T00:39:44.079 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310/remote/smithi027/crash 2024-09-21T00:39:44.081 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/crash -- . 2024-09-21T00:39:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:tar: /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T00:39:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:tar: Error is not recoverable: exiting now 2024-09-21T00:39:44.157 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310/remote/smithi089/crash 2024-09-21T00:39:44.168 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/crash -- . 2024-09-21T00:39:44.211 INFO:teuthology.orchestra.run.smithi089.stderr:tar: /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T00:39:44.211 INFO:teuthology.orchestra.run.smithi089.stderr:tar: Error is not recoverable: exiting now 2024-09-21T00:39:44.214 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310/remote/smithi098/crash 2024-09-21T00:39:44.216 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/crash -- . 2024-09-21T00:39:44.253 INFO:teuthology.orchestra.run.smithi098.stderr:tar: /var/lib/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T00:39:44.253 INFO:teuthology.orchestra.run.smithi098.stderr:tar: Error is not recoverable: exiting now 2024-09-21T00:39:44.254 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T00:39:44.255 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-21T00:39:44.399 INFO:tasks.cephadm:Compressing logs... 2024-09-21T00:39:44.400 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T00:39:44.404 DEBUG:teuthology.orchestra.run.smithi089:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T00:39:44.407 DEBUG:teuthology.orchestra.run.smithi098:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T00:39:44.440 INFO:teuthology.orchestra.run.smithi098.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T00:39:44.441 INFO:teuthology.orchestra.run.smithi089.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T00:39:44.450 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T00:39:44.450 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T00:39:44.450 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T00:39:44.450 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T00:39:44.451 INFO:teuthology.orchestra.run.smithi089.stderr:gzip/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.7.log: -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T00:39:44.451 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T00:39:44.451 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T00:39:44.451 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T00:39:44.451 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T00:39:44.451 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T00:39:44.451 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.log 2024-09-21T00:39:44.451 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T00:39:44.451 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T00:39:44.451 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T00:39:44.451 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T00:39:44.452 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T00:39:44.452 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T00:39:44.453 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T00:39:44.453 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T00:39:44.453 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T00:39:44.453 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T00:39:44.453 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T00:39:44.453 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T00:39:44.453 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.log 2024-09-21T00:39:44.453 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T00:39:44.453 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T00:39:44.453 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T00:39:44.453 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T00:39:44.453 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T00:39:44.453 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T00:39:44.470 INFO:teuthology.orchestra.run.smithi027.stderr:find: ‘/var/log/rbd-target-api’gzip: No such file or directory 2024-09-21T00:39:44.470 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-21T00:39:44.471 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T00:39:44.474 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T00:39:44.475 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T00:39:44.475 INFO:teuthology.orchestra.run.smithi027.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-21T00:39:44.475 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T00:39:44.475 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T00:39:44.475 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T00:39:44.475 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T00:39:44.476 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T00:39:44.477 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T00:39:44.477 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.log 2024-09-21T00:39:44.478 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T00:39:44.479 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T00:39:44.480 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T00:39:44.481 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T00:39:44.482 INFO:teuthology.orchestra.run.smithi027.stderr: 87.1% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T00:39:44.482 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T00:39:44.484 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.j.log: /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.9% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T00:39:44.489 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 81.2% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T00:39:44.490 INFO:teuthology.orchestra.run.smithi027.stderr: 94.9% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T00:39:44.491 INFO:teuthology.orchestra.run.smithi027.stderr: 94.7% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T00:39:44.495 INFO:teuthology.orchestra.run.smithi027.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T00:39:44.511 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.f.log: /var/log/ceph/cephadm.log: 81.2% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T00:39:44.558 INFO:teuthology.orchestra.run.smithi089.stderr: 94.8% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T00:39:44.564 INFO:teuthology.orchestra.run.smithi089.stderr: 94.9% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T00:39:44.564 INFO:teuthology.orchestra.run.smithi089.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T00:39:44.576 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T00:39:44.577 INFO:teuthology.orchestra.run.smithi098.stderr: 90.1% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T00:39:44.588 INFO:teuthology.orchestra.run.smithi089.stderr: 90.1% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T00:39:44.591 INFO:teuthology.orchestra.run.smithi098.stderr: 92.9% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T00:39:44.598 INFO:teuthology.orchestra.run.smithi089.stderr: 92.8% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T00:39:44.599 INFO:teuthology.orchestra.run.smithi027.stderr: 90.9% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T00:39:44.614 INFO:teuthology.orchestra.run.smithi089.stderr: 87.1% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T00:39:44.615 INFO:teuthology.orchestra.run.smithi098.stderr: 94.9% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T00:39:44.651 INFO:teuthology.orchestra.run.smithi027.stderr: 84.1% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T00:39:44.722 INFO:teuthology.orchestra.run.smithi098.stderr: 90.9% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T00:39:44.725 INFO:teuthology.orchestra.run.smithi098.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T00:39:44.757 INFO:teuthology.orchestra.run.smithi027.stderr: 90.0% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T00:39:44.765 INFO:teuthology.orchestra.run.smithi089.stderr: 91.0% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T00:39:45.835 INFO:teuthology.orchestra.run.smithi027.stderr: 90.3% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T00:39:46.296 INFO:teuthology.orchestra.run.smithi089.stderr: 91.6% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T00:39:47.329 INFO:teuthology.orchestra.run.smithi098.stderr: 91.5% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T00:39:48.406 INFO:teuthology.orchestra.run.smithi027.stderr: 90.5% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T00:40:12.637 INFO:teuthology.orchestra.run.smithi098.stderr: 94.9% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T00:40:14.694 INFO:teuthology.orchestra.run.smithi098.stderr: 95.2% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T00:40:22.021 INFO:teuthology.orchestra.run.smithi089.stderr: 95.1% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T00:40:48.059 INFO:teuthology.orchestra.run.smithi098.stderr: 91.7% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T00:41:05.101 INFO:teuthology.orchestra.run.smithi089.stderr: 91.7% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T00:41:24.922 INFO:teuthology.orchestra.run.smithi098.stderr: 91.6% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T00:41:56.790 INFO:teuthology.orchestra.run.smithi098.stderr: 91.5% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T00:42:00.777 INFO:teuthology.orchestra.run.smithi098.stderr: 91.4% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T00:42:26.116 INFO:teuthology.orchestra.run.smithi027.stderr: 91.3% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T00:42:37.048 INFO:teuthology.orchestra.run.smithi098.stderr: 94.2% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T00:42:37.051 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-21T00:42:37.051 INFO:teuthology.orchestra.run.smithi098.stderr:real 2m52.629s 2024-09-21T00:42:37.051 INFO:teuthology.orchestra.run.smithi098.stderr:user 4m12.934s 2024-09-21T00:42:37.051 INFO:teuthology.orchestra.run.smithi098.stderr:sys 0m9.500s 2024-09-21T00:42:38.264 INFO:teuthology.orchestra.run.smithi027.stderr: 91.3% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T00:42:47.540 INFO:teuthology.orchestra.run.smithi089.stderr: 91.8% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T00:42:49.215 INFO:teuthology.orchestra.run.smithi089.stderr: 91.4% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T00:43:13.790 INFO:teuthology.orchestra.run.smithi027.stderr: 91.3% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T00:43:20.405 INFO:teuthology.orchestra.run.smithi027.stderr: 91.3% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T00:44:15.828 INFO:teuthology.orchestra.run.smithi027.stderr: 94.0% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T00:44:15.866 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-21T00:44:15.866 INFO:teuthology.orchestra.run.smithi027.stderr:real 4m31.397s 2024-09-21T00:44:15.867 INFO:teuthology.orchestra.run.smithi027.stderr:user 4m42.429s 2024-09-21T00:44:15.867 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m28.082s 2024-09-21T00:44:47.222 INFO:teuthology.orchestra.run.smithi089.stderr: 91.6% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T00:46:43.652 INFO:teuthology.orchestra.run.smithi089.stderr: 93.7% -- replaced with /var/log/ceph/9c21de84-77ad-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T00:46:43.654 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-09-21T00:46:43.654 INFO:teuthology.orchestra.run.smithi089.stderr:real 6m59.233s 2024-09-21T00:46:43.654 INFO:teuthology.orchestra.run.smithi089.stderr:user 5m23.844s 2024-09-21T00:46:43.654 INFO:teuthology.orchestra.run.smithi089.stderr:sys 0m13.034s 2024-09-21T00:46:43.655 INFO:tasks.cephadm:Archiving logs... 2024-09-21T00:46:43.656 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310/remote/smithi027/log 2024-09-21T00:46:43.657 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T00:47:25.568 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310/remote/smithi089/log 2024-09-21T00:47:25.569 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T00:48:16.360 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310/remote/smithi098/log 2024-09-21T00:48:16.361 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T00:48:56.039 INFO:tasks.cephadm:Removing cluster... 2024-09-21T00:48:56.039 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 --force 2024-09-21T00:48:56.438 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:48:57.953 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 --force 2024-09-21T00:48:58.155 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:48:58.822 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9c21de84-77ad-11ef-baf5-efdc52797490 --force 2024-09-21T00:48:59.027 INFO:teuthology.orchestra.run.smithi098.stdout:Deleting cluster with fsid: 9c21de84-77ad-11ef-baf5-efdc52797490 2024-09-21T00:48:59.610 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T00:48:59.611 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T00:48:59.653 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T00:48:59.673 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T00:48:59.696 INFO:tasks.cephadm:Teardown complete 2024-09-21T00:48:59.696 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T00:48:59.712 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T00:48:59.712 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T00:48:59.722 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T00:48:59.729 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T00:48:59.855 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T00:48:59.867 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T00:48:59.868 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T00:48:59.872 DEBUG:teuthology.orchestra.run.smithi089:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T00:48:59.874 DEBUG:teuthology.orchestra.run.smithi098:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T00:48:59.886 INFO:teuthology.orchestra.run.smithi089.stderr:bash: line 1: ntpq: command not found 2024-09-21T00:48:59.888 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-09-21T00:48:59.898 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T00:48:59.899 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2024-09-21T00:48:59.899 INFO:teuthology.orchestra.run.smithi089.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 +551us[ +549us] +/- 51ms 2024-09-21T00:48:59.899 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 -1137us[-1137us] +/- 53ms 2024-09-21T00:48:59.899 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +916us[ +914us] +/- 89ms 2024-09-21T00:48:59.899 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:48:59.903 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T00:48:59.903 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-09-21T00:48:59.903 INFO:teuthology.orchestra.run.smithi098.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 +583us[ +599us] +/- 51ms 2024-09-21T00:48:59.903 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 -1093us[-1093us] +/- 53ms 2024-09-21T00:48:59.903 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 +955us[ +955us] +/- 89ms 2024-09-21T00:48:59.904 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:48:59.904 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-09-21T00:48:59.998 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T00:48:59.998 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-09-21T00:48:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv01.front.sepia.ceph.com 3 6 377 29 +570us[ +561us] +/- 51ms 2024-09-21T00:48:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 -1122us[-1122us] +/- 53ms 2024-09-21T00:48:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +930us[ +930us] +/- 89ms 2024-09-21T00:48:59.999 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:49:00.000 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T00:49:00.011 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T00:49:00.011 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T00:49:00.035 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T00:49:00.209 DEBUG:teuthology.orchestra.run.smithi089:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T00:49:00.286 DEBUG:teuthology.orchestra.run.smithi098:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T00:49:00.404 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T00:49:00.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:49:00.469 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T00:49:00.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:49:00.497 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T00:49:00.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:49:00.524 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T00:49:00.535 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T00:49:00.585 INFO:teuthology.task.internal:Duration was 2850.201113 seconds 2024-09-21T00:49:00.585 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T00:49:00.618 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T00:49:00.618 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T00:49:00.622 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T00:49:00.624 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T00:49:00.683 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T00:49:00.717 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T00:49:00.723 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T00:49:01.057 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T00:49:01.057 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T00:49:01.057 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T00:49:01.108 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi089.front.sepia.ceph.com 2024-09-21T00:49:01.108 DEBUG:teuthology.orchestra.run.smithi089:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T00:49:01.133 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi098.front.sepia.ceph.com 2024-09-21T00:49:01.133 DEBUG:teuthology.orchestra.run.smithi098:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T00:49:01.155 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T00:49:01.155 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T00:49:01.159 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T00:49:01.176 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T00:49:01.222 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T00:49:01.222 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T00:49:01.226 DEBUG:teuthology.orchestra.run.smithi089:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T00:49:01.242 DEBUG:teuthology.orchestra.run.smithi098:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T00:49:02.133 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T00:49:02.151 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T00:49:02.151 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T00:49:02.210 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T00:49:02.240 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T00:49:02.284 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T00:49:02.294 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T00:49:02.298 DEBUG:teuthology.orchestra.run.smithi089:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T00:49:02.301 DEBUG:teuthology.orchestra.run.smithi098:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T00:49:02.328 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = core 2024-09-21T00:49:02.341 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = core 2024-09-21T00:49:02.392 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2024-09-21T00:49:02.423 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T00:49:02.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:49:02.457 DEBUG:teuthology.orchestra.run.smithi089:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T00:49:02.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:49:02.473 DEBUG:teuthology.orchestra.run.smithi098:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T00:49:02.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:49:02.489 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T00:49:02.503 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T00:49:02.505 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310/remote/smithi027 2024-09-21T00:49:02.505 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T00:49:02.589 DEBUG:teuthology.misc:Transferring archived files from smithi089:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310/remote/smithi089 2024-09-21T00:49:02.589 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T00:49:02.629 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913310/remote/smithi098 2024-09-21T00:49:02.629 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T00:49:02.666 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T00:49:02.666 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T00:49:02.670 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T00:49:02.673 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T00:49:02.715 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T00:49:02.729 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T00:49:02.729 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T00:49:02.762 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T00:49:02.762 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T00:49:02.766 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T00:49:02.769 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T00:49:02.784 INFO:teuthology.orchestra.run.smithi098.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 00:49 /home/ubuntu/cephtest 2024-09-21T00:49:02.784 INFO:teuthology.orchestra.run.smithi089.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 00:49 /home/ubuntu/cephtest 2024-09-21T00:49:02.798 INFO:teuthology.orchestra.run.smithi027.stdout: 393723 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 00:49 /home/ubuntu/cephtest 2024-09-21T00:49:02.809 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T00:49:02.820 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T00:49:02.877 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 2850.2011132240295 failure_reason: 'error during quiesce thrashing: Error quiescing set ''47099ce0'': 1 (EPERM)' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-09-21T00:49:02.878 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T00:49:02.966 INFO:teuthology.run:FAIL